From bfb8c782ee3545801116987829b7c5f7c06543cf Mon Sep 17 00:00:00 2001 From: Alex Blank Date: Wed, 13 Aug 2025 15:55:43 +0200 Subject: [PATCH] further work on results --- ..._fertility_results_by_input_resolution.png | Bin 254872 -> 254872 bytes .../lstm_ov_over_results_by_input_length.png | Bin 332620 -> 332620 bytes ...tm_ov_over_results_by_input_resolution.png | Bin 397930 -> 397930 bytes thesis/sections/background.tex | 8 +- thesis/sections/methodology.tex | 6 + thesis/sections/results.tex | 228 +++++++++++++----- 6 files changed, 180 insertions(+), 62 deletions(-) diff --git a/thesis/resources/figures/results/lstm_fertility_results_by_input_resolution.png b/thesis/resources/figures/results/lstm_fertility_results_by_input_resolution.png index a0faa00692ed15be7f2f446b80c447ad8a1c7e04..54631c5f037f7b5f8476c4b250ca6fd055f17380 100644 GIT binary patch delta 52 zcmbPnoPWk~{t2FP2098EB_##LR{Hw6i6sR&`6W4-NqYH3>H1TOB8nQ*Thkd^)0wuW IGaEhz01EXJz5oCK delta 52 zcmbPnoPWk~{t2FP#yScaB_##LR{Hw6i6sR&`6W4-NqYH3>H2A^Te}+5Thkd^)0wuW IGaEhz018_Zy8r+H diff --git a/thesis/resources/figures/results/lstm_ov_over_results_by_input_length.png b/thesis/resources/figures/results/lstm_ov_over_results_by_input_length.png index e6ff11264cf35c27d94dbb0714134e85d8c3b4fa..f725103ccc08cd566b01fafdb760cb16af1d225d 100644 GIT binary patch delta 58 zcmX@JOytZmkqMr12098EB_##LR{Hw6i6sR&`6W4-NqYH3>H1TOB8nQ*Thkd^)0tY+ OnOoCYwx+W_6#@W|gB7v> delta 58 zcmX@JOytZmkqMr1#yScaB_##LR{Hw6i6sR&`6W4-NqYH3>H2A^Te}+5Thkd^)0tY+ OnOoCYwx+W_6#@W{*A=e- diff --git a/thesis/resources/figures/results/lstm_ov_over_results_by_input_resolution.png b/thesis/resources/figures/results/lstm_ov_over_results_by_input_resolution.png index 4cdc5055065e3152ba4f0d72997617d7ea05db19..da1abc8f02f1e371ec128f40d191fa1b06600e60 100644 GIT binary patch delta 61 zcmaF0MB>#Fi3y%^2098EB_##LR{Hw6i6sR&`6W4-NqYH3>H1TOB8nQ*Thkd^)0tY+ RnOoCYTGLs#rn4~%0RZ2776JeO delta 61 zcmaF0MB>#Fi3y%^#yScaB_##LR{Hw6i6sR&`6W4-NqYH3>H2A^Te}+5Thkd^)0tY+ RnOoCYTGLs#rn4~%0RY~{75@ML diff --git a/thesis/sections/background.tex b/thesis/sections/background.tex index b8ceef1..a8ffdd0 100644 --- a/thesis/sections/background.tex +++ b/thesis/sections/background.tex @@ -136,10 +136,10 @@ For women aiming to avoid pregnancy, minimizing false-negative predictions is cr Although false-positives may lead to unnecessary abstinence, this outcome is generally considered less severe. Consequently, prediction algorithms should be conservative, erring on the side of higher fertility estimates to prioritize safety. -Conversely, for women aiming to conceive, false-positive predictions could misdirect efforts toward incorrect cycle days, -causing frustration or delays. -False-negatives have fewer negative consequences. -Therefore, algorithms for this group should prefer cautious fertility estimates, reducing the risk of misdirected effort. +For women aiming to conceive, incorrect predictions are less severe, as they might, at worst, result in misdirected +pregnancy effort, causing frustration or delays, but no potential dangers to the health of a woman. +Therefore, algorithms for this group don't need to be as conservative. +It remains to be seen, where the middle ground lies and how different algorithms perform for different use cases. \subsubsection{Physiological Signs of Ovulation}\label{subsubsec:physiological_signs} Several physiological signs correlate with ovulation and can be used for prediction. diff --git a/thesis/sections/methodology.tex b/thesis/sections/methodology.tex index 2f11598..b252ec5 100644 --- a/thesis/sections/methodology.tex +++ b/thesis/sections/methodology.tex @@ -500,6 +500,9 @@ To identify suitable configurations for each model architecture, model-specific The selected value ranges were intentionally broad to explore the trade-off between model complexity and generalization. This allowed assessment of whether increased architectural depth and capacity contribute meaningfully to performance, or whether simpler models are sufficient for the task. +Due to resource and time limitations, not all configuration permutations can be tested. +Thus, the parameters will be tested on the input configuration with the best MSE on the fertile days for each model, +as this is the metric that represents the use cases and overall intention the best. \begin{table}[htbp] \centering @@ -539,6 +542,9 @@ Tables~\ref{tab:lstm_model_hyperparameters} and~\ref{tab:transformer_model_hyper for the LSTM-based and Transformer-based models, respectively. Note that the same settings were used for the hybrid models, as their architecture beyond the convolutional front end is structurally identical. +Based on the results of the model parameter search, a best parameter set will be selected for each model architecture, +based on the MSE of the fertility-probability target during the fertile days. +These model configurations will then be used for further evaluations. \vspace{0.5em} We acknowledge that interactions between input and model parameters may influence final model performance, diff --git a/thesis/sections/results.tex b/thesis/sections/results.tex index 7a2c182..002833c 100644 --- a/thesis/sections/results.tex +++ b/thesis/sections/results.tex @@ -25,12 +25,11 @@ \subsection{Overall Model Performance Across Architectures}\label{subsec:overall_model_performance_across_architectures} -\subsection{Fertility Probability Prediction Accuracy}\label{subsec:fertility_probability_precition_accuracy} % show why I selected the individual input configs for model config training % selected by best mse fertility, use 2nd best, as it provides basically the same performance, but more input data for more complex model configs -\subsubsection{Fertility Probability Prediction}\label{subsubsec:fertility_probability_prediction} +\subsection{Fertility Probability Prediction}\label{subsec:fertility_probability_prediction} \subsubsection{Impact of Input Window Length}\label{subsubsec:fert_impact_of_historical_context} \begin{landscape} @@ -78,34 +77,31 @@ \end{table} \end{landscape} -Table~\ref{tab:fertility_results_by_window_length} presents the performance of different model architectures across various metrics and input window lengths. +Table~\ref{tab:fertility_results_by_window_length} summarises performance across metrics and input window lengths for the fertility-probability target. -It is evident that for both the LSTM and Transformer models, \textbf{longer input windows generally yield better performance}. -Specifically, the LSTM achieves the best results for four out of six metrics with the 160-day window. -The only exception is the error metrics for non-fertile days, where shorter input windows perform better. -Notably, the LSTM achieves the \textbf{global best performance} for four out of six metrics, including MAE and MSE on non-fertile and fertile days. +For both the LSTM and Transformer, longer input windows generally yield better results. +The LSTM achieves the top scores in four of six metrics with the 160-day window, including the global best for MAE and MSE on both fertile and non-fertile days. +The only exception is non-fertile day errors, where shorter windows perform better. -For the Transformer model, the best performance is observed with the 160-day input window in three out of six metrics. -However, for non-fertile day metrics, performance improves with shorter input windows. -It is consistently outperformed by the LSTM model. +The Transformer peaks at the 160-day window in three of six metrics but improves on non-fertile days with shorter inputs. +Across all window lengths, it is consistently outperformed by the LSTM\@. -The \textbf{convolutional models} tend to perform best with \textbf{medium-length input windows} (20 to 40 days). -The \textbf{convolutional LSTM} shows its best performance with a 40-day window for most metrics, -except for non-fertile day errors, where the 20-day window yields smaller errors. -It also achieves the \textbf{global best performance} for non-fertile day MSE with the 20-day window. -Similarly, the \textbf{convolutional Transformer} excels with 40-day windows for the overall and non-fertile-day metrics, -while the best fertile-day metrics are seen with 20-day windows. -The \textbf{convolutional Transformer} also achieves the \textbf{global best performance} for 40-day windows in both MAE and MSE\@. -In all but non-fertile day metrics, the \textbf{convolutional Transformer} outperforms the LSTM variant. +Convolutional models perform best with medium-length windows (20--40 days). +The convolutional LSTM reaches its lowest errors for most metrics at 40 days, except for non-fertile MSE, where 20 days is optimal---also a global best. +The convolutional Transformer similarly favours 40 days for overall and non-fertile-day metrics, while fertile-day metrics perform best with 20-day inputs. +It achieves global best scores for 40-day MAE and MSE, and outperforms the convolutional LSTM in all but non-fertile day metrics. \subsubsection{Impact of Input Resolution}\label{subsubsec:fert_impact_of_input_resolution} +Shifting focus from temporal span to sampling density, Table~\ref{tab:fertility_results_by_window_resolution} reports performance for the fertility-probability target across varying input resolutions. +Convolutional models are omitted, as their convolution layers inherently perform learnable resampling. -Figure~\ref{fig:results_performance_lstm_fertility_input_resolution} presents the results for different input resolutions used in the LSTM models, -alongside the corresponding mean squared error (MSE) metrics. -The plot shows the MSE values for three target categories: non-fertile days, overall MSE, and fertile days MSE\@. +The LSTM outperforms the Transformer at all resolutions, with best-performing metrics scattered across the medium-to-high range (4--288 values/day) and no single optimum. +Fertile-day errors are lowest at 288 values/day, while non-fertile day errors peak at 12 values/day. +Overall MAE and MSE minima occur at 48 and 4 values/day, respectively. + +The Transformer shows a clearer trend towards higher-resolution inputs. +Its fertile-day metrics are best at 4 values/day, whereas non-fertile day, overall MAE, and overall MSE scores peak between 48 and 288 values/day. -Notably, the MSE for non-fertile days appears to behave inversely in relation to the other two metrics. -However, there is no consistent upward or downward trend when changing the input window length with a fixed input resolution. \begin{landscape} \begin{table} @@ -145,29 +141,80 @@ However, there is no consistent upward or downward trend when changing the input \label{tab:fertility_results_by_window_resolution} \end{table} \end{landscape} -\begin{figure} - \centering - \includegraphics[width=1.0\textwidth]{resources/figures/results/transformer_fertility_results_by_input_resolution} - \caption{Performance of transformer models with different input window lengths and fixes input resolutions for the fertility probability target} - \label{fig:results_performance_transformer_fertility_input_resolution} -\end{figure} - -\begin{figure} - \centering - \includegraphics[width=1.00\textwidth]{resources/figures/results/lstm_fertility_results_by_input_resolution} - \caption{Performance of LSTM models with different input resolutions and fixes input window lengths for the fertility probability target} - \label{fig:results_performance_lstm_fertility_input_resolution} -\end{figure} +\subsubsection{Impact of Model Parameters}\label{subsubsec:fert_impaoct_of_model_parameters} +\begin{landscape} + \begin{table} + \scriptsize + \begin{tabularx}{\linewidth}{l*{8}{X}} + \toprule + \multirow{2}{*}{Hidden Layer Size} & + \multirow{2}{*}{\# LSTM Layers} & + \multicolumn{3}{c}{MAE} & + \multicolumn{3}{c}{MSE} \\ + \cmidrule(lr){3-5} \cmidrule(lr){6-8} + & & Fert Overall & Fert Days & Non-Fert Days & Fert Overall & Fert Days & Non-Fert Days \\ + \midrule + 16 & 1 & 0.0485 & 0.1050 & 0.0252 & 0.0059 & 0.0161 & 0.0016 \\ + 32 & 1 & 0.0477 & 0.1029 & 0.0248 & 0.0057 & 0.0155 & 0.0016 \\ + 32 & 2 & 0.0458 & 0.0922 & 0.0269 & 0.0052 & \textbf{0.0122} & 0.0024 \\ + 64 & 2 & 0.0447 & 0.0925 & 0.0250 & 0.0051 & 0.0123 & 0.0021 \\ + 128 & 2 & 0.0424 & 0.0951 & 0.0201 & 0.0051 & 0.0131 & 0.0018 \\ + 128 & 4 & 0.0427 & 0.0989 & 0.0191 & 0.0053 & 0.0145 & 0.0014 \\ + 256 & 4 & 0.0433 & 0.1050 & \textbf{0.0175} & 0.0055 & 0.0161 & \textbf{0.0011} \\ + 512 & 4 & \textbf{0.0399} & \textbf{0.0911} & 0.0189 & \textbf{0.0047} & \textbf{0.0122} & 0.0016 \\ + \bottomrule + \end{tabularx} + \caption{Evaluation Metrics for the Fertility-Probability Target across Different Model Parameters for the LSTM model with + a fixed input window length of 160 days and an input resolution of 12 values per day. + \textbf{Bold} values represent the best value for each metric within a model.} + \label{tab:fertility_results_by_model_parameters_lstm} + \end{table} + \begin{table} + \scriptsize + \begin{tabularx}{\linewidth}{l*{8}{X}} + \toprule + \multirow{2}{*}{Size of Embedding} & + \multirow{2}{*}{\# Encoder Layers} & + \multirow{2}{*}{\# Attention Heads} & + \multicolumn{3}{c}{MAE} & + \multicolumn{3}{c}{MSE} \\ + \cmidrule(lr){4-6} \cmidrule(lr){7-9} + & & & Fert Overall & Fert Days & Non-Fert Days & Fert Overall & Fert Days & Non-Fert Days \\ + \midrule + 16 & 1 & 1 & 0.0708 & 0.1262 & 0.0475 & 0.0084 & 0.0227 & 0.0024 \\ + 32 & 1 & 1 & 0.0599 & 0.1163 & 0.0378 & 0.0070 & 0.0196 & 0.0019 \\ + 64 & 1 & 1 & 0.0601 & 0.1178 & 0.0373 & 0.0071 & 0.0200 & 0.0020 \\ + 64 & 2 & 2 & 0.0477 & 0.0908 & 0.0301 & 0.0049 & 0.0121 & 0.0020 \\ + 128 & 2 & 2 & 0.0449 & 0.0904 & 0.0263 & 0.0047 & 0.0119 & 0.0017 \\ + 128 & 4 & 4 & 0.0461 & 0.0979 & 0.0247 & 0.0050 & 0.0143 & 0.0013 \\ + 256 & 4 & 4 & 0.0475 & 0.0919 & 0.0293 & 0.0048 & 0.0125 & 0.0017 \\ + 512 & 4 & 4 & 0.0403 & \textbf{0.0831} & 0.0229 & \textbf{0.0043} & \textbf{0.0103} & 0.0018 \\ + 512 & 8 & 8 & \textbf{0.0395} & 0.0967 & \textbf{0.0159} & 0.0048 & 0.0139 & \textbf{0.0011} \\ + \bottomrule + \end{tabularx} + \caption{Evaluation Metrics for the Fertility-Probability Target across Different Model Parameters for the Transformer model with + a fixed input window length of 160 days and an input resolution of 12 values per day. + \textbf{Bold} values represent the best value for each metric within a model.} + \label{tab:fertility_results_by_model_parameters_transformer} + \end{table} +\end{landscape} \subsubsection{Comparison with Baselines}\label{subsubsec:fert_comparison_with_baselines} \subsection{Ovulation-Over Prediction}\label{subsubsec:ov_over_prediction} -\subsubsection{Performance around Ovulation}\label{subsubsec:ov_over_performance_around_ovulation} - \subsubsection{Impact of Input Window Length}\label{subsubsec:ov_over_impact_of_historical_context} +Table~\ref{tab:ov_over_results_by_window_length} presents performance across metrics and input window lengths for the ovulation-over target. + +The LSTM and Transformer both outperform their convolutional counterparts. +The LSTM benefits from longer windows (160 days) except for the before-ovulation metric, where 20 days is optimal---also the \textbf{global best} across models. +The Transformer performs best with mid-length windows (40 days) for most metrics, with exceptions in after-ovulation performance (MAE: 80 days, MSE: 20 days). + +Among convolutional models, the convolutional LSTM peaks at mid-length windows, reaching lowest MAE at 40 days and lowest MSE at 20 days. +The convolutional Transformer also favours 40 days overall, but before-ovulation performance benefits from longer inputs (160 days). + \begin{landscape} \begin{table} \small @@ -179,36 +226,37 @@ However, there is no consistent upward or downward trend when changing the input \midrule \multicolumn{7}{c}{\textbf{LSTM}} \\ \midrule - 10 & 0.1218 & 0.1044 & 0.1223 & \underline{0.0612} & 0.0289 & 0.0711 \\ - 20 & 0.1153 & \underline{\textbf{0.0745}} & 0.1312 & 0.0641 & \underline{\textbf{0.0212}} & 0.0822 \\ - 40 & 0.1066 & 0.0820 & 0.1128 & 0.0616 & 0.0281 & 0.0740 \\ - 80 & 0.1173 & 0.0842 & 0.1291 & 0.0647 & 0.0263 & 0.0801 \\ - 160 & \underline{0.1039} & 0.1139 & \underline{0.0973} & 0.0557 & 0.0462 & \underline{0.0580} \\ + 10 & 0.1218 & 0.1044 & 0.1223 & 0.0612 & 0.0289 & 0.0711 \\ + 20 & 0.1153 & \underline{\textbf{0.0745}} & 0.1312 & 0.0641 & \underline{\textbf{0.0212}} & 0.0822 \\ + 40 & 0.1066 & 0.0820 & 0.1128 & 0.0616 & 0.0281 & 0.0740 \\ + 80 & 0.1173 & 0.0842 & 0.1291 & 0.0647 & 0.0263 & 0.0801 \\ + 160 & \underline{0.1039} & 0.1139 & \underline{0.0973} & \underline{0.0557} & 0.0462 & \underline{0.0580} \\ \midrule \multicolumn{7}{c}{\textbf{Transformer}} \\ \midrule - 10 & 0.1137 & 0.1006 & 0.1186 & 0.0618 & 0.0336 & 0.0740 \\ - 20 & 0.1204 & 0.0771 & 0.1366 & 0.0690 & \underline{0.0255} & 0.0864 \\ - 40 & \underline{\textbf{0.1017}} & 0.1409 & \underline{\textbf{0.0883}} & \underline{\textbf{0.0533}} & 0.0621 & \underline{\textbf{0.0520}} \\ - 80 & 0.1138 & \underline{0.0897} & 0.1234 & 0.0654 & 0.0356 & 0.0788 \\ - 160 & 0.1076 & 0.0959 & 0.1141 & 0.0606 & 0.0379 & 0.0714 \\ + 10 & 0.1137 & 0.1006 & 0.1186 & 0.0618 & 0.0336 & 0.0740 \\ + 20 & 0.1204 & 0.0771 & 0.1366 & 0.0690 & \underline{0.0255} & 0.0864 \\ + 40 & \underline{\textbf{0.1017}} & 0.1409 & \underline{\textbf{0.0883}} & \underline{\textbf{0.0533}} & 0.0621 & \underline{\textbf{0.0520}} \\ + 80 & 0.1138 & \underline{0.0897} & 0.1234 & 0.0654 & 0.0356 & 0.0788 \\ + 160 & 0.1076 & 0.0959 & 0.1141 & 0.0606 & 0.0379 & 0.0714 \\ \midrule \multicolumn{7}{c}{\textbf{Convolution-LSTM}} \\ \midrule - 10 & 0.1847 & 0.1878 & 0.1840 & 0.0819 & 0.0581 & 0.0949 \\ - 20 & 0.1493 & 0.1274 & 0.1562 & \underline{0.0699} & \underline{0.0389} & \underline{0.0833} \\ - 40 & \underline{0.1455} & \underline{0.1089} & \underline{0.1561} & 0.0722 & 0.0358 & 0.0852 \\ - 80 & 0.2327 & 0.1796 & 0.2507 & 0.1168 & 0.0686 & 0.1327 \\ - 160 & 0.2518 & 0.2040 & 0.2757 & 0.1293 & 0.0878 & 0.1503 \\ + 10 & 0.1847 & 0.1878 & 0.1840 & 0.0819 & 0.0581 & 0.0949 \\ + 20 & 0.1493 & 0.1274 & 0.1562 & \underline{0.0699} & \underline{0.0389} & \underline{0.0833} \\ + 40 & \underline{0.1455} & \underline{0.1089} & \underline{0.1561} & 0.0722 & 0.0358 & 0.0852 \\ + 80 & 0.2327 & 0.1796 & 0.2507 & 0.1168 & 0.0686 & 0.1327 \\ + 160 & 0.2518 & 0.2040 & 0.2757 & 0.1293 & 0.0878 & 0.1503 \\ \midrule \multicolumn{7}{c}{\textbf{Convolution-Transformer}} \\ \midrule - 10 & 0.1472 & 0.1053 & 0.1651 & 0.0768 & 0.0317 & 0.0966 \\ - 20 & 0.1530 & 0.1307 & 0.1637 & 0.0745 & 0.0443 & 0.0886 \\ - 40 & \underline{0.1448} & 0.1228 & \underline{0.1514} & \underline{0.0709} & 0.0435 & \underline{0.0820} \\ - 80 & 0.1685 & 0.1164 & 0.1889 & 0.0865 & 0.0345 & 0.1089 \\ - 160 & 0.2440 & \underline{0.1051} & 0.3117 & 0.1403 & \underline{0.0286} & 0.1946 \\ + 10 & 0.1472 & 0.1053 & 0.1651 & 0.0768 & 0.0317 & 0.0966 \\ + 20 & 0.1530 & 0.1307 & 0.1637 & 0.0745 & 0.0443 & 0.0886 \\ + 40 & \underline{0.1448} & 0.1228 & \underline{0.1514} & \underline{0.0709} & 0.0435 & \underline{0.0820} \\ + 80 & 0.1685 & 0.1164 & 0.1889 & 0.0865 & 0.0345 & 0.1089 \\ + 160 & 0.2440 & \underline{0.1051} & 0.3117 & 0.1403 & \underline{0.0286} & 0.1946 \\ \bottomrule + \end{tabularx} \caption{Evaluation Metrics for the Ovulation-Over Target across Different Model Architectures and Input Lengths on a fixed Input Resolution of 12 Values per Day. \underline{Underlined} values represent the best value for each metric within a model. @@ -219,6 +267,12 @@ However, there is no consistent upward or downward trend when changing the input \subsubsection{Impact of Input Resolution}\label{subsubsec:ov_over_impact_of_input_resolution} +Table~\ref{tab:ov_over_results_by_resolution} shows results for the ovulation-over target across different resolutions. +Convolutional models are excluded, as their convolution layers inherently perform resampling. + +The LSTM generally performs best at medium resolutions (12--48 values/day), except for the after-ovulation metric, where higher resolutions yield better results. +The Transformer prefers higher resolutions, with optimal performance for all but the before-ovulation metric at 72 or 288 values/day. +Before-ovulation performance peaks at a medium resolution of 12 values/day. \begin{landscape} \begin{table} @@ -259,6 +313,64 @@ However, there is no consistent upward or downward trend when changing the input \end{table} \end{landscape} +\subsubsection{Impact of Model Parameters}\label{subsubsec:ov_over_impaoct_of_model_parameters} +\begin{landscape} + \begin{table} + \scriptsize + \begin{tabularx}{\linewidth}{l*{8}{X}} + \toprule + \multirow{2}{*}{Hidden Layer Size} & + \multirow{2}{*}{\# LSTM Layers} & + \multicolumn{3}{c}{MAE} & + \multicolumn{3}{c}{MSE} \\ + \cmidrule(lr){3-5} \cmidrule(lr){6-8} + & & Fert Overall & Fert Days & Non-Fert Days & Fert Overall & Fert Days & Non-Fert Days \\ + \midrule + 16 & 1 & 0.2253 & 0.1353 & 0.2735 & 0.1196 & 0.0397 & 0.1621 \\ + 32 & 1 & 0.1746 & 0.1255 & 0.2005 & 0.0860 & 0.0404 & 0.1109 \\ + 32 & 2 & 0.1349 & 0.1240 & 0.1399 & 0.0674 & 0.0396 & 0.0786 \\ + 64 & 2 & 0.1270 & 0.0943 & 0.1365 & 0.0645 & 0.0292 & 0.0772 \\ + 128 & 2 & 0.1151 & \textbf{0.0861} & 0.1243 & 0.0626 & \textbf{0.0274} & 0.0755 \\ + 128 & 4 & 0.1326 & 0.1064 & 0.1407 & 0.0658 & 0.0291 & 0.0811 \\ + 256 & 4 & 0.1353 & 0.1111 & 0.1407 & 0.0662 & 0.0360 & 0.0765 \\ + 512 & 4 & \textbf{0.1120} & 0.1358 & \textbf{0.0983} & \textbf{0.0616} & 0.0603 & \textbf{0.0613} \\ + \bottomrule + \end{tabularx} + \caption{Evaluation Metrics for the Ovulation-Over Target across Different Model Parameters for the LSTM model with + a fixed input window length of 160 days and an input resolution of 12 values per day. + \textbf{Bold} values represent the best value for each metric within a model.} + \label{tab:ov_over_results_by_model_parameters_lstm} + \end{table} + \begin{table} + \scriptsize + \begin{tabularx}{\linewidth}{l*{8}{X}} + \toprule + \multirow{2}{*}{Size of Embedding} & + \multirow{2}{*}{\# Encoder Layers} & + \multirow{2}{*}{\# Attention Heads} & + \multicolumn{3}{c}{MAE} & + \multicolumn{3}{c}{MSE} \\ + \cmidrule(lr){4-6} \cmidrule(lr){7-9} + & & & Fert Overall & Fert Days & Non-Fert Days & Fert Overall & Fert Days & Non-Fert Days \\ + \midrule + 16 & 1 & 1 & 0.4792 & 0.5280 & 0.4675 & 0.2336 & 0.2820 & 0.2220 \\ + 32 & 1 & 1 & 0.4406 & 0.4572 & 0.4417 & 0.2046 & 0.2179 & 0.2068 \\ + 64 & 1 & 1 & 0.3614 & 0.3411 & 0.3796 & 0.1696 & 0.1604 & 0.1800 \\ + 64 & 2 & 2 & 0.1532 & \textbf{0.0810} & 0.1913 & 0.0904 & 0.0302 & 0.1214 \\ + 128 & 2 & 2 & 0.1476 & 0.0979 & 0.1739 & 0.0816 & 0.0332 & 0.1064 \\ + 128 & 4 & 4 & \textbf{0.1114} & 0.0887 & 0.1236 & 0.0668 & 0.0368 & 0.0821 \\ + 256 & 4 & 4 & 0.1310 & 0.0920 & 0.1433 & 0.0722 & \textbf{0.0293} & 0.0876 \\ + 512 & 4 & 4 & 0.1222 & 0.1047 & 0.1261 & 0.0610 & 0.0321 & 0.0708 \\ + 512 & 8 & 8 & 0.1126 & 0.1965 & \textbf{0.0753} & \textbf{0.0543} & 0.0868 & \textbf{0.0410} \\ + \bottomrule + \end{tabularx} + \caption{Evaluation Metrics for the Ovulation-Over Target across Different Model Parameters for the Transformer model with + a fixed input window length of 160 days and an input resolution of 12 values per day. + \textbf{Bold} values represent the best value for each metric within a model.} + \label{tab:ov_over_results_by_model_parameters_transformer} + \end{table} +\end{landscape} + \subsubsection{Comparison with Baselines}\label{subsubsec:ov_over_comparison_with_baselines} \subsection{Stratified Analysis}\label{subsec:stratified_analysis}