alex/temperature-based-fertility-prediction-thesis
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
e71f1cd2e34c809f5d1c3a4ff5d05b3b93065295
temperature-based-fertility…/thesis/sections/appendix.tex
T
Alex Blank e71f1cd2e3 fixes and improvements
2025-09-09 13:44:45 +00:00

688 lines
51 KiB
TeX
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
\appendix
\appendixpage % prints "Appendices"
\section{Supplementary Results: Prediction of OV-over Indicator}\label{sec:appendix_ov_over_results}
\subsection{Comparative Study of Model Architectures and Parameters}\label{subsec:appendix_ov_over_architecture_results}
I will evaluate prediction of a binary indicator denoting whether ovulation has already occurred and
examine the influence of input window length, input resolution, and model capacity.
\paragraph{Impact of input window length.}
For predicting whether ovulation has occurred (OV-over),
optimal window lengths again vary by architecture and target phase (pre- vs post-ovulation).
The \textbf{Transformer} yields the lowest overall (0.0533) and post-ovulation MSE (0.0520) at 40 days.
The \textbf{LSTM} performs best before ovulation (MSE 0.0212 at 20 days), while its overall MSE improves with longer context (160 days).
The \textbf{Convolutional LSTM} favors short windows, with the best overall MSE (0.0699), before-OV (0.0389),
and after-OV (0.0833) all occurring at 20 days.
The \textbf{Convolutional Transformer} performs best overall at 40 days and best before ovulation at 160 days (MSE 0.0286).
See Table~\ref{tab:ovover_windows_compact_mse} for a summary.
\begin{table}[t]
\small
\renewcommand{\arraystretch}{1.15}
\setlength{\tabcolsep}{6pt}
\begin{tabularx}{\linewidth}{l*{3}{>{\centering\arraybackslash}X}}
\toprule
\textbf{Architecture} & \multicolumn{3}{c}{\textbf{MSE}} \\
\cmidrule(r){2-4}
& \shortstack[c]{Overall best\\(days / MSE)}
& \shortstack[c]{Before-OV best\\(days / MSE)}
& \shortstack[c]{After-OV best\\(days / MSE)} \\
\midrule
LSTM
& 160 / 0.0557
& \textbf{20 / 0.0212}
& 160 / 0.0580 \\
Transformer
& \textbf{40 / 0.0533}
& 20 / 0.0255
& \textbf{40 / 0.0520} \\
Convolutional LSTM
& 20 / 0.0699
& 20 / 0.0389
& 20 / 0.0833 \\
Convolutional Transformer
& 40 / 0.0709
& 160 / 0.0286
& 40 / 0.0820 \\
\bottomrule
\end{tabularx}
\caption{OV-Over: best input window per architecture (MSE only) at a fixed input resolution of 12 values/day.
Bold entries are bests within a column.}
\label{tab:ovover_windows_compact_mse}
\end{table}
\paragraph{Impact of input resolution.}
Fixing the input window to 20 days, I have compared input resolutions for LSTM and Transformer models.
For the \textbf{LSTM}, the lowest overall MSE (0.0633) occurs at 24/day,
with best before-OV and after-OV MSEs at 12/day (0.0212) and 48/day (0.0550), respectively.
The \textbf{Transformer} performs best overall at 72/day (0.0585),
with lowest before-OV MSE at 12/day (0.0255) and after-OV MSE at 288/day (0.0578).
Results are summarized in Table~\ref{tab:ovover_resolution_compact_mse};
full resolution grids are in Appendix Table~\ref{tab:ov_over_results_by_resolution}.
\begin{table}[t]
\small
\renewcommand{\arraystretch}{1.15}
\setlength{\tabcolsep}{6pt}
\begin{tabularx}{\linewidth}{l*{3}{>{\centering\arraybackslash}X}}
\toprule
\textbf{Architecture} & \multicolumn{3}{c}{\textbf{MSE}} \\
\cmidrule(r){2-4}
& \shortstack[c]{Overall best\\(values/day / MSE)}
& \shortstack[c]{Before-OV best\\(values/day / MSE)}
& \shortstack[c]{After-OV best\\(values/day / MSE)} \\
\midrule
LSTM
& 24 / 0.0633
& \textbf{12 / 0.0212}
& \textbf{48 / 0.0550} \\
Transformer
& \textbf{72 / 0.0585}
& 12 / 0.0255
& 288 / 0.0578 \\
\bottomrule
\end{tabularx}
\caption{OV-Over: best input resolution per architecture (MSE only) at a fixed input-window length of 20\,days.
Bold entries are bests within a column. Convolutional models are excluded (they consume 288 values/day internally).}
\label{tab:ovover_resolution_compact_mse}
\end{table}
\paragraph{Impact of model parameters.}
Using fixed input settings (160 days at 12/day for LSTM/Transformer; 40 days for convolutional models), I have evaluated model scaling.
The \textbf{Transformer} achieves the lowest overall (0.0543) and after-ovulation MSE (0.0410) at a large configuration (512×8×8).
Before-OV MSE is lowest at 256×4×4 (0.0293).
For the \textbf{LSTM}, the best overall and after-OV performance is at 512×4,
while the best before-OV MSE occurs at 128×2 (0.0274).
The \textbf{Convolutional LSTM} performs best overall and after-OV at 256×4, with the best before-OV MSE at 128×2.
Finally, the \textbf{Convolutional Transformer} achieves its lowest overall and after-OV MSE at 512×4×4, and best before-OV MSE at 512×8×8.
Table~\ref{tab:ovover_params_compact_mse} summarizes these parameter-dependent results;
full comparisons are included in Appendix Tables~\ref{tab:ov_over_results_by_model_parameters_lstm}–\ref{tab:ov_over_results_by_model_parameters_conv_transformer}.
\begin{table}[t]
\scriptsize
\renewcommand{\arraystretch}{1.15}
\setlength{\tabcolsep}{6pt}
\begin{tabularx}{\linewidth}{l*{3}{>{\centering\arraybackslash}X}}
\toprule
\textbf{Architecture} & \multicolumn{3}{c}{\textbf{MSE}} \\
\cmidrule(r){2-4}
& \shortstack[c]{Overall best\\(params / MSE)}
& \shortstack[c]{Before-OV best\\(params / MSE)}
& \shortstack[c]{After-OV best\\(params / MSE)} \\
\midrule
LSTM
& \(512\times4\) / 0.0616
& \textbf{\(128\times2\) / 0.0274}
& \(512\times4\) / 0.0613 \\
Transformer
& \textbf{\(512\times8\times8\) / 0.0543}
& \(256\times4\times4\) / 0.0293
& \textbf{\(512\times8\times8\) / 0.0410} \\
Convolutional LSTM
& \(256\times4\) / 0.0687
& \(128\times2\) / 0.0357
& \(256\times4\) / 0.0715 \\
Convolutional Transformer
& \(512\times4\times4\) / 0.0703
& \(512\times8\times8\) / 0.0339
& \(512\times4\times4\) / 0.0814 \\
\bottomrule
\end{tabularx}
\caption{OV-Over: best parameter settings per architecture (MSE only).
Parameters are formatted as \(\text{hidden}\times\text{layers}\) (LSTM/Conv.\ LSTM) and \(\text{embedding}\times\text{encoder layers}\times\text{heads}\) (Transformer/Conv.\ Transformer).
Bold values indicate bests within a column. Fixed input settings: 160\,days with 12 values/day for LSTM/Transformer; 40\,days for convolutional models.}
\label{tab:ovover_params_compact_mse}
\end{table}
\subsection{Regular vs. Irregular Cycles}\label{subsec:appendix_ov_over_regular_vs_irregular}
\begin{table}[htbp]
\centering
\small
\begin{tabularx}{\linewidth}{l*{3}{X}}
\toprule
\multirow{2}{*}{Model} & \multicolumn{3}{c}{MSE} \\
\cmidrule(r){2-4}
& OV-Over Overall & OV-Over Before OV & OV-Over After OV \\
\midrule
\multicolumn{4}{c}{\textbf{Regular Cycle Group}} \\
\midrule
LSTM & \textbf{0.028519} & 0.031695 & \textbf{0.026116} \\
Transformer & 0.032929 & \textbf{0.024774} & 0.033411 \\
Convolutional LSTM & 0.034224 & 0.028962 & 0.033929 \\
Convolutional Transformer & 0.033333 & 0.039045 & 0.029329 \\
Last-Cycle Baseline & 0.087916 & 0.093465 & 0.074191 \\
Population Mean Baseline & 0.285887 & 0.008876 & 0.418879 \\
User Mean Baseline & 0.069245 & 0.069152 & 0.055724 \\
\midrule
\multicolumn{4}{c}{\textbf{Irregular Cycle Group}} \\
\midrule
LSTM & \textbf{0.051747} & \textbf{0.016777} & 0.070419 \\
Transformer & 0.055990 & 0.026159 & 0.067520 \\
Convolutional LSTM & 0.054523 & 0.030092 & \textbf{0.066643} \\
Convolutional Transformer & 0.056075 & 0.029638 & 0.067965 \\
Last-Cycle Baseline & 0.222906 & 0.134945 & 0.258047 \\
Population Mean Baseline & 0.171368 & 0.130366 & 0.140363 \\
User Mean Baseline & 0.180702 & 0.088884 & 0.217144 \\
\bottomrule
\end{tabularx}
\caption{MSEs for OV-over target across different model architectures for regular and irregular cycle groups. Bold values denote the best scores per column.}
\label{tab:ov_over_mse_regular_irregular}
\end{table}
\noindent
Table~\ref{tab:ov_over_mse_regular_irregular} shows the MSE metric results for all models on the regular and irregular cycle groups for the OV-over target.
For this target, performance patterns differ more distinctly across phases.
In the regular group, the LSTM performs best overall and after ovulation, while the Transformer performs best before ovulation.
Convolutional models perform slightly worse than their recurrent and transformer-based counterparts across all phases.
The population mean baseline shows a spurious low MSE before ovulation, but this is not consistent across phases, suggesting it is not reliable.
In general, baselines are notably weaker than learned models across all splits.
For irregular cycles, all model performances deteriorate relative to regular cycles.
The LSTM remains the most robust, achieving the best MSEs both overall and before ovulation.
Convolutional LSTM performs best after ovulation.
Transformer-based models perform less consistently in this group, suggesting decreased robustness to cycle irregularity.
As before, all learned models outperform the baselines by a wide margin.
The full table with MSE and MAE for all models can be found in the appendix, Table~\ref{tab:regular_vs_irregular_ov_over_results}.
\section{Extra Figures}\label{app:figs}
\begin{landscape}
\begin{table}
\small
\begin{tabularx}{\linewidth}{l*{6}{X}}
\toprule
\multirow{2}{*}{Input-Length in Days} & \multicolumn{3}{c}{MAE} & \multicolumn{3}{c}{MSE} \\
\cmidrule(r){2-4} \cmidrule(r){5-7}
& Fertility Overall & Fertile Days & Non-Fertile Days & Fertility Overall & Fertile Days & Non-Fertile Days \\
\midrule
\multicolumn{7}{c}{\textbf{LSTM}} \\ \midrule
10 & 0.0399 & 0.0857 & 0.0212 & 0.0045 & 0.0105 & 0.0021 \\
20 & 0.0421 & 0.0994 & 0.0180 & 0.0052 & 0.0144 & \underline{\textbf{0.0013}} \\
40 & 0.0406 & 0.0938 & 0.0184 & 0.0049 & 0.0128 & 0.0016 \\
80 & 0.0420 & 0.1006 & \underline{\textbf{0.0176}} & 0.0053 & 0.0147 & 0.0014 \\
160 & \underline{0.0378} & \underline{\textbf{0.0834}} & 0.0192 & \underline{0.0043} & \underline{\textbf{0.0102}} & 0.0019 \\
\midrule
\multicolumn{7}{c}{\textbf{Transformer}} \\ \midrule
10 & 0.0443 & 0.0868 & 0.0273 & 0.0046 & 0.0110 & 0.0021 \\
20 & 0.0472 & 0.0949 & 0.0274 & 0.0051 & 0.0133 & 0.0018 \\
40 & 0.0413 & 0.0861 & 0.0233 & \underline{0.0044} & 0.0111 & 0.0017 \\
80 & 0.0437 & \underline{0.0859} & 0.0269 & 0.0045 & \underline{0.0108} & 0.0021 \\
160 & \underline{0.0411} & 0.0882 & \underline{0.0218} & 0.0045 & 0.0116 & \underline{0.0016} \\
\midrule
\multicolumn{7}{c}{\textbf{Convolutional-LSTM}} \\ \midrule
10 & 0.0435 & 0.0954 & 0.0224 & 0.0050 & 0.0133 & 0.0017 \\
20 & 0.0414 & 0.0996 & \underline{0.0179} & 0.0050 & 0.0145 & \underline{\textbf{0.0013}} \\
40 & \underline{0.0394} & \underline{0.0911} & 0.0184 & \underline{0.0045} & \underline{0.0122} & 0.0014 \\
80 & 0.0481 & 0.1005 & 0.0278 & 0.0054 & 0.0146 & 0.0019 \\
160 & 0.0505 & 0.1067 & 0.0290 & 0.0060 & 0.0166 & 0.0020 \\
\midrule
\multicolumn{7}{c}{\textbf{Convolutional-Transformer}} \\ \midrule
10 & 0.0459 & 0.0962 & 0.0256 & 0.0049 & 0.0136 & \underline{0.0015} \\
20 & 0.0423 & \underline{0.0848} & 0.0257 & 0.0045 & \underline{\textbf{0.0102}} & 0.0022 \\
40 & \underline{\textbf{0.0376}} & 0.0854 & \underline{0.0186} & \underline{\textbf{0.0041}} & 0.0108 & \underline{0.0015} \\
80 & 0.0414 & 0.0932 & 0.0210 & 0.0048 & 0.0128 & 0.0018 \\
160 & 0.0523 & 0.1018 & 0.0337 & 0.0059 & 0.0148 & 0.0026 \\
\bottomrule
\end{tabularx}
\caption{Evaluation Metrics for the Fertility-Probability Target across Different Model Architectures and Input Lengths at a fixed Input Resolution of 12 Values per Day.
\underline{Underlined} values represent the best value for each metric within a model.
\textbf{\underline{Bold + Underlined}} values represent the global best values across all models for a given metric.}
\label{tab:fertility_results_by_window_length}
\end{table}
\end{landscape}
\begin{landscape}
\begin{table}
\small
\begin{tabularx}{\linewidth}{l*{6}{X}}
\toprule
\multirow{2}{*}{Values Per Day} & \multicolumn{3}{c}{MAE} & \multicolumn{3}{c}{MSE} \\
\cmidrule(r){2-4} \cmidrule(r){5-7}
& Fertility Overall & Fertile Days & Non-Fertile Days & Fertility Overall & Fertile Days & Non-Fertile Days \\
\midrule
\multicolumn{7}{c}{\textbf{LSTM}} \\
\midrule
1 & 0.0471 & 0.0987 & 0.0220 & 0.0062 & 0.0145 & 0.0022 \\
2 & 0.0457 & 0.0962 & 0.0231 & 0.0052 & 0.0133 & 0.0016 \\
4 & 0.0421 & 0.0898 & 0.0223 & \underline{\textbf{0.0046}}& 0.0116 & 0.0018 \\
12 & 0.0421 & 0.0994 & \underline{\textbf{0.0180}}& 0.0052 & 0.0144 & \underline{\textbf{0.0013}}\\
24 & 0.0419 & 0.0949 & 0.0201 & 0.0050 & 0.0132 & 0.0017 \\
48 & \underline{\textbf{0.0402}}& 0.0929 & 0.0182 & 0.0049 & 0.0126 & 0.0018 \\
72 & 0.0433 & 0.0972 & 0.0216 & 0.0052 & 0.0137 & 0.0018 \\
288 & 0.0410 & \underline{\textbf{0.0872}}& 0.0223 & 0.0049 & \underline{\textbf{0.0110}}& 0.0025 \\
\midrule
\multicolumn{7}{c}{\textbf{Transformer}} \\
\midrule
1 & 0.0541 & 0.1015 & 0.0313 & 0.0063 & 0.0146 & 0.0023 \\
2 & 0.0468 & 0.0946 & 0.0259 & 0.0054 & 0.0128 & 0.0021 \\
4 & 0.0456 & \underline{0.0891}& 0.0277 & 0.0050 & \underline{0.0115}& 0.0023 \\
12 & 0.0472 & 0.0949 & 0.0274 & 0.0051 & 0.0133 & 0.0018 \\
24 & 0.0502 & 0.0960 & 0.0323 & 0.0052 & 0.0133 & 0.0021 \\
48 & 0.0447 & 0.0912 & \underline{0.0254}& \underline{0.0048}& 0.0122 & 0.0017 \\
72 & 0.0468 & 0.0975 & 0.0257 & 0.0051 & 0.0140 & \underline{0.0014} \\
288 & \underline{0.0449}& 0.0937 & 0.0257 & \underline{0.0048}& 0.0128 & 0.0017 \\
\bottomrule
\end{tabularx}
\caption{Evaluation Metrics for the Fertility-Probability Target across Different Model Architectures and Input Resolutions at a fixed Input-Window-Length of 20 Days.
\underline{Underlined} values represent the best value for each metric within a model.
\textbf{\underline{Bold + Underlined}} values represent the global best values across all models for a given metric.}
\label{tab:fertility_results_by_window_resolution}
\end{table}
\end{landscape}
\begin{landscape}
\begin{table}
\scriptsize
\begin{tabularx}{\linewidth}{l*{8}{X}}
\midrule
\multicolumn{8}{c}{\textbf{LSTM}} \\
\toprule
\multirow{2}{*}{Hidden Layer Size} &
\multirow{2}{*}{\# LSTM Layers} &
\multicolumn{3}{c}{MAE} &
\multicolumn{3}{c}{MSE} \\
\cmidrule(lr){3-5} \cmidrule(lr){6-8}
& & Fert Overall & Fert Days & Non-Fert Days & Fert Overall & Fert Days & Non-Fert Days \\
\midrule
16 & 1 & 0.0485 & 0.1050 & 0.0252 & 0.0059 & 0.0161 & 0.0016 \\
32 & 1 & 0.0477 & 0.1029 & 0.0248 & 0.0057 & 0.0155 & 0.0016 \\
32 & 2 & 0.0458 & 0.0922 & 0.0269 & 0.0052 & \textbf{0.0122} & 0.0024 \\
64 & 2 & 0.0447 & 0.0925 & 0.0250 & 0.0051 & 0.0123 & 0.0021 \\
128 & 2 & 0.0424 & 0.0951 & 0.0201 & 0.0051 & 0.0131 & 0.0018 \\
128 & 4 & 0.0427 & 0.0989 & 0.0191 & 0.0053 & 0.0145 & 0.0014 \\
256 & 4 & 0.0433 & 0.1050 & \textbf{0.0175} & 0.0055 & 0.0161 & \textbf{0.0011} \\
512 & 4 & \textbf{0.0399} & \textbf{0.0911} & 0.0189 & \textbf{0.0047} & \textbf{0.0122} & 0.0016 \\
\bottomrule
\end{tabularx}
\caption{Evaluation Metrics for the fertility-probability target across Different Model Parameters for the LSTM model with
a fixed input window length of 160 days and an input resolution of 12 values per day.
\textbf{Bold} values represent the best value for each metric within a model.}
\label{tab:fertility_results_by_model_parameters_lstm}
\end{table}
\begin{table}
\scriptsize
\begin{tabularx}{\linewidth}{l*{8}{X}}
\midrule
\multicolumn{9}{c}{\textbf{Transformer}} \\
\toprule
\multirow{2}{*}{Size of Embedding} &
\multirow{2}{*}{\# Encoder Layers} &
\multirow{2}{*}{\# Attention Heads} &
\multicolumn{3}{c}{MAE} &
\multicolumn{3}{c}{MSE} \\
\cmidrule(lr){4-6} \cmidrule(lr){7-9}
& & & Fert Overall & Fert Days & Non-Fert Days & Fert Overall & Fert Days & Non-Fert Days \\
\midrule
16 & 1 & 1 & 0.0708 & 0.1262 & 0.0475 & 0.0084 & 0.0227 & 0.0024 \\
32 & 1 & 1 & 0.0599 & 0.1163 & 0.0378 & 0.0070 & 0.0196 & 0.0019 \\
64 & 1 & 1 & 0.0601 & 0.1178 & 0.0373 & 0.0071 & 0.0200 & 0.0020 \\
64 & 2 & 2 & 0.0477 & 0.0908 & 0.0301 & 0.0049 & 0.0121 & 0.0020 \\
128 & 2 & 2 & 0.0449 & 0.0904 & 0.0263 & 0.0047 & 0.0119 & 0.0017 \\
128 & 4 & 4 & 0.0461 & 0.0979 & 0.0247 & 0.0050 & 0.0143 & 0.0013 \\
256 & 4 & 4 & 0.0475 & 0.0919 & 0.0293 & 0.0048 & 0.0125 & 0.0017 \\
512 & 4 & 4 & 0.0403 & \textbf{0.0831} & 0.0229 & \textbf{0.0043} & \textbf{0.0103} & 0.0018 \\
512 & 8 & 8 & \textbf{0.0395} & 0.0967 & \textbf{0.0159} & 0.0048 & 0.0139 & \textbf{0.0011} \\
\bottomrule
\end{tabularx}
\caption{Evaluation Metrics for the fertility probability target across Different Model Parameters for the Transformer model with
a fixed input window length of 160 days and an input resolution of 12 values per day.
\textbf{Bold} values represent the best value for each metric within a model.}
\label{tab:fertility_results_by_model_parameters_transformer}
\end{table}
\begin{table}
\scriptsize
\begin{tabularx}{\linewidth}{l*{8}{X}}
\midrule
\multicolumn{8}{c}{\textbf{Convolutional-LSTM}} \\
\toprule
\multirow{2}{*}{Hidden Layer Size} &
\multirow{2}{*}{\# LSTM Layers} &
\multicolumn{3}{c}{MAE} &
\multicolumn{3}{c}{MSE} \\
\cmidrule(lr){3-5} \cmidrule(lr){6-8}
& & Fert Overall & Fert Days & Non-Fert Days & Fert Overall & Fert Days & Non-Fert Days \\
\midrule
16 & 1 & 0.0487 & 0.1029 & 0.0264 & 0.0056 & 0.0154 & \textbf{0.0016} \\
32 & 1 & 0.0461 & 0.0956 & 0.0266 & 0.0050 & 0.0133 & 0.0017 \\
32 & 2 & 0.0448 & 0.0975 & 0.0232 & 0.0051 & 0.0139 & \textbf{0.0016} \\
64 & 2 & 0.0432 & 0.0921 & 0.0241 & 0.0048 & 0.0123 & 0.0019 \\
128 & 2 & 0.0400 & 0.0895 & 0.0204 & 0.0045 & 0.0119 & \textbf{0.0016} \\
128 & 4 & 0.0398 & 0.0871 & 0.0204 & 0.0044 & 0.0113 & \textbf{0.0016} \\
256 & 4 & 0.0389 & \textbf{0.0826} & 0.0209 & \textbf{0.0042} & \textbf{0.0100} & 0.0018 \\
512 & 4 & \textbf{0.0380} & 0.0869 & \textbf{0.0184} & 0.0044 & 0.0112 & 0.0017 \\
\bottomrule
\end{tabularx}
\caption{Evaluation Metrics for the fertility probability target across Different Model Parameters for the convolutional LSTM model with
a fixed input window length of 40 days.
\textbf{Bold} values represent the best value for each metric within a model.}
\label{tab:fertility_results_by_model_parameters_conv_lstm}
\end{table}
\begin{table}
\scriptsize
\begin{tabularx}{\linewidth}{l*{8}{X}}
\midrule
\multicolumn{8}{c}{\textbf{Convolutional-Transformer}} \\
\toprule
\multirow{2}{*}{Size of Embedding} &
\multirow{2}{*}{\# Encoder Layers} &
\multirow{2}{*}{\# Attention Heads} &
\multicolumn{3}{c}{MAE} &
\multicolumn{3}{c}{MSE} \\
\cmidrule(lr){4-6} \cmidrule(lr){7-9}
& & & Fert Overall & Fert Days & Non-Fert Days & Fert Overall & Fert Days & Non-Fert Days \\
\midrule
16 & 1 & 1 & 0.0574 & 0.1005 & 0.0427 & 0.0061 & 0.0146 & 0.0032 \\
32 & 1 & 1 & 0.0462 & 0.0963 & 0.0266 & 0.0051 & 0.0135 & 0.0018 \\
64 & 1 & 1 & 0.0546 & 0.1006 & 0.0379 & 0.0059 & 0.0145 & 0.0027 \\
64 & 2 & 2 & 0.0427 & 0.0903 & 0.0237 & 0.0046 & 0.0118 & 0.0018 \\
128 & 2 & 2 & 0.0406 & 0.0868 & 0.0225 & 0.0044 & 0.0113 & 0.0017 \\
128 & 4 & 4 & 0.0411 & 0.0886 & 0.0230 & 0.0044 & 0.0116 & 0.0017 \\
256 & 4 & 4 & 0.0420 & 0.0866 & 0.0240 & \textbf{0.0043} & 0.0113 & 0.0016 \\
512 & 4 & 4 & \textbf{0.0377} & 0.0878 & \textbf{0.0177} & \textbf{0.0043} & 0.0117 & \textbf{0.0013} \\
512 & 8 & 8 & 0.0399 & \textbf{0.0842} & 0.0224 & 0.0044 & \textbf{0.0107} & 0.0019 \\
\bottomrule
\end{tabularx}
\caption{Evaluation Metrics for the fertility probability target across Different Model Parameters for the convolutional Transformer model with
a fixed input window length of 40 days.
\textbf{Bold} values represent the best value for each metric within a model.}
\label{tab:fertility_results_by_model_parameters_conv_transformer}
\end{table}
\end{landscape}
\begin{landscape}
\begin{table}
\small
\begin{tabularx}{\linewidth}{l*{6}{X}}
\toprule
\multirow{2}{*}{Input-Length in Days} & \multicolumn{3}{c}{MAE} & \multicolumn{3}{c}{MSE} \\
\cmidrule(r){2-4} \cmidrule(r){5-7}
& OV-Over Overall & OV-Over Before OV & OV-Over After OV & OV-Over Overall & OV-Over Before OV & OV-Over After OV \\
\midrule
\multicolumn{7}{c}{\textbf{LSTM}} \\
\midrule
10 & 0.1218 & 0.1044 & 0.1223 & 0.0612 & 0.0289 & 0.0711 \\
20 & 0.1153 & \underline{\textbf{0.0745}} & 0.1312 & 0.0641 & \underline{\textbf{0.0212}} & 0.0822 \\
40 & 0.1066 & 0.0820 & 0.1128 & 0.0616 & 0.0281 & 0.0740 \\
80 & 0.1173 & 0.0842 & 0.1291 & 0.0647 & 0.0263 & 0.0801 \\
160 & \underline{0.1039} & 0.1139 & \underline{0.0973} & \underline{0.0557} & 0.0462 & \underline{0.0580} \\
\midrule
\multicolumn{7}{c}{\textbf{Transformer}} \\
\midrule
10 & 0.1137 & 0.1006 & 0.1186 & 0.0618 & 0.0336 & 0.0740 \\
20 & 0.1204 & 0.0771 & 0.1366 & 0.0690 & \underline{0.0255} & 0.0864 \\
40 & \underline{\textbf{0.1017}} & 0.1409 & \underline{\textbf{0.0883}} & \underline{\textbf{0.0533}} & 0.0621 & \underline{\textbf{0.0520}} \\
80 & 0.1138 & \underline{0.0897} & 0.1234 & 0.0654 & 0.0356 & 0.0788 \\
160 & 0.1076 & 0.0959 & 0.1141 & 0.0606 & 0.0379 & 0.0714 \\
\midrule
\multicolumn{7}{c}{\textbf{Convolutional-LSTM}} \\
\midrule
10 & 0.1847 & 0.1878 & 0.1840 & 0.0819 & 0.0581 & 0.0949 \\
20 & 0.1493 & 0.1274 & 0.1562 & \underline{0.0699} & \underline{0.0389} & \underline{0.0833} \\
40 & \underline{0.1455} & \underline{0.1089} & \underline{0.1561} & 0.0722 & 0.0358 & 0.0852 \\
80 & 0.2327 & 0.1796 & 0.2507 & 0.1168 & 0.0686 & 0.1327 \\
160 & 0.2518 & 0.2040 & 0.2757 & 0.1293 & 0.0878 & 0.1503 \\
\midrule
\multicolumn{7}{c}{\textbf{Convolutional-Transformer}} \\
\midrule
10 & 0.1472 & 0.1053 & 0.1651 & 0.0768 & 0.0317 & 0.0966 \\
20 & 0.1530 & 0.1307 & 0.1637 & 0.0745 & 0.0443 & 0.0886 \\
40 & \underline{0.1448} & 0.1228 & \underline{0.1514} & \underline{0.0709} & 0.0435 & \underline{0.0820} \\
80 & 0.1685 & 0.1164 & 0.1889 & 0.0865 & 0.0345 & 0.1089 \\
160 & 0.2440 & \underline{0.1051} & 0.3117 & 0.1403 & \underline{0.0286} & 0.1946 \\
\bottomrule
\end{tabularx}
\caption{Evaluation Metrics for the OV-over Target across Different Model Architectures and Input Lengths at a fixed Input Resolution of 12 Values per Day.
\underline{Underlined} values represent the best value for each metric within a model.
\textbf{\underline{Bold + Underlined}} values represent the global best values across all models for a given metric.}
\label{tab:ov_over_results_by_window_length}
\end{table}
\end{landscape}
\begin{landscape}
\begin{table}
\small
\begin{tabularx}{\linewidth}{l*{6}{X}}
\toprule
\multirow{2}{*}{Values Per Day} & \multicolumn{3}{c}{MAE} & \multicolumn{3}{c}{MSE} \\
\cmidrule(r){2-4} \cmidrule(r){5-7}
& OV-Over Overall & OV-Over Before OV & OV-Over After OV & OV-Over Overall & OV-Over Before OV & OV-Over After OV \\
\midrule
\multicolumn{7}{c}{\textbf{LSTM}} \\
\midrule
1 & 0.1691 & 0.1280 & 0.1819 & 0.0914 & 0.0411 & 0.1099 \\
2 & 0.1431 & 0.1239 & 0.1471 & 0.0747 & 0.0442 & 0.0849 \\
4 & 0.1344 & 0.1130 & 0.1417 & 0.0680 & 0.0371 & 0.0807 \\
12 & \underline{0.1153} & \underline{\textbf{0.0745}} & 0.1312 & 0.0641 & \underline{\textbf{0.0212}} & 0.0822 \\
24 & 0.1192 & 0.0975 & 0.1240 & \underline{0.0633} & 0.0288 & 0.0755 \\
48 & 0.1463 & 0.2493 & \underline{0.0973} & 0.0768 & 0.1203 & \underline{\textbf{0.0550}} \\
72 & 0.1587 & 0.1113 & 0.1844 & 0.0857 & 0.0341 & 0.1126 \\
288 & 0.1353 & 0.1801 & 0.1169 & 0.0726 & 0.0799 & 0.0713 \\
\midrule
\multicolumn{7}{c}{\textbf{Transformer}} \\
\midrule
1 & 0.1687 & 0.1582 & 0.1709 & 0.0883 & 0.0578 & 0.0996 \\
2 & 0.1469 & 0.0895 & 0.1676 & 0.0823 & 0.0260 & 0.1044 \\
4 & 0.1282 & 0.0999 & 0.1393 & 0.0704 & 0.0329 & 0.0862 \\
12 & 0.1204 & \underline{0.0771}& 0.1366 & 0.0690 & \underline{0.0255}& 0.0864 \\
24 & 0.1952 & 0.1649 & 0.2102 & 0.0904 & 0.0638 & 0.1040 \\
48 & 0.1137 & 0.1159 & 0.1073 & 0.0610 & 0.0480 & 0.0629 \\
72 & \underline{\textbf{0.1041}}& 0.1180 & \underline{\textbf{0.0947}}& \underline{\textbf{0.0585}}& 0.0538 & 0.0581 \\
288 & 0.1319 & 0.1879 & 0.1097 & 0.0617 & 0.0769 & \underline{0.0578}\\
\bottomrule
\end{tabularx}
\caption{Evaluation Metrics for the OV-over Target across Different Model Architectures and Input Resolutions at a fixed Input-Window-Length of 20 Days.
\underline{Underlined} values represent the best value for each metric within a model.
\textbf{\underline{Bold + Underlined}} values represent the global best values across all models for a given metric.}
\label{tab:ov_over_results_by_resolution}
\end{table}
\end{landscape}
\begin{landscape}
\begin{table}
\scriptsize
\begin{tabularx}{\linewidth}{l*{8}{X}}
\midrule
\multicolumn{8}{c}{\textbf{LSTM}} \\
\toprule
\multirow{2}{*}{Hidden Layer Size} &
\multirow{2}{*}{\# LSTM Layers} &
\multicolumn{3}{c}{MAE} &
\multicolumn{3}{c}{MSE} \\
\cmidrule(lr){3-5} \cmidrule(lr){6-8}
& & Fert Overall & Fert Days & Non-Fert Days & Fert Overall & Fert Days & Non-Fert Days \\
\midrule
16 & 1 & 0.2253 & 0.1353 & 0.2735 & 0.1196 & 0.0397 & 0.1621 \\
32 & 1 & 0.1746 & 0.1255 & 0.2005 & 0.0860 & 0.0404 & 0.1109 \\
32 & 2 & 0.1349 & 0.1240 & 0.1399 & 0.0674 & 0.0396 & 0.0786 \\
64 & 2 & 0.1270 & 0.0943 & 0.1365 & 0.0645 & 0.0292 & 0.0772 \\
128 & 2 & 0.1151 & \textbf{0.0861} & 0.1243 & 0.0626 & \textbf{0.0274} & 0.0755 \\
128 & 4 & 0.1326 & 0.1064 & 0.1407 & 0.0658 & 0.0291 & 0.0811 \\
256 & 4 & 0.1353 & 0.1111 & 0.1407 & 0.0662 & 0.0360 & 0.0765 \\
512 & 4 & \textbf{0.1120} & 0.1358 & \textbf{0.0983} & \textbf{0.0616} & 0.0603 & \textbf{0.0613} \\
\bottomrule
\end{tabularx}
\caption{Evaluation Metrics for the OV-over Target across Different Model Parameters for the LSTM model with
a fixed input window length of 160 days and an input resolution of 12 values per day.
\textbf{Bold} values represent the best value for each metric within a model.}
\label{tab:ov_over_results_by_model_parameters_lstm}
\end{table}
\begin{table}
\scriptsize
\begin{tabularx}{\linewidth}{l*{8}{X}}
\midrule
\multicolumn{9}{c}{\textbf{Transformer}} \\
\toprule
\multirow{2}{*}{Size of Embedding} &
\multirow{2}{*}{\# Encoder Layers} &
\multirow{2}{*}{\# Attention Heads} &
\multicolumn{3}{c}{MAE} &
\multicolumn{3}{c}{MSE} \\
\cmidrule(lr){4-6} \cmidrule(lr){7-9}
& & & Fert Overall & Fert Days & Non-Fert Days & Fert Overall & Fert Days & Non-Fert Days \\
\midrule
16 & 1 & 1 & 0.4792 & 0.5280 & 0.4675 & 0.2336 & 0.2820 & 0.2220 \\
32 & 1 & 1 & 0.4406 & 0.4572 & 0.4417 & 0.2046 & 0.2179 & 0.2068 \\
64 & 1 & 1 & 0.3614 & 0.3411 & 0.3796 & 0.1696 & 0.1604 & 0.1800 \\
64 & 2 & 2 & 0.1532 & \textbf{0.0810} & 0.1913 & 0.0904 & 0.0302 & 0.1214 \\
128 & 2 & 2 & 0.1476 & 0.0979 & 0.1739 & 0.0816 & 0.0332 & 0.1064 \\
128 & 4 & 4 & \textbf{0.1114} & 0.0887 & 0.1236 & 0.0668 & 0.0368 & 0.0821 \\
256 & 4 & 4 & 0.1310 & 0.0920 & 0.1433 & 0.0722 & \textbf{0.0293} & 0.0876 \\
512 & 4 & 4 & 0.1222 & 0.1047 & 0.1261 & 0.0610 & 0.0321 & 0.0708 \\
512 & 8 & 8 & 0.1126 & 0.1965 & \textbf{0.0753} & \textbf{0.0543} & 0.0868 & \textbf{0.0410} \\
\bottomrule
\end{tabularx}
\caption{Evaluation Metrics for the OV-over Target across Different Model Parameters for the Transformer model with
a fixed input window length of 160 days and an input resolution of 12 values per day.
\textbf{Bold} values represent the best value for each metric within a model.}
\label{tab:ov_over_results_by_model_parameters_transformer}
\end{table}
\begin{table}
\scriptsize
\begin{tabularx}{\linewidth}{l*{8}{X}}
\midrule
\multicolumn{8}{c}{\textbf{Convolutional-LSTM}} \\
\toprule
\multirow{2}{*}{Hidden Layer Size} &
\multirow{2}{*}{\# LSTM Layers} &
\multicolumn{3}{c}{MAE} &
\multicolumn{3}{c}{MSE} \\
\cmidrule(lr){3-5} \cmidrule(lr){6-8}
& & Fert Overall & Fert Days & Non-Fert Days & Fert Overall & Fert Days & Non-Fert Days \\
\midrule
16 & 1 & 0.2920 & 0.2198 & 0.3344 & 0.1264 & 0.0602 & 0.1653 \\
32 & 1 & 0.2141 & 0.1598 & 0.2377 & 0.0912 & 0.0398 & 0.1149 \\
32 & 2 & 0.1990 & 0.1556 & 0.2135 & 0.0852 & 0.0419 & 0.1025 \\
64 & 2 & 0.1743 & 0.1345 & 0.1845 & 0.0789 & 0.0400 & 0.0923 \\
128 & 2 & 0.1524 & 0.1138 & 0.1633 & 0.0717 & \textbf{0.0357} & 0.0845 \\
128 & 4 & 0.1579 & 0.1185 & 0.1670 & 0.0777 & 0.0401 & 0.0894 \\
256 & 4 & \textbf{0.1424} & 0.1425 & \textbf{0.1369} & \textbf{0.0687} & 0.0546 & \textbf{0.0715} \\
512 & 4 & 0.1436 & \textbf{0.1166} & 0.1523 & 0.0699 & 0.0382 & 0.0820 \\
\bottomrule
\end{tabularx}
\caption{Evaluation Metrics for the OV-over Target across Different Model Parameters for the convolutional LSTM model with
a fixed input window length of 40 days.
\textbf{Bold} values represent the best value for each metric within a model.}
\label{tab:ov_over_results_by_model_parameters_conv_lstm}
\end{table}
\begin{table}
\scriptsize
\begin{tabularx}{\linewidth}{l*{8}{X}}
\midrule
\multicolumn{8}{c}{\textbf{Convolutional-Transformer}} \\
\toprule
\multirow{2}{*}{Size of Embedding} &
\multirow{2}{*}{\# Encoder Layers} &
\multirow{2}{*}{\# Attention Heads} &
\multicolumn{3}{c}{MAE} &
\multicolumn{3}{c}{MSE} \\
\cmidrule(lr){4-6} \cmidrule(lr){7-9}
& & & Fert Overall & Fert Days & Non-Fert Days & Fert Overall & Fert Days & Non-Fert Days \\
\midrule
16 & 1 & 1 & 0.3759 & 0.3602 & 0.3855 & 0.1597 & 0.1498 & 0.1659 \\
32 & 1 & 1 & 0.1757 & 0.1119 & 0.2017 & 0.0916 & 0.0314 & 0.1167 \\
64 & 1 & 1 & 0.2771 & 0.2395 & 0.2978 & 0.1303 & 0.1133 & 0.1402 \\
64 & 2 & 2 & 0.1610 & 0.1149 & 0.1777 & 0.0816 & \textbf{0.0310} & 0.1021 \\
128 & 2 & 2 & 0.1501 & 0.0997 & 0.1725 & 0.0783 & 0.0316 & 0.0991 \\
128 & 4 & 4 & 0.1668 & 0.1472 & 0.1758 & 0.0801 & 0.0488 & 0.0953 \\
256 & 4 & 4 & 0.1529 & 0.1317 & 0.1597 & 0.0742 & 0.0435 & 0.0861 \\
512 & 4 & 4 & \textbf{0.1495} & 0.1298 & \textbf{0.1551} & \textbf{0.0703} & 0.0409 & \textbf{0.0814} \\
512 & 8 & 8 & 0.1576 & \textbf{0.1025} & 0.1848 & 0.0825 & 0.0339 & 0.1066 \\
\bottomrule
\end{tabularx}
\caption{Evaluation Metrics for the OV-over Target across Different Model Parameters for the convolutional Transformer model with
a fixed input window length of 40 days.
\textbf{Bold} values represent the best value for each metric within a model.}
\label{tab:ov_over_results_by_model_parameters_conv_transformer}
\end{table}
\end{landscape}
\begin{landscape}
\begin{table}
\small
\begin{tabularx}{\linewidth}{l*{6}{X}}
\toprule
\multirow{2}{*}{Model} & \multicolumn{2}{c}{MAE} & \multicolumn{3}{c}{MSE} \\
\cmidrule(r){2-4} \cmidrule(r){5-7}
& Fertility Overall & Fertile Days & Non-Fertile Days & Fertility Overall & Fertile Days & Non-Fertile Days \\
\midrule
\multicolumn{7}{c}{\textbf{Regular Cycle Group}} \\
\midrule
LSTM & 0.026034 & 0.072578 & \textbf{0.007398} & 0.002563 & 0.007850 & \textbf{0.000415} \\
Transformer & 0.025847 & \textbf{0.065979} & 0.009693 & \textbf{0.002376} & \textbf{0.006651} & 0.000626 \\
Convolutional LSTM & 0.026750 & 0.070371 & 0.009774 & 0.002460 & 0.007313 & 0.000573 \\
Convolutional Transformer & \textbf{0.025161} & 0.066934 & 0.008789 & 0.002424 & 0.006919 & 0.000655 \\
Last-Cycle Baseline & 0.037961 & 0.085792 & 0.016133 & 0.006592 & 0.014177 & 0.003198 \\
Population Mean Baseline & 0.080743 & 0.139663 & 0.052513 & 0.016657 & 0.027671 & 0.011292 \\
User Mean Baseline & 0.032624 & 0.077341 & 0.012325 & 0.005195 & 0.011715 & 0.002312 \\
\midrule
\multicolumn{7}{c}{\textbf{Irregular Cycle Group}} \\
\midrule
LSTM & \textbf{0.036813} & 0.093108 & \textbf{0.014657} & 0.004490 & 0.013415 & \textbf{0.001076} \\
Transformer & 0.039474 & \textbf{0.084206} & 0.022359 & 0.004122 & \textbf{0.010774} & 0.001783 \\
Convolutional LSTM & 0.037182 & 0.085657 & 0.018791 & \textbf{0.004096} & 0.011188 & 0.001538 \\
Convolutional Transformer & 0.038007 & 0.086143 & 0.019175 & 0.004281 & 0.011374 & 0.001639 \\
Last-Cycle Baseline & 0.049164 & 0.124943 & 0.022958 & 0.009423 & 0.023650 & 0.004613 \\
Population Mean Baseline & 0.051673 & 0.123722 & 0.027299 & 0.009795 & 0.022330 & 0.005751 \\
User Mean Baseline & 0.054256 & 0.129195 & 0.028745 & 0.010548 & 0.023935 & 0.006159 \\
\bottomrule
\end{tabularx}
\caption{Evaluation Metrics for the fertility probability target across Different Model Architectures for the Regular and Irregular Cycle Groups.
\textbf{Bold} values represent the best values across all models for a given metric.}
\label{tab:regular_vs_irregular_fertility_results}
\end{table}
\end{landscape}
\begin{landscape}
\begin{table}
\small
\begin{tabularx}{\linewidth}{l*{6}{X}}
\toprule
\multirow{2}{*}{Model} & \multicolumn{3}{c}{MAE} & \multicolumn{3}{c}{MSE} \\
\cmidrule(r){2-4} \cmidrule(r){5-7}
& OV-Over Overall & OV-Over Before OV & OV-Over After OV & OV-Over Overall & OV-Over Before OV & OV-Over After OV \\
\midrule
\multicolumn{7}{c}{\textbf{Regular Cycle Group}} \\
\midrule
LSTM & \textbf{0.059506} & 0.087519 & \textbf{0.047856}&\textbf{0.028519}& 0.031695 & \textbf{0.026116} \\
Transformer & 0.075639 & 0.083443 & 0.069310 & 0.032929 & \textbf{0.024774} & 0.033411 \\
Convolutional LSTM & 0.076335 & 0.090550 & 0.068022 & 0.034224 & 0.028962 & 0.033929 \\
Convolutional Transformer & 0.075421 & 0.111256 & 0.060485 & 0.033333 & 0.039045 & 0.029329 \\
Last-Cycle Baseline & 0.087916 & 0.093465 & 0.074191 & 0.087916 & 0.093465 & 0.074191 \\
Population Mean Baseline & 0.285887 & \textbf{0.008876} & 0.418879 & 0.285887 & 0.008876 & 0.418879 \\
User Mean Baseline & 0.069245 & 0.069152 & 0.055724 & 0.069245 & 0.069152 & 0.055724 \\
\midrule
\multicolumn{7}{c}{\textbf{Irregular Cycle Group}} \\
\midrule
LSTM & \textbf{0.094527} & \textbf{0.063073} & \textbf{0.106232} & \textbf{0.051747} & \textbf{0.016777} & 0.070419 \\
Transformer & 0.117531 & 0.096482 & 0.116790 & 0.055990 & 0.026159 & 0.067520 \\
Convolutional LSTM & 0.108166 & 0.092732 & 0.110939 & 0.054523 & 0.030092 & \textbf{0.066643} \\
Convolutional Transformer & 0.111817 & 0.094195 & 0.114122 & 0.056075 & 0.029638 & 0.067965 \\
Last-Cycle Baseline & 0.222906 & 0.134945 & 0.258047 & 0.222906 & 0.134945 & 0.258047 \\
Population Mean Baseline & 0.171368 & 0.130366 & 0.140363 & 0.171368 & 0.130366 & 0.140363 \\
User Mean Baseline & 0.180702 & 0.088884 & 0.217144 & 0.180702 & 0.088884 & 0.217144 \\
\bottomrule
\end{tabularx}
\caption{Evaluation Metrics for the OV-over Target across Different Model Architectures for the Regular and Irregular Cycle Groups.
\textbf{Bold} values represent the best values across all models for a given metric.}
\label{tab:regular_vs_irregular_ov_over_results}
\end{table}
\end{landscape}
Reference in New Issue View Git Blame Copy Permalink