alex/temperature-based-fertility-prediction-thesis
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

further work and refinements

Browse Source
This commit is contained in:
Alex
2025-08-05 17:42:49 +02:00
parent 2c5108d174
commit 4e6e43c5c1
17 changed files with 2327 additions and 2061 deletions
Download Patch File Download Diff File Expand all files Collapse all files
thesis/sections/methodology.tex
+47 -21
View File
@@ -438,7 +438,8 @@ training conditions across models.
The framework is designed to allow plug-and-play experimentation with model types (e.g., LSTM, Transformer),
and it supports automated logging, early stopping, and checkpointing.
Training was conducted on a GPU cluster equipped with NVIDIA A30 GPUs using the PyTorch framework~\cite{noauthor_pytorch_nodate}.
The trainings were performed on the scientific compute cluster of Leipzig University on NVIDIA A30 GPUs,
using the PyTorch framework~\cite{noauthor_pytorch_nodate}.
All experiments were implemented in Python and run with mixed precision for improved memory efficiency.
A detailed technical report on the training framework is planned for future work.
@@ -543,21 +544,25 @@ Note that the same settings were used for the hybrid models, as their architectu
We acknowledge that interactions between input and model parameters may influence final model performance,
and our two-stage tuning procedure may miss globally optimal combinations.
Future work may incorporate more advanced hyperparameter optimization techniques, such as
Bayesian Optimization, Genetic Algorithms, or Neural Architecture Search (NAS), to better explore the
joint parameter space in a more efficient and principled manner.
\subsubsection{Training Details}\label{subsubsec:training_details}
The trainings were performed on the scientific compute cluster of Leipzig University on NVIDIA A30 GPUs.
Due to the heterogeneity of both the models and the trainings, a dynamic batch size algorithm was used, that estimated
the batch size dynamically during training to optimize the resource usage.
The learning rate was scaled linearly with the batch size to allow for equivalent convergence behaviour~\cite{goyal_accurate_2018}
The batch size was capped at 2048 to avoid OOM errors during data preparation.
Due to the heterogeneity of both the models and training configurations,
a dynamic batch size algorithm was used to optimize resource usage by estimating the batch size during training.
The learning rate was scaled linearly with the batch size to maintain equivalent convergence behavior~\cite{goyal_accurate_2018}.
The maximum batch size was capped at 2048 to avoid out-of-memory errors during data preparation.
All training runs employed a \emph{One Cycle Scheduler}, which has proven effective for recurrent, convolutional,
and transformer-based models.
This schedule begins with a warm-up phase, followed by an annealing phase, allowing the model to converge more smoothly~\cite{smith_disciplined_2018}.
Optimization was performed using \emph{AdamW}, a decoupled weight decay variant of the \emph{Adam} optimizer,
designed to correct its flawed implementation of L2 regularization~\cite{loshchilov_decoupled_2019}.
Each model was trained for up to 30 epochs, with early stopping based on validation performance.
\subsection{Evaluation}\label{subsec:evaluation}
To meaningfully compare model performance, we define a set of metrics that capture both overall accuracy and behavior at key points in the prediction sequence.
To meaningfully compare model performance, we define a set of metrics according to the research objectives,
that capture both overall accuracy and behavior at key points in the prediction sequence as well as cover the use cases introduced in~\ref{subsubsec:practical_use_cases}.
This includes metrics for different temporal segments, enabling a more detailed understanding of model strengths and limitations.
\subsubsection{Baseline Comparisons}\label{subsubsec:baseline_comparisons}
@@ -585,6 +590,8 @@ and is defined as:
\text{MAE} = \frac{1}{n} \sum_{i=1}^{n} \left| y_i - \hat{y}_i \right|
\end{align}
where \(y_i\) is the observed value and \(\hat{y}_i\) the predicted value.
MAE was selected for its intuitive interpretability.
In particular, the fertility probability target lends itself well to an absolute error interpretation,
making MAE a natural choice for evaluating prediction accuracy.
@@ -593,15 +600,28 @@ To complement this, we also report the mean squared error (MSE):
\begin{align}
\text{MSE} = \frac{1}{n} \sum_{i=1}^{n} (y_i - \hat{y}_i)^2
\end{align}
where \(y_i\) is the observed value and \(\hat{y}_i\) the predicted value.
MSE penalizes larger errors more heavily than MAE, making it useful for highlighting substantial deviations.
This is particularly relevant for model comparison, where disproportionate errors can skew performance.
Moreover, since the fertility probability target was trained using an MSE-based loss function,
this metric directly reflects the optimization objective.
Additionally, we add the coefficient of determination (\(R^2\)) regression score:
\begin{align}
R^2 = 1 - \frac{\sum_{i=1}^n (y_i - \hat{y}_i)^2}{\sum_{i=1}^n (y_i - \bar{y})^2}
\end{align}
where \(y_i\) is the observed value, \(\hat{y}_i\) the predicted value,
\(\bar{y}\) is the mean of observed values and \(n\) is the number of observations.
This coefficient indicates the proportion of total variance in the target that is explained by the model.
Since \( R^2 \) is specific to regression tasks, it is only applied to the fertility probability target.
All models operate on the same inputs and targets, so adjusted \( R^2 \) is not required.
To enable a more nuanced comparison of model performance,
we complement the overall error metrics with targeted evaluations at biologically relevant subregions of the prediction sequence.
This decomposition allows us to identify localized strengths and weaknesses in different models—
This decomposition allows us to identify localized strengths and weaknesses in different models,
for example, a model may accurately capture the onset of fertility but underperform closer to ovulation.
It also facilitates a clearer comparison of false-positive tendencies,
@@ -623,19 +643,25 @@ Tables~\ref{tab:fertility_mae_metrics} and~\ref{tab:ov_over_mae_metrics} summari
\renewcommand{\arraystretch}{1.3} % spacing between rows
\begin{tabular}{@{}p{0.35\linewidth}p{0.60\linewidth}@{}}
\toprule
\textbf{Metric Name} & \textbf{Description} \\
\textbf{Metric Name} & \textbf{Description} \\
\midrule
\multicolumn{2}{@{}l}{\textbf{Mean Absolute Error}} \\
\midrule
Fertility Overall & MAE over the entire sequence. \\
During-Fertility & MAE during the fertile phase. \\
Non-Fertility & MAE on the non-fertile days. \\
Fertility Overall & MAE over the entire sequence. \\
During-Fertility & MAE during the fertile phase. \\
Non-Fertility & MAE on the non-fertile days. \\
\midrule
\multicolumn{2}{@{}l}{\textbf{Mean Squared Error}} \\
\midrule
Fertility Overall & MSE over the entire sequence. \\
During-Fertility & MSE during the fertile phase. \\
Non-Fertility & MSE on the non-fertile days. \\
Fertility Overall & MSE over the entire sequence. \\
During-Fertility & MSE during the fertile phase. \\
Non-Fertility & MSE on the non-fertile days. \\
\midrule
\multicolumn{2}{@{}l}{\textbf{Coefficient of Determination (\(R^2\))}} \\
\midrule
Fertility Overall & \(R^2\) over the entire sequence. \\
During-Fertility & \(R^2\) during the fertile phase. \\
Non-Fertility & \(R^2\) on the non-fertile days. \\
\bottomrule
\end{tabular}
\caption{Evaluation metrics of the fertility probability target based on mean absolute error (MAE) at various intervals across the predicted fertility window.}
@@ -674,7 +700,7 @@ the next ovulation day is likely to be identical or very close to the previous o
To evaluate how different algorithms handle varying degrees of cycle variability,
we compare model performance on two distinct user groups: one with highly regular cycles and another with highly irregular cycles.
These groups are defined by the standard deviation of ovulation day across cycles.
These groups are defined by the standard deviation of the ovulation day across cycles.
Specifically, we select the 100 users with the lowest standard deviation (regular group)
and the 100 users with the highest standard deviation (irregular group).
This comparison allows us to assess not only overall performance,
@@ -759,7 +785,7 @@ For each day in a full year we count occurrences of the following states:
\end{itemize}
This classification measures both fertility prediction accuracy and the impact of incorrect deferrals.
A model overly predicting fertility would increase pregnancy rates but negatively affect usability due to misdirected efforts.
A model overly predicting fertility would increase potential pregnancy rates but negatively affect usability and trust due to misdirected efforts.
\subsection{Ethical Considerations}\label{subsec:ethical_considerations}