alex/temperature-based-fertility-prediction-thesis
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

further work on discussion

Browse Source
This commit is contained in:
Alex Blank
2025-09-04 17:30:43 +00:00
parent a1b2662b8a
commit a2fdf2f0a3
25 changed files with 2707 additions and 2621 deletions
Download Patch File Download Diff File Expand all files Collapse all files
thesis/sections/methodology.tex
+89 -49
View File
@@ -10,6 +10,8 @@ To address these limitations, we develop a data-driven framework based on a larg
heterogeneous dataset of real-world menstrual cycles.
Our approach emphasizes model transparency, adaptability to irregular patterns, and the predictive utility of
high-resolution core body temperature measurements.
The overall goal is to train a model to predict the fertility-probability and information about the ovulation for
a given day, only relying on past information prior to that day.
This section outlines the methodology used, including preprocessing, labeling, feature extraction, and model architectures.
\subsection{Data Preprocessing}\label{subsec:data_preprocessing}
@@ -64,15 +66,15 @@ The next section details how these labels are incorporated into feature represen
The features used as model inputs have been divided into three categories:
\begin{itemize}
\item \textbf{Static features} - Characteristics, that remain constant across a user's cycle, such as age, height, or average ovulation day
\item \textbf{Known features} — Inputs known a priori at each time step, such as time of day or calendar-based variables.
\item \textbf{Observable features} — Inputs available at the current time step, including raw and derived temperature values.
\item \textbf{Known features} — Inputs known a priori at each time step, such as time of day or calendar-based variables (e.g., month of the year).
\item \textbf{Observable features} — Inputs available at the current time step, including raw and derived temperature values (e.g, rolling averages).
\end{itemize}
The target variables predicted by the model—like ovulation status or fertility probability—are described separately.
Each feature type can handle categorical and continuous features.
This allows for mixed inputs, such as scalar measurements and class labels, within the same category.
The categorization into four feature types is intended to clarify the conceptual roles of different input types.
The categorization into three feature types is intended to clarify the conceptual roles of different input types.
While the current models concatenate all features into a single input stream, the distinction allows for flexibility—future models
may process each feature group differently depending on their architectural design.
@@ -106,11 +108,12 @@ Static features provide user-specific context that helps the model learn individ
%\end{table}
%Table~\ref{tab:static_features} shows all static features and their descriptions.
Prior research by \citeauthor{li_menstrual_2023} has shown that menstrual cycle characteristics vary significantly with age and BMI~\cite{li_menstrual_2023}.
Menstrual cycle characteristics vary significantly with age and BMI~\cite{li_menstrual_2023}.
Including such information is therefore expected to improve predictive performance.
In addition, summary statistics from previous cycles—such as ovulation timing, temperature levels, or the fraction of ovulatory cycles—provide useful individual context.
In addition, summary statistics from previous cycles, such as ovulation timing, temperature levels, or the fraction of ovulatory cycles, provide useful individual context.
These features help the model learn subject-specific variability and better estimate the likelihood and timing of ovulation in the current cycle.
If no user-specific context is available yet, population-based mean values are used.
Table~\ref{tab:feature_overview} shows the full list of static input features.
All historical features are computed using only data available prior to the current cycle, ensuring no data leakage and supporting robust, user-adaptive learning.
@@ -141,7 +144,7 @@ The cyclical nature of the variable is clearly visible in the transformation.
Although the model architectures used are sequential, the explicit inclusion of these time features allows the models to interpret each time step in a broader context.
More importantly, they enable the detection of gaps in the recording, which would otherwise not be visible from the data alone.
Additionally, prior research has shown that the menstrual cycle may be influenced by weekly rhythms~\cite{ecochard_menstrual_2024}.
Menstrual cycles can be influenced by weekly rhythms~\cite{ecochard_menstrual_2024}.
For example, menstruation has been found to begin more frequently on Thursdays or Fridays, suggesting that behavioral or social factors may modulate certain events in the cycle.
Including this information could therefore improve the predictive quality of the models.
@@ -242,20 +245,44 @@ The model outputs represent data-driven estimates and do not constitute medical
\label{tab:feature_overview}
\end{table}
\subsubsection{Time-Series Input Representation}
\label{subsubsec:time_series_input_representation}
All features were normalized based on their empirical distributions.
A \textit{standard scaler} was applied to approximately normal features without outliers,
a \textit{robust scaler} was used for distributions with outliers, and a \textit{MinMax scaler} was used for all others.
Different Scalers were used for the train, validation and test sets to avoid data leakage.
For the final data matrix, all features were stacked per timestep.
The static features were repeated for each timestep.
For the final data matrix, all features are stacked per timestep.
The static features are repeated for each timestep.
We are aware of possible inefficiencies here.
A side channel for static features might improve predictive efficiency and potential quality, but
this was left out to keep the interfaces the same for compatibility purposes between all tested models
\subsubsection{Time-Series Input Representation}
\label{subsubsec:time_series_input_representation}
Let
\begin{itemize}
\item \( T \) be the sequence length (number of time steps)
\item \( x_t^{\text{obs}} \in \mathbb{R}^{d_{\text{obs}}} \): observable features at time \( t \)
\item \( x_t^{\text{known}} \in \mathbb{R}^{d_{\text{known}}} \): known features at time \( t \)
\item \( x^{\text{static}} \in \mathbb{R}^{d_{\text{static}}} \): static features (repeated across time steps)
\end{itemize}
Then each input token at time \( t \in \{1, \dots, T\} \) is:
\[
x_t = \left[ x_t^{\text{obs}} \;\middle|\; x_t^{\text{known}} \;\middle|\; x^{\text{static}} \right]
\in \mathbb{R}^{d_{\text{obs}} + d_{\text{known}} + d_{\text{static}}}
\]
The full input sequence is then represented as a matrix:
\[
X = \begin{bmatrix}
x_1 \\
x_2 \\
\vdots \\
x_T
\end{bmatrix}
\in \mathbb{R}^{T \times (d_{\text{obs}} + d_{\text{known}} + d_{\text{static}})}
\]
The high temporal resolution of the temperature data, 288 measurements per day, results in very long input sequences
that are impractical for most deep learning models to process directly.
@@ -269,9 +296,6 @@ where values from distinct categories (e.g., hours 23 and 0) might otherwise be
The effect of different sampling resolutions and aggregation strategies is evaluated in Section~\ref{sec:results}.
To simulate real-time prediction rather than retrospective analysis, a sliding-window approach is employed.
This allows the model to make predictions based only on data available up to a specific point in the cycle.
To simulate real-time prediction, each cycle is split into overlapping,
fixed-length input windows that capture all available data up to a given time step.
As the cycle progresses, these windows slide forward, allowing the model to update its prediction based on growing historical context.
@@ -318,6 +342,9 @@ Ultimately, the models used in this study were selected based on their ability t
\item Generalize across users while incorporating personalized cycle context
\end{itemize}
In this section we will introduce the base architectures \textbf{Long-Short-Term-Memory Models} and \textbf{Transformer Models}.
Additionally, we'll show, how the convolutional hybrids extend their functionality.
\subsubsection{LSTM-Architecture}\label{subsubsec:lstm_architecture}
\begin{figure}[htbp]
\centering
@@ -373,7 +400,7 @@ Since the task does not require sequence-to-sequence modeling, only the encoder
Its output—one vector per input token—is aggregated via 1D adaptive average pooling, resulting in a single vector representation per sequence.
This vector is then passed through a linear projection layer to produce the two target outputs:
fertility probability and ovulation-over indicator.
Note, that in contrast to the original use case of machine-translation, not special tokens are necessary here, as we do not perform sequence-to-sequence prediction.
Note, that in contrast to the original use case of machine-translation, no special tokens are necessary here, as we do not perform sequence-to-sequence prediction.
Figure~\ref{fig:methodology_transformer_architecture} shows the overall architecture.
The stacked inputs and outputs indicate batch processing.
@@ -434,11 +461,20 @@ The models were trained using a configurable framework developed specifically fo
allowing for flexible experimentation with different architectures, input feature sets, and
hyperparameter configurations.
The training process is organized into distinct \textit{runs}, each representing a set of model experiments with a shared base configuration.
Within a run, variable parameters—such as input sequence length, hidden layer size, dropout rate,
or specific feature subsets—are systematically swept across predefined value ranges.
The training process is organized into distinct \textit{runs}, each representing a set of model experiments
(e.g., a sweep over possible input sequence lengths), with a shared base configuration (e.g., fixed input resolution for the input sequence length sweep).
Within a run, variable parameters, such as input sequence length, hidden layer size, dropout rate,
or specific feature subsets, are swept across predefined value ranges.
For each combination of parameters, a dedicated training and evaluation procedure is performed.
We define 3 runs for each base architecture and 2 for each convolutional architecture.
\begin{itemize}
\item \textbf{Input-Sequence-Length Run:} explores different input sequence lengths
\item \textbf{Input Resolution Run:} explores different input resolutions (only for base architectures)
\item \textbf{Model Parameter Run:} explores different model hyperparameters, e.g., hidden layer size
\end{itemize}
For each combination of parameters (e.g., 20 day input sequence length, resolution of 12 values per day),
a dedicated training and evaluation procedure is performed.
This structure supports efficient hyperparameter exploration and ensures consistent, reproducible
training conditions across models.
@@ -576,7 +612,8 @@ Each model was trained for up to 30 epochs, with early stopping based on validat
To meaningfully compare model performance, we define a set of metrics according to the research objectives,
that capture both overall accuracy and behavior at key points in the prediction sequence as well as cover the use cases introduced in~\ref{subsubsec:practical_use_cases}.
This includes metrics for different temporal segments, enabling a more detailed understanding of model strengths and limitations.
This includes metrics for different temporal segments, such as before and after the ovulation,
enabling a more detailed understanding of model strengths and limitations.
\subsubsection{Baseline Comparisons}\label{subsubsec:baseline_comparisons}
@@ -595,30 +632,32 @@ In many cases, it remains unclear whether proposed models genuinely outperform s
\subsubsection{Evaluation Metrics}\label{subsubsec:evaluation_metrics}
The base metric used for all categories is the mean absolute error (MAE),
which describes the average absolute deviation of the prediction from the target value,
The base metric used for all categories is the mean squared error (MAE),
which describes the average squared deviation of the prediction from the target value,
and is defined as:
\begin{align}
\text{MSE} = \frac{1}{n} \sum_{i=1}^{n} (y_i - \hat{y}_i)^2
\end{align}
where \(y_i\) is the observed value and \(\hat{y}_i\) the predicted value.
MSE penalizes larger errors more heavily, making it useful for highlighting substantial deviations.
This is particularly relevant for model comparison, where disproportionate errors can skew performance.
Moreover, since the fertility probability target was trained using an MSE-based loss function,
this metric directly reflects the optimization objective.
To complement this, we also report the mean squared error (MAE):
\begin{align}
\text{MAE} = \frac{1}{n} \sum_{i=1}^{n} \left| y_i - \hat{y}_i \right|
\end{align}
where \(y_i\) is the observed value and \(\hat{y}_i\) the predicted value.
MAE was selected for its intuitive interpretability.
In particular, the fertility probability target lends itself well to an absolute error interpretation,
making MAE a natural choice for evaluating prediction accuracy.
To complement this, we also report the mean squared error (MSE):
\begin{align}
\text{MSE} = \frac{1}{n} \sum_{i=1}^{n} (y_i - \hat{y}_i)^2
\end{align}
where \(y_i\) is the observed value and \(\hat{y}_i\) the predicted value.
MSE penalizes larger errors more heavily than MAE, making it useful for highlighting substantial deviations.
This is particularly relevant for model comparison, where disproportionate errors can skew performance.
Moreover, since the fertility probability target was trained using an MSE-based loss function,
this metric directly reflects the optimization objective.
However, we will only be using the MAE as a secondary metric, as we will base our further interpretation of
the model performances on the use-case evaluations, that provide inherent real-world interpretability.
We considered including the coefficient of determination (\(R^2\)) as an evaluation metric.
However, we found that the target windows frequently exhibited very low variance,
@@ -652,20 +691,20 @@ Tables~\ref{tab:fertility_mae_metrics} and~\ref{tab:ov_over_mae_metrics} summari
\toprule
\textbf{Metric Name} & \textbf{Description} \\
\midrule
\multicolumn{2}{@{}l}{\textbf{Mean Absolute Error}} \\
\midrule
Fertility Overall & MAE over the entire sequence. \\
During-Fertility & MAE during the fertile phase. \\
Non-Fertility & MAE on the non-fertile days. \\
\midrule
\multicolumn{2}{@{}l}{\textbf{Mean Squared Error}} \\
\midrule
Fertility Overall & MSE over the entire sequence. \\
During-Fertility & MSE during the fertile phase. \\
Non-Fertility & MSE on the non-fertile days. \\
\midrule
\multicolumn{2}{@{}l}{\textbf{Mean Absolute Error}} \\
\midrule
Fertility Overall & MAE over the entire sequence. \\
During-Fertility & MAE during the fertile phase. \\
Non-Fertility & MAE on the non-fertile days. \\
\bottomrule
\end{tabular}
\caption{Evaluation metrics of the fertility probability target based on mean absolute error (MAE) and mean squared error (MSE) at various intervals across the predicted fertility window.}
\caption{Evaluation metrics of the fertility probability target based on mean squared error (MSE) and mean absolute error (MAE) at various intervals across the predicted fertility window.}
\label{tab:fertility_mae_metrics}
\end{table}
@@ -677,20 +716,20 @@ Tables~\ref{tab:fertility_mae_metrics} and~\ref{tab:ov_over_mae_metrics} summari
\toprule
\textbf{Metric Name} & \textbf{Description} \\
\midrule
\multicolumn{2}{@{}l}{\textbf{Mean Absolute Error}} \\
\midrule
OV-Over Overall & MAE over the entire sequence. \\
Pre-OV & MAE before the ovulation. \\
Post-OV & MAE after the ovulation. \\
\midrule
\multicolumn{2}{@{}l}{\textbf{Mean Squared Error}} \\
\midrule
OV-Over Overall & MSE over the entire sequence. \\
Pre-OV & MSE before the ovulation. \\
Post-OV & MSE after the ovulation. \\
\midrule
\multicolumn{2}{@{}l}{\textbf{Mean Absolute Error}} \\
\midrule
OV-Over Overall & MAE over the entire sequence. \\
Pre-OV & MAE before the ovulation. \\
Post-OV & MAE after the ovulation. \\
\bottomrule
\end{tabular}
\caption{Evaluation metrics of the ovulation-over target based on mean absolute error (MAE) and mean squared error (MSE) at various intervals across the predicted fertility window.}
\caption{Evaluation metrics of the ovulation-over target based on mean squared error (MSE) and mean absolute error (MAE) at various intervals across the predicted fertility window.}
\label{tab:ov_over_mae_metrics}
\end{table}
@@ -701,8 +740,9 @@ This step was necessary to keep the computational effort manageable,
as exhaustively testing all possible configurations for every subsequent metric would have been prohibitively expensive.
Selection was based primarily on the \textbf{Fertility-Overall MSE} metric,
as it most directly reflects the main objective of this study: predicting fertility.
In cases where the difference between configurations was small,
as it most directly reflects the main objective of this study: predicting fertility,
and we want to penalize larger errors more, as they are much more problematic for our use-case scenarios.
In cases where the difference between configurations was small (less than \(\pm\) 5\% metric value),
we preferred the option that aligned with the general tendency of the model architecture.
For example, if an architecture consistently performed better with more input data or longer sequences,
but the Fertility-Overall MSE was only marginally better for a shorter window, we selected the longer window.