further work on methodology

thesis/main.tex

@@ -6,12 +6,14 @@
 
 % Packages
 \usepackage{amsmath}
+\usepackage{array}
 \usepackage[a4paper, margin=1in]{geometry}
 
 \usepackage{graphicx}
 \graphicspath{{resources/figures/}}
 
 \usepackage{blindtext}
+%\usepackage[style=ieee, backend=biber]{biblatex}
 \usepackage[style=ieee, backend=biber]{biblatex}
 \addbibresource{../main.bib}
 \usepackage{booktabs}

thesis/sections/background.tex

@@ -45,15 +45,14 @@ This process, known as menstruation, marks the beginning of a new cycle.
 
 The menstrual cycle typically lasts around 28 days, with ovulation occurring near the midpoint.
 However, variations, particularly in the follicular phase length, are common and can be influenced by factors such as stress, diet, exercise and age~\cite{silberstein_physiology_2000}.
-Figure~\ref{fig:background_menstrual_cycle_physiology} provides a detailed overview of the hormonal and physiological changes
-throughout the menstrual cycle.
+Figure~\ref{fig:background_menstrual_cycle_physiology} provides a detailed overview of the hormonal and physiological changes throughout the menstrual cycle.
 
-%\begin{figure}[htb]
-%    \centering
-%    \includegraphics[width=0.9\textwidth]{background_labeled_cycle}
-%    \caption{A cycles temperature curve with its phases and ovulation}
-%    \label{fig:background_labeled_cycle}
-%\end{figure}
+\begin{figure}[htb]
+    \centering
+    \includegraphics[width=0.9\textwidth]{background_labeled_cycle}
+    \caption{A cycles temperature curve with its phases and ovulation}
+    \label{fig:background_labeled_cycle}
+\end{figure}
 
 Figure~\ref{fig:background_labeled_cycle} shows the temperature curve over the course of a menstrual cycle with the
 menstruation, fertile phase and ovulation marked.

thesis/sections/methodology.tex

@@ -79,7 +79,7 @@ use to learn patterns based not only on the temperature data, but also on this c
 
 \begin{table}[htbp]
     \centering
-    \begin{tabular}{@{}lp{0.6\linewidth}@{}}
+    \begin{tabular}{l>{\raggedright\arraybackslash}p{0.65\linewidth}}
         \toprule
         \textbf{Feature}                & \textbf{Description}                                        \\
         \midrule
@@ -125,12 +125,12 @@ These help the model place each observation in temporal context:
 
 Except for \textit{time since cycle start}, all features are encoded using sine and cosine transforms to preserve their cyclical nature and make them more interpretable for the model.
 
-%\begin{figure}[htbp]
-%    \centering
-%    \includegraphics[width=0.9\textwidth]{methodology_time_feature_sine_encoded}
-%    \caption{Sine and cosine encoding of the day-of-week feature.}
-%    \label{fig:methodology_time_feature_encoding}
-%\end{figure}
+\begin{figure}[htbp]
+    \centering
+    \includegraphics[width=0.9\textwidth]{methodology_time_feature_sine_encoded}
+    \caption{Sine and cosine encoding of the day-of-week feature.}
+    \label{fig:methodology_time_feature_encoding}
+\end{figure}
 
 Figure~\ref{fig:methodology_time_feature_encoding} illustrates the sine and cosine encoding of the day-of-week feature.
 The cyclical nature of the variable is clearly visible in the transformation.
@@ -147,6 +147,13 @@ Including this information could therefore improve the predictive quality of the
 Observable features include all time-series inputs available up to the current time step.
 They represent real-time physiological signals from which the model must infer ovulatory status.
 
+\begin{figure}[htbp]
+    \centering
+    \includegraphics[width=0.9\textwidth]{methodology_observable_features}
+    \caption{Observable features for a cycle.}
+    \label{fig:methodology_observable_features}
+\end{figure}
+
 \begin{itemize}
     \item \textbf{Temperature} — Raw intravaginal temperature as recorded by the OvulaRing sensor, sampled every 5 minutes.
     \item \textbf{Rolling Average Temperature} — The mean temperature over a 1-day (288-sample) sliding window, linearly interpolated to preserve the original input resolution.
@@ -158,9 +165,12 @@ These derived features are intended to reduce model complexity by providing smoo
 The \textit{rolling average} allows the model to capture broader trends without having to learn temporal aggregation from scratch.
 The \textit{rolling minimum} and \textit{maximum} support the detection of boundary behavior (e.g., temperature shifts, sustained elevation, extreme values)
 without requiring explicit memory or aggregation.
-
+Special care was taken, so that the sliding window can only look backwards, so that no data leakage can happen.
 The 1-day window length reflects the expected circadian cycle and strikes a balance between temporal sensitivity and signal stability.
 
+Figure~\ref{fig:methodology_observable_features} illustrates the behavior of all observable features within a single cycle.
+The rolling extrema delineate the amplitude of the daily temperature variation and accentuate phase transitions.
+
 \subsubsection{Target Features}
 
 Target features represent the outputs that the models are trained to predict.
@@ -393,10 +403,95 @@ might offer valuable insight into the interpretability and robustness of learned
 
 \subsection{Model Training}\label{subsec:model_training}
 
+The models described in the previous sections were trained to predict fertility probability and ovulation status based on daily temperature and contextual features.
+This section outlines the training process, including the overall setup, hyperparameter optimization strategies, and implementation details.
+Each model type was trained using the same preprocessing pipeline and evaluation protocol to ensure comparability across architectures.
+
 \subsubsection{Training Setup}\label{subsubsec:training_setup}
 
+The models were trained using a configurable framework developed specifically for this study,
+allowing for flexible experimentation with different architectures, input feature sets, and
+hyperparameter configurations.
+
+The training process is organized into distinct \textit{runs}, each representing a set of model experiments with a shared base configuration.
+Within a run, variable parameters—such as input sequence length, hidden layer size, dropout rate,
+or specific feature subsets—are systematically swept across predefined value ranges.
+
+For each combination of parameters, a dedicated training and evaluation procedure is performed.
+This structure supports efficient hyperparameter exploration and ensures consistent, reproducible
+training conditions across models.
+
+The framework is designed to allow plug-and-play experimentation with model types (e.g., LSTM, Transformer),
+and it supports automated logging, early stopping, and checkpointing.
+
+Training was conducted on a GPU cluster equipped with NVIDIA A30 GPUs using the PyTorch framework~\cite{noauthor_pytorch_nodate}.
+All experiments were implemented in Python and run with mixed precision for improved memory efficiency.
+
+A detailed technical report on the training framework is planned for future work.
+
 \subsubsection{Hyperparameter Tuning}\label{subsubsec:hyperparameter_tuning}
-% what name to use? -> fix experimental setup reference from earlier
+
+To identify performant configurations without exhaustively searching the entire hyperparameter space,
+a structured subset of sensible parameter values was defined based on prior experience and preliminary tests.
+A multi-fidelity strategy with early stopping was used to reduce training time during large-scale sweeps.
+
+Hyperparameter tuning was divided into two stages:
+(1) tuning of input-related parameters such as resampling rate, and
+(2) tuning of model-specific architectural parameters such as hidden size or attention heads.
+
+\paragraph{Input Parameter Tuning}% \mbox{}\\
+This stage involved identifying optimal settings for data preprocessing and input representation.
+Key variables included the resampling rate (temporal resolution) and historical context length (input window size).
+These parameters strongly influence the structure of the input signal and can significantly affect model performance.
+
+The goal was to determine whether there is a sweet spot between too little and too much temporal context,
+and whether incorporating data from previous cycles improves learning or introduces noise.
+
+\begin{table}[htbp]
+    \centering
+    \begin{tabular}{l>{\raggedright\arraybackslash}p{0.45\linewidth}>{\raggedright\arraybackslash}p{0.3\linewidth}}
+        \toprule
+        \textbf{Parameter} & \textbf{Description}                           & \textbf{Values Tested}       \\
+        \midrule
+        Resampling rate    & Number of temperature measurements per day     & 1, 2, 4, 12, 24, 48, 72, 288 \\
+        Window length      & Historical context in days (input window size) & 10, 20, 40, 80, 160          \\
+        \bottomrule
+    \end{tabular}
+    \caption{Input-related hyperparameters for LSTM and Transformer models.}
+    \label{tab:input_hyperparameters_basic}
+\end{table}
+
+\begin{table}[ht]
+    \centering
+    \begin{tabular}{l>{\raggedright\arraybackslash}p{0.45\linewidth}>{\raggedright\arraybackslash}p{0.3\linewidth}}
+        \toprule
+        \textbf{Parameter} & \textbf{Description} & \textbf{Values Tested} \\
+        \midrule
+        Window length & Historical context in days (input window size) & 10, 20, 40, 80, 160 \\
+        \bottomrule
+    \end{tabular}
+    \caption{Input-related hyperparameters for convolutional LSTM and Transformer hybrids.}
+    \label{tab:input_hyperparameters_conv}
+\end{table}
+
+
+Table~\ref{tab:input_hyperparameters_basic} shows the value ranges used for the LSTM and Transformer models,
+while Table~\ref{tab:input_hyperparameters_conv} lists those for the convolutional hybrid variants.
+Note that convolutional models do not require an explicit resampling parameter, as they perform learned downsampling internally
+(see Section~\ref{subsubsec:temporal_convolution_layer}).
+
+\paragraph{Model Parameter Tuning}% \mbox{}\\
+Once suitable input configurations were established, model-specific hyperparameters were tuned.
+For LSTM-based models, this included hidden layer size and number of recurrent layers.
+For Transformer models, relevant parameters included the number of attention heads and encoder depth.
+
+\vspace{0.5em}
+We acknowledge that interactions between input and model parameters may influence final model performance,
+and our two-stage tuning procedure may miss globally optimal combinations.
+
+Future work may incorporate more advanced hyperparameter optimization techniques, such as
+Bayesian Optimization, Genetic Algorithms, or Neural Architecture Search (NAS), to better explore the
+joint parameter space in a more efficient and principled manner.
 
 \subsubsection{Training Details}\label{subsubsec:training_details}