further work on methodology

thesis/main.tex

@@ -11,8 +11,8 @@
 \usepackage{graphicx}
 \graphicspath{{resources/figures/}}
 
-\usepackage[style=ieee, backend=biber]{biblatex}
 \usepackage{blindtext}
+\usepackage[style=ieee, backend=biber]{biblatex}
 \addbibresource{../main.bib}
 \usepackage{booktabs}
 
@@ -25,7 +25,7 @@
 
         \includegraphics[width=7cm]{leipzig_university_logo}\\[1cm] % Adjust size as needed
 
-        {\huge \textbf{Ovulation Prediction with Machine Learning}}\\[1.5cm]
+        {\huge \textbf{Body-Core Temperature based Ovulation Prediction with Machine Learning}}\\[1.5cm]
 
         \textbf{Master’s Thesis}\\[1cm]
 
@@ -38,7 +38,8 @@
         \textbf{Date:} September 2025\\[2cm]
 
         \textbf{Master of Science: Data Science}\\
-        \textbf{Leipzig University}
+        \textbf{Leipzig University}\\
+        \textit{In collaboration with VivoSensMedical GmbH}
     \end{titlepage}
 
     \pagebreak
@@ -47,6 +48,7 @@
     \end{abstract}
     \pagebreak
 
+
     \tableofcontents
 
     \include{sections/introduction}
@@ -63,6 +65,19 @@
 
     \include{sections/conclusion}
 
+    \section*{Declaration of Use of AI-Assisted Writing Tools}
+
+    Parts of this thesis were prepared with the assistance of generative AI tools, including OpenAI’s ChatGPT .
+    These tools were used to improve clarity, coherence, and fluency of the written language,
+    to rephrase technical explanations for greater precision, and to support structural editing.
+
+    At no point were generative models used to produce scientific content, generate results, or conduct analysis.
+    All AI-generated suggestions were critically reviewed, adapted, and verified by me to ensure their accuracy and relevance.
+
+    The outputs served solely as editorial support and did not replace my own academic work or authorship.
+    Responsibility for the content, structure, and conclusions of this thesis remains entirely my own.
+    \pagebreak
+
     \printbibliography
 
 \end{document}

thesis/sections/methodology.tex

@@ -165,6 +165,7 @@ The 1-day window length reflects the expected circadian cycle and strikes a bala
 
 Target features represent the outputs that the models are trained to predict.
 These can include exogenous biological outcomes or interpretable derivatives of input features.
+For this study, the following exogenous variables are used:
 
 \begin{itemize}
     \item \textbf{Fertility / Pregnancy Probability} — The estimated probability of conception from unprotected intercourse on the current day.
@@ -185,6 +186,7 @@ These can include exogenous biological outcomes or interpretable derivatives of
 The combination of these two targets is intended to provide the user with both physiological and practical insight:
 \textit{Fertility probability} conveys the likelihood of conception, but alone does not indicate whether ovulation is yet to come or has already passed.
 A fertility probability near zero could mean that ovulation is either in the past or still ahead—information the model alone cannot disambiguate.
+Figure~\ref{fig:methodology_target_features} shows an example cycle where the target feature values during the course of a cycle can be seen.
 
 The \textit{ovulation-over indicator}, by contrast, explicitly marks the post-ovulatory phase, but does not describe conception risk.
 Together, the two outputs offer complementary information and improve interpretability for real-time user-facing applications.
@@ -192,14 +194,50 @@ Together, the two outputs offer complementary information and improve interpreta
 As discussed in Section~\ref{sec:discussion}, all predictions are subject to further interpretation before presentation in the product interface.
 The model outputs represent data-driven estimates and do not constitute medical advice or diagnostic statements.
 
-\subsection{Time-Series Modeling Approach}\label{subsec:time-series_modeling_approach}
+\begin{table}[htbp]
+    \centering
+    \begin{tabular}{@{}lp{0.62\linewidth}@{}}
+        \toprule
+        \textbf{Feature Name}           & \textbf{Description}                                        \\
+        \midrule
+        \multicolumn{2}{@{}l}{\textbf{Static Features}} \\
+        \midrule
+        User age                        & Age in years; mean imputed if missing                       \\
+        User height                     & Height in centimeters; mean imputed if missing              \\
+        User weight                     & Weight in kilograms; mean imputed if missing                \\
+        Average cycle length            & Mean length of all previous cycles for this user            \\
+        Cycle length STD                & Standard deviation of previous cycle lengths                \\
+        Ovulatory fraction              & Proportion of prior cycles classified as ovulatory          \\
+        Cycle count                     & Number of previously completed cycles available             \\
+        Avg. pre-ovulatory temperature  & Mean temperature in the follicular phase of previous cycles \\
+        Avg. post-ovulatory temperature & Mean temperature in the luteal phase of previous cycles     \\
+        \midrule
+        \multicolumn{2}{@{}l}{\textbf{Known (Time-Based) Features}} \\
+        \midrule
+        Time since cycle start          & Hours since the start of the current cycle                  \\
+        Hour of day                     & Hour of the day (0–23)                                      \\
+        Day of the week                 & Day of the week (0–6)                                       \\
+        Month of the year               & Month of the year (0–11)                                    \\
+        \midrule
+        \multicolumn{2}{@{}l}{\textbf{Observable Features}} \\
+        \midrule
+        Raw temperature                 & Temperature value recorded by the sensor                    \\
+        Rolling average temperature     & 1-day rolling average (288 measurements)                    \\
+        Rolling window temp. minimum    & Minimum temperature within 1-day window                     \\
+        Rolling window temp. maximum    & Maximum temperature within 1-day window                     \\
+        \bottomrule
+    \end{tabular}
+    \caption{Overview of all features and their descriptions.}
+    \label{tab:feature_overview}
+\end{table}
 
-\subsubsection{Input \& Output Modeling}\label{subsubsec:input_output_modeling}
 
-% build in the probability curve for the pregnancy chance
+All features were normalized based on their empirical distributions.
+A \textit{standard scaler} was applied to approximately normal features without outliers,
+a \textit{robust scaler} was used for distributions with outliers, and a \textit{MinMax scaler} was used for all others.
 
-\subsubsection{Sequence Representation and Sampling}
-\label{subsubsec:input_sequence_construction}
+\subsubsection{Time-Series Input Representation}
+\label{subsubsec:time_series_input_representation}
 
 Due to the high temporal resolution of the temperature data (288 measurements per day), raw input sequences can become prohibitively long for most model types.
 
@@ -207,19 +245,25 @@ To manage input size and evaluate the impact of temporal resolution on predictiv
 Consecutive time steps are aggregated into bins of configurable size, and each bin is reduced to a single value using a feature-specific aggregation function.
 
 For most continuous features, the \texttt{mean} is used.
-For cyclic or categorical features—such as \textit{day of the week}— the \texttt{max} or \texttt{mode} is applied to avoid introducing
-artifacts at bin boundaries, where values from distinct categories (e.g., hours 23 and 0) might otherwise be averaged into a nonexistent intermediate state.
+For cyclic or categorical features—such as \textit{day of the week}—the \texttt{max} or \texttt{mode} is applied to avoid introducing artifacts at bin boundaries,
+where values from distinct categories (e.g., hours 23 and 0) might otherwise be averaged into a nonexistent intermediate state.
 
 The effect of different sampling resolutions and aggregation strategies is evaluated in Section~\ref{sec:results}.
 
 To simulate real-time prediction rather than retrospective analysis, a sliding-window approach is employed.
-This allows the model to make partial predictions based on the data available up to a certain point in the cycle.
+This allows the model to make predictions based only on data available up to a specific point in the cycle.
 
-Each cycle is split into overlapping input windows, where each window includes data from the start of the cycle up to a specific day.
+Each cycle is split into overlapping input windows, where each window includes data from the cycle start up to a defined time step.
 The window length is fixed and configurable.
 As the cycle progresses, the window slides forward, allowing the model to incorporate increasing historical context over time.
 
-This setup enables temporally resolved predictions at different stages of the cycle and supports evaluation of how predictive accuracy changes with increasing context.
+For the model types used in this study, each window produces a single output vector.
+By default, this corresponds to the predicted target values at the final time step of the window, though this can be offset depending on configuration.
+While the architecture could be extended to produce output sequences (e.g., one prediction per input step), this study focuses on single-vector outputs.
+
+This setup enables temporally resolved predictions at different stages of the cycle and supports analysis of how predictive accuracy evolves with increasing context.
+Depending on the configuration, downsampling and windowing can be skipped to allow the raw data to be processed by the models themselves.
+This is used primarily in the convolutional flavours of the models, to allow them to learn the best way of reducing the input complexity based on the data itself.
 
 \begin{figure}[htbp]
     \centering
@@ -234,20 +278,125 @@ This setup enables temporally resolved predictions at different stages of the cy
 Fixed-length input windows would normally prevent early-cycle predictions when insufficient data is available.
 To address this, left-padding is applied using masked values.
 
-In this study, predictions are enabled once at least 4 days of data are available.
+In this study, predictions are enabled once at least four days of data are available.
 A padding value of 0.0 is used for all features, and the padding length is adjusted accordingly.
-This choice ensures that the model learns to ignore tokens consisting entirely of padding.
+This design ensures that the model learns to ignore tokens consisting entirely of padding.
 
-The feature \textit{hours since start}, which encodes the time elapsed since cycle onset, will be zero for all padded tokens—explicitly indicating
-that these entries contain no meaningful information.
+The feature \textit{hours since start}, which encodes the time elapsed since cycle onset, is also set to 0.0 for all padded tokens—
+explicitly indicating that these entries contain no usable information.
 
 Figure~\ref{fig:methodology_padding_example} shows an example of such padding during early-cycle input preparation.
 
+\subsection{Model Architecture and Selection}\label{subsec:model_architecture_and_selection}
+
+The primary objective of this study is to find models that accurately predict the features introduced in~\ref{fig:methodology_target_features},
+based on the historical data and context variables.
+This task presents several modelling challenges: capturing temporal dependencies across varying cycle lengths, handling irregular menstrual patterns
+and adapting to user-specific variability.
+
+Ultimately, the models used in this study were selected based on their ability to:
+\begin{itemize}
+    \item Leverage sequential input efficiently across multiple time scales
+    \item Learn temporal patterns from partially observed data
+    \item Generalize across users while incorporating personalized cycle context
+\end{itemize}
+
+\subsubsection{LSTM-Architecture}\label{subsubsec:lstm_architecture}
+\begin{figure}[htbp]
+    \centering
+    \includegraphics[width=0.7\textwidth]{methodology_lstm_architecture}
+    \caption{
+        Schematic overview of the LSTM architecture used in this study.
+    }
+    \label{fig:methodology_lstm_architecture}
+\end{figure}
+
+Recurrent neural networks, particularly GRUs and LSTMs, were tested for their ability to model long-term dependencies in the time series.
+Their sequential memory structure allows them to retain information across cycle days, but they may struggle with high-resolution input and longer sequences.
+LSTMs, in particular, have a long history of strong performance in sequence prediction tasks.
+For this study, a stack of LSTM layers was followed by a linear projection layer, mapping the hidden state at the final time
+step to the two target variables: fertility probability and ovulation-over indicator.
+Figure~\ref{fig:methodology_lstm_architecture} shows the overall architecture pipeline used for the LSTM-based model.
+The stacked inputs and outputs denote the batch processing of the model.
+
+Model-specific architectural parameters are:
+\begin{itemize}
+    \item \textbf{Input Length} — Number of time steps included in each input sequence.
+    \item \textbf{Hidden Size} — Dimensionality of the LSTM’s internal hidden state.
+    \item \textbf{Number of Layers} — Depth of the LSTM stack.
+\end{itemize}
+The specific values and tuning ranges for these parameters are discussed in Section~\ref{subsubsec:hyperparameter_tuning}.
+
+\subsubsection{Transformer Architecture}\label{subsubsec:transformer_architecture}
+
+\begin{figure}[htbp]
+    \centering
+    \includegraphics[width=0.7\textwidth]{methodology_transformer_architecture}
+    \caption{
+        Schematic overview of the Transformer architecture used in this study.
+    }
+    \label{fig:methodology_transformer_architecture}
+\end{figure}
+
+Self-attention models such as the Transformer were used to capture long-range dependencies in the sequence without relying on recurrence.
+These models have demonstrated state-of-the-art performance in various sequential tasks and are more robust to vanishing gradients than RNN-based alternatives.
+
+An overview of the original Transformer design is provided in Section~\ref{subsubsec:transformer_models}.
+For this study, the architecture was adapted for multivariate time-series prediction.
+
+As the input consists of continuous features rather than discrete tokens, no embedding layer is used.
+However, positional encodings are still added to allow the model to interpret the relative positions of tokens—essential for effective attention computation.
+While time-dependent features like \textit{hour-of-day} or \textit{time-since-cycle-start} carry positional information implicitly,
+positional encoding was retained for architectural consistency.
+A future direction could investigate the impact of omitting it in such naturally ordered domains.
+
+Since the task does not require sequence-to-sequence modeling, only the encoder part of the Transformer is used.
+Its output—one vector per input token—is aggregated via 1D adaptive average pooling, resulting in a single vector representation per sequence.
+This vector is then passed through a linear projection layer to produce the two target outputs:
+fertility probability and ovulation-over indicator.
+
+Figure~\ref{fig:methodology_transformer_architecture} shows the overall architecture.
+The stacked inputs and outputs indicate batch processing.
+
+\subsubsection{Temporal Convolution Layer}
+\label{subsubsec:temporal_convolution_layer}
+
+\begin{figure}[htbp]
+    \centering
+    \includegraphics[width=0.7\textwidth]{methodology_convolution_architecture}
+    \caption{
+        Schematic overview of the temporal convolution layer architecture used in this study.
+    }
+    \label{fig:methodology_convolution_architecture}
+\end{figure}
+
+For both the LSTM and Transformer models, long input sequences can substantially increase model complexity and training time.
+While input downsampling mitigates this, static resampling risks discarding relevant temporal patterns and reducing predictive quality.
+
+To address this, a learnable temporal downsampling module was introduced, based on 1D convolutional layers combined with adaptive pooling.
+This allows the model to reduce sequence length in a data-driven manner while preserving important features.
+
+Specifically, a two-stage convolutional block is used to reduce the input resolution from 288 to 48 measurements per day.
+Each stage consists of a 1D convolution followed by an adaptive average pooling layer.
+The feature dimension remains unchanged throughout the downsampling process.
+Figure~\ref{fig:methodology_convolution_architecture} shows the full pipeline of this convolutional preprocessing module.
+The output is subsequently passed to the LSTM or Transformer model as described in Sections~\ref{subsubsec:lstm_architecture} and~\ref{subsubsec:transformer_architecture}.
+
+The convolutional layer is trained end-to-end with the main model, allowing gradient-based optimization
+to adaptively learn which local patterns are most informative for the downstream prediction tasks.
+
+Further research may be necessary to identify optimal parameters for the convolutional downsampling process.
+Both the number of convolutional stages and the final output resolution can be tuned to balance model capacity and temporal fidelity.
+Future work could also explore alternative, potentially more interpretable downsampling strategies.
+In particular, a qualitative analysis of which temporal motifs are preserved—or lost—through the convolutional layers
+might offer valuable insight into the interpretability and robustness of learned representations.
+
 \subsection{Model Training}\label{subsec:model_training}
 
 \subsubsection{Training Setup}\label{subsubsec:training_setup}
 
 \subsubsection{Hyperparameter Tuning}\label{subsubsec:hyperparameter_tuning}
+% what name to use? -> fix experimental setup reference from earlier
 
 \subsubsection{Training Details}\label{subsubsec:training_details}
 

thesis/sections/results.tex

@@ -2,3 +2,5 @@
 %! Date = 3/6/25
 
 \section{Results}\label{sec:results}
+
+% provide information about the training behaviour and statistic of the different models