further work

thesis/sections/background.tex

@@ -16,43 +16,55 @@ Around day 14 of a typical cycle, ovulation occurs, marking the transition to th
 Ovulation refers to the rupture of the mature ovarian follicle and the release of an egg cell into the fallopian tube.
 Figure~\ref{fig:background_basic_female_reproductive_system} illustrates the female reproductive system,
 including the ovaries and the fallopian tubes.
-\\
-\begin{figure}
+
+\begin{figure}[htb]
     \centering
     \includegraphics[width=0.4\textwidth]{background_female_reproductive_organs}
     \caption{The basic female reproductive system~\cite{wikimedia_commons_basic_2019}.}
     \label{fig:background_basic_female_reproductive_system}
 \end{figure}
+
 Ovulation is triggered by a surge in \textbf{luteinizing hormone (LH)}
 and \textbf{follicle-stimulating hormone (FSH)}, following a peak in estradiol levels.
 As ovulation occurs, estradiol levels drop, progesterone levels begin to rise and a slight increase in body temperature
 (typically around 0.5°C) can be observed.
 This marks the beginning of the luteal phase.
-\\
+
 During the luteal phase, the endometrium thickens further, creating an optimal environment for embryo implantation.
 LH and FSH levels decrease, while progesterone remains elevated to support endometrial maintenance.
 If fertilization does not occur, progesterone levels drop, leading to the shedding of the endometrial lining along
 with the unfertilized egg.
 This process, known as menstruation, marks the beginning of a new cycle.
-\\
-The menstrual cycle typically lasts around 28 days, with ovulation occurring near the midpoint.
-However, variations, particularly in the follicular phase length, are common and can be influenced by factors such as stress, diet, exercise and age~\cite{silberstein_physiology_2000}.
-Figure~\ref{fig:background_menstrual_cycle_physiology} provides a detailed overview of the hormonal and physiological changes
-throughout the menstrual cycle.
 
-%TODO: implement a graph of a "normal" cycle with labeled phases for a better understanding of its structure
-
-\begin{figure}[htbp]
+\begin{figure}[htb]
     \centering
     \includegraphics[width=0.6\textwidth]{background_menstrual_cycle_physiology}
     \caption{Physiological changes during the menstrual cycle~\cite{pedroso_menstrual_2022}.}
     \label{fig:background_menstrual_cycle_physiology}
 \end{figure}
 
+The menstrual cycle typically lasts around 28 days, with ovulation occurring near the midpoint.
+However, variations, particularly in the follicular phase length, are common and can be influenced by factors such as stress, diet, exercise and age~\cite{silberstein_physiology_2000}.
+Figure~\ref{fig:background_menstrual_cycle_physiology} provides a detailed overview of the hormonal and physiological changes
+throughout the menstrual cycle.
+
+\begin{figure}[htb]
+    \centering
+    \includegraphics[width=0.9\textwidth]{background_labeled_cycle}
+    \caption{A cycles temperature curve with its phases and ovulation}
+    \label{fig:background_labeled_cycle}
+\end{figure}
+
+Figure~\ref{fig:background_labeled_cycle} shows the temperature curve over the course of a menstrual cycle with the
+menstruation, fertile phase and ovulation marked.
+It starts with a menstruation and ends just before the next menstruation.
+The follicular phase starts at the beginning and goes on until the ovulation.
+The luteal phase begins at the ovulation and continues until the next menstruation.
+
 Not every cycle results in ovulation—a phenomenon known as anovulation—which leads to a monophasic temperature pattern.
 Anovulation can have various causes, including hormonal imbalances, stress, or underlying health conditions~\cite{rosenfield_adolescent_2013}.
 
-\begin{figure}[htbp]
+\begin{figure}[htb]
     \centering
     \includegraphics[width=0.9\textwidth]{background_anovulatory_cycle}
     \caption{Example of a cycle without an ovulation and the resulting absence of a temperature rise}
@@ -71,14 +83,14 @@ Figure~\ref{fig:background_anovulation} shows a cycle that does not have an ovul
 To illustrate the diversity of real-world menstrual cycles, Figures~\ref{fig:background_long_cycle} and~\ref{fig:background_short_cycle}
 show examples of cycles that are significantly longer or shorter than a normative 28-day cycle.
 
-\begin{figure}[htbp]
+\begin{figure}[htb]
     \centering
     \includegraphics[width=0.9\textwidth]{background_long_cycle}
     \caption{Example of a long cycle with a length of 111 days}
     \label{fig:background_long_cycle}
 \end{figure}
 
-\begin{figure}[htbp]
+\begin{figure}[htb]
     \centering
     \includegraphics[width=0.9\textwidth]{background_short_cycle}
     \caption{Example of a short cycle with a length of 22 days}
@@ -97,7 +109,7 @@ rather than relying on population-wide assumptions.
 For a regular cycle pattern, sophisticated analysis or predictions are often not necessary, as the last ovulation day
 can reliably be used as the next.
 
-\begin{figure}[htbp]
+\begin{figure}[htb]
     \centering
     \includegraphics[width=0.9\textwidth]{background_irregular_cycle_example}
     \caption{Example of a woman with irregular menstrual rhythm}
@@ -355,7 +367,7 @@ linearly projected from the input and used to compute attention scores.
 \end{figure}
 
 
-Figure~\ref{fig:background_transformer_architecture} illustrates the original encoder-decoder model introduced by~\citeyear{vaswani_attention_2017}.
+Figure~\ref{fig:background_transformer_architecture} illustrates the original encoder-decoder model introduced by~\citeauthor{vaswani_attention_2017}.
 
 The Transformer architecture consists of two components: an \emph{Encoder} and a \emph{Decoder}.
 

thesis/sections/discussion.tex

@@ -8,3 +8,11 @@
 % talk about whether bbt / temperature can be used for such a task, discuss bbt doubt papers
 % While previous work has argued against the predictive value of BBT~\cite{some_author_2010}, our findings suggest otherwise.
 %Using continuous core body temperature data from 40,000 cycles, we demonstrate that temperature-based models can reliably detect ovulatory patterns, even in the presence of physiological noise or mild irregularity.
+
+
+\section{Future Work}\label{sec:future_work}
+
+% inclusion of markers
+% better labeling process
+% more advanced models
+%

thesis/sections/methodology.tex

@@ -8,46 +8,19 @@
 % you did I apply it
 % implementation details
 
-\subsection{Data Collection \& Preprocessing}\label{subsec:data_collection_preprocessing}
+\subsection{Data Preprocessing}\label{subsec:data_preprocessing}
 
-\subsubsection{Data Collection}\label{subsubsec:data_collection}
-The dataset used in this work was collected by \textit{VivoSensMedical GmbH}, a medical technology company based in Leipzig, Germany,
-specializing in fertility tracking devices and applications.
-Each entry in the dataset is derived from temperature measurements recorded by the \textit{OvulaRing} wearable
-device\cite{alexander_fertilitatsmonitoring_2014}, which continuously measures core body temperature every 5 minutes.
-The sensor is worn intra-vaginally and is designed for extended use, requiring removal only for data synchronization.
-This enables high-resolution, long-term temperature tracking, providing a detailed view of menstrual cycle patterns.
+\subsubsection{Data Filtering}\label{subsubsec:data_filtering}
 
-% add a few example cycles
-% TODO: stats about dataset
-The dataset used for training and evaluation consists of 45000 cycles,
+As briefly mentioned in~\ref{subsec:data_background}, not all cycles in the dataset are suitable for training.
+Cycles that are either too short (\textless 10 days) or too long (\textgreater 150 days) are excluded,
+as they typically indicate erroneous entries, pregnancies, sensor failures, or data processing issues.
 
-In addition to raw temperature values, the dataset includes user-specific metadata, such as health status, weight, and age.
-Users can also manually input time-dependent markers, indicating events such as menstruation, ovulation, and intercourse,
-which serve as contextual features for the model.
-While these additional data points can enhance predictive accuracy, it is important to note that they are self-reported
-and may be subject to errors or biases.
+In addition, incomplete cycles are filtered out, since new cycles continuously arrive from active users
+and may not contain the full sequence of data required for retrospective labeling.
 
-\subsubsection{Data Preprocessing}\label{subsubsec:data_preprocessing}
-Not all cycles in the dataset can be used for training.
-Cycles that are too short are excluded, as they usually do not contain enough information to make reliable predictions.
-Additionally, cycles that are too long are also excluded, as they might contain multiple ovulation events or
-pregnancies, which would make the prediction task ambiguous.
-
-\subsubsection{Feature Engineering}\label{subsubsec:feature_engineering}
-
-%TODO: show that a model without marker context and one with them is trained, to show the impact of the markers
-
-%TODO: show that for initial testing and fine tuning the dataset was reduced to 10% of size to speed up early impressions of performance
-%TODO: show that initial feature set with only temperature as observable and only pregnancy "chance" as target
-% did not show promise on small 10% dataset, so the features got extended
-% show the modifications that were made to account for multi target prediction
-% show, the mothod of circumventing the non-padding-implementation by using a dedicated feature
-% show, that a different loss / temporal loss wheighting or position aware loss could help
-% show, the problem of the model using its past targets, even though they are not observable and how I dealt with this -> explain and use exogenous
-% introduce change of quantiles to single variable output and why, quantiles loss other than .5 was almost always just 0 or 1, so no significant predicitve quality in this, and this only makes predictions harder for model
-% explain why class imablances are there, and what I did to counteract this, also explain, why special weighting is useful for something like the pregnancy risk prediction
-% show the work that needed to be done for inference -> padding values for future observables
+Temperature values outside the physiologically plausible range—below 35\textdegree C or above 43\textdegree C—are also excluded,
+as they typically result from sensor malfunction or transmission errors.
 
 \subsubsection{Data Labeling}\label{subsubsec:data_labeling}
 
@@ -70,19 +43,117 @@ The algorithm operates in two stages:
 
 This retrospective labeling provides a practical and scalable proxy for ground truth, enabling training and evaluation across a large, real-world dataset,
 especially, as temperature is at least an excellent retrospective marker for ovulation.
-%TODO: provide statistics for the retrospective algorithm
+In internal evaluations, the estimated ovulation day fell within a \(\pm\)2-day window of the expert reference in approximately 86\% of labeled cycles.
 
-These labels are then used as ground truth labels for the prediction algorithms.
-We are aware of potential shortcomings with this approach.
-We acknowledge that this approach introduces some label noise—particularly in edge cases where temperature signals are
-ambiguous or distorted (e.g., illness, missed measurements).
-These inaccuracies may propagate into the predictive models.
-However, label quality is continuously reviewed and can be refined as part of future model iterations.
+These labels serve as the supervisory signal for model training and evaluation.
+We acknowledge the limitations of this method: ambiguous or noisy temperature patterns—due to illness, dropout,
+or sensor error—can lead to mislabeled examples, which may propagate to downstream models.
+However, label quality is continuously reviewed and may be refined iteratively as model performance improves.
 
-The usage of the ovulation day labels in the model input features is documented in section~\ref{subsubsec:input_output_modeling}
+The specific usage of ovulation labels in feature construction is described in the next section.
 
+\subsection{Feature Engineering}\label{subsec:feature_engineering}
 
-% show, that for very regular cycles, no sophisticated methods are necessary
+The features used as model inputs have been divided into three categories:
+\begin{itemize}
+    \item \textbf{Static features} - Characteristics, that remain constant across a user's cycle, such as age, height, or average ovulation day
+    \item \textbf{Known features} — Inputs known a priori at each time step, such as time of day or calendar-based variables.
+    \item \textbf{Observable features} — Inputs available at the current time step, including raw and derived temperature values.
+    \item \textbf{Target features} — Outputs the model is trained to predict, such as the fertility probability.
+\end{itemize}
+
+Each feature type can handle categorical and continuous features.
+This allows for mixed inputs, such as scalar measurements and class labels, within the same category.
+
+The categorization into four feature types is intended to clarify the conceptual roles of different input types.
+While the current models concatenate all features into a single input stream, the distinction allows for flexibility—future models
+may process each feature group differently depending on their architectural design.
+
+\subsubsection{Static Features}\label{subsubsec:static_features}
+
+Static features are the features that do not change over the course of a cycle.
+They might even be static for all cycles from a specific user, such as age, height and weight.
+
+The static features are supposed to create contextual information about the cycle and the user that each model can then
+use to learn patterns based not only on the temperature data, but also on this context.
+
+\begin{table}[htbp]
+    \centering
+    \begin{tabular}{@{}lp{0.6\linewidth}@{}}
+        \toprule
+        \textbf{Feature}                & \textbf{Description}                                        \\
+        \midrule
+        User age                        & Age in years; mean imputed if missing                       \\
+        User height                     & Height in centimeters; mean imputed if missing              \\
+        User weight                     & Weight in kilograms; mean imputed if missing                \\
+        Average cycle length            & Mean length of all previous cycles for this user            \\
+        Cycle length SD                 & Standard deviation of previous cycle lengths                \\
+        Average ovulation day           & Mean day of ovulation from previous cycles                  \\
+        Ovulation SD                    & Standard deviation of ovulation day of previous cycles      \\
+        Ovulatory fraction              & Proportion of prior cycles classified as ovulatory          \\
+        Cycle count                     & Number of previous completed cycles available               \\
+        Avg. pre-ovulation temperature  & Mean temperature in the follicular phase of previous cycles \\
+        Avg. post-ovulation temperature & Mean temperature in the luteal phase of previous cycles     \\
+        \bottomrule
+    \end{tabular}
+    \caption{Static features used as model inputs}
+    \label{tab:static_features}
+\end{table}
+
+Table~\ref{tab:static_features} shows all static features and their descriptions.
+Prior research by \citeauthor{li_menstrual_2023} has shown that menstrual cycle characteristics vary significantly with age and BMI~\cite{li_menstrual_2023}.
+Including such information is therefore expected to improve predictive performance.
+
+In addition, summary statistics from previous cycles—such as ovulation timing, temperature levels, or the fraction of ovulatory cycles—provide useful individual context.
+These features help the model learn subject-specific variability and better estimate the likelihood and timing of ovulation in the current cycle.
+
+All historical features are computed using only data available prior to the current cycle, ensuring no data leakage and supporting robust, user-adaptive learning.
+
+%
+The idea here is to provide as much context information to the models as possible to help them predict the ovulation.
+
+\subsubsection{Known Features}\label{subsubsec:known_features}
+In the context of this study, known features correspond to time-dependent inputs.
+These help the model place each observation in temporal context:
+
+\begin{itemize}
+    \item \textbf{Time since cycle start} — Provides the model with a relative position within the menstrual cycle.
+    \item \textbf{Hour of day} — Helps distinguish between daytime and nighttime patterns, especially relevant for circadian rhythms.
+    \item \textbf{Day of the week} — Encodes potential behavioral differences between weekdays and weekends.
+    \item \textbf{Month of the year} — Captures seasonal variations in temperature patterns or user behavior.
+\end{itemize}
+
+Except for \textit{time since cycle start}, all features are encoded using sine and cosine transforms to preserve their cyclical nature and make them more interpretable for the model.
+
+\begin{figure}[htbp]
+    \centering
+    \includegraphics[width=0.9\textwidth]{methodology_time_feature_sine_encoded}
+    \caption{Sine and cosine encoding of the day-of-week feature.}
+    \label{fig:methodology_time_feature_encoding}
+\end{figure}
+
+Figure~\ref{fig:methodology_time_feature_encoding} illustrates the sine and cosine encoding of the day-of-week feature.
+The cyclical nature of the variable is clearly visible in the transformation.
+
+Although the model architectures used are sequential, the explicit inclusion of these time features allows the models to interpret each time step in a broader context.
+More importantly, they enable the detection of gaps in the recording, which would otherwise not be visible from the data alone.
+
+Additionally, prior research has shown that the menstrual cycle may be influenced by weekly rhythms~\cite{ecochard_menstrual_2024}.
+For example, menstruation has been found to begin more frequently on Thursdays or Fridays, suggesting that behavioral or social factors may modulate certain events in the cycle.
+Including this information could therefore improve the predictive quality of the models.
+
+\subsubsection{Observable Features}\label{subsubsec:observable_features}
+
+The observable features constitute the input features that are directly observable, but only until the current moment.
+
+\begin{itemize}
+    \item \textbf{Temperature} - the raw temperature as recorded by the OvulaRing sensor
+    \item \textbf{Rolling Average Temperature} - the rolling average of the temperature over 1 day (288 measurements)
+    \item \textbf{Rolling Window Temperature Minimum} - the minimum temperature over a rolling window of 1 day
+    \item \textbf{Rolling Window Temperature Maximum} - the maximum temperature over a rolling window of 1 day
+\end{itemize}
+
+% table with all features
 
 \subsection{Time-Series Modeling Approach}\label{subsec:time-series_modeling_approach}
 

thesis/sections/related_work.tex

@@ -3,7 +3,7 @@
 
 
 \section{Related Work}\label{sec:related_work}
-There has been a variety of works in menstrual cycle analysis.
+There has been a variety of works in menstrual cycle analysis and ovulation prediction based on different physiological signs.
 
 \subsection{Temperature-Based Approaches}\label{subsec:temperature_based_approaches}