further work on discussion

thesis/main.tex

@@ -50,7 +50,8 @@
         Alexander Blank\\[0.5cm]
 
         \textbf{Supervisor:}\\
-        Prof. Bogdan Franczyk\\[1.5cm]
+        Prof. Bogdan Franczyk\\
+        Dr. Christian Alvermann\\[1.5cm]
 
         \textbf{Date:} September 2025\\[2cm]
 

thesis/sections/appendix.tex

@@ -488,3 +488,4 @@
         \label{tab:regular_vs_irregular_ov_over_results}
     \end{table}
 \end{landscape}
+

thesis/sections/background.tex

@@ -11,11 +11,12 @@ The menstrual cycle consists of physiological changes preparing the female body
 typically spanning around 28 days but varying considerably among individuals.
 It includes two main phases: the follicular phase, beginning with menstruation, and the luteal phase, following ovulation.
 
-During the follicular phase, ovarian follicles mature under the influence of rising estradiol levels, thickening the uterine lining (endometrium).
+During the follicular phase (Figure~\ref{fig:background_menstrual_cycle_physiology} until day 14),
+ovarian follicles mature under the influence of rising estradiol levels, thickening the uterine lining (endometrium).
 Around mid-cycle, a surge of luteinizing hormone (LH) and follicle-stimulating hormone (FSH), triggered by peak estradiol,
 induces ovulation—the release of a mature egg into the fallopian tube.
 
-After ovulation, the luteal phase begins.
+After ovulation, the luteal phase begins (Figure~\ref{fig:background_menstrual_cycle_physiology} day 14 to 28).
 Progesterone increases substantially, maintaining endometrial thickness for potential embryo implantation.
 In parallel, a subtle rise in body temperature (~0.5°C) occurs due to progesterone elevation.
 If fertilization does not happen, progesterone and temperature decline back to baseline levels, resulting in menstruation and initiating a new cycle.
@@ -55,13 +56,17 @@ The luteal phase begins at the ovulation and continues until the next menstruati
 
 
 While many cycles exhibit a characteristic biphasic pattern, deviations from this norm are common.
-Some remain monophasic, which might be an indication for an anovulatory cycle, which is a menstrual cycle, where no ovulation occurs.
+Some do not show the typical temperature surge, which might be an indication for an anovulatory cycle.
+Anovulatory cycles don't have an ovulation, and thus cannot result in a pregnancy.
 Anovulation can have various causes, including hormonal imbalances, stress, or underlying health conditions~\cite{rosenfield_adolescent_2013}.
-Monophasic cycles with a confirmed ovulation event have been observed, so there seems to be no clear indication that it is a direct cause of anovulation~\cite{moghissi_accuracy_1976}.
+Monophasic cycles with a confirmed ovulation event have also been observed, so there seems to be no clear indication that it is a direct cause of anovulation~\cite{moghissi_accuracy_1976}.
 Thus, distinguishing between ovulatory and anovulatory cycles is challenging, as the only definitive confirmation of
 successful ovulation in a clinical sense is a pregnancy.
 Even ultrasound imaging can only confirm that an egg was released from its follicle—not whether it was successfully implanted or fertilized.
-Figure~\ref{fig:background_anovulation} shows a cycle with a monophasic temperature pattern.
+
+Figure~\ref{fig:background_labeled_cycle} shows a cycle with a typical biphasic temperature pattern.
+In contrast, Figure~\ref{fig:background_anovulation} shows a cycle with a monophasic temperature pattern.
+Over time, the temperature does not show any significant or longer lasting temperature changes.
 
 To illustrate the diversity of real-world menstrual cycles, Figures~\ref{fig:background_long_cycle} and~\ref{fig:background_short_cycle}
 show examples of cycles that are significantly longer or shorter than a normative 28-day cycle.
@@ -107,6 +112,7 @@ Fertility varies throughout the menstrual cycle, centered around the ovulation e
 An egg remains viable for about 24 hours post-ovulation, while sperm can survive up to 6 days in a woman's reproductive tract,
 thus the fertile period extends to approximately five days prior to ovulation~\cite{dunson_day-specific_1999}.
 Consequently, the whole fertile window generally spans six days: five days preceding ovulation and one day after.
+The distribution of fertility probability is not dependent on the length of the cycle.
 
 \begin{figure}[htbp]
     \centering
@@ -116,7 +122,8 @@ Consequently, the whole fertile window generally spans six days: five days prece
     \label{fig:background_pregnancy_chance}
 \end{figure}
 
-Figure~\ref{fig:background_pregnancy_chance} demonstrates the probability of fertilization peaking one day before ovulation, emphasizing the critical timing for fertility prediction.
+Figure~\ref{fig:background_pregnancy_chance} demonstrates the probability of fertilization peaking one day before ovulation,
+emphasizing the critical timing for fertility prediction.
 
 Fertility prediction fundamentally depends on accurate ovulation timing.
 However, since the goal is to identify the fertile window before ovulation occurs, detection must be early and precise.
@@ -141,6 +148,11 @@ pregnancy effort, causing frustration or delays, but no potential dangers to the
 Therefore, algorithms for this group don't need to be as conservative.
 It remains to be seen, where the middle ground lies and how different algorithms perform for different use cases.
 
+In this study, we'll focus on fertility prediction, which incorporates both use cases, and thus we will not train
+different models for each use-case.
+However, we will test the thresholds used for decision-making to find use-case dependent optimums.
+Section~\ref{subsubsec:use_case_evaluation} will introduce the methodology in more detail.
+
 \subsubsection{Physiological Signs of Ovulation}\label{subsubsec:physiological_signs}
 Several physiological signs correlate with ovulation and can be used for prediction.
 As shown in Figure~\ref{fig:background_menstrual_cycle_physiology}, these include hormonal fluctuations (LH and FSH surges) and
@@ -177,7 +189,7 @@ Thanks to a battery life of at least six months, the device supports continuous
 
 A known limitation of manual cycle annotations is the potential for misalignment.
 Intermediate bleeding events unrelated to menstruation (e.g., ovulatory spotting or irregular shedding) or missing menstruation entries can lead to ambiguous cycle definitions.
-Therefore, all user-entered cycle starts undergo manual review to reduce annotation errors.
+Manual review of all user-entered cycle starts would improve the quality of the annotations, but is currently not done.
 
 \begin{figure}[htbp]
     \centering
@@ -198,7 +210,7 @@ contained hardware-related anomalies, or fell outside a reasonable cycle length
 
 Cycles shorter than 10 days typically result from incorrect cycle start entries or premature termination of temperature recordings.
 Long cycles, longer than 150 days, are often due to data entry errors or pregnancy-related recordings,
-where the sensor was worn continuously throughout gestation—sometimes producing sequences up to nine months long.
+where the sensor was worn continuously throughout gestation, sometimes producing sequences up to nine months long.
 
 While such cases may still contain useful information, they were excluded from this analysis to avoid complications in preprocessing and labeling.
 In most instances, only a small portion of these extended cycles contributes meaningfully to the study objectives.
@@ -207,8 +219,10 @@ The cutoff values for cycle length are 10 and 150 days, respectively.
 The cleaned dataset has 40{,}266 menstrual cycles from 6{,}245 users.
 The median number of cycles per user is 4 (IQR: 2--8) and the median cycle length is 28 days (IQR: 26--32).
 
-The average data density—defined as the fraction of available measurements out of the theoretical maximum of 288 measurements per day—is 0.90.
+The average data density, defined as the fraction of available measurements out of the theoretical maximum of 288 measurements per day, is 0.90.
 This corresponds to an average data availability of 90\% per cycle, with an average loss of 10\%.
+Data loss is usually caused by users not wearing the sensor for a longer time, or an erroneous sensor that needed replacement,
+but was not immediately delivered.
 
 37934 cycles (94\%) were classified as biphasic and 2333 (6\%) as monophasic.
 
@@ -217,7 +231,7 @@ Users had a median age of 32 years (IQR: 29–36), median weight of 65 kg (IQR:
 \subsubsection{Irregularities and Confounding Factors}
 Despite careful data collection, real-world measurements are subject to physiological and behavioral noise.
 Especially core body temperature is influenced by various factors unrelated to the menstrual cycle.
-Illnesses—especially those involving fever—can significantly affect temperature patterns.
+Illnesses, especially those involving fever, can significantly affect temperature patterns.
 This poses a challenge for any analysis relying on temperature data, as one of the key physiological indicators of ovulation is a post-ovulatory temperature rise (see Section~\ref{subsubsec:physiological_signs}).
 
 Figure~\ref{fig:background_fever_cycle} shows an example of a cycle where an illness caused a marked increase in temperature.
@@ -403,10 +417,9 @@ In the case of machine translation, this input would be a sentence in the source
 
 
 The input tokens are first mapped to dense continuous vector representations (embeddings).
-Since the attention mechanism permutation-invariant---that is, it does not inherently encode the order of tokens in the sequence---
+Since the attention mechanism permutation-invariant, that is, it does not inherently encode the order of tokens in the sequence,
 \emph{positional encodings} are added to the token embeddings to provide information about the token positions in the sequence.
 
-
 Without positional encoding, repeated tokens such as `The` would be indistinguishable
 to the model regardless of their location, even if they play different syntactic or semantic roles.
 Positional encodings, often based on sinusoidal functions, inject a unique position-dependent signal
@@ -419,6 +432,7 @@ In this work, we use sine and cosine functions of different frequencies:
 \end{align}
 where \(i\) is the dimension of the input and \(pos\) is the position in the sequence,
 as in the original paper~\cite{vaswani_attention_2017}.
+Positional encodings are added to the embedded inputs through simple addition.
 
 \begin{figure}[htbp]
     \centering
@@ -562,6 +576,8 @@ These techniques reduce the computational load while maintaining salient informa
 
 Figure~\ref{fig:background_convolution_example} illustrates a simple one-dimensional convolution applied to a sequence using a filter of size 3.
 The stride determines how far the filter moves at each step, affecting both the resolution and length of the resulting feature map.
+As a recap, resolution in this context means, how many of the daily measurements (initially 288) are retained for the input of the models.
+The sequence length is the overall number of measurements available.
 
 \begin{figure}[htbp]
     \centering
@@ -571,6 +587,6 @@ The stride determines how far the filter moves at each step, affecting both the
 \end{figure}
 
 In summary, time-series modeling offers a range of approaches, each with specific trade-offs.
-RNNs and LSTMs provide explicit sequential modeling but suffer from training inefficiencies.
+RNNs and LSTMs provide explicit sequential modeling but suffer from vanishing gradients, especially for longer sequences.
 Transformers excel at long-range context capture but demand more memory and parallelization.
-Convolutional layers offer efficient local feature extraction and often serve as useful pre-processing stages for both model families.
+Convolutional layers might offer efficient local feature extraction and might mitigate the shortcomings of both architecture types.

thesis/sections/discussion.tex

@@ -4,6 +4,165 @@
 
 \section{Discussion}\label{sec:discussion}
 
+\subsection{RQ1: Model Architecture Evuation an Optimization}\label{subsec:discussion_rq1_model_evaluation}
+
+
+The first research objective considered different model architectures and how they perform
+under various hyperparameter configurations, with respect to the optimization goals of this study.
+
+We found that non-convolutional models performed best with medium resolutions and longer input windows,
+where they also outperformed their convolutional counterparts.
+This suggests that including more historical context improves prediction accuracy.
+However, there appears to be a trade-off:
+while more information can improve predictions, it also introduces additional noise, which the models struggle to handle effectively.
+The strong performance at medium resolutions indicates a possible sweet spot—balancing information richness with model capacity and generalization.
+
+The convolutional variants performed best with medium-length input windows, which was somewhat surprising.
+One would expect convolutional downsampling to enable more efficient data representation,
+allowing the model to extract relevant patterns from high-resolution data and to perform better on longer sequences.
+However, this was not observed in our experiments.
+Instead, convolutional models performed worse than non-convolutional models on longer input windows,
+suggesting that the benefits of downsampling may be offset by limitations in capturing long-range dependencies.
+For medium and short input windows, however, convolutional models did outperform the non-convolutional
+variants—indicating that the convolution itself provides an advantage when input length is limited.
+
+To our knowledge, no prior studies have compared different deep learning architectures for real-time fertility prediction.
+This highlights both the novelty and the exploratory nature of our approach.
+
+All these results must be interpreted with caution, as they are based on single-shot experiments and are not statistically robust.
+A more rigorous evaluation would involve training all models multiple times to minimize the
+impact of random initialization and other stochastic processes in training.
+For most metrics, differences in performance were small, often less than 1\%, and could easily be attributed to such variability.
+
+We are also aware of potential losses in predictive quality due to class imbalance in the labels.
+Especially in long cycles, the targets are mostly zero, except for a narrow window around ovulation
+(for the fertility-probability target) or the area after ovulation (for the ov-over target).
+Positive-to-negative target ratios can reach 10–20:1 in long cycles, meaning for each non-zero target day,
+there are 10 to 20 days with all-zero targets.
+This imbalance may introduce instability during training and reduce predictive performance.
+Future iterations of this study could incorporate class-weighting or loss-balancing mechanisms to address this issue.
+
+Future work should include a more thorough parameter search and statistically more robust evaluation methodology.
+A more sophisticated hyperparameter search may be necessary to identify globally optimal configurations for each model architecture.
+Time and resource constraints could be alleviated using more efficient search algorithms such as Bayesian Optimization,
+Genetic Algorithms, or Neural Architecture Search (NAS) to better explore the joint parameter space.
+
+Additionally, we only focused on basic models—LSTMs and Transformers, with convolutional hybrids.
+A closer look into alternative architectures, or tailoring architectures more specifically
+to the characteristics of menstrual cycle data, could significantly improve predictions.
+Recent Transformer variants for time series modeling, such as \emph{TimeXer}~\cite{wang_timexer_2024}
+or \emph{MAMBA}~\cite{wang_is_2024}, may offer better performance.
+Alternatively, a custom architecture could be developed to reflect the domain-specific structure of biological temperature data more closely.
+This could be combined with a more sophisticated convolutional setup or preprocessing strategy
+to handle high-resolution inputs more efficiently.
+Such preprocessing might help overcome computational bottlenecks that raw LSTMs and Transformers encounter when dealing with large input sequences.
+
+Moreover, a more statistically grounded approach to feature selection could further improve predictive performance.
+Both raw and engineered features could be evaluated for their impact on model output.
+Especially promising are intermediate features such as cycle-level aggregates,
+temperature accumulation over time or space, and static user-level characteristics.
+The database used in this study also contains an extensive set of user-logged marker events, such as intercourse,
+intermediate bleeding, or illnesses, which could help the models learn correlations between these events and temperature fluctuations.
+
+Finally, exploring alternative prediction targets may help address some of the core challenges of fertility prediction.
+Beyond the fertility-probability and ov-over indicators used in this study,
+targets such as the number of days to the next ovulation (or since the last), or the ovulation day
+as a direct regression target, might prove more stable and informative.
+As we will see in the next section, the current targets are highly sensitive to anomalies,
+which may not be the case for alternative formulations.
+
+\subsection{RQ2: Factors and Patterns that Influence Prediction}\label{susbsec:discussion_rq2_factors_and_patterns}
+
+Our second research objective focused on identifying potential factors and patterns in the data
+that influence fertility prediction.
+
+To our knowledge, no prior work has explicitly analyzed the relationship between temperature patterns
+and ovulation—including the corresponding fertile window—in a way that could directly inform machine learning models.
+This underscores the exploratory nature of this analysis.
+
+We identified a consistent pre-ovulatory temperature dip that aligned closely with the ground-truth fertility curve.
+This drop was particularly pronounced in short and regular cycles.
+In contrast, irregular cycles often exhibited too much variability and noise in the follicular phase
+for the models to reliably detect this temperature drop, or to associate it meaningfully with increased fertility probability.
+Interestingly, the magnitude of the temperature dip was strongly correlated with the predicted fertility probability:
+larger dips tended to produce higher model confidence.
+
+This observed correlation may be coincidental, or it may indicate a meaningful biological marker
+for fertility and successful ovulation.
+We hypothesize that there is a real association between the magnitude of the pre-ovulatory temperature dip and the likelihood of ovulation.
+
+However, our method has a critical limitation: the ground-truth labels used were not based on clinically confirmed ovulation events.
+This limitation may result not only in fertility-probability curves that are arbitrarily offset from their true value,
+but also in the inclusion of cycles labeled as ovulatory that in fact were anovulatory.
+As shown in Section~\ref{subsubsec:physiological_signs}, the only reliable indicator
+of ovulation is daily transvaginal ultrasound, which was not feasible given the scale of our dataset.
+
+Future studies could apply the current model to a smaller subset of cycles with clinically
+confirmed ovulation and/or cycles exhibiting a clear temperature rise, to test the hypothesis
+that a significant pre-ovulatory temperature drop is either necessary for ovulation or strongly correlated with fertility.
+
+This approach may also help clarify the role of temperature anomalies in fertility prediction,
+for example, the sharp spike shown in Figure~\ref{fig:results_rq2_anomaly_in_temperature_drop}.
+Anomalies like this, which may be due to illness with fever symptoms, likely affect both
+the menstrual cycle and the model's ability to detect pre-ovulatory fertility.
+This case highlights that body temperature is highly sensitive to external physiological factors
+that cannot be inferred from the temperature curve alone.
+
+It is likely that many additional patterns or confounding factors in
+the temperature data directly or indirectly affect model predictions.
+Future research could explore this further, ideally in collaboration with experts in reproductive health.
+This could lead to a more biologically grounded interpretation of the patterns uncovered by the models.
+
+Moreover, improvements in label quality will naturally improve the interpretability and accuracy of predictions.
+Better labels would also allow more reliable medical interpretation of the prediction outputs and the patterns driving them, as previously discussed.
+
+\subsection{RQ3: Performance Across Regular and Irregular Cycles}\label{subsec:discussion_rq3_regular_vs_irregular}
+
+The third research objective of this study was to compare model performance across subsets of users
+with regular and irregular menstrual cycles.
+
+We found that predictive performance is clearly sensitive to cycle irregularity.
+Both overall accuracy and the improvement in performance over a growing user history
+were substantially better for regular cycles compared to irregular ones.
+This suggests that past cycles contain valuable information that helps the models
+predict the current cycle more reliably, especially when those past cycles follow consistent patterns.
+
+Some of this improvement can also be attributed to static features, several of which are
+derived from aggregated past cycles, such as average cycle length or average ovulation day.
+These features appear to be more informative and stable in the regular cycle group.
+
+Based on the clear difference in performance, we hypothesize, that all models rely more on those static features
+than on patterns identified in the temperature curves, since if that were the case, the difference would be smaller
+or non-existent.
+There might be additional factors that differentiate regular from irregular cycles, that might explain the results,
+which have not been identified yet.
+This could also point out, that it is hard or even impossible to reliably predict fertility based on temperature alone,
+but this hypothesis needs to be investigated in further research.
+
+\citeauthor{yu_tracking_2022} also report a significant drop in performance for irregular cycles
+in their study~\cite{yu_tracking_2022}, particularly in terms of sensitivity.
+While we cannot directly compare results due to methodological differences, especially since
+our models do not perform explicit binary classification into fertile vs.\ non-fertile days, a similar trend is observable.
+In our case, the models tend to overestimate fertility probability when faced with uncertainty, particularly in irregular cycles
+(see Figure~\ref{fig:results_rq2_irregular_cycle_predictions_example}).
+As a result, we do not report specificity or sensitivity values.
+However, the use-case scenarios in the next section offer a more binary evaluation framework.
+
+A promising direction for future work would be to further investigate the causes of
+performance degradation in irregular cycle groups.
+One potential experiment could involve removing static features derived from past cycles to evaluate whether the models
+are genuinely using the temperature patterns from previous cycles that fall within
+the current input window, or relying mainly on those engineered features.
+
+It may also be valuable to explore the effect of increasing the input window length further.
+Ideally, the model would have access to the entire cycle history of a user for training and decision-making.
+However, this would likely exceed the memory and capacity constraints of the architectures used in this study.
+Overcoming this limitation may require more efficient or hierarchical time series architectures, as previously discussed.
+
+\subsection{RQ4: Use-case Evaluations}\label{subsec:discussion_rq4_use_case_evaluations}
+
+
+% -------------------------------------------------------------------------------------
 This study set out to evaluate the predictive value of body temperature for ovulation and fertility,
 to assess the performance of different model architectures across use cases,
 and to compare machine learning models with rule-based baselines.
@@ -12,21 +171,24 @@ Below we discuss the findings in relation to these objectives, their implication
 \subsection{Predictive Value of Body Temperature}
 Our results confirm that body core temperature carries meaningful predictive value for ovulation and fertility,
 particularly in regular cycles with clear pre-ovulatory dips.
+This dip before the
 In such cases, fertility-probability curves aligned closely with the ground truth,
 demonstrating that models can reliably exploit this physiological marker.
-However, anomalies in the signal—such as irregular spikes or absent dips—frequently led to erroneous predictions.
+However, anomalies in the signal, such as irregular spikes or absent dips, frequently led to erroneous predictions.
 This suggests that the models strongly rely on short-term temperature fluctuations without
 distinguishing between ovulation-related and unrelated changes.
 
 The irregular-versus-regular cycle analysis reinforces this interpretation:
 prediction accuracy was consistently higher in regular cycles, indicating that models leverage cyclical regularity in
 addition to absolute thermal changes.
+Especially for regular cycles, the performance of the trained models
+
+%[continue here: bring in your figure-based examples of temperature drops, anomalies, and unclear cases, with interpretation rather than description.]
+
 Taken together, these findings highlight both the utility and the limitations of body temperature as a single biomarker.
 While it provides a strong signal in favorable cases, its variability across users and cycles limits robustness in real-world applications.
 Additional biomarkers or contextual information are likely required to disambiguate genuine ovulatory patterns from noise.
 
-%[continue here: bring in your figure-based examples of temperature drops, anomalies, and unclear cases, with interpretation rather than description.]
-
 \subsection{Model Architectures and Use-Case Performance}
 Across architectures, LSTMs consistently outperformed both Transformers and convolutional models in the fertility-probability task.
 This was somewhat unexpected given the recent dominance of Transformer-based approaches in sequence modeling.
@@ -49,44 +211,6 @@ For pregnancy planning, results were less directly comparable, reflecting the mu
 Nonetheless, the analysis illustrates how probabilistic outputs could be adapted to individual risk preferences,
 highlighting the flexibility of machine learning approaches over fixed rules.
 
-%[continue here: expand with more detail on threshold effects, how different models fared in contraception vs. pregnancy contexts, and the implications for user-facing tools.]
-
-\subsection{Comparison to Rule-Based Baselines}
-Compared with simple heuristic baselines, all machine learning models demonstrated superior predictive performance,
-particularly in irregular cycles where rule-based approaches break down.
-This highlights the advantage of data-driven methods, which can learn subtle patterns and adapt to user-specific variability
-that static rules cannot capture.
-Nevertheless, rule-based methods retain value for their simplicity and interpretability,
-and could complement machine learning models in hybrid approaches where transparency is essential.
-
-%[continue here: connect this explicitly to natural family planning methods in the literature, and comment on where ML truly adds value.]
-
-\subsection{Limitations}
-Several limitations constrain interpretation.
-First, results were reported on aggregate test sets without uncertainty quantification;
-per-user predictions were not retained, preventing bootstrap confidence intervals or paired statistical testing.
-Second, labels were generated by a retrospective algorithm trained on expert annotations.
-Any inaccuracies in this algorithm propagate directly into the training data and may bias model learning.
-Third, reliance on temperature alone leaves the models vulnerable to anomalies caused by illness, lifestyle, or measurement error.
-Finally, resource constraints limited the scope of hyperparameter optimization and the exploration of more advanced architectures.
-
-%[continue here: add dataset-specific limitations such as age range, long-cycle imbalance, and lack of prospective evaluation.]
-
-\subsection{Future Work}
-Future extensions of this work should pursue several directions.
-Incorporating additional user-entered markers—such as bleeding, stress, illness,
-or intercourse—may provide critical context to disambiguate temperature anomalies.
-Alternative target formulations, such as predicting the day of ovulation or the time-to-ovulation,
-could reduce class imbalance and better reflect clinical needs.
-Training separate models for fertility and ovulation-over tasks may also improve performance by reducing task interference.
-
-From a methodological perspective, exploring architectures tailored for time-series forecasting—such as
-TimeXer or MAMBA—alongside principled hyperparameter optimization techniques (e.g., Bayesian optimization, NAS) could yield further gains.
-Most importantly, future studies should evaluate these approaches prospectively,
-with clinically validated ovulation labels and diverse populations,
-to establish their real-world utility in natural family planning and contraception.
-
-%[continue here: add your own vision for clinical applications, e.g. integration into fertility apps, medical oversight, or regulatory implications.]
 
 
 %\section{Discussion}\label{sec:discussion}
@@ -126,16 +250,6 @@ for potential users of the predictions.
 
 All prediction curve in this section have been generated with the best model selected in section~\ref{subsubsec:results_best_model_config_selection},
 i.e., the LSTM model.
-
-\begin{figure}[htbp]
-    \centering
-    \includegraphics[width=1.0\textwidth]{resources/figures/discussion/regular_cycle_fertility_prediction}
-    \caption{
-        Temperature rolling average and fertility-probability prediction for a user with a regular cycle pattern. (Values are scaled features)
-    }
-    \label{fig:discussion_regular_cycle_fertility_prediction}
-\end{figure}
-
 Figure~\ref{fig:discussion_regular_cycle_fertility_prediction} shows the prediction curve for the fertility-probability target
 for a user with a regular cycle pattern.
 For such a regular cycle pattern, the predictions almost exactly match the targets.
@@ -166,33 +280,6 @@ The identified temperature drop and the fertility that seems to come with it cou
 with more accurate ovulation labeling.
 It might be an indicator for a successful upcoming ovulation.
 
-\begin{figure}[htbp]
-    \centering
-    \includegraphics[width=1.0\textwidth]{resources/figures/discussion/temperature_drop_fertility}
-    \caption{
-        Temperature rolling average and fertility-probability prediction with a clear correlation between the temperature drop
-        pre-ovulation and the fertility. (Values are scaled features)
-    }
-    \label{fig:discussion_temperature_drop_fertility_prediction}
-\end{figure}
-\begin{figure}[htbp]
-    \centering
-    \includegraphics[width=1.0\textwidth]{resources/figures/discussion/temperature_spike_in_temperature_dop}
-    \caption{
-        Temperature rolling average and fertility-probability prediction with a disruptive temperature spike during the pre-ovulatory phase,
-        leading to an early end of the predicted fertile window.
-    }
-    \label{fig:discussion_spike_in_temperature_drop}
-\end{figure}
-\begin{figure}[htbp]
-    \centering
-    \includegraphics[width=1.0\textwidth]{resources/figures/discussion/temperature_unclear_temperature_drop}
-    \caption{
-        Temperature rolling average and fertility-probability prediction with no clear temperature drop and a resulting
-        incorrect prediction. (Values are scaled features)
-    }
-    \label{fig:discussion_unclear_temperature_drop}
-\end{figure}
 
 The results of the irregular vs regular cycle study, as shown in~\ref{subsubsec:regular_vs_irregular_cycles},
 indicate, that the models seem to learn recurrent cycle pattern on a per-user basis.
@@ -240,53 +327,6 @@ While the results themselves can give a clear indication of both the fertility p
 of a given cycle is already over for any given day, there a variety of external factors that should be taken into
 consideration for a direct output to the user.
 
-\subsection{Limitations}\label{subsec:limiations}
-This analysis reports aggregate test-set metrics without uncertainty quantification.
-Because we did not retain per-user predictions, we cannot compute user-level bootstrap confidence intervals or perform paired significance testing.
-As a result, apparent performance differences, especially small ones, may reflect sampling variability.
-Future re-evaluation that stores per-user predictions will enable user-level bootstrapping, calibration assessment, and formal comparisons.
-
-\subsection{Future Work}\label{sec:future_work}
-There are several directions in which this study could be extended,
-most of which were omitted due to time and resource constraints but represent valuable areas for future exploration.
-
-One major area is feature selection.
-The dataset used includes additional user-entered markers such as physiological signs (e.g., bleeding, illness, stress)
-and external events (e.g., intercourse, pregnancy tests).
-These markers were not included in the present analysis but may carry predictive value and could meaningfully improve model performance.
-Similarly, the introduction of engineered or intermediate features, derived from raw inputs, may help models better
-capture relevant patterns and temporal dependencies.
-Additionally, the target features could be modelled in a better way, as, especially for long cycles,
-there is a large imbalance of value distribution.
-If a cycle has a length of 100 days with an ovulation at day 90, only 10\% of the ovulation-over targets are one.
-The same applies to the fertility target, which will be zero throughout almost the whole sequence,
-which will make it harder for the models to learn useful information.
-
-Alternative target formulations could also be explored to better reflect the structure of the fertile window and ovulation.
-For example, instead of predicting a daily fertility probability, models could aim to identify the absolute day of ovulation,
-or estimate the time until the next (or since the last) ovulation event.
-Additionally, multiple models could be trained individually for each target, to avoid confusing the models
-on two different prediction tasks.
-While the fertility-probability and the timing of the ovulation have an inherent causal relationship,
-there might be anomalies in the data that confuse one or both targets, and independent predictions might perform better.
-
-
-As noted in Section~\ref{subsubsec:data_labeling}, ovulation labels were assigned using a
-retrospective algorithm trained on expert-annotated data.
-Any inaccuracies in this algorithm propagate directly to the supervised learning labels.
-Thus, improving prediction quality may require a newly labeled dataset—ideally combining expert review with algorithmic assistance.
-
-Finally, while the models used in this study (LSTMs and Transformers) are well-established for time-series analysis,
-they were not extensively customized.
-Future work could involve tailoring architectures more specifically to the characteristics of menstrual cycle data.
-Recent transformer variants designed for time series, such as \emph{TimeXer}~\cite{wang_timexer_2024} or \emph{MAMBA}~\cite{wang_is_2024},
-may offer improved performance.
-Alternatively, a custom architecture could be developed to better reflect the domain-specific structure of biological temperature data.
-
-Future work may incorporate more advanced hyperparameter optimization techniques,
-such as Bayesian Optimization, Genetic Algorithms, or Neural Architecture Search (NAS),
-to better explore the joint parameter space in a more efficient and principled manner.
-
 
 % talk about whether bbt / temperature can be used for such a task, discuss bbt doubt papers
 % While previous work has argued against the predictive value of BBT~\cite{some_author_2010}, our findings suggest otherwise.

thesis/sections/introduction.tex

@@ -8,17 +8,15 @@ While textbooks often describe a menstrual cycle as lasting 28 to 30 days with o
 such regularity is the exception rather than the rule~\cite{munster_length_1992, bull_real-world_2019}.
 For individuals with consistent cycle patterns, simple calendar-based predictions may suffice.
 However, for the majority, especially with increasing age and associated irregularity, more sophisticated methods are necessary.
-This, combined with an ever higher age of pregnancy in industrialized and industrializing countries,
-underscores the growing need for accurate understanding of the menstrual cycle~\cite{sauer_reproduction_2015}.
+This, combined with the rising maternal age in industrialized and industrializing countries,
+underscores the growing demand for accurate, individualized menstrual cycle prediction methods~\cite{sauer_reproduction_2015}.
 
-For many women, the practical use cases of menstrual cycle monitoring are \emph{Natural Family Planning} (NFP) and contraception~\cite{earle_use_2021}.
+For many individuals, the practical use cases of menstrual cycle monitoring are \emph{Natural Family Planning} (NFP) and contraception~\cite{earle_use_2021}.
 For these use cases, it is essential to identify the ovulation and its corresponding fertile and infertile days in a cycle,
 to either avoid or achieve pregnancy more effectively.
 
-Ovulation is the process in which an egg cell is released from the ovaries, making fertilization possible.
-This process is regulated by hormonal changes, including fluctuations in luteinizing hormone (LH) and
-follicle-stimulating hormone (FSH), and is accompanied by other physiological changes such as an increase in electrical resistance
-and viscosity of the cervical mucus or an increase in body temperature~\cite{wallach_prediction_1980}.
+Ovulation, the release of an egg cell from the ovaries, is triggered by hormonal changes and accompanied
+by physiological shifts such as changes in cervical mucus and a rise in body temperature~\cite{wallach_prediction_1980}.
 These processes remain incompletely understood and are influenced by lifestyle factors such as stress, diet, exercise,
 or health-related conditions such as Polycystic Ovary Syndrome (PCOS), making ovulation difficult to predict.
 
@@ -49,10 +47,17 @@ NFP and contraception.
 Finally, we demonstrate that high predictive accuracy on highly regular, curated datasets, as commonly reported in prior work,
 may overestimate real-world applicability, since such datasets tend to favor even simple, rule-based approaches.
 
+The main research objective is to evaluate the fundamental feasibility of predicting fertility and
+ovulation from body temperature data using machine learning,
+with a focus on how prediction performance varies across menstrual cycle types and real-world use cases.
 
-The research objectives are:
+These objectives support the broader question of whether temperature-based models can provide robust predictions
+across real-world variability in cycle patterns and user needs:
 \begin{itemize}
-    \item To evaluate the predictive value of body temperature for ovulation and fertility across diverse menstrual cycle types.
-    \item To assess the performance of different model architectures on the use cases of natural family planning and contraception, across both regular and irregular cycles.
-    \item To compare sophisticated machine learning models with simple rule-based baseline approaches.
+    \item To train and evaluate a set of machine learning models for ovulation and fertility prediction,
+    using standard performance metrics such as MAE and MSE\@.
+    \item To identify influential factors and patterns that affect the prediction.
+    \item To compare prediction performance across regular and irregular cycles to assess how cycle variability affects feasibility.
+    \item To evaluate model outputs in the context of practical use cases,
+    such as contraception and natural family planning, using task-specific evaluation criteria.
 \end{itemize}

thesis/sections/methodology.tex

@@ -10,6 +10,8 @@ To address these limitations, we develop a data-driven framework based on a larg
 heterogeneous dataset of real-world menstrual cycles.
 Our approach emphasizes model transparency, adaptability to irregular patterns, and the predictive utility of
 high-resolution core body temperature measurements.
+The overall goal is to train a model to predict the fertility-probability and information about the ovulation for
+a given day, only relying on past information prior to that day.
 This section outlines the methodology used, including preprocessing, labeling, feature extraction, and model architectures.
 
 \subsection{Data Preprocessing}\label{subsec:data_preprocessing}
@@ -64,15 +66,15 @@ The next section details how these labels are incorporated into feature represen
 The features used as model inputs have been divided into three categories:
 \begin{itemize}
     \item \textbf{Static features} - Characteristics, that remain constant across a user's cycle, such as age, height, or average ovulation day
-    \item \textbf{Known features} — Inputs known a priori at each time step, such as time of day or calendar-based variables.
-    \item \textbf{Observable features} — Inputs available at the current time step, including raw and derived temperature values.
+    \item \textbf{Known features} — Inputs known a priori at each time step, such as time of day or calendar-based variables (e.g., month of the year).
+    \item \textbf{Observable features} — Inputs available at the current time step, including raw and derived temperature values (e.g, rolling averages).
 \end{itemize}
 The target variables predicted by the model—like ovulation status or fertility probability—are described separately.
 
 Each feature type can handle categorical and continuous features.
 This allows for mixed inputs, such as scalar measurements and class labels, within the same category.
 
-The categorization into four feature types is intended to clarify the conceptual roles of different input types.
+The categorization into three feature types is intended to clarify the conceptual roles of different input types.
 While the current models concatenate all features into a single input stream, the distinction allows for flexibility—future models
 may process each feature group differently depending on their architectural design.
 
@@ -106,11 +108,12 @@ Static features provide user-specific context that helps the model learn individ
 %\end{table}
 %Table~\ref{tab:static_features} shows all static features and their descriptions.
 
-Prior research by \citeauthor{li_menstrual_2023} has shown that menstrual cycle characteristics vary significantly with age and BMI~\cite{li_menstrual_2023}.
+Menstrual cycle characteristics vary significantly with age and BMI~\cite{li_menstrual_2023}.
 Including such information is therefore expected to improve predictive performance.
 
-In addition, summary statistics from previous cycles—such as ovulation timing, temperature levels, or the fraction of ovulatory cycles—provide useful individual context.
+In addition, summary statistics from previous cycles, such as ovulation timing, temperature levels, or the fraction of ovulatory cycles, provide useful individual context.
 These features help the model learn subject-specific variability and better estimate the likelihood and timing of ovulation in the current cycle.
+If no user-specific context is available yet, population-based mean values are used.
 Table~\ref{tab:feature_overview} shows the full list of static input features.
 
 All historical features are computed using only data available prior to the current cycle, ensuring no data leakage and supporting robust, user-adaptive learning.
@@ -141,7 +144,7 @@ The cyclical nature of the variable is clearly visible in the transformation.
 Although the model architectures used are sequential, the explicit inclusion of these time features allows the models to interpret each time step in a broader context.
 More importantly, they enable the detection of gaps in the recording, which would otherwise not be visible from the data alone.
 
-Additionally, prior research has shown that the menstrual cycle may be influenced by weekly rhythms~\cite{ecochard_menstrual_2024}.
+Menstrual cycles can be influenced by weekly rhythms~\cite{ecochard_menstrual_2024}.
 For example, menstruation has been found to begin more frequently on Thursdays or Fridays, suggesting that behavioral or social factors may modulate certain events in the cycle.
 Including this information could therefore improve the predictive quality of the models.
 
@@ -242,20 +245,44 @@ The model outputs represent data-driven estimates and do not constitute medical
     \label{tab:feature_overview}
 \end{table}
 
+\subsubsection{Time-Series Input Representation}
+\label{subsubsec:time_series_input_representation}
 
 All features were normalized based on their empirical distributions.
 A \textit{standard scaler} was applied to approximately normal features without outliers,
 a \textit{robust scaler} was used for distributions with outliers, and a \textit{MinMax scaler} was used for all others.
 Different Scalers were used for the train, validation and test sets to avoid data leakage.
 
-For the final data matrix, all features were stacked per timestep.
-The static features were repeated for each timestep.
+For the final data matrix, all features are stacked per timestep.
+The static features are repeated for each timestep.
 We are aware of possible inefficiencies here.
 A side channel for static features might improve predictive efficiency and potential quality, but
 this was left out to keep the interfaces the same for compatibility purposes between all tested models
 
-\subsubsection{Time-Series Input Representation}
-\label{subsubsec:time_series_input_representation}
+Let
+\begin{itemize}
+    \item \( T \) be the sequence length (number of time steps)
+    \item \( x_t^{\text{obs}} \in \mathbb{R}^{d_{\text{obs}}} \): observable features at time \( t \)
+    \item \( x_t^{\text{known}} \in \mathbb{R}^{d_{\text{known}}} \): known features at time \( t \)
+    \item \( x^{\text{static}} \in \mathbb{R}^{d_{\text{static}}} \): static features (repeated across time steps)
+\end{itemize}
+
+Then each input token at time \( t \in \{1, \dots, T\} \) is:
+\[
+    x_t = \left[ x_t^{\text{obs}} \;\middle|\; x_t^{\text{known}} \;\middle|\; x^{\text{static}} \right]
+    \in \mathbb{R}^{d_{\text{obs}} + d_{\text{known}} + d_{\text{static}}}
+\]
+
+The full input sequence is then represented as a matrix:
+\[
+    X = \begin{bmatrix}
+            x_1 \\
+            x_2 \\
+            \vdots \\
+            x_T
+    \end{bmatrix}
+    \in \mathbb{R}^{T \times (d_{\text{obs}} + d_{\text{known}} + d_{\text{static}})}
+\]
 
 The high temporal resolution of the temperature data, 288 measurements per day, results in very long input sequences
 that are impractical for most deep learning models to process directly.
@@ -269,9 +296,6 @@ where values from distinct categories (e.g., hours 23 and 0) might otherwise be
 
 The effect of different sampling resolutions and aggregation strategies is evaluated in Section~\ref{sec:results}.
 
-To simulate real-time prediction rather than retrospective analysis, a sliding-window approach is employed.
-This allows the model to make predictions based only on data available up to a specific point in the cycle.
-
 To simulate real-time prediction, each cycle is split into overlapping,
 fixed-length input windows that capture all available data up to a given time step.
 As the cycle progresses, these windows slide forward, allowing the model to update its prediction based on growing historical context.
@@ -318,6 +342,9 @@ Ultimately, the models used in this study were selected based on their ability t
     \item Generalize across users while incorporating personalized cycle context
 \end{itemize}
 
+In this section we will introduce the base architectures \textbf{Long-Short-Term-Memory Models} and \textbf{Transformer Models}.
+Additionally, we'll show, how the convolutional hybrids extend their functionality.
+
 \subsubsection{LSTM-Architecture}\label{subsubsec:lstm_architecture}
 \begin{figure}[htbp]
     \centering
@@ -373,7 +400,7 @@ Since the task does not require sequence-to-sequence modeling, only the encoder
 Its output—one vector per input token—is aggregated via 1D adaptive average pooling, resulting in a single vector representation per sequence.
 This vector is then passed through a linear projection layer to produce the two target outputs:
 fertility probability and ovulation-over indicator.
-Note, that in contrast to the original use case of machine-translation, not special tokens are necessary here, as we do not perform sequence-to-sequence prediction.
+Note, that in contrast to the original use case of machine-translation, no special tokens are necessary here, as we do not perform sequence-to-sequence prediction.
 
 Figure~\ref{fig:methodology_transformer_architecture} shows the overall architecture.
 The stacked inputs and outputs indicate batch processing.
@@ -434,11 +461,20 @@ The models were trained using a configurable framework developed specifically fo
 allowing for flexible experimentation with different architectures, input feature sets, and
 hyperparameter configurations.
 
-The training process is organized into distinct \textit{runs}, each representing a set of model experiments with a shared base configuration.
-Within a run, variable parameters—such as input sequence length, hidden layer size, dropout rate,
-or specific feature subsets—are systematically swept across predefined value ranges.
+The training process is organized into distinct \textit{runs}, each representing a set of model experiments
+(e.g., a sweep over possible input sequence lengths), with a shared base configuration (e.g., fixed input resolution for the input sequence length sweep).
+Within a run, variable parameters, such as input sequence length, hidden layer size, dropout rate,
+or specific feature subsets, are swept across predefined value ranges.
 
-For each combination of parameters, a dedicated training and evaluation procedure is performed.
+We define 3 runs for each base architecture and 2 for each convolutional architecture.
+\begin{itemize}
+    \item \textbf{Input-Sequence-Length Run:} explores different input sequence lengths
+    \item \textbf{Input Resolution Run:} explores different input resolutions (only for base architectures)
+    \item \textbf{Model Parameter Run:} explores different model hyperparameters, e.g.,  hidden layer size
+\end{itemize}
+
+For each combination of parameters (e.g., 20 day input sequence length, resolution of 12 values per day),
+a dedicated training and evaluation procedure is performed.
 This structure supports efficient hyperparameter exploration and ensures consistent, reproducible
 training conditions across models.
 
@@ -576,7 +612,8 @@ Each model was trained for up to 30 epochs, with early stopping based on validat
 
 To meaningfully compare model performance, we define a set of metrics according to the research objectives,
 that capture both overall accuracy and behavior at key points in the prediction sequence as well as cover the use cases introduced in~\ref{subsubsec:practical_use_cases}.
-This includes metrics for different temporal segments, enabling a more detailed understanding of model strengths and limitations.
+This includes metrics for different temporal segments, such as before and after the ovulation,
+enabling a more detailed understanding of model strengths and limitations.
 
 \subsubsection{Baseline Comparisons}\label{subsubsec:baseline_comparisons}
 
@@ -595,30 +632,32 @@ In many cases, it remains unclear whether proposed models genuinely outperform s
 
 \subsubsection{Evaluation Metrics}\label{subsubsec:evaluation_metrics}
 
-The base metric used for all categories is the mean absolute error (MAE),
-which describes the average absolute deviation of the prediction from the target value,
+The base metric used for all categories is the mean squared error (MAE),
+which describes the average squared deviation of the prediction from the target value,
 and is defined as:
+\begin{align}
+    \text{MSE} = \frac{1}{n} \sum_{i=1}^{n} (y_i - \hat{y}_i)^2
+\end{align}
 
+
+where \(y_i\) is the observed value and \(\hat{y}_i\) the predicted value.
+
+MSE penalizes larger errors more heavily, making it useful for highlighting substantial deviations.
+This is particularly relevant for model comparison, where disproportionate errors can skew performance.
+Moreover, since the fertility probability target was trained using an MSE-based loss function,
+this metric directly reflects the optimization objective.
+
+To complement this, we also report the mean squared error (MAE):
 \begin{align}
     \text{MAE} = \frac{1}{n} \sum_{i=1}^{n} \left| y_i - \hat{y}_i \right|
 \end{align}
-
 where \(y_i\) is the observed value and \(\hat{y}_i\) the predicted value.
 
 MAE was selected for its intuitive interpretability.
 In particular, the fertility probability target lends itself well to an absolute error interpretation,
 making MAE a natural choice for evaluating prediction accuracy.
-
-To complement this, we also report the mean squared error (MSE):
-\begin{align}
-    \text{MSE} = \frac{1}{n} \sum_{i=1}^{n} (y_i - \hat{y}_i)^2
-\end{align}
-where \(y_i\) is the observed value and \(\hat{y}_i\) the predicted value.
-
-MSE penalizes larger errors more heavily than MAE, making it useful for highlighting substantial deviations.
-This is particularly relevant for model comparison, where disproportionate errors can skew performance.
-Moreover, since the fertility probability target was trained using an MSE-based loss function,
-this metric directly reflects the optimization objective.
+However, we will only be using the MAE as a secondary metric, as we will base our further interpretation of
+the model performances on the use-case evaluations, that provide inherent real-world interpretability.
 
 We considered including the coefficient of determination (\(R^2\)) as an evaluation metric.
 However, we found that the target windows frequently exhibited very low variance,
@@ -652,20 +691,20 @@ Tables~\ref{tab:fertility_mae_metrics} and~\ref{tab:ov_over_mae_metrics} summari
         \toprule
         \textbf{Metric Name} & \textbf{Description}          \\
         \midrule
-        \multicolumn{2}{@{}l}{\textbf{Mean Absolute Error}} \\
-        \midrule
-        Fertility Overall    & MAE over the entire sequence. \\
-        During-Fertility     & MAE during the fertile phase. \\
-        Non-Fertility        & MAE on the non-fertile days.  \\
-        \midrule
         \multicolumn{2}{@{}l}{\textbf{Mean Squared Error}} \\
         \midrule
         Fertility Overall    & MSE over the entire sequence. \\
         During-Fertility     & MSE during the fertile phase. \\
         Non-Fertility        & MSE on the non-fertile days.  \\
+        \midrule
+        \multicolumn{2}{@{}l}{\textbf{Mean Absolute Error}} \\
+        \midrule
+        Fertility Overall    & MAE over the entire sequence. \\
+        During-Fertility     & MAE during the fertile phase. \\
+        Non-Fertility        & MAE on the non-fertile days.  \\
         \bottomrule
     \end{tabular}
-    \caption{Evaluation metrics of the fertility probability target based on mean absolute error (MAE) and mean squared error (MSE) at various intervals across the predicted fertility window.}
+    \caption{Evaluation metrics of the fertility probability target based on mean squared error (MSE) and mean absolute error (MAE) at various intervals across the predicted fertility window.}
     \label{tab:fertility_mae_metrics}
 \end{table}
 
@@ -677,20 +716,20 @@ Tables~\ref{tab:fertility_mae_metrics} and~\ref{tab:ov_over_mae_metrics} summari
         \toprule
         \textbf{Metric Name} & \textbf{Description}          \\
         \midrule
-        \multicolumn{2}{@{}l}{\textbf{Mean Absolute Error}} \\
-        \midrule
-        OV-Over Overall      & MAE over the entire sequence. \\
-        Pre-OV               & MAE before the ovulation.     \\
-        Post-OV              & MAE after the ovulation.      \\
-        \midrule
         \multicolumn{2}{@{}l}{\textbf{Mean Squared Error}} \\
         \midrule
         OV-Over Overall      & MSE over the entire sequence. \\
         Pre-OV               & MSE before the ovulation.     \\
         Post-OV              & MSE after the ovulation.      \\
+        \midrule
+        \multicolumn{2}{@{}l}{\textbf{Mean Absolute Error}} \\
+        \midrule
+        OV-Over Overall      & MAE over the entire sequence. \\
+        Pre-OV               & MAE before the ovulation.     \\
+        Post-OV              & MAE after the ovulation.      \\
         \bottomrule
     \end{tabular}
-    \caption{Evaluation metrics of the ovulation-over target based on mean absolute error (MAE) and mean squared error (MSE) at various intervals across the predicted fertility window.}
+    \caption{Evaluation metrics of the ovulation-over target based on mean squared error (MSE) and mean absolute error (MAE) at various intervals across the predicted fertility window.}
     \label{tab:ov_over_mae_metrics}
 \end{table}
 
@@ -701,8 +740,9 @@ This step was necessary to keep the computational effort manageable,
 as exhaustively testing all possible configurations for every subsequent metric would have been prohibitively expensive.
 
 Selection was based primarily on the \textbf{Fertility-Overall MSE} metric,
-as it most directly reflects the main objective of this study: predicting fertility.
-In cases where the difference between configurations was small,
+as it most directly reflects the main objective of this study: predicting fertility,
+and we want to penalize larger errors more, as they are much more problematic for our use-case scenarios.
+In cases where the difference between configurations was small (less than \(\pm\) 5\% metric value),
 we preferred the option that aligned with the general tendency of the model architecture.
 For example, if an architecture consistently performed better with more input data or longer sequences,
 but the Fertility-Overall MSE was only marginally better for a shorter window, we selected the longer window.

thesis/sections/related_work.tex

@@ -4,6 +4,12 @@
 
 \section{Related Work}\label{sec:related_work}
 
+This section will introduce related work of both ovulation detection and ovulation prediction.
+First, we'll introduce early work on the detection of the ovulation based on biomarkers.
+Then, we'll show how others have used body temperature to predict ovulation and what their limitations are.
+Lastly, we will take a closer look at related work that uses other biomarkers as base, or as an addition to the body
+temperature for ovulation and fertility prediction.
+
 A variety of approaches have historically been explored for ovulation detection and prediction,
 ranging from hormonal assays to physiological signal tracking.
 
@@ -73,7 +79,8 @@ For menstruation prediction, the model detected 70.70\% of menstruation days in
 These results indicate that the model performed well in regular cycles but struggled with irregularity, particularly in detecting the fertile window.
 
 Complementing academic efforts, several commercial products have adopted temperature-based tracking,
-such as \textit{Ava}~\cite{sl_ava_nodate}, \textit{Daysy}~\cite{electronics_zykluscomputer_nodate} or \textit{Trackle}~\cite{noauthor_trackle_nodate}.
+such as \textit{Ava}~\cite{sl_ava_nodate}, \textit{Daysy}~\cite{electronics_zykluscomputer_nodate}, \textit{Trackle}~\cite{noauthor_trackle_nodate}
+or \textit{Natural Cycles}~\cite{noauthor_natural_nodate}.
 However, these products typically rely on proprietary algorithms, and no peer-reviewed publications are available detailing their methodology or performance.
 This lack of transparency limits their scientific evaluation and comparability.
 In contrast, the present study provides an open and data-driven approach to ovulation prediction based on continuous temperature data, aiming to contribute reproducible evidence to the field.
@@ -81,7 +88,7 @@ In contrast, the present study provides an open and data-driven approach to ovul
 \subsection{Other Physiological Signals}\label{subsec:other_physiolocical_signals}
 In addition to temperature, other physiological signals have been explored for ovulation and cycle phase prediction.
 
-As early as \citeyear{moreno_temporal_1988}, researchers investigated ovulation prediction based on the electrical resistance of salivary and vaginal secretions~\cite{moreno_temporal_1988}.
+As early as~\citeyear{moreno_temporal_1988}, researchers investigated ovulation prediction based on the electrical resistance of salivary and vaginal secretions~\cite{moreno_temporal_1988}.
 Their study analyzed 29 cycles from 11 women, with daily recordings of BBT, urinary LH, pelvic ultrasound, and ovulation predictor kit results.
 Participants were under the age of 35, had cycle lengths between 25 and 35 days, and had abstained from hormone therapies for at least two months prior to the study.
 
@@ -92,7 +99,7 @@ These findings suggest that electrical resistance is a strong physiological mark
 A related modern implementation is the commercial product \textit{kegg}~\cite{noauthor_kegg_nodate}, which measures the electrical resistance of cervical mucus.
 The device uses an undisclosed algorithm to estimate fertility status based on these readings, although no peer-reviewed validation studies are currently available.
 
-\citeauthor{masuda_machine_2025} developed a machine learning algorithm to classify phases of the menstrual cycle
+In~\citeyear{masuda_machine_2025}, \citeauthor{masuda_machine_2025} developed a machine learning algorithm to classify phases of the menstrual cycle
 (follicular vs. luteal) based on sleeping heart rate, as recorded by a fitness tracker~\cite{masuda_machine_2025}.
 They used an XGBoost classifier for this binary task and additionally performed ovulation day prediction,
 although the details of this task were not fully specified.
@@ -119,7 +126,7 @@ waking—they report classification accuracies between 0.843 and 0.864, dependin
 with very similar numbers for precision, recall, specificity and F1 score.
 Ovulation day prediction yielded an average absolute error between 3.6 and 4.1 days.
 
-\paragraph{Summary:}
+\subsection{Summary}\label{subsec:related_work_summary}
 While various physiological signals and modeling strategies have been explored for ovulation prediction,
 many existing studies are limited by small, highly selective datasets, assumptions of cycle regularity, or reliance on proprietary algorithms.
 The present work extends prior approaches by leveraging a large, heterogeneous dataset of real-world cycles and applying transparent,

thesis/sections/results.tex

@@ -3,19 +3,18 @@
 
 
 \section{Results}\label{sec:results}
-We report results for two targets: (i) fertility probability and (ii) an indicator that ovulation has already occurred (OV-Over).
-Unless stated otherwise, \textbf{MSE} is the primary metric (equivalent to the Brier score for probabilistic targets),
-and \textbf{MAE} is secondary.
-We first summarize overall performance across architectures, then analyze sensitivity to input window length,
-input resolution, and model capacity.
-Finally, we compare the best configurations to three baselines and present stratified and use-case analyses.
 
-\subsection{Overall Model Performance Across Architectures}\label{subsec:overall_model_performance_across_architectures}
+% find introduction
+
+\subsection{RQ1: Comparative Study of Model Architectures}\label{subsec:rq1_temp_predictive_value}
 
 We systematically evaluated multiple model architectures to assess their performance in
-predicting fertility probability and ovulation-over targets.
+predicting the targets.
+We report results for two targets: (i) fertility probability and (ii) an indicator that ovulation has already occurred (OV-Over).
 Each architecture was tested across a range of input window lengths, temporal resolutions, and model capacities.
 The best-performing configurations for each architecture were selected for downstream analyses.
+Unless stated otherwise, \textbf{MSE} is the primary metric (equivalent to the Brier score for probabilistic targets),
+and \textbf{MAE} is secondary.
 
 All metrics reported in this section are point estimates without confidence intervals; therefore, comparisons are descriptive.
 
@@ -124,7 +123,7 @@ Results are summarized in Table~\ref{tab:fertility_resolution_compact_mse}; full
 \paragraph{Impact of model parameters.}
 
 We next evaluate how architectural complexity, i.e., hidden size, number of layers,
-and number of attention heads—influences performance under fixed input settings (160 days at 12/day for LSTM/Transformer, 40 days for convolutional models).
+and number of attention heads, influences performance under fixed input settings (160 days at 12/day for LSTM/Transformer, 40 days for convolutional models).
 Parameters are reported as (hidden size × number of layers) for LSTM models and (embedding size × number of encoders × attention heads) for Transformers.
 
 For the \textbf{LSTM}, the best overall and fertile-day MSEs are both observed at a configuration with 512 hidden units and 4 layers.
@@ -194,9 +193,6 @@ The \textbf{LSTM} performs best before ovulation (MSE 0.0212 at 20 days), while
 The \textbf{Convolutional LSTM} favors short windows, with the best overall MSE (0.0699), before-OV (0.0389),
 and after-OV (0.0833) all occurring at 20 days.
 The \textbf{Convolutional Transformer} performs best overall at 40 days and best before ovulation at 160 days (MSE 0.0286).
-
-These trends suggest that intermediate windows (20–40 days) often balance short- and long-term signal,
-while long windows (e.g., 160 days) help capture post-ovulation patterns.
 See Table~\ref{tab:ovover_windows_compact_mse} for a summary.
 
 \begin{table}[t]
@@ -242,7 +238,6 @@ with best before-OV and after-OV MSEs at 12/day (0.0212) and 48/day (0.0550), re
 
 The \textbf{Transformer} performs best overall at 72/day (0.0585),
 with lowest before-OV MSE at 12/day (0.0255) and after-OV MSE at 288/day (0.0578).
-This indicates that high resolution benefits post-ovulation prediction, while lower rates suffice for pre-ovulation.
 
 Results are summarized in Table~\ref{tab:ovover_resolution_compact_mse};
 full resolution grids are in Appendix Table~\ref{tab:ov_over_results_by_resolution}.
@@ -334,7 +329,8 @@ while also considering the general performance trends of each model.
 Tables~\ref{tab:best_configs_lstm} and~\ref{tab:best_configs_transformer} summarize the selected hyperparameters for each architecture,
 including input window length, temporal resolution, and model complexity.
 For convolutional models, the input resolution was always fixed at the full 288 measurements per day.
-
+For the Transformer architecture, we made the decision to use longer input windows, as the performance on longer
+input windows was competitive to the best measured at 40 days, but includes much more context information.
 
 \begin{table}[htbp]
     \centering
@@ -389,9 +385,9 @@ All trained architectures outperform the baseline models across all evaluation m
         & Fertility Overall           & Fertile Days                & Non-Fertile Days            & OV-Over Overall           & OV-Over Before OV & OV-Over After OV \\
         \midrule
         Transformer                 & 0.0037          & \textbf{0.0089} &  0.0017 &          0.0508 &          0.0236 &          0.0581 \\
-        Convolutional Transformer   & 0.0038          & 0.0098 &           0.0014 &          0.0517 &          0.0305 &          0.0566 \\
         LSTM                        & \textbf{0.0036} & 0.0104 &           \textbf{0.0008} & \textbf{0.0437} & \textbf{0.0233} & \textbf{0.0492} \\
         Convolutional LSTM          & 0.0037          & 0.0092 &           0.0014 &          0.0499 &          0.0281 &          0.0563 \\
+        Convolutional Transformer   & 0.0038          & 0.0098 &           0.0014 &          0.0517 &          0.0305 &          0.0566 \\
         User-Based-Mean Baseline    & 0.0064          & 0.0154 &           0.0028 &          0.1072 &          0.0872 &          0.0983 \\
         Last-Cycle Baseline         & 0.0080          & 0.0202 &           0.0031 &          0.1453 &          0.1099 &          0.1762 \\
         Population-Mean Baseline    & 0.0127          & 0.0258 &           0.0074 &          0.2145 &          0.0400 &          0.3749 \\
@@ -410,9 +406,89 @@ Compared to the strongest baseline (User-Mean), the LSTM reduces the overall fer
 The performance gap is even larger when compared to the population-mean and last-cycle baselines,
 confirming the advantage of personalized and temporally-aware modeling.
 
-\subsection{Stratified Analysis}\label{subsec:stratified_analysis}
+% -------------------------------------------------------------------------------------------------------------------------------
+
+\subsection{RQ2: Factors and Patterns that Influence Predictions}\label{subsec:rq2_factors_and_patterns}
+
+\subsubsection{Representative predictions}\label{subsubsec:rq2_examples}
+
+All following prediction were made with the best model for each architecture as selected in the previous section.
+
+\begin{figure}[htbp]
+    \centering
+    \includegraphics[width=1.0\textwidth]{resources/figures/results/regular_cycle_pattern_fertility_prediction_example}
+    \caption{
+        Temperature rolling average and fertility-probability prediction for a user with a regular cycle pattern. (Values are scaled features)
+    }
+    \label{fig:results_rq2_regular_cycle_predictions_example}
+\end{figure}
+
+To visualize potential patterns in the predictions, we will show some representative prediction plots.
+
+All models have a similar predictive behaviour throughout both the regular and irregular cycles.
+Figure~\ref{fig:results_rq2_regular_cycle_predictions_example} shows, that all models come very close to the ground truth
+for a regular cycle pattern.
+Slight deviations from the regular cycle length, such as in cycle 5 (Measurement 150--220) show, that the models
+tend to overestimate the fertility-probability in such cases.
+Additionally, all models never reach the full range of fertility-probability as indicated by the ground-truth.
+
+These predictions show a clear correlation with a temperature drop before the ovulation.
+
+\begin{figure}[htbp]
+    \centering
+    \includegraphics[width=1.0\textwidth]{resources/figures/results/temperature_drop_pattern}
+    \caption{
+        Temperature rolling average and fertility-probability prediction showing a correlation between fertility and
+        the pre-ovulator temperature drop.
+    }
+    \label{fig:results_temperature_drop_pattern}
+\end{figure}
+
+This correlation between a pre-ovulator temperature drop and the fertility can be seen more pronounced in Figure~\ref{fig:results_temperature_drop_pattern}.
+The ground-truth fertility seems to be centered around this temperature drop and all models seem to be able
+to pick it up, if it is clearly distinguishable and visible in the data.
+For regular cycles, this temperature drop seems to be much easier distinguishable,
+as there is not as much temperature variability throughout the first cycle phase.
+Additionally, there also seems to be a correlation between the height of the temperature drop and the corresponding
+fertility-probability prediction by the models.
+The second cycle (Measurement 35--90) has a smaller temperature drop and all models predict a smaller fertility-probability
+as for the following two cycles (Measurements 90--145).
+This association of temperature-drop height and fertility-probability can also be seen in Figure~\ref{fig:results_rq2_regular_cycle_predictions_example}.
+The first two cycles have a smaller temperature drop compared to the following cycles and also a less pronounced prediction.
+
+Figure~\ref{fig:results_rq2_irregular_cycle_predictions_example} shows an irregular cycle pattern
+and the fertility-probability predictions for it.
+For irregular cycles, all models show much more struggle in determining the ground-truth fertility.
+The pre-ovulator temperature drop is far less distinguishable.
+The fifth cycle (measurement 450--700) shows a much more gradual temperature drop and all model's predictions
+are far off target.
+The previous 2 cycles (cycle 3 and 4) are shorter and have a much more distinguishable temperature drop
+and thus all models show indication for a detection of the fertile phase.
+
+\begin{figure}[htbp]
+    \centering
+    \includegraphics[width=1.0\textwidth]{resources/figures/results/irregular_cycle_pattern_fertility_prediction_example}
+    \caption{
+        Temperature rolling average and fertility-probability prediction for a user with an irregular cycle pattern. (Values are scaled features)
+    }
+    \label{fig:results_rq2_irregular_cycle_predictions_example}
+\end{figure}
+
+All models seem to be easily confused by anomalies during this characteristic temperature drop, even for short and regular cycles.
+Figure~\ref{fig:results_rq2_anomaly_in_temperature_drop} shows a cycle with a clear anomaly during the pre-ovulatory drop in cycle 3 (Measurements 90--125).
+This confuses all models into ending the fertile phase earlier, even though fertility is likely still elevated.
+
+\begin{figure}[htbp]
+    \centering
+    \includegraphics[width=1.0\textwidth]{resources/figures/results/spike_in_temperature_drop}
+    \caption{
+        Temperature rolling average and fertility-probability prediction for a user with an anomaly during the characteristic temperature drop. (Values are scaled features)
+    }
+    \label{fig:results_rq2_anomaly_in_temperature_drop}
+\end{figure}
+
+\subsubsection{Effect of user history depth}\label{subsubsec:rq2_history_depth}
 
-\subsubsection{Influence of User History Depth}\label{subsubsec:influence_of_past_user_data}
 \begin{figure}[htbp]
     \centering
     \includegraphics[width=0.8\textwidth]{resources/figures/results/performance_on_different_historical_contexts}
@@ -426,16 +502,22 @@ confirming the advantage of personalized and temporally-aware modeling.
 Figure~\ref{fig:results_performance_on_different_historical_context} shows the MSE for the fertility probability
 and OV-over targets as a function of the number of past cycles available per user, for all model architectures and baselines.
 
-All four models improve on both metrics as the amount of historical data increases.
+All four models improve slightly on both metrics as the amount of historical data increases.
 The LSTM-based models show a larger relative improvement with longer user history than the Transformer-based variants.
-The User-Mean Baseline also improves substantially, with the Last-Cycle Baseline showing a smaller but still notable gain.
-In contrast, the Population-Mean Baseline performs worse (i.e., MSE increases) as more historical cycles are included.
+
+All baselines seem produce worse prediction with larger historical contexts.
+The last-cycle and population-mean baselines show the largest decrease in performance,
+especially for the fertility-probability prediction target.
+The user-mean baselines offers largely unchanged fertility prediction performance,
+but worsens on the ov-over prediction with more historical cycles available.
 
 Across all models and baselines, the variability of results increases with greater history depth:
 results are more tightly clustered around the trend line with short histories,
 but show greater scatter for users with longer data records.
 
-\subsubsection{Regular vs Irregular Cycles}\label{subsubsec:regular_vs_irregular_cycles}
+% -------------------------------------------------------------
+
+\subsection{RQ3: Performance across Regular and Irregular Cycles}\label{subsec:rq3_regular_vs_irregular}
 
 \begin{table}[htbp]
     \centering
@@ -538,7 +620,6 @@ As before, all learned models outperform the baselines by a wide margin.
 
 The full table with MSE and MAE for all models can be found in the appendix, Table~\ref{tab:regular_vs_irregular_ov_over_results}.
 
-\vspace{1em}
 
 \begin{figure}[htbp]
     \centering
@@ -559,10 +640,12 @@ The full table with MSE and MAE for all models can be found in the appendix, Tab
 \end{figure}
 
 \paragraph{Evaluation on Historical Context Depth.}
-Figures~\ref{fig:results_performance_on_regular_cycles} and~\ref{fig:results_performance_on_irregular_cycles} visualize performance improvements with increasing numbers of past cycles.
+Figures~\ref{fig:results_performance_on_regular_cycles} and~\ref{fig:results_performance_on_irregular_cycles}
+visualize performance improvements with increasing numbers of past cycles.
 For regular cycles, all models except the population-mean baseline benefit from more historical data, showing consistent MSE reductions for both targets.
 Interestingly, the population-mean baseline performs worse as more cycles are added.
 The largest improvement can be seen for the last-cycle baseline and user-mean baseline models.
+The outliers in the last-cycle and user-mean baseline are MSE values of 0.0, which result in these vertical lines.
 
 In contrast, irregular cycles exhibit more variability and less performance gain with more cycles available.
 All trained models improve on both targets with more context.
@@ -571,255 +654,242 @@ In contrast, their MSE increases with more historical context for the fertility-
 The last-cycle baseline shows marginal performance improvement for the OV-over target and the same increase for the fertility-probability
 target as the other two baselines.
 
+The selected irregular cycle group has cycles with longer context, thus the irregular cycle history reaches up to 80 cycles,
+while regular cycle history only reaches up to 35.
+
 All models show increased output variability as the number of available past cycles grows,
 as indicated by the wider deviations from the trend lines.
 
-\subsection{Use-Case Evaluation Results}\label{subsec:use_case_evaluation_results}
+% -------------------------------------------------------------
+
+\subsection{RQ4: Use-Case Evaluations}\label{subsec:rq4_use_case_evaluations}
 
 We evaluated the use-case scenarios described in Section~\ref{subsubsec:practical_use_cases} using the algorithms
 in Section~\ref{subsubsec:use_case_evaluation} across multiple fertility thresholds.
 Each evaluation used the same test set of 100 users (100 user-years) and was repeated for 200 iterations
 to get statistically more meaningful results.
-we report means and 95\% confidence intervals (CIs).
+All results represent the means over all runs.
 
 \subsubsection{Contraception Use-Case Results}\label{subsubsec:use_case_contraception_results}
 
-\paragraph{Threshold 0.01 (Table~\ref{tab:results_contraception_use_case_0_01}).}
-At the strictest threshold of 0.01, Transformer and LSTM models achieve the lowest pregnancy rates, only 4.4 to 4.6
-on average, despite allowing approximately 3,200 intercourse events.
-This equates to about 1.4 pregnancies per 1,000 intercourse events, a strong result for contraceptive reliability.
-
-In contrast, convolutional variants (ConvLSTM and ConvTransformer) restrict intercourse events to around 1,150,
-leading to roughly 34–35 pregnancies, or about 30 per 1,000 events—far less efficient in terms of balance between access and protection.
-All baseline models perform substantially worse, with pregnancy counts exceeding 100 in all cases, confirming the value of personalized predictions.
-
-\begin{table}
+\begin{figure}[htbp]
     \centering
+    \includegraphics[width=0.9\textwidth]{resources/figures/results/contraception_use_case_results_by_fertility_threshold}
+    \caption{
+        Contraception use-case evaluation study results by fertility threshold.
+    }
+    \label{fig:results_contraception_use_case_results_by_fertility_threshold}
+\end{figure}
 
-    \begin{subtable}{\textwidth}
+Figure~\ref{fig:results_contraception_use_case_results_by_fertility_threshold} shows the outcomes of
+the contraception use-case across fertility thresholds for each model architecture, along with the baselines.
+The four evaluated metrics are: number of pregnancies, number of intercourse events,
+number of correct denials (i.e., appropriately flagged fertile days),
+and number of incorrect denials (i.e., fertile days incorrectly flagged as infertile).
+
+\paragraph{Pregnancies.}
+All trained models start with very low pregnancy rates (Transformer: 0, LSTM: 4)
+and then increase roughly linearly with the threshold,
+reaching between 99 (Convolutional Transformer) and 122 (Transformer) at the highest setting.
+The LSTM performs worst overall in this metric.
+The Transformer starts with the lowest pregnancy rate but rises steeply,
+ending with the highest number of pregnancies at threshold 0.1.
+By comparison, the baselines show much higher pregnancy rates throughout,
+starting between 103 (user-mean) and 150 (population-mean) and increasing by about 50 pregnancies over the range of thresholds.
+
+\paragraph{Intercourse Events.}
+All trained models except the LSTM begin with low values of about 1000 intercourse events.
+The LSTM starts much higher, around 3000, and increases sharply, converging toward approximately 6100.
+All trained model variants show curved growth resembling logarithmic convergence, approaching the same upper range.
+In contrast, the baselines start near 6000 events and converge toward the same levels as the trained models at higher thresholds.
+
+\paragraph{Correct Denials.}
+The number of correct denials decreases by about half across the tested thresholds.
+All trained models cluster closely, starting at around 1800 correct denials,
+with the Transformer and its convolutional variant slightly outperforming the LSTMs.
+The baselines start at substantially lower levels: 1012 (user-mean), 902 (last-cycle), and 737 (population-mean).
+
+\paragraph{Incorrect Denials.}
+For incorrect denials, the Transformer-based models (except LSTM) start at very high values near 5000
+but sharply decrease and converge between 500 and 1000.
+The LSTM shows the same convergence pattern but starts considerably lower at 2654.
+Baselines follow a different trend: the last-cycle and user-mean baselines decrease
+modestly by about 100 over the range (starting at 749 and 762, respectively),
+while the population-mean baseline instead decreases from 1102 to around 879.
+
+\begin{figure}[htbp]
     \centering
-        \scriptsize
-        \begin{tabularx}{\linewidth}{l*{5}{X}}
-            \toprule
-            Model                     & No. of Intercourse Events         & Pregnancies      & Correct Denials & Incorrect Denials \\
-            \midrule
-            Convolutional LSTM        & 1154 (1149-1158) &    34.9 (34.1-35.7) &     1615 (1611-1620) &       4963 (4954-4971) \\
-            Transformer               & 3200 (3194-3207) &       4.4 (4.1-4.7) &     1817 (1812-1822) &       2717 (2710-2724) \\
-            Convolutional Transformer & 1156 (1151-1160) &    34.3 (33.5-35.0) &     1613 (1608-1619) &       4974 (4965-4983) \\
-            LSTM                      & 3206 (3199-3213) &       4.6 (4.3-4.9) &     1832 (1827-1838) &       2711 (2705-2717) \\
-            Last-Cycle Baseline       & 6066 (6057-6075) & 127.5 (125.9-129.1) &        918 (915-922) &          762 (759-766) \\
-            Population-Mean Baseline  & 5874 (5865-5883) & 153.7 (151.9-155.5) &        749 (746-752) &       1121 (1116-1125) \\
-            User-Mean Baseline        & 5933 (5924-5942) & 105.8 (104.4-107.2) &     1030 (1026-1034) &          776 (772-779) \\
-            \bottomrule
-        \end{tabularx}
-        \caption{Contraception metrics at threshold \textbf{0.01} for all models.}
-        \label{tab:results_contraception_use_case_0_01}
-    \end{subtable}
+    \includegraphics[width=0.9\textwidth]{resources/figures/results/contraception_use_case_pregnancy_statistics}
+    \caption{
+        Contraception use-case results: pregnancies per 100 users per year and per 1000 intercourse events, stratified by fertility threshold.
+    }
+    \label{fig:results_contraception_use_case_pregnancy_statistics}
+\end{figure}
 
-    \vspace{1.5em}
+To facilitate comparison, Figure~\ref{fig:results_contraception_use_case_pregnancy_statistics} presents two
+normalized pregnancy metrics across fertility thresholds: the number of pregnancies per 100 users per year (only one potential pregnancy is counted per user),
+and the number of pregnancies per 1000 intercourse events.
 
-    \begin{subtable}{\textwidth}
-        \centering
-        \scriptsize
-        \begin{tabularx}{\linewidth}{l*{5}{X}}
-            \toprule
-            Model                     & No. of Intercourse Events         & Pregnancies      & Correct Denials & Incorrect Denials \\
-            \midrule
-            Convolutional LSTM        &        2509 (2502-2515) &    55.0 (54.0-56.0) &     1448 (1444-1452) &       3768 (3760-3775) \\
-            Transformer               &        3801 (3793-3810) &       8.2 (7.8-8.6) &     1768 (1763-1773) &       2162 (2157-2168) \\
-            Convolutional Transformer &        2510 (2503-2516) &    55.6 (54.5-56.6) &     1448 (1443-1453) &       3776 (3769-3783) \\
-            LSTM                      &        3801 (3794-3809) &       8.2 (7.8-8.7) &     1772 (1766-1777) &       2160 (2154-2165) \\
-            Last-Cycle Baseline       &        6106 (6096-6116) & 130.2 (128.7-131.7) &        888 (885-892) &          750 (746-753) \\
-            Population-Mean Baseline  &        5924 (5914-5933) & 156.1 (154.3-157.8) &        716 (713-720) &       1100 (1096-1104) \\
-            User-Mean Baseline        &        5988 (5978-5997) & 108.8 (107.3-110.3) &       998 (994-1002) &          764 (761-768) \\
-            \bottomrule
-        \end{tabularx}
-        \caption{Contraception metrics at threshold \textbf{0.05} for all models.}
-        \label{tab:results_contraception_use_case_0_05}
-    \end{subtable}
+Here, the difference between the models becomes clearer.
+The baselines start at 62 to 73 pregnancies per 100 users and become linearly worse over the threshold range.
 
-    \vspace{1.5em}
+The LSTM performs worse than all other trained models, except for the upper end of the threshold interval.
+The Transformer, while being the best model for small thresholds, becomes worse over growing values until being on par with
+the LSTM for the maximum tested value (0.1).
 
-    \begin{subtable}{\textwidth}
-        \centering
-        \scriptsize
-        \begin{tabularx}{\linewidth}{l*{5}{X}}
-            \toprule
-            Model                     & No. of Intercourse Events         & Pregnancies      & Correct Denials & Incorrect Denials \\
-            \midrule
-            Convolutional LSTM        &        3596 (3588-3603) &    75.2 (74.0-76.4) &     1292 (1288-1297) &       2847 (2840-2853) \\
-            Transformer               &        4226 (4217-4234) &    13.3 (12.8-13.7) &     1692 (1687-1696) &       1822 (1817-1827) \\
-            Convolutional Transformer &        3592 (3584-3599) &    73.8 (72.7-75.0) &     1296 (1292-1301) &       2846 (2840-2852) \\
-            LSTM                      &        4222 (4214-4230) &    13.3 (12.8-13.8) &     1694 (1688-1699) &       1818 (1812-1823) \\
-            Last-Cycle Baseline       &        6145 (6135-6155) & 134.8 (133.3-136.3) &        865 (861-868) &          744 (740-747) \\
-            Population-Mean Baseline  &        5960 (5951-5969) & 160.3 (158.5-162.1) &        694 (691-697) &       1088 (1084-1093) \\
-            User-Mean Baseline        &        6014 (6005-6022) & 111.8 (110.3-113.3) &        969 (965-973) &          758 (755-762) \\
-            \bottomrule
-        \end{tabularx}
-        \caption{Contraception metrics at threshold \textbf{0.10} for all models.}
-        \label{tab:results_contraception_use_case_0_10}
-    \end{subtable}
-
-    \caption{grouped contraception metrics at thresholds 0.01, 0.05, and 0.10. values are means over 200 iterations; 95\% confidence intervals in parentheses.}
-    \label{tab:results_contraception_grouped}
-\end{table}
-
-\paragraph{Threshold 0.05 (Table~\ref{tab:results_contraception_use_case_0_05}).}
-Increasing the threshold to 0.05 improves access: Transformer and LSTM models now permit approximately 3,800 intercourse
-events while still limiting pregnancies to around 8.2 (2.2 per 1,000).
-This represents a ~19\% increase in events over threshold 0.01, at the cost of a modest rise in pregnancies (+3.8 absolute, +86\% relative).
-
-Convolutional models also allow more events (~2,510) but continue to produce significantly more pregnancies (~55),
-yielding a less favorable risk-benefit profile.
-Baselines remain underperforming.
-
-\paragraph{Threshold 0.10 (Table~\ref{tab:results_contraception_use_case_0_10}).}
-A further increase to 0.10 raises Transformer/LSTM events to ~4,220, but also raises pregnancies to ~13.3 (3.1 per 1,000).
-This is a ~11\% gain in access compared to 0.05, but the pregnancy count increases by ~62\%.
-
-Meanwhile, convolutional models permit ~3,590 events and result in 74–75 pregnancies, indicating a consistent trade-off in favor of sequential models.
-
-\paragraph{Contraception—Recommended Threshold.}
-Threshold 0.05 achieves a favorable balance between access and effectiveness.
-Transformer and LSTM models perform best, allowing a relatively high number of intercourse events while keeping pregnancies low.
-This threshold offers the best compromise and is selected as the most promising setting for contraceptive use.
+\paragraph{Summary.}
+In the contraception use-case, the trained models achieve substantially lower pregnancy rates than the baselines,
+though at the cost of fewer intercourse opportunities and more denials, both correct and incorrect.
+This results in less pregnancies for both 100 users for a year and per 1000 intercourse events for all trained models.
+Within the trained models, Transformers and their convolutional variants perform best overall, while LSTMs lag behind across most metrics.
+There is an edge for the non-convolutional models for smaller thresholds that switches for higher values.
 
 \subsubsection{Pregnancy Use-Case Results}\label{subsubsec:use_case_pregnancy_results}
 
-\begin{table}
+\begin{figure}[htbp]
     \centering
+    \includegraphics[width=0.9\textwidth]{resources/figures/results/pregnancy_use_case_results_by_fertility_threshold}
+    \caption{
+        Contraception use-case evaluation study results by fertility threshold.
+    }
+    \label{fig:results_pregnancy_use_case_results_by_fertility_threshold}
+\end{figure}
 
-    \begin{subtable}{\textwidth}
+
+Figure~\ref{fig:results_pregnancy_use_case_results_by_fertility_threshold} presents the results for the pregnancy use-case
+across varying fertility thresholds, comparing all model architectures and baselines.
+The four reported metrics are: number of pregnancies, number of intercourse events, number of correct deferrals (i.e., correctly flagged infertile days),
+and number of incorrect deferrals (i.e., fertile days incorrectly flagged as infertile).
+
+The overall pattern mirrors the contraception use-case, but the optimization goal is reversed:
+here, a higher number of pregnancies is desirable.
+
+\paragraph{Pregnancies.}
+Trained models achieve substantially higher pregnancy rates than the baselines.
+They begin at around 275 pregnancies for low thresholds and decline linearly to between 157 and 176 at higher thresholds.
+By contrast, the baselines start much lower (user-mean: 159, last-cycle: 139, population-mean: 117) and also decrease linearly,
+reaching 119, 102, and 71 pregnancies, respectively.
+This places all trained models well above baseline performance across the full threshold range.
+
+\paragraph{Intercourse Events.}
+Trained models start with high numbers of intercourse events, near 7000, except for the LSTM which begins lower at 4252.
+All decline sublinearly and converge toward approximately 1500 at the highest thresholds.
+In contrast, the baselines start far lower, between 1700 and 1900, and decrease linearly to about 1300.
+Thus, the improved pregnancy rates of trained models come at the cost of substantially higher intercourse event counts.
+
+\paragraph{Correct Deferrals.}
+The baselines initially show higher numbers of correct deferrals, with values that increase linearly as thresholds rise.
+Trained models begin much lower but follow a logarithmic-like growth pattern,
+eventually converging with the baselines at high thresholds.
+Within the trained group, the LSTM starts with noticeably higher values than the other models, though all converge toward a similar range.
+
+\paragraph{Incorrect Deferrals.}
+For trained models, incorrect deferrals start very low at small thresholds
+(Transformer: 11, LSTM: 190) and grow with a curved, logarithmic-like pattern.
+At high thresholds they reach between 1906 (Convolutional Transformer) and 2378 (Transformer).
+The LSTM consistently produces more incorrect deferrals than the other trained models, except upper end of the threshold value range.
+The Transformer once again shows the familiar pattern of excelling at low thresholds
+but converging to the weakest performance among trained models at higher thresholds.
+
+\begin{figure}[htbp]
     \centering
-        \scriptsize
-        \begin{tabularx}{\linewidth}{l*{5}{X}}
-            \toprule
-            Model                     & No. of Intercourse Events        & Pregnancies      & Correct Deferrals & Incorrect Deferrals \\
-            \midrule
-            Convolutional LSTM        & 4899 (4890-4908) & 216.9 (214.8-219.1) & 10038            & 1734               \\
-            Transformer               & 3685 (3678-3692) & 264.9 (262.7-267.0) & 17489            & 350                \\
-            Convolutional Transformer & 4903 (4894-4912) & 214.3 (212.2-216.4) & 10038            & 1734               \\
-            LSTM                      & 3688 (3680-3696) & 266.9 (264.7-269.1) & 17489            & 350                \\
-            Last-Cycle Baseline       & 1536 (1531-1541) & 138.7 (137.1-140.2) & 24593            & 4038               \\
-            Population-Mean Baseline  & 1704 (1699-1709) & 113.5 (112.1-114.9) & 22956            & 4841               \\
-            User-Mean Baseline        & 1652 (1647-1656) & 158.4 (156.8-160.0) & 24528            & 3523               \\
-            \bottomrule
-        \end{tabularx}
-        \caption{Pregnancy metrics at threshold \textbf{0.01} for all models. }
-        \label{tab:results_pregnancy_use_case_0_01}
-    \end{subtable}
+    \includegraphics[width=0.9\textwidth]{resources/figures/results/pregnancy_use_case_pregnancy_statistics}
+    \caption{
+        Contraception use-case evaluation study results by fertility threshold.
+    }
+    \label{fig:results_pregnancy_use_case_pregnancy_statistics}
+\end{figure}
 
-    \vspace{1.5em}
+To facilitate comparison here as well, Figure~\ref{fig:results_pregnancy_use_case_pregnancy_statistics} presents two
+normalized pregnancy metrics across fertility thresholds: the number of pregnancies per 100 users per year,
+and the number of pregnancies per 1000 intercourse events.
 
-    \begin{subtable}{\textwidth}
-        \centering
-        \scriptsize
-        \begin{tabularx}{\linewidth}{l*{5}{X}}
-            \toprule
-            Model                     & No. of Intercourse Events         & Pregnancies      & Correct Deferrals & Incorrect Deferrals \\
-            \midrule
-            Convolutional LSTM        & 6174 (6164-6184) & 236.1 (234.1-238.2) &              4332 &                1076 \\
-            Transformer               & 4253 (4245-4261) & 270.9 (268.6-273.1) &             14825 &                 190 \\
-            Convolutional Transformer & 6173 (6164-6182) & 235.7 (233.6-237.9) &              4332 &                1076 \\
-            LSTM                      & 4255 (4247-4262) & 273.0 (270.6-275.3) &             14825 &                 190 \\
-            Last-Cycle Baseline       & 1581 (1576-1586) & 139.8 (138.2-141.3) &             24327 &                4091 \\
-            Population-Mean Baseline  & 1752 (1747-1757) & 117.1 (115.6-118.6) &             22655 &                4888 \\
-            User-Mean Baseline        & 1695 (1690-1701) & 158.3 (156.6-160.1) &             24271 &                3567 \\
-            \bottomrule
-        \end{tabularx}
-        \caption{Pregnancy metrics at threshold \textbf{0.05} for all models.}
-        \label{tab:results_pregnancy_use_case_0_05}
-    \end{subtable}
+All trained models clearly result in more pregnancies per 100 users overall.
+However, for lower fertility thresholds, there are fewer pregnancies per 1000 intercourse events for the trained models
+compared to the baselines.
 
-    \vspace{1.5em}
+\paragraph{Summary.}
+In the pregnancy use-case, trained models clearly outperform baselines in terms of pregnancy rates,
+though at the expense of more intercourse events and higher incorrect deferrals.
+This results in less pregnancies per 1000 intercourse events for the trained models on smaller thresholds.
+However, for 100 user-years, the pregnancy rates are noticeably higher than the baselines.
+The LSTM underperforms relative to the Transformer-based architectures across most metrics,
+while the Transformer itself exhibits strong performance at low thresholds but deteriorates more rapidly with increasing thresholds.
+There is an edge for the non-convolutional models for smaller thresholds that switches for higher values.
 
-    \begin{subtable}{\textwidth}
-        \centering
-        \scriptsize
-        \begin{tabularx}{\linewidth}{l*{5}{X}}
-            \toprule
-            Model                     & No. of Intercourse Events          & Pregnancies      & Correct Deferrals & Incorrect Deferrals \\
-            \midrule
-            Convolutional LSTM        & 3887 (3880-3895) & 198.0 (196.1-199.9) &             14544 &                2296 \\
-            Transformer               & 3291 (3284-3298) & 260.7 (258.6-262.7) &             19261 &                 551 \\
-            Convolutional Transformer & 3883 (3875-3891) & 198.4 (196.6-200.3) &             14544 &                2296 \\
-            LSTM                      & 3296 (3289-3303) & 261.1 (259.1-263.2) &             19261 &                 551 \\
-            Last-Cycle Baseline       & 1509 (1504-1513) & 135.2 (133.6-136.8) &             24778 &                4010 \\
-            Population-Mean Baseline  & 1668 (1663-1673) & 109.5 (108.1-110.9) &             23156 &                4811 \\
-            User-Mean Baseline        & 1619 (1614-1624) & 154.0 (152.4-155.6) &             24717 &                3501 \\
-            \bottomrule
-        \end{tabularx}
-        \caption{Pregnancy metrics at threshold \textbf{0.10} for all models.}
-        \label{tab:results_pregnancy_use_case_0_10}
-    \end{subtable}
 
-    \caption{grouped pregnancy metrics at thresholds 0.01, 0.05, and 0.10. values are means over 200 iterations; 95\% confidence intervals in parentheses.
-    Under our simulation at a fixed threshold, correct/incorrect deferrals are deterministic; CIs are therefore omitted for these columns.}
-    \label{tab:results_pregnancy_grouped}
-\end{table}
+% -------------------------------------------------------------
 
-\paragraph{Threshold 0.01 (Table~\ref{tab:results_pregnancy_use_case_0_01}).}
-In the conception use-case, Transformer and LSTM models generate the highest pregnancy counts (~265–267) at ~3,685 intercourse events (72 per 1,000).
-Convolutional models yield fewer pregnancies (~214–217) but allow ~4,900 intercourse events, resulting in ~44 pregnancies per 1,000.
+\subsection{Summary of key results}\label{subsec:key_results}
+% Keep this as a tight bulleted list mirroring RQs; strictly findings, no causes/interpretations.
 
-Baselines underperform on both metrics, allowing fewer events and achieving lower pregnancy counts, suggesting they are
-overly conservative without yielding benefits in effectiveness.
+%\begin{itemize}[leftmargin=*]
+%    \item \textbf{RQ1:} Temperature-based models achieve lowest MSEs on regular cycles; errors increase on irregular cycles.
+%    More history generally reduces MSE, with greater variance at higher history depth.
+%    \item \textbf{RQ2:} In contraception and conception scenarios, sequential models (LSTM/Transformer) yield
+%    the most favorable access–outcome trade-offs at threshold 0.05.
+%    \item \textbf{RQ3:} All ML models outperform rule-based baselines across targets and splits;
+%    the largest margins occur on post-ovulation metrics.
+%\end{itemize}
 
-\paragraph{Threshold 0.05 (Table~\ref{tab:results_pregnancy_use_case_0_05}).}
-At threshold 0.05, Transformer/LSTM models slightly increase intercourse access (~4,250 events) with pregnancies rising to ~271–273 (64 per 1,000).
-Notably, these models also achieve very low incorrect deferral counts (~190), indicating they rarely block opportunities for conception when they shouldn’t.
+\noindent\textbf{RQ1 (Architectures, context, and capacity).}
+\begin{itemize}
+    \item \emph{Architectures.} All learned models beat baselines by a wide margin.
+    The \textbf{LSTM} is the most reliable overall and post-ovulation; the \textbf{Transformer} is strongest on fertile days.
+    Convolutional variants are competitive but rarely best.
 
-Convolutional models allow substantially more intercourse (~6,170) with lower pregnancy counts (~236),
-but at the cost of higher incorrect deferrals (~1,076).
-This suggests they are more permissive but less selective.
+    \item \emph{Temporal context.} Performance depends on window length and differs by model.
+    LSTM benefits from long context (up to 160\,days), Transformers peak around 40–80\,days; conv models prefer mid-range (40\,days).
+    (Tables~\ref{tab:fertility_windows_compact_mse},~\ref{tab:ovover_windows_compact_mse}.)
 
-\paragraph{Threshold 0.10 (Table~\ref{tab:results_pregnancy_use_case_0_10}).}
-At the highest threshold, Transformer/LSTM models see a drop in access (~3,290 events) and in pregnancies (~261),
-but with an increase in incorrect deferrals (~551).
-Convolutional models again show a more permissive profile (~3,880 events, ~198 pregnancies), but with higher rates of incorrect deferrals (~2,296).
+    \item \emph{Input resolution.} Low–medium rates (4–48/day) minimize \emph{overall} error for sequence models; high rates help fertile-day signals.
+    Conv models learn their own compression from full resolution. (Tables~\ref{tab:fertility_resolution_compact_mse},~\ref{tab:ovover_resolution_compact_mse}.)
 
-This threshold leads to fewer pregnancies and more unnecessary blocks, especially for the Transformer/LSTM models.
+    \item \emph{Capacity.} Moderate-to-large configurations improve accuracy but with target- and phase-specific optima
+    (e.g., LSTM \(512\times4\); Transformer \(512\times4\times4\) for fertility overall, \(512\times8\times8\) for OV-over after-OV).
+    (Tables~\ref{tab:fertility_params_compact_mse}, \ref{tab:ovover_params_compact_mse}.)
 
-\paragraph{Pregnancy—Recommended Threshold.}
-Threshold 0.05 strikes the best balance for conception as well.
-Transformer and LSTM models provide high pregnancy counts and good access with minimal incorrect deferrals.
-Compared to 0.10, it results in more successful conceptions with fewer missed opportunities, making it the optimal setting for this use-case.
+    \item Best configs (Tables~\ref{tab:best_configs_lstm},~\ref{tab:best_configs_transformer})
+    generalize on the held-out test set (Table~\ref{tab:results_model_selection_metrics}).
 
-\subsection{Summary of Key Findings}\label{subsec:summary_of_key_findings}
+    \item LSTM is best \emph{overall}: fertility MSE \(0.0036\) (best overall; best non-fertile \(0.0008\)) and OV-over overall \(0.0437\)
+    (best; best after-OV \(0.0492\)). Transformer is best on fertile days (MSE \(0.0089\)).
 
-\paragraph{Model Performance.}
-LSTM models consistently achieved the lowest overall mean squared errors (MSE) across both prediction targets,
-fertility probability and ovulation-over (OV-over), on the held-out test set.
-Transformer models slightly outperformed LSTMs on fertile-day predictions, suggesting higher sensitivity to short-term signals.
-Convolutional models (ConvLSTM and ConvTransformer) performed competitively in some configurations but generally
-exhibited higher error and less favorable trade-offs in practical scenarios.
+    \item Versus the strongest baseline (User-Mean), LSTM cuts fertility MSE by \(\sim 44\%\) and post-ovulatory MSE by \(\sim 59\%\).
+\end{itemize}
 
-\paragraph{Effect of Input Settings.}
-\textit{Input window length} significantly influenced performance.
-LSTMs and Transformers benefited from longer windows (e.g., 160\,days), especially for post-ovulation detection,
-while convolutional models favored shorter windows (e.g., 20--40\,days).
-\textit{Input resolution} showed that intermediate sampling rates (4--48\,values/day) often minimized overall error,
-though full resolution (288/day) was most useful for detecting fertile windows in LSTM models.
+\noindent\textbf{RQ2 (Factors and Patterns).}
+\begin{itemize}
+    \item Models consistently key on the \emph{pre-ovulatory temperature drop}; larger drops yield higher predicted fertility.
+    This pattern weakens in irregular cycles with noisier temperature traces (Figures~\ref{fig:results_rq2_regular_cycle_predictions_example}–\ref{fig:results_rq2_irregular_cycle_predictions_example}, \ref{fig:results_temperature_drop_pattern}).
 
-\paragraph{Model Scaling.}
-Larger configurations (e.g., 512 hidden units, 4--8 layers, 4--8 attention heads) generally improved performance across architectures,
-especially for Transformers in OV-over prediction.
+    \item More user history helps modestly; LSTM variants benefit most.
+    Baselines do not: Last-Cycle and Population-Mean often degrade with more history (Figure~\ref{fig:results_performance_on_different_historical_context}).
+\end{itemize}
 
-\paragraph{Comparison to Baselines.}
-All trained models substantially outperformed baseline predictors (user mean, last-cycle, population mean) across metrics and targets.
-The LSTM model reduced fertility MSE by 44\% and post-ovulation MSE by 59\% compared to the strongest baseline.
+\noindent\textbf{RQ3 (Regular vs.\ Irregular).}
+\begin{itemize}
+    \item All models are better on \emph{regular} cycles.
+    For fertility, Transformer leads overall and on fertile days; LSTM leads on non-fertile days.
+    For OV-over, LSTM leads overall and after-OV; Transformer leads before-OV (Tables~\ref{tab:fertility_mse_regular_irregular},~\ref{tab:ov_over_mse_regular_irregular}).
 
-\paragraph{Stratified Analyses.}
-\textit{User history depth:} More historical cycles consistently improved prediction accuracy, especially for LSTM models.
-However, output variability increased with longer histories.
-\textit{Cycle regularity:} All models performed better on regular cycles than irregular ones.
-LSTM remained the most robust across both groups.
+    \item In \emph{irregular} cycles, errors rise across the board.
+    LSTM remains most robust; Convolutional LSTM is competitive for fertility overall; Transformers are less stable before/after phase splits.
 
-\paragraph{Use-Case Scenarios.}
-In \textit{contraceptive settings}, Transformer and LSTM models maintained pregnancy rates below 2.2 per 1{,}000 intercourse events
-at a 0.05 threshold---demonstrating both high reliability and user access.
-In \textit{conception settings}, the same models yielded the highest pregnancy rates ($\sim$64 per 1{,}000 events)
-with minimal missed opportunities, again at the 0.05 threshold.
-Convolutional models allowed more events but were less selective,
-resulting in higher pregnancy rates in contraception and lower in conception scenarios.
-Baseline methods were consistently outperformed in both use-cases, often sacrificing either effectiveness or access.
+    \item Variance of predictions grow with longer context lengths.
+\end{itemize}
+
+\noindent\textbf{RQ4 (Use-case Evaluations).}
+\begin{itemize}
+    \item \emph{Contraception:} Trained models cut pregnancies sharply vs baselines,
+    at the cost of more denials and fewer intercourse opportunities.
+    Transformer variants are best at low thresholds; LSTM lags overall.
+    Convolutional models excel at higher thresholds
+    (Figures~\ref{fig:results_contraception_use_case_results_by_fertility_threshold},~\ref{fig:results_contraception_use_case_pregnancy_statistics}).
+
+    \item \emph{Pregnancy seeking:} Trained models yield many more pregnancies than baselines but require
+    more intercourse events and accept more incorrect deferrals at high thresholds.
+    Transformer excels at low thresholds; LSTM underperforms overall.
+    Convolutional models again excel at higher thresholds
+    (Figures~\ref{fig:results_pregnancy_use_case_results_by_fertility_threshold},~\ref{fig:results_pregnancy_use_case_pregnancy_statistics}).
+\end{itemize}