alex/temperature-based-fertility-prediction-thesis
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

further work on discussion

Browse Source
This commit is contained in:
Alex Blank
2025-09-04 17:30:43 +00:00
parent a1b2662b8a
commit a2fdf2f0a3
25 changed files with 2707 additions and 2621 deletions
Download Patch File Download Diff File Expand all files Collapse all files
thesis/sections/results.tex
+299 -229
View File
@@ -3,19 +3,18 @@
\section{Results}\label{sec:results}
We report results for two targets: (i) fertility probability and (ii) an indicator that ovulation has already occurred (OV-Over).
Unless stated otherwise, \textbf{MSE} is the primary metric (equivalent to the Brier score for probabilistic targets),
and \textbf{MAE} is secondary.
We first summarize overall performance across architectures, then analyze sensitivity to input window length,
input resolution, and model capacity.
Finally, we compare the best configurations to three baselines and present stratified and use-case analyses.
\subsection{Overall Model Performance Across Architectures}\label{subsec:overall_model_performance_across_architectures}
% find introduction
\subsection{RQ1: Comparative Study of Model Architectures}\label{subsec:rq1_temp_predictive_value}
We systematically evaluated multiple model architectures to assess their performance in
predicting fertility probability and ovulation-over targets.
predicting the targets.
We report results for two targets: (i) fertility probability and (ii) an indicator that ovulation has already occurred (OV-Over).
Each architecture was tested across a range of input window lengths, temporal resolutions, and model capacities.
The best-performing configurations for each architecture were selected for downstream analyses.
Unless stated otherwise, \textbf{MSE} is the primary metric (equivalent to the Brier score for probabilistic targets),
and \textbf{MAE} is secondary.
All metrics reported in this section are point estimates without confidence intervals; therefore, comparisons are descriptive.
@@ -124,7 +123,7 @@ Results are summarized in Table~\ref{tab:fertility_resolution_compact_mse}; full
\paragraph{Impact of model parameters.}
We next evaluate how architectural complexity, i.e., hidden size, number of layers,
and number of attention heads—influences performance under fixed input settings (160 days at 12/day for LSTM/Transformer, 40 days for convolutional models).
and number of attention heads, influences performance under fixed input settings (160 days at 12/day for LSTM/Transformer, 40 days for convolutional models).
Parameters are reported as (hidden size × number of layers) for LSTM models and (embedding size × number of encoders × attention heads) for Transformers.
For the \textbf{LSTM}, the best overall and fertile-day MSEs are both observed at a configuration with 512 hidden units and 4 layers.
@@ -194,9 +193,6 @@ The \textbf{LSTM} performs best before ovulation (MSE 0.0212 at 20 days), while
The \textbf{Convolutional LSTM} favors short windows, with the best overall MSE (0.0699), before-OV (0.0389),
and after-OV (0.0833) all occurring at 20 days.
The \textbf{Convolutional Transformer} performs best overall at 40 days and best before ovulation at 160 days (MSE 0.0286).
These trends suggest that intermediate windows (20–40 days) often balance short- and long-term signal,
while long windows (e.g., 160 days) help capture post-ovulation patterns.
See Table~\ref{tab:ovover_windows_compact_mse} for a summary.
\begin{table}[t]
@@ -242,7 +238,6 @@ with best before-OV and after-OV MSEs at 12/day (0.0212) and 48/day (0.0550), re
The \textbf{Transformer} performs best overall at 72/day (0.0585),
with lowest before-OV MSE at 12/day (0.0255) and after-OV MSE at 288/day (0.0578).
This indicates that high resolution benefits post-ovulation prediction, while lower rates suffice for pre-ovulation.
Results are summarized in Table~\ref{tab:ovover_resolution_compact_mse};
full resolution grids are in Appendix Table~\ref{tab:ov_over_results_by_resolution}.
@@ -334,7 +329,8 @@ while also considering the general performance trends of each model.
Tables~\ref{tab:best_configs_lstm} and~\ref{tab:best_configs_transformer} summarize the selected hyperparameters for each architecture,
including input window length, temporal resolution, and model complexity.
For convolutional models, the input resolution was always fixed at the full 288 measurements per day.
For the Transformer architecture, we made the decision to use longer input windows, as the performance on longer
input windows was competitive to the best measured at 40 days, but includes much more context information.
\begin{table}[htbp]
\centering
@@ -389,9 +385,9 @@ All trained architectures outperform the baseline models across all evaluation m
& Fertility Overall & Fertile Days & Non-Fertile Days & OV-Over Overall & OV-Over Before OV & OV-Over After OV \\
\midrule
Transformer & 0.0037 & \textbf{0.0089} & 0.0017 & 0.0508 & 0.0236 & 0.0581 \\
Convolutional Transformer & 0.0038 & 0.0098 & 0.0014 & 0.0517 & 0.0305 & 0.0566 \\
LSTM & \textbf{0.0036} & 0.0104 & \textbf{0.0008} & \textbf{0.0437} & \textbf{0.0233} & \textbf{0.0492} \\
Convolutional LSTM & 0.0037 & 0.0092 & 0.0014 & 0.0499 & 0.0281 & 0.0563 \\
Convolutional Transformer & 0.0038 & 0.0098 & 0.0014 & 0.0517 & 0.0305 & 0.0566 \\
User-Based-Mean Baseline & 0.0064 & 0.0154 & 0.0028 & 0.1072 & 0.0872 & 0.0983 \\
Last-Cycle Baseline & 0.0080 & 0.0202 & 0.0031 & 0.1453 & 0.1099 & 0.1762 \\
Population-Mean Baseline & 0.0127 & 0.0258 & 0.0074 & 0.2145 & 0.0400 & 0.3749 \\
@@ -410,9 +406,89 @@ Compared to the strongest baseline (User-Mean), the LSTM reduces the overall fer
The performance gap is even larger when compared to the population-mean and last-cycle baselines,
confirming the advantage of personalized and temporally-aware modeling.
\subsection{Stratified Analysis}\label{subsec:stratified_analysis}
% -------------------------------------------------------------------------------------------------------------------------------
\subsection{RQ2: Factors and Patterns that Influence Predictions}\label{subsec:rq2_factors_and_patterns}
\subsubsection{Representative predictions}\label{subsubsec:rq2_examples}
All following prediction were made with the best model for each architecture as selected in the previous section.
\begin{figure}[htbp]
\centering
\includegraphics[width=1.0\textwidth]{resources/figures/results/regular_cycle_pattern_fertility_prediction_example}
\caption{
Temperature rolling average and fertility-probability prediction for a user with a regular cycle pattern. (Values are scaled features)
}
\label{fig:results_rq2_regular_cycle_predictions_example}
\end{figure}
To visualize potential patterns in the predictions, we will show some representative prediction plots.
All models have a similar predictive behaviour throughout both the regular and irregular cycles.
Figure~\ref{fig:results_rq2_regular_cycle_predictions_example} shows, that all models come very close to the ground truth
for a regular cycle pattern.
Slight deviations from the regular cycle length, such as in cycle 5 (Measurement 150--220) show, that the models
tend to overestimate the fertility-probability in such cases.
Additionally, all models never reach the full range of fertility-probability as indicated by the ground-truth.
These predictions show a clear correlation with a temperature drop before the ovulation.
\begin{figure}[htbp]
\centering
\includegraphics[width=1.0\textwidth]{resources/figures/results/temperature_drop_pattern}
\caption{
Temperature rolling average and fertility-probability prediction showing a correlation between fertility and
the pre-ovulator temperature drop.
}
\label{fig:results_temperature_drop_pattern}
\end{figure}
This correlation between a pre-ovulator temperature drop and the fertility can be seen more pronounced in Figure~\ref{fig:results_temperature_drop_pattern}.
The ground-truth fertility seems to be centered around this temperature drop and all models seem to be able
to pick it up, if it is clearly distinguishable and visible in the data.
For regular cycles, this temperature drop seems to be much easier distinguishable,
as there is not as much temperature variability throughout the first cycle phase.
Additionally, there also seems to be a correlation between the height of the temperature drop and the corresponding
fertility-probability prediction by the models.
The second cycle (Measurement 35--90) has a smaller temperature drop and all models predict a smaller fertility-probability
as for the following two cycles (Measurements 90--145).
This association of temperature-drop height and fertility-probability can also be seen in Figure~\ref{fig:results_rq2_regular_cycle_predictions_example}.
The first two cycles have a smaller temperature drop compared to the following cycles and also a less pronounced prediction.
Figure~\ref{fig:results_rq2_irregular_cycle_predictions_example} shows an irregular cycle pattern
and the fertility-probability predictions for it.
For irregular cycles, all models show much more struggle in determining the ground-truth fertility.
The pre-ovulator temperature drop is far less distinguishable.
The fifth cycle (measurement 450--700) shows a much more gradual temperature drop and all model's predictions
are far off target.
The previous 2 cycles (cycle 3 and 4) are shorter and have a much more distinguishable temperature drop
and thus all models show indication for a detection of the fertile phase.
\begin{figure}[htbp]
\centering
\includegraphics[width=1.0\textwidth]{resources/figures/results/irregular_cycle_pattern_fertility_prediction_example}
\caption{
Temperature rolling average and fertility-probability prediction for a user with an irregular cycle pattern. (Values are scaled features)
}
\label{fig:results_rq2_irregular_cycle_predictions_example}
\end{figure}
All models seem to be easily confused by anomalies during this characteristic temperature drop, even for short and regular cycles.
Figure~\ref{fig:results_rq2_anomaly_in_temperature_drop} shows a cycle with a clear anomaly during the pre-ovulatory drop in cycle 3 (Measurements 90--125).
This confuses all models into ending the fertile phase earlier, even though fertility is likely still elevated.
\begin{figure}[htbp]
\centering
\includegraphics[width=1.0\textwidth]{resources/figures/results/spike_in_temperature_drop}
\caption{
Temperature rolling average and fertility-probability prediction for a user with an anomaly during the characteristic temperature drop. (Values are scaled features)
}
\label{fig:results_rq2_anomaly_in_temperature_drop}
\end{figure}
\subsubsection{Effect of user history depth}\label{subsubsec:rq2_history_depth}
\subsubsection{Influence of User History Depth}\label{subsubsec:influence_of_past_user_data}
\begin{figure}[htbp]
\centering
\includegraphics[width=0.8\textwidth]{resources/figures/results/performance_on_different_historical_contexts}
@@ -426,16 +502,22 @@ confirming the advantage of personalized and temporally-aware modeling.
Figure~\ref{fig:results_performance_on_different_historical_context} shows the MSE for the fertility probability
and OV-over targets as a function of the number of past cycles available per user, for all model architectures and baselines.
All four models improve on both metrics as the amount of historical data increases.
All four models improve slightly on both metrics as the amount of historical data increases.
The LSTM-based models show a larger relative improvement with longer user history than the Transformer-based variants.
The User-Mean Baseline also improves substantially, with the Last-Cycle Baseline showing a smaller but still notable gain.
In contrast, the Population-Mean Baseline performs worse (i.e., MSE increases) as more historical cycles are included.
All baselines seem produce worse prediction with larger historical contexts.
The last-cycle and population-mean baselines show the largest decrease in performance,
especially for the fertility-probability prediction target.
The user-mean baselines offers largely unchanged fertility prediction performance,
but worsens on the ov-over prediction with more historical cycles available.
Across all models and baselines, the variability of results increases with greater history depth:
results are more tightly clustered around the trend line with short histories,
but show greater scatter for users with longer data records.
\subsubsection{Regular vs Irregular Cycles}\label{subsubsec:regular_vs_irregular_cycles}
% -------------------------------------------------------------
\subsection{RQ3: Performance across Regular and Irregular Cycles}\label{subsec:rq3_regular_vs_irregular}
\begin{table}[htbp]
\centering
@@ -538,7 +620,6 @@ As before, all learned models outperform the baselines by a wide margin.
The full table with MSE and MAE for all models can be found in the appendix, Table~\ref{tab:regular_vs_irregular_ov_over_results}.
\vspace{1em}
\begin{figure}[htbp]
\centering
@@ -559,10 +640,12 @@ The full table with MSE and MAE for all models can be found in the appendix, Tab
\end{figure}
\paragraph{Evaluation on Historical Context Depth.}
Figures~\ref{fig:results_performance_on_regular_cycles} and~\ref{fig:results_performance_on_irregular_cycles} visualize performance improvements with increasing numbers of past cycles.
Figures~\ref{fig:results_performance_on_regular_cycles} and~\ref{fig:results_performance_on_irregular_cycles}
visualize performance improvements with increasing numbers of past cycles.
For regular cycles, all models except the population-mean baseline benefit from more historical data, showing consistent MSE reductions for both targets.
Interestingly, the population-mean baseline performs worse as more cycles are added.
The largest improvement can be seen for the last-cycle baseline and user-mean baseline models.
The outliers in the last-cycle and user-mean baseline are MSE values of 0.0, which result in these vertical lines.
In contrast, irregular cycles exhibit more variability and less performance gain with more cycles available.
All trained models improve on both targets with more context.
@@ -571,255 +654,242 @@ In contrast, their MSE increases with more historical context for the fertility-
The last-cycle baseline shows marginal performance improvement for the OV-over target and the same increase for the fertility-probability
target as the other two baselines.
The selected irregular cycle group has cycles with longer context, thus the irregular cycle history reaches up to 80 cycles,
while regular cycle history only reaches up to 35.
All models show increased output variability as the number of available past cycles grows,
as indicated by the wider deviations from the trend lines.
\subsection{Use-Case Evaluation Results}\label{subsec:use_case_evaluation_results}
% -------------------------------------------------------------
\subsection{RQ4: Use-Case Evaluations}\label{subsec:rq4_use_case_evaluations}
We evaluated the use-case scenarios described in Section~\ref{subsubsec:practical_use_cases} using the algorithms
in Section~\ref{subsubsec:use_case_evaluation} across multiple fertility thresholds.
Each evaluation used the same test set of 100 users (100 user-years) and was repeated for 200 iterations
to get statistically more meaningful results.
we report means and 95\% confidence intervals (CIs).
All results represent the means over all runs.
\subsubsection{Contraception Use-Case Results}\label{subsubsec:use_case_contraception_results}
\paragraph{Threshold 0.01 (Table~\ref{tab:results_contraception_use_case_0_01}).}
At the strictest threshold of 0.01, Transformer and LSTM models achieve the lowest pregnancy rates, only 4.4 to 4.6
on average, despite allowing approximately 3,200 intercourse events.
This equates to about 1.4 pregnancies per 1,000 intercourse events, a strong result for contraceptive reliability.
In contrast, convolutional variants (ConvLSTM and ConvTransformer) restrict intercourse events to around 1,150,
leading to roughly 34–35 pregnancies, or about 30 per 1,000 events—far less efficient in terms of balance between access and protection.
All baseline models perform substantially worse, with pregnancy counts exceeding 100 in all cases, confirming the value of personalized predictions.
\begin{table}
\begin{figure}[htbp]
\centering
\includegraphics[width=0.9\textwidth]{resources/figures/results/contraception_use_case_results_by_fertility_threshold}
\caption{
Contraception use-case evaluation study results by fertility threshold.
}
\label{fig:results_contraception_use_case_results_by_fertility_threshold}
\end{figure}
\begin{subtable}{\textwidth}
\centering
\scriptsize
\begin{tabularx}{\linewidth}{l*{5}{X}}
\toprule
Model & No. of Intercourse Events & Pregnancies & Correct Denials & Incorrect Denials \\
\midrule
Convolutional LSTM & 1154 (1149-1158) & 34.9 (34.1-35.7) & 1615 (1611-1620) & 4963 (4954-4971) \\
Transformer & 3200 (3194-3207) & 4.4 (4.1-4.7) & 1817 (1812-1822) & 2717 (2710-2724) \\
Convolutional Transformer & 1156 (1151-1160) & 34.3 (33.5-35.0) & 1613 (1608-1619) & 4974 (4965-4983) \\
LSTM & 3206 (3199-3213) & 4.6 (4.3-4.9) & 1832 (1827-1838) & 2711 (2705-2717) \\
Last-Cycle Baseline & 6066 (6057-6075) & 127.5 (125.9-129.1) & 918 (915-922) & 762 (759-766) \\
Population-Mean Baseline & 5874 (5865-5883) & 153.7 (151.9-155.5) & 749 (746-752) & 1121 (1116-1125) \\
User-Mean Baseline & 5933 (5924-5942) & 105.8 (104.4-107.2) & 1030 (1026-1034) & 776 (772-779) \\
\bottomrule
\end{tabularx}
\caption{Contraception metrics at threshold \textbf{0.01} for all models.}
\label{tab:results_contraception_use_case_0_01}
\end{subtable}
Figure~\ref{fig:results_contraception_use_case_results_by_fertility_threshold} shows the outcomes of
the contraception use-case across fertility thresholds for each model architecture, along with the baselines.
The four evaluated metrics are: number of pregnancies, number of intercourse events,
number of correct denials (i.e., appropriately flagged fertile days),
and number of incorrect denials (i.e., fertile days incorrectly flagged as infertile).
\vspace{1.5em}
\paragraph{Pregnancies.}
All trained models start with very low pregnancy rates (Transformer: 0, LSTM: 4)
and then increase roughly linearly with the threshold,
reaching between 99 (Convolutional Transformer) and 122 (Transformer) at the highest setting.
The LSTM performs worst overall in this metric.
The Transformer starts with the lowest pregnancy rate but rises steeply,
ending with the highest number of pregnancies at threshold 0.1.
By comparison, the baselines show much higher pregnancy rates throughout,
starting between 103 (user-mean) and 150 (population-mean) and increasing by about 50 pregnancies over the range of thresholds.
\begin{subtable}{\textwidth}
\centering
\scriptsize
\begin{tabularx}{\linewidth}{l*{5}{X}}
\toprule
Model & No. of Intercourse Events & Pregnancies & Correct Denials & Incorrect Denials \\
\midrule
Convolutional LSTM & 2509 (2502-2515) & 55.0 (54.0-56.0) & 1448 (1444-1452) & 3768 (3760-3775) \\
Transformer & 3801 (3793-3810) & 8.2 (7.8-8.6) & 1768 (1763-1773) & 2162 (2157-2168) \\
Convolutional Transformer & 2510 (2503-2516) & 55.6 (54.5-56.6) & 1448 (1443-1453) & 3776 (3769-3783) \\
LSTM & 3801 (3794-3809) & 8.2 (7.8-8.7) & 1772 (1766-1777) & 2160 (2154-2165) \\
Last-Cycle Baseline & 6106 (6096-6116) & 130.2 (128.7-131.7) & 888 (885-892) & 750 (746-753) \\
Population-Mean Baseline & 5924 (5914-5933) & 156.1 (154.3-157.8) & 716 (713-720) & 1100 (1096-1104) \\
User-Mean Baseline & 5988 (5978-5997) & 108.8 (107.3-110.3) & 998 (994-1002) & 764 (761-768) \\
\bottomrule
\end{tabularx}
\caption{Contraception metrics at threshold \textbf{0.05} for all models.}
\label{tab:results_contraception_use_case_0_05}
\end{subtable}
\paragraph{Intercourse Events.}
All trained models except the LSTM begin with low values of about 1000 intercourse events.
The LSTM starts much higher, around 3000, and increases sharply, converging toward approximately 6100.
All trained model variants show curved growth resembling logarithmic convergence, approaching the same upper range.
In contrast, the baselines start near 6000 events and converge toward the same levels as the trained models at higher thresholds.
\vspace{1.5em}
\paragraph{Correct Denials.}
The number of correct denials decreases by about half across the tested thresholds.
All trained models cluster closely, starting at around 1800 correct denials,
with the Transformer and its convolutional variant slightly outperforming the LSTMs.
The baselines start at substantially lower levels: 1012 (user-mean), 902 (last-cycle), and 737 (population-mean).
\begin{subtable}{\textwidth}
\centering
\scriptsize
\begin{tabularx}{\linewidth}{l*{5}{X}}
\toprule
Model & No. of Intercourse Events & Pregnancies & Correct Denials & Incorrect Denials \\
\midrule
Convolutional LSTM & 3596 (3588-3603) & 75.2 (74.0-76.4) & 1292 (1288-1297) & 2847 (2840-2853) \\
Transformer & 4226 (4217-4234) & 13.3 (12.8-13.7) & 1692 (1687-1696) & 1822 (1817-1827) \\
Convolutional Transformer & 3592 (3584-3599) & 73.8 (72.7-75.0) & 1296 (1292-1301) & 2846 (2840-2852) \\
LSTM & 4222 (4214-4230) & 13.3 (12.8-13.8) & 1694 (1688-1699) & 1818 (1812-1823) \\
Last-Cycle Baseline & 6145 (6135-6155) & 134.8 (133.3-136.3) & 865 (861-868) & 744 (740-747) \\
Population-Mean Baseline & 5960 (5951-5969) & 160.3 (158.5-162.1) & 694 (691-697) & 1088 (1084-1093) \\
User-Mean Baseline & 6014 (6005-6022) & 111.8 (110.3-113.3) & 969 (965-973) & 758 (755-762) \\
\bottomrule
\end{tabularx}
\caption{Contraception metrics at threshold \textbf{0.10} for all models.}
\label{tab:results_contraception_use_case_0_10}
\end{subtable}
\paragraph{Incorrect Denials.}
For incorrect denials, the Transformer-based models (except LSTM) start at very high values near 5000
but sharply decrease and converge between 500 and 1000.
The LSTM shows the same convergence pattern but starts considerably lower at 2654.
Baselines follow a different trend: the last-cycle and user-mean baselines decrease
modestly by about 100 over the range (starting at 749 and 762, respectively),
while the population-mean baseline instead decreases from 1102 to around 879.
\caption{grouped contraception metrics at thresholds 0.01, 0.05, and 0.10. values are means over 200 iterations; 95\% confidence intervals in parentheses.}
\label{tab:results_contraception_grouped}
\end{table}
\begin{figure}[htbp]
\centering
\includegraphics[width=0.9\textwidth]{resources/figures/results/contraception_use_case_pregnancy_statistics}
\caption{
Contraception use-case results: pregnancies per 100 users per year and per 1000 intercourse events, stratified by fertility threshold.
}
\label{fig:results_contraception_use_case_pregnancy_statistics}
\end{figure}
\paragraph{Threshold 0.05 (Table~\ref{tab:results_contraception_use_case_0_05}).}
Increasing the threshold to 0.05 improves access: Transformer and LSTM models now permit approximately 3,800 intercourse
events while still limiting pregnancies to around 8.2 (2.2 per 1,000).
This represents a ~19\% increase in events over threshold 0.01, at the cost of a modest rise in pregnancies (+3.8 absolute, +86\% relative).
To facilitate comparison, Figure~\ref{fig:results_contraception_use_case_pregnancy_statistics} presents two
normalized pregnancy metrics across fertility thresholds: the number of pregnancies per 100 users per year (only one potential pregnancy is counted per user),
and the number of pregnancies per 1000 intercourse events.
Convolutional models also allow more events (~2,510) but continue to produce significantly more pregnancies (~55),
yielding a less favorable risk-benefit profile.
Baselines remain underperforming.
Here, the difference between the models becomes clearer.
The baselines start at 62 to 73 pregnancies per 100 users and become linearly worse over the threshold range.
\paragraph{Threshold 0.10 (Table~\ref{tab:results_contraception_use_case_0_10}).}
A further increase to 0.10 raises Transformer/LSTM events to ~4,220, but also raises pregnancies to ~13.3 (3.1 per 1,000).
This is a ~11\% gain in access compared to 0.05, but the pregnancy count increases by ~62\%.
The LSTM performs worse than all other trained models, except for the upper end of the threshold interval.
The Transformer, while being the best model for small thresholds, becomes worse over growing values until being on par with
the LSTM for the maximum tested value (0.1).
Meanwhile, convolutional models permit ~3,590 events and result in 74–75 pregnancies, indicating a consistent trade-off in favor of sequential models.
\paragraph{Contraception—Recommended Threshold.}
Threshold 0.05 achieves a favorable balance between access and effectiveness.
Transformer and LSTM models perform best, allowing a relatively high number of intercourse events while keeping pregnancies low.
This threshold offers the best compromise and is selected as the most promising setting for contraceptive use.
\paragraph{Summary.}
In the contraception use-case, the trained models achieve substantially lower pregnancy rates than the baselines,
though at the cost of fewer intercourse opportunities and more denials, both correct and incorrect.
This results in less pregnancies for both 100 users for a year and per 1000 intercourse events for all trained models.
Within the trained models, Transformers and their convolutional variants perform best overall, while LSTMs lag behind across most metrics.
There is an edge for the non-convolutional models for smaller thresholds that switches for higher values.
\subsubsection{Pregnancy Use-Case Results}\label{subsubsec:use_case_pregnancy_results}
\begin{table}
\begin{figure}[htbp]
\centering
\includegraphics[width=0.9\textwidth]{resources/figures/results/pregnancy_use_case_results_by_fertility_threshold}
\caption{
Contraception use-case evaluation study results by fertility threshold.
}
\label{fig:results_pregnancy_use_case_results_by_fertility_threshold}
\end{figure}
\begin{subtable}{\textwidth}
\centering
\scriptsize
\begin{tabularx}{\linewidth}{l*{5}{X}}
\toprule
Model & No. of Intercourse Events & Pregnancies & Correct Deferrals & Incorrect Deferrals \\
\midrule
Convolutional LSTM & 4899 (4890-4908) & 216.9 (214.8-219.1) & 10038 & 1734 \\
Transformer & 3685 (3678-3692) & 264.9 (262.7-267.0) & 17489 & 350 \\
Convolutional Transformer & 4903 (4894-4912) & 214.3 (212.2-216.4) & 10038 & 1734 \\
LSTM & 3688 (3680-3696) & 266.9 (264.7-269.1) & 17489 & 350 \\
Last-Cycle Baseline & 1536 (1531-1541) & 138.7 (137.1-140.2) & 24593 & 4038 \\
Population-Mean Baseline & 1704 (1699-1709) & 113.5 (112.1-114.9) & 22956 & 4841 \\
User-Mean Baseline & 1652 (1647-1656) & 158.4 (156.8-160.0) & 24528 & 3523 \\
\bottomrule
\end{tabularx}
\caption{Pregnancy metrics at threshold \textbf{0.01} for all models. }
\label{tab:results_pregnancy_use_case_0_01}
\end{subtable}
\vspace{1.5em}
Figure~\ref{fig:results_pregnancy_use_case_results_by_fertility_threshold} presents the results for the pregnancy use-case
across varying fertility thresholds, comparing all model architectures and baselines.
The four reported metrics are: number of pregnancies, number of intercourse events, number of correct deferrals (i.e., correctly flagged infertile days),
and number of incorrect deferrals (i.e., fertile days incorrectly flagged as infertile).
\begin{subtable}{\textwidth}
\centering
\scriptsize
\begin{tabularx}{\linewidth}{l*{5}{X}}
\toprule
Model & No. of Intercourse Events & Pregnancies & Correct Deferrals & Incorrect Deferrals \\
\midrule
Convolutional LSTM & 6174 (6164-6184) & 236.1 (234.1-238.2) & 4332 & 1076 \\
Transformer & 4253 (4245-4261) & 270.9 (268.6-273.1) & 14825 & 190 \\
Convolutional Transformer & 6173 (6164-6182) & 235.7 (233.6-237.9) & 4332 & 1076 \\
LSTM & 4255 (4247-4262) & 273.0 (270.6-275.3) & 14825 & 190 \\
Last-Cycle Baseline & 1581 (1576-1586) & 139.8 (138.2-141.3) & 24327 & 4091 \\
Population-Mean Baseline & 1752 (1747-1757) & 117.1 (115.6-118.6) & 22655 & 4888 \\
User-Mean Baseline & 1695 (1690-1701) & 158.3 (156.6-160.1) & 24271 & 3567 \\
\bottomrule
\end{tabularx}
\caption{Pregnancy metrics at threshold \textbf{0.05} for all models.}
\label{tab:results_pregnancy_use_case_0_05}
\end{subtable}
The overall pattern mirrors the contraception use-case, but the optimization goal is reversed:
here, a higher number of pregnancies is desirable.
\vspace{1.5em}
\paragraph{Pregnancies.}
Trained models achieve substantially higher pregnancy rates than the baselines.
They begin at around 275 pregnancies for low thresholds and decline linearly to between 157 and 176 at higher thresholds.
By contrast, the baselines start much lower (user-mean: 159, last-cycle: 139, population-mean: 117) and also decrease linearly,
reaching 119, 102, and 71 pregnancies, respectively.
This places all trained models well above baseline performance across the full threshold range.
\begin{subtable}{\textwidth}
\centering
\scriptsize
\begin{tabularx}{\linewidth}{l*{5}{X}}
\toprule
Model & No. of Intercourse Events & Pregnancies & Correct Deferrals & Incorrect Deferrals \\
\midrule
Convolutional LSTM & 3887 (3880-3895) & 198.0 (196.1-199.9) & 14544 & 2296 \\
Transformer & 3291 (3284-3298) & 260.7 (258.6-262.7) & 19261 & 551 \\
Convolutional Transformer & 3883 (3875-3891) & 198.4 (196.6-200.3) & 14544 & 2296 \\
LSTM & 3296 (3289-3303) & 261.1 (259.1-263.2) & 19261 & 551 \\
Last-Cycle Baseline & 1509 (1504-1513) & 135.2 (133.6-136.8) & 24778 & 4010 \\
Population-Mean Baseline & 1668 (1663-1673) & 109.5 (108.1-110.9) & 23156 & 4811 \\
User-Mean Baseline & 1619 (1614-1624) & 154.0 (152.4-155.6) & 24717 & 3501 \\
\bottomrule
\end{tabularx}
\caption{Pregnancy metrics at threshold \textbf{0.10} for all models.}
\label{tab:results_pregnancy_use_case_0_10}
\end{subtable}
\paragraph{Intercourse Events.}
Trained models start with high numbers of intercourse events, near 7000, except for the LSTM which begins lower at 4252.
All decline sublinearly and converge toward approximately 1500 at the highest thresholds.
In contrast, the baselines start far lower, between 1700 and 1900, and decrease linearly to about 1300.
Thus, the improved pregnancy rates of trained models come at the cost of substantially higher intercourse event counts.
\caption{grouped pregnancy metrics at thresholds 0.01, 0.05, and 0.10. values are means over 200 iterations; 95\% confidence intervals in parentheses.
Under our simulation at a fixed threshold, correct/incorrect deferrals are deterministic; CIs are therefore omitted for these columns.}
\label{tab:results_pregnancy_grouped}
\end{table}
\paragraph{Correct Deferrals.}
The baselines initially show higher numbers of correct deferrals, with values that increase linearly as thresholds rise.
Trained models begin much lower but follow a logarithmic-like growth pattern,
eventually converging with the baselines at high thresholds.
Within the trained group, the LSTM starts with noticeably higher values than the other models, though all converge toward a similar range.
\paragraph{Threshold 0.01 (Table~\ref{tab:results_pregnancy_use_case_0_01}).}
In the conception use-case, Transformer and LSTM models generate the highest pregnancy counts (~265–267) at ~3,685 intercourse events (72 per 1,000).
Convolutional models yield fewer pregnancies (~214–217) but allow ~4,900 intercourse events, resulting in ~44 pregnancies per 1,000.
\paragraph{Incorrect Deferrals.}
For trained models, incorrect deferrals start very low at small thresholds
(Transformer: 11, LSTM: 190) and grow with a curved, logarithmic-like pattern.
At high thresholds they reach between 1906 (Convolutional Transformer) and 2378 (Transformer).
The LSTM consistently produces more incorrect deferrals than the other trained models, except upper end of the threshold value range.
The Transformer once again shows the familiar pattern of excelling at low thresholds
but converging to the weakest performance among trained models at higher thresholds.
Baselines underperform on both metrics, allowing fewer events and achieving lower pregnancy counts, suggesting they are
overly conservative without yielding benefits in effectiveness.
\begin{figure}[htbp]
\centering
\includegraphics[width=0.9\textwidth]{resources/figures/results/pregnancy_use_case_pregnancy_statistics}
\caption{
Contraception use-case evaluation study results by fertility threshold.
}
\label{fig:results_pregnancy_use_case_pregnancy_statistics}
\end{figure}
\paragraph{Threshold 0.05 (Table~\ref{tab:results_pregnancy_use_case_0_05}).}
At threshold 0.05, Transformer/LSTM models slightly increase intercourse access (~4,250 events) with pregnancies rising to ~271–273 (64 per 1,000).
Notably, these models also achieve very low incorrect deferral counts (~190), indicating they rarely block opportunities for conception when they shouldn’t.
To facilitate comparison here as well, Figure~\ref{fig:results_pregnancy_use_case_pregnancy_statistics} presents two
normalized pregnancy metrics across fertility thresholds: the number of pregnancies per 100 users per year,
and the number of pregnancies per 1000 intercourse events.
Convolutional models allow substantially more intercourse (~6,170) with lower pregnancy counts (~236),
but at the cost of higher incorrect deferrals (~1,076).
This suggests they are more permissive but less selective.
All trained models clearly result in more pregnancies per 100 users overall.
However, for lower fertility thresholds, there are fewer pregnancies per 1000 intercourse events for the trained models
compared to the baselines.
\paragraph{Threshold 0.10 (Table~\ref{tab:results_pregnancy_use_case_0_10}).}
At the highest threshold, Transformer/LSTM models see a drop in access (~3,290 events) and in pregnancies (~261),
but with an increase in incorrect deferrals (~551).
Convolutional models again show a more permissive profile (~3,880 events, ~198 pregnancies), but with higher rates of incorrect deferrals (~2,296).
\paragraph{Summary.}
In the pregnancy use-case, trained models clearly outperform baselines in terms of pregnancy rates,
though at the expense of more intercourse events and higher incorrect deferrals.
This results in less pregnancies per 1000 intercourse events for the trained models on smaller thresholds.
However, for 100 user-years, the pregnancy rates are noticeably higher than the baselines.
The LSTM underperforms relative to the Transformer-based architectures across most metrics,
while the Transformer itself exhibits strong performance at low thresholds but deteriorates more rapidly with increasing thresholds.
There is an edge for the non-convolutional models for smaller thresholds that switches for higher values.
This threshold leads to fewer pregnancies and more unnecessary blocks, especially for the Transformer/LSTM models.
\paragraph{Pregnancy—Recommended Threshold.}
Threshold 0.05 strikes the best balance for conception as well.
Transformer and LSTM models provide high pregnancy counts and good access with minimal incorrect deferrals.
Compared to 0.10, it results in more successful conceptions with fewer missed opportunities, making it the optimal setting for this use-case.
% -------------------------------------------------------------
\subsection{Summary of Key Findings}\label{subsec:summary_of_key_findings}
\subsection{Summary of key results}\label{subsec:key_results}
% Keep this as a tight bulleted list mirroring RQs; strictly findings, no causes/interpretations.
\paragraph{Model Performance.}
LSTM models consistently achieved the lowest overall mean squared errors (MSE) across both prediction targets,
fertility probability and ovulation-over (OV-over), on the held-out test set.
Transformer models slightly outperformed LSTMs on fertile-day predictions, suggesting higher sensitivity to short-term signals.
Convolutional models (ConvLSTM and ConvTransformer) performed competitively in some configurations but generally
exhibited higher error and less favorable trade-offs in practical scenarios.
%\begin{itemize}[leftmargin=*]
% \item \textbf{RQ1:} Temperature-based models achieve lowest MSEs on regular cycles; errors increase on irregular cycles.
% More history generally reduces MSE, with greater variance at higher history depth.
% \item \textbf{RQ2:} In contraception and conception scenarios, sequential models (LSTM/Transformer) yield
% the most favorable access–outcome trade-offs at threshold 0.05.
% \item \textbf{RQ3:} All ML models outperform rule-based baselines across targets and splits;
% the largest margins occur on post-ovulation metrics.
%\end{itemize}
\paragraph{Effect of Input Settings.}
\textit{Input window length} significantly influenced performance.
LSTMs and Transformers benefited from longer windows (e.g., 160\,days), especially for post-ovulation detection,
while convolutional models favored shorter windows (e.g., 20--40\,days).
\textit{Input resolution} showed that intermediate sampling rates (4--48\,values/day) often minimized overall error,
though full resolution (288/day) was most useful for detecting fertile windows in LSTM models.
\noindent\textbf{RQ1 (Architectures, context, and capacity).}
\begin{itemize}
\item \emph{Architectures.} All learned models beat baselines by a wide margin.
The \textbf{LSTM} is the most reliable overall and post-ovulation; the \textbf{Transformer} is strongest on fertile days.
Convolutional variants are competitive but rarely best.
\paragraph{Model Scaling.}
Larger configurations (e.g., 512 hidden units, 4--8 layers, 4--8 attention heads) generally improved performance across architectures,
especially for Transformers in OV-over prediction.
\item \emph{Temporal context.} Performance depends on window length and differs by model.
LSTM benefits from long context (up to 160\,days), Transformers peak around 40–80\,days; conv models prefer mid-range (40\,days).
(Tables~\ref{tab:fertility_windows_compact_mse},~\ref{tab:ovover_windows_compact_mse}.)
\paragraph{Comparison to Baselines.}
All trained models substantially outperformed baseline predictors (user mean, last-cycle, population mean) across metrics and targets.
The LSTM model reduced fertility MSE by 44\% and post-ovulation MSE by 59\% compared to the strongest baseline.
\item \emph{Input resolution.} Low–medium rates (4–48/day) minimize \emph{overall} error for sequence models; high rates help fertile-day signals.
Conv models learn their own compression from full resolution. (Tables~\ref{tab:fertility_resolution_compact_mse},~\ref{tab:ovover_resolution_compact_mse}.)
\paragraph{Stratified Analyses.}
\textit{User history depth:} More historical cycles consistently improved prediction accuracy, especially for LSTM models.
However, output variability increased with longer histories.
\textit{Cycle regularity:} All models performed better on regular cycles than irregular ones.
LSTM remained the most robust across both groups.
\item \emph{Capacity.} Moderate-to-large configurations improve accuracy but with target- and phase-specific optima
(e.g., LSTM \(512\times4\); Transformer \(512\times4\times4\) for fertility overall, \(512\times8\times8\) for OV-over after-OV).
(Tables~\ref{tab:fertility_params_compact_mse}, \ref{tab:ovover_params_compact_mse}.)
\paragraph{Use-Case Scenarios.}
In \textit{contraceptive settings}, Transformer and LSTM models maintained pregnancy rates below 2.2 per 1{,}000 intercourse events
at a 0.05 threshold---demonstrating both high reliability and user access.
In \textit{conception settings}, the same models yielded the highest pregnancy rates ($\sim$64 per 1{,}000 events)
with minimal missed opportunities, again at the 0.05 threshold.
Convolutional models allowed more events but were less selective,
resulting in higher pregnancy rates in contraception and lower in conception scenarios.
Baseline methods were consistently outperformed in both use-cases, often sacrificing either effectiveness or access.
\item Best configs (Tables~\ref{tab:best_configs_lstm},~\ref{tab:best_configs_transformer})
generalize on the held-out test set (Table~\ref{tab:results_model_selection_metrics}).
\item LSTM is best \emph{overall}: fertility MSE \(0.0036\) (best overall; best non-fertile \(0.0008\)) and OV-over overall \(0.0437\)
(best; best after-OV \(0.0492\)). Transformer is best on fertile days (MSE \(0.0089\)).
\item Versus the strongest baseline (User-Mean), LSTM cuts fertility MSE by \(\sim 44\%\) and post-ovulatory MSE by \(\sim 59\%\).
\end{itemize}
\noindent\textbf{RQ2 (Factors and Patterns).}
\begin{itemize}
\item Models consistently key on the \emph{pre-ovulatory temperature drop}; larger drops yield higher predicted fertility.
This pattern weakens in irregular cycles with noisier temperature traces (Figures~\ref{fig:results_rq2_regular_cycle_predictions_example}–\ref{fig:results_rq2_irregular_cycle_predictions_example}, \ref{fig:results_temperature_drop_pattern}).
\item More user history helps modestly; LSTM variants benefit most.
Baselines do not: Last-Cycle and Population-Mean often degrade with more history (Figure~\ref{fig:results_performance_on_different_historical_context}).
\end{itemize}
\noindent\textbf{RQ3 (Regular vs.\ Irregular).}
\begin{itemize}
\item All models are better on \emph{regular} cycles.
For fertility, Transformer leads overall and on fertile days; LSTM leads on non-fertile days.
For OV-over, LSTM leads overall and after-OV; Transformer leads before-OV (Tables~\ref{tab:fertility_mse_regular_irregular},~\ref{tab:ov_over_mse_regular_irregular}).
\item In \emph{irregular} cycles, errors rise across the board.
LSTM remains most robust; Convolutional LSTM is competitive for fertility overall; Transformers are less stable before/after phase splits.
\item Variance of predictions grow with longer context lengths.
\end{itemize}
\noindent\textbf{RQ4 (Use-case Evaluations).}
\begin{itemize}
\item \emph{Contraception:} Trained models cut pregnancies sharply vs baselines,
at the cost of more denials and fewer intercourse opportunities.
Transformer variants are best at low thresholds; LSTM lags overall.
Convolutional models excel at higher thresholds
(Figures~\ref{fig:results_contraception_use_case_results_by_fertility_threshold},~\ref{fig:results_contraception_use_case_pregnancy_statistics}).
\item \emph{Pregnancy seeking:} Trained models yield many more pregnancies than baselines but require
more intercourse events and accept more incorrect deferrals at high thresholds.
Transformer excels at low thresholds; LSTM underperforms overall.
Convolutional models again excel at higher thresholds
(Figures~\ref{fig:results_pregnancy_use_case_results_by_fertility_threshold},~\ref{fig:results_pregnancy_use_case_pregnancy_statistics}).
\end{itemize}