temperature-based-fertility-prediction-thesis/thesis/sections/discussion.tex

%! Author = alex
%! Date = 3/6/25


\section{Discussion}\label{sec:discussion}

\paragraph{Limitations.}


The model selection study based on the input window length, input resolution and model complexity parameters has shown,
longer input windows and higher complexity seem to be favourable for the purposes of this study.
The convolutional models show better performance for medium length input windows, which might be a limitation
of the model or its convolutional layers.
The LSTM and Transformer models also work better with medium resolution input, which suggests a sweet-spot of information.
Too small a resolution might omit important information, while the opposite might introduce too much noise,
that the models in the current configuration cannot handle.
It could be worth investigating how more complex models handle more complex input data,
as this grid-search was not possible in this study due to time and resource limitations.

The study also showed, that the LSTM and Transformer base models outperform the convolutional models.
However, this is only the case for longer input sequences.
When isolating a 40-day input window, the convolutional models outperform the non-convolution models,
which suggests, that there is an informational benefit in the convolution.
However, the convolutional models seem to struggle with longer input windows,
which clearly represent a benefit for the predictive tasks, as the normal LSTM and Transformer models perform better
on longer input windows.
Additional work might be necessary to investigate, whether the convolutional models need additional complexity
or an optimized convolutional pre-processing to deal with longer input sequences.


All prediction curve in this section have been generated with the best model selected in section~\ref{subsubsec:results_best_model_config_selection},
i.e., the LSTM model.

\begin{figure}[htbp]
    \centering
    \includegraphics[width=1.0\textwidth]{resources/figures/discussion/regular_cycle_fertility_prediction}
    \caption{
        Temperature rolling average and fertility-probability prediction for a user with a regular cycle pattern. (Values are scaled features)
    }
    \label{fig:discussion_regular_cycle_fertility_prediction}
\end{figure}

Figure~\ref{fig:discussion_regular_cycle_fertility_prediction} shows the prediction curve for the fertility-probability target
for a user with a regular cycle pattern.
For such a regular cycle pattern, the predictions almost exactly match the targets.
Another noteworthy observation is the correlation between a clear temperature drop preceding ovulation and
the fertility rising.
This can be seen more prominently in Figure~\ref{fig:discussion_temperature_drop_fertility_prediction}.
The model seems to identify that this drop must be an indication for a heightened fertility.
However, this decrease in temperature is not visible for all cycles, and thus significantly impacts the prediction qualities of the model.
Figure~\ref{fig:discussion_spike_in_temperature_drop} shows an anomaly that results in a temperature spike during
the pre-ovulatory temperature dip.
This confuses the model into ending the fertile phase earlier.

This suggests that the model may interpret the termination of the pre-ovulatory temperature decline as a
signal that ovulation is imminent or has just occurred, thus marking the end of the fertile window.
Another pattern that is visible for many users is a cycle with no clear temperature drop.
Figure~\ref{fig:discussion_unclear_temperature_drop} shows such a curve.
Here, potentially unrelated circumstances lead to a drop in temperature that the model incorrectly identifies as fertile days.
The actual fertile days don't result in an equally visible temperature drop.
This indicates, that a clear temperature drop is a precursor to the ovulation and comes with the fertile days,
but its intensity varies between users and also between cycles of the same user.
Such a pattern can turn out to be a useful predictor for ovulation / fertility, but the models we trained don't seem
to be able to differentiate between ovulation-related and unrelated temperature drops.

This also points to a potential flaw in our workflow.
We don't yet have clinically accurate labels for the cycles that were used for the training of our models.
In fact, we cannot guarantee an ovulation, not even for the cycles with a clear temperature rise after the apparent ovulation.
It is not uncommon for women to have a clear temperature rise without an ovulation and vice versa,
to have an ovulation but no clear temperature rise.
The identified temperature drop and the fertility that seems to come with it could be a base for further research
with more accurate ovulation labeling.
It might be an indicator for a successful upcoming ovulation.


\begin{figure}[htbp]
    \centering
    \includegraphics[width=1.0\textwidth]{resources/figures/discussion/temperature_drop_fertility}
    \caption{
        Temperature rolling average and fertility-probability prediction with a clear correlation between the temperature drop
        pre-ovulation and the fertility. (Values are scaled features)
    }
    \label{fig:discussion_temperature_drop_fertility_prediction}
\end{figure}
\begin{figure}[htbp]
    \centering
    \includegraphics[width=1.0\textwidth]{resources/figures/discussion/temperature_spike_in_temperature_dop}
    \caption{
        Temperature rolling average and fertility-probability prediction with a disruptive temperature spike during the pre-ovulatory phase,
        leading to an early end of the predicted fertile window.
    }
    \label{fig:discussion_spike_in_temperature_drop}
\end{figure}
\begin{figure}[htbp]
    \centering
    \includegraphics[width=1.0\textwidth]{resources/figures/discussion/temperature_unclear_temperature_drop}
    \caption{
        Temperature rolling average and fertility-probability prediction with no clear temperature drop and a resulting
        incorrect prediction. (Values are scaled features)
    }
    \label{fig:discussion_unclear_temperature_drop}
\end{figure}

The results of the irregular vs regular cycle study, as shown in~\ref{subsubsec:regular_vs_irregular_cycles},
indicate, that the models seem to learn recurrent cycle pattern on a per-user basis.
We conclude this based on the fact, that regular cycle performance is significantly better than the performance on the irregular cycle set.
If there is a reliable indicator in the temperature data, or its ablations, then there would not be a large difference between the two subsets.
The temperature patterns should show up in both regular and irregular cycle users.
Thus, all models must learn or extract information beyond the actual temperature data, which they seem to rely on more than
the temperature related features.
This is likely information indicating some form of regularity, which the models then use to predict the targets for new cycles.
It would be interesting to take a close look at how and in what intensity the models use certain features,
and whether the performance changes upon omitting certain features.

\paragraph{Use-Case Study.}

The threshold has a large effect on the overall effectiveness of the different use cases.
For the contraception use case, changing the threshold doubles and even quadruples the pregnancy rate.
It has to be noted, that the contraception use case is naive use-case, where the woman does not take any other
measures next to the prediction of our models.
In a real-world scenario, this is largely not the case, and thus the pregnancy rates should be even lower.
A measure often taken is abstinence during the first cycle phase (luteal phase), which should significantly lower
unwanted pregnancies.
Even without additional measures, a threshold of 0.05 leads to an approximate \emph{Pearl-Index} (pregnancy rate over 1 year for 100 women)
of 4--5, which is significantly better than methods such as the contraceptive pill (7) or the condom (13) for a typical use case.
However, these numbers have to be taken with caution, as this is not an actual study, but a naive theoretical projection.
Further research is necessary to find more reliable results.

The pregnancy use case is harder to contextualize, as there are no comparable results for other methods,
and the actual pregnancy probability is subject to many more factors, we could not take into consideration for this study.




%
%In general, we expect the transformer based model to outperform the LSTM basd models, as they have proven to be
%a more effective for time-series analysis tasks especially for longer sequences.
%We also expect to find similar performance on irregular cycles compared to regular cycles,
%if the temperature is a reliable predictive indicator for the ovulation.
%If the performance on irregular cycles is significantly worse, and the predicted fertility curves see no upward trend around
%the actual fertility curves, we have no reason to believe that there is any predictive value in the temperature as is.
%
%Results do not show any clear indication that the temperature contains any patterns useful for the prediction of
%fertility or the ovulation.

% regarding the different performance increases on historical contexts: It might have been useful to take a closer look
% at the characteristics / differences for users with many cycles available

% talk about whether bbt / temperature can be used for such a task, discuss bbt doubt papers
% While previous work has argued against the predictive value of BBT~\cite{some_author_2010}, our findings suggest otherwise.
%Using continuous core body temperature data from 40,000 cycles, we demonstrate that temperature-based models can reliably detect ovulatory patterns, even in the presence of physiological noise or mild irregularity.
% explain the need for further medical interpretation of the results of either model


% regular vs irregular cycles: mean baselines perform well on regular cycles as expected, as they get closer to the regularities with every past cycle

% transformer can deal with more data and does also perform better for some cases
% lstm outperforms

%The findings do not show a clear indication, that the temperature can be used as a predictive target.
%The results of irregular cycles should be significantly better to infer, that there are usable patters in the
%temperature readings before the ovulation or the fertile days.
%
%For a practical use case of any model's prediction, a medical interpretation should be performed.
%While the results themselves can give a clear indication of both the fertility probability and whether the ovulation
%of a given cycle is already over for any given day, there a variety of external factors that should be taken into
%consideration for a direct output to the user.

\subsection{Limitations}\label{subsec:limiations}
This analysis reports aggregate test-set metrics without uncertainty quantification.
Because we did not retain per-user predictions, we cannot compute user-level bootstrap confidence intervals or perform paired significance testing.
As a result, apparent performance differences, especially small ones, may reflect sampling variability.
Future re-evaluation that stores per-user predictions will enable user-level bootstrapping, calibration assessment, and formal comparisons.

\subsection{Future Work}\label{sec:future_work}
There are several directions in which this study could be extended,
most of which were omitted due to time and resource constraints but represent valuable areas for future exploration.

One major area is feature selection.
The dataset used includes additional user-entered markers such as physiological signs (e.g., bleeding, illness, stress)
and external events (e.g., intercourse, pregnancy tests).
These markers were not included in the present analysis but may carry predictive value and could meaningfully improve model performance.
Similarly, the introduction of engineered or intermediate features, derived from raw inputs, may help models better
capture relevant patterns and temporal dependencies.
Additionally, the target features could be modelled in a better way, as, especially for long cycles,
there is a large imbalance of value distribution.
If a cycle has a length of 100 days with an ovulation at day 90, only 10\% of the ovulation-over targets are one.
The same applies to the fertility target, which will be zero throughout almost the whole sequence,
which will make it harder for the models to learn useful information.

Alternative target formulations could also be explored to better reflect the structure of the fertile window and ovulation.
For example, instead of predicting a daily fertility probability, models could aim to identify the absolute day of ovulation,
or estimate the time until the next (or since the last) ovulation event.

As noted in Section~\ref{subsubsec:data_labeling}, ovulation labels were assigned using a
retrospective algorithm trained on expert-annotated data.
Any inaccuracies in this algorithm propagate directly to the supervised learning labels.
Thus, improving prediction quality may require a newly labeled dataset—ideally combining expert review with algorithmic assistance.

Finally, while the models used in this study (LSTMs and Transformers) are well-established for time-series analysis,
they were not extensively customized.
Future work could involve tailoring architectures more specifically to the characteristics of menstrual cycle data.
Recent transformer variants designed for time series, such as \emph{TimeXer}~\cite{wang_timexer_2024} or \emph{MAMBA}~\cite{wang_is_2024},
may offer improved performance.
Alternatively, a custom architecture could be developed to better reflect the domain-specific structure of biological temperature data.

Future work may incorporate more advanced hyperparameter optimization techniques,
such as Bayesian Optimization, Genetic Algorithms, or Neural Architecture Search (NAS),
to better explore the joint parameter space in a more efficient and principled manner.

% add improvements in experiments / comparison
% add better decision base