temperature-based-fertility-prediction-thesis/thesis/sections/results.tex

%! Author = alex
%! Date = 3/6/25


\section{Results}\label{sec:results}

We begin by comparing the overall performance of all trained models and baselines using four key evaluation metrics:
mean absolute error (MAE), mean squared error (MSE), and the coefficient of determination (\(R^2\)) for the regression target,
as well as the metrics for their respective sub-intervals.
Figure~\ref{fig:results_performance_overview_by_model_type} provides an overview of these metrics across all model types.

Overall, transformer-based models consistently outperformed LSTM variants and baseline methods across most evaluation criteria.
Among the baselines, [e.g., "the rule-based method"] showed the weakest performance,
while the [e.g., "windowed logistic regression"] performed competitively in certain contexts.
Differences across models were most pronounced in MSE and \(R^2\),
indicating that advanced architectures better captured higher-order dynamics and reduced large prediction errors.

The previous section detailed the design and implementation of our fertility prediction pipeline,
including data preprocessing, feature engineering, input encoding, and the development of several deep learning architectures.
We now present the results of our evaluation, focusing on the predictive accuracy of the proposed models across different temporal resolutions,
cycle types and use cases.
Model performance is assessed using both overall metrics and biologically targeted subintervals,
allowing for a nuanced comparison of approaches and their practical relevance to real-time fertility forecasting.
Additionally, model performance is compared to the three baseline models introduced.

% provide information about the training behaviour and statistic of the different models??

\subsection{Overall Model Performance Across Architectures}\label{subsec:overall_model_performance_across_architectures}

\begin{figure}
    \centering
    \includegraphics[width=0.9\textwidth]{resources/figures/results/model_performance_overview}
    \caption{Overview of the performances of all model types including the baselines on 4 selected performance metrics.}
    \label{fig:results_performance_overview_by_model_type}
\end{figure}

\subsection{Fertility Probability Prediction Accuracy}\label{subsec:fertility_probability_precition_accuracy}

\begin{landscape}
    \begin{table}[ht]
        \centering
        \caption{Model comparison for fertility prediction using MAE and MSE}
        \begin{adjustbox}{max width=\linewidth}
            \begin{tabular}{lllcccccc}
                \toprule
                \textbf{Model} & \textbf{Window} & \textbf{Daily} &
                \textbf{MAE$_{fert}$} & \textbf{MAE$_{fert,during}$} & \textbf{MAE$_{fert,non}$} &
                \textbf{MSE$_{fert}$} & \textbf{MSE$_{fert,during}$} & \textbf{MSE$_{fert,non}$} \\
                \midrule
                ModelA & 7  & Yes & 0.67 & 0.59 & 0.73 & 0.89 & 0.82 & 0.95 \\
                ModelB & 14 & No  & 0.65 & 0.58 & 0.71 & 0.87 & 0.80 & 0.93 \\
% More rows...
                \bottomrule
            \end{tabular}
        \end{adjustbox}
        \label{tab:fertility_comparison}
    \end{table}
\end{landscape}


% show why I selected the individual input configs for model config training
% selected by best mse fertility, use 2nd best, as it provides basically the same performance, but more input data for more complex model configs

\subsubsection{Fertility Probability Prediction}\label{subsubsec:fertility_probability_prediction}

\subsubsection{Impact of Input Window Length}\label{subsubsec:fert_impact_of_historical_context}
\begin{figure}
    \centering
    \includegraphics[width=1.0\textwidth]{resources/figures/results/lstm_fertility_results_by_input_length}
    \caption{Performance of LSTM models with different input window lengths and fixes input resolutions for the fertility probability target}
    \label{fig:results_performance_lstm_fertility_input_window_length}
\end{figure}
\begin{tabular}{lrrrrrrr}
    \toprule
    {} & MAE Fertility Overall & MAE Fertile Days & MAE Non-Fertile Days & MSE Fertility Overall & MSE Fertile Days & MSE Non-Fertile Days & Input-Window-Length in Days \\
    \midrule
    4  & 0.0378                & 0.0834           & 0.0192               & 0.0043                & 0.0102           & 0.0019               & 160                         \\
    5  & 0.0399                & 0.0857           & 0.0212               & 0.0045                & 0.0105           & 0.0021               & 10                          \\
    11 & 0.0421                & 0.0994           & 0.0180               & 0.0052                & 0.0144           & 0.0013               & 20                          \\
    13 & 0.0420                & 0.1006           & 0.0176               & 0.0053                & 0.0147           & 0.0014               & 80                          \\
    14 & 0.0406                & 0.0938           & 0.0184               & 0.0049                & 0.0128           & 0.0016               & 40                          \\
    \bottomrule
\end{tabular}


For the LSTM model, the error metrics show a small improvement with larger input window size.
Also, the MSE metric is numerically smaller than the MAE, which suggests, that the errors are rather small and tightly clustered with few large outliers.
For the MSE metric, the non-fertile phase seem to behave inversely proportional to the other both the fertile phase error and the overall error.
This is not the case for the MAE metric.
Overall, the improvement over a growing input window size is very small

\begin{figure}
    \centering
    \includegraphics[width=1.0\textwidth]{resources/figures/results/transformer_fertility_results_by_input_length}
    \caption{Performance of Transformer models with different input window lengths and fixes input resolutions for the fertility probability target}
    \label{fig:results_performance_transformer_fertility_input_window_length}
\end{figure}

\subsubsection{Impact of Input Resolution}\label{subsubsec:fert_impact_of_input_resolution}

Figure~\ref{fig:results_performance_lstm_fertility_input_resolution} presents the results for different input resolutions used in the LSTM models,
alongside the corresponding mean squared error (MSE) metrics.
The plot shows the MSE values for three target categories: non-fertile days, overall MSE, and fertile days MSE\@.

Notably, the MSE for non-fertile days appears to behave inversely in relation to the other two metrics.
However, there is no consistent upward or downward trend when changing the input window length with a fixed input resolution.

\begin{figure}
    \centering
    \includegraphics[width=1.0\textwidth]{resources/figures/results/transformer_fertility_results_by_input_resolution}
    \caption{Performance of transformer models with different input window lengths and fixes input resolutions for the fertility probability target}
    \label{fig:results_performance_transformer_fertility_input_resolution}
\end{figure}


\begin{figure}
    \centering
    \includegraphics[width=1.00\textwidth]{resources/figures/results/lstm_fertility_results_by_input_resolution}
    \caption{Performance of LSTM models with different input resolutions and fixes input window lengths for the fertility probability target}
    \label{fig:results_performance_lstm_fertility_input_resolution}
\end{figure}

\subsubsection{Comparison with Baselines}\label{subsubsec:fert_comparison_with_baselines}

\subsection{Ovulation-Over Prediction}\label{subsubsec:ov_over_prediction}

\subsubsection{Performance around Ovulation}\label{subsubsec:ov_over_performance_around_ovulation}

\subsubsection{Impact of Input Window Length}\label{subsubsec:ov_over_impact_of_historical_context}

\begin{figure}
    \centering
    \includegraphics[width=1.0\textwidth]{resources/figures/results/lstm_ov_over_results_by_input_length}
    \caption{Performance of LSTM models with different input window lengths and fixes input resolutions for the ovulation-over target}
    \label{fig:results_performance_lstm_ov_over_input_window_length}
\end{figure}

\begin{figure}
    \centering
    \includegraphics[width=1.0\textwidth]{resources/figures/results/transformer_ov_over_results_by_input_length}
    \caption{Performance of Transformer models with different input window lengths and fixes input resolutions for the ovulation-over target}
    \label{fig:results_performance_transformer_ov_over_input_window_length}
\end{figure}

\subsubsection{Impact of Input Resolution}\label{subsubsec:ov_over_impact_of_input_resolution}


\begin{figure}
    \centering
    \includegraphics[width=1.00\textwidth]{resources/figures/results/lstm_ov_over_results_by_input_resolution}
    \caption{Performance of LSTM models with different input resolutions and fixes input window lengths for the ovulation-over target}
    \label{fig:results_performance_lstm_ov_over_input_resolution}
\end{figure}
\begin{figure}
    \centering
    \includegraphics[width=1.0\textwidth]{resources/figures/results/transformer_ov_over_results_by_input_resolution}
    \caption{Performance of transformer models with different input window lengths and fixes input resolutions for the ovulation-over target}
    \label{fig:results_performance_transformer_ov_over_input_resolution}
\end{figure}

\subsubsection{Comparison with Baselines}\label{subsubsec:ov_over_comparison_with_baselines}

\subsection{Stratified Analysis}\label{subsec:stratified_analysis}

\subsubsection{Regular vs Irregular Cycles}\label{subsubsec:regular_vs_irregular_cycles}

\subsubsection{Influence of User History Depth}\label{subsubsec:influence_of_past_user_data}

\subsection{Use-Case Evaluation Results}\label{subsec:use_case_evaluation_results}

\subsubsection{Contraception Use-Case Results}\label{subsubsec:use_case_contraception_results}

\subsubsection{Pregnancy Use-Case Results}\label{subsubsec:use_case_pregnancy_results}

\subsection{Summary of Key Findings}\label{subsec:summary_of_key_findings}