alex/temperature-based-fertility-prediction-thesis
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fixes

Browse Source
This commit is contained in:
Alex Blank
2025-08-12 19:00:08 +00:00
parent 5655022d74
commit f9b1b63a12
3 changed files with 158 additions and 133 deletions
Download Patch File Download Diff File Expand all files Collapse all files
thesis/resources/figures/results/lstm_ov_over_results_by_input_resolution.png
BIN
View File
Binary file not shown.
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 389 KiB

After

Width:  |  Height:  |  Size: 389 KiB

thesis/sections/discussion.tex
+28 -25
View File
@@ -4,24 +4,24 @@
\section{Discussion}\label{sec:discussion}
In this study, we investigated the performance of different machine learning architectures on the task of fertility prediction,
with the aim to find a model that performs well for natural family planning and natural contraception on regular and irregular cycles.
Based on an extensive real-world database and established model architectures for timeseries analysis,
we expect our models to outperform the rule-based baselines.
We think, that for regular cycles, the performance difference will be lower than irregular cycles,
as the baseline models have no way of adapting to irregularities.
In general, we expect the transformer based model to outperform the LSTM basd models, as they have proven to be
a more effective for time-series analysis tasks especially for longer sequences.
We also expect to find similar performance on irregular cycles compared to regular cycles,
if the temperature is a reliable predictive indicator for the ovulation.
If the performance on irregular cycles is significantly worse, and the predicted fertility curves see no upward trend around
the actual fertility curves, we have no reason to believe that there is any predictive value in the temperature as is.
Results do not show any clear indication that the temperature contains any patterns useful for the prediction of
fertility or the ovulation.
%In this study, we investigated the performance of different machine learning architectures on the task of fertility prediction,
%with the aim to find a model that performs well for natural family planning and natural contraception on regular and irregular cycles.
%
%
%Based on an extensive real-world database and established model architectures for timeseries analysis,
%we expect our models to outperform the rule-based baselines.
%We think, that for regular cycles, the performance difference will be lower than irregular cycles,
%as the baseline models have no way of adapting to irregularities.
%
%In general, we expect the transformer based model to outperform the LSTM basd models, as they have proven to be
%a more effective for time-series analysis tasks especially for longer sequences.
%We also expect to find similar performance on irregular cycles compared to regular cycles,
%if the temperature is a reliable predictive indicator for the ovulation.
%If the performance on irregular cycles is significantly worse, and the predicted fertility curves see no upward trend around
%the actual fertility curves, we have no reason to believe that there is any predictive value in the temperature as is.
%
%Results do not show any clear indication that the temperature contains any patterns useful for the prediction of
%fertility or the ovulation.
% talk about whether bbt / temperature can be used for such a task, discuss bbt doubt papers
@@ -30,14 +30,17 @@ fertility or the ovulation.
% explain the need for further medical interpretation of the results of either model
The findings do not show a clear indication, that the temperature can be used as a predictive target.
The results of irregular cycles should be significantly better to infer, that there are usable patters in the
temperature readings before the ovulation or the fertile days.
% transformer can deal with more data and does also perform better for some cases
% lstm outperforms
For a practical use case of any model's prediction, a medical interpretation should be performed.
While the results themselves can give a clear indication of both the fertility probability and whether the ovulation
of a given cycle is already over for any given day, there a variety of external factors that should be taken into
consideration for a direct output to the user.
%The findings do not show a clear indication, that the temperature can be used as a predictive target.
%The results of irregular cycles should be significantly better to infer, that there are usable patters in the
%temperature readings before the ovulation or the fertile days.
%
%For a practical use case of any model's prediction, a medical interpretation should be performed.
%While the results themselves can give a clear indication of both the fertility probability and whether the ovulation
%of a given cycle is already over for any given day, there a variety of external factors that should be taken into
%consideration for a direct output to the user.
\section{Future Work}\label{sec:future_work}
thesis/sections/results.tex
+130 -108
View File
@@ -3,77 +3,39 @@
\section{Results}\label{sec:results}
We begin by comparing the overall performance of all trained models and baselines using four key evaluation metrics:
mean absolute error (MAE), mean squared error (MSE), and the coefficient of determination (\(R^2\)) for the regression target,
as well as the metrics for their respective sub-intervals.
Figure~\ref{fig:results_performance_overview_by_model_type} provides an overview of these metrics across all model types.
Overall, transformer-based models consistently outperformed LSTM variants and baseline methods across most evaluation criteria.
Among the baselines, [e.g., "the rule-based method"] showed the weakest performance,
while the [e.g., "windowed logistic regression"] performed competitively in certain contexts.
Differences across models were most pronounced in MSE and \(R^2\),
indicating that advanced architectures better captured higher-order dynamics and reduced large prediction errors.
The previous section detailed the design and implementation of our fertility prediction pipeline,
including data preprocessing, feature engineering, input encoding, and the development of several deep learning architectures.
We now present the results of our evaluation, focusing on the predictive accuracy of the proposed models across different temporal resolutions,
cycle types and use cases.
Model performance is assessed using both overall metrics and biologically targeted subintervals,
allowing for a nuanced comparison of approaches and their practical relevance to real-time fertility forecasting.
Additionally, model performance is compared to the three baseline models introduced.
%
%We begin by comparing the overall performance of all trained models and baselines using four key evaluation metrics:
%mean absolute error (MAE) and mean squared error (MSE), as well as the metrics for their respective sub-intervals.
%
%Overall, transformer-based models consistently outperformed LSTM variants and baseline methods across most evaluation criteria.
%Among the baselines, [e.g., "the rule-based method"] showed the weakest performance,
%while the [e.g., "windowed logistic regression"] performed competitively in certain contexts.
%Differences across models were most pronounced in MSE and \(R^2\),
%indicating that advanced architectures better captured higher-order dynamics and reduced large prediction errors.
%
%The previous section detailed the design and implementation of our fertility prediction pipeline,
%including data preprocessing, feature engineering, input encoding, and the development of several deep learning architectures.
%We now present the results of our evaluation, focusing on the predictive accuracy of the proposed models across different temporal resolutions,
%cycle types and use cases.
%Model performance is assessed using both overall metrics and biologically targeted subintervals,
%allowing for a nuanced comparison of approaches and their practical relevance to real-time fertility forecasting.
%Additionally, model performance is compared to the three baseline models introduced.
%
% provide information about the training behaviour and statistic of the different models??
\subsection{Overall Model Performance Across Architectures}\label{subsec:overall_model_performance_across_architectures}
\begin{figure}
\centering
\includegraphics[width=0.9\textwidth]{resources/figures/results/model_performance_overview}
\caption{Overview of the performances of all model types including the baselines on 4 selected performance metrics.}
\label{fig:results_performance_overview_by_model_type}
\end{figure}
\subsection{Fertility Probability Prediction Accuracy}\label{subsec:fertility_probability_precition_accuracy}
\begin{landscape}
\begin{table}[ht]
\centering
\caption{Model comparison for fertility prediction using MAE and MSE}
\begin{adjustbox}{max width=\linewidth}
\begin{tabular}{lllcccccc}
\toprule
\textbf{Model} & \textbf{Window} & \textbf{Daily} &
\textbf{MAE$_{fert}$} & \textbf{MAE$_{fert,during}$} & \textbf{MAE$_{fert,non}$} &
\textbf{MSE$_{fert}$} & \textbf{MSE$_{fert,during}$} & \textbf{MSE$_{fert,non}$} \\
\midrule
ModelA & 7 & Yes & 0.67 & 0.59 & 0.73 & 0.89 & 0.82 & 0.95 \\
ModelB & 14 & No & 0.65 & 0.58 & 0.71 & 0.87 & 0.80 & 0.93 \\
% More rows...
\bottomrule
\end{tabular}
\end{adjustbox}
\label{tab:fertility_comparison}
\end{table}
\end{landscape}
% show why I selected the individual input configs for model config training
% selected by best mse fertility, use 2nd best, as it provides basically the same performance, but more input data for more complex model configs
\subsubsection{Fertility Probability Prediction}\label{subsubsec:fertility_probability_prediction}
\subsubsection{Impact of Input Window Length}\label{subsubsec:fert_impact_of_historical_context}
\begin{figure}
\centering
\includegraphics[width=1.0\textwidth]{resources/figures/results/lstm_fertility_results_by_input_length}
\caption{Performance of LSTM models with different input window lengths and fixes input resolutions for the fertility probability target}
\label{fig:results_performance_lstm_fertility_input_window_length}
\end{figure}
\begin{landscape}
\begin{table}
% \begin{tabular}{lrrrrrrr}
\small
\begin{tabularx}{\linewidth}{l*{6}{X}}
\toprule
\multirow{2}{*}{Input-Length in Days} & \multicolumn{3}{c}{MAE} & \multicolumn{3}{c}{MSE} \\
@@ -95,9 +57,9 @@ Additionally, model performance is compared to the three baseline models introdu
160 & \underline{0.0411} & 0.0882 & \underline{0.0218} & 0.0045 & 0.0116 & \underline{0.0016} \\
\midrule
\multicolumn{7}{c}{\textbf{Convolution-LSTM}} \\ \midrule
10 & 0.0435 & \underline{0.0954} & 0.0224 & 0.0050 & 0.0133 & 0.0017 \\
20 & \underline{0.0414} & 0.0996 & \underline{0.0179} & 0.0050 & 0.0145 & \underline{\textbf{0.0013}} \\
40 & 0.0394 & 0.0911 & 0.0184 & \underline{0.0045} & \underline{0.0122} & 0.0014 \\
10 & 0.0435 & 0.0954 & 0.0224 & 0.0050 & 0.0133 & 0.0017 \\
20 & 0.0414 & 0.0996 & \underline{0.0179} & 0.0050 & 0.0145 & \underline{\textbf{0.0013}} \\
40 & \underline{0.0394} & \underline{0.0911} & 0.0184 & \underline{0.0045} & \underline{0.0122} & 0.0014 \\
80 & 0.0481 & 0.1005 & 0.0278 & 0.0054 & 0.0146 & 0.0019 \\
160 & 0.0505 & 0.1067 & 0.0290 & 0.0060 & 0.0166 & 0.0020 \\
\midrule
@@ -108,26 +70,33 @@ Additionally, model performance is compared to the three baseline models introdu
80 & 0.0414 & 0.0932 & 0.0210 & 0.0048 & 0.0128 & 0.0018 \\
160 & 0.0523 & 0.1018 & 0.0337 & 0.0059 & 0.0148 & 0.0026 \\
\bottomrule
% \end{tabular}
\end{tabularx}
\caption{Evaluation Metrics for the Fertility-Probability target for each model architecture across different input lenghts.
Underlined values indicate the best value per metric for a model and bold and underlined indicate global best values for a metric.}
\label{tab:fertility_results}
\caption{Evaluation Metrics for the Fertility-Probability Target across Different Model Architectures and Input Lengths on a fixed Input Resolution of 12 Values per Day.
\underline{Underlined} values represent the best value for each metric within a model.
\textbf{\underline{Bold + Underlined}} values represent the global best values across all models for a given metric.}
\label{tab:fertility_results_by_window_length}
\end{table}
\end{landscape}
For the LSTM model, the error metrics show a small improvement with larger input window size.
Also, the MSE metric is numerically smaller than the MAE, which suggests, that the errors are rather small and tightly clustered with few large outliers.
For the MSE metric, the non-fertile phase seem to behave inversely proportional to the other both the fertile phase error and the overall error.
This is not the case for the MAE metric.
Overall, the improvement over a growing input window size is very small
Table~\ref{tab:fertility_results_by_window_length} presents the performance of different model architectures across various metrics and input window lengths.
\begin{figure}
\centering
\includegraphics[width=1.0\textwidth]{resources/figures/results/transformer_fertility_results_by_input_length}
\caption{Performance of Transformer models with different input window lengths and fixes input resolutions for the fertility probability target}
\label{fig:results_performance_transformer_fertility_input_window_length}
\end{figure}
It is evident that for both the LSTM and Transformer models, \textbf{longer input windows generally yield better performance}.
Specifically, the LSTM achieves the best results for four out of six metrics with the 160-day window.
The only exception is the error metrics for non-fertile days, where shorter input windows perform better.
Notably, the LSTM achieves the \textbf{global best performance} for four out of six metrics, including MAE and MSE on non-fertile and fertile days.
For the Transformer model, the best performance is observed with the 160-day input window in three out of six metrics.
However, for non-fertile day metrics, performance improves with shorter input windows.
It is consistently outperformed by the LSTM model.
The \textbf{convolutional models} tend to perform best with \textbf{medium-length input windows} (20 to 40 days).
The \textbf{convolutional LSTM} shows its best performance with a 40-day window for most metrics,
except for non-fertile day errors, where the 20-day window yields smaller errors.
It also achieves the \textbf{global best performance} for non-fertile day MSE with the 20-day window.
Similarly, the \textbf{convolutional Transformer} excels with 40-day windows for the overall and non-fertile-day metrics,
while the best fertile-day metrics are seen with 20-day windows.
The \textbf{convolutional Transformer} also achieves the \textbf{global best performance} for 40-day windows in both MAE and MSE\@.
In all but non-fertile day metrics, the \textbf{convolutional Transformer} outperforms the LSTM variant.
\subsubsection{Impact of Input Resolution}\label{subsubsec:fert_impact_of_input_resolution}
@@ -138,6 +107,44 @@ The plot shows the MSE values for three target categories: non-fertile days, ove
Notably, the MSE for non-fertile days appears to behave inversely in relation to the other two metrics.
However, there is no consistent upward or downward trend when changing the input window length with a fixed input resolution.
\begin{landscape}
\begin{table}
\small
\begin{tabularx}{\linewidth}{l*{6}{X}}
\toprule
\multirow{2}{*}{Values Per Day} & \multicolumn{3}{c}{MAE} & \multicolumn{3}{c}{MSE} \\
\cmidrule(r){2-4} \cmidrule(r){5-7}
& Fertility Overall & Fertile Days & Non-Fertile Days & Fertility Overall & Fertile Days & Non-Fertile Days \\
\midrule
\multicolumn{7}{c}{\textbf{LSTM}} \\
\midrule
1 & 0.0471 & 0.0987 & 0.0220 & 0.0062 & 0.0145 & 0.0022 \\
2 & 0.0457 & 0.0962 & 0.0231 & 0.0052 & 0.0133 & 0.0016 \\
4 & 0.0421 & 0.0898 & 0.0223 & \underline{\textbf{0.0046}}& 0.0116 & 0.0018 \\
12 & 0.0421 & 0.0994 & \underline{\textbf{0.0180}}& 0.0052 & 0.0144 & \underline{\textbf{0.0013}}\\
24 & 0.0419 & 0.0949 & 0.0201 & 0.0050 & 0.0132 & 0.0017 \\
48 & \underline{\textbf{0.0402}}& 0.0929 & 0.0182 & 0.0049 & 0.0126 & 0.0018 \\
72 & 0.0433 & 0.0972 & 0.0216 & 0.0052 & 0.0137 & 0.0018 \\
288 & 0.0410 & \underline{\textbf{0.0872}}& 0.0223 & 0.0049 & \underline{\textbf{0.0110}}& 0.0025 \\
\midrule
\multicolumn{7}{c}{\textbf{Transformer}} \\
\midrule
1 & 0.0541 & 0.1015 & 0.0313 & 0.0063 & 0.0146 & 0.0023 \\
2 & 0.0468 & 0.0946 & 0.0259 & 0.0054 & 0.0128 & 0.0021 \\
4 & 0.0456 & \underline{0.0891}& 0.0277 & 0.0050 & \underline{0.0115}& 0.0023 \\
12 & 0.0472 & 0.0949 & 0.0274 & 0.0051 & 0.0133 & 0.0018 \\
24 & 0.0502 & 0.0960 & 0.0323 & 0.0052 & 0.0133 & 0.0021 \\
48 & 0.0447 & 0.0912 & \underline{0.0254}& \underline{0.0048}& 0.0122 & 0.0017 \\
72 & 0.0468 & 0.0975 & 0.0257 & 0.0051 & 0.0140 & \underline{0.0014} \\
288 & \underline{0.0449}& 0.0937 & 0.0257 & \underline{0.0048}& 0.0128 & 0.0017 \\
\bottomrule
\end{tabularx}
\caption{Evaluation Metrics for the Fertility-Probability Target across Different Model Architectures and Input Resolutions on a fixed Input-Window-Length of 20 Days.
\underline{Underlined} values represent the best value for each metric within a model.
\textbf{\underline{Bold + Underlined}} values represent the global best values across all models for a given metric.}
\label{tab:fertility_results_by_window_resolution}
\end{table}
\end{landscape}
\begin{figure}
\centering
\includegraphics[width=1.0\textwidth]{resources/figures/results/transformer_fertility_results_by_input_resolution}
@@ -156,8 +163,14 @@ However, there is no consistent upward or downward trend when changing the input
\subsubsection{Comparison with Baselines}\label{subsubsec:fert_comparison_with_baselines}
\subsection{Ovulation-Over Prediction}\label{subsubsec:ov_over_prediction}
\subsubsection{Performance around Ovulation}\label{subsubsec:ov_over_performance_around_ovulation}
\subsubsection{Impact of Input Window Length}\label{subsubsec:ov_over_impact_of_historical_context}
\begin{landscape}
\begin{table}
\small
\begin{tabularx}{\linewidth}{l*{6}{X}}
\toprule
\multirow{2}{*}{Input-Length in Days} & \multicolumn{3}{c}{MAE} & \multicolumn{3}{c}{MSE} \\
@@ -197,45 +210,54 @@ However, there is no consistent upward or downward trend when changing the input
160 & 0.2440 & \underline{0.1051} & 0.3117 & 0.1403 & \underline{0.0286} & 0.1946 \\
\bottomrule
\end{tabularx}
\caption{Evaluation Metrics for the Ovulation-Over target for each model architecture across different input lenghts.
Underlined values indicate the best value per metric for a model and bold and underlined indicate global best values for a metric.}
\label{tab:ov_over_results}
\caption{Evaluation Metrics for the Ovulation-Over Target across Different Model Architectures and Input Lengths on a fixed Input Resolution of 12 Values per Day.
\underline{Underlined} values represent the best value for each metric within a model.
\textbf{\underline{Bold + Underlined}} values represent the global best values across all models for a given metric.}
\label{tab:ov_over_results_by_window_length}
\end{table}
\end{landscape}
\subsubsection{Performance around Ovulation}\label{subsubsec:ov_over_performance_around_ovulation}
\subsubsection{Impact of Input Window Length}\label{subsubsec:ov_over_impact_of_historical_context}
\begin{figure}
\centering
\includegraphics[width=1.0\textwidth]{resources/figures/results/lstm_ov_over_results_by_input_length}
\caption{Performance of LSTM models with different input window lengths and fixes input resolutions for the ovulation-over target}
\label{fig:results_performance_lstm_ov_over_input_window_length}
\end{figure}
\begin{figure}
\centering
\includegraphics[width=1.0\textwidth]{resources/figures/results/transformer_ov_over_results_by_input_length}
\caption{Performance of Transformer models with different input window lengths and fixes input resolutions for the ovulation-over target}
\label{fig:results_performance_transformer_ov_over_input_window_length}
\end{figure}
\subsubsection{Impact of Input Resolution}\label{subsubsec:ov_over_impact_of_input_resolution}
\begin{figure}
\centering
\includegraphics[width=1.00\textwidth]{resources/figures/results/lstm_ov_over_results_by_input_resolution}
\caption{Performance of LSTM models with different input resolutions and fixes input window lengths for the ovulation-over target}
\label{fig:results_performance_lstm_ov_over_input_resolution}
\end{figure}
\begin{figure}
\centering
\includegraphics[width=1.0\textwidth]{resources/figures/results/transformer_ov_over_results_by_input_resolution}
\caption{Performance of transformer models with different input window lengths and fixes input resolutions for the ovulation-over target}
\label{fig:results_performance_transformer_ov_over_input_resolution}
\end{figure}
\begin{landscape}
\begin{table}
\small
\begin{tabularx}{\linewidth}{l*{6}{X}}
\toprule
\multirow{2}{*}{Values Per Day} & \multicolumn{3}{c}{MAE} & \multicolumn{3}{c}{MSE} \\
\cmidrule(r){2-4} \cmidrule(r){5-7}
& OV-Over Overall & OV-Over Before OV & OV-Over After OV & OV-Over Overall & OV-Over Before OV & OV-Over After OV \\
\midrule
\multicolumn{7}{c}{\textbf{LSTM}} \\
\midrule
1 & 0.1691 & 0.1280 & 0.1819 & 0.0914 & 0.0411 & 0.1099 \\
2 & 0.1431 & 0.1239 & 0.1471 & 0.0747 & 0.0442 & 0.0849 \\
4 & 0.1344 & 0.1130 & 0.1417 & 0.0680 & 0.0371 & 0.0807 \\
12 & \underline{0.1153} & \underline{\textbf{0.0745}} & 0.1312 & 0.0641 & \underline{\textbf{0.0212}} & 0.0822 \\
24 & 0.1192 & 0.0975 & 0.1240 & \underline{0.0633} & 0.0288 & 0.0755 \\
48 & 0.1463 & 0.2493 & \underline{0.0973} & 0.0768 & 0.1203 & \underline{\textbf{0.0550}} \\
72 & 0.1587 & 0.1113 & 0.1844 & 0.0857 & 0.0341 & 0.1126 \\
288 & 0.1353 & 0.1801 & 0.1169 & 0.0726 & 0.0799 & 0.0713 \\
\midrule
\multicolumn{7}{c}{\textbf{Transformer}} \\
\midrule
1 & 0.1687 & 0.1582 & 0.1709 & 0.0883 & 0.0578 & 0.0996 \\
2 & 0.1469 & 0.0895 & 0.1676 & 0.0823 & 0.0260 & 0.1044 \\
4 & 0.1282 & 0.0999 & 0.1393 & 0.0704 & 0.0329 & 0.0862 \\
12 & 0.1204 & \underline{0.0771}& 0.1366 & 0.0690 & \underline{0.0255}& 0.0864 \\
24 & 0.1952 & 0.1649 & 0.2102 & 0.0904 & 0.0638 & 0.1040 \\
48 & 0.1137 & 0.1159 & 0.1073 & 0.0610 & 0.0480 & 0.0629 \\
72 & \underline{\textbf{0.1041}}& 0.1180 & \underline{\textbf{0.0947}}& \underline{\textbf{0.0585}}& 0.0538 & 0.0581 \\
288 & 0.1319 & 0.1879 & 0.1097 & 0.0617 & 0.0769 & \underline{0.0578}\\
\bottomrule
\end{tabularx}
\caption{Evaluation Metrics for the Ovulation-Over Target across Different Model Architectures and Input Resolutions on a fixed Input-Window-Length of 20 Days.
\underline{Underlined} values represent the best value for each metric within a model.
\textbf{\underline{Bold + Underlined}} values represent the global best values across all models for a given metric.}
\label{tab:ov_over_results_by_resolution}
\end{table}
\end{landscape}
\subsubsection{Comparison with Baselines}\label{subsubsec:ov_over_comparison_with_baselines}