added user specific metrics

thesis/sections/methodology.tex

@@ -573,6 +573,21 @@ The batch size was capped at 2048 to avoid OOM errors during data preparation.
 To meaningfully compare model performance, we define a set of metrics that capture both overall accuracy and behavior at key points in the prediction sequence.
 This includes metrics for different temporal segments, enabling a more detailed understanding of model strengths and limitations.
 
+\subsubsection{Baseline Comparisons}\label{subsubsec:baseline_comparisons}
+
+To place model results into context, a set of simple baseline models was established.
+These models rely on minimal assumptions and serve to evaluate whether the more complex models actually learn meaningful patterns beyond basic cycle regularities.
+
+Three types of baselines are considered:
+\begin{itemize}
+    \item \textbf{Population-Mean OV Day}: predicts the mean ovulation day across the full population.
+    \item \textbf{User-Mean OV Day}: predicts the user’s average ovulation day; uses the population mean for the first cycle.
+    \item \textbf{Previous OV Day}: predicts the ovulation day from the previous cycle; uses the population mean for the first cycle.
+\end{itemize}
+
+These baselines also help assess performance on regular menstrual patterns, something often overlooked in related work (see Section~\ref{sec:related_work}).
+In many cases, it remains unclear whether proposed models genuinely outperform such simple heuristics.
+
 \subsubsection{Evaluation Metrics}\label{subsubsec:evaluation_metrics}
 
 The base metric used for all categories is the mean absolute error (MAE),
@@ -682,20 +697,80 @@ For a user to be included in the analysis, they must have at least five complete
 Additionally, ovulation must occur no later than cycle day 150, as later values likely indicate measurement errors or
 biologically atypical cases that fall outside the scope of this study.
 
-\subsubsection{Baseline Comparisons}\label{subsubsec:baseline_comparisons}
+\subsubsection{Use Case Evaluation}
+We further evaluate the two distinct use cases introduced in Section~\ref{subsubsec:use_cases_of_ovulation_prediction}.
+For this purpose, two specialized evaluation algorithms were developed,
+enabling comparability between models and providing interpretable performance metrics for each scenario.
 
-To place model results into context, a set of simple baseline models was established.
-These models rely on minimal assumptions and serve to evaluate whether the more complex models actually learn meaningful patterns beyond basic cycle regularities.
+\paragraph{Contraception Use-Case:}
+For evaluating contraceptive effectiveness, we developed an algorithm inspired by the classical \emph{Pearl Index},
+initially proposed by~\citeauthor{pearl_factors_1933} in~\citeyear{pearl_factors_1933}.
 
-Three types of baselines are considered:
+\begin{figure}[htbp]
+    \centering
+    \includegraphics[width=0.9\textwidth]{resources/figures/methodology/methodology_use_case_contraception_decision_diagram}
+    \caption{
+        Decision diagram outlining the evaluation procedure for the contraception use case.
+    }
+    \label{fig:methodology_use_case_contraception_decision_diagram}
+\end{figure}
+
+Figure~\ref{fig:methodology_use_case_contraception_decision_diagram} illustrates the decision logic of the evaluation algorithm.
+The fertility threshold can be adjusted, as will be explored in Section~\ref{subsec:use_case_evaluation_results}.
+A day-specific probability of intercourse is computed for each user based on age distributions reported by~\cite{twenge_declines_2017}.
+We assume, that the users don't have any health-related or non-health-related issues affecting fertility.
+If a user's age is unknown, it is randomly drawn from the overall dataset distribution.
+Only users with at least one continuous year of data are included.
+
+Each day of data for a full year is categorized by the algorithm into one of the following outcomes:
 \begin{itemize}
-    \item \textbf{Population-Mean OV Day}: predicts the mean ovulation day across the full population.
-    \item \textbf{User-Mean OV Day}: predicts the user’s average ovulation day; uses the population mean for the first cycle.
-    \item \textbf{Previous OV Day}: predicts the ovulation day from the previous cycle; uses the population mean for the first cycle.
+    \item \emph{No Sex}: no intercourse occurred.
+    \item \emph{Correct Denial}: fertility prediction correctly indicated abstinence during a fertile period.
+    \item \emph{Incorrect Denial}: fertility prediction incorrectly indicated abstinence during an infertile period.
+    \item \emph{Pregnancy}: fertility prediction allowed intercourse during a potentially fertile period, and it led to a pregnancy.
+    \item \emph{No Pregnancy}: fertility prediction allowed intercourse during an infertile or potentially fertile period, but it did not lead to a pregnancy.
 \end{itemize}
 
-These baselines also help assess performance on regular menstrual patterns, something often overlooked in related work (see Section~\ref{sec:related_work}).
-In many cases, it remains unclear whether proposed models genuinely outperform such simple heuristics.
+This categorization captures both the contraceptive accuracy (avoiding pregnancy) and the practicality
+(minimizing unnecessary abstinence) of the predictive algorithm.
+An algorithm achieving perfect contraceptive accuracy by always recommending abstinence would score highly but
+significantly limit user acceptability and utility.
+
+\paragraph{Pregnancy Use-Case:}
+
+For the pregnancy use-case, we developed a complementary algorithm to evaluate model performance for users attempting to conceive.
+
+\begin{figure}[htbp]
+    \centering
+    \includegraphics[width=0.9\textwidth]{resources/figures/methodology/methodology_use_case_pregnancy_decision_diagram}
+    \caption{
+        Decision diagram outlining the evaluation procedure for the pregnancy use case.
+    }
+    \label{fig:methodology_use_case_pregnancy_decision_diagram}
+\end{figure}
+
+Figure~\ref{fig:methodology_use_case_pregnancy_decision_diagram} illustrates the decision logic of this algorithm.
+The fertility threshold is adjustable and is explored further in Section~\ref{subsec:use_case_evaluation_results}.
+Since sexual intercourse frequency differs slightly for couples trying to conceive~\cite{gaskins_predictors_2018},
+we assume an average frequency of six times per month.
+We derive a daily probability of intercourse based on remaining fertile days predicted for the month,
+ensuring that intercourse frequency averages out to this monthly rate.
+
+We assume no health-related fertility impairments for comparative simplicity,
+though we acknowledge that real-world fertility is influenced by numerous complex factors.
+Similar to the contraception scenario, only users with at least one continuous year of data are considered.
+
+Each day in a full year is classified into one of the following categories:
+\begin{itemize}
+    \item \emph{No Sex}: Day predicted as fertile, but no intercourse occurred.
+    \item \emph{Pregnancy}: Correct fertile prediction, intercourse occurred, resulting in pregnancy.
+    \item \emph{No Pregnancy}: Correct fertile prediction, intercourse occurred, but no pregnancy occurred.
+    \item \emph{Incorrect Deferral}: Incorrect non-fertile prediction, actual fertility was above threshold.
+    \item \emph{Correct Deferral}: Correct non-fertile prediction.
+\end{itemize}
+
+This classification measures both fertility prediction accuracy and the impact of incorrect deferrals.
+A model overly predicting fertility would increase pregnancy rates but negatively affect usability due to misdirected efforts.
 
 \subsection{Ethical Considerations}\label{subsec:ethical_considerations}