%! Author = alex
%! Date = 3/6/25


\section{Methodology}\label{sec:methodology}

% why did I select tft over other methods -> include examples of time series and why I belief a complex model could help
% you did I apply it
% implementation details

\subsection{Data Preprocessing}\label{subsec:data_preprocessing}

\subsubsection{Data Filtering}\label{subsubsec:data_filtering}

As briefly mentioned in~\ref{subsec:data_background}, not all cycles in the dataset are suitable for training.
Cycles that are either too short (\textless 10 days) or too long (\textgreater 150 days) are excluded,
as they typically indicate erroneous entries, pregnancies, sensor failures, or data processing issues.

In addition, incomplete cycles are filtered out, since new cycles continuously arrive from active users
and may not contain the full sequence of data required for retrospective labeling.

Temperature values outside the physiologically plausible range—below 35\textdegree C or above 43\textdegree C—are also excluded,
as they typically result from sensor malfunction or transmission errors.

\subsubsection{Data Labeling}\label{subsubsec:data_labeling}

Supervised machine learning requires labeled data—i.e., known target values for each input.
In the context of this study, the relevant target is the ovulation day within each menstrual cycle.

The dataset contains over 40{,}000 cycles, making manual annotation unfeasible.
Instead, a retrospective ovulation detection algorithm is employed to assign labels:
(1) whether a cycle is ovulatory or anovulatory, and
(2) if ovulatory, the estimated day of ovulation.

This labeling algorithm was developed in collaboration with a gynecologist and reproductive medicine specialist.
It is trained on a curated reference set of cycles with expert-assigned labels based on domain knowledge and characteristic temperature patterns.

The algorithm operates in two stages:
\begin{enumerate}
    \item \textbf{Cycle classification:} Each cycle is classified as either monophasic (anovulatory) or biphasic (ovulatory), based on the presence of a luteal-phase temperature shift.
    \item \textbf{Ovulation estimation:} For biphasic cycles, the most likely day of ovulation is identified retrospectively using the full temperature curve.
\end{enumerate}

This retrospective labeling provides a practical and scalable proxy for ground truth, enabling training and evaluation across a large, real-world dataset,
especially, as temperature is at least an excellent retrospective marker for ovulation.
In internal evaluations, the estimated ovulation day fell within a \(\pm\)2-day window of the expert reference in approximately 86\% of labeled cycles.

These labels serve as the supervisory signal for model training and evaluation.
We acknowledge the limitations of this method: ambiguous or noisy temperature patterns—due to illness, dropout,
or sensor error—can lead to mislabeled examples, which may propagate to downstream models.
However, label quality is continuously reviewed and may be refined iteratively as model performance improves.

The specific usage of ovulation labels in feature construction is described in the next section.

\subsection{Feature Engineering}\label{subsec:feature_engineering}

The features used as model inputs have been divided into three categories:
\begin{itemize}
    \item \textbf{Static features} - Characteristics, that remain constant across a user's cycle, such as age, height, or average ovulation day
    \item \textbf{Known features} — Inputs known a priori at each time step, such as time of day or calendar-based variables.
    \item \textbf{Observable features} — Inputs available at the current time step, including raw and derived temperature values.
    \item \textbf{Target features} — Outputs the model is trained to predict, such as the fertility probability.
\end{itemize}

Each feature type can handle categorical and continuous features.
This allows for mixed inputs, such as scalar measurements and class labels, within the same category.

The categorization into four feature types is intended to clarify the conceptual roles of different input types.
While the current models concatenate all features into a single input stream, the distinction allows for flexibility—future models
may process each feature group differently depending on their architectural design.

\subsubsection{Static Features}\label{subsubsec:static_features}

Static features are the features that do not change over the course of a cycle.
They might even be static for all cycles from a specific user, such as age, height and weight.

The static features are supposed to create contextual information about the cycle and the user that each model can then
use to learn patterns based not only on the temperature data, but also on this context.

\begin{table}[htbp]
    \centering
    \begin{tabular}{@{}lp{0.6\linewidth}@{}}
        \toprule
        \textbf{Feature}                & \textbf{Description}                                        \\
        \midrule
        User age                        & Age in years; mean imputed if missing                       \\
        User height                     & Height in centimeters; mean imputed if missing              \\
        User weight                     & Weight in kilograms; mean imputed if missing                \\
        Average cycle length            & Mean length of all previous cycles for this user            \\
        Cycle length SD                 & Standard deviation of previous cycle lengths                \\
        Average ovulation day           & Mean day of ovulation from previous cycles                  \\
        Ovulation SD                    & Standard deviation of ovulation day of previous cycles      \\
        Ovulatory fraction              & Proportion of prior cycles classified as ovulatory          \\
        Cycle count                     & Number of previous completed cycles available               \\
        Avg. pre-ovulation temperature  & Mean temperature in the follicular phase of previous cycles \\
        Avg. post-ovulation temperature & Mean temperature in the luteal phase of previous cycles     \\
        \bottomrule
    \end{tabular}
    \caption{Static features used as model inputs}
    \label{tab:static_features}
\end{table}

Table~\ref{tab:static_features} shows all static features and their descriptions.
Prior research by \citeauthor{li_menstrual_2023} has shown that menstrual cycle characteristics vary significantly with age and BMI~\cite{li_menstrual_2023}.
Including such information is therefore expected to improve predictive performance.

In addition, summary statistics from previous cycles—such as ovulation timing, temperature levels, or the fraction of ovulatory cycles—provide useful individual context.
These features help the model learn subject-specific variability and better estimate the likelihood and timing of ovulation in the current cycle.

All historical features are computed using only data available prior to the current cycle, ensuring no data leakage and supporting robust, user-adaptive learning.

%
The idea here is to provide as much context information to the models as possible to help them predict the ovulation.

\subsubsection{Known Features}\label{subsubsec:known_features}
In the context of this study, known features correspond to time-dependent inputs.
These help the model place each observation in temporal context:

\begin{itemize}
    \item \textbf{Time since cycle start} — Provides the model with a relative position within the menstrual cycle.
    \item \textbf{Hour of day} — Helps distinguish between daytime and nighttime patterns, especially relevant for circadian rhythms.
    \item \textbf{Day of the week} — Encodes potential behavioral differences between weekdays and weekends.
    \item \textbf{Month of the year} — Captures seasonal variations in temperature patterns or user behavior.
\end{itemize}

Except for \textit{time since cycle start}, all features are encoded using sine and cosine transforms to preserve their cyclical nature and make them more interpretable for the model.

%\begin{figure}[htbp]
%    \centering
%    \includegraphics[width=0.9\textwidth]{methodology_time_feature_sine_encoded}
%    \caption{Sine and cosine encoding of the day-of-week feature.}
%    \label{fig:methodology_time_feature_encoding}
%\end{figure}

Figure~\ref{fig:methodology_time_feature_encoding} illustrates the sine and cosine encoding of the day-of-week feature.
The cyclical nature of the variable is clearly visible in the transformation.

Although the model architectures used are sequential, the explicit inclusion of these time features allows the models to interpret each time step in a broader context.
More importantly, they enable the detection of gaps in the recording, which would otherwise not be visible from the data alone.

Additionally, prior research has shown that the menstrual cycle may be influenced by weekly rhythms~\cite{ecochard_menstrual_2024}.
For example, menstruation has been found to begin more frequently on Thursdays or Fridays, suggesting that behavioral or social factors may modulate certain events in the cycle.
Including this information could therefore improve the predictive quality of the models.

\subsubsection{Observable Features}\label{subsubsec:observable_features}

Observable features include all time-series inputs available up to the current time step.
They represent real-time physiological signals from which the model must infer ovulatory status.

\begin{itemize}
    \item \textbf{Temperature} — Raw intravaginal temperature as recorded by the OvulaRing sensor, sampled every 5 minutes.
    \item \textbf{Rolling Average Temperature} — The mean temperature over a 1-day (288-sample) sliding window, linearly interpolated to preserve the original input resolution.
    \item \textbf{Rolling Window Temperature Minimum} — The minimum temperature observed within a 1-day window, highlighting potential overnight lows or phase-specific dips.
    \item \textbf{Rolling Window Temperature Maximum} — The maximum temperature within a 1-day window, capturing transient peaks or elevated plateaus.
\end{itemize}

These derived features are intended to reduce model complexity by providing smoothed or extremal summaries of the raw signal.
The \textit{rolling average} allows the model to capture broader trends without having to learn temporal aggregation from scratch.
The \textit{rolling minimum} and \textit{maximum} support the detection of boundary behavior (e.g., temperature shifts, sustained elevation, extreme values)
without requiring explicit memory or aggregation.

The 1-day window length reflects the expected circadian cycle and strikes a balance between temporal sensitivity and signal stability.

\subsubsection{Target Features}

Target features represent the outputs that the models are trained to predict.
These can include exogenous biological outcomes or interpretable derivatives of input features.

\begin{itemize}
    \item \textbf{Fertility / Pregnancy Probability} — The estimated probability of conception from unprotected intercourse on the current day.
    \item \textbf{Ovulation-Over Indicator} — A binary variable indicating whether ovulation has already occurred in the current cycle.
\end{itemize}

\begin{figure}[htbp]
    \centering
    \includegraphics[width=0.9\textwidth]{methodology_target_features}
    \caption{
        Target features plotted for a single cycle.
        The ovulation-over indicator switches on the day of ovulation;
        the fertility probability follows a curve based on known day-specific fecundability~\cite{dunson_day-specific_1999}.
    }
    \label{fig:methodology_target_features}
\end{figure}

The combination of these two targets is intended to provide the user with both physiological and practical insight:
\textit{Fertility probability} conveys the likelihood of conception, but alone does not indicate whether ovulation is yet to come or has already passed.
A fertility probability near zero could mean that ovulation is either in the past or still ahead—information the model alone cannot disambiguate.

The \textit{ovulation-over indicator}, by contrast, explicitly marks the post-ovulatory phase, but does not describe conception risk.
Together, the two outputs offer complementary information and improve interpretability for real-time user-facing applications.

As discussed in Section~\ref{sec:discussion}, all predictions are subject to further interpretation before presentation in the product interface.
The model outputs represent data-driven estimates and do not constitute medical advice or diagnostic statements.

\subsection{Time-Series Modeling Approach}\label{subsec:time-series_modeling_approach}

\subsubsection{Input \& Output Modeling}\label{subsubsec:input_output_modeling}

% build in the probability curve for the pregnancy chance

\subsubsection{Sequence Representation and Sampling}
\label{subsubsec:input_sequence_construction}

Due to the high temporal resolution of the temperature data (288 measurements per day), raw input sequences can become prohibitively long for most model types.

To manage input size and evaluate the impact of temporal resolution on predictive performance, a parameterized resampling strategy is applied.
Consecutive time steps are aggregated into bins of configurable size, and each bin is reduced to a single value using a feature-specific aggregation function.

For most continuous features, the \texttt{mean} is used.
For cyclic or categorical features—such as \textit{day of the week}— the \texttt{max} or \texttt{mode} is applied to avoid introducing
artifacts at bin boundaries, where values from distinct categories (e.g., hours 23 and 0) might otherwise be averaged into a nonexistent intermediate state.

The effect of different sampling resolutions and aggregation strategies is evaluated in Section~\ref{sec:results}.

To simulate real-time prediction rather than retrospective analysis, a sliding-window approach is employed.
This allows the model to make partial predictions based on the data available up to a certain point in the cycle.

Each cycle is split into overlapping input windows, where each window includes data from the start of the cycle up to a specific day.
The window length is fixed and configurable.
As the cycle progresses, the window slides forward, allowing the model to incorporate increasing historical context over time.

This setup enables temporally resolved predictions at different stages of the cycle and supports evaluation of how predictive accuracy changes with increasing context.

\begin{figure}[htbp]
    \centering
    \includegraphics[width=0.9\textwidth]{methodology_padding_example}
    \caption{
        Padded input window during early cycle phases,
        where historical data is still sparse.
    }
    \label{fig:methodology_padding_example}
\end{figure}

Fixed-length input windows would normally prevent early-cycle predictions when insufficient data is available.
To address this, left-padding is applied using masked values.

In this study, predictions are enabled once at least 4 days of data are available.
A padding value of 0.0 is used for all features, and the padding length is adjusted accordingly.
This choice ensures that the model learns to ignore tokens consisting entirely of padding.

The feature \textit{hours since start}, which encodes the time elapsed since cycle onset, will be zero for all padded tokens—explicitly indicating
that these entries contain no meaningful information.

Figure~\ref{fig:methodology_padding_example} shows an example of such padding during early-cycle input preparation.

\subsection{Model Training}\label{subsec:model_training}

\subsubsection{Training Setup}\label{subsubsec:training_setup}

\subsubsection{Hyperparameter Tuning}\label{subsubsec:hyperparameter_tuning}

\subsubsection{Training Details}\label{subsubsec:training_details}

\subsection{Evaluation}\label{subsec:evaluation}

\subsubsection{Evaluation Metrics}\label{subsubsec:evaluation_metrics}

\subsubsection{Baseline Comparisons}\label{subsubsec:baseline_comparisons}

