260 lines
16 KiB
TeX
260 lines
16 KiB
TeX
%! Author = alex
|
|
%! Date = 3/6/25
|
|
|
|
|
|
\section{Methodology}\label{sec:methodology}
|
|
|
|
% why did I select tft over other methods -> include examples of time series and why I belief a complex model could help
|
|
% you did I apply it
|
|
% implementation details
|
|
|
|
\subsection{Data Preprocessing}\label{subsec:data_preprocessing}
|
|
|
|
\subsubsection{Data Filtering}\label{subsubsec:data_filtering}
|
|
|
|
As briefly mentioned in~\ref{subsec:data_background}, not all cycles in the dataset are suitable for training.
|
|
Cycles that are either too short (\textless 10 days) or too long (\textgreater 150 days) are excluded,
|
|
as they typically indicate erroneous entries, pregnancies, sensor failures, or data processing issues.
|
|
|
|
In addition, incomplete cycles are filtered out, since new cycles continuously arrive from active users
|
|
and may not contain the full sequence of data required for retrospective labeling.
|
|
|
|
Temperature values outside the physiologically plausible range—below 35\textdegree C or above 43\textdegree C—are also excluded,
|
|
as they typically result from sensor malfunction or transmission errors.
|
|
|
|
\subsubsection{Data Labeling}\label{subsubsec:data_labeling}
|
|
|
|
Supervised machine learning requires labeled data—i.e., known target values for each input.
|
|
In the context of this study, the relevant target is the ovulation day within each menstrual cycle.
|
|
|
|
The dataset contains over 40{,}000 cycles, making manual annotation unfeasible.
|
|
Instead, a retrospective ovulation detection algorithm is employed to assign labels:
|
|
(1) whether a cycle is ovulatory or anovulatory, and
|
|
(2) if ovulatory, the estimated day of ovulation.
|
|
|
|
This labeling algorithm was developed in collaboration with a gynecologist and reproductive medicine specialist.
|
|
It is trained on a curated reference set of cycles with expert-assigned labels based on domain knowledge and characteristic temperature patterns.
|
|
|
|
The algorithm operates in two stages:
|
|
\begin{enumerate}
|
|
\item \textbf{Cycle classification:} Each cycle is classified as either monophasic (anovulatory) or biphasic (ovulatory), based on the presence of a luteal-phase temperature shift.
|
|
\item \textbf{Ovulation estimation:} For biphasic cycles, the most likely day of ovulation is identified retrospectively using the full temperature curve.
|
|
\end{enumerate}
|
|
|
|
This retrospective labeling provides a practical and scalable proxy for ground truth, enabling training and evaluation across a large, real-world dataset,
|
|
especially, as temperature is at least an excellent retrospective marker for ovulation.
|
|
In internal evaluations, the estimated ovulation day fell within a \(\pm\)2-day window of the expert reference in approximately 86\% of labeled cycles.
|
|
|
|
These labels serve as the supervisory signal for model training and evaluation.
|
|
We acknowledge the limitations of this method: ambiguous or noisy temperature patterns—due to illness, dropout,
|
|
or sensor error—can lead to mislabeled examples, which may propagate to downstream models.
|
|
However, label quality is continuously reviewed and may be refined iteratively as model performance improves.
|
|
|
|
The specific usage of ovulation labels in feature construction is described in the next section.
|
|
|
|
\subsection{Feature Engineering}\label{subsec:feature_engineering}
|
|
|
|
The features used as model inputs have been divided into three categories:
|
|
\begin{itemize}
|
|
\item \textbf{Static features} - Characteristics, that remain constant across a user's cycle, such as age, height, or average ovulation day
|
|
\item \textbf{Known features} — Inputs known a priori at each time step, such as time of day or calendar-based variables.
|
|
\item \textbf{Observable features} — Inputs available at the current time step, including raw and derived temperature values.
|
|
\item \textbf{Target features} — Outputs the model is trained to predict, such as the fertility probability.
|
|
\end{itemize}
|
|
|
|
Each feature type can handle categorical and continuous features.
|
|
This allows for mixed inputs, such as scalar measurements and class labels, within the same category.
|
|
|
|
The categorization into four feature types is intended to clarify the conceptual roles of different input types.
|
|
While the current models concatenate all features into a single input stream, the distinction allows for flexibility—future models
|
|
may process each feature group differently depending on their architectural design.
|
|
|
|
\subsubsection{Static Features}\label{subsubsec:static_features}
|
|
|
|
Static features are the features that do not change over the course of a cycle.
|
|
They might even be static for all cycles from a specific user, such as age, height and weight.
|
|
|
|
The static features are supposed to create contextual information about the cycle and the user that each model can then
|
|
use to learn patterns based not only on the temperature data, but also on this context.
|
|
|
|
\begin{table}[htbp]
|
|
\centering
|
|
\begin{tabular}{@{}lp{0.6\linewidth}@{}}
|
|
\toprule
|
|
\textbf{Feature} & \textbf{Description} \\
|
|
\midrule
|
|
User age & Age in years; mean imputed if missing \\
|
|
User height & Height in centimeters; mean imputed if missing \\
|
|
User weight & Weight in kilograms; mean imputed if missing \\
|
|
Average cycle length & Mean length of all previous cycles for this user \\
|
|
Cycle length SD & Standard deviation of previous cycle lengths \\
|
|
Average ovulation day & Mean day of ovulation from previous cycles \\
|
|
Ovulation SD & Standard deviation of ovulation day of previous cycles \\
|
|
Ovulatory fraction & Proportion of prior cycles classified as ovulatory \\
|
|
Cycle count & Number of previous completed cycles available \\
|
|
Avg. pre-ovulation temperature & Mean temperature in the follicular phase of previous cycles \\
|
|
Avg. post-ovulation temperature & Mean temperature in the luteal phase of previous cycles \\
|
|
\bottomrule
|
|
\end{tabular}
|
|
\caption{Static features used as model inputs}
|
|
\label{tab:static_features}
|
|
\end{table}
|
|
|
|
Table~\ref{tab:static_features} shows all static features and their descriptions.
|
|
Prior research by \citeauthor{li_menstrual_2023} has shown that menstrual cycle characteristics vary significantly with age and BMI~\cite{li_menstrual_2023}.
|
|
Including such information is therefore expected to improve predictive performance.
|
|
|
|
In addition, summary statistics from previous cycles—such as ovulation timing, temperature levels, or the fraction of ovulatory cycles—provide useful individual context.
|
|
These features help the model learn subject-specific variability and better estimate the likelihood and timing of ovulation in the current cycle.
|
|
|
|
All historical features are computed using only data available prior to the current cycle, ensuring no data leakage and supporting robust, user-adaptive learning.
|
|
|
|
%
|
|
The idea here is to provide as much context information to the models as possible to help them predict the ovulation.
|
|
|
|
\subsubsection{Known Features}\label{subsubsec:known_features}
|
|
In the context of this study, known features correspond to time-dependent inputs.
|
|
These help the model place each observation in temporal context:
|
|
|
|
\begin{itemize}
|
|
\item \textbf{Time since cycle start} — Provides the model with a relative position within the menstrual cycle.
|
|
\item \textbf{Hour of day} — Helps distinguish between daytime and nighttime patterns, especially relevant for circadian rhythms.
|
|
\item \textbf{Day of the week} — Encodes potential behavioral differences between weekdays and weekends.
|
|
\item \textbf{Month of the year} — Captures seasonal variations in temperature patterns or user behavior.
|
|
\end{itemize}
|
|
|
|
Except for \textit{time since cycle start}, all features are encoded using sine and cosine transforms to preserve their cyclical nature and make them more interpretable for the model.
|
|
|
|
%\begin{figure}[htbp]
|
|
% \centering
|
|
% \includegraphics[width=0.9\textwidth]{methodology_time_feature_sine_encoded}
|
|
% \caption{Sine and cosine encoding of the day-of-week feature.}
|
|
% \label{fig:methodology_time_feature_encoding}
|
|
%\end{figure}
|
|
|
|
Figure~\ref{fig:methodology_time_feature_encoding} illustrates the sine and cosine encoding of the day-of-week feature.
|
|
The cyclical nature of the variable is clearly visible in the transformation.
|
|
|
|
Although the model architectures used are sequential, the explicit inclusion of these time features allows the models to interpret each time step in a broader context.
|
|
More importantly, they enable the detection of gaps in the recording, which would otherwise not be visible from the data alone.
|
|
|
|
Additionally, prior research has shown that the menstrual cycle may be influenced by weekly rhythms~\cite{ecochard_menstrual_2024}.
|
|
For example, menstruation has been found to begin more frequently on Thursdays or Fridays, suggesting that behavioral or social factors may modulate certain events in the cycle.
|
|
Including this information could therefore improve the predictive quality of the models.
|
|
|
|
\subsubsection{Observable Features}\label{subsubsec:observable_features}
|
|
|
|
Observable features include all time-series inputs available up to the current time step.
|
|
They represent real-time physiological signals from which the model must infer ovulatory status.
|
|
|
|
\begin{itemize}
|
|
\item \textbf{Temperature} — Raw intravaginal temperature as recorded by the OvulaRing sensor, sampled every 5 minutes.
|
|
\item \textbf{Rolling Average Temperature} — The mean temperature over a 1-day (288-sample) sliding window, linearly interpolated to preserve the original input resolution.
|
|
\item \textbf{Rolling Window Temperature Minimum} — The minimum temperature observed within a 1-day window, highlighting potential overnight lows or phase-specific dips.
|
|
\item \textbf{Rolling Window Temperature Maximum} — The maximum temperature within a 1-day window, capturing transient peaks or elevated plateaus.
|
|
\end{itemize}
|
|
|
|
These derived features are intended to reduce model complexity by providing smoothed or extremal summaries of the raw signal.
|
|
The \textit{rolling average} allows the model to capture broader trends without having to learn temporal aggregation from scratch.
|
|
The \textit{rolling minimum} and \textit{maximum} support the detection of boundary behavior (e.g., temperature shifts, sustained elevation, extreme values)
|
|
without requiring explicit memory or aggregation.
|
|
|
|
The 1-day window length reflects the expected circadian cycle and strikes a balance between temporal sensitivity and signal stability.
|
|
|
|
\subsubsection{Target Features}
|
|
|
|
Target features represent the outputs that the models are trained to predict.
|
|
These can include exogenous biological outcomes or interpretable derivatives of input features.
|
|
|
|
\begin{itemize}
|
|
\item \textbf{Fertility / Pregnancy Probability} — The estimated probability of conception from unprotected intercourse on the current day.
|
|
\item \textbf{Ovulation-Over Indicator} — A binary variable indicating whether ovulation has already occurred in the current cycle.
|
|
\end{itemize}
|
|
|
|
\begin{figure}[htbp]
|
|
\centering
|
|
\includegraphics[width=0.9\textwidth]{methodology_target_features}
|
|
\caption{
|
|
Target features plotted for a single cycle.
|
|
The ovulation-over indicator switches on the day of ovulation;
|
|
the fertility probability follows a curve based on known day-specific fecundability~\cite{dunson_day-specific_1999}.
|
|
}
|
|
\label{fig:methodology_target_features}
|
|
\end{figure}
|
|
|
|
The combination of these two targets is intended to provide the user with both physiological and practical insight:
|
|
\textit{Fertility probability} conveys the likelihood of conception, but alone does not indicate whether ovulation is yet to come or has already passed.
|
|
A fertility probability near zero could mean that ovulation is either in the past or still ahead—information the model alone cannot disambiguate.
|
|
|
|
The \textit{ovulation-over indicator}, by contrast, explicitly marks the post-ovulatory phase, but does not describe conception risk.
|
|
Together, the two outputs offer complementary information and improve interpretability for real-time user-facing applications.
|
|
|
|
As discussed in Section~\ref{sec:discussion}, all predictions are subject to further interpretation before presentation in the product interface.
|
|
The model outputs represent data-driven estimates and do not constitute medical advice or diagnostic statements.
|
|
|
|
\subsection{Time-Series Modeling Approach}\label{subsec:time-series_modeling_approach}
|
|
|
|
\subsubsection{Input \& Output Modeling}\label{subsubsec:input_output_modeling}
|
|
|
|
% build in the probability curve for the pregnancy chance
|
|
|
|
\subsubsection{Sequence Representation and Sampling}
|
|
\label{subsubsec:input_sequence_construction}
|
|
|
|
Due to the high temporal resolution of the temperature data (288 measurements per day), raw input sequences can become prohibitively long for most model types.
|
|
|
|
To manage input size and evaluate the impact of temporal resolution on predictive performance, a parameterized resampling strategy is applied.
|
|
Consecutive time steps are aggregated into bins of configurable size, and each bin is reduced to a single value using a feature-specific aggregation function.
|
|
|
|
For most continuous features, the \texttt{mean} is used.
|
|
For cyclic or categorical features—such as \textit{day of the week}— the \texttt{max} or \texttt{mode} is applied to avoid introducing
|
|
artifacts at bin boundaries, where values from distinct categories (e.g., hours 23 and 0) might otherwise be averaged into a nonexistent intermediate state.
|
|
|
|
The effect of different sampling resolutions and aggregation strategies is evaluated in Section~\ref{sec:results}.
|
|
|
|
To simulate real-time prediction rather than retrospective analysis, a sliding-window approach is employed.
|
|
This allows the model to make partial predictions based on the data available up to a certain point in the cycle.
|
|
|
|
Each cycle is split into overlapping input windows, where each window includes data from the start of the cycle up to a specific day.
|
|
The window length is fixed and configurable.
|
|
As the cycle progresses, the window slides forward, allowing the model to incorporate increasing historical context over time.
|
|
|
|
This setup enables temporally resolved predictions at different stages of the cycle and supports evaluation of how predictive accuracy changes with increasing context.
|
|
|
|
\begin{figure}[htbp]
|
|
\centering
|
|
\includegraphics[width=0.9\textwidth]{methodology_padding_example}
|
|
\caption{
|
|
Padded input window during early cycle phases,
|
|
where historical data is still sparse.
|
|
}
|
|
\label{fig:methodology_padding_example}
|
|
\end{figure}
|
|
|
|
Fixed-length input windows would normally prevent early-cycle predictions when insufficient data is available.
|
|
To address this, left-padding is applied using masked values.
|
|
|
|
In this study, predictions are enabled once at least 4 days of data are available.
|
|
A padding value of 0.0 is used for all features, and the padding length is adjusted accordingly.
|
|
This choice ensures that the model learns to ignore tokens consisting entirely of padding.
|
|
|
|
The feature \textit{hours since start}, which encodes the time elapsed since cycle onset, will be zero for all padded tokens—explicitly indicating
|
|
that these entries contain no meaningful information.
|
|
|
|
Figure~\ref{fig:methodology_padding_example} shows an example of such padding during early-cycle input preparation.
|
|
|
|
\subsection{Model Training}\label{subsec:model_training}
|
|
|
|
\subsubsection{Training Setup}\label{subsubsec:training_setup}
|
|
|
|
\subsubsection{Hyperparameter Tuning}\label{subsubsec:hyperparameter_tuning}
|
|
|
|
\subsubsection{Training Details}\label{subsubsec:training_details}
|
|
|
|
\subsection{Evaluation}\label{subsec:evaluation}
|
|
|
|
\subsubsection{Evaluation Metrics}\label{subsubsec:evaluation_metrics}
|
|
|
|
\subsubsection{Baseline Comparisons}\label{subsubsec:baseline_comparisons}
|
|
|