|
|
|
@@ -8,46 +8,19 @@
|
|
|
|
|
% you did I apply it
|
|
|
|
|
% implementation details
|
|
|
|
|
|
|
|
|
|
\subsection{Data Collection \& Preprocessing}\label{subsec:data_collection_preprocessing}
|
|
|
|
|
\subsection{Data Preprocessing}\label{subsec:data_preprocessing}
|
|
|
|
|
|
|
|
|
|
\subsubsection{Data Collection}\label{subsubsec:data_collection}
|
|
|
|
|
The dataset used in this work was collected by \textit{VivoSensMedical GmbH}, a medical technology company based in Leipzig, Germany,
|
|
|
|
|
specializing in fertility tracking devices and applications.
|
|
|
|
|
Each entry in the dataset is derived from temperature measurements recorded by the \textit{OvulaRing} wearable
|
|
|
|
|
device\cite{alexander_fertilitatsmonitoring_2014}, which continuously measures core body temperature every 5 minutes.
|
|
|
|
|
The sensor is worn intra-vaginally and is designed for extended use, requiring removal only for data synchronization.
|
|
|
|
|
This enables high-resolution, long-term temperature tracking, providing a detailed view of menstrual cycle patterns.
|
|
|
|
|
\subsubsection{Data Filtering}\label{subsubsec:data_filtering}
|
|
|
|
|
|
|
|
|
|
% add a few example cycles
|
|
|
|
|
% TODO: stats about dataset
|
|
|
|
|
The dataset used for training and evaluation consists of 45000 cycles,
|
|
|
|
|
As briefly mentioned in~\ref{subsec:data_background}, not all cycles in the dataset are suitable for training.
|
|
|
|
|
Cycles that are either too short (\textless 10 days) or too long (\textgreater 150 days) are excluded,
|
|
|
|
|
as they typically indicate erroneous entries, pregnancies, sensor failures, or data processing issues.
|
|
|
|
|
|
|
|
|
|
In addition to raw temperature values, the dataset includes user-specific metadata, such as health status, weight, and age.
|
|
|
|
|
Users can also manually input time-dependent markers, indicating events such as menstruation, ovulation, and intercourse,
|
|
|
|
|
which serve as contextual features for the model.
|
|
|
|
|
While these additional data points can enhance predictive accuracy, it is important to note that they are self-reported
|
|
|
|
|
and may be subject to errors or biases.
|
|
|
|
|
In addition, incomplete cycles are filtered out, since new cycles continuously arrive from active users
|
|
|
|
|
and may not contain the full sequence of data required for retrospective labeling.
|
|
|
|
|
|
|
|
|
|
\subsubsection{Data Preprocessing}\label{subsubsec:data_preprocessing}
|
|
|
|
|
Not all cycles in the dataset can be used for training.
|
|
|
|
|
Cycles that are too short are excluded, as they usually do not contain enough information to make reliable predictions.
|
|
|
|
|
Additionally, cycles that are too long are also excluded, as they might contain multiple ovulation events or
|
|
|
|
|
pregnancies, which would make the prediction task ambiguous.
|
|
|
|
|
|
|
|
|
|
\subsubsection{Feature Engineering}\label{subsubsec:feature_engineering}
|
|
|
|
|
|
|
|
|
|
%TODO: show that a model without marker context and one with them is trained, to show the impact of the markers
|
|
|
|
|
|
|
|
|
|
%TODO: show that for initial testing and fine tuning the dataset was reduced to 10% of size to speed up early impressions of performance
|
|
|
|
|
%TODO: show that initial feature set with only temperature as observable and only pregnancy "chance" as target
|
|
|
|
|
% did not show promise on small 10% dataset, so the features got extended
|
|
|
|
|
% show the modifications that were made to account for multi target prediction
|
|
|
|
|
% show, the mothod of circumventing the non-padding-implementation by using a dedicated feature
|
|
|
|
|
% show, that a different loss / temporal loss wheighting or position aware loss could help
|
|
|
|
|
% show, the problem of the model using its past targets, even though they are not observable and how I dealt with this -> explain and use exogenous
|
|
|
|
|
% introduce change of quantiles to single variable output and why, quantiles loss other than .5 was almost always just 0 or 1, so no significant predicitve quality in this, and this only makes predictions harder for model
|
|
|
|
|
% explain why class imablances are there, and what I did to counteract this, also explain, why special weighting is useful for something like the pregnancy risk prediction
|
|
|
|
|
% show the work that needed to be done for inference -> padding values for future observables
|
|
|
|
|
Temperature values outside the physiologically plausible range—below 35\textdegree C or above 43\textdegree C—are also excluded,
|
|
|
|
|
as they typically result from sensor malfunction or transmission errors.
|
|
|
|
|
|
|
|
|
|
\subsubsection{Data Labeling}\label{subsubsec:data_labeling}
|
|
|
|
|
|
|
|
|
@@ -70,19 +43,117 @@ The algorithm operates in two stages:
|
|
|
|
|
|
|
|
|
|
This retrospective labeling provides a practical and scalable proxy for ground truth, enabling training and evaluation across a large, real-world dataset,
|
|
|
|
|
especially, as temperature is at least an excellent retrospective marker for ovulation.
|
|
|
|
|
%TODO: provide statistics for the retrospective algorithm
|
|
|
|
|
In internal evaluations, the estimated ovulation day fell within a \(\pm\)2-day window of the expert reference in approximately 86\% of labeled cycles.
|
|
|
|
|
|
|
|
|
|
These labels are then used as ground truth labels for the prediction algorithms.
|
|
|
|
|
We are aware of potential shortcomings with this approach.
|
|
|
|
|
We acknowledge that this approach introduces some label noise—particularly in edge cases where temperature signals are
|
|
|
|
|
ambiguous or distorted (e.g., illness, missed measurements).
|
|
|
|
|
These inaccuracies may propagate into the predictive models.
|
|
|
|
|
However, label quality is continuously reviewed and can be refined as part of future model iterations.
|
|
|
|
|
These labels serve as the supervisory signal for model training and evaluation.
|
|
|
|
|
We acknowledge the limitations of this method: ambiguous or noisy temperature patterns—due to illness, dropout,
|
|
|
|
|
or sensor error—can lead to mislabeled examples, which may propagate to downstream models.
|
|
|
|
|
However, label quality is continuously reviewed and may be refined iteratively as model performance improves.
|
|
|
|
|
|
|
|
|
|
The usage of the ovulation day labels in the model input features is documented in section~\ref{subsubsec:input_output_modeling}
|
|
|
|
|
The specific usage of ovulation labels in feature construction is described in the next section.
|
|
|
|
|
|
|
|
|
|
\subsection{Feature Engineering}\label{subsec:feature_engineering}
|
|
|
|
|
|
|
|
|
|
% show, that for very regular cycles, no sophisticated methods are necessary
|
|
|
|
|
The features used as model inputs have been divided into three categories:
|
|
|
|
|
\begin{itemize}
|
|
|
|
|
\item \textbf{Static features} - Characteristics, that remain constant across a user's cycle, such as age, height, or average ovulation day
|
|
|
|
|
\item \textbf{Known features} — Inputs known a priori at each time step, such as time of day or calendar-based variables.
|
|
|
|
|
\item \textbf{Observable features} — Inputs available at the current time step, including raw and derived temperature values.
|
|
|
|
|
\item \textbf{Target features} — Outputs the model is trained to predict, such as the fertility probability.
|
|
|
|
|
\end{itemize}
|
|
|
|
|
|
|
|
|
|
Each feature type can handle categorical and continuous features.
|
|
|
|
|
This allows for mixed inputs, such as scalar measurements and class labels, within the same category.
|
|
|
|
|
|
|
|
|
|
The categorization into four feature types is intended to clarify the conceptual roles of different input types.
|
|
|
|
|
While the current models concatenate all features into a single input stream, the distinction allows for flexibility—future models
|
|
|
|
|
may process each feature group differently depending on their architectural design.
|
|
|
|
|
|
|
|
|
|
\subsubsection{Static Features}\label{subsubsec:static_features}
|
|
|
|
|
|
|
|
|
|
Static features are the features that do not change over the course of a cycle.
|
|
|
|
|
They might even be static for all cycles from a specific user, such as age, height and weight.
|
|
|
|
|
|
|
|
|
|
The static features are supposed to create contextual information about the cycle and the user that each model can then
|
|
|
|
|
use to learn patterns based not only on the temperature data, but also on this context.
|
|
|
|
|
|
|
|
|
|
\begin{table}[htbp]
|
|
|
|
|
\centering
|
|
|
|
|
\begin{tabular}{@{}lp{0.6\linewidth}@{}}
|
|
|
|
|
\toprule
|
|
|
|
|
\textbf{Feature} & \textbf{Description} \\
|
|
|
|
|
\midrule
|
|
|
|
|
User age & Age in years; mean imputed if missing \\
|
|
|
|
|
User height & Height in centimeters; mean imputed if missing \\
|
|
|
|
|
User weight & Weight in kilograms; mean imputed if missing \\
|
|
|
|
|
Average cycle length & Mean length of all previous cycles for this user \\
|
|
|
|
|
Cycle length SD & Standard deviation of previous cycle lengths \\
|
|
|
|
|
Average ovulation day & Mean day of ovulation from previous cycles \\
|
|
|
|
|
Ovulation SD & Standard deviation of ovulation day of previous cycles \\
|
|
|
|
|
Ovulatory fraction & Proportion of prior cycles classified as ovulatory \\
|
|
|
|
|
Cycle count & Number of previous completed cycles available \\
|
|
|
|
|
Avg. pre-ovulation temperature & Mean temperature in the follicular phase of previous cycles \\
|
|
|
|
|
Avg. post-ovulation temperature & Mean temperature in the luteal phase of previous cycles \\
|
|
|
|
|
\bottomrule
|
|
|
|
|
\end{tabular}
|
|
|
|
|
\caption{Static features used as model inputs}
|
|
|
|
|
\label{tab:static_features}
|
|
|
|
|
\end{table}
|
|
|
|
|
|
|
|
|
|
Table~\ref{tab:static_features} shows all static features and their descriptions.
|
|
|
|
|
Prior research by \citeauthor{li_menstrual_2023} has shown that menstrual cycle characteristics vary significantly with age and BMI~\cite{li_menstrual_2023}.
|
|
|
|
|
Including such information is therefore expected to improve predictive performance.
|
|
|
|
|
|
|
|
|
|
In addition, summary statistics from previous cycles—such as ovulation timing, temperature levels, or the fraction of ovulatory cycles—provide useful individual context.
|
|
|
|
|
These features help the model learn subject-specific variability and better estimate the likelihood and timing of ovulation in the current cycle.
|
|
|
|
|
|
|
|
|
|
All historical features are computed using only data available prior to the current cycle, ensuring no data leakage and supporting robust, user-adaptive learning.
|
|
|
|
|
|
|
|
|
|
%
|
|
|
|
|
The idea here is to provide as much context information to the models as possible to help them predict the ovulation.
|
|
|
|
|
|
|
|
|
|
\subsubsection{Known Features}\label{subsubsec:known_features}
|
|
|
|
|
In the context of this study, known features correspond to time-dependent inputs.
|
|
|
|
|
These help the model place each observation in temporal context:
|
|
|
|
|
|
|
|
|
|
\begin{itemize}
|
|
|
|
|
\item \textbf{Time since cycle start} — Provides the model with a relative position within the menstrual cycle.
|
|
|
|
|
\item \textbf{Hour of day} — Helps distinguish between daytime and nighttime patterns, especially relevant for circadian rhythms.
|
|
|
|
|
\item \textbf{Day of the week} — Encodes potential behavioral differences between weekdays and weekends.
|
|
|
|
|
\item \textbf{Month of the year} — Captures seasonal variations in temperature patterns or user behavior.
|
|
|
|
|
\end{itemize}
|
|
|
|
|
|
|
|
|
|
Except for \textit{time since cycle start}, all features are encoded using sine and cosine transforms to preserve their cyclical nature and make them more interpretable for the model.
|
|
|
|
|
|
|
|
|
|
\begin{figure}[htbp]
|
|
|
|
|
\centering
|
|
|
|
|
\includegraphics[width=0.9\textwidth]{methodology_time_feature_sine_encoded}
|
|
|
|
|
\caption{Sine and cosine encoding of the day-of-week feature.}
|
|
|
|
|
\label{fig:methodology_time_feature_encoding}
|
|
|
|
|
\end{figure}
|
|
|
|
|
|
|
|
|
|
Figure~\ref{fig:methodology_time_feature_encoding} illustrates the sine and cosine encoding of the day-of-week feature.
|
|
|
|
|
The cyclical nature of the variable is clearly visible in the transformation.
|
|
|
|
|
|
|
|
|
|
Although the model architectures used are sequential, the explicit inclusion of these time features allows the models to interpret each time step in a broader context.
|
|
|
|
|
More importantly, they enable the detection of gaps in the recording, which would otherwise not be visible from the data alone.
|
|
|
|
|
|
|
|
|
|
Additionally, prior research has shown that the menstrual cycle may be influenced by weekly rhythms~\cite{ecochard_menstrual_2024}.
|
|
|
|
|
For example, menstruation has been found to begin more frequently on Thursdays or Fridays, suggesting that behavioral or social factors may modulate certain events in the cycle.
|
|
|
|
|
Including this information could therefore improve the predictive quality of the models.
|
|
|
|
|
|
|
|
|
|
\subsubsection{Observable Features}\label{subsubsec:observable_features}
|
|
|
|
|
|
|
|
|
|
The observable features constitute the input features that are directly observable, but only until the current moment.
|
|
|
|
|
|
|
|
|
|
\begin{itemize}
|
|
|
|
|
\item \textbf{Temperature} - the raw temperature as recorded by the OvulaRing sensor
|
|
|
|
|
\item \textbf{Rolling Average Temperature} - the rolling average of the temperature over 1 day (288 measurements)
|
|
|
|
|
\item \textbf{Rolling Window Temperature Minimum} - the minimum temperature over a rolling window of 1 day
|
|
|
|
|
\item \textbf{Rolling Window Temperature Maximum} - the maximum temperature over a rolling window of 1 day
|
|
|
|
|
\end{itemize}
|
|
|
|
|
|
|
|
|
|
% table with all features
|
|
|
|
|
|
|
|
|
|
\subsection{Time-Series Modeling Approach}\label{subsec:time-series_modeling_approach}
|
|
|
|
|
|
|
|
|
|
Alex Blank