%! Author = alex
%! Date = 9/17/24

% Preamble
\documentclass[11pt]{article}

% Packages
\usepackage{amsmath}
% biber bibliography
\usepackage[style=ieee, backend=biber]{biblatex}
\addbibresource{../../main.bib}

% Document
\begin{document}

    \title{Finding Predictors for Human Ovulation with Attention Mechanisms}
    \author{Alexander Blank}
    \date{February 2025}
    \maketitle


    \section{Introduction}\label{sec:introduction}
    Women's health is a topic that has been neglected for a long time, even though it has direct implications for
    more than half of the world's population.
    The menstrual cycle and its corresponding ovulation are essential for reproductive health,
    family planning and natural contraception.
    Part of the challenge lies in the lack of comprehensive datasets, but even more critically,
    in the incomplete understanding of the underlying physiological mechanisms.
    New advances in machine learning have shown, that there is potential in predicting the menstrual cycle and ovulation
    with high accuracy based on physiological data, such as basal body temperature,
    heart rate or cervicovaginal fluid\cite{noauthor_cervicovaginal_2005, luo_detection_2020, masuda_machine_2025, yu_tracking_2022}.
    A problem with the current landscape of research is often the lack of interpretability of the models as well as insufficient
    data to train the models.
    This lack of understanding is compounded by the opacity of many machine learning models, making their predictions difficult to trust and validate in clinical settings.
    In female health, a wrong prediction can have severe consequences, such as unwanted pregnancies, health risks or even death.
    This thesis aims to improve upon that by using a large dataset of body core temperature and markers to train an
    interpretable model based on attention mechanisms to predict ovulation and to identify the most important predictors for ovulation.


    \section{Related Work}\label{sec:related-work}
    Research on fertility prediction goes back as far as the 1990s, where the first purely statistical models were
    developed to predict both the ovulation and the fertility over time\cite{royston_identifying_1991}.
    Since then, there has been a long pause in research until the rise of machine learning in the late 2010s.
    Both data quality and quantity have improved significantly, which has led to more accurate models\cite{maman_prediction_2023, noauthor_pdf_2024}.
    However, since the female menstrual cycle is a biological process, prone to noise and irregularities, the models
    often lack generalizability and interpretability, especially for women with irregular cycles.
    There is a variety of biological features that can be used to predict ovulation, but most of it have shown to be
    impractical for large scale use, as they require daily blood samples or ultrasound measurements.
    The most promising are models based on temperature data, as it is a quantity that can be recorded on a large scale
    without intrusive methods (ear-piece or vaginal sensor monitoring)\cite{alexander_fertilitatsmonitoring_2014, luo_detection_2020}.
    However, body temperature alone is an imperfect predictor of ovulation,
    as it is influenced by external factors such as stress, sport, illness, and sleep disturbances.
    While external measurement (e.g.\ skin or earpiece sensors) are even less reliable,
    even vaginal temperature measurements can be influenced by these factors.
    This is where this research aims to improve upon, by incorporating more predictors, such as markers for illness, sport,
    stress or similar things.


    \section{Research Questions and Objectives}\label{sec:research-questions-and-objectives}
    The main research objective is to create a model that can predict ovulation and fertile days with high accuracy
    while offering a high level of interpretability in its underlying attention mechanisms.

    \begin{itemize}
        \item Can attention-based models improve the prediction of ovulation?
        \item Can attention-based models improve the interpretability of the female menstrual cycle?
        \item What are the most important predictors for ovulation, both in space and time?
        \item How does an attention-based model compare to other models in terms of accuracy and interpretability?
        \item Can we account for external influences (e.g. stress, illness, sport) to improve prediction robustness?
        \item Optional: Use a non-interpretable transformer model to see impact of interpretability on prediction accuracy.
    \end{itemize}


    \section{Methodology}\label{sec:methodology}
    The research will be based on an extensive dataset, generously provided by the FemTech company\cite{noauthor_vivosens_nodate},
    which incorporates 50,000 cycles with temperature measurements every 5 minutes by a intravaginal sensor ring.
    The dataset contains a wide variety of cycle types and predispositions, including regular and irregular cycles,
    PCOS, endometriosis, and other conditions.
    Additionally, the dataset contains both user generated and automatically generated markers for cervical fluid, menstruation,
    mood and other relevant factors.

    The model will be based on a transformer architecture, which has shown to be very effective in time series prediction\cite{vaswani_attention_2017}.
    This architecture is particularly suited for ovulation prediction because it can capture dependencies across different time points—identifying,
    for instance, how temperature fluctuations in the follicular phase influence predictions in the luteal phase.
    Transformer-based models have been further improved for specific use cases.
    Here we will use a model that has shown to be effective in time-series analysis with the addition of an
    interpretable attention mechanism, called the \emph{Temporal Fusion Transformer}(TFT)\cite{lim_temporal_2020}.
    TFT is specifically designed for multivariate time-series forecasting, allowing it to incorporate static covariates
    (e.g., age) alongside dynamic features (e.g., temperature, stress markers) to improve prediction robustness.
    By training an interpretable attention mechanism, we will identify the most crucial factors for ovulation and
    fertility prediction.
    These factors will then be presented to a reproductive health expert to validate and interpret the findings.
    Given the sensitivity of fertility prediction, particular attention
    will be paid to data privacy and ethical considerations in model deployment.

    The model will be evaluated on its accuracy and interpretability and compared to other models in the field such as
    OvulaRing's previous methods\cite{alexander_fertilitatsmonitoring_2014} other machine learning models (LSTM, GRU, etc.)
    and simpler statistical models ((S)ARIMA, etc.).
    Next to standard metrics, such as accuracy, precision, recall, F1-score, we will also use fertility-specific metrics,
    that incorporate the probability curve of the fertile window according to \citeauthor{dunson_day-specific_1999}\cite{dunson_day-specific_1999},
    and the severity of the consequences of a wrong prediction.


    \section{Conclusion}
    The research aims to improve the prediction of ovulation and fertility by using an interpretable attention-based model.
    The model will be trained on a large dataset of intravaginal temperature measurements and set of time-varying and constant features.
    The model will be evaluated on its accuracy and interpretability and compared to other models in the field.
    The most important predictors for ovulation will be identified and presented to a reproductive health expert for validation.

    % bibliography
    \printbibliography


\end{document}