temperature-based-fertility-prediction-thesis/thesis/sections/background.tex

%! Author = alex
%! Date = 3/7/25

\section{Background}\label{sec:background}

\subsection{Physiological Background}\label{subsec:physiological_background}

\subsubsection{Menstrual Cycle}\label{subsec:menstrual_cycle}
The menstrual cycle describes the physiological changes in the female body that prepare it for pregnancy.
It is divided into two phases: the \textbf{follicular phase} and the \textbf{luteal phase}.
\\
During the follicular phase, the ovarian follicles mature, and the endometrium (the inner lining of the uterus) thickens
in preparation for a potential implantation of a fertilized egg.
Around day 14 of a typical cycle, ovulation occurs, marking the transition to the luteal phase.
Ovulation refers to the rupture of the mature ovarian follicle and the release of an egg cell into the fallopian tube.
Figure\ref{fig:background_basic_female_reproductive_system} illustrates the female reproductive system,
including the ovaries and the fallopian tubes.
\\
\begin{figure}
    \centering
    \includegraphics[width=0.4\textwidth]{background_female_reproductive_organs}
    \caption{The basic female reproductive system\cite{wikimedia_commons_basic_2019}.}
    \label{fig:background_basic_female_reproductive_system}
\end{figure}
Ovulation is triggered by a surge in \textbf{luteinizing hormone (LH)}
and \textbf{follicle-stimulating hormone (FSH)}, following a peak in estradiol levels.
As ovulation occurs, estradiol levels drop, progesterone levels begin to rise and a slight increase in body temperature
(typically around 0.5°C) can be observed.
This marks the beginning of the luteal phase.
\\
During the luteal phase, the endometrium thickens further, creating an optimal environment for embryo implantation.
LH and FSH levels decrease, while progesterone remains elevated to support endometrial maintenance.
If fertilization does not occur, progesterone levels drop, leading to the shedding of the endometrial lining along
with the unfertilized egg.
This process, known as menstruation, marks the beginning of a new cycle.
\\
The menstrual cycle typically lasts around 28 days, with ovulation occurring near the midpoint.
However, variations, particularly in the follicular phase length, are common and can be influenced by factors such as stress, diet, exercise and age\cite{silberstein_physiology_2000}.
Figure~\ref{fig:background_menstrual_cycle_physiology} provides a detailed overview of the hormonal and physiological changes
throughout the menstrual cycle.

\begin{figure}
    \centering
    \includegraphics[width=0.6\textwidth]{background_menstrual_cycle_physiology}
    \caption{Physiological changes during the menstrual cycle\cite{pedroso_menstrual_2022}.}
    \label{fig:background_menstrual_cycle_physiology}
\end{figure}

Not every cycle results in ovulation—a phenomenon known as anovulation—which leads to a monophasic temperature pattern.
Anovulation can have various causes, including hormonal imbalances, stress, or underlying health conditions\cite{rosenfield_adolescent_2013}.

Anovulation is reflected in temperature data as either an absence of a clear temperature rise or a rise
that is insufficient in magnitude or duration to be considered a reliable indicator of ovulation.
Distinguishing between ovulatory and anovulatory cycles is challenging, as the only definitive confirmation of
successful ovulation in a clinical sense is a positive pregnancy test.
Even ultrasound imaging can only confirm that an egg was released from its follicle—not whether it was fertilized or successfully implanted.

%TODO: find source for this

%TODO: show plot of different cycle types

\subsubsection{Fertility Prediction}\label{subsec:fertility_prediction}
Throughout the menstrual cycle, the chance of fertilization varies significantly.
An egg cell released from the ovary during ovulation, can be fertilized for up to 24 hours.
However, since male sperm cells can survive up to 6 days inside the female reproductive tract,
the fertile window is typically defined as the five days before ovulation until one day after ovulation\cite{dunson_day-specific_1999}.
Research by \citeauthor{dunson_day-specific_1999} has shown that the highest chance of fertilization is around one day before ovulation,
as illustrated in Figure~\ref{fig:background_pregnancy_chance}.

\begin{figure}
    \centering
    \includegraphics[width=0.7\textwidth]{background_pregnancy_chance_over_time}
    \caption{Chance of fertilization depending on the day of the menstrual cycle.
    The highest chance is around one day before ovulation\cite{dunson_day-specific_1999}.}
    \label{fig:background_pregnancy_chance}
\end{figure}

It is important to note that fertility prediction is inherently dependent on ovulation prediction.
Since the probability of conception is tightly linked to ovulation timing, the accuracy of fertility prediction methods
is constrained by the precision of ovulation detection.
This relationship underscores the necessity of developing reliable ovulation prediction models,
as even small inaccuracies can significantly impact fertility assessments.

\subsubsection{Physiological Signs of Ovulation}\label{subsec:physiological_signs}
Several physiological signs correlate with ovulation and can be used for prediction.
As shown in Figure~\ref{fig:background_menstrual_cycle_physiology}, these include hormonal fluctuations (LH and FSH surges) and
changes in body temperature.
Additionally, variations in cervical mucus consistency, salivary ferning patterns,
and electrical resistance of the skin and vaginal mucosa have been observed\cite{silberstein_physiology_2000}.

Among these, ultrasonography provides the most accurate confirmation of ovulation by detecting follicular changes,
but it is costly and requires specialized equipment.
Hormone measurements in urine and blood are widely used and available in at-home test kits,
but they require frequent testing.
Temperature-based methods, particularly body temperature tracking, offer a non-invasive alternative by
detecting the slight temperature rise that follows ovulation.
Advances in wearable technology have further enabled continuous and automated temperature monitoring,
improving accessibility and usability\cite{alexander_fertilitatsmonitoring_2014, luo_detection_2020, yu_tracking_2022}.

\subsection{Technical Background}\label{subsec:technological_background}

\subsubsection{Time Series Analysis}\label{subsubsec:time_series_analysis}
Time series analysis is a fundamental tool for studying sequential data that evolves over time.
Unlike other data types, time series data has an inherent temporal order, where each data point is associated
with a timestamp, capturing its dependence on past values.
There are usually two main goals:
understanding the underlying mechanisms that lead to the observed data and predicting future data points based on the
historical information and potentially external factors\cite{cryer_time_2008}
\\
Time series analysis encompasses various methods, ranging from simple statistical models to complex deep learning architectures.
Classical methods

\subsubsection{Transformer Models}\label{subsubsec:transformer_models}

\subsubsection{Temporal-Fusion-Transformer Models}\label{subsubsec:temporal_fusion_transformer_models}
The Temporal Fusion Transformer (TFT) is a deep learning model designed for time series forecasting.
It's a novel approach at time series analysis and forecasting presented by Google in 2020\cite{lim_temporal_2020}.
The main focus of this architecture is to both achieve high prediction accuracy while maintaining interpretability.
It differs from a traditional implementation of a transformer architecture by incorporating the support for:
\begin{itemize}
    \item Multiple inputs (static, past and future variables)
    \item Long sequences
    \item Uncertainty
\end{itemize}
Figure~\ref{fig:background_tft_architecture} provides an overview of the TFT architecture.

\paragraph{1. Gating Mechanisms}
As the precise relationship between input variables is hard to anticipate in advance, the authors introduce a mechanisms,
that allows the model to learn this relationship, giving it the ability to choose whether to apply a non-linear
transformation to the inputs.
Sometimes, the relationship between variables is linear, and applying a non-linear transformation would only add noise.
They call this mechanism Gated Residual Network (GRN).
GRNs take the primary input vector $a$ and an optional context vector $c$ and compute the output as follows:
\begin{align}
\text{GRN}_{\omega}(a, c) &= \text{LayerNorm}(a + \text{GLU}_{\omega}(\eta_1)) \label{eq:grn} \\
\eta_1 &= W_{1,\omega} \eta_2 + b_{1,\omega} \label{eq:eta1} \\
\eta_2 &= \text{ELU}(W_{2,\omega} a + W_{3,\omega} c + b_{2,\omega}) \label{eq:eta2}
\end{align}

ELU refers to the Exponential Linear Unit activation function, which is a variant of the ReLU function,
that speeds up training and alleviates the vanishing gradient problem\cite{clevert_fast_2016}.
LayerNorm refers to a more robust normalization technique compared to batch normalization, which is used to stabilize
and speed up training in deep neural networks\cite{ba_layer_2016}.
Here, $\omega$ denotes weight sharing.

For large $W_{2,\omega}\ a + W_{3\omega}\   c + b_{2,\omega} \gg 0$ the output of the ELU is approximately $a$, acting
as identity function.
For negative values, it generates a constant output, resulting in a linear transformation of the input.
Gating layers based on the Gated Linear Unit (GLU)\cite{dauphin_language_2017} provide the model with the ability to
suppress any parts of the architecture not needed for the prediction in a dataset.


\paragraph{2. Variable Selection Networks (VSNs)}
Variable Selection Networks (VSNs) are a key component of the TFT architecture, allowing the model to both
select input variables most relevant to the prediction and removing unnecessary, noisy inputs, that could negatively
impact prediction performance.
Each component, i.e., static, past, and future variables, use separate VSNs to select the most relevant features.
Inputs to the VSN use entity embeddings for categorical features or a linear transformation for continuous features.
This transforms the inputs into the ($d_{model}$)-dimensional space, where $d_{model}$ is the model's hidden dimension.


\begin{figure}
    \centering
    \includegraphics[width=0.8\textwidth]{background_tft_architecture}
    \caption{TFT architecture overview\cite{lim_temporal_2020}.}
    \label{fig:background_tft_architecture}
\end{figure}