fixes and improvements

thesis/sections/background.tex

@@ -7,7 +7,7 @@
 \subsection{Physiological Background}\label{subsec:physiological_background}
 
 \subsubsection{Menstrual Cycle}\label{subsec:menstrual_cycle}
-The menstrual cycle consists of physiological changes preparing the female body for potential pregnancy,
+The menstrual cycle consists of physiological changes preparing the body for potential pregnancy,
 typically spanning around 28 days but varying considerably among individuals.
 It includes two main phases: the follicular phase, beginning with menstruation, and the luteal phase, following ovulation.
 
@@ -19,7 +19,8 @@ induces ovulation—the release of a mature egg into the fallopian tube.
 After ovulation, the luteal phase begins (Figure~\ref{fig:background_menstrual_cycle_physiology} day 14 to 28).
 Progesterone increases substantially, maintaining endometrial thickness for potential embryo implantation.
 In parallel, a subtle rise in body temperature (~0.5°C) occurs due to progesterone elevation.
-If fertilization does not happen, progesterone and temperature decline back to baseline levels, resulting in menstruation and initiating a new cycle.
+If fertilization does not happen, progesterone and temperature decline back to baseline levels,
+resulting in menstruation and initiating a new cycle~\cite{owen_physiology_1975}.
 
 \begin{figure}[htbp]
     \centering
@@ -57,7 +58,7 @@ The luteal phase begins at the ovulation and continues until the next menstruati
 
 While many cycles exhibit a characteristic biphasic pattern, deviations from this norm are common.
 Some do not show the typical temperature surge, which might be an indication for an anovulatory cycle.
-Anovulatory cycles don't have an ovulation, and thus cannot result in a pregnancy.
+Anovulatory cycles do not have an ovulation, and thus cannot result in a pregnancy.
 Anovulation can have various causes, including hormonal imbalances, stress, or underlying health conditions~\cite{rosenfield_adolescent_2013}.
 Monophasic cycles with a confirmed ovulation event have also been observed, so there seems to be no clear indication that it is a direct cause of anovulation~\cite{moghissi_accuracy_1976}.
 Thus, distinguishing between ovulatory and anovulatory cycles is challenging, as the only definitive confirmation of
@@ -130,7 +131,8 @@ However, since the goal is to identify the fertile window before ovulation occur
 For individuals trying to conceive or avoid pregnancy, knowing the window of fertility is more actionable than identifying the ovulation event itself.
 
 \subsubsection{Practical Use Cases}\label{subsubsec:practical_use_cases}
-In this study, we will focus on \emph{natural family planning} (NFP), which includes preventing and achieving pregnancy.
+In this study, I will focus on \emph{natural family planning} (NFP),
+which constitutes achieving and preventing pregnancy through non-invasive and natural means~\cite{smoley_natural_nodate}.
 Individuals aiming to avoid pregnancy identify fertile days to abstain from intercourse,
 whereas those seeking pregnancy aim to focus intercourse around days with the highest fertility probability.
 
@@ -144,13 +146,14 @@ Although false-positives may lead to unnecessary abstinence, this outcome is gen
 Consequently, prediction algorithms should be conservative, erring on the side of higher fertility estimates to prioritize safety.
 
 For women aiming to conceive, incorrect predictions are less severe, as they might, at worst, result in misdirected
-pregnancy effort, causing frustration or delays, but no potential dangers to the health of a woman.
-Therefore, algorithms for this group don't need to be as conservative.
-It remains to be seen, where the middle ground lies and how different algorithms perform for different use cases.
+pregnancy effort, causing frustration or delays, but no severe dangers to the health of a woman.
+Therefore, algorithms for this group do not need to be as conservative.
+It remains to be seen where the middle ground lies and how different algorithms perform for different use cases.
+I acknowledge potential mental health risks associated to it an unfulfilled pregnancy-wish.
 
-In this study, we'll focus on fertility prediction, which incorporates both use cases, and thus we will not train
-different models for each use-case.
-However, we will test the thresholds used for decision-making to find use-case dependent optimums.
+In this study, I will focus on fertility prediction, which incorporates both use cases, and thus I will not train
+different models for each use case.
+However, I will test the thresholds used for decision-making to find use case dependent optimums.
 Section~\ref{subsubsec:use_case_evaluation} will introduce the methodology in more detail.
 
 \subsubsection{Physiological Signs of Ovulation}\label{subsubsec:physiological_signs}
@@ -180,7 +183,7 @@ OvulaRing has been financed by the startup grant `EXIST Gründerstipendium` from
 The device continuously records intravaginal core body temperature at 5-minute intervals.
 The sensor itself measures approximately 1\,cm $\times$ 1\,cm $\times$ 2\,cm and is embedded in a silicone ring with a diameter of 5\,cm for ease of use.
 It pairs with a smartphone via Bluetooth to synchronize and upload recorded data to a secure database.
-Figure~\ref{fig:background_ovularing} shows an image of the ring attached to its silicone ring.
+Figure~\ref{fig:background_ovularing} shows an image of the sensor attached to its silicone ring.
 
 The product has been on the market for over a decade, resulting in an extensive longitudinal dataset of menstrual cycles.
 Cycle boundaries are defined by self-reported menstruation, which users manually log in the accompanying app to mark the beginning of each cycle.
@@ -269,7 +272,7 @@ With a large, high-resolution dataset of longitudinal temperature measurements a
 the next challenge lies in how to model such sequential data effectively.
 Accurate ovulation prediction requires algorithms that can handle temporal dependencies,
 irregularities, and physiological variability across users.
-To this end, we turn to machine learning techniques designed for time series analysis,
+To this end, I turn to machine learning techniques designed for time series analysis,
 beginning with foundational concepts and progressing to modern neural architectures.
 
 \subsubsection{Time Series Analysis}\label{subsubsec:time_series_analysis}
@@ -284,7 +287,8 @@ These recurrent connections allow the network to retain information from previou
 from a prior time step into the current one, enabling a form of temporal memory.
 Recurrent Neural Networks (RNNs) process sequences in a strictly sequential manner,
 where each token's representation depends on the preceding token.
-This results in a per-layer time complexity of $\mathcal{O}(n \cdot d^2)$, where $n$ is the sequence length and $d$ is the hidden state size.
+This results in a per-layer time complexity of $\mathcal{O}(n \cdot d^2)$, where $n$ is
+the sequence length and $d$ is the hidden state size~\cite{elman_finding_nodate}.
 
 
 In practice, this means that input data is processed sequentially, one step at a time.
@@ -304,7 +308,7 @@ if they grow exponentially, it results in \emph{exploding gradients}~\cite{hochr
 In both cases, learning is significantly impaired.
 
 Exploding gradients can often be mitigated using techniques such as \emph{gradient clipping},
-where the magnitude of the gradient is capped---typically within a range of \([-1, 1]\)---to stabilize training.
+where the magnitude of the gradient is capped---typically within a range of \([-1, 1]\)---to stabilize training~\cite{pascanu_difficulty_2013}.
 
 Figure~\ref{fig:rnn_unfolded} illustrates the unfolded structure of an RNN across three time steps.
 This technique, known as \emph{unfolding}, clarifies how sequential inputs update the hidden state and generate
@@ -386,15 +390,7 @@ update, and expose information, solving key limitations of vanilla RNNs in seque
 
 LSTMs are widely used in biomedical applications due to their capacity to handle sequences of variable length and complexity.
 In the context of fertility prediction, where hormonal patterns exhibit periodicity but also irregularity,
-LSTMs are well-suited to learn relevant time-dependent signals from sequential physiological measurements.
-
-While powerful, LSTMs can be computationally intensive and sensitive to hyperparameter tuning.
-Therefore, they are often compared with alternative architectures,
-including simpler feedforward networks and more recent attention-based models,
-to evaluate trade-offs in performance, interpretability, and computational cost.
-
-Given their ability to learn from sequences with noisy periodic structure,
-LSTMs offer a natural choice for modeling hormonal and temperature fluctuations across menstrual cycles.
+LSTMs are well-suited to learn relevant time-dependent signals from sequential physiological measurements~\cite{tsukiyama_lstm-phv_2021, min_deep_2016}.
 
 The next section introduces the \emph{Transformer} architecture, a more recent alternative that forgoes
 recurrence in favor of attention mechanisms.
@@ -409,7 +405,7 @@ This design mitigates the limitations of recurrent models, such as long-term mem
 and vanishing gradients.
 However, the per-layer time complexity is $\mathcal{O}(n^2 \cdot d)$, due to the quadratic cost of computing pairwise attention across the $n$ tokens.
 Unlike RNNs, Transformers also cannot inherently process sequences of variable length.
-Additional preprocessing steps, such as padding or windowing must be applied to support these.
+Additional preprocessing steps, such as padding and masking must be applied to support these.
 
 Originally introduced for machine translation, Transformers have proven broadly applicable to
 various sequence modeling tasks due to their flexibility, scalability, and strong performance
@@ -447,7 +443,7 @@ to the model regardless of their location, even if they play different syntactic
 Positional encodings, often based on sinusoidal functions, inject a unique position-dependent signal
 into each token, enabling the model to distinguish between identical tokens in different positions.
 
-In this work, we use sine and cosine functions of different frequencies:
+In this work, I use sine and cosine functions of different frequencies:
 \begin{align}
     PE_{\text{pos}, 2i} &= \sin\left(\frac{\text{pos}}{10000^{\frac{2i}{d_{\text{model}}}}}\right), \\
     PE_{\text{pos}, 2i+1} &= \cos\left(\frac{\text{pos}}{10000^{\frac{2i}{d_{\text{model}}}}}\right)
@@ -582,19 +578,24 @@ This makes Transformers well-suited for learning long-range temporal dependencie
 such as ovulatory trends spanning multiple days or cycles.
 
 \subsubsection{Convolutional Layers as Temporal Feature Extractors}
-For high-resolution time-series data, the input dimensionality can become large,
-especially in models like Transformers that process the entire sequence in parallel.
-This can lead to increased memory consumption and slower training.
-To mitigate this and retain as much information as possible, convolutional layers can be used
-to reduce the sequence length while preserving important local patterns~\cite{lecun_gradient-based_nodate}.
 
-In this context, one-dimensional convolutions act as learnable filters that slide over the input sequence to extract temporal features.
-Each filter is parameterized to respond to specific local structures in the data, such as peaks, slopes, or short-term motifs.
-By adjusting the \emph{stride}, the step size of the convolution, the model can control the degree of downsampling,
-effectively reducing the number of time steps passed to subsequent layers.
+Convolutional neural networks (CNNs) can also be applied to time-series by using 1D convolutions across the temporal dimension.
+In this setting, each convolutional filter acts as a learnable temporal pattern detector (e.g. for local peaks, slopes, or motifs).
+Convolutions exploit the local correlation structure of time series: adjacent measurements are often highly related.
+As~\cite{lecun_convolutional_1998} note, “time-series have a strong 1D structure – variables that are temporally nearby are highly correlated.
+Local correlations are the reason for the well-known advantages of extracting and combining local features”~\cite{lecun_convolutional_1998}.
+Convolutional layers enforce locality by restricting each neuron’s receptive field to a contiguous segment of time.
+By adjusting the convolutional stride and using pooling,
+CNNs can downsample the sequence length (reducing resolution) while preserving salient information.
+This reduces the input dimensionality for subsequent layers and can speed up training.
 
-Additional dimensionality reduction can be achieved using pooling operations, such as \emph{max pooling}, which retains only the maximum value within a given window.
-These techniques reduce the computational load while maintaining salient information for downstream processing.
+In practice, convolutional architectures have achieved strong performance on sequential tasks.
+For example,~\cite{lecun_convolutional_1998} show that a simple Temporal Convolutional Network often outperforms canonical
+recurrent models (like LSTMs) across diverse sequence modeling benchmarks.
+Their experiments suggest that CNNs are “a natural starting point for sequence modeling,”
+especially when temporal features are local or multi-scale.
+In summary, 1D convolutions provide an efficient way to extract local temporal features and compress high-resolution sequences,
+complementing recurrent and attention-based models in time-series analysis
 
 Figure~\ref{fig:background_convolution_example} illustrates a simple one-dimensional convolution applied to a sequence using a filter of size 3.
 The stride determines how far the filter moves at each step, affecting both the resolution and length of the resulting feature map.