alex/temperature-based-fertility-prediction-thesis
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

hopefully final commit

Browse Source
This commit is contained in:
Alex Blank
2025-09-05 20:02:41 +00:00
parent a2fdf2f0a3
commit bf2e79cc4c
15 changed files with 709 additions and 449 deletions
Download Patch File Download Diff File Expand all files Collapse all files
thesis/sections/background.tex
+55 -33
View File
@@ -71,7 +71,7 @@ Over time, the temperature does not show any significant or longer lasting tempe
To illustrate the diversity of real-world menstrual cycles, Figures~\ref{fig:background_long_cycle} and~\ref{fig:background_short_cycle}
show examples of cycles that are significantly longer or shorter than a normative 28-day cycle.
\citeauthor{bull_real-world_2019} have done an extensive study on cycle variability,
highlighting that women frequently deviate from the normative cycle, especially with age\cite{bull_real-world_2019}.
highlighting that women frequently deviate from the normative cycle, especially with age~\cite{bull_real-world_2019}.
\begin{figure}[htbp]
\centering
@@ -116,7 +116,7 @@ The distribution of fertility probability is not dependent on the length of the
\begin{figure}[htbp]
\centering
\includegraphics[width=0.7\textwidth]{resources/figures/background/background_pregnancy_chance_over_time}
\includegraphics[width=0.7\textwidth]{resources/figures/background/fertility_probability_relative_to_ovulation}
\caption{Chance of fertilization depending on the day of the menstrual cycle.
The highest chance is around one day before ovulation~\cite{dunson_day-specific_1999}.}
\label{fig:background_pregnancy_chance}
@@ -202,7 +202,8 @@ In addition to temperature measurements, the database includes contextual metada
These markers provide further physiological context and may include information about intermediate bleeding, sexual intercourse, or positive pregnancy tests.
At the time of writing, the dataset contains approximately 65{,}000 annotated cycles,
comprising more than 350 million individual temperature measurements~\footnote{This is the largest dataset of continuous body core temperature data used in any study so far.}.
comprising more than 350 million individual temperature measurements.
This is the largest dataset of continuous body core temperature data used in any study so far.
\subsubsection{Dataset Summary}
For the present study, the dataset was reduced to approximately 40{,}000 cycles after filtering out entries that were incomplete,
@@ -281,7 +282,10 @@ Recurrent Neural Networks (RNNs) and the more recent Transformer architecture.
Recurrent Neural Networks (RNNs) are extensions of classical neural networks that incorporate cyclic connections between neurons.
These recurrent connections allow the network to retain information from previous inputs by feeding the hidden state
from a prior time step into the current one, enabling a form of temporal memory.
They process input \emph{sequentially}, maintaining this hidden state over time.
Recurrent Neural Networks (RNNs) process sequences in a strictly sequential manner,
where each token's representation depends on the preceding token.
This results in a per-layer time complexity of $\mathcal{O}(n \cdot d^2)$, where $n$ is the sequence length and $d$ is the hidden state size.
In practice, this means that input data is processed sequentially, one step at a time.
At each step \(t\), the input \(x_t\) is combined with the previous hidden state \(h_{t-1}\) to produce a new
@@ -324,46 +328,61 @@ information: the \emph{forget gate}, the \emph{input gate}, and the \emph{output
\item The \emph{output gate} selects relevant parts of the current cell state to produce the output and the next hidden state.
\end{itemize}
Each gate employs a sigmoid activation function to regulate the flow of information,
allowing LSTMs to preserve and update memory over long sequences.
Each gate uses a sigmoid activation to produce values in \([0, 1]\),
regulating how much information passes through.
This gating mechanism enables LSTMs to maintain relevant information over long sequences
while discarding irrelevant parts.
\begin{figure}[htbp]
\centering
\includegraphics[width=0.6\textwidth]{resources/figures/background/background_lstm_structure}
\caption{Architecture of a memory cell \( c_j \) and its gate units \( \text{in}_j \) and \( \text{out}_j \).
At the center, the self-recurrent linear unit (with weight 1.0) maintains the internal state~\cite{hochreiter_long_1997}.
See text for more details.}
\includegraphics[width=0.6\textwidth]{resources/figures/background/lstm_cell_diagram}
\caption{Architecture of a memory cell and its gate units (Image source: \cite{chevalier_english_2018}).}
\label{fig:lstm_architecture}
\end{figure}
The LSTM unit maintains an internal state \( s_{c_j}(t) \) for each cell $c_j$ that evolves over time to capture long-term dependencies.
Figure~\ref{fig:lstm_architecture} shows the structure of a memory cell along with its input and output gates.
The \emph{input gate} activation \( y_j^{\text{in}}(t) \) controls the influence of the new candidate input \( g(\text{net}_{c_j}(t)) \) on the state update (Eq.~\eqref{eq:state_update}):
The LSTM maintains an internal cell state \(c_t\), which is updated at each time step.
First, the forget gate activation \(f_t\) determines how much of the previous state is retained:
\begin{equation}
s_{c_j}(t) = s_{c_j}(t - 1) + y_j^{\text{in}}(t) \cdot g(\text{net}_{c_j}(t))
f_t = \sigma\left(W_f \cdot [h_{t-1}, x_t] + b_f\right)
\label{eq:forget_gate}
\end{equation}
The input gate \(i_t\) and the candidate cell state \(\tilde{c}_t\) are computed as:
\begin{align}
i_t &= \sigma\left(W_i \cdot [h_{t-1}, x_t] + b_i\right)
\label{eq:input_gate} \\
\tilde{c}_t &= \tanh\left(W_c \cdot [h_{t-1}, x_t] + b_c\right)
\label{eq:candidate_state}
\end{align}
The new cell state \(c_t\) is then updated by combining the retained previous state
with the gated candidate state:
\begin{equation}
c_t = f_t \cdot c_{t-1} + i_t \cdot \tilde{c}_t
\label{eq:state_update}
\end{equation}
The \emph{output gate} \( y_j^{\text{out}}(t) \), computed from \( \text{net}_{\text{out}_j}(t) \), determines how much of the transformed internal state is passed on as visible output (Eq.~\eqref{eq:cell_output}):
\begin{equation}
y_j^{c}(t) = y_j^{\text{out}}(t) \cdot h(s_{c_j}(t))
\label{eq:cell_output}
\end{equation}
\(g\) and \(h\) are differentiable functions, here \emph{tanh}.
The gate activations themselves are derived from net inputs formed as weighted sums of previous outputs \( y^u(t-1) \), as given by:
Next, the output gate \(o_t\) determines how much of the transformed cell state
is exposed as hidden state:
\begin{align}
\text{net}_{\text{out}_j}(t) &= \sum_u w_{\text{out}_j,u} \, y^u(t - 1) \label{eq:net_out} \\
\text{net}_{\text{in}_j}(t) &= \sum_u w_{\text{in}_j,u} \, y^u(t - 1) \label{eq:net_in} \\
\text{net}_{c_j}(t) &= \sum_u w_{c_j,u} \, y^u(t - 1) \label{eq:net_c}
o_t &= \sigma\left(W_o \cdot [h_{t-1}, x_t] + b_o\right)
\label{eq:output_gate} \\
h_t &= o_t \cdot \tanh(c_t)
\label{eq:cell_output}
\end{align}
During training, the weights associated with the inputs \( x_t \) and previous outputs \( h_{t-1} \),
as well as the biases for each gate in a cell, are learned through backpropagation.
To build more expressive models, memory cells can be stacked in multiple layers, and their outputs concatenated or passed sequentially to higher layers.
Here, \(\sigma\) denotes the sigmoid function.
All gates depend on both the previous hidden state \(h_{t-1}\) and the current input \(x_t\).
The weight matrices \(W_f, W_i, W_c, W_o\) and biases \(b_f, b_i, b_c, b_o\)
are learned during training via backpropagation through time.
By combining multiplicative gates and additive memory updates, LSTMs can learn when to forget,
update, and expose information, solving key limitations of vanilla RNNs in sequence modeling tasks.
LSTMs are widely used in biomedical applications due to their capacity to handle sequences of variable length and complexity.
In the context of fertility prediction, where hormonal patterns exhibit periodicity but also irregularity,
@@ -388,6 +407,9 @@ Unlike recurrent neural networks (RNNs), Transformers process input sequences \e
allowing them to model relationships between any pair of input tokens or timesteps directly.
This design mitigates the limitations of recurrent models, such as long-term memory constraints
and vanishing gradients.
However, the per-layer time complexity is $\mathcal{O}(n^2 \cdot d)$, due to the quadratic cost of computing pairwise attention across the $n$ tokens.
Unlike RNNs, Transformers also cannot inherently process sequences of variable length.
Additional preprocessing steps, such as padding or windowing must be applied to support these.
Originally introduced for machine translation, Transformers have proven broadly applicable to
various sequence modeling tasks due to their flexibility, scalability, and strong performance
@@ -401,7 +423,7 @@ linearly projected from the input and used to compute attention scores.
\begin{figure}[htbp]
\centering
\includegraphics[width=0.4\textwidth]{resources/figures/background/background_transformer_architecture}
\caption{The Transformer - architecture for an encoder-decoder model~\cite{vaswani_attention_2017}}
\caption{The Transformer - architecture for an encoder-decoder model (Image source: \cite{vaswani_attention_2017})}
\label{fig:background_transformer_architecture}
\end{figure}
@@ -417,7 +439,7 @@ In the case of machine translation, this input would be a sentence in the source
The input tokens are first mapped to dense continuous vector representations (embeddings).
Since the attention mechanism permutation-invariant, that is, it does not inherently encode the order of tokens in the sequence,
Since the attention mechanism is permutation-invariant, that is, it does not inherently encode the order of tokens in the sequence,
\emph{positional encodings} are added to the token embeddings to provide information about the token positions in the sequence.
Without positional encoding, repeated tokens such as `The` would be indistinguishable
@@ -564,11 +586,11 @@ For high-resolution time-series data, the input dimensionality can become large,
especially in models like Transformers that process the entire sequence in parallel.
This can lead to increased memory consumption and slower training.
To mitigate this and retain as much information as possible, convolutional layers can be used
to reduce the sequence length while preserving important local patterns.
to reduce the sequence length while preserving important local patterns~\cite{lecun_gradient-based_nodate}.
In this context, one-dimensional convolutions act as learnable filters that slide over the input sequence to extract temporal features.
Each filter is parameterized to respond to specific local structures in the data, such as peaks, slopes, or short-term motifs.
By adjusting the \emph{stride}—the step size of the convolution—the model can control the degree of downsampling,
By adjusting the \emph{stride}, the step size of the convolution, the model can control the degree of downsampling,
effectively reducing the number of time steps passed to subsequent layers.
Additional dimensionality reduction can be achieved using pooling operations, such as \emph{max pooling}, which retains only the maximum value within a given window.