fixes

thesis/sections/methodology.tex

@@ -347,6 +347,8 @@ Model-specific architectural parameters are:
     \item \textbf{Number of Layers} — Depth of the LSTM stack.
 \end{itemize}
 The specific values and tuning ranges for these parameters are discussed in Section~\ref{subsubsec:hyperparameter_tuning}.
+A fixed dropout rate of \(p = 0.5\) has been selected for each LSTM-layer expect the last,
+as was shown to be an effective value~\cite{pham_dropout_2014}
 
 \subsubsection{Transformer Architecture}\label{subsubsec:transformer_architecture}
 
@@ -375,6 +377,7 @@ Since the task does not require sequence-to-sequence modeling, only the encoder
 Its output—one vector per input token—is aggregated via 1D adaptive average pooling, resulting in a single vector representation per sequence.
 This vector is then passed through a linear projection layer to produce the two target outputs:
 fertility probability and ovulation-over indicator.
+Note, that in contrast to the original use case of machine-translation, not special tokens are necessary here, as we do not perform sequence-to-sequence prediction.
 
 Figure~\ref{fig:methodology_transformer_architecture} shows the overall architecture.
 The stacked inputs and outputs indicate batch processing.
@@ -386,6 +389,8 @@ Model-specific architectural parameters are:
     \item \textbf{Number of Encoder-Layers} — Number of encoder layers to stack
     \item \textbf{Number of Attention-Heads} — Number of attention heads to use in each layer
 \end{itemize}
+For the Transformer models, a fixed dropout of \(p = 0.1\) was selected, as proposed by\cite{vaswani_attention_2017} in
+the original paper.
 The specific values and tuning ranges for these parameters are discussed in Section~\ref{subsubsec:hyperparameter_tuning}.
 
 \subsubsection{Temporal Convolution Layer}
@@ -555,6 +560,13 @@ joint parameter space in a more efficient and principled manner.
 
 \subsubsection{Training Details}\label{subsubsec:training_details}
 
+The trainings were performed on the scientific compute cluster of Leipzig University on NVIDIA A30 GPUs.
+Due to the heterogeneity of both the models and the trainings, a dynamic batch size algorithm was used, that estimated
+the batch size dynamically during training to optimize the resource usage.
+The learning rate was scaled linearly with the batch size to allow for equivalent convergence behaviour~\cite{goyal_accurate_2018}
+The batch size was capped at 2048 to avoid OOM errors on the system.
+%TODO sources, for square rule
+
 \subsection{Evaluation}\label{subsec:evaluation}
 
 A variety of evaluation metrics have been defined, to best capture each aspect of the performance of an ovulation prediction.