further work on discussion

This commit is contained in:
Alex Blank
2025-08-25 11:38:26 +00:00
parent b71da9edc6
commit 56bfc2b95d
3 changed files with 169 additions and 110 deletions
+137 -108
View File
@@ -582,24 +582,6 @@ Each evaluation used a test set of 100 users (100 user-years) and was repeated f
we report means and 95\% confidence intervals (CIs).
\subsubsection{Contraception Use-Case Results}\label{subsubsec:use_case_contraception_results}
\begin{table}
\scriptsize
\begin{tabularx}{\linewidth}{l*{5}{X}}
\toprule
Model & No. of Inter. Events & Pregnancies & Correct Denials & Incorrect Denials \\
\midrule
Convolutional LSTM & 1154 (1149-1158) & 34.9 (34.1-35.7) & 1615 (1611-1620) & 4963 (4954-4971) \\
Transformer & 3200 (3194-3207) & 4.4 (4.1-4.7) & 1817 (1812-1822) & 2717 (2710-2724) \\
Convolutional Transformer & 1156 (1151-1160) & 34.3 (33.5-35.0) & 1613 (1608-1619) & 4974 (4965-4983) \\
LSTM & 3206 (3199-3213) & 4.6 (4.3-4.9) & 1832 (1827-1838) & 2711 (2705-2717) \\
Last-Cycle Baseline & 6066 (6057-6075) & 127.5 (125.9-129.1) & 918 (915-922) & 762 (759-766) \\
Population-Mean Baseline & 5874 (5865-5883) & 153.7 (151.9-155.5) & 749 (746-752) & 1121 (1116-1125) \\
User-Mean Baseline & 5933 (5924-5942) & 105.8 (104.4-107.2) & 1030 (1026-1034) & 776 (772-779) \\
\bottomrule
\end{tabularx}
\caption{Contraception metrics at threshold \textbf{0.01} for all models. Values are means over 200 iterations; 95\% CIs in parentheses.}
\label{tab:results_contraception_use_case_0_01}
\end{table}
\paragraph{Threshold 0.01 (Table~\ref{tab:results_contraception_use_case_0_01}).}
At the strictest threshold of 0.01, Transformer and LSTM models achieve the lowest pregnancy rates, only 4.4 to 4.6
@@ -611,22 +593,74 @@ leading to roughly 3435 pregnancies, or about 30 per 1,000 events—far less
All baseline models perform substantially worse, with pregnancy counts exceeding 100 in all cases, confirming the value of personalized predictions.
\begin{table}
\scriptsize
\begin{tabularx}{\linewidth}{l*{5}{X}}
\toprule
Model & No. of Inter. Events & Pregnancies & Correct Denials & Incorrect Denials \\
\midrule
Convolutional LSTM & 2509 (2502-2515) & 55.0 (54.0-56.0) & 1448 (1444-1452) & 3768 (3760-3775) \\
Transformer & 3801 (3793-3810) & 8.2 (7.8-8.6) & 1768 (1763-1773) & 2162 (2157-2168) \\
Convolutional Transformer & 2510 (2503-2516) & 55.6 (54.5-56.6) & 1448 (1443-1453) & 3776 (3769-3783) \\
LSTM & 3801 (3794-3809) & 8.2 (7.8-8.7) & 1772 (1766-1777) & 2160 (2154-2165) \\
Last-Cycle Baseline & 6106 (6096-6116) & 130.2 (128.7-131.7) & 888 (885-892) & 750 (746-753) \\
Population-Mean Baseline & 5924 (5914-5933) & 156.1 (154.3-157.8) & 716 (713-720) & 1100 (1096-1104) \\
User-Mean Baseline & 5988 (5978-5997) & 108.8 (107.3-110.3) & 998 (994-1002) & 764 (761-768) \\
\bottomrule
\end{tabularx}
\caption{Contraception metrics at threshold \textbf{0.05} for all models. Values are means over 200 iterations; 95\% CIs in parentheses.}
\label{tab:results_contraception_use_case_0_05}
\centering
\begin{subtable}{\textwidth}
\centering
\scriptsize
\begin{tabularx}{\linewidth}{l*{5}{X}}
\toprule
Model & No. of Intercourse Events & Pregnancies & Correct Denials & Incorrect Denials \\
\midrule
Convolutional LSTM & 1154 (1149-1158) & 34.9 (34.1-35.7) & 1615 (1611-1620) & 4963 (4954-4971) \\
Transformer & 3200 (3194-3207) & 4.4 (4.1-4.7) & 1817 (1812-1822) & 2717 (2710-2724) \\
Convolutional Transformer & 1156 (1151-1160) & 34.3 (33.5-35.0) & 1613 (1608-1619) & 4974 (4965-4983) \\
LSTM & 3206 (3199-3213) & 4.6 (4.3-4.9) & 1832 (1827-1838) & 2711 (2705-2717) \\
Last-Cycle Baseline & 6066 (6057-6075) & 127.5 (125.9-129.1) & 918 (915-922) & 762 (759-766) \\
Population-Mean Baseline & 5874 (5865-5883) & 153.7 (151.9-155.5) & 749 (746-752) & 1121 (1116-1125) \\
User-Mean Baseline & 5933 (5924-5942) & 105.8 (104.4-107.2) & 1030 (1026-1034) & 776 (772-779) \\
\bottomrule
\end{tabularx}
\caption{Contraception metrics at threshold \textbf{0.01} for all models.}
\label{tab:results_contraception_use_case_0_01}
\end{subtable}
\vspace{1.5em}
\begin{subtable}{\textwidth}
\centering
\scriptsize
\begin{tabularx}{\linewidth}{l*{5}{X}}
\toprule
Model & No. of Intercourse Events & Pregnancies & Correct Denials & Incorrect Denials \\
\midrule
Convolutional LSTM & 2509 (2502-2515) & 55.0 (54.0-56.0) & 1448 (1444-1452) & 3768 (3760-3775) \\
Transformer & 3801 (3793-3810) & 8.2 (7.8-8.6) & 1768 (1763-1773) & 2162 (2157-2168) \\
Convolutional Transformer & 2510 (2503-2516) & 55.6 (54.5-56.6) & 1448 (1443-1453) & 3776 (3769-3783) \\
LSTM & 3801 (3794-3809) & 8.2 (7.8-8.7) & 1772 (1766-1777) & 2160 (2154-2165) \\
Last-Cycle Baseline & 6106 (6096-6116) & 130.2 (128.7-131.7) & 888 (885-892) & 750 (746-753) \\
Population-Mean Baseline & 5924 (5914-5933) & 156.1 (154.3-157.8) & 716 (713-720) & 1100 (1096-1104) \\
User-Mean Baseline & 5988 (5978-5997) & 108.8 (107.3-110.3) & 998 (994-1002) & 764 (761-768) \\
\bottomrule
\end{tabularx}
\caption{Contraception metrics at threshold \textbf{0.05} for all models.}
\label{tab:results_contraception_use_case_0_05}
\end{subtable}
\vspace{1.5em}
\begin{subtable}{\textwidth}
\centering
\scriptsize
\begin{tabularx}{\linewidth}{l*{5}{X}}
\toprule
Model & No. of Intercourse Events & Pregnancies & Correct Denials & Incorrect Denials \\
\midrule
Convolutional LSTM & 3596 (3588-3603) & 75.2 (74.0-76.4) & 1292 (1288-1297) & 2847 (2840-2853) \\
Transformer & 4226 (4217-4234) & 13.3 (12.8-13.7) & 1692 (1687-1696) & 1822 (1817-1827) \\
Convolutional Transformer & 3592 (3584-3599) & 73.8 (72.7-75.0) & 1296 (1292-1301) & 2846 (2840-2852) \\
LSTM & 4222 (4214-4230) & 13.3 (12.8-13.8) & 1694 (1688-1699) & 1818 (1812-1823) \\
Last-Cycle Baseline & 6145 (6135-6155) & 134.8 (133.3-136.3) & 865 (861-868) & 744 (740-747) \\
Population-Mean Baseline & 5960 (5951-5969) & 160.3 (158.5-162.1) & 694 (691-697) & 1088 (1084-1093) \\
User-Mean Baseline & 6014 (6005-6022) & 111.8 (110.3-113.3) & 969 (965-973) & 758 (755-762) \\
\bottomrule
\end{tabularx}
\caption{Contraception metrics at threshold \textbf{0.10} for all models.}
\label{tab:results_contraception_use_case_0_10}
\end{subtable}
\caption{grouped contraception metrics at thresholds 0.01, 0.05, and 0.10. values are means over 200 iterations; 95\% confidence intervals in parentheses.}
\label{tab:results_contraception_grouped}
\end{table}
\paragraph{Threshold 0.05 (Table~\ref{tab:results_contraception_use_case_0_05}).}
@@ -638,25 +672,6 @@ Convolutional models also allow more events (~2,510) but continue to produce sig
yielding a less favorable risk-benefit profile.
Baselines remain underperforming.
\begin{table}
\scriptsize
\begin{tabularx}{\linewidth}{l*{5}{X}}
\toprule
Model & No. of Inter. Events & Pregnancies & Correct Denials & Incorrect Denials \\
\midrule
Convolutional LSTM & 3596 (3588-3603) & 75.2 (74.0-76.4) & 1292 (1288-1297) & 2847 (2840-2853) \\
Transformer & 4226 (4217-4234) & 13.3 (12.8-13.7) & 1692 (1687-1696) & 1822 (1817-1827) \\
Convolutional Transformer & 3592 (3584-3599) & 73.8 (72.7-75.0) & 1296 (1292-1301) & 2846 (2840-2852) \\
LSTM & 4222 (4214-4230) & 13.3 (12.8-13.8) & 1694 (1688-1699) & 1818 (1812-1823) \\
Last-Cycle Baseline & 6145 (6135-6155) & 134.8 (133.3-136.3) & 865 (861-868) & 744 (740-747) \\
Population-Mean Baseline & 5960 (5951-5969) & 160.3 (158.5-162.1) & 694 (691-697) & 1088 (1084-1093) \\
User-Mean Baseline & 6014 (6005-6022) & 111.8 (110.3-113.3) & 969 (965-973) & 758 (755-762) \\
\bottomrule
\end{tabularx}
\caption{Contraception metrics at threshold \textbf{0.10} for all models. Values are means over 200 iterations; 95\% CIs in parentheses.}
\label{tab:results_contraception_use_case_0_10}
\end{table}
\paragraph{Threshold 0.10 (Table~\ref{tab:results_contraception_use_case_0_10}).}
A further increase to 0.10 raises Transformer/LSTM events to ~4,220, but also raises pregnancies to ~13.3 (3.1 per 1,000).
This is a ~11\% gain in access compared to 0.05, but the pregnancy count increases by ~62\%.
@@ -669,24 +684,77 @@ Transformer and LSTM models perform best, allowing a relatively high number of i
This threshold offers the best compromise and is selected as the most promising setting for contraceptive use.
\subsubsection{Pregnancy Use-Case Results}\label{subsubsec:use_case_pregnancy_results}
\begin{table}
\scriptsize
\begin{tabularx}{\linewidth}{l*{5}{X}}
\toprule
Model & No. of Inter. Events & Pregnancies & Correct Deferrals & Incorrect Deferrals \\
\midrule
Convolutional LSTM & 4899 (4890-4908) & 216.9 (214.8-219.1) & 10038 & 1734 \\
Transformer & 3685 (3678-3692) & 264.9 (262.7-267.0) & 17489 & 350 \\
Convolutional Transformer & 4903 (4894-4912) & 214.3 (212.2-216.4) & 10038 & 1734 \\
LSTM & 3688 (3680-3696) & 266.9 (264.7-269.1) & 17489 & 350 \\
Last-Cycle Baseline & 1536 (1531-1541) & 138.7 (137.1-140.2) & 24593 & 4038 \\
Population-Mean Baseline & 1704 (1699-1709) & 113.5 (112.1-114.9) & 22956 & 4841 \\
User-Mean Baseline & 1652 (1647-1656) & 158.4 (156.8-160.0) & 24528 & 3523 \\
\bottomrule
\end{tabularx}
\caption{Pregnancy metrics at threshold \textbf{0.01} for all models. Values are means over 200 iterations; 95\% CIs in parentheses.
\centering
\begin{subtable}{\textwidth}
\centering
\scriptsize
\begin{tabularx}{\linewidth}{l*{5}{X}}
\toprule
Model & No. of Intercourse Events & Pregnancies & Correct Deferrals & Incorrect Deferrals \\
\midrule
Convolutional LSTM & 4899 (4890-4908) & 216.9 (214.8-219.1) & 10038 & 1734 \\
Transformer & 3685 (3678-3692) & 264.9 (262.7-267.0) & 17489 & 350 \\
Convolutional Transformer & 4903 (4894-4912) & 214.3 (212.2-216.4) & 10038 & 1734 \\
LSTM & 3688 (3680-3696) & 266.9 (264.7-269.1) & 17489 & 350 \\
Last-Cycle Baseline & 1536 (1531-1541) & 138.7 (137.1-140.2) & 24593 & 4038 \\
Population-Mean Baseline & 1704 (1699-1709) & 113.5 (112.1-114.9) & 22956 & 4841 \\
User-Mean Baseline & 1652 (1647-1656) & 158.4 (156.8-160.0) & 24528 & 3523 \\
\bottomrule
\end{tabularx}
\caption{Pregnancy metrics at threshold \textbf{0.01} for all models. }
\label{tab:results_pregnancy_use_case_0_01}
\end{subtable}
\vspace{1.5em}
\begin{subtable}{\textwidth}
\centering
\scriptsize
\begin{tabularx}{\linewidth}{l*{5}{X}}
\toprule
Model & No. of Intercourse Events & Pregnancies & Correct Deferrals & Incorrect Deferrals \\
\midrule
Convolutional LSTM & 6174 (6164-6184) & 236.1 (234.1-238.2) & 4332 & 1076 \\
Transformer & 4253 (4245-4261) & 270.9 (268.6-273.1) & 14825 & 190 \\
Convolutional Transformer & 6173 (6164-6182) & 235.7 (233.6-237.9) & 4332 & 1076 \\
LSTM & 4255 (4247-4262) & 273.0 (270.6-275.3) & 14825 & 190 \\
Last-Cycle Baseline & 1581 (1576-1586) & 139.8 (138.2-141.3) & 24327 & 4091 \\
Population-Mean Baseline & 1752 (1747-1757) & 117.1 (115.6-118.6) & 22655 & 4888 \\
User-Mean Baseline & 1695 (1690-1701) & 158.3 (156.6-160.1) & 24271 & 3567 \\
\bottomrule
\end{tabularx}
\caption{Pregnancy metrics at threshold \textbf{0.05} for all models.}
\label{tab:results_pregnancy_use_case_0_05}
\end{subtable}
\vspace{1.5em}
\begin{subtable}{\textwidth}
\centering
\scriptsize
\begin{tabularx}{\linewidth}{l*{5}{X}}
\toprule
Model & No. of Intercourse Events & Pregnancies & Correct Deferrals & Incorrect Deferrals \\
\midrule
Convolutional LSTM & 3887 (3880-3895) & 198.0 (196.1-199.9) & 14544 & 2296 \\
Transformer & 3291 (3284-3298) & 260.7 (258.6-262.7) & 19261 & 551 \\
Convolutional Transformer & 3883 (3875-3891) & 198.4 (196.6-200.3) & 14544 & 2296 \\
LSTM & 3296 (3289-3303) & 261.1 (259.1-263.2) & 19261 & 551 \\
Last-Cycle Baseline & 1509 (1504-1513) & 135.2 (133.6-136.8) & 24778 & 4010 \\
Population-Mean Baseline & 1668 (1663-1673) & 109.5 (108.1-110.9) & 23156 & 4811 \\
User-Mean Baseline & 1619 (1614-1624) & 154.0 (152.4-155.6) & 24717 & 3501 \\
\bottomrule
\end{tabularx}
\caption{Pregnancy metrics at threshold \textbf{0.10} for all models.}
\label{tab:results_pregnancy_use_case_0_10}
\end{subtable}
\caption{grouped pregnancy metrics at thresholds 0.01, 0.05, and 0.10. values are means over 200 iterations; 95\% confidence intervals in parentheses.
Under our simulation at a fixed threshold, correct/incorrect deferrals are deterministic; CIs are therefore omitted for these columns.}
\label{tab:results_pregnancy_use_case_0_01}
\label{tab:results_pregnancy_grouped}
\end{table}
\paragraph{Threshold 0.01 (Table~\ref{tab:results_pregnancy_use_case_0_01}).}
@@ -696,25 +764,6 @@ Convolutional models yield fewer pregnancies (~214217) but allow ~4,900 inter
Baselines underperform on both metrics, allowing fewer events and achieving lower pregnancy counts, suggesting they are
overly conservative without yielding benefits in effectiveness.
\begin{table}
\scriptsize
\begin{tabularx}{\linewidth}{l*{5}{X}}
\toprule
Model & No. of Inter. Events & Pregnancies & Correct Deferrals & Incorrect Deferrals \\
\midrule
Convolutional LSTM & 6174 (6164-6184) & 236.1 (234.1-238.2) & 4332 & 1076 \\
Transformer & 4253 (4245-4261) & 270.9 (268.6-273.1) & 14825 & 190 \\
Convolutional Transformer & 6173 (6164-6182) & 235.7 (233.6-237.9) & 4332 & 1076 \\
LSTM & 4255 (4247-4262) & 273.0 (270.6-275.3) & 14825 & 190 \\
Last-Cycle Baseline & 1581 (1576-1586) & 139.8 (138.2-141.3) & 24327 & 4091 \\
Population-Mean Baseline & 1752 (1747-1757) & 117.1 (115.6-118.6) & 22655 & 4888 \\
User-Mean Baseline & 1695 (1690-1701) & 158.3 (156.6-160.1) & 24271 & 3567 \\
\bottomrule
\end{tabularx}
\caption{Pregnancy metrics at threshold \textbf{0.05} for all models. Values are means over 200 iterations; 95\% CIs in parentheses.}
\label{tab:results_pregnancy_use_case_0_05}
\end{table}
\paragraph{Threshold 0.05 (Table~\ref{tab:results_pregnancy_use_case_0_05}).}
At threshold 0.05, Transformer/LSTM models slightly increase intercourse access (~4,250 events) with pregnancies rising to ~271273 (64 per 1,000).
Notably, these models also achieve very low incorrect deferral counts (~190), indicating they rarely block opportunities for conception when they shouldnt.
@@ -723,26 +772,6 @@ Convolutional models allow substantially more intercourse (~6,170) with lower pr
but at the cost of higher incorrect deferrals (~1,076).
This suggests they are more permissive but less selective.
\begin{table}
\scriptsize
\begin{tabularx}{\linewidth}{l*{5}{X}}
\toprule
Model & No. of Inter. Events & Pregnancies & Correct Deferrals & Incorrect Deferrals \\
\midrule
Convolutional LSTM & 3887 (3880-3895) & 198.0 (196.1-199.9) & 14544 & 2296 \\
Transformer & 3291 (3284-3298) & 260.7 (258.6-262.7) & 19261 & 551 \\
Convolutional Transformer & 3883 (3875-3891) & 198.4 (196.6-200.3) & 14544 & 2296 \\
LSTM & 3296 (3289-3303) & 261.1 (259.1-263.2) & 19261 & 551 \\
Last-Cycle Baseline & 1509 (1504-1513) & 135.2 (133.6-136.8) & 24778 & 4010 \\
Population-Mean Baseline & 1668 (1663-1673) & 109.5 (108.1-110.9) & 23156 & 4811 \\
User-Mean Baseline & 1619 (1614-1624) & 154.0 (152.4-155.6) & 24717 & 3501 \\
\bottomrule
\end{tabularx}
\caption{Pregnancy metrics at threshold \textbf{0.10} for all models. Values are means over 200 iterations; 95\% CIs in parentheses.}
\label{tab:results_pregnancy_use_case_0_10}
\end{table}
\paragraph{Threshold 0.10 (Table~\ref{tab:results_pregnancy_use_case_0_10}).}
At the highest threshold, Transformer/LSTM models see a drop in access (~3,290 events) and in pregnancies (~261),
but with an increase in incorrect deferrals (~551).