5.1 Notation

We observe RCT data \((Z, X, Y)\), where for each patient \(Z_i= 0, 1\) is the treatment status, \(Y_i = 0, 1\) is the observed outcome and \(X_i\) is a set of covariates measured. Let \(\{Y_i(z), z=0, 1\}\) denote the unobservable potential outcomes. We observe \(Y_i = Z_iY_i(1) + (1 - Z_i)Y_i(0)\). We are interested in predicting the conditional average treatment effect (CATE), \[\tau(x) = E\{Y(0) - Y(1)|X=x\}\] Assuming that \((Z, X, Y)\) is a random sample from the target population and that \(\big(Y(0), Y(1)\big)\perp \!\!\! \perp Z|X\), as we are in the RCT setting, we can predict CATE from \[\begin{align*} \tau(x) &= E\{Y(0)\:\vert\:X=x\}-E\{Y(1)\:\vert\:X=x\}\\ &=E\{Y\:\vert\:X=x, Z=0\}-E\{Y\:\vert\:X=x, Z=1\} \end{align*}\]

Based on an estimate of baseline risk \[ E\{Y\:\vert\:X=x, Z=0\}=g\big(\hat{lp}(x)\big) \] with \(\hat{u}=\hat{lp}(x)=x^t\hat{\beta}\) the linear predictor and \(g\) the link function, we predict CATE from \[ \hat{\tau}(x) = g\big(f(\hat{u}, 0)\big) - g\big(f(\hat{u}, 1)\big) \] where \(f(u,z)\) describes interactions of the baseline risk linear predictor with treatment.

5.2 Simulation settings

For all patients we observe covariates \(X_1,\dots,X_8\), of which \(4\) are continuous and \(4\) are binary. More specifically,

\[\begin{equation*} X_1,\dots,X_4 \sim N(0, 1) \end{equation*}\] \[\begin{equation*} X_5,\dots,X_8 \sim B(1, 0.2) \end{equation*}\]

We first, generate the binary outcomes \(Y\) for the untreated patients (\(Z=0\)), based on

\[\begin{equation} P(Y=1\:\vert\:X=x, Z=0) = g(\beta_0 + \beta_1x_1+\dots+\beta_8x_8) = g(lp_0), \tag{5.1} \end{equation}\]

where \[g(x) = \frac{e^x}{1+e^x}\]

For treated patients, outcomes are generated from:

\[\begin{equation} P(Y = 1\:\vert\:X=x, Z=1) = g(lp_1) \tag{5.2} \end{equation}\]

where \[lp_1 = \gamma_2(lp_0-c)^2+\gamma_1(lp_0-c)+\gamma_0\]

5.3 Base-case scenario

The base-case scenario assumes a constant odds ratio of \(0.8\) in favor of treatment. The simulated datasets are of size \(n=4250\), where treatment is allocated at random using a 50/50 split (80% power for the detection of an unadjusted OR of 0.8, assuming an event rate of 20% in the untreated arm). Outcome incidence in the untreated population is set at \(20\%\). For the development of the prediction model we use the model defined in (5.1) including a constant treatment effect. When doing predictions, \(Z\) is set to \(0\). The value of the true \(\beta\) is such that the above prediction model has an AUC of \(0.75\).

The previously defined targets are achieved when \(\beta=(-2.08, 0.49,\dots,0.49)^t\). For the derivations in the treatment arm we use \(\gamma=(\log(0.8), 1, 0)^t\).

5.4 Deviations from base-case

We deviate from the base-case scenario in two ways. First, we alter the overall target settings of sample size, overall treatment effect and prediction model AUC. In a second stage, we consider settings that violate the assumption of a constant relative treatment effect, using a model-based approach.

For the first part, we consider:

• Sample size:
  • \(n=1064\)
  • \(n=17000\)
• Overall treatment effect:
  • \(OR=0.5\)
  • \(OR=1\)
• Prediction performance:
  • \(AUC=0.65\)
  • \(AUC=0.85\)

We set the true risk model coefficients to be \(\bm{\beta} = \big(-1.63,0.26,\dots,0.26\big)^t\) for \(AUC=0.65\) and \(\bm{\beta} = \big(-2.7,0.82,\dots,0.82\big)^t\) for \(AUC=0.85\). In both cases, \(\beta_0\) is selected so that an event rate of \(20\%\) is maintained in the control arm.

For the second part linear, quadratic and non-monotonic deviations from the assumption of constant relative effect are considered. We also consider different intensity levels of these deviations. Finally, constant absolute treatment-related harms are introduced, i.e. positive (\(0.25\times\text{true average benefit}\)), strong positive (\(0.50\times\text{true average benefit}\)) and negative (\(-0.25\times\text{true average benefit}\); i.e. constant absolute treatment-related benefit). In case of true absent treatment effects, treatment-related harms are set to \(1\%, 2\%\) and \(-1\%\) for positive, strong positive and negative setting, respectively. The settings for these deviations are defined in Table 5.1.