Allocating Multiple Patients at a Time in Multiple-Objective Response-Adaptive Repeated Measurement Designs

An adaptive allocation rule was previously proposed to build response-adaptive repeated measurement designs that accommodate multiple objectives. Such designs are desirable because they increase both estimation precision and treatment benefit by assigning more patients to a better treatment sequence, but it still leaves something to be desired. In particular, the rule may not be practically useful because responses from all existing patients may not become available by the time a new patient is enrolled for experimentation. In this paper, we extend the allocation strategy from allocating a single patient at a time (one-step look ahead) to allocating several patients at a time (multi-step look ahead). Through the simulations on two-treatment three-period designs, we demonstrate that, with carefully chosen design parameters, adaptive designs are still more efficient than the fixed design ABB/BAA in terms of the mean squared error, and at the same time they maintain well-balanced patient care, even when we allocate two patients at a time. In addition, we observe that there is a curvature relationship between the design efficacy and the sample size in the initial stage of a response-adaptive design.


Introduction
In repeated measurement designs (RMDs), two or more treatments are administered on the study subjects in one of following two ways: 1) in a parallel design, the same treatment is repeatedly applied to a given subject over different periods; 2) in a crossover design, different treatments are applied to a given subject over different periods.In clinical trials, the use of RMDs is very popular for comparing the efficacy of several different treatments (Sindrup et al., 1999;Bate & Jones, 2008).In recent years, there has been a growing interest in designing clinical trials by utilizing past experiences to update the protocols of the trials continuously based on available evidence, such as response-adaptive designs (RADs) (Hu & Rosenberger, 2006).
RADs have conventionally been designed to achieve a single objective, for example, obtaining a precise estimate of a treatment effect (Kushner, 2003), minimizing the number of subjects required for a study (Armitage, 1975), or skewing allocations toward beneficial treatments (Zelen, 1969;Wei & Durham, 1978;Durham & Yu, 1990;Rosenberger et al., 2001;Flournoy et al., 2010).To date, more research has been done to conduct multiple objective designs (Cook & Wong, 1994;Clyde & Chaloner, 1996;Antognini & Giovagnoli, 2010).More recently, we proposed an adaptive allocation strategy to construct multiple-objective RADs, that increase both estimation precision and treatment benefit for trials with continuous outcomes (Liang & Carriere, 2009).Subsequently, we also provided a unified strategy to construct multiple-objective designs for both continuous and dichotomous outcomes (Carriere & Liang, 2010).
In a RAD, the treatment assignment for a new patient depends on the cumulative experiences from all previous patients.In other words, we predict an optimal allocation strategy for an incoming new patient based on all available data (one-step look ahead).However, all outcomes from existing patients may not become available as soon as a new patient enters the study.A straightforward solution to this problem is to predict an allocation strategy for multiple future patients (multi-step look ahead).In this paper, we extend the current multiple-objective adaptive allocation strategy from allocating a single patient at a time (Liang & Carriere, 2009) to allocating multiple patients at a time (multi-step look ahead).In the simulation studies, we focus on two-treatment three-period designs allocating one or two patients at a time, and we compare the two adaptive allocation strategies with two fixed optimal designs under various parameter settings and explore the effect of the sample size in the initial stage of a RAD on design efficacy.This paper is organized as follows.Section 2 reviews and extend the general allocation strategy for building multiple-objective RADs to assign several patients at a time.Section 3 describes the methodology details of the allocation strategy for clinical trials with continuous outcomes under the self and mixed carryover effects model with random subject effects.Section 4 presents the properties of the two-treatment three-period RADs via simulations and compares them with optimal fixed designs.Finally, we present our conclusions and suggestions for further research.

Multi-Step Look Ahead Allocation Strategy
An adaptive allocation strategy in RMDs can accommodate multiple-objectives, to increase both estimation precision and treatment benefit (Liang & Carriere, 2009).To construct RADs accommodating two objectives, we pre-specify the total number of subjects needed (N) and the percentage weight given to the primary objective (λ), and then (1 − λ) is the percentage weight given to a secondary objective.For example, the first objective may be to maximize the information matrix, and the second objective may be to increase favorable treatment experiences in the study.Depending on the nature of the trials, one can use different objectives.In principle, the usual optimal design construction methods are advocated (Laska et al., 1983;Kershner, 1986;Kushner, 2003;Atkinson, et al., 2007) to adaptively determine the treatment sequence for a future patient (Liang & Carriere, 2009).In addition, this adaptive approach is applicable to both discrete and continuous responses, under suitable model assumptions (Carriere & Liang, 2010).
In this paper, we extend this adaptive allocation rule to assign several patients at a time as follows.
Step 1: Choose a desired evaluation function g (Liang & Carriere, 2009).Without loss of generality, we assume that a higher value of g indicates a better treatment sequence from this point forward.Then, randomly assign the first m (m < N) patients to all possible treatments or treatment sequences.
Step 2: Assume that a cohort of (q + 1) new patients (q ≥ 0) are enrolled, denoted as the l th , the (l + 1) th , ... , and the (l + q) th patients (m + 1 ≤ l ≤ N − q), and they will be treated by treatment sequence k l , k l+1 , ... , and k l+q , respectively.Let K = (k l , k l+1 , ..., k l+q ), where the domain of K is all possible combinations of (q + 1) treatment sequences.Note that k l , k l+1 , ... , and k l+q can be identical.For each treatment sequence combination K, calculate the estimated information matrix for the first (l + q) patients, ÂK l+q , based on the first (l − 1) patients' responses and the assumption that the cohort of the new patients will receive the treatment sequence k l , k l+1 , ... , and k l+q , respectively, and compute the evaluation function for each new patient given a particular treatment sequence (i.e.g l−1,k l , g l−1,k l+1 , ... , and g l−1,k l+q ).
Step 3: Choose a treatment sequence combination K * = (k * l , k * l+1 , ..., k * l+q ) from the s q+1 possible treatment sequence combinations for the l th , (l + 1) th , ... , (l + q) th patients, where s = t p is the number of possible treatment sequences in a t-treatment p-period RMD, in such a way that where Equation (1) is our selection criterion, which is designed to achieve two objectives, balancing between estimation precision and overall treatment benefit at the current stage.We choose the criterion in such a way that the first part aims to favor a treatment sequence combination that maximizes the information matrix, ÂK l+q , measured by an optimality criterion Θ(•), which could be the determinant (D-optimality), the trace (A-optimality), or the maximum eigenvalue (E-optimality) of the information matrix.The second part aims to choose a treatment sequence combination that gives the best performance based on the pre-specified evaluation function g.At each stage, one treatment sequence combination K o = (k o l , k o l+1 , ..., k o l+q ) maximizes the optimality criterion, Θ(•), and possibly another treatment sequence combination K B = (k B l , k B l+1 , ..., k B l+q ) has the highest value on the evaluation function.Prior to the experiment, investigators can choose a parameter λ ∈ [0, 1] to weight and balance the two objectives.
Similar to the allocation rule for assigning one patient at a time (q = 0), we have the traditional RAD problem when λ = 1 (Kushner, 2003).When λ = 0, the overall treatment benefit of a treatment sequence evaluated by the pre-specified evaluation function is the only concern.The choice is often driven by what the investigators want to emphasize.In situations where more than one treatment sequence combination achieve the highest score based on the selection criterion, we can randomly assign the patients to one of them.An example of computing the information matrix, ÂK l+q , is provided in Section 3. D-optimality is used in the simulations (Section 4).
Step 4: Repeat steps 2 to 3 until all N patients have been allocated.

The Design Model
It is well known that optimal designs are highly dependent on the chosen design model for the experiment.In general, models for t-treatment p-period RMDs have included effects for an overall mean, periods, direct treatments and carryovers.For carryover effects, much work has assumed simple first-order carryover effects (Hedayat & Afsarinejad, 1978;Carriere, 1994).More recently, another form of carryover effects model was proposed that allowed two different types of carryover effects from each treatment: 1) a self carryover effect which occurs when one treatment is followed by itself, and 2) a mixed carryover effect which occurs when one treatment is followed by another different treatment (Afsarinejad & Hedayat, 2002).We extended Afsarinejad and Hedayat's design model by including random subject effects (Liang & Carriere, 2009), which is the model we consider in this paper.In general, the model is written as: where y i jk denotes the response variable for the j th subject given treatment sequence k in period i, i = 1, 2, ..., p, j = 1, 2, ..., N k , k = 1, 2, ..., s, p is the number of periods, N k is the number of subjects given treatment sequence k, and s = t p is the number of available treatment sequences in a t-treatment p-period RMD.The μ, π i , ξ jk , and ε i jk are the overall mean, period, random subject, and random measurement error effects, respectively.Denoting d k [i, j] as the treatment used for subject j given treatment sequence k in period i, we have represent mixed and self carryover effects, respectively, of a treatment d k [i − 1, j] given in the period (i − 1) into the next period for subject j given treatment sequence k.The δ i jk is an indicator variable, taking Assume that ξ jk , and ε i jk are mutually independent random effects with mean 0 and variance σ 2 ξ and σ 2 ε , respectively.

Computing Information Matrix Adaptively for Two-treatment Three-period Designs
In the selection criterion in Section 2, we need to compute the information matrix and the evaluation functions based on the first (l − 1) patients by the time a cohort of (q + 1) new patients enters the study to be allocated.In this section, we demonstrate how to compute the information matrix for a two-treatment three-period design.Let us consider a matrix representation of model ( 2) where Y jk = (y 1 jk , y 2 jk , ..., y p jk ) T is a p × 1 vector of observations from subject j in treatment sequence k, β is the column vector of unknown parameters, and X k is the design matrix for treatment sequence k.In a two-treatment three-period design, there are eight possible treatment sequences (s = 8): AAA, AAB, ABA, ABB, BBB, BBA, BAB, and BAA, and β = (μ, π 2 , π 3 , τ, γ, ϕ) T , where π 2 and π 3 are dummy variables for the second-and third-period effects, respectively, and τ = (τ A − τ B )/2, γ = (γ A − γ B )/2, and ϕ = (ϕ A − ϕ B )/2 are the contrasts for direct treatment effects, mixed carryover effects, and self carryover effects, respectively.For example, the design matrix X k for a given treatment sequence k = ABB is defined as The observed information matrix given the data from the first (l − 1) patients, H l−1 , is obtained as, where Ĉl−1 is the estimated variance-covariance matrix for the response vector Y jk , N k is the number of patients given treatment sequence k, and the summation is over all treatment sequences used among the first (l − 1) patients.
Under the equi-correlated covariance assumption, the variance-covariance matrix can be estimated as (5) where σ2 ε,l−1 and σ2 ξ,l−1 are the restricted maximum likelihood estimates for σ 2 ε,l−1 and σ 2 ξ,l−1 , respectively, and I [p] is a p × p identity matrix, and 1 [p] is a p × 1 vector of ones (Laird & Ware, 1982).The allocation history and the responses from the first (l − 1) patients are used to estimate σ2 ε,l−1 and σ2 ξ,l−1 .The estimated information matrix, given the history H l−1 and the assumption that the cohort of the (q + 1) new patients will receive the treatment sequence combination K = (k l , k l+1 , ..., k l+q ), becomes where the unknown parameters in Equation ( 6) can be estimated using Equations ( 4) and ( 5).

Numerical Examples
In this section, we illustrate our allocation strategy to construct RMDs for comparing two treatments under the design model ( 2).The parameter of interest is the direct treatment effect contrast, τ = (τ A − τ B )/2.Researchers have shown that, for fixed dual-balanced two-treatment two-period RMDs under the design model that allows for two different types of carryover effects from each treatment, the best linear unbiased estimator of the direct treatment effect contrast is obtained using the data in the first period only (Afsarinejad & Hedayat, 2002;Liang & Carriere, 2010).In addition, we have previously shown that, adaptive allocation strategies can result in unbalanced two-treatment two-period designs and utilize the data from both periods to estimate the direct treatment effect contrast, hence clearly improve the design efficiency (Liang & Carriere, 2009).Therefore, in the paper, we focus on the next simplest type of RMDs, that is, the two-treatment three-period RMDs.

The Simulated Settings
Assume that, at the initial stage, m patients are equally assigned to 8 treatment sequences.Let m (sample size in the initial stage) be 8, 16, 24 and 32, respectively, σ 2 ξ = 2, σ 2 ε = 1, and μ = 100.To be comparable with the previous study (Liang & Carriere, 2009), we define the evaluation function for a given treatment sequence as the summation of all responses from this treatment sequence divided by the number of patients given this treatment sequence; and we assume that a higher value indicates a better treatment sequence.In addition, we consider two sets of parameters: 1) π 2 = π 3 = τ = γ = ϕ = 0 (absence of treatment difference); and 2) π 2 = π 3 = τ = ϕ = 2.5, γ = −2.5 (presence of treatment difference).For example, τ = 2.5 simulates that the treatment A is better than B by 5 units.
The expected outcome for each treatment sequence based on the parameter values used for simulation is summarized in Table 1.When treatment differences are present as described in Table 1, the expected outcome from each treatment sequence in decreasing order is AAA, BAA, AAB/ABA, BAB/BBA, ABB and BBB.That is, treatment sequence AAA is the best of these eight possible treatment sequences, while BBB is the worst.The overall performance of the treatment sequences AAB and ABA is indistinguishable; and similarly, the overall performance of the treatment sequences BAB and BBA is indistinguishable.
To assess the efficiency of a design for estimating τ, a mean squared error (MSE) for τ is computed as MS E = E(τ − τ) 2 , where τ is an estimate of τ.In simulation studies, we estimate MSE by where τ(b) is a maximum likelihood estimator of τ obtained in the b th simulation run for the total B number of simulations.
Denote MS E 1 as the MSE for a proposed adaptive design and MS E 0 as the MSE for a reference design.The relative efficiency (RE) of the adaptive design compared with the reference design is defined as When RE = a > 1, the adaptive design is (a−1)×100% more efficient than the reference design.When RE = a < 1, the adaptive design is only a × 100% as efficient as the reference design.
We consider λ (weight between the two objectives) of 1, 0.9, 0.5, 0.7, 0.3, and 0, and N (total sample size) of 40, 80, and 100.We examine two adaptive allocation rules: assigning one patient at a time (one-step look ahead, q = 0) and assigning two patients at a time (two-step look ahead, q = 1) as described in Section 2. To smooth out the randomness, we report the average allocation results to treatment sequences from 5,000 repetitions.In addition, we compare the adaptive designs with fixed optimal designs ABB/BAA (Laska et al., 1983;Kershner, 1986) and ABA/BAB (Hedayat & Stufken, 2003;Liang & Carriere, 2009;Liang & Carriere, 2010).The computations were done in R version 2.10.1 (The R Foundation for Statistical Computing, Vienna, Austria, 2009).Due to space limit, the results are presented in details in Tables 2 and 3 for λ = 1, 0.5 and 0, N = 40 and 100, and m = 8, 16, and 32.The allocation pattern for other parameter settings of λ, m and N is similar and the detailed simulation results are available upon request.

Absence of Treatment Difference
When π 2 = π 3 = τ = γ = ϕ = 0, there is no treatment advantage.When λ = 0, Table 2 shows that adaptive designs assign an approximately equal number of subjects to each of the eight treatment sequences for all m values under both one-and two-step look ahead strategies.When λ = 1, adaptive designs assign an approximately equal number of subjects to a dual block (a treatment sequence and its dual with treatments in a reverse order), and more subjects are assigned to ABB and its dual.However, adaptive designs use six of the eight sequences rather uniformly.
Estimation of each design parameter with its standard error (data not shown) indicates that the estimated values are unbiased and very close to the true values of the parameters in all cases for both one-and two-step look ahead strategies.For a fixed value of N and m, the standard errors decrease as λ increases.This happens because, as λ increases, we give more emphasis to the precision of the estimators than to the ethnical criteria.For a fixed value of N and λ, the standard errors increase and then decrease as m increases.This reveals a nonlinear relationship between the sample size at the initial stage and the standard errors in estimating the parameters of interest.
The left panel in Table 4 summarizes the empirical coverage probabilities of 95% confidence intervals (CIs) for τ = 0 based on the 5,000 simulations under various adaptive designs and the width of each CI.When λ = 0 and m > 8, the estimate of τ is more precise (narrower CI) but less accurate (lower coverage) as N increases from 40 to 100.This is because, compared to N = 40, the allocation distribution is less uniform for N = 100 (Table 2).However, when λ = 1, both precision and accuracy (coverage) improve as N increases.When λ is 0.7 or less, adaptive designs with m > 8 have a low coverage of about 90% even when sample size is 100.However, in the special case when m = N (not shown), where the adaptive design becomes the fixed design using all 8 treatment sequences equally often, the coverage probabilities quickly improve with increasing sample sizes.This indicates the coverage probability decreases with λ as expected, but it first decreases and then bounces back when m increases.The non-linear relationship between the initial sample size m and the design efficiency is also evident in Figure 1.For estimating the direct treatment contrast, the design using ABA and BAB is the best because it is the optimal design under Model (2) (Hedayat & Stufken, 2003;Liang & Carriere, 2009;Liang & Carriere, 2010).RADs with m = 8 and large λ are almost as efficient as the optimal design ABA/BAB.Design efficiency first decreases, but it bounces back when m approaches to N (fixed design with all 8 treatment sequences).In addition, Figure 1 shows that RADs are more efficient than the fixed design ABB/BAA for large λ under both one-step and two-step look ahead strategies.Overall, there is no discernable disadvantage or loss for allocating more than one patient at a time.

Presence of Treatment Difference
When treatment differences are present, the expected performance of each treatment sequence, in decreasing order, is AAA, BAA, equivalently AAB and ABA, equivalently BAB and BBA, ABB, and BBB (Table 1).Table 3 shows that, when λ = 1, adaptive designs assign an approximately equal number of subjects to a dual block, and more subjects are given to ABB and its dual.However, as λ < 1 and decreases, adaptive designs assign more subjects to the best treatment sequence AAA and fewer subjects to the worst treatment sequence BBB.This is consistent with the parameter setting (Table 1).The allocation is rapidly skewed as λ decreases especially for small m.When treatment differences are present, the impact of using small λ is evident.If the emphasis is placed on increasing the treatment benefit (λ = 0), Design AAA is optimal, followed by BAA.Again allocating two patients at a time does not leave any appreciable impact.
, and μ = 100.Parameter setting II: Similar to the case when the treatment difference is absent, the estimation of each design parameter is very close to the true value and the same pattern in the change of the standard errors is observed as N, m, and λ vary (data not shown).The coverage probability is low when λ is 0.7 or less, and it decreases and then increases as m increases (Table 4).When λ = 0 and m > 8, the estimate of τ is more precise but less accurate as N increases from 40 to 100.This is because, compared to N=40, the allocation distribution is more skewed for N=100 (Table 3).However, when λ = 1, both precision and accuracy improve as N increases.Figure 2 illustrates that adaptive designs with a large value of λ are nearly as efficient as the fixed optimal design ABA/BAB, while also taking the treatment performance into consideration.In general, a RAD with a large λ and m = 8 has no real consequences to the design efficiency, especially when the sample size is large.Use of λ = 0 has low efficiency relative to the reference design ABB/BAA, as N increases.Design efficiency first decreases as m increases but it bounces back when m approaches to N. Again, there is no discernable disadvantage or loss for allocating two patients at a time.

Concluding Remarks
In this paper, we propose a new adaptive allocation strategy to assign multiple patients at a time.We provide a detailed allocation rule for constructing practically useful RMDs adaptively for two-treatment three-period trials using both one-step and two-step look ahead strategies under a mixed and self carryover effects model with random subject effects.We demonstrate that, on average, there is no discernable disadvantage or loss for allocating two patients at a time compared with allocating one patient at a time.Although the adaptive designs sometimes may not be as efficient as the fixed designs in terms of the mean squared error, as expected, they successfully allocate more patients to better treatment sequences, no doubt the best strategy depending upon the study goals.
The investigator can determine the value of λ to balance the two objectives of increasing estimation precision and decreasing the chance of assigning patients to inferior treatment sequences.Our simulated experiments show that the design with a high value of λ can result in allocating more patients to effective treatment sequences without much loss of estimation precision, no matter one-step or two-step look ahead allocation strategy is applied.We do not observe any discernable loss to the value of RADs for allocating two patients at a time.There is virtually no impact to the estimation results and the coverage probability.Further study is needed to examine the impact of allocating more than two patients at a time or under other design models.
It is interesting to observe the non-linear relationship between the initial sample size, m, and the design efficiency.The design efficiency and the coverage probability are the best when m is at a smallest possible, in our case m = 8.They decrease as m increases but it slowly bounces back when m approaches to the total sample size.In summary, based on our simulation, we suggest that one could utilize a two-treatment three-period RAD with 8 patients in the initial stage for the best results in design efficiency and the coverage probability.Further research is needed to confirm these findings and to determine the optimal initial sample size and the number of maximum patients to allocate at a time to further improve efficacy and practicability of RADs.
and μ = 100.Entries are the expected mean vectors Y jk and the summation of responses computed from model (2) under the respective parameters settings.

Table 1 .
Expected outcome for each treatment sequence based on the values used for simulations in a two-treatment three-period design

Table 3 .
Estimated number of patients receiving each treatment sequence using adaptive two-treatment three-period designs when there are treatment differences Entries are based on 5,000 computer replications where there are treatment differences with π 2