Early signs of efficacy are critical in drug development. Response Evaluation Criteria in Solid Tumors (RECIST) are commonly used to determine the efficacy of anti-cancer therapy in clinical trials. RECIST, however, emphasizes the value of tumor shrinkage, while many targeted agents induce prolonged tumor growth arrest. This limits its use for the detection of treatment efficacy for these more cytostatic regimens. Therefore, we designed an individualized variant of a time to progression (TTP) end point based on prospective volumetric measurements and an intra-patient control, the TTP ratio.
Patients and methods
Patients with any metastatic malignancy, without regular treatment options, were treated with the mTOR inhibitor everolimus. Treatment response was determined using both RECIST and the TTP ratio. The TTP ratio was defined as the volumetric pretreatment TTP divided by the volumetric on-treatment TTP. A patient was classified as a responder if the TTP ratio was <0.7. Consistency and reproducibility of volumetric measurements were determined.
Seventy-three patients were included of whom 59 started treatment. A TTP ratio could be established in 73% (n = 43) of the treated patients. The inter-observer agreement for volumetric progression was 0.78 (95% confidence interval 0.70–0.87) (Krippendorff's α-coefficient). According to RECIST, 35 patients (59%) had stable disease (SD) and 1 patient demonstrated a partial response (PR), whereas only 21 patients (36%) met the prespecified criteria for treatment efficacy according to the TTP ratio. Treatment response according to both the TTP ratio and RECIST (SD + PR) correlated with overall survival (OS) [P(log-rank) < 0.001]. The TTP ratio, however, was also able to differentiate which patients had a better OS within the RECIST SD group [P(log-rank) = 0.0496].
The TTP ratio had a high inter-observer agreement, correlated with OS and identified which patients within the RECIST SD group had a longer OS.
Early signs of clinical activity are important in the decision to further develop new drugs. At present, Response Evaluation Criteria in Solid Tumors 1.1 (RECIST)-based parameters such as the response rate (RR) or progression-free survival (PFS) are standard to determine drug efficacy in early clinical trials [
1]. The introduction of targeted and immunomodulatory agents, however, has intensified the debate on the validity of these commonly used endpoints in clinical trials [
2]. Although RR reliably measures significant tumor progression and regression, it lacks the capability to detect growth rate reduction, which may be of great clinical value. This is an important limitation because targeted agents often exert a more cytostatic effect than chemotherapy, resulting in delayed growth rather than objective tumor regression [
3]. Patients with indolent growing tumors will end up in the stable disease (SD) group, obscuring the distinction between a slow natural course of disease and treatment effect. The value of PFS in single-arm studies is also adversely affected by inter-tumor variation in the natural growth rate. A drug-induced decrease in growth rate will not be detected without knowledge of the intrinsic growth rate. Using only RR or PFS as an efficacy end point in early-phase clinical trials may therefore lead to wrongful interpretation of the results with all untoward consequences [
These limitations of RECIST emphasize the need for a reliable parameter of clinical benefit that corrects for growth characteristics of the individual patient's tumor. Such a parameter will not only improve detection of drug efficacy but also support drug development in early clinical trials. Here, we introduce and evaluate a new personalized response parameter to measure the efficacy of targeted therapy: the time to progression (TTP) ratio (Figure 1). The TTP ratio prospectively compares volumetric tumor growth off and on treatment and therefore serves as an intra-patient control for natural tumor growth rate.
Patients with any advanced malignancy, who progressed on their previous treatment and had no regular systemic treatment options left, were eligible for inclusion. Key eligibility criteria included an age of 18 years or older; Eastern Cooperative Oncology Group (ECOG) performance status of ≤2; volumetrically measurable disease; feasibility of histologic tumor biopsy; and adequate hepatic, renal and hematologic function.
study regulatory compliance
The protocol (ClinicalTrials.gov identifier NCT01566279) was approved by the ethical review board of The Netherlands Cancer Institute and complied with the Declaration of Helsinki, Dutch law and Good Clinical Practice. All patients provided written informed consent before study-related procedures.
The CPCT-03 study was an open-label, prospective, single-arm, multicenter intervention study. Study objectives included biomarker identification using RECIST and TTP ratio, evaluating the TTP ratio as a marker for treatment efficacy, and determining PFS, overall survival (OS) and Disease Control Rate (DCR) as defined by RECIST. Patients were accrued at the Netherlands Cancer Institute, UMC Utrecht Cancer Center and Erasmus MC Cancer Institute Rotterdam.
All patients received everolimus 10 mg once daily, orally, on a continuous basis until disease progression according to RECIST. Dose reductions to 5 mg once daily and 5 mg every other day were allowed. A third dose reduction, or treatment interruption of more than 3 weeks, was not allowed.
After study inclusion, the time to an either ≥30% volumetric increase in target lesions or the development of new lesions was determined in a prospective manner, before the treatment with everolimus. This period was called time to progression 1 (TTP1) and represented the natural tumor growth rate. In TTP1, tumor assessments were carried out at baseline, 4 weeks after baseline and every 6 weeks subsequently. In the case of obvious clinical progression during TTP1, a computed tomography (CT) scan was carried out immediately. Subsequently, treatment with everolimus was started and patients were again followed until a ≥30% volumetric increase in target lesions or the development of new lesions. This was called time to progression 2 (TTP2) and represented the growth speed of the tumor under treatment. In TTP2, tumor assessments were carried out every 8 weeks until progressive disease, according to RECIST, was observed. The only exception was patients who were already progressive in TTP1 at 4 weeks, they had their first on-treatment scan at 4 weeks. The TTP ratio was calculated by dividing TTP1 by TTP2. A patient was classified as a responder if the TTP ratio was <0.7. The 0.7 cut-off for response was based on the PFS ratio of Von Hoff et al. [
5]. Von Hoff et al. divided TTP2 by TTP1 (in contrast to TTP1 by TTP2) and used a threshold of >1.33 for response, which corresponds to 0.75 for our TTP ratio. They determined TTP1 under the previous treatment, whereas here it is determined without treatment and the more stringent cut-off 0.7 was chosen.
All tumor assessments were carried out using CT and sent to a central facility. Volumetric measurements were carried out using semiautomatic software (EncoreUnFoie, v5.0, Image Sciences, UMC Utrecht, the Netherlands, 2012). All CT scans were measured by at least two independent observers (GAC, FW, CGMG-H, IU) using the same set of target lesions. At study entry, volumetrically measurable target lesions were selected in adherence to RECIST guidelines [
1]. A lesion was considered volumetrically measurable if its borders could be delimited on every single CT scan slice. Volumetric measurements were carried out by manually contouring the lesion on all axial slices. Subsequently, the volume of each individual lesion was calculated automatically (supplementary Figure S1, available at Annals of Oncology online). The percent change in volume was calculated for the sum of volumes. If there was no consensus on the presence or absence of volumetric progressive disease, a third observer was consulted. An increase of 30% since nadir or more in the cumulative volume of target lesions or appearance of new lesions was considered PD. The 30% cut-off was chosen based on the work of van Kessel et al. [
6], who found that for individual observers, 95% of all repeated lesion measurements fell within the limit of -28.6% and 30.4%. Patients were also evaluated according to the conventional RECIST during the TTP2 period. For all TTP ratio assessable patients, the PFS ratio as described by Von Hoff et al. [
5] was also determined to enable comparison with the TTP ratio. The PFS ratio uses TTP on the most recent line of treatment as an intra-patient control.
- Supplement Figure S1
evaluability of patients
Patients were not evaluable for TTP ratio if they did not complete the TTP1 period or if they had a protocol violation, lost their volumetric measurability or stopped treatment due to reasons other than PD [with the exception of patients that had already passed the threshold of response (<0.7)]. Patients were evaluable for RECIST if treatment response was determined on at least one CT.
The majority of analyses were carried out using SPSS Statistics version 22 (IBM). Baseline data were reported with descriptive statistics. PFS and OS curves were constructed using the Kaplan–Meier technique, and analyzed using a log-rank test. Numbers of target lesions were compared using a paired t-test. A Spearman correlation was used to analyze the relation between TTP1 and TTP ratio, TTP1 and the wash-out period of the previous treatment, and baseline tumor volume and percentage change. Inter-observer variability was calculated using R version 3.2.0 (www.r-project.org) with Krippendorff's α-coefficient.
Seventy-three patients were included between 15 August 2012 and 23 April 2014 (supplementary Figure S2, available at Annals of Oncology online). Fifty-nine patients started treatment with everolimus. Reasons for drop-out during TTP1 included clinical deterioration (n = 8), initiation of other treatment (n = 2), toxicity from a previous treatment (n = 1), screen failure (n = 1) or withdrawal of informed consent (n = 1). Baseline patient characteristics are depicted in Table 1.
- Supplement Figure S2
Table 1Baseline patient characteristics
|Demographic or clinical characteristic||No. of patients||%|
|No. of patients||73|
|Head and neck||2||2.7|
|Time since initial diagnosis|
|>6 months to ≤2 years||27||37.0|
|>2 to ≤5 years||23||31.5|
|No. of organs involved|
TTP ratio versus RECIST
To compare the TTP ratio and RECIST, we evaluated several factors, including number of target lesions, concordance of change and response classification. Forty-three (73%) patients reached TTP2 and were evaluable for efficacy using the TTP ratio. Fifty-one (86%) patients were evaluable using RECIST. Reasons for non-evaluability according to TTP ratio included protocol violation (n = 1), loss of volumetric measurability (n = 3) and stop of treatment due to reasons other than PD (n = 12). Patients were not evaluable for RECIST when treatment response was not determined (n = 8). Because not all lesions can be measured volumetrically, we compared the number of target lesions used for RECIST and volumetric measurements. Within patients evaluable for both methods, fewer lesions were selected as target lesions for volumetric measurements [mean 2.5 (±1.0 SD)] compared with RECIST [mean 3.0 (±1.2 SD)]. This difference was statistically significant (P < 0.001, paired t-test). Volumetric and RECIST measurements were concordant in measuring either tumor growth or regression in 79% of cases (supplementary Figure S3, available at Annals of Oncology online).
- Supplement Figure S3
Using standard RECIST, most patients were classified as having SD (59%, Table 2). Twenty-five percent of patients were classified as progressive (PD) and one patient had a partial response (PR). Using the TTP ratio, 36% of patients were classified as responders and 37% as non-responders. The RECIST SD cohort could be split in 20 TTP ratio responders and 8 non-responders.
Table 2Efficacy of everolimus
|All patients (n = 59)||Colorectal (n = 17)||Neuro-endocrine (n = 9)||Esophageal (n = 5)||Breast (n = 4)|
|TTP ratio (n, %)|
|Response (<0.7)||21 (36%)||5 (29%)||2 (22%)||3 (60%)||3 (75%)|
|Non-response (≥0.7)||22 (37%)||10 (59%)||2 (22%)||0 (—)||0 (—)|
|Unknown||16 (27%)||2 (12%)||5 (56%)||2 (40%)||1 (25%)|
|Best response (n, %)|
|CR||0 (—)||0 (—)||0 (—)||0 (—)||0 (—)|
|PR||1 (2%)||0 (—)||0 (—)||1 (20%)||0 (—)|
|SD||35 (59%)||8 (47%)||6 (67%)||3 (60%)||3 (75%)|
|PD||15 (25%)||8 (47%)||1 (11%)||1 (20%)||0 (—)|
|Unknown||8 (14%)||1 (6%)||2 (22%)||0 (—)||1 (25%)|
|Disease control rate||36 (61%)||8 (47%)||6 (67%)||4 (80%)||4 (75%)|
|Events (n, %)||45 (62%)||14 (82%)||3 (33%)||4 (80%)||3 (75%)|
|Events (n, %)||50 (85%)||17 (100%)||6 (67%)||4 (80%)||4 (100%)|
TTP ratio as an efficacy end point
To evaluate the consistency of measuring volumetric progressive disease, the inter-observer agreement was calculated using Krippendorff's α-coefficient. The inter-observer agreement was 0.78 [95% confidence interval (CI) 0.70–0.87] with 199 evaluated scans. Baseline tumor volume was not correlated to the percentage of change in target lesions (P = 0.413, Spearman). TTP1 was not correlated to TTP ratio (P = 0.551, Spearman) or the wash-out period of the previous treatment (P = 0.251, Spearman).
To explore the predictive value of outcome according to TTP ratio, we analyzed its relation with OS in the TTP ratio evaluable cohort (n = 43). Figure 2A shows OS of responders versus non-responders according to TTP ratio. A significant difference in OS between responders [median 12 months (95% CI 6.0–18.0)] and non-responders [median 4 months (95% CI 2.9–5.1)] was observed [P(log-rank) < 0.001]. There was also a significant difference in OS between the RECIST SD and PR patients versus the PD patients in the same cohort [P(log-rank) < 0.001, Figure 2B]. Because a large proportion of the RECIST SD population were TTP ratio responders, we carried out a separate analysis within the RECIST SD cohort to evaluate if TTP ratio response was correlated to OS within this subgroup (n = 28). The median OS was significantly longer in the TTP ratio responder group [median 11 months (95% CI 4.4–17.6)] than in the non-responder group [median 5 months (95% CI 3.2–6.8)] [P(log-rank) = 0.0496, Figure 2C]. PFS ratio response also correlated to OS [P(log-rank) = 0.008]. However, in contrast to the TTP ratio, response according to PFS ratio was not correlated to OS in the RECIST SD cohort [P(log-rank) = 0.311].
efficacy of everolimus
Within this study, we also evaluated the efficacy of everolimus according to both end points among different tumor types (Table 2). Individual TTP times and ratios are shown in supplementary Figure S4, available at Annals of Oncology online. According to RECIST, high disease control rates (PR + SD) were observed for breast (75%) and esophageal (80%) cancer, including a PR for esophageal cancer. Both tumor types also had a high rate of responders according to the TTP ratio: 60% for esophageal cancer and 75% for breast cancer.
- Supplement Figure S4
All TTP ratio evaluable breast cancers and esophageal adenocarcinomas had a short TTP1 and a response according to the TTP ratio. The squamous cell esophageal carcinomas included a patient with a long TTP1 and response according to the TTP ratio (this patient also had a RECIST PR). The second patient was not evaluable for response according to the TTP ratio. RECIST response was SD. On CT, however, necrosis of the lung metastases was observed (supplementary Figure S5, available at Annals of Oncology online).
- Supplement Figure S5
The majority of patients stopped treatment due to PD (n = 35). Other reasons to stop treatment were adverse events (AEs) (n = 7); toxicity (n = 3); patient refusal (n = 1); clinical deterioration (n = 1); death (n = 1); other (n = 7). At the time of analysis, four patients were still on treatment. AEs are summarized in supplementary Table S1, available at Annals of Oncology online.
- Supplement Table S1
The results of our study suggest that the TTP ratio has additional value when determining the clinical benefit of targeted therapies in early-phase clinical studies. In this phase I population, both TTP ratio and RECIST correlated to OS. However, TTP ratio was also able to differentiate within the RECIST SD group which patients had a longer OS, which could be interpreted as a sign of clinical benefit. TTP ratio measurements were highly reproducible among observers in this study. If validated in other cohorts, this provides an opportunity to determine whether patients classified as having SD actually experienced clinical benefit, and gives more insight as to which patient groups benefit from treatment. Ultimately, we believe the TTP ratio could support drug development by improved detection of early signs of clinical activity.
Furthermore, TTP ratio as an outcome measure was able to detect the efficacy of everolimus in breast and esophageal cancer. Previous studies show that everolimus combined with exemestane is active in breast cancer. However, the beneficial effect of everolimus for esophageal cancer patients has never been fully explored. Early phase studies by Werner et al. [
7] and Wainberg et al. [
8] report low RRs and a large SD population. Because it remains unclear if patients with SD actually benefit from treatment, further studies were discontinued. Our data, however, suggest that we were able to evaluate whether patients within the SD group indeed had a drug-attributable decrease in tumor growth rate. For all esophageal cancer patients in this study (n = 5), this was, in fact, the case. Despite their heavily pretreated status, these patients seemed to benefit from treatment with everolimus. Taking into account small patient numbers, these results may spark an interest to further investigate everolimus in esophageal cancer.
Despite the advantages discussed above, using the TTP ratio as an end point in clinical studies also has several limitations. First, it has been a laborious effort to perform volumetric measurements (in duplicate) of each CT scan. Volumetric measurements are, and will remain, time-consuming procedures until robust and reliable fully automatic software is developed. Secondly, the wait-and-see period to assess natural growth rate initially raised concerns with physicians and patients. Eight patients (11%) were not able to start treatment due to clinical deterioration during the waiting period. Percentagewise, this is comparable to the early drop-out rate in large phase I cohorts [
9]. In this regard, it is important to realize that participants in this study had no other treatment options besides best supportive care or phase I study participation, with a small chance of treatment success, possible suboptimal dosing and unknown toxicity profiles. A wait-and-see approach is also not necessarily disadvantageous. In this study, six patients had a TTP1 of >100 days. These patients had no strict indication to start treatment immediately and their quality of life was not negatively affected by treatment-related AEs during their waiting period. In addition, a first follow-up CT taking place at 4 weeks ensured early detection of highly progressive tumors with a low threshold to start treatment because a volumetric increase of 30% equals a much smaller increase in diameter [
6]. Altogether we feel that the aforementioned considerations legitimate the design of this study and exploratory end point. We cannot exclude the possibility of pseudoprogression in some patients. When adopting the TTP ratio to evaluate the efficacy of treatments that can result in pseudoprogression, we recommend a similar approach as the Immune-Related Response Criteria, namely performing a consecutive CT after 4 weeks to confirm PD.
Previous studies have also recognized the limitations of on-treatment RECIST for targeted therapies and several alternative end points have been explored [
11] such as the tumor growth rate (TGR), by Ferté et al. [
11], which compared tumor growth rate on-treatment and before treatment. They compared TGR and RECIST in a large cohort of renal cancer patients treated with sorafenib or everolimus and found that it facilitated detection of early signs of efficacy and was associated with PFS and OS. However, growth rate before treatment was determined retrospectively in the wash-out period, making it a less reliable end point. Another example is the PFS ratio by Von Hoff et al. where PFS according to RECIST was compared with PFS on the previous treatment [
5]. Although an intra-patient control is used, the success of the previous treatment is a major determinant of efficacy of the treatment of interest. Although PFS ratio also correlated to OS, PFS ratio was not able to differentiate within the RECIST SD group which patients had a longer OS. The TTP ratio, in contrast to the aforementioned examples, is thus far the only efficacy end point in which natural growth rate (via intra-patient control) was prospectively determined and which correlated to OS in the RECIST SD group.
To summarize, we believe that measuring clinical benefit according to TTP ratio is of additional value to standard RECIST measurements when determining the efficacy of targeted therapeutics in early-phase clinical studies as it (i) corrects for the natural growth rate of the tumor, (ii) corresponds well with OS in a phase I population of patients, (iii) is able to differentiate which patients had a longer OS within the SD cohort, (iv) shows high inter-observer agreement and (v) is able to identify potential patient groups (i.e. esophageal cancer) that might benefit from treatment. Our findings warrant further exploration and validation of this approach as it could greatly facilitate early detection of drug efficacy and thereby support drug development.
- Supplement Materials
This work was supported by The Center for Personalized Cancer Treatment, which is financially supported by: Dutch Cancer Society , Amsterdam, The Netherlands (grant HUBR 2011-4880 ); NutsOhra Foundation , Amsterdam, The Netherlands (project 1102-062 ); Barcode for Life , Utrecht, The Netherlands (www.barcodeforlife.nl); Friends of the UMC Utrecht Foundation , Utrecht, The Netherlands (www.vriendenumcutrecht.nl). Everolimus was provided by Novartis.
The authors have declared no conflicts of interest.
- New response evaluation criteria in solid tumours: revised RECIST guideline (version 1.1).Eur J Cancer. 2009; 45: 228-247
- The challenge of choosing appropriate end points in single-arm phase II studies of rare diseases.J Clin Oncol. 2012; 30: 896-898
- Phase II studies of modern drugs directed against new targets: if you are fazed, too, then resist RECIST.J Clin Oncol. 2004; 22: 4442-4445
- Alternate endpoints for screening phase II studies.Clin Cancer Res. 2009; 15: 1873-1882
- Pilot study using molecular profiling of patients’ tumors to find potential targets and select treatments for their refractory cancers.J Clin Oncol. 2010; 28: 4877-4883
- Semi-automatic software increases CT measurement accuracy but not response classification of colorectal liver metastases after chemotherapy.Eur J Radiol. 2012; 81: 2543-2549
- Phase I study of everolimus and mitomycin C for patients with metastatic esophagogastric adenocarcinoma.Cancer Med. 2013; 2: 325-333
- Phase II trial of everolimus in patients with refractory metastatic adenocarcinoma of the esophagus, gastroesophageal junction and stomach: possible role for predictive biomarkers.Cancer Chemother Pharmacol. 2015; 76: 61-67
- Patient selection for oncology phase I trials: a multi-institutional study of prognostic factors.J Clin Oncol. 2012; 30: 996-1004
- Tumor growth rate is an early indicator of antitumor drug activity in phase I clinical trials.Clin Cancer Res. 2014; 20: 246-252
- Tumor growth rate provides useful information to evaluate sorafenib and everolimus treatment in metastatic renal cell carcinoma patients: an integrated analysis of the TARGET and RECORD phase 3 trial data.Eur Urol. 2014; 65: 713-720
Accepted: May 24, 2016
Received in revised form: May 10, 2016
Received: March 7, 2016
© 2016 European Society for Medical Oncology. Published by Elsevier Inc.
User LicenseElsevier user license |
How you can reuse
Elsevier's open access license policy
Elsevier user license
For non-commercial purposes:
- Read, print & download
- Text & data mine
- Translate the article
- Reuse portions or extracts from the article in other works
- Redistribute or republish the final article
- Sell or re-use for commercial purposes
Elsevier's open access license policy