Advertisement

ESMO-Magnitude of Clinical Benefit Scale version 1.1

      ABSTRACT

      Background

      The ESMO Magnitude of Clinical Benefit Scale (ESMO-MCBS) version 1.0 (v1.0) was published in May 2015 and was the first version of a validated and reproducible tool to assess the magnitude of clinical benefit from new cancer therapies. The ESMO-MCBS was designed to be a dynamic tool with planned revisions and updates based upon recognition of expanding needs and shortcomings identified since the last review.

      Methods

      The revision process for the ESMO-MCBS incorporates a nine-step process: Careful review of critiques and suggestions, and identification of problems in the application of v1.0; Identification of shortcomings for revision in the upcoming version; Proposal and evaluation of solutions to address identified shortcomings; Field testing of solutions; Preparation of a near-final revised version for peer review for reasonableness by members of the ESMO Faculty and Guidelines Committee; Amendments based on peer review for reasonableness; Near-final review by members of the ESMO-MCBS Working Group and the ESMO Executive Board; Final amendments; Final review and approval by members of the ESMO-MCBS Working Group and the ESMO Executive Board.

      Results

      Twelve issues for revision or amendment were proposed for consideration; proposed amendments were formulated for eight identified shortcomings. The proposed amendments are classified as either structural, technical, immunotherapy triggered or nuanced. All amendments were field tested in a wide range of studies comparing scores generated with ESMO-MCBS v1.0 and version 1.1 (v1.1).

      Conclusions

      ESMO-MCBS v1.1 incorporates 10 revisions and will allow for scoring of single-arm studies. Scoring remains very stable; revisions in v1.1 alter the scores of only 12 out of 118 comparative studies and facilitate scoring for single-arm studies.
      ESMO10.13039/501100007075

      Introduction

      The ESMO Magnitude of Clinical Benefit Scale (ESMO-MCBS) v1.0 was published in May 2015 [1] and was the first version of a validated and reproducible tool to assess the magnitude of the clinical benefit from new and effective cancer therapies. The tool uses a rational, structured and consistent approach to derive a relative ranking of the magnitude of clinically meaningful benefit that can be expected from a new anti-cancer treatment.
      The ESMO-MCBS was designed to be a dynamic tool with planned revisions and updates based upon recognition of expanding needs and shortcomings identified since the last review. For instance, while there is a recent trend for more medication to be approved on the basis of single-arm studies, the first version only scored comparative studies. Therefore, there was a manifest imperative to formulate and to incorporate an approach to grade the clinical benefit data derived from the registration trials of medications approved on the basis of these single-arm studies. Additional impetus for revision was derived from experience accrued in evaluating a further large cohort of studies published between 2014 and 2016; feedback and queries from clinicians, patients, researchers and representatives of the pharmaceutical industry; and a dynamic process of internal peer review.
      Commitment to ‘accountability for reasonableness’, which represents the ethical gold-standard for a fair priority setting process in public policy [
      • Gruskin S.
      • Daniels N.
      Process is the point: justice and human rights: priority setting and fair deliberative process.
      ,
      • Daniels N.
      Accountability for reasonableness.
      ], demands a process for challenges and appeals and requires that the deliberative process be transparent. In accordance with this standard, the revised v1.1 of the ESMO-MCBS is presented with point-by-point deliberative explanation, including the shortcomings identified in v1.0, description of the amendment in v1.1, and explanation of the rationale including relevant index cases.

      Methodology

      The revision process for the ESMO-MCBS incorporates a nine-step process: 1. Careful review of critiques and suggestions, and identification of problems in the application of v1.0; 2. Identification of shortcomings for revision in the upcoming version; 3. Proposal and evaluation of solutions to address identified shortcomings; 4. Field testing of solutions; 5. Preparation of a near-final revised version for peer review for reasonableness by members of the ESMO Faculty and Guidelines Committee; 6. Amendments based on peer review for reasonableness; 7. Near-final review by members of the ESMO-MCBS Working Group and the ESMO Executive Board; 8. Final amendments; 9. Final review and approval by members of the ESMO-MCBS Working Group and the ESMO Executive Board.
      Twelve issues for revision or amendment were proposed for consideration (Table 1). Of these, one, the use of the lower limit of the 95% confidence interval (CI) rather than the point estimate, had been reviewed extensively by the Working Group and subjected to extensive statistical modelling (to be published in full inESMO Open) and a decision was made not to subject this issue to revision. Three issues have been deferred for further consultation with patient representative organisations. These include toxicity penalties, credit for improvements in quality of life (QoL) or deferred deterioration in QoL and credits for single symptom improvements.
      Table 1Issues generating planned amendments in V1.1 ESMO-MCBS
      1.Use of lower limit of 95% CI versus point estimate
      2.Crediting 2- and 3-year survival advantage <10%
      3.Unscorable randomised studies because of scale deficiencies
      4.No mechanism to credit plateauing in PFS curve (when median PFS gain is small)
      5.Toxicity penalties: differential or not, which adverse effects
      6.Validity of QoL credit
      7.Crediting of single symptom improvements
      8.Inability to score FDA and EMA approved neoadjuvant therapies based on pCR
      9.PFS studies with early termination (based on planned interim survival analysis)
      10.Inability to score medications approved based on single-arm studies
      11.No recognition that some patients treated with non-curable intent, may be cured
      12.PFS penalty for no improvement in OS and QoL did not explicitly refer to mature survival data
      Proposed amendments were formulated for the remaining eight identified shortcomings. The proposed amendments are classified as either structural, technical, immunotherapy triggered or nuanced. Structural amendments (n = 2) involve either the addition of a new form or a restructuring of an existing form with the addition of a new prognostic stratification. Technical amendments (n = 1) involve changes to the instructions regarding the application of the ESMO-MCBS. Immunotherapy triggered amendments (n = 3) refer to amendments triggered by evolving issues derived from the expanding experience in evaluating new immunotherapy-based studies. Nuanced amendments (n = 4) correct shortcomings identified in the scoring of existing forms. The amendments for each of the revised sections of the ESMO-MCBS: Introduction, form 1, forms 2a and 2b, and the new form 3 are detailed in this manuscript.
      Field testing, comparing scores generated on v1.0 and v1.1, was undertaken in 118 comparative studies (supplementary Tables S1–S15, available atAnnals of Oncology online). A pre-final version and the field testing results derived from the proposed amendments were peer reviewed by members of the ESMO Faculty and Guidelines Committee for ‘reasonableness’. Based on feedback from this peer review process, amendments were further adjusted and revised. The final amendments resulted in a change in score for 12 studies: 4 comparative studies having scores upgraded by 1 point and 3 achieving scores in situations where they were not previously scorable (Table 2), and 5 having scores downgraded (Table 3); along with new scoring of 10 single-arm studies (Table 4). The final version (v1.1) is presented in Figure 1 and has been approved by the ESMO-MCBS Working Group and the ESMO Executive Board.
      Table 2Randomized studies which were previously unscorable or whose scores are upgraded m ESMO-MCBS v1.1
      DrugTrial nameDiseaseSettingPrimary outcomePFS controlPFS gainPFS HROS controlOS gainOS HRpCRQoLToxicityESM0- MCBS V1.0ESM0- MCBS V1.1Ref.
      Dacarbazine± ipilimumabMelanoma1st line metastaticOS (crossover allowed)5-year survival 8.8%9.40% 2-year gain 11%4A/4[
      • Robert C.
      • Thomas L.
      • Bondarenko I.
      • et al.
      Ipilimumab plus dacarbazine for previously untreated metastatic melanoma.
      ,
      • Maio M.
      • Grob J.J.
      • Aamdal S.
      • et al.
      Five-year survival rates for treatment-naive patients with advanced melanoma who received ipilimumab plus dacarbazine in a phase III trial.
      ]
      9.1 months2.1 months0.33 (0.24–0.53)
      Pembrolizumab 2 or 10 mg/kg versus docetaxelKEYNOTE- 010Lung2nd line after platinum based therapy OR TKI (for EGFR/ALK mutated) advanced NSCLC >1% tumour cell PD-L1OS All >1%8.5 months (all)1.9 months (2)0.71 (0.58–0.88)Reduced grade 3/4 adverse eventsNo score3[
      • Herbst R.S.
      • Baas P.
      • Kim D.W.
      • et al.
      Pembrolizumab versus docetaxel for previously treated, PD-L1-positive, advanced non-small-cell lung cancer (KEYNOTE-010): a randomised controlled trial.
      ]
      4.2 months (10)0.61 (0.49–0.75)55
      PD-L1 >50%8.2 months6.7 months (2)0.54 (0.38–0.77)55
      9.1 months (10)0.50 (0.36–0.70)55
      Olaparib versus placeboOvarian cancerBRCA ovarian in remissionPFS4.3 months6.9 months0·18 (0·10–0·31)27.8 months2 months0.73 (0.55–0.96)No benefit2a3a[
      • Ledermann J.
      • Harter P.
      • Gourley C.
      • et al.
      Olaparib maintenance therapy in platinum-sensitive relapsed ovarian cancer.
      ,
      • Ledermann J.
      • Harter P.
      • Gourley C.
      • et al.
      Olaparib maintenance therapy in patients with platinum-sensitive relapsed serous ovarian cancer: a preplanned retrospective analysis of outcomes by BRCA status in a randomised phase 2 trial.
      ,
      • Ledermann J.A.
      • Harter P.
      • Gourley C.
      • et al.
      Quality of life during olaparib maintenance therapy in platinum-sensitive relapsed serous ovarian cancer.
      ]
      Trastuzumab and docetaxel± pertuzumabNeoSphereBreastNeo-adjuvant HER2 overexpressed invasive ductal breastpCRNS29% versus 46%No scoreCa[
      • Gianni L.
      • Pienkowski T.
      • Im Y.-H.
      • et al.
      Efficacy and safety of neoadjuvant pertuzumab and trastuzumab in women with locally advanced, inflammatory, or early HER2-positive breast cancer (NeoSphere): a randomised multicentre, open-label, phase 2 trial.
      ,
      • Gianni L.
      • Pienkowski T.
      • Im Y.H.
      • et al.
      5-year analysis of neoadjuvant pertuzumab and trastuzumab in patients with locally advanced, inflammatory, or early-stage HER2-positive breast cancer (NeoSphere): a multicentre, open-label, phase 2 randomised trial.
      ]
      Imiquimod 5% cream (5 or 7 days) versus placeboBasal CellSuperficial basal cell carcinoma in adults with normal immune systems when surgical methods are less appropriatepCR (histologic clearance rates)3% versus 79% (5  days)– 82% (7  days)No scoreC[
      • Geisse J.
      • Caro I.
      • Lindholm J.
      • et al.
      Imiquimod 5% cream for the treatment of superficial basal cell carcinoma: results from two phase III, randomized, vehicle-controlled studies.
      ]
      Combination versus nivolumab versus ipilimumabMelanoma1st line advanced or metastatic melanomaPFSAll 2.9 monthsCombination 8.6 months0.42 (0.31–0.57) Plateau with >10% gain at 1 yearGr 3+ 55% versus 27%23[
      • Larkin J.
      • Chiarion-Sileni V.
      • Gonzalez R.
      • et al.
      Combined nivolumab and ipilimumab or monotherapy in untreated melanoma.
      ]
      Nivolumab 4 months0.57 (0.43–0.76)Gr 3+ 16% versus 29%44
      PD-L1+: 3.9 monthsCombination 10.1 months
      Nivolumab 10.1 months
      PD-L1−: 2.8 monthsCombination 8.4 monthsPlateau with >10% gain at 1 year23
      Nivolumab 2.3 months
      Pembrolizumab (2 versus 10 mg/kg) versus investigator-choiceKEYNOTE- 002Melanoma2nd line metastatic after failure of ipilimumab BRAF or MEK inhibitorPFS2.6 months1.6 months (2 mg/kg)0.45 (0.35–0..57) Plateau PFS gain >10% gain at 1 year3a4a[
      • Ribas A.
      • Puzanov I.
      • Dummer R.
      • et al.
      Pembrolizumab versus investigator-choice chemotherapy for ipilimumab-refractory melanoma (KEYNOTE-002): a randomised, controlled, phase 2 trial.
      ]
      2 months (10 mg/kg)0.39 (0.30–0.51) Plateau with >10% gain at 1 year3a4a
      aRandomised phase II study.
      Table 3Randomised studies whose scores are downgraded in ESMO-MCBS v1.1
      DrugTrial nameDiseaseSettingPrimary outcomePFS controlPFS gainPFS HROS controlOS gainOS HRpCRQoLToxicityESM0- MCBS V1.0ESM0- MCBS V1.1Ref
      T-DM1 versus lapatinib+ capecitabineEMILIABreast2nd line metastatic after trastuzumab failurePFS and OS6.4 months3.2 months0.65 (0.55– 0.77)25.1 months5.8 months0.68 (0.55–0.85)Delayed deterioration54[
      • Verma S.
      • Miles D.
      • Gianni L.
      • et al.
      Trastuzumab emtansine for HER2-positive advanced breast cancer.
      ,
      • Welslau M.
      • Dieras V.
      • Sohn J.H.
      • et al.
      Patient-reported outcomes from EMILIA, a randomized phase 3 study of trastuzumab emtansine (T-DM1) versus capecitabine and lapatinib in human epidermal growth factor receptor 2-positive locally advanced or metastatic breast cancer.
      ]
      Paclitaxel and carboplatin (5 or 6 cycles)± bevacizumab till 18 cycles or progressionICON7OvarianHigh-risk, early stage post-resection or advanced ovarian or primary peritonealPFS stratified for stage and risk of progression(All) 17.4 months2.4 months0.87 (0.77–0.99)(0.77–0.99)NSMinor deterioration11[
      • Stark D.
      • Nankivell M.
      • Pujade-Lauraine E.
      • et al.
      Standard chemotherapy with or without bevacizumab in advanced ovarian cancer: quality-of-life outcomes from the International Collaboration on Ovarian Neoplasms (ICON7) phase 3 randomised trial.
      ,
      • Perren T.J.
      • Swart A.M.
      • Pfisterer J.
      • et al.
      A phase 3 trial of bevacizumab in ovarian cancer.
      ]
      (high risk)
      10.5 months5.5 months0.73(0.60– 0.93)28.8 months7.8 months0.64 (0.48–0.85)43
      FOLFIRI± ramucirumabRAISEColorectal2nd line metastatic after bevacizumab, oxaliplatin, fluoropyrimidineOS11.7 months1.6 months, 2-year survival gain 5%–10%0.84 (0.73–0.97)31[
      • Tabernero J.
      • Yoshino T.
      • Cohn A.L.
      • et al.
      Ramucirumab versus placebo in combination with second-line FOLFIRI in patients with metastatic colorectal carcinoma that progressed during or after first-line therapy with bevacizumab, oxaliplatin, and a fluoropyrimidine (RAISE): a randomised, double-blind, multicentre, phase 3 study.
      ]
      Gemcitabine± nab-paclitaxelPancreatic1st line advanced or metastatic. Good performance status (KPS>70%)OS6.7 months 2-year survival 4%1.8 months 2-year survival gain 5%0.72 (0.62–0.83)32[
      • Von Hoff D.D.
      • Ervin T.
      • Arena F.P.
      • et al.
      Increased survival in pancreatic cancer with nab-paclitaxel plus gemcitabine.
      ]
      Docetaxel± ramucirumabREVELLung2nd line after platinum based therapy NSCLCOS9.1 months1.4 months 2-year survival gain 3–5%0.86 (0.75–0.98)21[
      • Garon E.B.
      • Ciuleanu T.E.
      • Arrieta O.
      • et al.
      Ramucirumab plus docetaxel versus placebo plus docetaxel for second-line treatment of stage IV non-small-cell lung cancer after disease progression on platinum-based therapy (REVEL): a multicentre, double-blind, randomised phase 3 trial.
      ]
      Table 4Single-arm studies scored using form 3 in ESMO-MCBS v1.1
      DrugDiseaseSettingStratificationORRCRDoRPFSSurvivalQoLToxicityESMO- MCBS 1.0ESM0- MCBS 1.1Ref.
      AZD9291 (Osimertib)LungEGFR Inhibitor–Resistant NSCLC phase 2, RR, stratified for EGFR T790M mutationMUT61%6+ months9.6 monthsNo score3[
      • Janne P.A.
      • Yang J.C.
      • Kim D.W.
      • et al.
      AZD9291 in EGFR inhibitor-resistant non-small-cell lung cancer.
      ]
      WT21%2.8 monthsNo score1
      CeritinibLungALK mutated NSCLC stratified for prior/no-prior crizotinibPrior crizotinib56%8.2+ months6.9 monthsNo score3[
      • Shaw A.T.
      • Kim D.W.
      • Mehra R.
      • et al.
      Ceritinib in ALK-rearranged non-small-cell lung cancer.
      ]
      No prior crizotinib62%10.2 monthsNo score3
      CrizotinibLung2nd line metastatic NSCLC whose tumours are ROS1 positive.72%7%17.6 months19.2 monthsNo score3[
      • Shaw A.T.
      • Ou S.H.
      • Bang Y.J.
      • et al.
      Crizotinib in ROS1-rearranged non-small-cell lung cancer.
      ]
      CrizotinibLungNSCLC whose tumours are ALK mutated61.8%10 months9.7 monthsNo score3[
      • Kwak E.L.
      • Bang Y.J.
      • Camidge D.R.
      • et al.
      Anaplastic lymphoma kinase inhibition in non-small-cell lung cancer.
      ,
      • Camidge D.R.
      • Bang Y.J.
      • Kwak E.L.
      • et al.
      Activity and safety of crizotinib in patients with ALK-positive non-small-cell lung cancer: updated results from a phase 1 study.
      ]
      NivolumabHodgkin’sClassical Hodgkin lymphoma that has relapsed or progressed after autologous haematological stem cell transplantation and post-transplantation brentuximab vedotin87%17%8.7 months24+ monthsNo score3[
      • Ansell S.M.
      • Lesokhin A.M.
      • Borrello I.
      • et al.
      PD-1 blockade with nivolumab in relapsed or refractory Hodgkin’s lymphoma.
      ]
      OlaparibOvarianAfter three lines therapy BRCA mutated (germline OR somatic)31%3%7 months7 months16 monthsNo score3[
      • Kaufman B.
      • Shapira-Frommer R.
      • Schmutzler R.K.
      • et al.
      Olaparib monotherapy in patients with advanced cancer and a germline BRCA1/2 mutation.
      ]
      AtezolizumabUrothelialPlatinum resistant locally advanced or metastatic transitional cell cancer stratified by PD-L1 immunohistochemistry score (IC 2/3)26%11%11.7 monthsNo score3[
      • Powles T.
      • Eder J.P.
      • Fine G.D.
      • et al.
      MPDL3280A (anti-PD-L1) treatment leads to clinical activity in metastatic bladder cancer.
      ,
      • Rosenberg J.E.
      • Hoffman-Censits J.
      • Powles T.
      • et al.
      Atezolizumab in patients with locally advanced and metastatic urothelial carcinoma who have progressed following treatment with platinum-based chemotherapy: a single-arm, multicentre, phase 2 trial.
      ]
      EverolimusBrainSubependymal giant-cell astrocytomas in tuberous sclerosisRR: tumour volume (vol.)100% (decrease volume >30%)24+ monthsImprovedNo score4[
      • Krueger D.A.
      • Care M.M.
      • Holland K.
      • et al.
      Everolimus for subependymal giant-cell astrocytomas in tuberous sclerosis.
      ]
      PembrolizumabHead and NeckPD-L1-positive squamous cell H+N carcinoma after 1st line16%5%6+ monthsNo score1[
      • Seiwert T.Y.
      • Burtness B.
      • Mehra R.
      • et al.
      Safety and clinical activity of pembrolizumab for treatment of recurrent or metastatic squamous cell carcinoma of the head and neck (KEYNOTE-012): an open-label, multicentre, phase 1b trial.
      ]
      PembrolizumabColorectal2nd line+metastatic colorectal cancer (MMR-d) (MSI-high (microsatellite instability) who have received prior therapy62%5.9+ monthsNo score3[
      • Le D.T.
      • Uram J.N.
      • Wang H.
      • et al.
      PD-1 blockade in tumors with mismatch-repair deficiency.
      ]
      Lynch27%
      Somatic100%
      INSTRUCTIONS: SHORTCOMINGS IN v1.0 AND AMENDMENT IN v1.1.
      Shortcoming: Instructions regarding subgroup analyses were not adequately clear.
      Amendment: Instructions regarding subgroup analyses were redrafted to more clearly reflect the published eligibility guidelines for the application of the ESMO-MCBS [
      • Cherny N.I.
      • Sullivan R.
      • Dafni U.
      • et al.
      A standardised, generic, validated approach to stratify the magnitude of clinical benefit that can be anticipated from anti-cancer therapies: the European Society for Medical Oncology Magnitude of Clinical Benefit Scale (ESMO-MCBS).
      ]. The rules for subgroup analyses are more clearly specified in the instructions as following:
      • Studies with pre-planned subgroup analyses with a maximum of three subgroups can be graded (provided there is adjustment for multiple comparisons).
      • When statistically significant results are reported for any subgroup, then each of these should be graded separately.
      • Subgroups not showing statistically significant results are not graded.
      • Except for studies that incorporate collection of tissue samples to enable re-stratification based on new genetic or other biomarkers, findings from un-planned (posthoc) subgroup analysis cannot be graded and they can only be used as foundation for hypothesis generation.
      Rationale: Point of clarification.
      Type: Technical.
      Index case: The SQUIRE study of cisplatin and gemcitabine with or without the addition of necitumumab in first-line treatment of patients with metastatic squamous non-small-cell lung cancer included a planned subgroup analysis for tumours expressing EGFR H-score less than or greater than 200 which did not demonstrate any predictive significance [
      • Thatcher N.
      • Hirsch F.R.
      • Luft A.V.
      • et al.
      Necitumumab plus gemcitabine and cisplatin versus gemcitabine and cisplatin alone as first-line therapy in patients with stage IV squamous non-small-cell lung cancer (SQUIRE): an open-label, randomised, controlled phase 3 trial.
      ]. In a subsequent publication, the authors published results of an unplanned exploratory re-analysis of the data re-stratifying the cohorts to EGFR-expressing or non-EGFR-expressing [
      • Paz-Ares L.
      • Socinski M.
      • Shahidi J.
      • et al.
      Correlation of EGFR-expression with safety and efficacy outcomes in SQUIRE: a randomized, multicenter, open-label, phase III study of gemcitabine-cisplatin plus necitumumab versus gemcitabine-cisplatin alone in the first-line treatment of patients with stage IV squamous non-small cell lung cancer.
      ]. In the EGFR-expressing subgroup an OS advantage was reported: HR 0.79 (0.69–0.92) absolute gain in median survival 1.7 months and with a 2-year survival gain of 3%–5%.
      An appeal was submitted by the study sponsor to have the unplanned exploratory re-analysis accredited with ESMO-MCBS score 3.
      The ESMO-MCBS Working Group evaluated the appeal and also noted the authors’ conclusions ‘these data should be interpreted in the context of the SQUIRE ITT results and caution should be exercised, especially when it comes to the outcomes of smaller subgroups…. Further analyses of the data are needed in order to better understand the potential predictive role of EGFR copy number gain in this setting’. The appeal to score the unplanned exploratory analysis was declined.
      FORM 1: SHORTCOMINGS IN v1.0 AND AMENDMENT IN v1.1.
      Shortcoming: Version 1.0 had no framework to score adjuvant therapies approved on the basis of pathological complete remission (pCR).
      Amendment: New criteria for grade C have been inserted ‘Improvements in pCR (pathological complete remission) alone (primary end point) by ≥30% relative gain AND ≥15% absolute gain in studies without mature survival data’.
      Type: Nuanced.
      Rationale: Pertuzumab was approved by both the European Medicines Agency (EMA) and US Food and Drug Administration (FDA) in addition to chemotherapy and trastuzumab in the neoadjuvant treatment of HER2 overexpressed invasive ductal breast cancer, on the basis of substantial improvement in the pCR rate of patients with HER2 overexpressing invasive ductal breast cancer. The approval by the FDA is an accelerated (provisional) approval. There continues to be substantial controversy regarding the validity of pCR as a surrogate for overall survival (OS) in neoadjuvant studies in solid tumours. It is graded at a level C because of this on-going controversy [
      • Rose B.S.
      • Winer E.P.
      • Mamon H.J.
      Perils of the pathologic complete response.
      ].
      Index case: In the randomised phase II NeoSphere trial of neoadjuvant therapy with trastuzumab and docetaxel ±pertuzumab for HER2 overexpressed invasive ductal breast cancer, there was a 17% increase in the pCR rate (46% versus 29%) [
      • Gianni L.
      • Pienkowski T.
      • Im Y.-H.
      • et al.
      Efficacy and safety of neoadjuvant pertuzumab and trastuzumab in women with locally advanced, inflammatory, or early HER2-positive breast cancer (NeoSphere): a randomised multicentre, open-label, phase 2 trial.
      ]. Based on this, the FDA granted accelerated (provisional) approval for the addition of pertuzumab to chemotherapy and trastuzumab in the neoadjuvant setting and the EMA approved the indication with a provision for ‘additional monitoring’. ESMO-MCBS v1.0 was not able to score this trial; in v1.1 it scores grade C (Improvements in pCR alone, as primary end point, by ≥30% relative AND ≥15% absolute gain in studies without mature survival data).
      FORM 2a: SHORTCOMINGS IN v1.0 AND AMENDMENT IN v1.1.
      Shortcoming: The thresholds for absolute gain in median survival in the subsection for patients with a median OS >12 months, provided inadequate constraints in the situation of clinical studies where the median OS was very long.
      Amendment: The prognostic stratification for form 2a has been revised, v1.1 incorporates a three-level prognostic stratification: ≤12 months, >12 to ≤24 months, and >24 months. The >24-month stratification is introduced to achieve maximal score if either: HR ≤0.70 AND Gain ≥9 months or increase in 7-year survival of >10%.
      Type: Structural.
      Rationale: For grade assignment in the ESMO-MCBS, the highest grade is assigned to successful trials providing a meaningful relative benefit, taking into account the variability of the estimate coupled with a minimum relevant observed absolute gain (quantitative component). It is thus guided by a dual rule based on the comparison to pre-specified threshold values of the observed relative benefit and the absolute benefit achieved by the therapy. Statistical simulation studies on the scoring criteria for v1.0 identified that for the subgroup of studies in which the median survival for the control arm was very long (>2 years), the absolute gain threshold restricted to 5 months did not apply adequate constraint on scores credited on the basis of HR ≤ 0.70. This was identified as an important deficiency requiring amendment.
      Index cases: In the EMILIA study of T-DM1 versus the combination therapy of lapatinib and capecitabine, the control arm had an OS of 25.1 months and the survival gain was 5.8 months [
      • Verma S.
      • Miles D.
      • Gianni L.
      • et al.
      Trastuzumab emtansine for HER2-positive advanced breast cancer.
      ,
      • Welslau M.
      • Dieras V.
      • Sohn J.H.
      • et al.
      Patient-reported outcomes from EMILIA, a randomized phase 3 study of trastuzumab emtansine (T-DM1) versus capecitabine and lapatinib in human epidermal growth factor receptor 2-positive locally advanced or metastatic breast cancer.
      ], thus generating the same preliminary score of 4 as the CLEOPATRA study of paclitaxel and trastuzumab with or without the addition or pertuzumab in which the control arm OS was 40.8 months and the gain was 15.7 months [
      • Swain S.M.
      • Kim S.-B.
      • Cortés J.
      • et al.
      Pertuzumab, trastuzumab, and docetaxel for HER2-positive metastatic breast cancer (CLEOPATRA study): overall survival results from a randomised, double-blind, placebo-controlled, phase 3 study.
      ,
      • Baselga J.
      • Cortés J.
      • Kim S.-B.
      • et al.
      Pertuzumab plus trastuzumab plus docetaxel for metastatic breast cancer.
      ,
      • Swain S.M.
      • Baselga J.
      • Kim S.-B.
      • et al.
      Pertuzumab, trastuzumab, and docetaxel in HER2-positive metastatic breast cancer.
      ,
      • Cortes J.
      • Baselga J.
      • Im Y.H.
      • et al.
      Health-related quality-of-life assessment in CLEOPATRA, a phase III study combining pertuzumab with trastuzumab and docetaxel in metastatic breast cancer.
      ]. In v1.1, this anomaly is corrected; the preliminary ESMO-MCBS score for the EMILIA study is downgraded from 4 to 3, and the final ESMO-MCBS score derived from the credit for co-primary outcome of PFS and quality of life benefit is 4.
      Shortcoming: Studies in which there was a long-term plateau in the survival curve (such as in the case of ipilimumab in melanoma) were not credited as having curative potential.
      Amendment: There is a new adjustment to the preliminary scoring: ‘If there is a long-term plateau in the survival curve, and OS advantage continues to be observed at 5 years (or 7 years for diseases with median survival >24 months), also score according to form 1 (treatments with curative potential) and present both scores, i.e. A/4’.
      Type: Immunotherapy triggered.
      Rationale: In some diseases exceptional responders may achieve protracted complete remission (CR) with the possibility of long-term disease control and survival. Historically, this has been described in the setting of small-cell-lung cancer [
      • Takada M.
      • Fukuoka M.
      • Kawahara M.
      • et al.
      Phase III study of concurrent versus sequential thoracic radiotherapy in combination with cisplatin and etoposide for limited-stage small-cell lung cancer: results of the Japan Clinical Oncology Group Study 9104.
      ]. More recently, long-term follow-up data after immunotherapy for metastatic melanoma suggested some patients may achieve long-term disease control with the potential for cure. The potential for cure is indicated by a sustained plateau in the long-term survival data. This amendment recognises that some treatments may have benefits both in a non-curative and curative setting and that the clinical benefit scale ought to credit both. It is important to emphasise that prolonged (5 or 7 year) survival advantage alone, without a plateau in the curve (as has been observed in some instances of breast and prostate cancer), does not meet this criterion.
      Index case: The study of dacarbazine ±ipilimumab in the treatment of metastatic melanoma initially demonstrated a modest survival gain of 2.1 months over a control of 9.1 months with HR 0.69 (0.57–0.84) and with 2-year survival gain of 10.6% scoring ESMO-MCBS 4 [15]. Long-term follow-up subsequently demonstrated a plateau on the survival curve after 3 years, with a 9.4% absolute increase in 5-year OS [
      • Maio M.
      • Grob J.J.
      • Aamdal S.
      • et al.
      Five-year survival rates for treatment-naive patients with advanced melanoma who received ipilimumab plus dacarbazine in a phase III trial.
      ] which meets form 1 criteria or grade A. Consequently, the ESMO-MCBS v1.1 score is A/4.
      Shortcoming: Anomalies in the absolute gain criteria for grade 2 scoring, did not allow scoring for some studies.
      Amendment: Grade 2 scoring across all prognostic groups (based on median OS of the control arm) has been amended to credit studies achieving optimal HR targets but small absolute gains, as well as studies achieving an intermediate HR which achieve median survival gains above a minimal threshold.
      Type: Nuanced.
      Rationale: Statistical modelling of plausible outcome scenarios using v1.0, which incorporated both a lower and upper limit for median survival gain, identified the problem that some studies with intermediate HRs could not be scored because survival gain was outside the defining limits. This problem was also observed in the setting of some low HR scenarios in which the absolute gain in median OS was very modest. The revision corrects these anomalies in the scale function and corrects the lacunae in the scoring continuum.
      Index case: The KEYNOTE-010 study [
      • Herbst R.S.
      • Baas P.
      • Kim D.W.
      • et al.
      Pembrolizumab versus docetaxel for previously treated, PD-L1-positive, advanced non-small-cell lung cancer (KEYNOTE-010): a randomised controlled trial.
      ] compared docetaxel to pembrolizumab 2 or 10 mg/kg in advanced non-small-cell lung cancer with PD-L1 expression on at least 1% of tumour cells. The experimental arm of pembrolizumab 2 mg/kg achieved a 1.9-month gain in median survival (10.4 versus 8.5 months), HR 0.71 (0.58–0.88). In v1.0, there was a grading lacuna that resulted in this scenario being unscorable. With the v1.1 amendment, this scenario is now scorable, and achieves a preliminary score of 2 which is upgraded to a final score of 3 based on reduced toxicity.
      Shortcoming: Grade 1 and grade 2 credit for low magnitude improvements in 2-and 3-year survival, corresponding to gains of<10%, lacked adequate statistical justification.
      Amendment: Grade 1 and grade 2 credit for low magnitude improvements of <10% absolute gain 2-year survival (for studies with control median OS <12 months) and 3-year survival in studies (for studies with median control OS >12 months) have been deleted.
      Type: Nuanced.
      Rationale: Published OS plots typically do not include confidence intervals. When small differences are identified in the tail of a survival curve, when the number of surviving cases is small, the likelihood that confidence intervals would overlap is high and therefore crediting benefit based on such small differences is not statistically justifiable.
      Index cases: According to ESMO-MCBS v1.0, the REVEL study of docetaxel versus ramucirumab in the second-line treatment of non-small-cell lung cancer [
      • Garon E.B.
      • Ciuleanu T.E.
      • Arrieta O.
      • et al.
      Ramucirumab plus docetaxel versus placebo plus docetaxel for second-line treatment of stage IV non-small-cell lung cancer after disease progression on platinum-based therapy (REVEL): a multicentre, double-blind, randomised phase 3 trial.
      ] with control median OS of 9.1 months and median gain of 1.4 months with HR 0.86 (0.75–0.98) would have qualified it for an ESMO-MCBS score of 1 on these considerations alone. However, the score was upgraded to 2 on the basis of a 2-year survival gain of 3%–5% that was observed in the tail of the survival curve. In another two instances, studies with relatively small gains in median survival had been upgraded on the basis of a 2-year survival gain of 5%–10% in the tail of the curve: The addition of ramucirumab to FOLFIRI in advanced colorectal cancer [
      • Tabernero J.
      • Yoshino T.
      • Cohn A.L.
      • et al.
      Ramucirumab versus placebo in combination with second-line FOLFIRI in patients with metastatic colorectal carcinoma that progressed during or after first-line therapy with bevacizumab, oxaliplatin, and a fluoropyrimidine (RAISE): a randomised, double-blind, multicentre, phase 3 study.
      ] was upgraded from 1 to 3 and the superiority of nab-paclitaxel in combination with gemcitabine over gemcitabine alone in advanced and metastatic pancreatic cancer [
      • Von Hoff D.D.
      • Ervin T.
      • Arena F.P.
      • et al.
      Increased survival in pancreatic cancer with nab-paclitaxel plus gemcitabine.
      ] was upgraded from 2 to 3. In v1.1, these upgrades are rescinded because it is highly unlikely such small differences in the tail of the curve are statistically significant.
      In contrast, the CheckMate 057 study demonstrating superiority of nivolumab over docetaxel as second line therapy after platinum-based therapy for non-squamous non-small-cell lung cancer [
      • Borghaei H.
      • Paz-Ares L.
      • Horn L.
      • et al.
      Nivolumab versus docetaxel in advanced nonsquamous non-small-cell lung cancer.
      ,
      • Borghaei H.
      • Brahmer J.R.
      • Horn L.
      • et al.
      Nivolumab vs docetaxel in patients with advanced NSCLC: CheckMate 017/057 2-y update and exploratory cytokine profile analyses.
      ] was upgraded in v1.0 from 3 to 4 on the basis of a >10% increase in 2-year survival. Although this credit may also be statistically dubious, the possibility of it being significant is considered more plausible, and in v1.1 credit continues to be accrued for 2- and 3-year survival gain of this magnitude.
      FORM 2b: SHORTCOMINGS IN v1.0 AND AMENDMENT IN v1.1.
      Shortcoming 1: In v1.0 there was no mechanism to credit plateauing in progression-free survival (PFS) curves. This resulted in low grading for treatments with a small gain in median PFS but improved duration of response (DoR) among the patients who did respond.
      Amendment: The addition of a new adjustment. Upgrade 1 level if there is a long-term plateau in the PFS curve, and there is ≥10% improvement in PFS at 1 year for control median PFS <6 months or ≥10% improvement in PFS at 2 years for median PFS >6 months.
      Type: Immunotherapy triggered.
      Rationale: The rapidly evolving experience with immunotherapy interventions has highlighted the scenario in which very prolonged responses may be observed in a minority of patients. In this scenario, the relatively modest improvement in median PFS may be accompanied by a far more substantial widening of sustained PFS advantage with time, associated with a prolonged DoR. This amendment corrects the anomaly whereby scoring based only on median PFS, without looking at sustained advantage in PFS, understates the magnitude of clinical benefit.
      Index cases: In the study of combination therapy nivolumab plus ipilimumab versus nivolumab alone versus ipilimumab alone as first-line therapy for patients with locally advanced or metastatic melanoma [
      • Larkin J.
      • Chiarion-Sileni V.
      • Gonzalez R.
      • et al.
      Combined nivolumab and ipilimumab or monotherapy in untreated melanoma.
      ], the combination therapy scores only a grade of 2 according to v1.0. The maximal preliminary score of 3 is achieved on the basis of improved median PFS gain of 7.6 months (control 2.9 months), HR 0.42 (0.31–0.57). However, this is adjusted down to 2 because of high toxicity. In v1.1, additional credit is given for plateau in PFS curve with >10% gain at 1 year, thus upgrading the score to 3. Similarly, in the Keynote-002 study of pembrolizumab (2 versus 10 mg/kg) versus investigator-choice second-line therapy for metastatic melanoma after failure of ipilimumab, BRAF or MEK inhibitor [
      • Ribas A.
      • Puzanov I.
      • Dummer R.
      • et al.
      Pembrolizumab versus investigator-choice chemotherapy for ipilimumab-refractory melanoma (KEYNOTE-002): a randomised, controlled, phase 2 trial.
      ], additional credit is given for plateau in PFS curve with >10% gain at 1 year, raising the score from 3 with v1.0 to 4 with the additional adjustment in v1.1.
      Shortcoming 2: PFS studies with early crossover because of early stopping or crossover based on detection of survival advantage at interim analysis, and with no mature OS advantage, are not credited with improving OS in v1.0.
      Amendment: Addition of a new adjustment to ‘Upgrade 1 level if study had early crossover because of early stopping or crossover based on detection of survival advantage at interim analysis.
      Type: Immunotherapy triggered.
      Rationale: Preliminary PFS data are capped at a maximum of grade 3 because of the limitations of PFS as a surrogate for either improvement in OS or improvement in QoL. ESMO-MCBS v1.0 was designed to allow upgrade for PFS studies in which secondary outcome data demonstrated improved OS. In v1.0, the only mechanism for this upgrade was based on improvement in median OS. This approach penalised studies which incorporated early stopping or crossover based on detection of survival advantage at interim analysis, particularly in situations where the interim analysis was performed before median survival was reached. This amendment corrects this anomaly and provides an extra mechanism for crediting improved magnitude of clinical benefit evidenced by survival advantage at interim analysis.
      This upgrade does not apply to situations in which early stopping or crossover are based on detection of PFS advantage at interim analysis.
      Index case: None.
      Shortcoming 3: The penalty for studies with PFS advantage only, with no gain in OS or QoL, did not explicitly indicate that survival data must be mature.
      Amendment: The adjustment criterion has been amended ‘Downgrade 1 level if the drug ONLY leads to improved PFS (mature data shows no OS advantage) and QoL assessment does not demonstrate improved QoL’.
      Type: Nuanced.
      Rationale: It is inappropriate to penalise a study with inconclusive immature interim survival data when mature data may refute the penalty.
      Index case: The phase II Study 19 which evaluated the role of olaparib for BRCA-mutated ovarian cancer in first remission [
      • Ledermann J.
      • Harter P.
      • Gourley C.
      • et al.
      Olaparib maintenance therapy in platinum-sensitive relapsed ovarian cancer.
      ,
      • Ledermann J.
      • Harter P.
      • Gourley C.
      • et al.
      Olaparib maintenance therapy in patients with platinum-sensitive relapsed serous ovarian cancer: a preplanned retrospective analysis of outcomes by BRCA status in a randomised phase 2 trial.
      ,
      • Ledermann J.A.
      • Harter P.
      • Gourley C.
      • et al.
      Quality of life during olaparib maintenance therapy in platinum-sensitive relapsed serous ovarian cancer.
      ] demonstrated a PFS advantage but no survival advantage at an interim analysis of immature survival data. Incorrect application of the penalty for PFS studies which yield neither survival nor QoL benefit led to a downgrading from 3 to 2. This penalty is rescinded in v1.1.
      NEW FORM 3: SINGLE-ARM STUDIES IN‘ORPHAN DISEASES’AND FOR DISEASES WITH‘HIGH UNMET NEED’WHEN PRIMARY OUTCOME IS PROGRESSION FREE SURVIVAL (PFS) OR OVERALL RESPONSE RATE (ORR).
      Shortcoming: In v1.0,there was no mechanism to evaluate medications approved on the basis of single-arm studies.
      Amendment: Development of a new form 3 ‘For single-arm studies in ‘orphan diseases’ and for diseases with ‘high unmet need’ when primary outcome is PFS or ORR’.
      Type: Structural.
      Rationale: In the past 5 years, an increasing number of agents have been approved by regulatory authorities on the basis of data derived from single-arm studies. Overwhelmingly, these approvals have been in the setting of either uncommon diseases, cohorts of small target populations identified by biological markers or situations in which there is no alternative treatment strategy and ‘high unmet need’ for new therapeutic strategies.
      Even in the absence of a comparator arm, there is a perceived need to provide information regarding the magnitude of clinical benefit. The published papers describing the studies of agents which have been approved are characterised by a paucity of data regarding the natural history of untreated patients and the literature is characterised by a presumption of poor outcomes in the absence of disease modifying treatment.
      A review of published studies leading to license indications identified that three major indicators of clinical benefit leading to approval were response rate, either ORR or CR, DoR, and PFS. Infrequently, measurements of QoL have been incorporated.
      Based upon this review, the ESMO-MCBS Working Group has developed a three grade scale for single-arm studies in orphan diseases and for diseases with high unmet need when primary outcome is PFS or ORR. The highest grade of 3 is awarded to studies demonstrating either: median PFS >6 months; ORR > 60%; or ORR ≥20 <60% AND DoR ≥9 months.
      Preliminary scores are adjusted based on toxicity (Downgrade 1 level if there are ≥30% grade 3–4 toxicities impacting on daily well-being), QoL (Upgrade 1 level if improved QoL) or confirmatory phase 4 experience (Upgrade 1 level for confirmatory, adequately sized, phase 4 experience).

      Field testing

      All amendments incorporated into v1.1 have been field tested in 128 studies, reported in 142 publications across a range of 15 disease groups in the comparative studies and 5 disease groups in the single-arm setting (see Table 2, Table 3, Table 4 andsupplementary Tables S1–S15, available atAnnals of Oncology online). Field testing has been reviewed by the ESMO Guidelines Committee and ESMO Faculty for reasonableness. The amended version resulted in scoring changes in 12 randomised studies (Tables 2 and 3) and facilitated scoring for 10 single-arm studies (Table 4).

      Discussion

      The ESMO-MCBS has proven to be a valued and reproducible tool for the evaluation of the magnitude of benefit from clinical studies that is essential for both policy and clinical discourse. Dynamic discussion with end-users who have provided feedback, critiques and suggestions has helped facilitate a dynamic quality improvement process that is essential to maintain the integrity of this important endeavour.
      Since its launch in 2015, the ESMO-MCBS has been widely cited (source Scopus/Google Scholar). It has been applied in clinical and in health technology assessment settings [
      • Wild C.
      • Grössmann N.
      • Bucsics A.
      • et al.
      Utilisation of the ESMO-MCBS in practice of HTA.
      ,
      • Hammerman A.
      • Greenberg-Dotan S.
      • Feldhamer I.
      • et al.
      The Esmo magnitude of clinical benefit scale for novel cancer medicines – correspondence with prioritization decisions in updating the Israeli National List of Health Services.
      ], successfully employed by the ESMO Guidelines Committee [
      • European Society for Medical Oncology
      ESMO Clinical Practice Guidelines News.
      ], demonstrated clinical utility [
      • Kiesewetter B.
      • Raderer M.
      • Steger G.G.
      • et al.
      The European Society for Medical Oncology Magnitude of Clinical Benefit Scale in daily practice: a single institution, real-life experience at the Medical University of Vienna.
      ] and has served as the basis for benchmarking studies [
      • Del Paggio J.C.
      • Azariah B.
      • Sullivan R.
      • et al.
      Do contemporary randomized controlled trials meet ESMO thresholds for meaningful clinical benefit?.
      ].
      ESMO-MCBS v1.1 incorporates 9 amendments in forms 1, 2a and 2b and adds a new form 3 for scoring of single-arm studies. For the most part, the scoring remains very stable. The 10 revisions in v1.1 will alter the scores of only 12 out of 118 comparative studies. In the seven studies impacted by the revision, the newer version either upgrades the score or enables scoring which was previously not possible and in five instances a score is lowered by the amendments.
      A transparent process of development with scope for peer review, appeal and revision is one of the critical criteria for ‘accountability for reasonableness’ [
      • Gruskin S.
      • Daniels N.
      Process is the point: justice and human rights: priority setting and fair deliberative process.
      ,
      • Daniels N.
      Accountability for reasonableness.
      ]. The validity of this revision derives from a careful deliberative process followed by extensive field testing and peer review. This first revision is an integral part of the planned process for on-going development and the Working Group invites feedback, discussion, review and constructive critiques of the revised v1.1 as well as any appeals regarding scores derived using the new version.
      Not all issues could be addressed in this revision. For the next planned revision, the ESMO-MCBS Working Group will be soliciting stakeholder input from patients and patient advocates regarding the toxicity penalties, credit for improvements in QoL or deferred deterioration in QoL and credits for single symptom improvements. This process will involve formal consultations with patient advocacy groups and an open invitation for feedback or suggestions regarding these aspects of the scale.
      The amendments incorporated into v1.1 will be applied to studies previously scored and will prevail until the next planned revision.

       Conclusion

      The 10 amendments incorporated into ESMO-MCBS v1.1 improve the discriminatory capacity and utility of the grading system. These revisions are part of an organisational commitment by ESMO to the continual quality improvement that is essential for accountability for reasonableness, incorporating a transparent revision process based on accrued experience, feedback, consultation and the critical review necessary to promote the integrity of this important endeavour.

      Figure 1. ESMO Magnitude of Clinical Benefit Scale v1.1.

      Instructions
      1. There are five forms:
      ▪ Evaluation form 1: for new approaches to adjuvant therapy or new potentially curative therapies
      Hyper mature data from studies that were un-blinded after compelling early results with subsequent access to the superior arm are contaminated, subsequently late intention to treat (ITT) follow-up data are not evaluable.
      ▪ Evaluation form 2a: for therapies that are not likely to be curative with primary end point of OS with separate sheets for:
      • IF median OS with the standard treatment is ≤12 months
      • IF median OS with the standard treatment >12 months, ≤24 months
      • IF median OS with the standard treatment >24 months
      ▪ Evaluation form 2b: for therapies that are not likely to be curative with primary end point PFS with separate sheets for:
      • IF median PFS with standard treatment ≤6 months
      • IF median PFS with standard treatment >6 months
      ▪ Evaluation form 2c: for therapies that are not likely to be curative with primary end point other than OS or PFS or equivalence (non-inferiority) studies.
      ▪ Evaluation form 3: for single-arm studies in ‘orphan diseases’ and for diseases with ‘high unmet need’ when primary outcome is PFS or ORR.
      2. The highest grade of the ESMO-MCBS is A in the curative setting and this is restricted to new curative treatments; for non-curative indications 5 is the highest possible grade, yet sufficient to trigger rapid consideration for reimbursement is B and 4.
      3. Analysis of phase III trials
      • Adequately powered studies showing statistically significant improvement in the primary outcome (defined byP <0.050)
      • Careful analyses ‘control arm’ and identification of end points.
      • Check subgroup analysis.
        • a. Studies with pre-planned subgroup analyses with a maximum of three subgroups can be graded (provided there is adjustment for multiple comparisons).
        • b. When statistically significant results are reported for any subgroup, then each of these should be graded separately.
        • c. Subgroups not showing statistically significant results are not graded.
        • d. Except for studies that incorporate collection of tissue samples to enable re-stratification based on new genetic or other biomarkers, findings from un-planned (post hoc) subgroup analysis cannot be graded and they can only be used as foundation for hypothesis generation.
      4. More than one outcome may be applicable
      • The statistical significance of secondary outcomes are determined by the same criteria as for primary outcomes ie (defined byP <0.050)
      5. For a required HR, not the point estimate but the lower limit of 95% CI estimated based on the observed HR in the trial should encompass the required HR Example: for threshold set at HR ≤0.65, it is the lower limit of the 95% CI which has to be ≤0.65
      6. In the studies with a primary outcome of OS in the non-curative setting check for
      • Survival gain >10% at 2, 3 or 5 years on Kaplan Meyer plots
      • Long term survival with plateau in Kaplan Meyer plots at 5 or 7 years
      • Reduced toxicity
      • Improvement in quality of life
      • Report final adjusted grade taken into account toxicity, and QoL when relevant
      7. In studies with a primary outcome of PFS in the non-curative setting check for:
      • Indicators of toxicity
      • Survival data also available
      • Early termination with crossover based on planned interim survival analysis
      • Greater than 10% gain in PFA at 2 or 3 years (tail of curve)
      • Global QoL advantage using validated scale if applicable
      • Report final adjusted grade taken into account toxicity, survival advantage and QoL when applicable

      Appendix 1

      Table A1Oncology colleagues who participated in the field testing of ESMO-MCBS v1.1
      First nameLast nameCountryFirst nameLast nameCountry
      DirkArnoldGermany and PortugalAlanHorwichUK
      Paolo A.AsciertoItalyRobertHuddartUK
      IgorAurerCroatiaRavindranKanesvaranSingapore
      AristotelisBamiasGreeceVesaKatajaFinland
      SusanaBanerjeeUKBarbaraKiesewetterAustria
      FabriceBarlesiFranceRobertoLabiancaItaly
      RupertBartschAustriaMarcoLadettoItaly
      AxelBexNetherlandsSibylleLoiblGermany
      ChristianBlankNetherlandsFotiosLoupakisItaly
      GyorgyBodokyHungaryChristineMarosiAustria
      IoannisBoukovinasGreeceErikaMartinelliItaly
      EmilianoCalvoSpainMustafaÖzgüroğluTurkey
      FedericoCappuzzoItalyShaniPaluch-ShimonIsrael
      FatimaCardosoPortugalGeorgePentheroudakisGreece
      RaphaelCataneIsraelSolangePetersSwitzerland
      ThomasCernySwitzerlandSandroPignataItaly
      AndresCervantesSpainCamilloPortaItaly
      AlexandraChambersCanadaMatthiasPreusserAustria
      NicolettaColomboItalyMicheleReniItaly
      Pier FrancoConteItalyElzbietaSenkusPoland
      TanjaCuferSloveniaStefanSleijferNetherlands
      GiuseppeCuriglianoItalyEgbertSmitNetherlands
      SimonEkmanSwedenMichaelStahlGermany
      PilarGarridoSpainChristopherSteerAustralia
      OliverGautschiSwitzerlandKarina DahlSteffensenDenmark
      MicheleGhielminiSwitzerlandMartinVan den BentNetherlands
      SilkeGillessenSwitzerlandJohanVansteenkisteBelgium
      RosGlasspoolUKMarcelVerheijThe Netherlands
      CesareGridelliItalyJanVermorkenBelgium
      JohnHaanenNetherlandsUmbertoVitoloItaly
      AzzaHassanQatarJanWalewskiPoland

      Acknowledgements

      The authors wish to acknowledge the support and contribution of the ESMO Executive Board, the ESMO Faculty, Members of the ESMO Guidelines Committee and the logistics and organisational support provided by ESMO Staff.
      Appendix I lists our oncology colleagues who have participated on the field testing of ESMO-MCBS v1.1 and agreed to place their names in this publication. We also thank those who wished to remain anonymous.

      Funding

      ESMO (no grant numbers apply).

      Disclosure

      The authors have declared the following: JB: Director of EORTC. EORTC conducts many studies sponsored by, or otherwise supported by, a large number of companies. EORTC is an independent research organisation. MJP: Board member: Radius. Consultant (honoraria): AstraZeneca, Lilly, MSD, Novartis, Pfizer, Roche-Genentech, Crescendo Biologics, Periphagen, Huya, Debiopharm, PharmaMar. Research grants to Institute: AstraZeneca, Lilly, MDS, Novartis, Pfizer, Roche-Genentech, Synthon, Radius, Servier. Speakers bureau/stock ownership: none. GP: Consulting and advisory services, research support: Amgen, Merck, Astra Zeneca, Roche, BMS, MSD and Lilly. JYD: Compensated participation to Advisory boards, lecture, Symposia: Amgen, Merck Serono, Bayer, Roche/Genentech, Sanofi, AstraZeneca, Boehringer-Ingelheim, Sirtex until April 2016. No further compensated participation in industry events from May 2016 onwards. JT: Advisory boards for Amgen, Bayer, Boehringer Ingelheim, Celgene, Chugai, Genentech, Lilly, MSD, Merck Serono, Novartis, Pfizer, Roche, Sanofi, Symphogen, Taiho and Takeda. CZ: Honoraria: AstraZeneca, Celgene, Roche, Novartis, Bristol Myers Squibb, MSD, Ariad and Newgen. EGEV: Currently conducting research sponsored by the following companies: Amgen, Roche/Genentech, Chugai Pharma, Synthon, AstraZeneca, Radius Health, CytomX Therapeutics and Nordic Nanovector (all payments to the institution). Consulting or advisory role for the following companies: Synthon, Medication and Merck (all payments to the institution). Not a member of any speakers’ bureau. All remaining authors have declared no conflicts of interest.

      Supplementary data

      References

        • Cherny N.I.
        • Sullivan R.
        • Dafni U.
        • et al.
        A standardised, generic, validated approach to stratify the magnitude of clinical benefit that can be anticipated from anti-cancer therapies: the European Society for Medical Oncology Magnitude of Clinical Benefit Scale (ESMO-MCBS).
        Ann Oncol. 2015; 26: 1547-1573
        • Gruskin S.
        • Daniels N.
        Process is the point: justice and human rights: priority setting and fair deliberative process.
        Am J Public Health. 2008; 98: 1573-1577
        • Daniels N.
        Accountability for reasonableness.
        BMJ. 2000; 321: 1300-1301
        • Thatcher N.
        • Hirsch F.R.
        • Luft A.V.
        • et al.
        Necitumumab plus gemcitabine and cisplatin versus gemcitabine and cisplatin alone as first-line therapy in patients with stage IV squamous non-small-cell lung cancer (SQUIRE): an open-label, randomised, controlled phase 3 trial.
        Lancet Oncol. 2015; 16: 763-774
        • Paz-Ares L.
        • Socinski M.
        • Shahidi J.
        • et al.
        Correlation of EGFR-expression with safety and efficacy outcomes in SQUIRE: a randomized, multicenter, open-label, phase III study of gemcitabine-cisplatin plus necitumumab versus gemcitabine-cisplatin alone in the first-line treatment of patients with stage IV squamous non-small cell lung cancer.
        Ann Oncol. 2016; 27: 1573-1579
        • Rose B.S.
        • Winer E.P.
        • Mamon H.J.
        Perils of the pathologic complete response.
        J Clin Oncol. 2016; 34: 3959-3962
        • Gianni L.
        • Pienkowski T.
        • Im Y.-H.
        • et al.
        Efficacy and safety of neoadjuvant pertuzumab and trastuzumab in women with locally advanced, inflammatory, or early HER2-positive breast cancer (NeoSphere): a randomised multicentre, open-label, phase 2 trial.
        Lancet Oncol. 2012; 13: 25-32
        • Verma S.
        • Miles D.
        • Gianni L.
        • et al.
        Trastuzumab emtansine for HER2-positive advanced breast cancer.
        N Engl J Med. 2012; 367: 1783-1791
        • Welslau M.
        • Dieras V.
        • Sohn J.H.
        • et al.
        Patient-reported outcomes from EMILIA, a randomized phase 3 study of trastuzumab emtansine (T-DM1) versus capecitabine and lapatinib in human epidermal growth factor receptor 2-positive locally advanced or metastatic breast cancer.
        Cancer. 2014; 120: 642-651
        • Swain S.M.
        • Kim S.-B.
        • Cortés J.
        • et al.
        Pertuzumab, trastuzumab, and docetaxel for HER2-positive metastatic breast cancer (CLEOPATRA study): overall survival results from a randomised, double-blind, placebo-controlled, phase 3 study.
        Lancet Oncol. 2013; 14: 461-471
        • Baselga J.
        • Cortés J.
        • Kim S.-B.
        • et al.
        Pertuzumab plus trastuzumab plus docetaxel for metastatic breast cancer.
        N Engl J Med. 2012; 366: 109-119
        • Swain S.M.
        • Baselga J.
        • Kim S.-B.
        • et al.
        Pertuzumab, trastuzumab, and docetaxel in HER2-positive metastatic breast cancer.
        N Engl J Med. 2015; 372: 724-734
        • Cortes J.
        • Baselga J.
        • Im Y.H.
        • et al.
        Health-related quality-of-life assessment in CLEOPATRA, a phase III study combining pertuzumab with trastuzumab and docetaxel in metastatic breast cancer.
        Ann Oncol. 2013; 24: 2630-2635
        • Takada M.
        • Fukuoka M.
        • Kawahara M.
        • et al.
        Phase III study of concurrent versus sequential thoracic radiotherapy in combination with cisplatin and etoposide for limited-stage small-cell lung cancer: results of the Japan Clinical Oncology Group Study 9104.
        J Clin Oncol. 2002; 20: 3054-3060
        • Robert C.
        • Thomas L.
        • Bondarenko I.
        • et al.
        Ipilimumab plus dacarbazine for previously untreated metastatic melanoma.
        N Engl J Med. 2011; 364: 2517-2526
        • Maio M.
        • Grob J.J.
        • Aamdal S.
        • et al.
        Five-year survival rates for treatment-naive patients with advanced melanoma who received ipilimumab plus dacarbazine in a phase III trial.
        J Clin Oncol. 2015; 33: 1191-1196
        • Herbst R.S.
        • Baas P.
        • Kim D.W.
        • et al.
        Pembrolizumab versus docetaxel for previously treated, PD-L1-positive, advanced non-small-cell lung cancer (KEYNOTE-010): a randomised controlled trial.
        Lancet. 2016; 387: 1540-1550
        • Garon E.B.
        • Ciuleanu T.E.
        • Arrieta O.
        • et al.
        Ramucirumab plus docetaxel versus placebo plus docetaxel for second-line treatment of stage IV non-small-cell lung cancer after disease progression on platinum-based therapy (REVEL): a multicentre, double-blind, randomised phase 3 trial.
        Lancet. 2014; 384: 665-673
        • Tabernero J.
        • Yoshino T.
        • Cohn A.L.
        • et al.
        Ramucirumab versus placebo in combination with second-line FOLFIRI in patients with metastatic colorectal carcinoma that progressed during or after first-line therapy with bevacizumab, oxaliplatin, and a fluoropyrimidine (RAISE): a randomised, double-blind, multicentre, phase 3 study.
        Lancet Oncol. 2015; 16: 499-508
        • Von Hoff D.D.
        • Ervin T.
        • Arena F.P.
        • et al.
        Increased survival in pancreatic cancer with nab-paclitaxel plus gemcitabine.
        N Engl J Med. 2013; 369: 1691-1703
        • Borghaei H.
        • Paz-Ares L.
        • Horn L.
        • et al.
        Nivolumab versus docetaxel in advanced nonsquamous non-small-cell lung cancer.
        N Engl J Med. 2015; 373: 1627-1639
        • Borghaei H.
        • Brahmer J.R.
        • Horn L.
        • et al.
        Nivolumab vs docetaxel in patients with advanced NSCLC: CheckMate 017/057 2-y update and exploratory cytokine profile analyses.
        J Clin Oncol. 2016; 34 (suppl abstr 9025)
        • Larkin J.
        • Chiarion-Sileni V.
        • Gonzalez R.
        • et al.
        Combined nivolumab and ipilimumab or monotherapy in untreated melanoma.
        N Engl J Med. 2015; 373: 23-34
        • Ribas A.
        • Puzanov I.
        • Dummer R.
        • et al.
        Pembrolizumab versus investigator-choice chemotherapy for ipilimumab-refractory melanoma (KEYNOTE-002): a randomised, controlled, phase 2 trial.
        Lancet Oncol. 2015; 16: 908-918
        • Ledermann J.
        • Harter P.
        • Gourley C.
        • et al.
        Olaparib maintenance therapy in platinum-sensitive relapsed ovarian cancer.
        N Engl J Med. 2012; 366: 1382-1392
        • Ledermann J.
        • Harter P.
        • Gourley C.
        • et al.
        Olaparib maintenance therapy in patients with platinum-sensitive relapsed serous ovarian cancer: a preplanned retrospective analysis of outcomes by BRCA status in a randomised phase 2 trial.
        Lancet Oncol. 2014; 15: 852-861
        • Ledermann J.A.
        • Harter P.
        • Gourley C.
        • et al.
        Quality of life during olaparib maintenance therapy in platinum-sensitive relapsed serous ovarian cancer.
        Br J Cancer. 2016; 115: 1313-1320
        • Wild C.
        • Grössmann N.
        • Bucsics A.
        • et al.
        Utilisation of the ESMO-MCBS in practice of HTA.
        Ann Oncol. 2016; 27: 2134-2136
        • Hammerman A.
        • Greenberg-Dotan S.
        • Feldhamer I.
        • et al.
        The Esmo magnitude of clinical benefit scale for novel cancer medicines – correspondence with prioritization decisions in updating the Israeli National List of Health Services.
        Value Health. 2015; 18: A336
        • European Society for Medical Oncology
        ESMO Clinical Practice Guidelines News.
        in: ESMO, Lugano2016
        • Kiesewetter B.
        • Raderer M.
        • Steger G.G.
        • et al.
        The European Society for Medical Oncology Magnitude of Clinical Benefit Scale in daily practice: a single institution, real-life experience at the Medical University of Vienna.
        ESMO Open. 2016; 1: e000066
        • Del Paggio J.C.
        • Azariah B.
        • Sullivan R.
        • et al.
        Do contemporary randomized controlled trials meet ESMO thresholds for meaningful clinical benefit?.
        Ann Oncol. 2017; 28: 157-162
        • Gianni L.
        • Pienkowski T.
        • Im Y.H.
        • et al.
        5-year analysis of neoadjuvant pertuzumab and trastuzumab in patients with locally advanced, inflammatory, or early-stage HER2-positive breast cancer (NeoSphere): a multicentre, open-label, phase 2 randomised trial.
        Lancet Oncol. 2016; 17: 791-800
        • Geisse J.
        • Caro I.
        • Lindholm J.
        • et al.
        Imiquimod 5% cream for the treatment of superficial basal cell carcinoma: results from two phase III, randomized, vehicle-controlled studies.
        J Am Acad Dermatol. 2004; 50: 722-733
        • Stark D.
        • Nankivell M.
        • Pujade-Lauraine E.
        • et al.
        Standard chemotherapy with or without bevacizumab in advanced ovarian cancer: quality-of-life outcomes from the International Collaboration on Ovarian Neoplasms (ICON7) phase 3 randomised trial.
        Lancet Oncol. 2013; 14: 236-243
        • Perren T.J.
        • Swart A.M.
        • Pfisterer J.
        • et al.
        A phase 3 trial of bevacizumab in ovarian cancer.
        N Engl J Med. 2011; 365: 2484-2496
        • Janne P.A.
        • Yang J.C.
        • Kim D.W.
        • et al.
        AZD9291 in EGFR inhibitor-resistant non-small-cell lung cancer.
        N Engl J Med. 2015; 372: 1689-1699
        • Shaw A.T.
        • Kim D.W.
        • Mehra R.
        • et al.
        Ceritinib in ALK-rearranged non-small-cell lung cancer.
        N Engl J Med. 2014; 370: 1189-1197
        • Shaw A.T.
        • Ou S.H.
        • Bang Y.J.
        • et al.
        Crizotinib in ROS1-rearranged non-small-cell lung cancer.
        N Engl J Med. 2014; 371: 1963-1971
        • Kwak E.L.
        • Bang Y.J.
        • Camidge D.R.
        • et al.
        Anaplastic lymphoma kinase inhibition in non-small-cell lung cancer.
        N Engl J Med. 2010; 363: 1693-1703
        • Camidge D.R.
        • Bang Y.J.
        • Kwak E.L.
        • et al.
        Activity and safety of crizotinib in patients with ALK-positive non-small-cell lung cancer: updated results from a phase 1 study.
        Lancet Oncol. 2012; 13: 1011-1019
        • Ansell S.M.
        • Lesokhin A.M.
        • Borrello I.
        • et al.
        PD-1 blockade with nivolumab in relapsed or refractory Hodgkin’s lymphoma.
        N Engl J Med. 2015; 372: 311-319
        • Kaufman B.
        • Shapira-Frommer R.
        • Schmutzler R.K.
        • et al.
        Olaparib monotherapy in patients with advanced cancer and a germline BRCA1/2 mutation.
        J Clin Oncol. 2015; 33: 244-250
        • Powles T.
        • Eder J.P.
        • Fine G.D.
        • et al.
        MPDL3280A (anti-PD-L1) treatment leads to clinical activity in metastatic bladder cancer.
        Nature. 2014; 515: 558-562
        • Rosenberg J.E.
        • Hoffman-Censits J.
        • Powles T.
        • et al.
        Atezolizumab in patients with locally advanced and metastatic urothelial carcinoma who have progressed following treatment with platinum-based chemotherapy: a single-arm, multicentre, phase 2 trial.
        Lancet. 2016; 387: 1909-1920
        • Krueger D.A.
        • Care M.M.
        • Holland K.
        • et al.
        Everolimus for subependymal giant-cell astrocytomas in tuberous sclerosis.
        N Engl J Med. 2010; 363: 1801-1811
        • Seiwert T.Y.
        • Burtness B.
        • Mehra R.
        • et al.
        Safety and clinical activity of pembrolizumab for treatment of recurrent or metastatic squamous cell carcinoma of the head and neck (KEYNOTE-012): an open-label, multicentre, phase 1b trial.
        Lancet Oncol. 2016; 17: 956-965
        • Le D.T.
        • Uram J.N.
        • Wang H.
        • et al.
        PD-1 blockade in tumors with mismatch-repair deficiency.
        N Engl J Med. 2015; 372: 2509-2520