Professor Nathan Cherny

Nathan Cherny: Epistemology is the study of how we know what we know, and oncologic knowledge is largely derived from clinical studies and that knowledge assumes integrity and freedom from bias. The challenges are: are these assumptions true? If they are not always true, what are the implications regarding therapeutic knowledge? Bias and research refer to areas in design, implementation, or analysis the deviate results from true findings. These commonly lead to either an overestimation of true benefits, or an underestimation of harm. Research integrity is about the steps we take to minimize bias. Bias is important for several reasons. Typically, it leads to the potential for scientific harm due to compromised generalizability of the data. Direct harm to patients by causing misleading expectations. Societal harms when the data is used for inappropriate resource allocation. Finally, reputational harm to oncologists as a profession due to loss of credibility. These are what I refer to as the as the perils of bias.

The term scientific scepticism encourages alertness to potential for bias in research. This is an important part of professional integrity. Scientific scepticism should not be confused with nihilism, and it requires critical evaluation skills - some of which we're going to discuss today. A wonderful aphorism that I quote all the time to my students, is that not all sceptics are scientists, but all good scientists should be sceptics. In evaluating research data, we have 2 approaches. We have the approach of looking at data through rose-coloured glasses with great enthusiasm of the wonderful breakthroughs, or using an approach of scientific scepticism and asking the questions, are these benefits real? Sources of bias can derive from design issues, and I'll just walk through some of these. When there's no control arm or a substandard control arm. From surrogates because of the limited predictive reliability of surrogates. In non-inferiority studies when you've got a high non-inferiority threshold that can give a false impression of non-inferiority which is not true. There are multiple issues of crossover which can impact on post progression treatment and overall survival. Early stopping rules can influence interpretation of survival benefit.

In terms of study implementation issues - issues not intrinsic to the design. Where you have substandard post protocol treatment, this can distort overall survival. We have a common problem in terms of quality of life. Where the quality-of-life data is collected, but there is publication bias and reporting of quality-of-life data, and often very delayed reporting. It can affect analysis - things like informative censoring can substantially impact on the analysis of surrogate outcomes and we're going to talk about conjectural findings from exploratory and unplanned analyses of either the surrogate outcome findings or overall survival findings. This is a map of the multiple different sources of bias that can be introduced into clinical studies. This is all covered in a study in a paper that the ESMO magnitude of clinical benefit working group published in 2001. This article will be made available to you. Most of the information in this talk is derived from that paper. Based on this analysis, one can develop a checklist of design issues, implementation issues, and analysis issues to check forward when reading papers. What we're going to do is we're going to walk through some of the common issues that we find are commonly causing bias. The issues we're going to talk about are going to be related to conclusions based on surrogate endpoints and we're going to talk about predictive reliability, single arm surrogate studies, and the problem of informative censoring, and how this impact surrogate endpoints. We're going to talk about factors that can distort overall survival data, particularly this issue of lack of appropriate crossover and substandard progression treatments. Then I'm going to talk finally about some of the analytic issues with conclusions based on conjectural rather than confirmatory findings.

Let’s slip into surrogate endpoints. The first key message is that all surrogate endpoints have limited predictive reliability for true clinical benefit. The validity of a surrogate depends on its predictable reliability of true clinical benefit. Does it predict for either living longer or living better? The limitation of surrogate endpoints is that they have this variable predictive reliability. No surrogate has absolute surrogacy for true clinical benefit across all diseases and treatments, and the surrogates themselves may or may not directly benefit the patient. There is a hierarchy of surrogates. At the top of the high of the hierarchy is disease free survival; this is used in adjuvant studies and studies with curative intent. This has moderate predictive reliability for overall survival, and it is claimed by patient advocacy groups that the delay in recurrence, even if it's not predictive of overall survival, is a benefit for patients because this is time without treatment or disease. Next down the list is progression-free survival (PFS). Progression-free survival is applied in a non-curative setting. The predictive reliability is intermediate and very variable. PFS is not a reliable predictor of either improved quality of life or even delayed deterioration in quality of life. The PFS itself, just the fact that X-ray images change earlier or later, does not in itself provide direct patient benefit. At the bottom of the list is response rate and in a cure setting - pathological complete response rate. These are weak indicators of overall survival. Response rate is a very weak predictor of quality of life, and so it has no intrinsic benefit.

We have a study where the primary endpoint is a surrogate outcome. This is the importance of follow-up for overall survival data, and, particularly in the non-curative setting, has quality of life being evaluated as a secondary outpoint? Because these can give some degree of verification of the surrogacy when possible, and sometimes when they're negative, they give you the message that the surrogacy has been disproven. The quality of life becomes a very important outcome in surrogacy trials. It can verify surrogacy for true clinical benefit by demonstrating either an improvement or delayed deterioration in quality of life. The issue or issues arise, however, related to the quality-of-life data. ESMO has recently published standards for evaluating quality of life data. Essentially, we will only credit quality of life data when it has been a secondary outcome, not an exploratory outcome, and has been evaluated with a validated scale. In the methods we want to see a predefined hypothesis and a predefined, clinically meaningful threshold. You want to see there's a high compliance, and ultimately in the results you want to see that there is a statistically significant and clinically meaningful significant difference in global quality of life, not in just sub parameters. This is a checklist which we've developed, which is available on the ESMO website.

The second issue in surrogacy relates to single armed studies, using an outcome of overall response rate and duration of response. What you can see is that, looking at the evidence presented to HTA bodies, we’re seeing a rapid rise in the number of phase 1 and 2 studies being presented. In 2020 it exceeded the number of studies being brought to approval based on randomized studies. Are single armed studies always justified? According to the FDA, they write that they are acceptable for accelerated approval in settings where there's no available therapy, and where major tumour regressions can be presumed to be attributed to the tested drug. They are increasingly used where standard treatments don't exist, where standard treatments are clearly inferior to new treatments, when diseases are rare, and in many situations where it is perceived that randomized studies are not feasible. However, they provide a much less reliable level of evidence for real clinical benefits. So, it leads us with a much greater degree of uncertainty. Often there has been a reasonable alternative therapy that could have been used as a comparator which they have not used. The use is not always based on empiric evidence of the infeasibility of randomized studies. We know that the overall response rates observed in single arm studies are often inflated, because when you take the same drug into a randomized clinical study, on average, the response rate is about 8% lower in the randomized study. This is an illustration of the problem. In yellow, many of these drugs that are withdrawn were proved based on the single armed studies. The data is weak and when they do confirmatory studies, they do not confirm the degree of benefit. In 2021, this is a nice study which are evaluated 31 FDA approvals that were based on single arm studies. They found that an alternative treatment could have been used as a control arm in 28 of the 31 studies. 5 of the drugs were approved despite having inferior efficacy to drugs were previously the standard of care. In more than 85% of them it would have been feasible to conduct a complete randomized study within a reasonable duration of time. Many of the approvals were not for rare conditions but they were for common conditions. This study concluded that this was not good science, it often generated wasteful, accelerated approvals that are subsequently rescinded, and many of these accelerated approvals linger without having the confirmatory studies done.

The last issue that I'm going to talk about related to surrogate endpoints is the issue of informative censoring and the impact of this on surrogate outcomes. I’m very grateful to Ian Tannock because a lot of what I’m going to be saying is based upon his work. Censoring refers to patients who don't complete the study in full and drop out without further measurements. When there are balanced dropouts between the 2 arms of a comparative superiority study, it is assumed that this is not going to impact on the results. This is called an uninformative censoring. However, in contrast, when there were more dropouts in the treatment arm or the experimental arm due to side effects, this can lead to unbalanced dropouts, and this can impact on results. This is what's called informative censoring because patients, particularly in PFS studies, patients who drop out before progressing for reasons other than death are no longer be evaluated. When you have more censoring of patients on the experimental arm than the control arm and you're censoring these poorly performing patients, this can lead to exaggerated progression free survival benefit. This is a study of everolimus along with hormonal therapy in first line, postmenopausal, estrogen receptor positive breast cancer patients. In this study they found a very large difference in PFS. There was a lot of informative censoring. 24% of patients on the everolimus arm dropped out because of toxicity compared to 6% on the control arm. Ultimately, when you look at the overall survival, there is no difference. This has been a particularly prevalent problem in the whole recent generation of studies looking at PARP inhibitors as maintenance therapy for patients in ovarian cancer. PARP inhibitors have lots of side effects and all these studies to date have been looking at PFS as the primary outcome. You can see very unbalanced censoring, with almost 20% of patients on in this study dropping out of this evaluation.

The next issues I want to talk about are the ones that can impact on overall survival data. We’re going to talk about crossover and substandard post progression treatment. Firstly, the crossover issue. In studies that are up streaming studies, they are taking an agent which is already approved as a later line therapy, and you're up streaming it in patients in the control arm when they progress. They should always be receiving the experimental agent because this is part of the subsequent standard of care. Failure to do this is a massive problem which is almost endemic. The scientific concerns are: if there is inadequate crossover of the control arm patients when they progress, that means they are receiving suboptimal post progression therapy, and this can clearly exaggerate the observed overall survival benefits. The second problem is that if the patients haven't been crossed over, it fails to answer the question whether using the same drug is better in the first or the second line. Ethically the failure to incorporate crossover to ensure standard of care causes harm to the control arm patients. This is looking at the study of abiraterone and prednisone in metastatic, castration-sensitive prostate cancer. This is up streaming of abiraterone, which was already approved in castrate- resistant patients. They found this very substantial improvement of overall survival. This was the data on castrate-resistant patients that had previously shown that abiraterone improved overall survival. Going back to the study in question, what we found was, when you look at subsequent treatments in patients in the control arm, there was a lack of crossover of abiraterone in 89% or to even a biological equivalent in 73%. Only 27% of patients received one of these agents in second line therapy. So, they didn't get standard of care, and this may have exaggerated the benefit. We’re still left with a question in patients with metastatic prostate cancer - we don't know if abiraterone needs to be given in a castrate-sensitive or a castrate-resistant setting. This is a whole spectrum of studies, a range of studies looking at the upstreaming of immunotherapy in renal cancer. Immunotherapy has an established role of second line therapy. These were studies which will bring upstreaming immunotherapy to the first line and looking at the subsequent therapies with patients in the control arm. You can see in none of these studies did more than 50% of the patients receive post-progression immunotherapy. All these studies showed improvements in overall survival and still because of the failure to incorporate appropriate crossover, we don't know if immunotherapy needs to be given upfront, or if patients would not receive the same degree of benefit if they received the immunotherapy as a second line therapy.

In contrast to appropriate crossover, when you're up streaming a drug which previously has an established proven role, in a novel agent which has been up streamed, it has no role. It has no established role in post progression therapy, and so for control arm patients when they progress, they do not need to be receiving the experimental agent. If you give the experimental agent, 1) scientifically, it undermines the ability to determine if overall survival is improved or not, and 2) there are ethical concerns because if a crossover delays initiation of a proven therapy, you're harming patients. Since the new medicines efficacy is unknown, there is no ethical mandate for the control arm patients to receive the medicine upon relapse. In this same area of patients who have progressed on a surrogate and the impact of overall survival, what happens here in the post progression phase becomes very important. Most randomized studies involve evaluating a single period of randomization between a novel treatment and an active control. Generally, it was first taken looking at surrogates. In studies of first- or second-line therapies almost all patients will receive one or more lines of post-progression therapy, which can also influence overall survival. In fact, in hormone receptor positive breast cancer, it is not uncommon for patients to receive more than 5 subsequent lines of therapy, and I want you to remember that number. I'm going to show you a couple of different scenarios. This is scenario one where you have an experimental treatment which has an approved progression free survival. All patients receive optimal post progression therapy, and the benefit persists. This is what was observed when trastuzumab was first tested in metastatic breast cancer. An alternative scenario is where there is a big benefit in PFS, but after optimal post-progression therapy that benefit disappears or becomes trivial. This was seen when bevacizumab was used in first line metastatic breast cancer and this data subsequently led to the withdrawal by the FDA of the approval for bevacizumab. The problem arises when patients have not got optimal postprogression therapy, because this may be that the suboptimal post progression therapy may have exaggerated the impact of PFS gain on the ultimate overall survival. This underscores the importance of credentialing centers to make sure they have the capacity to provide the standard of care post-progression therapy, and documenting post progression subsequent therapies until death. In fact, in the IHC guidelines, they say very clearly that efforts should be made to collect all data pertinent to relevant outcomes, including the occurrence and timing of inter-current events. Clinical trials are less generalizable if the sponsor tries to avoid or minimize these issues and post-progression treatments constitute an intercurrent event that is pertinent to overall survival. Let’s look at a scary example of this problem.

Now let's look at a great example. This is Mona Lisa 7, a study looking at the role of the addition of ribociclib to endocrine therapy in pre-menopausal patients. These are young pre-menopausal women, first line therapy, with or without ribociclib. This study showed a substantial improvement in progression-free survival and subsequently showed in follow up an improvement in overall survival. Wherein lies the problem. This is the number of patients who received any subsequent treatment. In the placebo arm, 27% of the patients received no further subsequent treatment. It is plausible that the failure to provide subsequent standard therapy to this 25% of patients who progressed on the control arm may have exaggerated the overall survival gain for ribociclib compared to placebo. This becomes even more intriguing when you look at the forest plot. Looking at the forest plot, this is a drug that had demonstrated no significant benefit in either North America, Europe, or Australia. All the benefit that was observed was derived from this agent cohort. It begs the question, did these Asian patients receive sub-optimal post-progression treatment? It is plausible that the overall survival gain may have been much smaller if patients received the standard treatment. I contacted the principal investigator to raise this issue, and he tried to get the data from Novartis and Novartis would not provide the data on the breakdown of subsequent therapies either to the principal investigator or to me. This is also a terrible issue of lack of data transparency.

The last issue that I want to address is conclusions that are based on conjectural findings. I want to make this distinction between confirmatory and conjectural findings. Confirmatory findings are derived from the primary or secondary analysis of the study, and they may include pre-specified subgroup analyses where there is appropriate adjustment for multiplicity testing and alpha splitting. One also needs to have a justified statistical plan, and ultimately a statistically significant positive outcome. A conjecture is an unproven proposition suspected to be true based on preliminary supporting evidence. In clinical studies, conjectural findings relate to the evaluation of efficacy based on incomplete or suboptimal data and these include findings from either exploratory endpoints or subgroup analyses that were outside of the statistical plan. Just to tabulate this distinction. For analysis - confirmatory outcomes are primary or secondary endpoints, and conjectural outcomes are post hoc or exploratory endpoints. In the confirmatory outcomes the subgroups are pre-specified, and the statistical plan is pre-specified. In conjectural outcomes these are post hoc analyses and exploratory analyses that didn’t have a pre-specified statistical plan. There’s often no adjustment for multiplicity. So conjectural findings have a higher risk of false positive or false negative findings. Therefore, it is important to make this distinction. Now, when is it common to see these? It is common to see post hoc conjectural analysis either in situations where the primary analysis showed a very small benefit, and the post hoc interest is to identify subgroups who are more or less likely to derive benefit which is a very reasonable question. The other common situation is when the primary analysis shows no significant benefit, and there's a post-hoc interest in identifying was there a subgroup with a patient where the treatment may have been effective? Different organizations have different guidelines. The ICH warns that the findings of post hoc subgroup analysis should be interpreted cautiously. The FDA, interestingly, has a draft guideline which has not been ratified and expresses concern that investigators incentives can influence the choice of analysis to identify one or more positive findings. The EMA has a structured approach saying that they will consider evidence based on conjecture where the subgroup of interest is well defined and clinically relevant, when it's biologically plausible, when there are substantially different results, and where there's been replication of similar subgroup findings from other relevant studies.

Here is a great example. This is one where the primary analysis showed a very small benefit. This is the addition of pertuzumab to trastuzumab in HER2-overexpressed breast cancer in an adjuvant setting. This was the primary analysis of the invasive disease-free survival rate, and you can almost get a laser pointer between the 2 curves. In the protocol they write, “the exploratory analysis will be performed for IDFS to ascertain whether the magnitude of the effectiveness of the addition of pertuzumab might differ according to patient sub-populations.” They looked at 12 different subgroups as post-hoc analyses. This is their forest plot, and they highlighted this issue of nodal status. If you look here, the p-value for the interaction is not significant, but then they still mapped out this curve and said, “Ha! We have found an improvement in disease free survival of about 2.5% in patients who are not positive.” This became the basis for the FDA and EMA approval. The concern is that this is 1 of 12 exploratory subgroups, the interaction was not significant, and there was no adjustment for multiplicity. Many countries are using this, and it's reimbursed, and I believe Canada looked at this more sceptically, and did not approve it. This is an example where the primary analysis showed no benefit. This is atezolizumab added to nab-paclitaxel in advanced triple negative breast cancer as a first-line treatment for metastatic disease. This had hierarchical testing for overall survival, first in the attention to the tree population, and only then in the pdl1 positive population. In the study, looking at progression free survival, there was a small benefit which became greater in the pdl1 positive subgroup. The overall survival was not significant and yet they still look at the overall survival data and got an accelerated approval based on that data which again was subsequently rescinded.

What I have tried to highlight here is the difference between the rose-coloured glasses approaches just looking at the data and regurgitating it uncritically and using a structured approach to critically evaluate the data to look for evidence of bias. Coming back to the question I started with about the epistemology of oncology. I asked the questions, are the assumptions of integrity for freedom of bias true? The answer is that in many cases they're probably not, and in terms of the implications for therapeutic knowledge, we need more humility and less hubris and integrity demand honest disclosure and a deeper evaluation and routine evaluation of these potential sources for bias. When it comes to the role of HTA bodies, all these issues that I've highlighted make it very difficult for HTA bodies to evaluate if there's a real added benefit to justify the expenditure of limited healthcare resources, and this then becomes not only a clinical challenge and a scientific challenge, but also a public policy challenge. Just to conclude again with this wonderful aphorism, not all sceptics are scientists, but all good scientists should be sceptics. What I wanted to do today is to review some of the key tools and the key things to look for in reading clinical research with an appropriately critical eye and a sensitivity for bias.

Ian Tannock: We've just been writing a couple of commentaries about recent adjuvant trials, which are making just huge amounts of money for companies. So, there's the adjuvant pembralizumab trial for renal cancer keynote 546 and the MONARCH E trial for abemaciclib and both trials are just fatally flawed but the companies are managing to push hugely expensive drugs into earlier stages of disease. 2 years of adjuvant abemaciclib in the United States costs more than $300,000 and I mean you pointed out the ways they do this. There’s informative centering which they don't even recognize or talk about. One of the ways that the companies do as you pointed out, is they do these global studies. These are huge studies, and they get people from all over the world to do them. But of course, as you point out, the people in these middle income or lower income countries can't afford the drugs. In MONARCH E which we're just working on a paper now, with your colleague and others, basically the biggest accrue to MONARCH E were in Brazil, Mexico, and China, and you just know the women in those countries are not going to be able to purchase the abemaciclib or another like drug. The trials are purposely, in my opinion, bias from the very beginning by the companies.

Nathan Cherny: This is the issue of credentialing of centers. In fact, it is a consort requirement that the centers should be accredited to be able to provide the standard of care. I agree, this is a global strategy. On the issue of informative censoring, it is often different in DFS studies from progression free survival studies and in fact we had a journal club this morning exactly on this study and the DFS was intention to treat so it was not impacted by informative censoring. But we've seen in many situations this study showed at 60 months, a 5-year improvement in invasive disease-free survival. That does not always translate into improved overall survival and time will tell. In the meantime, there's this pressure on countries and on political systems, and healthcare systems, to pay this very extravagant sum. The question is why 3 years and why not 1 year? The answer is that they will make more money on 3 years.

David Colquhoun: I've given something like 70 odd talks to sceptics in the pub, which I enjoyed while I could still get around, but the trouble is the term sceptic has now been adopted by the anti-science brigade. Vaccine deniers, you name whatever sort of scoundrel you want to name, they call themselves sceptics. This is awful, I removed sceptics from my twitter. Do you think it’s getting worse in the age of Trump?

Nathan Cherny: My problem is that people read studies with rose-coloured optimism, bias. People would like to feel that they are more clinically empowered then they are, and I think that there is a lack of scientific scepticism and reading, because people want to feel that they've got good arrows to cancers. I'm the constant party pooper in our journal clubs because I constantly find myself in the role of trying to water down the enthusiasm, and it is not contagious unfortunately. There’s a cultural issue as to whether one is going to look at data optimistically, or look at it critically, and many people just do not have that culture and do not want to adopt the culture and don't see the reason for adopting the culture of being critical readers.

David Colquhoun: I agree entirely, and I think I’m a bit of a party pooper. The other day pholcodine was banned from cough suppressants, so I was busy saying, “don't worry it didn't work anyway” which it doesn't of course. There were pharmacists on the radio saying, “we can recommend something else” which also doesn't work of course.

Leeza Osipenko: Nathan, you presented very important examples of trials, which is wonderful teaching material, wonderful ways to understand when and how things go wrong. If we just take oncology drug approval over the past 10 years, what proportion of these registration studies do you think have been depicted in such a way? So, where can we learn from them, and how much more work do we have left to do?

Nathan Cherny: I was asked by nature clinical reviews to look at all the drugs approved by the FDA over the past 5 years. What I found was that 70% of them were approved based on data just from surrogate data, only 30% of them had overall survival data. Even those that had overall survival data, the problem of inadequate crossover was widely prevalent. These are big issues. When we applied the ESMO MCBS to these 160 approvals, we found that only about a third scored highly, a third scored very low, and a third were somewhere in the middle. Even those high ones may not have been as good as they really are in many cases.

Leeza Osipenko: When you start depicting, there are so many other parameters which might not meet the criteria. There's one question in the chat from Sophie which I will read, “what you think of yesterday's announcement by Jeremy Hunt that the UK and MHRA will speed up the approval of drugs. Based on the presentation, how concerning is this?”

Nathan Cherny: The speed of the approvals reflects efficiency. The quality of the approvals is a more important decision. The policy of the FDA has been to enhance inclusiveness to lower the threshold so if anything has the chance of being beneficial to approve, and to have a very low level of discernment. What will be the policy of the English licensing authority, now that it’s not so much a matter of speed, as what will be the thresholds?

Lydie Meheus: As you know, we are supporting investigative driven trials, non-commercial trials, and from interaction with investigators we learned the harsh reality that if you want to collect the overall survival data, then you need to have your study open for 10 years, a much longer period which makes it much more expensive if you’re in the non-profit or public support trials. Some people are suggesting that maybe if from the start on you could do what they call registry-based RCT, where you could close your study with the surrogate endpoint, but then follow up with E-health records, or the administrative databases. Could that be something acceptable of course if it's pre-specified, or do you have other solutions to lower the cost of collecting overall survival data?

Nathan Cherny: Particularly in the post-progression setting, the issue is the completeness of the data. E-health records, one should be able to get detailed accounting for post-progression therapies, which is going to take work to do it, but the information should be readily available, and the time of death is going to be available. But it still requires a commitment of manpower to derive this data from the E-health records. When it's not done, you can see what happens. If you don't know what patients received from the time that they progressed, you've got no idea if they’ve received standard of care or not or if they’ve received popcorn and chocolate milk. That is a problem. The quality of data collection about subsequent therapies even in sponsored studies, is a real issue.

David Colquhoun: Another problem for the UK government is that they identified as a growth area – nutraceuticals - and there is no area where the evidence is worse than for the supplement industry. It's another aspect of Singapore-on-Thames I’m afraid.

Leeza Osipenko: You mentioned in your presentation and put examples where injustice happens sometimes and we withdraw drugs from the market which went through accelerated approval on single arm trial data, but we know that this withdrawal happens years after. During this time, the company manages to make large sums of money. What do you think we can do here? Can we build legislation around it to have the company liable to return this money to providers, patients, because one thing is just the earnings on the hype that doesn’t work, and another thing is potential harm of patients being on a toxic drug or just being on the drug.

Nathan Cherny: This is the role of a strong HTA body. We've been studying HTA decision making. We've had a project running with London School of Economics, and we know that when a drug is approved on a single arm study with an accelerated approval, firstly HTA bodies deliberate a lot longer. Even though the approval has been accelerated, the HTA process is slowed down, and we also know that in most cases there will also be some form of negotiation on price to give these drugs market access. The price that is ultimately being paid for by governments is generally substantially lower than the US space price, which is a point for negotiation, but it would also be totally reasonable to say that until the price will be much lower if there is a contingent or an accelerated approval, rather than a full approval. I don’t think we’re the one that can negotiate a payback and I’ve never heard of a HTA body trying to negotiate that.

Leeza Osipenko: It was my question because I think a HTA body will tell you it’s not their remit and it’s a political decision, that’s 1 point. If we look at the proportion of earnings, what is coming from the US market for each drug during that period versus the market in Europe?

Nathan Cherny: It's very high. The other thing is that because of the idiosyncrasies of the Medicare Part D laws. Medicare is the largest single purchaser of drugs in the United States and Medicare must provide all FDA approved drugs at a price set by the companies without price negotiation. There is no relationship between the clinical value of the drug and the cost of the drug.

Ian Tannock: It comes down to a matter of the people who are sitting on the guidelines committees and sitting on the FDA and the EMA advisory committees. Unfortunately, we already know that if you're invited to go on one of these committees, the amount that you declare under the Sunshine Act goes up, and so we need to be unbiased, and yet we don't appoint committees who are even financially unbiased. I think the only way for the future is we must do a better job of educating people, like oncologists, who sit on these committees. We need to have committee members who can't make profit out of what their decisions can lead to. The counterargument is that we'd never get anybody to serve on the committee. Well, there are some of us who don't take money from companies, and I just think that must become more and more acceptable. They have tightened up, universities have tightened up, and I think the whole system needs to be tightened up.

Nathan Cherny: The EMA and the FDA, their mandate is to approve drugs that are safe and effective. Their threshold for defining effectiveness is something equivalent or even better than the pre-existing standard, and it doesn't need to be better by much, it can be trivial. Approval does not mean market access, and therefore HTA bodies are so important.

Leeza Osipenko: They also have a very strict remit to what they can and cannot do and there is a big political reason behind. I really think there is a need for another layer here, either give more power to the HTA bodies, or have another layer of legislation, or the government playing the role. There is one comment in the chat, “there will be pricing negotiation in Medicare in the next few years, it has already started.” This is very limited and with a very small number of drugs. Unfortunately, it has nothing to do with newly approved drugs. This is only for drugs who have been on patent for at least 7 to 8 years, as I understand. Everything we're discussing right now - Medicare negotiation will not resolve in any way.

Lydie Meheus: Just want to say that with the revision of the pharmaceutical legislation in Europe, we are now pushing hard for mandatory scientific advice for the design of the clinical trials, for the registration trial, with input from HTA. At least then the design of the trial and the endpoints are approved by HTA. Do you believe that's an approach that could help?

Nathan Cherny: I think in randomized clinical studies, particularly upstream studies, if the Helsinki Committee were forced to look at this issue of crossover and credentialing as part of the approval process for a study and for site approval, I think that would make a substantial step forward. I've been trying to push this with the National Helsinki Committee or IRB Committee looking at randomized studies, and they are open to listening to this. Let’s say, particularly with the extensive drugs that have been brought upstream, that the sponsor would need to ensure that the patients on the control arm will have access to the drug post progression as part of the post progression therapy.

Leeza Osipenko: From experience of scientific advice, we need to remember that this advice is optional for companies to take on board and they need to see what the regulator suggested and mostly the FDA. If the US is the key market and that study is faster and easier to do, not considering what EUnetHTA suggested, chances that they will take those recommendations on board are slim. Let’s not forget that advice is highly confidential, and we will never find out what has been told to the company. That’s another massive lack of transparency and for us to check potential compliance with good practice.