Sir Rory Collins, FRS, FMedSci

Sir Rory Collins: Thank you very much for the invitation to speak, and the opportunity to try to make the case for the need really to look at the regulatory burden that is preventing the reliable evaluation of treatments through the obstacles that have been created to better the design and conduct of randomized, controlled trials.

To kind of get back in time, if you like, to the fifties or the end of the forties, when the first randomized controlled trial was done, in order to have an unbiased comparison between people who were randomly assigned either to control or to treatment, and that approach of randomization with its introduction into clinical medicine and evaluation of clinical medicine was a medical research Council trial in the UK of streptomycin in primary tuberculosis. Austin Bradford Hill, who advised the MRC on this trial, managed to introduce this by making the case that streptomycin was limited in supply at that time, and so he proposed that it be assigned to patients, you know essentially a lottery which he proposed would be a fair way of assigning it, but was also a way of introducing the concept of assigning at random, and so patients were assigned at random to either getting the standard care or just bed rest or bed rest and streptomycin, and, as you can see here, those who received streptomycin did very much better than those that got bed rest alone, and in some respects I think this moved things forward by introducing randomization but in other aspects may have set it back because it seemed to be a way in which to generate very clear results in relatively small studies. And unfortunately, most of the treatments that we have are not nearly as effective as a streptomycin, and so, as well as randomization, it was important to recognize the need to avoid not just systematic error bias through comparing like with like with randomization, but to do so on large enough numbers of people to reduce random error as well, and to avoid false negative results whereby the differences between those assigned at random to treatment and those assigned to control appear to have similar outcomes, just because the treatment effect is too small to be detected in a small trial, and it was really from the 90s on that we started to see much larger randomized controlled trials so that both false positive and false negatives were increasingly well controlled and avoided, and a trial which I was involved in, in the emergency treatment of heart attack is shown here, conducted in the mid-80s. streptokinase, a clock dissolving therapy had been tested in about 2 dozen trials in the sixties and seventies, but those trials were very small. It had been concluded that it didn't reduce mortality and acute myocardial infarction, but by doing a very large scale trial, the Isis-2 trial in 17000 patients during the emergency, the acute phase of a heart attack, it was shown that Streptokinase dissolving the clot in the coronary artery could reduce mortality, as you can see here by about one quarter, and because the numbers of deaths were so big that reduction from 13% dead with routine care which was essentially to give pain relief, and to monitor the patient could be reduced to 10% with a single injection of streptokinase, a generic drug. That then led on to some more effective clot-dissolving therapies such as tissue plasminogen activator and ultimately onto primary angioplasty as an alternative way of opening the coronary artery. We also assessed the effect of clot preventing therapy with low dose aspirin for a month, and that demonstrated also that it reduced mortality by a quarter and this was the first time that aspirin had been shown to be beneficial in the acute phase of a heart attack. A smaller trial done in the UK had failed to demonstrate this benefit, and the beauty of this particular design of a factorial design where we randomize half the patients to streptokinase and half to control and also half to aspirin and a half to control is that one is able to look at a separate and combined effects, and you can see that mortality by giving both treatments, was reduced from 13% to 8%. And because the numbers were large, as I say, these were highly significant differences, and the success of this trial was by keeping the participation of the doctor's and the other clinical staff in the trial very minimal, so they were able in the emergency setting of heart attack, to randomize large numbers of patients and get these very clear answers which then translated into clinical practice very rapidly.

However, from really actually the middle of the 90s, but particularly from the beginning of the century onwards. What we've seen is the impact of the International Committee on Harmonization. Good Clinical practice guidelines which are now increasingly preventing important public health questions from being answered reliably in randomised, controlled trials. For those of you not familiar with the International Committee, or, as it became, the International Council on Harmonization, this was an organization which was established by the pharmaceutical industry with the regulatory agencies from the 3 major markets of the world. The US, Europe and Japan, and I use the word markets because that was really the background to these guidelines. The AIM of the pharmaceutical industry here was to have a single set of rules that would allow them to get approval, for the evaluation of a new drug that would be consistent across the world, and so would lead to more rapid approval once they completed the trial, and there was nothing more to it than that there was a market, it was about getting to market faster. So, from the point of view of the pharmaceutical industry, it didn't really matter primarily whether the rules were good rules or bad rules, only that there was a single rule. So the objectives stated were to provide a unified standard, to facilitate a mutual acceptance of clinical data by the regulatory authorities. It’s about keeping things simple in getting the drug to market once the trials have been done.

It is asserted, but I would suggest is debatable in these guidelines, which again did not involve the academic community, did not involve both patients or the public and did not involve, as far as I'm aware, any ethical expertise. But it's claimed it is both an international, ethical, and scientific quality standard. The designing, conducting, recording, and reporting of trials. It's also a problem that these guidelines which were developed for mutual acceptance of clinical data, of new drugs, have had a huge mission creep. So they are now considered to be the principles under which pretty much all clinical trial investigations should be conducted. And so, as you can see here the principles established may also be applied to other clinical investigations, is written in the ICH guidelines, and that, indeed, is what has happened so charities, such as the gates foundation and the welcome trust. Government funders, such as the Medical Research Council and the NIH in the US and regulators across the world have adopted these for all clinical trials of drugs.

Now my concern is that although the objectives were perfectly reasonable, the idea of good clinical trial guidelines is to protect the safety rights, and well-being of study participants number one, and also to ensure the reliability and robustness of study results, good intentions don't necessarily suffice to ensure that one gets effective strategies that are both proportionate and efficient.

So what has been in the impact and particularly the adverse impact of the ICH guidelines as adopted into regulation and its impact on related bureaucracy. Well, first of all, I think there's now good evidence it has increased obstacles and delays and costs for trials to such an extent that many researchers and industry are saying, well, let's not do randomized trials. Let's use observational studies to try to evaluate effects of treatment, because then we won't be constrained by these regulatory guidelines. So we're moving away from a reliable way of evaluating treatment to an unreliable way of evaluating treatment in order to avoid the bureaucracy and the obstacles of the guidelines rather than thinking about how do we change the guidelines so that they facilitate the reliable evaluation treatment, and that is not just my view. So in 2001, the European Union issued a Clinical trial directive based on the ICH-GCP guidelines. So across Europe there was a directive that required compliance with the ICH-GCP guidelines. At the time academic researchers argued that that would have adverse consequences, but little notice was taken of that, however, in October 2009 the European Commission, having done a review of the impact of the directive, came to these conclusions. That it had increased the administrative costs of doing trials without added value, it had increased the scientific staff demands due to the inefficiencies of applying these guidelines. The directive had actually be implemented differently in different parts of Europe, to the detriment of safety of the clinical trial participants. And one example of 4 years after the directive was put into place from Northwick Park but I could give you other examples from other parts of Europe, was the disastrous Northwick Park study, conducted for Paraxel, where everything was done compliant with the Ich-gcp guidelines, a review showed that that was the case, but because there was a tick box mentality, if you like, to ensuring that there was compliance with the guidelines. There was a failure to actually think through the key principles of safety for these patients.

The European Commission also concluded that inconsistent application of the directive led to longer delays, that patients were not getting access to new and innovative treatments, and their conclusion was that performing clinical trials have become considerably more difficult and costly. So you would have thought as a consequence of that review, that they would think again but you would be wrong. So what they then did was change the directive into a regulation that then enforced this to an even greater extent across Europe.

What we've also seen is that the ICH-GCP and the way in which it's been implemented has distorted the research agenda and reduced creative collaboration between academia and industry. Pharmaceutical companies have become much more reluctant to work with academia and have academia doing clinical trials, and they have instead taken much greater control over them, and in indeed increasingly used contract research organizations to conduct those trials and if you look back to when ICH-GCP guidelines were released in 1995. There was essentially no CRO (contract research organization) industry at that time.

Since then it has become increasingly burden-full on clinical trial costs such that about 70 billion dollars a year goes into the CRO industry to conduct clinical trials, and as you can see here, it's pretty much an exponential curve with predicted increases so for those who set up CROs , this has been very lucrative, but all of this is money is going out of the research environment rather than being used to evaluate treatments effectively. And of course those costs get passed on in terms of the costs of the drugs that ultimately get to market. And in a paper in the new England journal in 2016, it was estimated on average, it cost about 2.6 billion dollars to develop a new drug.

When one looks at the kind of standard CRO run industry trial now done in, say, cardiovascular disease and I'm taking an example here which I will identify as an example of PCSK9 antibody therapy. So 30,000 patients, medium duration, 2 years. Too short, really to show the full benefits of a PCSK9 inhibitor, which lowers cholesterol because the benefits take longer than that to really emerge. It involved 50 countries and 1200 sites. Now this is people with existing coronary artery disease, a common condition. You have to ask why so many sites? Very small numbers of patients at each of those sites. So a lot of monitoring at large numbers of sites that are not doing very. They're not getting used to the protocol by recruiting large numbers of people, and the total cost of the trial, 2 billion dollars for a single trial in cardiovascular disease. So a cost per patient at 35000$, roughly. In parallel to this, the trial we're running of a PCSK9 Inhibitor , 15,000 people, 5 years. So you can see the fuller effects emerging just 2 countries, at 150 sites, much larger numbers of patients, each site so monitoring fewer sites, they get much more experience with doing the trial, so they get better at doing it, and far lower cost her patient and this is still being compliant with those ICH-GCP guidelines. If one streamlined further, going beyond what the guidelines would allow our view is that you could reduce the cost very much more. That means that the cost of these large-scale trials would be lower, more innovative treatments could be brought into large-scale clinical trials. It's now becoming very costly to go to the late stage trial. So fewer agents are being tested in late stage trial so we're losing opportunity there. And, of course, trials of older treatments would also be much less expensive and so we would get reliable evidence about care that is going on at the moment, for which we don't have good evidence, and the impact of this, I think, has been really quite chilling. We're seeing fewer new treatments developed by industry, fewer academic trials being done on current therapies and in an analysis done by Clive Meanwell at the Medicines Company, it's really disturbing.

So he looked at the top 10 drugs in the US in 2000 and 2015 and in terms of revenue, the revenue had increased for those top 10 drugs,  2 and a half fold. So that's good. A lot more money, if you like, for the companies .But from a public health perspective, there had been a 7 and a half fold decrease in the numbers of people who were being treated by those top  10 drugs. So there was a move away from treatments for common conditions, towards treatment for rarer conditions where the cost of the treatments were very high, but one was talking about the treatment of smaller numbers of people. So it's not that these newer treatments are not fantastic, they're very effective, but they're much more niche than the treatments for common conditions, and the reason why there was that move away from the development of treatments for common conditions is because they need larger trials to become more moderate effects, and those treatments are now not being developed to the extent that they were. And the undue focus on complying with these rules rather than on innovation, In fact the guidelines themselves prevent innovation. The way in which the regulatory authority auditors come round is they approach these trials as if they were being done in the 90s, where everything's on paper, where one can't link into health record systems to pull up health outcomes, etc. And so we're really applying rules of the nineties to the situation in the middle of the 2020s. The European Society of Cardiology in 2013, and more recently, in a 2022 paper from the ESC, The American Heart Association, American College Cardiology, and the World Heart Federation just reinforce this point, that these regulatory barriers are making it very much more difficult to conduct clinical trials and generate reliable evidence.

In the pandemic there were opportunities to apply the approach that was used in that trial of the emergency treatment of heart attack, the ISIS-2 trial that I mentioned, where again we ran this trial in the UK with very simple eligibility criteria, as with that trial of the emergency human heart attack which is patients hospitalized with infection, looked at major health outcomes such as death and other serious adverse events and a very simple follow-up with a one-page format discharge plus linkage into the health record systems that are available in the UK. It was possible to set up the trial very rapidly, and to recruit large numbers of people. 10,000 in the first 100 days of the pandemic starting in March 2020  across hundreds of hospitals in the UK. And this trial was able to demonstrate not only that treatments didn't work, it demonstrated, for example, hydroxychloroquine was ineffective, but was able to identify that loaded steroids in patients who were in intensive care could have their risk of death reduced by about a third, and this low cost treatment, costing about 5$, was enrolled out across the world for such patients. But the regulatory authority did not help. So in the middle of a pandemic while trying to evaluate and identify new treatments, to save lives. The UK regulator spent 6000 person hours of our staff time reviewing just one of the trial treatments in this trial. The auditors who came and spent time doing this had little or no expertise in doing trials or in statistics. Freedom of Information Act demonstrating that was true. Their focus was on the paperwork, and the compliance with various rules as they interpreted them, but not on the quality of the trial, and after 6000 person hours yielded the 119 page report with little in the way of material issues with the way in which the trial was conducted or the reliability of the data. So we have a problem of regulation that creates obstacles and then people who don't understand what matters in the trial applying those regulations in an inappropriate way.

So I guess somewhat inadvertently, presciently, in February of 2020, before we were aware, a pandemic was coming we had published this paper in the New England Journal, arguing for the need to streamline the approach to clinical trials and the regulations that were preventing innovation in the way in which clinical trials were done, and much of what I will say in the remainder of this talk is covered in that is covered in that.

https://www.nejm.org/doi/10.1056/NEJMsb1901642?url_ver=Z39.88-2003&rfr_id=ori:rid:crossref.org&rfr_dat=cr_pub%20%200pubmed

To start off with there's a real need to develop appropriate trial guidelines. That is guidelines that are not designed to get drugs to market. But actually guidelines that are based on the scientific principles that underlie randomized comparisons. They need to be guidelines that are developed by everybody who has knowledge about what's important. Not just the pharmaceutical industry, working with regulators to say what's important is we get our drug to market fast, but with public, with patients, with academics, with ethicists, so that we have appropriate trial guidelines that are appropriate in terms of the scientific principles, and appropriate for the current times when we have electronic systems that we didn't have in 1995 and also guidelines that can be future proofed, to be able to take advantage of innovation that occurs in the future.

So some years ago, we persuaded the welcome Trust and the Gates Foundation to fund the development of just such guidelines with a wide consultation across the world in high and low income countries. Researchers, regulators, patients, and the public, pharmaceutical companies and CROs. That what's called the Good Clinic Trials Collaborative and that continues to be funded by the welcome trust and the bill and Melinda Gates Foundation. Those draft guidelines have been put out consultation and really the key elements of them are good science and ethics, that they are clear and concise, that they are inclusively developed, and that they are progressive and durable, by which I mean that they can adapt to opportunities that occur in the future. So I think if you were interested and you wish to comment on these, please do go to the website.

https://www.goodtrials.org/

We need to think about ways in which to enhance recruitment strategies and there are fantastic opportunities now, because there's a lot of information in electronic health records, systems that can help us to identify potentially eligible participants for trials in order to be able to then go to them directly and invite them to take part in research, that that we need to be thinking about how we are much more inclusive in terms of the patients that go into trials to avoid excluding patients with certain criteria. Elderly patients, women, people from diverse backgrounds from around the world. So being broader and more generalized or avoiding unnecessary restrictive inclusion criteria. Think about ways in which we can use the technology that is available to us to improve trial quality. So not only design the studies better, but also think about how we can use electronic systems to manage those trials better. Not to pick up problems after the event, but to actually build quality into the way in which the trials are managed. To use centralized approaches to monitoring the trial to help improve both the patient safety, but also the performance of the trial. To pick up issues that are emerging in real time. And again we can use health record systems and other remote methods to improve the follow-up of patients so not only during the clinical trial, but in the long term if with the participants consent, we can link into their health record systems, then we can see whether benefits or adverse events effects emerged with a longer follow-up. We can also think about ways in which we can enhance the assessment of participants, not just by asking them questions, but perhaps by using remote methods to assess differences in quality of life or various aspects of health directly with the patients.

 I think we're getting some traction so the G7 emphasized the points I've described here about many therapeutic trials in the adequate in size. The FDA pointed out that most of the randomized trials, some about 95% of the randomized trials done during the pandemic had no chance of detecting plausible effects. That's a terrible waste of resources, a terrible waste of the patients who took part in those trials. We need to ensure that we do trials that do produce reliable answers. We need to make sure that our guidance supports that, and at present time does not.

So maybe with the support of people like you, maybe with the support and the understanding of regulators and governments and the wider community. From now onwards we could actually improve the trials that are done, facilitate the reliable evaluation of treatments and I think the key to that will be to establish and implement much more appropriate clinical trial guidelines and get rid of the plague of the ICH and its guidance.

Leeza Osipenko: Very interesting perspective, very sobering data.

Jack Scannell: I've got 2 questions, both partly related. On incentives. So arguably big drug companies like high barriers to entry because it reduces competition so it's not entirely amazing. I spent a lot of my career in drug and biotech investment. It's also the case that drug companies don't like spending 2 billion dollars on a trial that they should do for 150  million. So in a sense, are they pushing for cheap trials or actually, is the barriers to entry and competition caused by the status quo, a kind of equilibrating factor for them. The second question is around underpowered trials. In fact, before this talk, I was just looking at an article on exactly the same point and the question is how did these get approved? Someone must write a check for someone to then go and run an underpowered trial. And is it that the applicant forms for the money don't have power calculations, or the people writing the grants don't pair so I'm just intrigued to know how so many underpaid trials get done.

Sir Rory Collins: On your incentive issue I think the thing with the pharmaceutical industry is that even a single company is not going to be a single entity. So I think one of the things we've seen with ICH-GCP as you know, I've been doing clinical trials for the last 40 years, and the regulatory groups within pharmaceutical companies have become increasingly large and powerful and no one likes giving up power. So the CRO industry is very powerful and can lobby hard. The regulatory groups within pharmaceutical companies have become very powerful, and so there are certainly disincentives for change. I think the point you make about Big Pharma having a monopoly is an interesting one, because it's certainly the case that a lot of innovative treatments don't come from Big Pharma. They come from small companies. And those small companies can't afford to take those treatments, in many cases, forward to the late stage trials. So it is in the interest of Big pharma to be able to pick and choose and if you like, buy cheap from the people who come up with the inventive concepts and a very interesting example of that is Inclisiran a small interfering RNA that blocks the synthesis of PCSK9. The medicines company that I mentioned licensed it from the company called Alnylam where John Maraganore and his team had developed this small molecule that could block synthesis of PCSK9, half LDL cholesterol with injection every 6 months under the skin. Medicines Company could not afford to do a 1 billion dollar trial. They could not find any pharmaceutical industry that would work with them on it. That drug was going nowhere until we showed them how you could do it at much lower cost, and then the trial went forward, and I gave you the example of that trial at a much lower cost, and then the Medicines Company was bought by Novartis for 9 billion dollars. But there must be tons of examples of that of these small companies with great ideas that just don't move forward so we're really failing to take advantage of a lot of innovation and, as you say, it's in the interest of the big companies that they can then pick and choose like that. However, I think that the argument could be made, that there are plenty of smart ideas out there so if more of them went forward because the trials cost less than everybody could gain, the small companies could gain from their innovation, the large companies could take advantage of more opportunities to develop those treatments and get them to market and have billion dollar a year drugs, and indeed, even the CROs could benefit because there'll be more trials going on so they could be supporting those trials, but doing so much more cost effective way, and the cost of the treatments to patients would be much lower. So I think it could be an everybody wins one, but people don't like change with respect to your second point, you can come up with foul calculations it can make any trial look as if it could get a result. But I think the problem is that that requires putting implausibly large effects in and that's really what we saw. There's a wonderful paper, by Janet Woodcock from the FDA, just showing how few of the trials done in the pandemic plausibly could detect the source of effects that  one should expect so it’s a terrible waste of resource and that goes on all the time.

David Culquhoun: I'd just be going to ask about the power calculation, as to what extent they were done, but they had overestimated the effect size or underestimated the variability.

Sir Rory Collins: Yeah, I think the problem is that when they actually do the power calculation, I think they start with how much it would cost, and then they work out how many people they could get in and again, it's a perverse effect of the, in my view, the guidelines. ICH is an organization that is completely non transparent. You can't even go to the meetings. It is an industry regulatory clothes shop. They're actually going to come out with a revision of the guidelines but very few people have had any opportunity to give input to that. But I think, David, it is a perverse effect that people work at how much funding could I get for this study, whether it's from industry or from academia? From MRC or the NIH, or whatever, and then they work out how many patients they could recruit with that amount of money, and then they can come over with a power calculation that fits that. I mean that's a little bit cynical but not too.

Lawrence: I think looking at the natural tendency for any large group of people is to eventually try to assume power and that power progressively grows over time. But we see here, it appears to be a dichotomy. On one hand, you have overregulation, on the other you have 95% of the trials in the Covid environment being ineffective. Obviously under regulated trials, so we have this kind of dichotomy that exists. And we also see the same problem with over control in a scientific community itself. So we see, for instance, you identify the Dexamethasone trial as an important discovery but the reason that Dexamethasone was considered was that there was a belief based on evidence, randomized control trials, that Dexamethasone would not be effective in a ARDS due to covid because it had been studied in a ARDS previously so here we have a problem with regulatory by the scientific community of the definition of the disease, and therefore failure of evidence based medicine as a function of that. So help me understand how do we deal with the issue? Obviously on one hand, we are looking specifically at over regulation. It's a massive overregulation. On the other hand, we have this Wild West out there, where small groups of people get together and decide a consensus, and then everybody has to follow that consensus. So how do we approach those things?

Sir Rory Collins: I wish I knew how to change the regulatory guidance situation. It feels somewhat quixotic to be going up against ICH because it is such an embedded and powerful group of not just the 3 major market regulators now, but pretty much all the regulatory authorities around the world and the pharmaceutical industry. So it really needs the community by which I mean patients , the public, academia, and I think industry to say enough, because it's not getting better. It's getting worse. And even in a pandemic what we saw was even the FDA would not allow convalescent plasma to be randomized. They said it could be evaluated through all observational studies. So over 100,000 people in the US got convalescent plasma for Covid. It was randomized in the recovery trial and found not to produce benefit but over 100,000 people in the Us got it. That meant that you may not have got other things that could be more effective. The dexamethasone is another interesting example, as you say, before the trial was started, a number of academics wrote to the regulator, saying that it shouldn't be allowed to be evaluated in the recovery trial because you believed that it would not be beneficial. So I think that there's kind of a risk. That because it's hard to do trials because easy to do, observational studies that people shortcut and things are being pushed towards using observational data rather than doing randomized trials. Doing randomized trials that are too small because of the costs when there are plenty of patients that could go into those trials, so I don't know the answer. I can identify the problem. I think that there is an opportunity now because the G7 has identified that this is an issue. I think the US, FDA with the Commissioner like Robert Califf, who understands these problems with people like Janet Woodcock, who understand these problems. There is an opportunity, but it needs people to lobby.

Leeza Osipenko: People who lobby effectively have much better budgets than us.

Silvia Marsoni: I have several things to say, but perhaps the most important one is that really what you are talking about is the opacity of the bureaucracy, and certainly the 2 major culprits in this is one the regulators that are not on top of what they're supposed to be, and most of all, the industry. But there is a third party bad guy, which is the CRO. You said before that they could benefit from another way of doing clinical trial? I think that within CROs the same thing that’s happening in industry is happening. The big fish is eating the smaller fish. I had a year ago approximately a terrible role, we went to the knife with this guy, who is a vice director of IQVIA, one of the biggest CROs, who was going around saying randomized clinical trial is too expensive. We need to do observational trial in the real world population, which is, you know, the last shiny little object, and in reality it's IQVIA who is actually doing and paying through a sister program through the European community, a penetration seminars all over in big cancer centres to propel this idea that you know clinical trial randomization, it's an old thing, we need to do observational trial in the real world. And when you say that you can actually do randomized trials in the real world, and that there are 2 completely different problems, the one of clinical trial selection, they go berserk, and they're really pushing because they're seeing this as a new area of economic gain so what you are saying cynically I am quoting. It's really a question of the economic policy between the big industry and the big CRO. In order to disrupt this, we need to go politics, there is no other way. So of course, we need to lobby. But the point is to find out who to lobby. The G7 is a good idea, but we should lobby charities, these are people that have weight. And the society. But the problem of the Medical Society that they are also somehow in cahoots with the industry, because of what we know So we need to identify new partners to politically dismantle.

Sir Rory Collins: No, I absolutely agree. I mean, that was the reason for going to Gates and the welcome trust and encouraging them to support developmental guidelines, because of course, they don't have any kind of financial gain. No one can accuse them of benefiting financially from supporting the development guidelines. More recently, with the support of the European Society of Cardiology particularly, I mentioned in December 2022, a paper by the Esc, along with the American Heart Association…. They put out a paper jointly in all of their journals, at the same time supporting this need for new regulation. So I think the societies, the charities. You're absolutely right. I think that the Government, because the cost of doing these trials at such high cost and so inefficiently, is it cost to governments, because the drugs that then come through will be only the very costly ones, and the cost to their public because there will be treatments that don't come through because of the cost of doing trials, and I get back to the other example of convalescent plasma. Exactly your argument, your point, real-world evidence. So the FDA in its lack of wisdom, pushed for non-randomized evaluation of convalescent plasma during the pandemic and failed to get the right answer at the expense of a 100,000 people being given that treatment and the diversion of effective action during that time. So you know, these failures need to be called out, and the impact of these failures need to be called out. And we need to get patients and the public to understand what's happening to them as a consequence.

Lydie Meheus: What I've posted is about the revision of the ICH guidelines for the clinical trials. ICH8 If I'm correct. That was presented, I think last April (2022), and that's all about the new clinical trial design that should be really with input from stakeholders that push for patient inputs from stakeholders that they push for patient input and also downstream stakeholders like HTAs, so I would like to hear what you think of that revision. And then I would also ask Leeza to explain the efforts of WHO because you were involved on getting better clinical trials. Could they help on pushing for that with what they have in mind, with new regulations. The third thing is the revision of the pharmaceutical legislation in Europe, where there might be an opportunity to really push. I don't know if you are aware of that, but it has been postponed now already for 3 months so there's a lot of push from the industry. But there it's really a big thing. For the first time in history, the role of EMA is revised, and we are still working on a leaked version but the official version will come out normally now April 26th, and then the question will be who can act upon that to really bring to the table whatever you have been explaining, and I think it all that stuff should be explained to the European Commission or to the European Parliament as if you really want to do something on the lobbying, I think we really now have a window of opportunity but some of you should do it. We are a small organization focusing on cancer, so we are trying to do our best on some aspects of it, but I think we need bigger organization, and, as you rightly say, the most influential ones to come with your story to the European Commission and to the Parliament.

Sir Rory Collins: I think, actually, it's the organizations like yours that can probably have a huge impact, because it's the patient organizations. It's the organizations that focus on particular diseases that can make the case for the harm that is being caused. The WHO have actually formally adopted the good Clinical Trial collaborative guidance, and so WHO is actually supporting the guidance that we developed. Jeremy Farrar was the medical director of the Welcome Trust when we got the funding from welcome to Set of those guidelines, and he has now moved to the WHO as the chief scientist, so I think that there is a real opportunity to work with WHO on this. I don't hold out much hope with ICH, honestly, because the opportunity to input there is very limited. My colleague, Martin Landry, who ran the recovery trial, is one of about half a dozen people who have had any opportunity to look at the guidance. I've talked to him about it and he said made 1000s of comments on the earlier draft and he has no idea which ones will have been adopted, and as a process for developing guidelines that are meant to be relevant to all clinical trials the idea that it is not an open process. I think it’s inappropriate. It ought to be out there and debated properly. Not: you make comments, and they may or may not be taken seriously by people who frankly don't really understand what matters in clinical trials. In my view, I mean I don't think it's malevolence. I think it's ignorance, and the ignorance of what really matters in a randomized control trial. I think one of the areas where one could create improvement is in training the auditors within regulatory authorities because they learn the guidance and the regulation, but they don't get toward what matters in a clinical trial and again we see that consistently across the world. So finally the EU. When the UK was in the EU, I certainly had opportunities to go to the Commission, and I remember going along and someone just saying, well you know you can say that about how the how the directive should be written but let me tell you, once you get to the lawyers in the Commission, everything will change so I don't know how you actually get effective change within the Commission, but I do think that that's an important opportunity. The question is whether the EMA is actually able to push things forward in the right direction.

Lydie Meheus: I think, being a patient representative in the EMA, they just have to listen to what the Commission says. So I think it's important to explain to the Commission what has to happen. So can I share your presentation with someone.

Sir Rory Collins: Absolutely feel free to do so, and I am now European. 3 weeks ago I became an Italian.

Lydie Meheus: Good to know because I think we get requests from political just this week about what we think should change and then something like your presentation could be really helpful to explain, to that level of political journalists also, what has to happen. I think it's important. So if I can link to that.

Sir Rory Collins: and it's it is all summarized in that New England journal 2020 paper as well. The magic of Randomization.

Leeza Osipenko: I think one clear message that also comes from this problem is that there are too many authorities. ICH, WHO, FDA. They're all very powerful, plus the lobbying, and they're not talking to each other, I mean to an extent they are, but it's a big problem in this balance of power as well to try to move things forward.

Sir Rory Collins: I think the WHO can play a major role, and the key regulators are the FDA and the EMA. The FDA at this particular time, for the next couple of years with the people at the top, is a real opportunity.

Susan Bewley:  I was just trying to look at the opposite from the WHO to the patient on the ground, remembering Ian’s wonderful personal view about how he couldn't trust a surgeon who knew what the operation for his knee was. I can't trust you unless you randomize me because of your uncertainty, and I've always felt like that. But I was reflecting about how money does get in the way. It may not be malevolent, but it gets in the way. For example, nobody has ever replicated an incredible study about which way up suppositories go, which make a difference between 87% of fingers up or 3% and 3% of them falling out, and nobody's repeated. So I've written to nice multiple times. They've just relooked it all for me, and they've said, well we're going to ask the MHRA. Well what are they going to say, the product license says this, something says that nobody thinks it's urgent to save 3% of the suppository bill or some other bill. I think we need to be a little bit more revolutionary. I mean, we've got amazing international organizations, we've got amazing charities like sense about science. Okay, it might have to be trivial to say, if you do this exercise with your left leg or that with your haircut, you know, masses of people get online, get randomized and citizen science starts to educate everyone that this is a powerful tool for answering practical questions, and it wouldn't need that much money. But I think the things are so entrenched you almost have to write in the basement. Getting some rats eating through the cables at the bottom, but I don't know how to do that, and so I just wonder if we can be a bit more innovative and creative and use some of the other powerful tools, like we have with the Internet to educate people. Lawrence Fox lives in the next street to me, and he was sitting there, and I felt like knocking on the door, and saying: Ivermectin, you’re mad, we've actually got the answer. not except in research in the UK. And so many things could be except in research, except in our RCTs. Never mind, I'm ranting to the people who already agree with me.

Sir Rory Collins: This is not just preventing new drugs. It's preventing us knowing what to do in very practical terms. Do masks work? When should you give a second dose of the vaccine? Should we be giving boosters? I mean the ability with particularly electronic health record systems. The way in which one can randomize all sorts of things where we don't know which way is best. Is that actually being prevented because trials have been mystified by all of this regulatory bureaucracy and the concept is very straightforward, and we've made it very hard to do.

Susan Bewley:  And the last thing just to throw in is that if we could do some citizen science thing, the other thing you mentioned factorial trials. Obviously the 5 by 5 factorial trials of Caesar, and then Coronas, looking at 5 different techniques at the same time in the same operation are fantastically powerful. Again, if we had something creative that lots of people got involved with just on the ground, maybe not masks. That's a bit too controversial, but something incredibly practical, it would be great, I want to volunteer.