- Prof Carole Longson
- Eric Low
- Dr. Dan Ollendorf
- Dr. Wija Oortwijn
Guest Experts: Dr Rick Vreman and Rob Hemmings
Leeza Osipenko: What should HTAs do as evidence packages are becoming more and more questionable and as the volume of submissions increases?
And What can be done to improve the regulator-HTA interaction to make it better serve patients and the public?
Rob Hemmings: I’m firmly of the view that assessments of quality, safety efficacy and benefit risk that are conducted by the regulators and assessments of cost effectiveness and value in some sense, translated into whether or not to give the medicine the price in the context of the local market, are both fundamentally important decisions. And it's very clear that it's easier to centralize that regulatory assessment of quality, safety or efficacy, than it is to centralize a HTA assessment where the local context is so very important.
I don't actually have a strong feeling as to whether both of those important question are addressed all under one roof or separate roofs, I think Leeza you showed AIFA, you showed the Norwegian agency where different types of regulators, both sit in the same building, and Carol and Leeza I'm sure you remember, until when NICE was set up there was quite a big discussion as to whether everyone should be under one roof or whether NICE should be an entirely distinct body and the decision was made for the latter. But this is something of a double edged sword. It meant that when these two worlds came together there was a real demand to respect each other's jurisdictions.
As regulators, we were sort of invited and required to stick very closely to our legislative criteria and regulators are driven by the legislative basis for their benefit risk decisions.
So on one hand we get clarity of criteria, but on the other hand, it makes it harder to understand the world in which the other person lives in, harder to understand the processes, their requirements, etc. I see a couple of fundamental aspects where the world doesn't sit coherently together, and one is the assessment of value. And again I listened to some of the first webinar and if I was going to provocativly stereotype, I would say HTA only ever wants a product that makes people live longer or improves EQ-5D . And whereas, you know the stereotype of the regulator is anything with a p value less than naught point naught five. And of course, both of them are not true. But I think value is seen in a different way. And I don't know if we have some sort of confirmation bias, but I think that the different experts systems use may be expert positions, it may be expert patients.
I think it's far from true to say that every patient input every external physician input would make those same criticisms of the FDA handling of new oncology treatments, for example, and it would be really interesting to discuss some approvals, that they made on intermediate endpoints or something more closely related to the disease, and its impact on survival times or quality of life I think that's one really interesting question. And the other is about evidence, per se and the regulator I think being more able to make scientifically justified extrapolations beyond a population that's been studied in a pivotal trial is my perception that the HTA regulators finds harder to do and of course then it's incumbent on the regulator to be able to explain that decision, and perhaps that's something we can come on to later in talking about how the two systems might evolve.
So Leeza, last intervention you asked the question about whether the systems are appropriate functional and balanced and I didn't understand balance so I'm not going to talk about it. It is appropriate ? Yes. Both of these questions are extremely important. I believe both stakeholder groups, believe they are trying to serve society in the way that they make their decisions. I believe the impact of political pressure, industry pressure, etc is minimal.
I believe both stakeholder group genuinely believe they're doing the best they can, for the patients and the society that they serve.
It is appropriate also that regulators stick very strictly to their legal basis. We can't change regulation to meet the needs of HTA without changing the regulation that underpins the regulatory decision making.
Is it functional? Well that's a really interesting question because it works, new products are coming through, new products have been agreed and priced by HTA, not all of them, new products are being endorsed and authorized by regulators, not all of them.
Is it functional? Yes. Is it perfect? No, not even close and balanced I don't understand so I'll pause there.
Rick Vreman: I think I might open the floor also with the academic perspective from the research that we've done on the similarities and differences between regulation and HTA. I think one of the things that we found when we looked at the assessments of these organizations is that when they assess the same drug within the same indication they actually often highlights the same issues, and the same uncertainties. In the vast majority of cases, the issues that they highlight in the evidence are actually the same between these organizations, so I think this might also be sort of a nice starting point to realize that it's not all about the differences between the stakeholder groups but we also have a lot of commonalities and one specific examples for when we've looked at the orphan designation by the committee for Medicinal Products at the EMA. And we've compared that to relative effectiveness assessments of HTA organizations. This was a part of the EUnet work Plan. And we found that actually in almost all these cases the issues that were highlighted for these drugs where the same.
So these evaluations actually are very similar in the majority of cases and we've also found that when regulators already highlighted large uncertainties in their evaluation, Then we find that this is associated with more regular negative HTA decisions or reimbursement decisions. So, that emphasizes the point that when you're already struggle in the regulatory domain, then you can expect more struggles downstream, which is a I think a very good argument to make sure that we address uncertainties very early. And I think this sort of brings me to a concept or framework that I like, which is the drug life cycle, where we think about more prospectively, the interaction and the planning of the drug life cycle where we very early in the process we have dialogues between stakeholders and we know also from previous research that when regulators and HTA organizations give recommendations to manufacturers within joint scientific advice or parallel consultation, they often can agree on the recommendations that they want to give, and also that the manufacturer, often incorporates those suggestions. So that's a way to streamline evidence generation during the drug lifecycle, which might prevent some of the issues that we get where regulators find something from the benefit risk balance perspective acceptable but HTA organizations, find that when they actually need to really quantify the added benefits for the best available therapy that they are not so positive. And we've seen that when we looked at expedited pathways where many of these drugs get negative reimbursements recommendations. And then I think another phase of the drug lifecycle where I think we can really improve is the phase after the initial decisions. So after the first decision on market authorization and the first reimbursement decision, we found that we want to develop more evidence but we know from the regulatory domain that it is often quite difficult to timely generate additional evidence and we see that now also with the introduction of performance based manage entry agreements into HTA arena where we also have lots of examples of unsuccessful schemes where we struggle to get the evidence that that we want. So I think there's definitely some room for improvements. The final thing that I would like to mention is that when we talk about the cooperation between regulation and HTA at least in a European arena, you always also have to consider the cooperation between HTA organizations.
Leeza Osipenko: François writes that he thinks that the system of authorization regulation reimbursement of medicines is very confusing for patients. A lot of products authorized by regulators are subsequently knocked down by the HTA bodies. Wouldn't it be much more effective to authorize fully effective transformative products, which would eventually have a greater probability of being reimbursed by national health care systems. My question back to François, where would you take those products to begin with? And then, how would we know because a lot of times we don't know because we need more time?
Dan Ollendorf: Maybe my answer to Francois at this point is a qualified yes. Of course we want truly effective transformative therapies because that will improve population health and that is certainly a place where regulators and HTA are joined at the hip.
So, a few comments and this is obviously coming from the US perspective and for those of you not that familiar ICER is the most visible HTA body in the US but it does not have any official mandate, so its activity basically has to be tied to the regulatory timeline for decisions. Because in the US when regulatory authorization is given then market uptake essentially starts the next day. So in the role that ICER plays and so I've been collaborating with them on reviews, even though I'm in an academic setting now, there has to be a lot of guesswork about what looks like a good bit, what looks like it's going to make it through the regulatory process.
And I have to say, with all due respect to the approach that regulators take generally when I'm thinking about the FDA, I feel a bit like I'm in my 20s and dating again, because I'm always trying to guess what the other party's thinking. As an example, I'm currently engaged with ICER on a review of an investigation on a compound that was studied in a relatively robust randomized double blind setting. The comparitor is the active standard of care. So it's not a placebo. The primary endpoint was met, all of the secondary endpoints were met. Some potential safety signals, but on an overall basis the benefit risk profile look to be very favorable. There was a lot of public buzz about this product.
The company received a complete response letter from the FDA. That is a rejection, in other words, and while the contents of that letter are confidential, one of the elements that leaked out was that this was a novel method of action, and the FDA was interested in a second trial to replicate those findings.
So that made me feel like I was in a time warp. And we were 20 years ago when this is what the FDA said routinely. So I think that, where there is alignment and as Rick talked about where the uncertainty are greatest coming out of the regulatory conversation that tends to be where HTA decisions are more negative. That's one way to think about it and that's certainly a pattern that we see worldwide. But if HTA has an expectation that the regulator is going to act in one way and then it's completely opposite of what's expected, then it's really hard to know what to do from the HTA community's perspective.
Wija Oortwijn: I wanted to take a step back and also I see a comment responded in the chat about the need for a rising scanning and it was also exactly what I was going to raise because what we’re talking about I mean Rob was talking about that the system is appropriate and that I mean that the system is addressing the needs of the society but I would challenge that because it's product driven because everything that goes through the systems because it's technology push, it's not necessarily what we need as a society.
So, it might be appropriate but it might be appropriate from a certain perspective and that's something I would like to discuss also with the panelists.
And my urge indeed would be also to look at the horizon scanning systems but not necessarily from a product perspective but also from a societal perspective. So what are the trends that we see and why certain contexts might need certain products, are more needed than in others. I'm also a bit concerned about the system as it is now because what I see, especially in middle income countries is that, because we have managed entry agreements schemes they except lower evidence in order to have a policy instrument like manage entry agreements to have the push for example for high cost drugs for example quality products, for example in Kazakhstan or in Ukraine, well their essential medicines list is not up to date at all. So I think there's something wrong in the system and I'm not quite sure if I have the answers, but I think it's also because we have the system in place so there is a kind of expectation because we have the regulatory space and so we have the technology and then people want to have indeed those products, not necessarily because they address the needs of the society because it's also possible through certain policy instruments.
And they of course look at our countries as well. So I have some challenges back to maybe Rob and Rick in terms of: there are of course possibilities in how HTA can align with the regulatory space which is fine, but I think there is fundamentally something challenging in our system and our ecosystem and if we look at HTA as an ecosystem I'm not quite sure we address what we should address.
Rob Hemmings: Do we want transformative therapies? Yeah, I'm with that, of course, everyone wants transformative therapies, the interesting question though I guess is,
how do you want them and when do you them ? How soon should they be placed on the market and what uncertainty? if we are truly thinking of addressing an unmet medical needs, where the medicine that's potentially transformative, how long should we delay and wait for that evidence before regulators and HTA make their decision? What's the point at which that remains. and that's the challenging discussion, the argument that regulators are sometimes to quick and the argument that HTA is sometimes too slow. I should have said by the way I do think, both stakeholders have incredibly important jobs and incredibly difficult jobs. if I have to pick the HTA job is even more difficult, because the regulator has to say benefit is bigger than risk. HTA then has to go and put that into a value, has to be quantified and bound, and I think that is the tougher part of this. I don't only want transformative therapies though, I want alternatives.
I want products with incremental benefits I don't only want transformative therapies. And when we're working at the margins, with companies who are pushing for the best price possible. I don't blame them for that, but that's I guess the HTA’s law and sometimes the HTA will have to say no. The company hasn’t brought forward the dossier that substantiates the value of the product. There is nothing that enables a regulator to turn down the product has a positive benefit risk. And in the end you judge the products, not the development plan.
So even if scientific advice was not listened to and the company went a different way about it, the clinical trial design may not be perfect. In the end you have to scientifically judge the product, and its benefit risk, not the development, not the development plan. I think where they are absolutely right, of course, the system is driven by push and to some extent that's not surprising Is it, because developments in healthcare have to be driven by technology and have to be driven by science. We can all say, you know, this is the big medical need, but if there's no scientific answer to address that medical need it will remain a medical lead until the science does exist, and the companies are out there looking for the technology and the science.
What I think we can do in a resource constrained environment and maybe we'll talk a bit more about scientific advice later. In a resource constrained environment, we can put our resources, where we do want to try and pull the product through the system. And that's to some extent what prime tries to do, what the accelerated access collaborative in the UK tries to do and how parallel scientific advice might best on.
Carole Longson: I'm struggling to know what to say because there is so much that we can say about this topic, it's really difficult to think about where to focus. I'm going to start perhaps with Rob's point about the different functions, and I think he makes a really important point that we mustn't lose. The development of all new technologies is a journey, and along that journey, there are important decision points. And those decision points are both necessary and useful. And I think we have to respect that companies are making decisions about what they're going to invest their resources in healthcare systems are making decisions about what priorities they want to signal. And then in a sense the offices and the agencies that have to try to get a handle on the introduction of new technologies. also have to play a very discreet, and a very specific role. So recognizing those territorial boundaries is really important. Having said that, if I think back to quite a while back at NICE, we're looking at 10-15 years ago, we realized that two things: one, if we were so late undertaking our HTA compared to a regulatory approval we're not really adding a lot of value to the system. That delay in an HTA signal and HTA decision was becoming really problematic. And so as an HTA organization, we decided we are going to have to pull ourselves back as close as we possibly can to that regulatory decision in order to be the most effective in our own work. So it's nothing about what the regulators have done, it's realizing your own place in the ecosystem, realizing that gap between a regulatory approval and an HTA decision isn't good for everyone. So that's the first thing.
The second thing that we realized was that it wasn't very productive for an HTA organization just to talk about how problematic the regulatory environment was and how things that were happening in the regulatory environment weren't useful for subsequent HTA. So we took the decision that we have to start understanding that environment in order to be able to influence how the handover between a regulatory decision into an HTA environment could be optimized. That's why NICE scientific advice started and all the increase conversations between regulators and HTA bodies began.
So, the key point is that it's probably less useful to talk about the misalignment and more useful to talk about how do you optimize handover. How do you optimize what's coming through for the subsequent part of the journey.
It's a bit like a relay race right from the beginning from the original science, it's getting something useful into the hands of people that really need it. And if you do not have those handovers, not just the lines, but understanding what they need to do in order to hand over efficiently, then you do not have a whole system thinking and absolutely you do not have an efficient framework. And therefore, my thought would be, how do we further energize the key players in the ecosystem that's not just regulators and HTA bodies, but the key players in the ecosystem to realize, and therefore do something about the fact that we have to have that handover in the most effective way possible?
Hopefully we can explore some of those this evening.
Rick Vreman: It’s quite incomprehensible that there might be a very large delay between a regulatory decision and an HTA decision wherein the patient does not have access to a therapy during that phase. And I totally agree that we should try to minimize the lag here and make this relay race as efficient as possible and it also extends beyond that HTA part, into the involvement of clinicians and the patient as well but that's not the focus of today of course. I think one way to further enhance this transfer from regulation to HTA is more conversations early and I emphasize this again that the early dialogue between the different stakeholders and the sharing of information, explaining to each other, why you make certain decisions based on what considerations that really helps with the smooth transition from regulation to HTA.
And then an earlier point that was made was the point about unmet medical needs where we say well, “at the moment, drug development is not really driven by the medical needs that exist in society” and I think that is something that we also might want to focus on a bit more in the future where we say well, “actually, these are the unmet medical needs that we really would like to see addressed” and maybe even formulate what you then expect with respect to the evidence that you want to have generated for drugs addressing those unmet medical needs. Even maybe come to agreement on the endpoints, to what constitutes a clinically relevant effect. All these things can be discussed already in advance I think between the different stakeholder groups. I think Rob mentioned that we judge the product and not the development and I think that's true. I also think that we often do consider some aspects beyond the product, for example, eligibility or to an expedited pathways. We need to address an item of medical needs to be eligible for example for the conditional marketing authorization scheme in Europe. I think this definition of addressing an unmet medical needs is actually quite vague and I think coming to more agreement on how stringent those criteria for an unmet medical need would have to be, might also help the alignment between regulators and HTA organizations.
And maybe, coming back to some comments in the chat I think, this is also part of the pharmaceutical legislation review, the place of unmet medical needs and the definition of unmet medical needs within the regulation.
Leeza Osipenko I do think it's very interesting that Rob raised an issue whether we judge the product or the development plan or not and whether development plan should be judged. I think it’s a very important question so if some of you can comment on that: what's on to the development plans, the number of trials, the type of trials, the timelines?
I think the biggest problem we're seeing now is we wish we paid a lot more attention to the development plan and sort of influenced it early on, because if a proper comparative trial could have been done while we allowed for a regulatory pathway for an expedited path through a single arm small trial, this brings a lot of harm to the HTA process where we're lacking this comparative evidence, which could have been generated. (We do have technologies where it's impossible) so whether this kind of dialogue can happen early on to influence that and actually demonstrate whether those products are transformative rather than looking at overall response rate and hyping a product after phase two, which results in nothing after phase three.
Eric Low: The first thing is that patients deserve better. All the system players. It's really crazy, the silos, the disconnect there in the system and the lack of a relationship between demand and supply. It's just nuts, and it needs to be fixed. So the answer lies in a systems approach. The answer lies in getting the right legislation and the right stakeholder in the system. As I said previously, until we get vested interests unified around a common goal which is delivering benefit and value to patients, and we get the right incentives and rewards in the system, it will not work because all the players just do what they do without battling. We need to think about it in a systems approach. and the real analogy, of course is right. But in any system, unless the inputs are good, the outputs are going to be so rubbish. So each stakeholder needs to think carefully around their role in that to make sure that, and notwithstanding the commercial investor interest, regulatory interests, they need to do their very best to generate the right data at that stage to pass on to the next decision maker.
And that comes a little bit back to “is the system looking at the product or the development plan?” and of course, it looks at the product and it shouldn't. It should look at the development plan. Because it's the development plan that will ensure that the right questions are addressed at the right time point and the right evidence is generated for the appropriate decision maker in the system. It's not the job of the system, particularly when things come to HTA to mark poor homework.
So in the UK perspective, there are lots of flexibilities and I see the massive influence of pharma on decision making. Think about it in the UK context, the impact that the voluntary scheme has had on NICE is massive, largely because of industry pressure and patient group pressure. And the more that flexibilities are built into HTA decision making , not factual uncertainties that nobody could have done anything about, but
uncertainties that have arisen because the wrong decisions have been made, the wrong questions are asked. It's not HTA’s job to fix that through flexibilities, managed access, performance outcomes schemes. These should be reserved for people who have done a very good job of developing the evidence and it shouldn't be to fix that problem and we should push them back and say “Look, you need to go back and fix your development program because the system can't bend enough and shouldn't flex enough to mark your bad homework” Patients deserve better but the problem is patients don't know any better because we don't understand how can a regulator deem a drug to be safe and effective and a HTA body see actually there are huge uncertainties around ‘how good is that drug’ and we really don't know if it's safe because we've only got a six months follow up from a phase 2 study, it makes no sense.
And one other point, and this comes back to Rick's point about lifecycle management which is something that I have been a huge proponent of for my entire career is that one regulatory trial can’t answer all the questions the system needs and the different decision makers need. A regulatory trial is set up to answer regulatory questions. It doesn't matter if these questions are clinically relevant and plausible to patients and health systems around the world. So we need to think about evidence development as a package because we can't answer all the questions the system needs in one regulatory study.
I reflect on the UK, particularly England, we have consequential clinical pathways.
So we have these pathways that are constructed by random bits of marketing authorization for drugs designed largely for the US market and we shoehorn them into consequential pathways, they're not state of the art, optimal kind of pathways. So this idea that Rick mentioned is cost-marketing authorization, post HTA approval. There's still a massive job to do, to shape coherent, plausible, and sensible and patient centric clinical pathways in the real world. And we're so bad at collecting data on these clinical pathways, whether they're consequential or state of the art. But we're not then putting that information back into the system to further improve evidence development.
Dan Ollendorf: I am thinking I was going to be introducing my comments along a systems perspective already and so you've actually given me more to think about, so I'm trying to process real time. So one of the things that strikes me and, you know, in the U S we don't have any formal early scientific advice or parallel advice, even, The closest thing we have at the national level is parallel review. The regulator and the major public payer CMS, But that activity has been extremely limited, not focused on drugs at all, only on diagnostics. And there are a lot of political and logistical reasons behind that. But one of the things that strikes me as an observer is that there may be beneficial conversations that happen, but they're at the margin and there is potential for significant information loss, even when HTA is then appraising something that it has given early advice on, or regulators are then reviewing something they've given early advice on, or even within and across product teams and manufacturers.
Could we, as a group, envision the notion of an international collaborative that involves those in the regulatory and the HTA community coming together in a kind of a consensus development format, maybe even professional society driven to identify areas where considerations around measurement outcomes, populations are relatively well-established, to develop some consensus guidance for industry that's replicable across products that can be used broadly. And for those areas where there's more uncertainty, maybe even try to break those up into manageable bits. So there's been a lot of conversation in the chat going on about the orphan drug legislation, both in the US and in Europe and how there is the potential for perverse incentives and abuses, but that could mean that this consensus group then works on elements of orphan populations that are addressable. So those orphan diseases that don't have challenges necessarily with measurement or with such rarity that a trial enrollment could be compromised, or things like that, to try to develop guidance that again is replicable across studies. Then put in some additional guidance for those areas where measurement is still a challenge, accelerated pathways and creative solutions to manage entry need to be considered. But try to take the lion's share of the concerned out of the equation by developing this guidance proactively.
Rick Vreman: I can be quite brief because I completely agree with that. I think this prospectively defining what we actually want to see as regulators, as HTA bodies is immensely important. I think it's also maybe a bit unrealistic to expect that the manufacturers come up with an extremely comprehensive evidence package when we know that you can get regulatory approval and also get some form of reimbursement with way less comprehensive evidence package. So I think it's also a bit unfair towards manufacturers to say, well, they haven't given us the evidence that we want when you never made that clear what you would want within a certain disease area or for a certain group of patients. So I think specifying what you would expect and then also sticking to it and being more critical if you don't get what you like you have specified is immensely important.
And Dan also mentioned the accelerated approval. And I think managed entry agreements were mentioned also a few times before. These evidence generation posts, these initial decisions, these processes should be more aligned, I think because everybody's struggles to get their right evidence, but still everybody also requests their own evidence to address their own questions and that is also not very efficient. So I think we should try to align that a bit more. And the one good argument is also that regulators have struggled to force manufacturers to come up with evidence in a timely fashion also because it's quite difficult to revoke a marketing authorization based on delay in the generation of evidence. This is an opportunity for kind of a collaboration between regulators and HTA organizations, because it is easier to link a lack of evidence generation to the price of a drug. So you might envision a scheme where the price is actually linked to whether the evidence that is needed was developed in a timely fashion.
Leeza Osipenko: A question from Rita in a chat about increase in scientific advice in the EMA and whether quality of submissions or understanding improved. Maybe Rob you can remember from your experience. Because I think we try to measure it and understand it at NICE and It felt overall that things are getting better. And then you get a briefing book where you realize that they're not. So it really depended on some companies: there's a bit of internal learning going on through parallel processes. I think HTA agencies learned a lot and exchange a lot of information, so there's huge value in that. How much it approved industry it's patchy because it improved some organizations a lot while it didn't improve others. It would be interesting to hear from regulatory perspective.
Rob Hemmings: I think largely speaking companies understand what regulatory standards look like. There are indications where it’s appropriate still to do a regular development plan that includes phase one, phase two, those finding proof of concept, multiple phase three trials, unplanned for post-authorization, evidence generation, and lifecycles. There's other indications where that's simply not feasible. You can try and generate all of your evidence in one shot and get away with a single phase three trial. Even some beyond that way, where you might be able to show efficacy and benefit your product on a smaller data set. And I think those are the ones that the kind of mostly discussing around. But companies use scientific advice, not so much to understand what the regulatory standards are, but to try and test them, to try and push them to see, can we do something a bit smaller, a bit quicker? Do we really have to do X, Y, and Z? And so there'll always be that lens through which you have to look at scientific advice type interactions. The history of the parallel scientific advice has been a positive one, but it’s not been an easy ride, from those early interactions where you have the big EMA scientific advice working party machine, and people dropped into that conversation from HTA or payer bodies around Europe with ideas on what the target population would look like in their region, what the comparator would look like in their region, et cetera. but almost impossible task for the company to reconcile. So we gradually move to more towards EUnetHTA. Perhaps that got close to down to this idea of let's apply a consistent standard and line of thinking from the HTA bodies that we could actually learn to use and generalize from.
But then capacity becomes an issue. And I think EUnetHTA call for, is it eight or 12 procedures per year and EMA scientific advice and Anya who is in the chat run 16, 18 per month, of these discussions. It's a big old world out there to try and get every single development program, streamlined and harmonized across all of the stakeholders, not to say it's not worth trying just to highlight the operational challenge of doing so.
I fear a world where regulators and HTAs write down what the development program should look like and what the drug effect should look like. And industry have to go and meet those standards because I think things will get gold plated. Now it's fine to say, this is what the product looked like, that I would pay this much money for, but I don't think you could ever say, 'this is what a product would look like otherwise I'm won't approve it' because again, I value small benefits, incremental benefits, alternatives for prescribers and patients coming forward to the market.
Let's take the most difficult example, but everyone likes to criticize. The company bringing forward a drug in oncology based on shrinking the tumor for a durable amount of time and a relatively small safety things. Well, the regulatory step standard has been to try and authorize that product in patients who have no other good treatment options.
I can't say that’s wrong for the regulator to place the product on the market that shows a durable shrinkage of a patient's cancer. I can't say it's wrong for the regulator to do that and to make that product authorized with the possibility for the patients to use it. I can understand that the HTA says I'm not prepared to pay X hundred thousand pounds out of my health care budget for that I can understand that perfectly well, but I can't say that the regulator is wrong in placing that product on the market.
Wija Oortwijn: I agree with what Rob was saying. I think that HTA agencies and regulatory agencies understand something different, what they want to see as an outcome. Rick was trying to say, okay, maybe we can give industry more clear expectations, but the expectations are different because the risk benefit assessment is something different than what HTA assessment are asking for when they recommend or make a decision with regard to pricing and reimbursement or even procurement. So I think there has to be more alignment. We were talking about this parallel scientific advice that could be a kind of a solutions for this, but still, even though I'm quite challenging whether we as an HTA community really understand the same bar of evidence compared to the regulators. I'm struggling a bit with this common language. Do we really have the same problem? And are we also looking at the same solution? So, and I'm not quite sure about this. Because when I hear Rob talking he's correct, because he says, you know, industry knows what we are asking for, because this is all very well laid out and it's very, well-regulated. It's true. But of course, what we are asking from an HTA perspective is something different. What the industry has to provide to the regulatory agencies, is also something different than when they come to a national HTA authority. And then we are asking something different. So of course there is misalignment. And the question is, can we indeed make that more optimal from the beginning? And I'm not quite sure because the regulatory agency has another remit than an HTA agency. So I'm still struggling with this. And of course we can try to optimize this. And I wanted to make a comment that DIA is now also collaborating with HTAi at least to see what are the issues and do we have the same issues? And can we come up with solutions to address, for example, uncertainty, especially at the HTA and the regulatory interface? But I'm not quite sure because I still think we are not talking in the same language and I think we should focus a bit more on this. Of course collaboration is great and I think EUnetHTA already has done quite some important work in this and I also saw a response that it is not easy and it is time-consuming to get everybody aligned even within Europe. So I'm not quite sure how we could do this. Sorry. I have no answers.
Carole Longson: I'm not sure I'm going to give you any answers either because it is quite a knotty issue. And what I'm going to say now is going to be quite simplistic. But hopefully the concepts will help us work through. I'm concentrating still on this handover. If we can find a good way to hand over from one part of the journey to the other it would be much better. Simplistically, if you think about the remit of the regulators. Does our medicine work? Is it risk benefit? Positive. And if you think about the HTA again, caricaturing, how well does it work? So we're looking at magnitude of effects. So those two things are actually quite useful things to know individually. It would be quite useful also to have some connectivity between those things. So does it work? How well does it work? In that handover from, 'does it work' to 'how well it works', let's take an example. Rob's brought it out already, the use of a surrogate endpoint in answering the question 'does it work?' . We can use that surrogate end point and be able to have some conclusions on 'does it work' based on a surrogate end point, but you'd probably need to explain why that surrogate endpoint can be used for a 'does it work?' decision. Are we explaining that sufficiently well enough? Because that's very useful when it's moving into the, 'how well does it work' context and the issue with that move from ' does it work?' to 'how well does it work' is that I might not able to get a handle on how well it works from that surrogate as easily as the question, 'does it work at all?'.
So very simplistic issue about handover that hopefully gives you an example of the problem. And we're still not fixed that one between regulatory and HTA. Is there not a way of being able to at least come to some consensus between regulators and HTA agencies on the use of surrogates and in what circumstances, and possibly even qualifying them from an HTA and a regulatory perspective. Now I know I could be a rather heretical to some of my HTA colleagues in even contemplating the use of a surrogate in an HTA environment. But if there is a relationship, a proven relationship between a surrogate and a final endpoint, you have at least some scientific validity for being able to accept it for the final outcome that an HTA body might be interested in. So there's some way of being able to align, and as a consequence have a much smoother handover between a regulator and an HTA environment. We talk about it in the macro level. Let's do consensus documents. I think we're talking too large. I think we should get down into, into some very tangible, very small things that regulators and HTA bodies can align. And then from that, hopefully we will start to understand our language a little bit better.
Leeza Osipenko: I think it's very good challenge. And if we look back at history, what has been done, everyone was coming to the table with their own story, and we practically did not dig into any joint methodological work. And the big question is, are the parties ready to that level of interaction, or they're just ready to sit at the table and everyone preaching their truth and then figured out why they don't agree? So that would be a very interesting initiative, which could be led by I C H or by a EUnetHTA.
Eric Low: I'm going to do my very best to articulate three points succinctly. Hopefully will help answer some of the questions that have come in. So the first one is that I think part of the solution lies with patients and patient advocacy, because nothing happens without patients. Research mostly will not happen without kind of patients. And I would say on the whole, patients are in a difficult position regarding research. Local, regional and global advocacy is often not fit for purpose because it's advocating for the right things. I think if patient organizations and advocacy groups truly understood what was happening in the system, we wouldn't stand for it. I think the change agents has to be advocacy. So the more that we can educate, empower, explain, and provide evidence to patients and patient advocacy to change the system and it all goes back to government sometimes and it’s about legislation and all the different actors and players just work within the rules that they're given. So I think there needs to be a huge effort from a patient advocacy perspective and the representatives of patients to up their game and actually become a stronger voice and work on this broken system.
The second thing is going back to Rob's point about gold plated, which is well made and I agree. And I guess the point I was trying to make is, is it just in the context of that better relationship between demand and supply, the more the demand side understands what it needs, what it wants, the better it can articulate that with more strength and less flexibilities. And what I would say is that you shouldn't be seeking perfection. Perfection is the enemy of the good, I think that's the saying. But actually if we aim for marginal gains and marginal improvements, with every stakeholder, every step in the system, it will be transformative at the bedside. And so that's what we should be aiming for. Looking at how we can better connect the dots and looking at incentivizing and advocating and brokering for marginal improvements at each step in the system. And maybe that's a bit idealistic and naive, but I think that should be where we start.
And the third point, which is slightly different to what I think is being said is, and I think Heidi may have brought this up in the chat. Should the burden of evidence development be entirely on the shoulders of industry? And should the system trust industry? And I don't mean that in a negative context, but necessarily trust industry with the responsibility of producing all the evidence, that's the decision makers need and with this move towards at least, a nice perspective is looking at the totality of the evidence, obviously without randomized phase three study, that's clinically relevant, well-designed, the right end points being at the top of the tree. Is there a role for academic and medical research, charity funded research to actually share some of the burden, particularly at a local level where different HTA bodies might need different evidence to set it in a national context? So again, looking at an England and the UK perspective, it's always struck me of the complete disconnect between the incentives and drivers for academic research compared to the decisions that NICE have to make about evidence. And I'm on record as saying this several times so apologies for being repetitive. I sat on the board of the national cancer research Institute for six years or so and I kept asking the same question: is can you give me metrics to let me know the extent to which the results of NIHR funded MCRI badge trials, have materially changed clinical practice in any cancer?
So, what we can see is we funded this many trials, this many patients within the trials, but we can't tell you what happened to the results and the extent to which these results change clinical practice, because there's no commissioning pathway for the results of academic research, unless you've got marketing authorization. And unless you've got a nice badge, there's no obligation of the health system to kind of pay for it. So what contribution can academic research play in reducing uncertainty in generating evidence. And I think that's an untapped potential to help solve problem. And actually government should be incentivizing academic groups and incentivizing medical research charities to actually think about the relevance and plausibility’s of the research they're doing, change the incentives, change the metrics and actually get them to contribute to reducing uncertainty in getting better evidence into the system to support decision making.
Leeza Osipenko: I think during the first seminar we brought it up saying we should have trials running in parallel led by non-industry partners. The question is, would industry trust that side before the marketing authorization? I'm talking, not academics doing the trials on behalf of industry, but actually independent trials. So it's a huge gap and it's fantastic you raised it. And the question is we have it now not enough.
Dan Ollendorf: Just following up on one of Eric's very good points. So in my vision for this approach, I was thinking about something like a core outcome set, driven group of exercises, but not in a traditional sense. In a sense where patient input is over-sampled or overweighted or both. Because I think trying to understand exactly how the patient experience translates into both what has been measured in what should be measured is critically important. But in terms of the output of the exercise itself, I was thinking more in terms of, I don't know if this is a rugby term or not, blocking and tackling and holding everyone's feet to the fire. So if a regulator is saying for a new anti-diabetic that potentially has cardiovascular benefit, we need to see results of a cardiovascular outcomes trial before we make a decision. That guidance has been out there for a while, but the decision-making is now in advance of those results in many cases, or bringing in the Alzheimer's example again. If you're going to measure benefit primarily on the back of a fancy clinical instrument that is never used in clinical practice, then at least take one of those clinical instruments that is used in clinical practice and make that a secondary endpoint. So it's kind of basic instruction. I'm thinking of a minimum standard that HTA and regulators can agree on recognizing that there will always be a need for further information, particularly at the back end. But at least a starting point that industry can take as consistent guidance. And again, it's an individual company's decision as to whether they adhere to it or not, or argue for something different, but at least it's out there.
Leeza Osipenko: I have a quick challenge, at least theoretically, I can imagine what Carole raised as kind of early mythological work or to get into this kind of routine to give instruction to the industry to think in this way. My challenge to you, what are the chances with FDA and anyone else on the plate of ever considering this? Because no matter what, even the drugs that we get to see in Europe, you can see just how much of a role FDA plays. And it seems to be the only target but that's the kind of perception you feel.
Dan Ollendorf: It's like anything else, it's a leadership driven culture. The commissioner tries to bring some of that into play, then maybe we'll have those conversations. So yes, I agree that there has to be willing participation from all sides. I think one of the things I neglected to mention earlier is that as Carole mentioned, we need to understand details about product development plans. We also need to understand details about product commercialization plans, right? We need to understand as a community, when an IV drug is expected to go subcutaneous when supplemental indications will be filed for after the initial indications approved, some of that may be unknown, but I think a lot of it is known and right now it's held very closely to the vest.
Rob Hemmings: I'm going to pick you up on the words, Carol, just because I hope sort of illustrative for the people, listening. 'The regulatory the decision will be based not only on does it work, but how well it work' that's inherent in the benefit best trade off. There has to be a magnitude of benefit versus the risk. Of course, as I said in my opening comments, that doesn't mean that the regulator and HTA is agreeing on what benefit is and what value it is. But there has to be that judgment of a clinically relevant effect that offsets the toxicities. The regulators will not, of course go to the value of the product within a particular healthcare system, or even sort of relative value compared to other interventions outside certain specifics. The discussion of surrogates is fascinating. And to me there is hardly any surrogate endpoints. When the regulator makes the decision based on tumors shrinking, or delaying times to tumors progressing or LDL levels or HBA-1C. It's not because of surrogacy, per se, the idea that an effect on this endpoint of magnitude X will transfer into an effect on this endpoint of magnitude Y. Its because you see sufficient value in the effect of what I call intermediate endpoint per se. And then let's use outsiders that we've been talking about as an example. Should you authorize products because they reduce beta amyloid or plaques. You can have a view on that. Should you authorize products because they've changed these artificial scales. Or should you actually wait until patients progress to some extent in that functional disability and develop, perhaps full-blown Alzheimer’s from a pre dementia type of diagnosis at the start?
And those are legitimate conversations to be had .This whole conversation that we're having today it's not going to get easier as we go forward. Alzheimer's is a fascinating example because the science tells us to treat early and the outcomes that we don't like to know probably 10 years down the line from that treatment. But I doubt anyone, including the patient community wants 10 year RCTs before any treatments are made available. Let's ask, let's find out. So you have to choose some time point for that access decision, but importantly, you should still have the evidence generation plans in place. Whether it's continuations of the clinical trials or whether it's good epidemiology in clinical practice that eventually answer those questions or answer those questions as efficiently as possible. And where that's in everyone's interest, the regulators should be building that into the obligations for the marketing authorization or as recommendations to the marketing authorization.
I think the discussion of the patient input is extremely important. The part that doesn't get discussed enough is how do you understand the perspective of an entire patient community? And I'm interested in understanding the breadth of perspectives across the patient community. I will only ever believe there will be one voice that says this is the endpoint that's important in that setting to all of these patients or this risk is tolerable, or this effect size is acceptable. It must be about understanding the spectrum of views from a patient community as a whole. And stakeholders, such as regulators and HTA being aware of that spectrum of views when they take their decision. I can easily imagine there'll be some patients who really want regulators to approve drugs based on the fact that the tumor shrinks for a long time. And there'll be other patients that don't want to take a drug that causes grade three grade four toxicities until they've got a survival benefit, but that's not quantified. Just to finish on something more practical. Unless we start with a blank sheet of paper and rewrite trials focusing in on the right discussions and this handover, and I'm sure regulators can do better in assessment reports, particularly as I say, around extrapolations that are made from clinical trial populations, perspectives on patient defaulted outcomes, quality of life data, and detailed methodological assessments of those. Perhaps needing to change this perspective that that type of endpoint is an uninterpretable from a single-arm trial. I really like the initiative at the EMA to have sort of webinars, which was done at the time a product was approved. It was done as a pilot at the time a product was approved and people could dial in and ask question. Or maybe there should be a shock, but the relevant patient community, development parts of the HTA and the regulators to have a discussion about ' these questions are not addressed', or 'I don't understand why you're going to do that' and make sure the products, the assessment report and the post authorization work is more fit for purpose.