Introductions

Till Bruckner: I'm Till Bruckner from Consilium Scientific’s TranspariMED campaign. I’ve been working in the area for clinical trial transparency for about 7 years, and when I started off in this field it was intensely depressing what you saw coming out in terms of research because there was one paper after the other, saying, oh, there's a problem. And then they would  quantify the problem in some way, but they wouldn't tell you with which institutions exactly the problem was, they couldn't tell you who was responsible for the problem. And then in the conclusion they write that. Well, we've got a problem and it could be solved if everyone just did things better in future. It was really not constructive, and over the past few years we've seen a massive change in that, so I'd like to welcome our panel today, and they've all been part of that change and they'll be presenting on the work that they're doing in  4 broad areas. First of all, is the  dynamics, what actually drives clinical trial reporting at institutions. The second one is new approaches, new tools to research and advocate on this. The third one is country strategies. What people are doing in different countries around the world to promote clinical trial transparency, and then at the end, we're going to have a discussion, and that is going to be moderated by Emma Thompson from Cochrane.

Anthony Keyes: I work at Johns Hopkins University in this clinical trials.Gov workspace, and I also represent the clinical trials, registration and results reporting task Force, which is a consortium made up of over 200 US Academic institutions, and over 600 members who are collectively working to gain some grounds here. We represent the academic side so one of the most important parts of my job is helping individual academic centres meet their clinical trials.Gov reporting requirements. We find a lot of people want to do that but have some trouble understanding regulations. So we pulled our resources collectively and we work towards developing best practices so everybody can be more, more compliant with registration and results, reporting.

Daniel Strech: I am affiliated in Berlin, where I am core-directing the Quest centre for responsible research. You will hear a little bit about projects and products we work on later from Maya and I was invited to comment briefly on a certain first and second class transparency as we perceive it a little bit increasingly, at least here in Germany. So the point is we saw a heavy increase in the transparency so to say, for clinical trials over the past 5 years and this is a narrative that you hear in Germany from many stakeholders. And it's true, but it's true only for drug trials sponsored by university medical centres. That were registered at the European registry, and they improved heavily in summary results reporting which they are required to do by law. If they didn't do it for many years, 5 years ago there were below 5% of trials that reported summary results, with some pressure over the past 5 years long story short, they improved heavily. We now have universities such as here in Berlin, that has a summary result reporting rate of over 95%. So it's really a heavy increase in, let's say, clinical trial transparency. But it's only for this subgroup. If you go to clinical trials.gov.  Search Trials from German University medical centres and check for how often they report summary results, so including device trials, psychotherapy, surgery, and all the other non-drug trials. There is no increase. It's still below 5%. It's the same with manuscript publications. If you search for how many of them that maybe not report summary results in a registry. But how many published their results at all via a manuscript, for example, we have, of course, a higher publication rate, which is between 40 and 50% but you don't see an improvement there. So it really stays at this level. What increased is only this drug trial specific point of summary results reporting. This is why I would argue that we, increasingly, at least over the past 5 years, see those 3 important stakeholders, the Regulators in Germany, but also the academic institutions, and also funders, take very seriously the transparency of drug trials. And here, with regard to transparency, summary results reporting. But for both perspectives the clinical studies as one perspective, there are, of course, much more clinical studies that we should care about, not only drug trials and with regards to transparency I also see a certain first and second class aspect going on. First-class transparency, so to say, is the dimension of summary results reporting but transparency is much more than just summary results reporting. Publication of manuscripts and having them open access, and so on. All very important dimensions of transparency. So this is my input. I don't directly have a solution, but I think this was the idea here to bring into discussion this this danger, so to say, of first and second class transparency, for both clinical studies, and the different types of transparency

Gustav Nilsonne: I'm an associate professor at the Karolinska Institute in Stockholm, Sweden. I'm co-leading a project to follow up the reporting of clinical trials in the Nordic countries. We are strongly inspired by the work done by Daniel and his group, and I'm also happy that several other collaborators on this project are on the call today. I won't mention everyone in the interest of time. So what we are doing is to follow up trials registered in the EU Clinical trials, registry, and the clinical trials.Gov registry that are marked as completed and we're looking for results in the registries, as well as in the published literature, with the help of a large number of volunteers who are searching and decoding studies. Now I agree with what Till said in the beginning. What we are doing now is rather descriptive, and whatever the results are going to be, we are probably going to conclude that they could be much better for that reason we are talking to stakeholders at an early stage. We will soon have results, but I've already started conversations with the major stakeholders in Sweden research funders. The health technology assessment agency, the Ethics Review Agency, the drugs regulator and some of the medical universities, and so far everyone is quite enthusiastic about the data that we're going to deliver and understandably many of the stakeholders don't want to take on extra work unless they get a new mandate, and perhaps some funding to go with it. So basically, we are starting a conversation between stakeholders initially in Sweden, later on also in the other Nordic countries, and I'm hoping that that will lead to some way to find improved institutional responsibility, and the support to researchers.

Claire Veryard: I’m the program manager for the ISRCTN registry. And today I thought I'd take a bit of an opportunity to throw out some ideas. So I've been looking into research on things like household recycling, where you can see that there's similar. There isn’t really an immediate tangible benefit to the individual but there is a community benefit, perhaps, in the future. Trying to change people's behaviour means interacting with quite subconscious processes of cost-benefit analysis that are going on in people's brains and going on in animals’ brains, so it's a really deep thing. So I think we need to think about that, and I would like to add, make it worth it to the HRA’s Make it public transparency slogans. So just going back to those slogans, the first one is make it easy. So actually for people to do things it has to be easier than not doing it. So that's kind of difficult where you're expecting them to do a lot of work. So maybe what we could think about here is can we outsource and professionalize the actual transparency activities, you know, and make people, you know, so that there will be experts in doing that? That would free up the researchers to do what they're best at. The other thing obviously, is technology. So could AI create summary results based on published results? Or could we have an app where all the individual data is stored, and then can be shared by the app. As for making transparency, the norm, currently, the norm is actually quite a low bar. And yeah, that can help to normalize and justify bad behaviour. We can already point to a lot of research showing that people aren't doing what they should be doing. We need some good news stories. There has been impact from people sharing data or reporting what they think are inconclusive results. The other thing here is kind of trying to reduce people's tolerance of their own waste. It needs to be actually disgusting to people that there is waste out there that they have created. In terms of the worthy effort there needs to be a kind of timely personal benefit, not the vague promise of future benefit to others that doesn't really trigger the behaviour that we want people. Things like summary results really should have a separate citation that people can use as an output. Could funders withhold some of the funding until the study is reported and then release it, could investigators pay a deposit, and then get that back when they report. I'm thinking about a very successful bottle recycling in Germany where even if people themselves are, aren't recycling there are people who can make an income from that. So there's a kind of incentive to do it. So I'm hoping to hear about your ideas about how to incentivize people, because we really need to make it worthwhile on a deep subconscious level.

Delwen Franzen: I'm presenting also, together with my colleague Maya, and we're both based at the Quest Centre for responsible research just like Daniel, who presented a little while ago. So we are going to tell you a little bit about some of our work on to drive clinical trial transparency and so we use very similar methods to what Gustav presented. So, not going to go into detail, but essentially, you wanted to build first a status quo analysis of how trials that were conducted at German University medical centres were doing on several registration and reporting practices, and then we wanted to make that data set useful to drive change and so we piloted 2 different communication and change strategies, and the first one I'm going to talk to you about is an institutional dashboard and the motivation here was really to try and take that status quo analysis and communicate it in a clear way to leadership a university medical centres to help them support, improvements. It's really difficult to change if you don't have a clear idea of how you're currently doing. And so this is a screenshot of what this dashboard looks like. It's recently been published and as you can see, it just gives university medical centres, a quick overview of how they're doing on different registration and reporting practices, and this also includes the timely reporting both for summary results in the registry, but also as a journal publication. And specifically, this work on tracking trial results as general publication involved quite a lot of manual work. And this is maybe something to keep in mind when that is a bit different also to other trial trackers that are out there. So how did we use this dashboard to drive change? So, of course, this was a paper, so this would reach the relevant academic community, but we wanted to go beyond that, and we had 2 main target audiences. One was to really raise awareness of this issue among the general public, and so, together with amazing support from Till and others, we got a lot of attention from some media outlets in Germany, and that was really helpful, I think, in bringing this topic to the fore and second, we really just wanted to build some really concrete tools that could directly support university medical centres to improve. In that way we are actually engaging also with the relevant community here in Germany. So, for example, the association of medical Faculties, we're planning a workshop with them to discuss the dashboard and how to move forward from there. So all in all, we really think this has been very helpful in getting the conversation started, and to move into implementation and with that I'm passing onto Maya, who's going to tell you a bit more about our second communication strategy.

Maia Salholz Hillel: So alongside this dashboard that gives you summary statistics about an institution we wanted to make sure to engage with the individual trial stakeholder, because they're in a prime position to make sure that they're fulfilling transparency requirements. But they don't always know how their exact trials are doing, what the laws are, what the guidelines are, and what they can quickly do. I think Claire touched on that point. How do we make this easy for people? And so we designed a report card in discussion with several of the people in the call today to get this information in front of the trial list in as compact and clear away as possible. So we have a simple tabular format that shows the practice. Whether the trials doing it, some details about the trial, and crucially what can be done. And this is hyperlinks to generalist resources about the registry, or even institutional specific resources. So I'll just quickly show you one example, which is for summary results reporting in the registry. This trial, for example, did not do that, but still can, and here's hyperlink on how to do that. So we have a tool. But again, that's only the next step from data set tool. We want to make sure that this is actually helpful we are just wrapping up a feasibility study where we disseminated this at the Charité to over a hundred trialists. We got pretty positive feedback on a survey. There was interest in this tool. There were also some struggles with the bureaucracy around this again, Claire, you brought that up. How do we make this easier for folks? And we're now looking at whether this actually drove change? We're not seeing huge amounts of change, but we did see an increase in the number of publications that have been linked in the registry. So we're hoping that maybe we can maybe identify some practices that will be a step towards transparency. And that brings us to our next step in driving change for both the dashboard and the trial transparency report cards, which is, we want to make sure we're engaging the stakeholders in how we develop these tools and how we share these tools so we're moving into a more qualitative phase now that we have these prototypes that are going to involve interviews and cocreation workshops.

Elise Gamertsfelder: I'm a former clinical trials nurse in the US. I am in London for my masters in health policy, and I've been working with Till and Consilium for almost a year on assessing the trial policies of the 14 largest public and philanthropic funders in the US. The impact that this had was pretty mixed. It depended on the funder. Kind of unsurprisingly a lot of our large public funders didn't give any response. Some of our philanthropic funders were pretty receptive and quick to act just to name a couple: the Alzheimer's association immediately modified their language to remove some ambiguity in their policy documents. The American Heart Association is forming an advisory group, for when they start to scale up their clinical trials. The juvenile diabetes research foundation added some documents that were previously private to a public part of their website to increase transparency. I think that the 2 takeaways from doing this was that benchmarking was very helpful for the funders to compare themselves and see how well they're doing. I think that maybe injecting a little bit of competition is motivating for them. And then echoing what Delwen and Maia is saying, working with a journalist and helping to get this outside of the academic world and into the public, especially since all this money is coming from public funds is hugely helpful.

Nicholas DeVito: I'm a postdoc at University of Oxford. I work at the Benin Institute for Applied Data science under Ben Gold Acre as my supervisor. I run the trials tracker project, which for anyone who doesn't know, is these live automated trackers that track compliance with US and this is the EU one, for EU laws around trials reporting. The reason we created these was that you have these laws. But there was no way anyone knew if anyone was following the laws, and the people who made the laws that were implementing the laws had no interest, seemingly, in showing who was following the law and who was and wasn't following the law. So we thought that we could make a good run at it, based on developments in both the Us and the EU and how data was made available, and how the clarity around the laws and the rules governing trials being reported. So we created these, we published about it, you can read more details there, go find publications and check those out if you're interested. But it wasn't only  this accountability, that’s what some people like to dismiss as like a naming and shaming thing. But we really view these also as like a keyword resource for people like Tony Keyes who spoke earlier if you're managing a clinical trial portfolio, it maybe that this wasn't as on your radar as it could have been, or you're not as resourced to have the vision over your portfolio as much as you could right. You're that person for the University of Missouri, Columbia. You can click on that, and you can just get a view of all your clinical trials and their current status that you have sponsored on clinical trials.Gov when they're coming to up to do what to do. Things like that. And the impact this has had has really been positive you know what occurred. A lot of positive feedback from people in the Clinical Trial management community that they've used these trackers for that for those purposes. And then also, but also as part of attention to the issue. So here in the UK we had a Parliamentary committee give a lot of attention to  what was going on here with the trackers and the performance of UK public institutions. And what's going to show the slide, but I happen to have one ready that I just wanted to show really quickly, which is from when we first started collecting data for the EU trials tracker. This is every major non-commercial sponsor of trials in the UK and through today you can see just massive improvements due to the attention that the Parliamentary Committee gave to this and gave everyone a kick to get this going. But I will add, and to build on what Daniel was speaking on earlier. If I'm going to talk about some limitations of the trials tracker we're looking very narrowly at compliance with the laws, and only certain trials are covered by the laws. The EU's even more restrictive. It's just medicines, and the US you'll have medicines and devices and a few other things but there's lots that isn't covered and also I'm looking only in these trackers at results, appearing on registries, which is only one form of dissemination. Obviously there's lots of other ways to disseminate, this one just happens to be the one that's required by law so it's very easy to track a very easy metrics to use. But you know people like Gustav or the project that people at QUEST have worked on are doing the manual work of trying to find what this rate looks like in the broader literature. I think that's where I'm going to end it from my section, and we're going to move over to the I believe we are going to move on to the country section next.

Megan Curtin: So I'm from a health justice organization called Universities allied for essential Medicines. We work internationally, and I work with the North American Leadership Division. We work principally with universities, to ensure that there's affordable and equitable access to medicine developed there, and also within, you know, the broader international field. So I am a student at UC Berkeley, calling in from California. Thank you, Nick, for talking about trials tracker. It's been a wonderful resource for us in our campaign. So for anyone who's unfamiliar with us Law, FDAAA is the FDA Amendments Act from 2007. This law essentially required clinical trial sponsors to report results to clinical trials.Gov. And as a result of that law there's also ways to ensure enforcement from the FDA for clinical trial sponsors. So one way is through sending preliminary notices to those who are noncompliant with results reporting. Well, there's over 4,000 trials that are not compliant with the law. Only 92 have received preliminary notices of noncompliance, and this notice essentially says, please become complaints within this set of time. If they are not compliant within that amount of time, then they send a notice of noncompliance, which is 30 days in which they must submit results, otherwise the Government can levy fines so all of this has been incredibly successful at increasing compliance. It's just that this scope of enforcement has been pretty limited, and so far, the Government has never levied any fines, because this has been so successful in ensuring compliance. So we released a report in 2021 of the top 40 universities in the US who were not compliant with clinical trial results reporting, and after our report we saw that there was an increase of rates of above 80% from March 2019 to February 2021, and the total number of research institutions that were legally compliant with the law increased from 13 to 17. We've also worked congressionally with Senator Frank Pallone to help him write a letter to the FDA and NIH to ensure compliance of clinical trial results for reporting law, and most recently we have been working on a citizen petition to the FDA with Columbia Law School science health and information clinic. So this citizen petition is asking the FDA to increase its enforcement of the law to a minimum of 250 pre notices per year, and this is within the scope of over 4,000 trials being noncompliant. At the same time we'd like them to publicize a Pre-notice public dashboard. While notices of non-compliance are public there's only been 4. The Pre notice number is not available to the public. At the same time, we'd also like them to issue guidance about prioritization for trials That pose the greatest risk to human life because their guidance regarding where they enforce particular clinical trials that are non-compliant has been very unfocused. So the FDA must respond to this petition by August. We'll let you know how things pan out. At the same time we've been working on Congressional actions. We've been meeting with different health of representative offices, inquiring about increasing awareness for this issue, and also potentially writing letters in support. We've had other organizations comment on our petition docket. So anyone can submit comments until the FDA responds, and that is our campaign. 

Joerg Meerpohl: I'm director of Cochrane, Germany, and heading the Institute for Evidence and Medicine at Freiburg University. Obviously for Cochrane, both international organizations, but also in the national level access to data of clinical trials, also to public health trials is super important, because it's really the basis of what we do on in our daily routine. We need access to trial results, but also to information on trial methods. Hence Cochrane and also Cochrane Germany has quite a long history in work on the field of access to data, so that includes empirical work on just describing the problem as we’ve heard. And there's a long track record in particular, of colleagues at Cochrane internationally. We have been involved in methods development such as reporting guidelines to just improve the transparency of reporting on clinical trials, but also some advocacy and structure work. So, for example, in Germany, advocacy work over many years led to the set of the German Register for clinical trials by my predecessor. More recently, we've also been involved in some methods work on looking at publication bias and dissemination bias, qualitative research which has informed the guidance of the CERQual tool. And just lately we are trying to set up a coalition of partners in Germany with groups like the German network of evidence-based healthcare and the national HTA organization and others, reaching out with that coalition to important partners to raise awareness, as we've heard before, through the issue of non-reporting and lack of transparency and research. So that includes contact with our umbrella organization of ethic boards in Germany, with our national regulator trying to address that second tier that Daniel nicely alluded to that isn't being covered, as of now by national legislation so we really hope that we'll try to improve odds, and on that end access to trial data and improve the reporting.

Matthias Briel: I'm heading a Meta-Research centre here at the University of Basel, in Switzerland, and in Switzerland prospective registration of clinical trials is mandatory by law, but not the publication of any kind of results. So this is just recommended. So we are monitoring the registration and publication of clinical trials for several years on a national level, but also on a local level. So just around Basel, it's basically the north-western part of Switzerland. Here we see that since it became legally binding that you have to register your trial, the registration rates actually increased. So for industry, sponsored trials it is really at a high level, close to 100%. For investigator sponsored trials it is now about 90%. So there are still 10% clinical trials not registered at all and about 70%, are prospectively registered. So we also did some qualitative work and asked in a survey, but also directly approached colleagues who did not register their trial, why they did not register, and they mostly replied. More than half said that they were actually not aware of that law, and nobody really reminded them that it is mandatory by law, and nobody really reminded them that it is mandatory by law. A second point was then they just said that they did not have the time and the resources to do the registration, and finally a group of about 40% said that they would need further information or instructions or further help and support. So this made us reach out to various stakeholders that could personally provide the support so clinical trial units in Switzerland start setting up now registration support for clinical researchers. This seems to be appreciated so there are quite a lot of clinical research colleagues approaching now CTU colleagues to have their trial registered, and this includes even then regular updates of the trial on the registry so on a yearly basis. Secondly, we thought, okay the researchers need to be reminded and also informed that this is actually mandatory, and the ethics committees are happy to do this now in a systematic way, but they are still reluctant to really enforce the law. So they restrain from any penalties. Equally the university hospitals, or the universities do not think that it is relevant that they really assume any responsibility here, and do not want to enforce any registration or results publication for the trials that are run at their institutions. So we keep working together with the different stakeholders in Switzerland, the funding agency, the main public funder in Switzerland is very much interested in increasing the transparency, and also because the law has been changed in Switzerland only about 6 years ago, and it's possible then to further revise some aspects there, and we push that the publication off, or the availability of results from all clinical trials is also made mandatory by law.

Dinesh Thankur: I'm a public health activist, primarily focused on drug development and pharmaceutical regulation in India. Hearing all of you it just feels like, we are back in the dark ages because India, although it does have what's called a clinical trial registry of India, there are serious issues with that in the sense that if the first step, in order to encourage transparencies is to actually have a system in place. Until 2007 there was no mandate for the country to try and collect this information and disseminate it. India started running the clinical trial registry of India through an amendment to statute,  it's not in the law, but it's a delegated statute so enforcement is an issue. But the kind of work that gets done in India broadly falls within 2 capabilities. First are the studies that are done to develop modern medicine, whether they are indirect studies or direct studies, or clinical studies for bioequivalents where India has very large generic drug industry, that this industry conducts, and the second is the clinical studies of indigenous medicine which is a traditional Indian medicine. There are several arms of that. In both cases the quality of information and the complete list of information is highly suspect. So for the last couple of years we've been trying to collect information to at least baseline where we stand in order to make case for additional transparency, and I can share some information with you. I don't have any slides, but I can share some information with you as to what we have now been able to collect information about. What this tells us is that there is a significant need given the fact that India supplies a vast majority of generic drugs to both Europe and the United States and although these companies are seeking market optimization within Europe and in India require disclosure of these studies and approvals prior to the respected regulators in this country that doesn't necessarily often happen. We did a study between 2007 and November of 2021 and there were 37,000 studies on the Indian trial registry  number 37,389 studies that we are registered in the Indian Clinical Trial registry, and when we started looking at this data that was available there the Indian registry had a very large number of single-centred trials compared to what we see in the Clinical trials.gov’s number. So the clinicaltrials.gov has 38%, Indian equivalent has 84%. If you start looking at some of this data, just kind of share something with you 26 principal investigators were registered to more than 45 trial sites across 15 years, which is a highly suspect number that tells you about the quality of data that's in there. Average industry funding across PIs was about 82.5%, which is very high compared to clinicaltrials.gov so the information that is put in the CTRI is highly suspect. I think it's more of a checkbox exercise that pharmaceutical companies and institutions do to try and meet some form filling exercise, and the quality of information that is in the CTRI is highly suspect. Now, just to give you a sense of  the indigenous traditional medicine studies in the same time period when we looked at in between 2007 and 2021, we had 27,000 interventional studies, and if you started looking at this, a large proportion of these were single arm studies. About one quarter of that, and just again it's a highly suspect number, and we only had about 20% of these studies were blinded compared to a very high number, in 65% of modern medicine studies. So the reason I'm sharing this information with you is because we've never had a baseline information for the quality of clinical studies that are being done in India both by large pharma as a part of the studies, and also in genetic companies for market authorization in countries like Germany, UK, United States, there is a system available with the quality of information is highly suspect. So with this baseline, we are hoping to undertake more advocacy efforts to try and clean up the system make compatible and bring it up to standards of clinicaltrials.gov so I'll stop there.

Florian Naudet: I come from France and come from the west of France, from Brittany, and actually it's the west of Europe and actually, it's a wild west of clinical trial reporting. So I need to speak about France and what happened in France. So I don't have any conflict of interest with the pharmaceutical firms. I have some funding with those bodies. You say oh no he is going to talk about all of those body, those bodies who are funding him but in fact, it's very important, because those are the usual suspects, those who can do something actually in France, and I will introduce you to the French landscape. So first, we have in France a ministry for Research, and we have another ministry for health. Ministry of research is actually pushing a lot In favour of open science. So you have this initiative, which is part of the French National Open Science policy. So we have a big open science policy in France, and we have a French committee that strongly supports this policy. So actually, there is a political wish to make things transparent in France. But actually, this is from the Ministry of Research. So they have done this kind of thing. So we have also in France, a tracker-like thing, we have this kind of dashboard to measure openness and research reporting in a database, so you can see industry sponsor, we are around 70% of industries lead sponsor in France that reports their results, and for academically sponsor, it's 29%. That's why I say it was a wild west, because actually this number is quite low. There are perhaps some false negative in that, because the problems can be also that people do not report the clinical trial number of registration in their publication. So there are a little more publications that are not captured by this kind of thing, so the ministry of research helps a lot actually and helps to monitor things. But the monetary ministry of research, does not give funding to clinical research that much. Funders are the French national agency for research, but they don't really give funds to the big clinical trials. The body that gives a lot of funding to the clinical trial is a ministry of health. They give 130 million dollars for clinical research, and they don't have a specific policy, for instance, for the data sharing. They don't really care when we want to talk with them. It's very difficult for result reporting. They have a kind of policy, because they give to the hospital 10% of the total budgets if the hospital reports the trial. So that is some kind of incentives, but apparently, it's not sufficient for the hospital to move forward. We have 2 more bodies in France. So we have ANSM, which is a regulatory body, and HAS, which is a regulatory body. For ANSM they oversee clinical research and then they can enforce penalties for not complying with the law. But I'm not sure that the agency answered TranspariMED’s email, so we'll see. But I don't think they are very responsive. They could be very helpful in this. And for the HAS they cannot enforce anything for clinical trial reporting, so perhaps it's easier for them to express their interest in open science, and they did this. We were involved in this recent position of the National Agency, and they call it in a favour in transparency through register report publication that clinical trial reports are shared. So they are some supports and some initiatives that can help to support but actually, it's not sufficient. Inserm is a big institute for research, and it’s also now an initiative. I am part of this initiative, and we are going to try to push for transparency at the institutional level. But Inserm is not the leader in clinical trial? It's rather medical hospitals who are doing those clinical trials and we are going to partner with Osiris, a new European project. So Nick, for instance. So he is also involved in that. And we are going to try to see if we can increase reproducibility through initiative of transparency. So we are discussing a lot about that, and I must say that this is broader, only clinical trial and clinical research.

Discussion/Brainstorm

Emma Thompson: So after hearing all those fantastic introductions to the work that's being done, there's definitely a lot of themes going on there I think we can pick into. There's a great deal of work on quantifying the issue through dashboards, through accountability trackers, and also raising awareness of that this is an issue through those tools, and then the political level advocacy, but also thinking about the barriers and facilitators of reporting among researchers. We've heard some of that. It would be great to go into that again later on, but also, I take Daniel's very good point that we need to go beyond what we currently are doing in terms of you know what might be within the legal frameworks of the things that are easier to enforce. Seeing transparency as a broader issue, that's more complicated than just getting drug trials up in 12 months. So with that, I would like to kick off, and maybe I will start with a question. If you had one thing that you would want to achieve in the next 5 years in this area, what would it be?

Gustav Nilsonne: So the most immediate objective that we are trying to advocate for is to have someone, possibly the Ethics Review Agency, but someone should have the job following up the reporting of clinical trials so that it doesn't have to be run on a shoestring as a research project which we're doing now. Since I'm already talking, I'll use up the question a bit and suggest that once we've achieved that, the next goal that I would personally like to see would be to mobilize resources so that we can go back into the archives and dig out results for old trials that are probably sitting on shrubs here and there in our universities with researchers that have moved on to other things or other places, but where the results could still be salvaged with an effort of time and money.

Emma Thompson: Yes, speaking on behalf of Cochrane, I would say that would be very excellent for improving the work that we do, and I'm sure many would appreciate that. Going to the point about having people that are responsible specifically for this. I'd be interested to hear from people in different contexts about how that works in places outside to Sweden. Is there anywhere where there is really a dedicated resource or is this still all just overworked people with a very small budget?

Daniel Strech: Can I ask Till to respond to this with regard to what is currently going on in Great Britain? I mean, if I understand rightly, then you haven't introduced this Make it public initiative here. Right? This is, of course, what we in Germany maybe not in 5 years, but I would at least hope that within one year we might have a first response from both groups, those important stakeholder groups, the Ethics committees in Germany we have 53 of them. But also the regulator, the more legal regulator in this regard, that they will give us feedback whether they are willing in principle, to copy a little bit of what just recently happened in Great Britain with the make it public initiative, because, if I understand right, then you have it in a certain way and my interest would be to know when will we see the first results of what is currently going on in Great Britain, when will we see the database that is really listing all approved clinical trials, and then showing that 12 months after study completion, there are really automatic reminders going on to the trialist asking for results supporting not only drug trials. So can you say something on this?

Till Bruckner: Nick raised his hand and I think Claire is much closer to the process than I am, so I'll leave it to them to respond to that.

Nicholas DeVito: I'll just say first here in the UK. So we've even gotten, very recently, some much better news, even beyond just the make it public campaign, which is this idea that the HRA should be undertaking efforts to improve these sorts of things and I mean already we've seen the HRA doing these annual transparency audits and they’re promising to do more. We'll see more than are now which Claire can definitely talk more about that than I can. But the automatic registration is happening, slowly rolling out as a thing to ensure that everything that's happening in the UK gets registered as a condition of ethics approval which is supposed to already have happened to Emma’s earlier point but now it's like they're actually offering like a service to allow your stuff to get registered automatically and you don't have to worry about it. Lastly the best news we've had is now make it public is just like a campaign, like a guiding thing for the HRA to do but we now have gotten news that the incoming clinical trial legislation we  post Brexit overhaul of the UK regulatory structure is going to rule requirements for prospective registration and result reporting within 12 months and I don't know if there's things built in. They haven't talked about audit or reminders, or things like that but at the very least there do appear to be sanctions on the table, where, if you don't report, or if you fail to register report prior trials, they may hold up or not approve your future trials within compliance, so as always this sounds great, we'll see how it looks in practice. Obviously, we know as was laid out in the US context and a little bit by me in the EU contexts. Just having these things in law isn't always isn't always enough.

Emma Thompson: Absolutely. It's the enforcement in practice. Claire, did you want to add something?

Claire Veryard: Yeah, I just wanted to clarify that at the moment, again, this is just for clinical trials of investigational drugs that are being automatically fed into ISRCTN and also that this law will apply to so we will have to get similar things in place for the non-CTIMPS.

Emma Thompson: Devices probably not included as well, right?

Claire Veryard: At the moment combined drug and device trials are, yeah.

Nicholas DeVito: But not devices alone. Devices are a Wild West situation.

Emma Thompson: Yeah, I was looking for a missing device trial early on and yeah, the Wild West definitely applies.

Florian Naudet: I think that's we should think about that, because each time there's a rebel it would become a target. People start to gain about it, and I had some experience, so I would say that in France we don't have a lot of support to make the better public. So, for instance, in hospital we are only starting to try to communicate trial results as they should be, and we have started to benchmark a little bit before, by contacting hospitals in France and also in Germany who behaved very well on certain indicators. So it was very fun, because those hospitals we contacted in France. So we asked them why they are better than other hospitals. And actually they said we were better because they had an audit because of very bad study, and then they were audited by the health authority and the health authority just said that? It would be better if you would be better on this indicator. So they started to be able, just because they just had a lot of fear. But this does not happen, for the other centred, so the other centres don't see anything, so they won't do anything. And they started to behave like that because of that. And the health authority, asked them to be better on the European reporting system, but not on Clinical trial.gov. So actually, they are very bad at clinicaltrial.gov and very good at our CTR, so it's very interesting to look at that. And also when we contacted another centre, they said, oh so we had a lot of pressure to be good on this indicator, and in order to achieve this, we started to communicate about the results about the trial that's where suspended, because it's very simple. You just have to write suspended and then it's okay. It's considered as reported. But actually what we need is a results of trail that actually completed and have some results, and sometimes it was secondary. And it's quite a bit more difficult. You need more resources and more things like that. That's why I say, it's quite complex.

Emma Thompson: Certainly thanks for sharing those insights into those different conversations you had Florian.

David Culquhoun: I'm a rather older physicist come statistician. Yes, I'm interested in devices because I don't know much about this, but I have the impression that the NHS would welcome a lot of phone applications which will probably not be much good for anything but that would be sold as a great price to the NHS. Can someone cast light on how these things are actually tested?

Nicholas DeVito: I have second hand experience very briefly, through a colleague of mine, who if she were here, she could answer this question much more detailed, but I think that the very broad hand wavy answer to your question, David, is not much is going on there, and it's not done particularly well and there's a big question over like whether these should we be treating them as medical devices, should we be holding them to that standard of evidence. Are they something different? Should they be using a different standard of evidence? And you have all these things. You know you have these apps that do help things whether they're for mental health, awareness, or registration tracking. Things like that that that are used for these purposes. But they're not vetted. They make claims that are unsubstantiated, and the research backing those claimed is usually insufficient. That's my go but I don't know that anyone's done a full scale systematic audit of the area. But it's something that someone probably should do and people I know have thought about doing.

Emma Thompson: And I see that Claire mentioned in the chat. There are some studies in the ISRCTN. Maybe this is another frontier to add to the list.

Daniel Strech: I just wanted to come back to your very initial question, what is my personal priority for the next 5 years, and just wanted to come back again to that thought that we shouldn't just focus as a community on clinical trials of drugs or medical devices? I think as we've seen with the Covid pandemic, there are so many interventions that are important for health and health care, non-pharmaceutical interventions, public health interventions. So we really need transparency on a much broader scale than just clinical trial development. And that also implies from my point of view, that's another important issue, I think, to consider not just the results or results reporting. It's really also the methods of the trials. The information how they were conducted, statistical analysis, plans, and so on, and so forth, because otherwise we might have the results eventually, but we can't interpret them and that's not going to help us for decision making and that's what the ultimate goal is. So that is just a plea to the community from my end.

Maia Salholz Hillel: I want to piggyback on what you were saying Joerg about the importance to look beyond results supporting and methods and also highlight the importance of registry data quality which, thank you. We heard a lot about CTRI From Dinesh and the struggles there but work that Nick and I are wrapping up on Covid trials just really show us how poor registry data quality makes it hard to evaluate the rest of our questions. I mean, it's hard to know if a trial reported on time, if we don't know whether it was suspended, terminated, completed, ongoing, or we don't know when it was completed. And so I also think we need to come back to the fact that this is a holistic, this is a system, and we can't evaluate one part of the system if we don't consider the other part of the system. So yeah, it's a constant conversation between the different stakeholders involved.

Emma Thompson: I heard people talking about, and I want to come back to Claire's point about appealing to the social issues here, and kind of behavioural science I guess is what you call it, and how we overcome the barriers. People have done qualitative research. I'm wondering if people would want to talk about some of those insights that they've had from doing those studies. A few people mentioned that they've done some qualitative research with researchers about this.

Till Bruckner: I've done a very small study recently and I was looking at clinical trials completed by the 10 largest non-profit sponsors in the UK In 2017, before the Parliamentary Committee raised the profile of the issue, and I was looking at nondrug trials. And I looked through about a hundred 150 trials. Found out that about 20% haven't reported that results or 30%. I went back to the institutions one by one, and said, You know sorry you appear to have overlooked something there, and every single one of those 10 institutions said, we are committed to transparency, and we are going to follow up on this. So I think that goes back to what this stuff said with he'd like to dig out the results of those older trials, so the next plan is to really scale this up in the UK and systematically go through older trials, going back maybe 5 or 10 years, and just really clear up the old mess. But here in the UK we've got an incredible amount of goodwill from institutions, from researchers, from regulators. That might not be the case in all countries, but I think generally, if you tell people, look this trial of yours 6 years ago, you didn't report the result. But here's an easy way to do. It. There's a quick way to do it. Please just put the results up on a registry. I think the vast majority of researchers will do that. I think from an institutional level there's a question at some point. Couldn't there be a big funder like welcome trust, they could put out a prize saying any clinical trial that you haven't made public that's more than 4 years old. You make the report. You make the results public now; you get a $4,000 grant for your research team. I think that's the incentive that we could be thinking about. And that would be a very cost effective way of adding to the store of medical knowledge really, but I'd like to know what other people think about that.

Anthony Keyes: It's something we've noticed here in the US side. There's a lot of good wills as Till mentioned, but people just lack the knowledge, and that's why I was really excited to work with a report card. Being over there with Delwen and everybody there because it says, yeah, this is the issue, but this is how you fix it, and I think that's some low hanging fruit there. People want to do it they just don't know how to do it. I think just by a simple thing like that we could really make some changes.

Emma Thompson: Yeah, agreed. And I liked the kind of the positive framing of it within that. There are steps you can take, and here they are I'm going to come next to Daniel.

Daniel Strech: I think we can follow up 2 stakeholder groups. And just really asking them at least, would you be willing to do more if we support you in any way that is possible? So yes, we do it with the individual researchers via report cards, but we still have to see whether how much they are willing, here at least in Germany, to then really use the support tools, even if we have very easy hands-on support like what you are saying such as, look if you have a paper but you haven't linked it to the registry so nobody might find it at least those not to that one on the registry. Here's a link that explains to you how to link your publication to the registry in less than 5 minutes. We'll see how many then really do it, even for this very low cost time efforts. But I would say, it's another research. It's not necessarily qualitative research where we ask the researchers, what are the reasons? I also think we have identified most of the reasons, lack of time, lack of knowledge. So the next step would be to the to better investigate what really helped implementing support measures. And where are the limitations that, even with very good support measures, people then don't follow up and don't improve their practices. And the second group is the very high level decision makers. Now in Germany, based on this NICE use case in UK, where something at least happened that the Ethics Committees, together with other parties, have to do something that might result into a much broader coverage of studies, and then also results application. We are now contacting, together with several people here in this group. Joerg and Till and Stephanie just joined us here. We are currently inviting the umbrella organizations of the Ethics committees in Germany, and also the regulators and the German physician associations for telephone calls and interviews where we will suggest doing something similar to that what is currently starting to happen in Great Britain, and at least ask very direct questions. What can we do so that maybe together with you, we go into this direction to improve these things. But it's still, I would say, trying to better understand where all the reasons are that they finally don't do it, and it would be great to see this in other countries as well.

Emma Thompson: Would love to hear the outcomes of that work that you just outlined. So maybe in a follow up session sometime.

Megan Curtin: Yeah, echoing other people’s conversation about the goodwill to increase trial reporting when clinical trial sponsors are aware that they're noncompliant has also been something that we've seen in terms of our university report and the other thing that I think is really effective, is having Freedom of Information Act requests for your request. Our legal team at Chic, Columbia Law School has done a lot of FOIA requests where we've been able to look more into the FDA and NIH’s enforcement of clinical trial oversight law, and you know we have seen some improvements but we've seen that there is an automated system when a clinical trial sponsor is noncompliant and it doesn't indicate within that notification what exactly they've been missing in their reporting and so there's quite a bit of confusion we've seen about what they need to amend within that system. Many trial sponsors think they can just upload their paper, so like other people are saying, just the quality of the reporting is an issue in part because of the message that's been generated by clinical trials.gov, but there has been more guidelines released about how to become compliant. So, yeah, FOIA is one great way and then you can also use that information and to create publicity.

Emma Thompson: FOIA is a great tool for advocacy.

Stephanie Müller-Ohlraun: I'm wondering, but I do this actually in all kinds of meetings, where we talk about the subject of transparency, about the estimation of our colleagues and ask ourselves to about the willingness and interest of investigators to comply. I personally, think that we are overestimating this in a maximum. I think when we ask investigators, they just answer like we expect them to answer in a social, desirable way. Nobody would say well, I don't care about patients. I don't care about resources. I don't care about ethics. But behind the doors I mean why don't they do it? Because I mean why don't they do it? Because they just don’t, they do it, because they just don't care. I don't think that the question of having a lack of resources is the only one, so I'm always thinking that we are actually overestimating their willingness and their interest in complying to these regulations, because at least I think there are different motivations, to run a clinical trial. They can be, of course, like purely scientifically driven motivations. But we all know that there are also other kind of motivations for running clinical trials and that's why I think that we are in general overestimating their interest and their willingness, and their compliance. So this is something which I would be interested in when we're talking about qualitative research. I think we're just not estimating this correctly. And that's why, for this pessimistic reason, I say that incentivizing would probably not suffice, at least not in the German context. I think we really need de-incentivizing. I think we need funders to blacklist PIs who don't publish for further fundings. I think they need IRBs to disapprove second, third, and fourth trials of trialists who haven't complied to transparency guidelines. I think we need publishers who have to disapprove publishing if they don't have a link to the to the registry and so forth. I am sure we need much more de-incentivizing and incentivizing. Just to be compliant will not be enough.

Emma Thompson: It kind of links with what Claire was saying about appealing to the right motivations that people have on a social level. I think Nick put some references to some research in the chats that might be worth looking into. Going at them with the stick rather than the carrot perhaps.

Balendra Pratap Singh: I just listened to the presentation of Dinesh, and I must appreciate that we should publish the paper if we register in the Clinical trial registry of India. As far as my experience is concerned, and my suggestion is that can we create a flag or something in front of clinical trial, or maybe in front of the title that author has completed the project or not, the clinical trial or not. Or they have partially completed, or they have published the paper something like this. So if anybody just sees the title or sees the registration number can simply visualize that with the project was completed, published, or not published, or something like this, this will encourage more and more people to finish the trial and then publish the paper. This is one of the very small suggestions.

Emma Thompson: Does anyone else have experience of those kind of flagging tools or as a technique? Is that something that's already existing?

Claire Veryard: On the ISRCTN registry,  we have badges for reporting results. Any kind of results, you know adding protocol, adding a statistical analysis plan, and sharing data. And we also have a badge for regular updating while the study is ongoing. But that doesn't show kind of outside. I don't know whether Balendra is thinking about something that's visible externally outside of the registry that kind of demonstrates that but yes.

Emma Thompson: And we've also got someone in the chat saying that Sri Lanka have something, but not sure what, so interesting thing to dig into. Thank you for your idea, Balendra. Dinesh, would you like to say your question or your thoughts?

Dinesh Thankur: No, the only point I was trying to make was, one of the unique challenges that we have in India is a publication of studied results in what are called predatory journals. Pay per view journals.  This is something that I don't think I've seen elsewhere except in India, specifically around studies that are either poorly designed, poorly executed, and I was just curious if anybody's actually dealt with something of this nature, because publication itself doesn't mean anything. If you're basically making up data, which is what our studies show in the way that studies are at least ready to the CTRI have been designed or conducted. That was the first point. The second point was I think that that one of the things that we really have to think about is that there have been regulatory actions specifically in Europe, where large number of generic drugs that would that were approved by the European medicine agency were withdrawn because of issues with clinical studies. Now this is far beyond transparent. There's something that we really do need to keep in mind. Those are 3 points.

Delwen Franzen: Just a reflection I was having. A lot of us representative from organizations here and I think we talk about the role of different stakeholders so I'm just wondering what is the most useful thing that a given stakeholder can do. So a research institution, for example, can help with incentives? Or registries can maybe help make the whole process easy. It's just an open question to the group If someone wants to respond to this like. I think it might be helpful to think along those lines. To see how we can really work together as effectively as possible, given that we all have our limited resources, and a lot of these things take time. So that's kind of my question to the group of just like what each of us can do given our context and the organization that we're based at.

Emma Thompson: So what do you think is the most important thing that different stakeholders can do?

Balendra Pratap Singh: One of my suggestions is that for the different countries there may be different rule. But can we create a mentor for these things, for some institutions or some geographical areas? Maybe 2 or 4 people as a mentor for the clinical trial, so they can actually guide from the beginning till the end, so that if people do not have a solution that they want answers, so these mentors actually help them to do the clinic trial in a better way, and they can encourage publication also. So this is one of the ways we can actually encourage more and more people to publish.

Claire Veryard: I think that's good idea. That's very human response to it. To provide people with an inspiring example, somebody that they want to be like, yeah. So as long as those people obviously being as transparent as possible, then that should really work.

Florian Naudet: Just a simple thing for all that. It won't serve the problem, but it might help a little bit. I would like to join arms to adopt a registered report format. So registered reports are this new form of publication in which you submit your protocol, and it's Peer reviewed, based on the importance of the research question and those quality of the methodology to answer this question, and you receive an acceptance a priori based on that, and then you can start your research conduct your research. And then when you get the results, a paper will be published, whatever the result is so this is quite, I think, important.

Emma Thompson: It might help to appeal to those motivations that people have that are more about that kind of personal academic. Who will get publication at the end of it.

Till Bruckner: Deborah Zarin suggested something a few years ago in a paper and I'm surprised that more people haven't picked it up, and she said clinical trial results belong onto the clinical trial registry where it was registered so when people afterwards publish in journals why can't they just copy and paste essentially the results table from the registry into the journal paper and all else is just commentary that they have to write. The results are already there. So just write the commentary, and I think that could save a lot of time as well.

Nicholas DeVito: Less focusing on what any given stakeholders can do, and more that I think one thing that we can do for all stakeholders is to make sure that and this you know particularly people who we are talking to loads and loads of stakeholders from all different places like if Daniel is trying to bring together these people or Joerg or someone like Till is you can't let people pass the buck. It’s diffused amongst everyone's responsibility that everyone can say it's someone else's responsibility. It's the editors of the Journal's responsibilities. It's the funders responsibility, government responsibilities. It's the individual institutions responsibilities, and all of those will point to all the other ones and say they're the one with the real power here, when truly they all have some modicum of power, like some handle on the levers of power to make this happen. So you don't let when you're having these discussions, don't let people pass the buck, say, yes, we realized that it would be great if our national legislator passed a law that said it was required to do this, but in the absence of that, what can you do in the meantime. What can this institution do in the meantime, what can the editor of this journal we're talking to do in the meantime and commit to and find those areas that people can commit to no matter what they are and people are raising many of the ideas here rather than let them pass.

Maia Salholz Hillel: Yeah, I just went on this not passing the buck thing. I also want to say within this group, I think a lot of us have had conversations with each other, but a lot of us have different power and different levers and different roles that we're playing in this. So I think it's great to see how we have been able to share things. Gustav and Katherine, really taking spits and parts of methods that we've used here in Germany or on Covid work and pulling that aside into another scale. Or how can tools that we've developed then be serving registries or other folks in this room? And how can folks who are involved in advocacy? How can Megan use the work that we're doing to fuel your next FOIA request, and I think really asking each other, because at least we know we have enthusiasm and desire but we also all have limited resources we're always facing. So how can we support each other? Open invitation to reach out to if there's anything concretely that I said today that could support you, because I mean we have limited resources, but we have things to make other things useful to others.

Megan Curtin: I think in terms of supporting everyone on the call. There's never one straight initiative where people can send letters of support or comment such as the FDA Citizen Petition we filed. Different organizations are welcome to submit comment and support, and I think you know UAEM is more than happy to support other organizations in this area of clinical trial transparency, and I also agree with the sentiment that we need this multifaceted approach, where you know you have the resources where you can talk to universities, and also discuss oversight in the government. For example, we've been working with the Energy and Commerce Committee who oversees the FDA. So if you're able to identify agencies that have power over each other, and to try and reach out to a broad range of stakeholders then you're able to spread your resources and potentially see if the people in government aren't able to collaborate so those are some ideas and so just increasing enforcement. I think it's wonderful to have incentive, but also to have penalties and to have compulsory compliance is something we need to improve.