Sergio Sismondo: I'm going to talk today just about key opinion leaders and I'm going to talk about key opinion leaders in the abstract. For those who care about the history of the opinion leader concept, I think the standard idea of the discovery of the opinion leaders was by Paul Lazarsfeld in 1940. Lazarsfeld was working at that point in the United States and doing research on elections and how influenced risk was passed along in elections, and he was working shortly thereafter at an organization that became part of Columbia University that was the Bureau of Applied social research. And there he had a group of students, and they basically funded a lot of the work by doing applied work for industry of one kind or another.

And the pharmaceutical industry was not well represented for a bit. But then, in the late 1950s, Pfizer commissioned the study through the Bureau of Applied Social Research, and Lazarsfeld’s student Elihu Katz recalls that Pfizer wanted to know about the effectiveness of its ad campaign on doctors’ decision-making, and then Katz responded well. Let me tell you about this interesting research we have on opinion leaders. This is because Pfizer was spending a huge amount of money on its ads, and for those who have read the book on the Sackler's Empire of pain, it was Arthur Sackler's ad for tetracycline that was at issue in this particular study. But the study was really focused on opinion leaders.

So, Katz developed a study that used records from pharmacies, and this was novel at the time, although it's now become very standard in some areas to identify who was making the first, when people were making first prescriptions of tetracycline and then mapped social network to doctors through interviews. And the result was something like that there was a chain reaction for doctors who were well-integrated and then there's a certain amount of evidence that this was driven by the opinion leaders. I'll come back to this at the very end of the talk. So, this is an idealized model of what the opinion leader does. The opinion leader is somebody who is respected in the field and has an asymmetric influence on doctors around them, but they're all opinion leaders exist in all fields. So, Pfizer appreciated the research so much that they asked the Bureau of Applied Social research not to publish it, and when it was published, it was published with a pseudonym for the drug and they called it Gammanym, instead of tetracycline.

But it's unclear to me exactly what happened once this information was became part of Pfizer corporate structure. But then we start seeing over the following 20 or 30 years, different companies making use of the opinion leader idea and using it in some interesting and novel ways, not necessarily anticipated by Lazarsfeld and Katz, and others.

So, I'm going to talk about the pharmaceutical industries, key opinion leaders, or the industry, which always needs to aggrandize everything. I went from the opinion leader to the key opinion leader, and so that became standard within the industry where the KOL sometimes is thought leader, but more generally KOL is the standard term.

Sometimes the industry is very blunt about exactly what it is. So, in this ad, what is in this article, Watermeadow, which is a medical education communication company, writes that KOLs are a convenient shorthand for those people usually eminent physicians, who we co-opt into our development and marketing strategies. I use a combination of data from what I think of as the gray literature. This is the literature which internal newsletters for the pharmaceutical industry. But I also attend conferences and workshops that are also intended for people working in the pharmaceutical industry or working closely outside for agencies that might be employed by pharmaceutical companies.

So, I'm going to talk about 2 different kinds of KOLs. There are lots of distinctions that roughly map onto this. I've just simplified them to two, Local KOLs and Research KOLs.

So, the Local KOLs, this is where the bulk of the industries’ budget for KOLs get spent. Local KOLs give what are in the United States called promotional talks. And the promotional talks are in a sense highly regulated by the companies especially, but also to some extent by the FDA. So, we have people who say very clearly here like the sales representative writes I would give them the information that I wanted them to talk about. I would give them the slides. They would go through specific training programs on what to say, what not to say, how to answer specific questions, etc.

Then there are these training programs. We can see an example in this little bit snippet from an ad by Wave Healthcare. It's vital that advocates are able to communicate and influence colleagues with clarity and conviction to ensure speakers are at the top of the game. We have developed a communication skills program for clinicians. So, these local KOLs are not necessarily pre-existing opinion leaders. These Local KOLs are people who are in some sense recruited to become speakers, giving these pre-written talks and communicating them to others.

They're told that they cannot deviate from the script, as it were. So here we have Dr. Khan in this from an interview. These are pseudonyms. By the way, in the book I would give pseudonyms with names, starting with K to all the key opinion leaders. So, this is interview data. But I generally use pseudonyms for people I encounter at these workshops and conferences too, even if they are in a sense speaking on the record. So, Dr. Khan says when you're out there doing a talk you really have to follow those rules to a T. If you don't follow those rules, then you're at risk of breaking procedure, and I mean arguably, I guess you're at risk of breaking the law. And this is something that these local KOLs are told repeatedly, and this is in interviews with them. This comes through very clearly that they are told me must stick to the script. Of course, we can see advantages for the pharmaceutical industry beyond regulatory advantages. We can see in advantages for these speakers to be sticking to the script. The regulatory advantages are simple, the regulatory advantages are that there shouldn't be too much overt off-label prescription. But the company advantages are probably, in a sense, the more obvious ones. If we know that Dr. Khan and his colleagues are sticking to the script, then we know that they are doing the right promotional work.

Here we have a similar quote. I won't read this out, but Dr. King says something similar again in an interview, so he says, I have to use the slide deck that they provide me. I'm not allowed to alter it in any way. And this allows a consistency of message that the talks include a lot of the key words that the company might want to accompany, the product, and so on and so forth.

So, these Local KOLs who might not have been existing opinion leaders in Paul Lazarsfel’s sense become more prominent. They may give as many as a hundred talks in a year, and the sales reps and medical science liaisons, perhaps give them audiences. The sales reps recruit audiences, typically of other physicians, sometimes of other prescribers, and as a result the KOL’s careers are built.

So let me turn from the Local KOLs to Researcher KOLs. Researcher KOLs are the kind of people who are somewhat less controlled by the industry, and they are often asked to give talks not just to after dinner talks to physicians, or clinic talks in clinics or hospitals, or something like that, but maybe also at conferences. These are people who are presumed to have some presence in the research literature on their own, or at least appearing to be independent from the pharmaceutical companies. So, we have people that had a workshop on the KOL management. I'll come back to that briefly. But the workshop on KOL Management is the standard term that the people in the industry hate it, but they seem to need it somehow. For example, there was an invited KOL who was asked to talk about what it was like to give to be a KOL. And this is a person I’ll call Dr. Kessel. And Dr. Kessel started off as a local KOL giving promotional talks, speaking from those companies slide sets, but then he became established as a researcher perhaps with the help of pharmaceutical companies for which he was working. And so, he came to wear a hat not as a well-connected and thoughtful local physician, presenting other people's data, but as a research scientist presenting his own. So Local KOLs can graduate to become Researcher KOLs.

So, Dr. Kessel says at the workshop I attended, in the past decade, I've been a consultant to the manufacturer of every compound that has been developed for the treatment of depression or the treatment of bipolar disorder, and some number of other compounds that haven't made it through the multi phases stages of development. So, you can see that this is a prominent person. And indeed, Dr. Kessel is a very prominent person. Again, this is a pseudonym, so don't go looking for Dr. Kessel, please.

And then he becomes this nationally and internationally known figure. He can engage in all sorts of activities. He might, for example, work with the FDA in terms of regulatory reviews, and perhaps other regulators around the world. And he has served as the author on many publications that were developed by pharmaceutical companies. This is standard. I was just doing some research with a colleague on the opioid manufacturer Mallinckrodt, and we can really see how the development of KOLs by the firm who are then the very first people to whom the firm goes when they have a ghost-written manuscript, or a project that is going to result in a ghost written or ghost managed manuscript. They bring on board some of the people that they've been developing and recruiting over the years as KOLs. So, this is a fairly standard path.

KOL management, as I said, this is a standard term for the interaction between the companies and the KOLs, and so you have other companies’ agency. I'll call them agencies, not pharmaceutical companies that develop tools for KOL management, tools for the identification of potential KOLs, tools for the training and the introduction of those KOLs to the products and questions, and so on and so forth. So, in the end, this is a bit of a joke here I've put in, but as we can see from this image, from an advertisement, from the point of view of a pharmaceutical company, the different departments that they have dwarf KOLs, and so the KOLs can be expected to engage in the sorts of activities that the companies want them to engage in.

How are KOLs recruited? Well, the entry point is often the advisory board and advisory boards take all sorts of different shapes and have different kinds of relevance to the pharmaceutical companies. Sometimes they genuinely seek advice, but that's probably a minority of cases. Most of the advisory boards are about the development of KOLs rather than the development of products independently of all this marketing. You can see in complaints by people who are engaged in the work of advisory boards that there might be 20 parallel advisory boards, and all the information collected from these KOLs, all the information collected from these physicians just goes in the bottom of some file cabinet and never gets seen again. So advisory board might be sometimes collecting advice, but often they're about communication to the KOLs that they are valuable and communication about the particular products. And again, in this recent project on Mallinckrodt we see that very clearly. We see some advisory boards come out with no detailed notes and plans for different manuscripts, and so on and so forth. But I think that's the exception and other advisory boards are occasions to gather together KOLs and pay them essentially for them to bring their families to on a nice little vacation. But their way of communicating to these KOLs is that the company finds them valuable. So that's recruitment.

But then, once you've recruited, then you can have a continuing demonstration of the value of the KOLs. David Healy spoke to this group not that long ago. He said it’s a glittering social thing. They hire the Opera House in Copenhagen for a meal, etc. And he said his wife was upset that he dropped out of the KOL business.

And then there's just the basic academic thing. So, Dr. Katz again, this is a quote from another workshop. He says when you're being asked to be the thought Leader, that's a bull's-eye exactly where academics live. They want to be thought leaders. This is his explaining why he was participating with the companies because they were offering him an opportunity to be thought leader.

So, these researcher KOLs are paid to speak, first continuing medical educations, but also specialists at conferences. Sometimes they help with regulatory approval, and then their authors on a bunch of on medical papers, to which they may have contributed a greater or lesser extent. And I'm not going to talk about ghost written or ghost managed papers. I've done a lot of research on that in the past, and if we want to talk about that in the question period, I'll be happy to talk about that. But I'm going to leave it aside from here.

So again, in all of this, the KOLs are managed. So back to Dr. Kessel that physician researcher who graduated from being a Local KOL to a Researcher KOL. He describes in his talk how serving as an expert witness in a court case, he had seen as part of a plaintiff's efforts to discredit him, the quote individual management plan for him by the company being sued. And the individual management plan, he reports, says something like, so-and-so will meet with him on such a date with this expected result, and then we'll invite him to do this. And so, he said, this is really embarrassing to come up in a court case. The pharmaceutical companies shouldn't be talking in these terms. But they find it very difficult not to talk in these terms. Because this is indeed what they expect from him, and he says that he's not naive. He knows that they're expecting these things, but he wishes that they were a little bit more respect of his expertise, etc.

These KOLs are sometimes criticized by other physicians, and they are called paid stooges or something like that, or even “drug whores” one KOL says. When I did interviews with a group of very highly paid KOLs, my focus was actually on how they were thinking about and justifying their work, and they think of it in terms of if they believe what they are saying, then they can always justify their work in terms of communicating scientific or clinical truths, and as long as they maintain their integrity, and they can say that they believe the truths. But then they are comfortable in their position if working closely with the pharmaceutical industry being paid by the industry for engaging in these activities. But we can ask, of course, which truths, and I also think, perhaps more important than just which truths.

Which KOLs become promoted in this way? KOLs do end up with this asymmetric influence because they are given platforms and audiences, and what they communicate may very well be backed up by data. But the KOLs are chosen in terms of how they can communicate these things, that the industry wants them to communicate, and then the audiences are presumably going to value, what gets communicated. So that integrity that they might have in virtue of communicating the truths as they see them, is part of a system in which certain kinds of data get amplified, and other kinds of data do not get amplified. So, we can see this as part of a system for shaping the things that doctors know, the things that count as medical knowledge altogether.

As a result, the pharmaceutical industry has this asymmetric influence on opinions. This is really different from the model which Paul Lazarsfeld started in 1940, where he thought he was thinking in terms of pre-existing influential people. Here we have a system in which the pharmaceutical industry can make people influential and can create this asymmetric influence on opinions in a very widespread way in terms of the Local KOLs and when a project that of promoting a new drug might involve a thousand people giving talks for a short while, and that is deemed to be worth it for the pharmaceutical industry. Incidentally, if you question this, there is actually a small amount of research available showing that KOLs might be more expensive than hiring sales representatives to do the same job. KOLs have a higher level of influence on their colleagues in terms of the number of dollars of prescriptions, or the monetary value of prescriptions that are made. So, it can be worth the pharmaceutical industry's paying fair number of KOLs to communicate what they want to communicate.

And I think of this as resulting in an epistemic corruption of medicine. The industry has taken a set of forms of communication. The talk to scientific conferences, the talk that added to them and grafted onto medicine, a separate set of activities, but or similar sets of activities that sometimes-become part of medicine, like the clinic talk, like the after-dinner talk. These become an accepted part of medicine. So, I think of this as in a sense, as I say, grafted onto the activities, and are standard ways of communication that are accepted within medicine and sometimes overtaking all other forms of communication of information. The KOLs in their capacity as authors of ghost managed publication of information, if there you can see another way, in which the industry is taking a form of communication. The research paper published in medical journals and is making that a form is a useful way of communicating the specific results that companies might want to communicate about their products.

So, on this close, I want to suggest that the systems that by which medical knowledge is most often communicated can lose integrity be through this process when you have interested parties. The companies are deeply invested in certain kinds of data, and they're deeply invested in the data that support their products. And as a result, the systems of medical communication as a whole can lose integrity, and therefore we can see loss of trust in the knowledge that gets communicated. So, I will stop there. Thank you very much, and I look forward to the discussion.

Leeza Osipenko: I think there were quite a few changes in the recent years. A lot of regulation came into the industry. What kind of gifts are allowed, what kind of gifts are not allowed, as you just quoted, no more trips to the golf courses, has this changed the outcomes, or it's simply switch from one means of attracting KOLs to other means or using other technologies? As the influence diminished as it increased has to stay the same, even though the certain things have been forbidden so to speak.

Sergio Sismondo: Yeah, I don't think I have seen any evidence of the influence diminished. It does take different forms. Now, one can pay an honorarium to a speaker that is just a little bit larger, and the Opera House in Copenhagen doesn't have to be engaged. I also heard that there are even hotels that have dropped the number of stars that they claim because of the regulations about where people can be put up for events. And so, if the hotel can say, well, we're no longer a 5-Star Hotel or something like that. We're only a 3-Star Hotel, but that means that they become an acceptable venue for pharmaceutical industry events. So, I think that there are ways around the regulations, as there always are. I was introduced to a statement by the scholar, Jerry Ravetz, who said, every system comes with the racket. Basically, every time that you create a regulation, people find new ways of creating and evading the regulation. But I think, more importantly, the sets of activities that are going on haven't changed appreciably as long as the industry uses KOLs to communicate. It's preferred data. It's preferred results. It doesn't really matter that. For at least that activity, the gifts have diminished.

Mark – Finasterideinfo.org: So, I've been researching a particular drug that goes back to the 90s and I'm looking at this strafing of the literature both in the United States and in Europe. In dermatology journals, I noticed that there are disclosures of conflicts of interest in the American journals, and I see that the disclosures in the European dermatology journals seem like there won't be any. And I'm wondering is there a difference in disclosure policies in journals in Europe and in the US or North America?

Sergio Sismondo: Interesting question or interesting observation. I didn't think that there were different policies, but I'm sure you're right that you're seeing a pattern. So, I wish I could answer that properly, but I I honestly didn't think that there were different policies, at least across the board.

One thing that this may or may not be relevant to the material that you're looking at. I think that if something is an article reporting on a clinical study of some kind, let's say in RCT, but not necessarily even a RCT. Then there tends to be a fairly generous acknowledgement of conflicts, but also other contributors who are not necessarily authors, and I've seen that both in American and European journals. But then, when the contribution is something more like a letter or a narrative review, or something like that. Then the contributions of non-authors and typically, these are offered by a single or two people as opposed to the clinical studies that are authored by six or eight people. Then the disclosures are much more limited in those narrative reviews or letters, or something like that even if one finds out later on that they were written within the company or developed within the company.

Mark: I find that now, in the more recent trials like there was a dermatology trial in the New England Journal or Jama, the conflict-of-interest statement was so long that it was actually longer than some articles. It was like 1,800 words, and so in some sense, I think, maybe they're more explicit and thorough. But in another way, things haven't changed.

Sergio Sismondo: All right. Yeah. If every one of the authors has had connections to 5 different companies, and then you list all of that and yet then you find yourself swamped with information. You have nothing at the end.

Leeza Osipenko: Thank you, Mark. This is a very interesting observation how a particular policy or new rule introduction doesn't actually change the practice or reality in terms of sunshine act. And you can look up any clinician, and how much money they received from the industry, and you can see this long list of conflict-of-interest reporting in the journal. Everyone is very transparent. Everyone is just saying, yes, yes, I've done that. I've taken this money. Business as usual. Nothing changes, and very interesting how particular enforcement of transparency does not really bring results that one would have expected.

Michael: I got interested in tapering medications, because I had hard time with this, and I couldn't find a lot of information. So, I couldn't get good information from a lot of research as you do. And you read like 20 papers, and I finally figured out those steps to get off it more gently, but I immediately noticed all these disclosures, just savings that you're talking about as an engineer. When I see that, I read it with very high mass suspicion, and I assume it's alloyed by commercial interests, and it's not common, like all science. There's value in all science papers. But it's not common for me not to find any papers that aren't alloyed by obvious conflict of interest, or at least the credible appearance of a possible conflict of interest, and there was nothing that I thought was reliable enough to go on and as I went deeper into it like I'm right down in the rabbit hole now, and I attended a conference at which they were describing how drugs are homologated or approved, and they just look at published papers and they make bar graphs of effect sizes and the effect sizes are not that great honestly, and they're hyper focused on the statistics of the width of the bars of the effect size and it's very strange the way these drugs are approved, and they are talking now about real world evidence. So, if the evidence is to homologate or approve a drug, it seems quite weak. The real-world evidence to remove from the market why should there be a different standard and if there's 10,000 patients saying that there are problems, why should the standard be different, and I think it should be just as easy to remove it from the market like the bar should be equally low. So, I just wondered if you have any comments about that. I'm of course, like immersing this community. That's quite angry. But I trying to figure out what is going on. There was a lot, but I wondered if you have any comments on it.

Sergio Sismondo: There is a lot. I think that we are in an era where there is enough profit to be made from especially prescription pharmaceuticals, that the industry can engage with as many researchers as it wants to, or companies can engage with as many researchers as they want to. And this is why, when you are looking at the literature, you're not seeing too many independent researchers, or you're not seeing enough independent researchers. This is why I introduced the phrase epistemic corruption because I think that once you have a mass of this interested research, then, in a sense, the whole of medicine is no longer trustworthy. It isn't easy enough to find the independent research, and frankly, medical research is tricky at the best of times, and it's difficult to find a really clear results in most areas, whether it's about drugs or not about drugs. So, we're in a difficult terrain, and then you have an industry that can make a lot of money on this terrain, and that we depend upon scientific information for our evidence about drugs. This is the way that the whole system is set up. It no longer just circulates as doctors with their opinions because we expect for strong scientific evidence, for whatever it is that we might want to do or not to. As for the issue about regulation, I would go stronger. I think that we should probably have something like a 10-year moratorium on new drugs right now, while we just sort things out. But that's a radical position, for the doctors in the group, I think the one should generally not prescribe anything that has been approved in the past 10 or 15 years, and you'd be wise to just go back on the tried and true or at least the things where you know the effects and the adverse events. But that's just a view on which I have.

Francois Maignen: My question is slightly different. You have focused mostly on the clinicians, but do you have any evidence if you don't start the industry shifting the choice of European leaders towards patients to try to have patience advocates, to request products, or to do some promotion on products?

Sergio Sismondo: Absolutely! Thank you very much for bringing that. That's just another facet of the marketing activity. And I really think that standard now, and it's standard especially for very expensive treatments, treatments for rare diseases and things like that, where patient advocates can be easily found and then developed and recognized. And again, the process can work very much like the key opinion leader development process. So, I attended a meeting where there was a sub-portion of the meeting that was devoted to new patient advocates, and then it was a set of seminars for these new patient advocates about effective patient advocacy, and there was even a prize given to a patient advocate for their work on behalf of one or another drug. I don't remember what it was, but you see this a lot with cancer treatments and other things like that, where the drugs are extremely expensive, and often the research, especially as the conditions become rarer, shows less and less obvious benefits. So then, you need patient advocate to be saying, well, even though the research is showing only marginal benefits, we need to have this opportunity to choose this product. Because this situation is so dire and so everything that you suggested I have done some research on it, and I just didn't present it today. But I think you're absolutely right, and I think we see it especially in these very expensive treatments. We've been setting records in terms of expensive treatments basically every year for the past 15 years. This is a brand new one that beats all previous.

Leeza Osipenko: Thank you for this perspective. Indeed, a lot of patient organizations are far more funded, and a patient perspective sometimes is understandable, because when you're in a desperate situation, philosophical discussions will not get too far, neither hope. So, there's choice of as pharmaceutical drug or church. So, people make choices where they take money from, where they take their money to. Very controversial issues. But also, there are many sides to the story, and sometimes it's unfortunate that patients are influenced this way. But you can see why it's easy to persuade them, because for them it's the only hope sometimes, and the only way forward, at least the way they see it.

David Colquhoun: Yes, in 2011, I think there was a meeting at Imperial College on the open data, and so on. When I was on the panel sitting next to Mark Walport, who was then director of the Welcome Trust, subsequently became chief scientific advisor to the government, and I wondered why it was that the caring profession was the most corrupt of all professions, and Mark Walport was rather offended by that and afraid it's true, though. Do you think that's because of the lack of evidence in medicine, and the fact that most things can't be cured still leads to exaggerated claims. Or do you think our doctors really greedier than anybody else? Or is it just that there's more money available?

Sergio Sismondo: No, I think that the first and the last are true. I don't want it to pronounce on whether doctors are particularly greedy. As a general principle, I would assume that doctors are no more or less greedy than anybody else. If any other industry had the opportunity to be corrupt, I might question whether real estate is less corrupt than the medicine. But anyway, given the opportunity, almost any group of people would seize it. So there definitely is more money available, more money floating around. It seems to me untethered because in the sense that the pharmaceutical industry, especially the medical device industry which I didn't talk about at all here, is very similar, but those are the most profitable industries in the world. If you look at them as industries. They're more profitable than big oil. So, there's a lot of money that can just be spent on whatever projects these companies want to engage in. And I think you're right about the problems of evidence. We generally treat as an advance, any treatment that has a fairly marginal demonstrated benefit over the placebo. So, we don't usually get results that are just clean where we say this procedure just has a 95% success rate. That's a rare thing in medicine. So, then we're always working with the margins, and we have to interpret everything statistically. It's standardly joked that I think that's one of the best drugs developed over the past 30 or 40 years is Viagra, because most of the people who take Viagra see a fairly immediate response. They don't need to go to the statistics to either figure this out, whereas a lot of people would be critical of that drug and would say, no, this is part of the problem. The manufacture of diseases with which I agree, at least the drug most people who take it experiences what they're looking for. So, whereas say your average Statin, nobody can judge that the Statin they're taking is effective at reducing their risk of cardiovascular events, that is impulsive to judge. You have to go to the statistics, and then the statistical benefits very marginal, but we, as a society, accept that. But people in medicine accept these marginal benefits.

David Colquhoun: Yes, I take a Statin, but I took it mainly because my GP gets paid for by the health service with prescribing it. Thank you.

Adam Schindler:  I work in the pharmaceutical industry. And I wanted to bring to your attention a situation and get your thoughts on it, where the key opinion leaders sit on the Scientific Advisory Board, they're one on the same, and as members of the Scientific Advisory Board, they can get compensation and so it gets an end run around the issue of honorarium or fancy hotels by just flat out paying them and the form of stock options, or whatever to be on the board, and then they go out, and our principal off primary. First authors on papers as well as I would add in many cases, what I'd call shells for the company. They'll go out and say, hey, this is a great finding. When the data might not support it. So, it's a way to get around external people by just having them essentially as employees, but covering them in this veneer of independence, as they're not employees. They're independent doctors of this university or that university, but they're getting paid and they're out there running the trial. So, it's closing the loop and getting these people essentially as employees of the company. But I have to duck off for a meeting myself in a couple of minutes, but I just wanted to bring that to your attention and ask your thoughts on how that could be regulated slightly better.

Sergio Sismondo: That's one way in which high level KOLs get rewarded for what they're doing. How could this be regulated better well. This might be a case where medicine needs to self-regulate to some extent and starts creating maximum levels of reimbursement in terms of stock options and the rest. And if you want to maintain a license, then you have to be below this threshold. Maybe the threshold should be 0. I don't know but there might be a case for self-regulation as opposed to external regulation. There probably are ways of regulating it externally to. But I can't think of them off hand.

Joanna (question in the chat): What are your thoughts on the preparation that young doctors and medical students are given to be equipped for the conflicts of interest that they will face throughout their careers? And I know many medical universities here in Germany don't have conflict of interest policies. And from anecdotal U.S. evidence, it doesn't seem like conflict of interest is discussed in medical schools. Do future doctors even have a chance at navigating these waters, and to what degree is medical education biased if many KOL researchers are also faculty members, professors, etc.?

Sergio Sismondo: So, I'm going to give both a pessimistic and an optimistic response to that. Let me start with the optimistic one. I actually see a lot of medical students being aware of these issues and starting organizations that object to the things. So, there's no free lunch movement in the United States, which is a very much a medical student organization. So, we see that students are to some extent aware of that. This is entirely anecdotal, but I've heard people who are working with the company complaining about how medical students act, that medical students are being too critical of pharmaceutical companies and too aware of conflicts of interest. So, there's maybe some hope for optimism as there's as the awareness might be just growing gradually. And the students can do sometimes object to particular speakers coming in if the speakers are clearly presenting a position of the company for which they work, or for which they have conflicts of interest, or something like that, so that's one thing.

On the other hand, I think that the attitude within the industry is that this is just youthful exuberance, and when these medical students graduate, when they get 5 years out of that medical school, then they start realizing that there are advances to working with pharmaceutical companies in terms of the information that they receive, and in terms of the abilities to be in their practices. So, I think that's the pessimistic side, which is even if we could do a lot of work to convince medical students of the importance of conflicts of interest. They may very well lose that as real-world pressures bearing on them.

Leeza Osipenko: I agree with you. I do also see quite a bit of movements, and the younger generation trying to change the current practices, not just with conflicts of interest, but other things that seem to be going wrong in medicine interpretation and collection of evidence. So indeed, there is some hope. But Sergio, just to finish. Would you make one point of advice? Because I don't think we are improving very much in the situation we can really change things overnight. Mind you win, and industry can buy the opinions that they want, so one bit of advice to either people who read the papers or the patients, or good public, what are your thoughts?

Sergio Sismondo: I'm just going to say the same thing that I said before, for those of you who are in medicine, just discount almost everything that is about recently approved products. You can basically assume that it is the hands of the pharmaceutical industry have played an enormous role in that. And it's not until the products becoming profitable that the industry loses interest in the shaping of that information. So, then you start knowing much better what the pluses and minuses of any given treatment are. So, it's hard to give advice to everybody. But that's just for prescribers.