The modern medicine is a lot about numbers: chances of this being an infection, chances of failure a treatment, chances of complication from a surgery. All these numbers are usually presented as a RISK percentage. Making sense of this percentage is a little more difficult than making sense of the “Flat 50% off” percentage. And a lot of “CONDITIONS APPLY”. The risk is usually presented in a way it sounds most favorable to the premise of the study. At the other end is an odd entity, the odds ratio, which is never looked at respectfully.
1. Why do we need to quantify risk?
2. Study designs that can be used to reach the risk ratio?
3. Interpreting Risk?
4. Real Risk vs Presented Risk?
Lecturer: Dr. Siddharth Dikshit, DNB, FICO
DR DIKSHIT: I believe you are there, and a very, very warm welcome to everyone there. At the outset, I would like to thank Cybersight and Orbis for bringing us all together, and then I would like to thank my colleague, Sayan, for letting me use some of his slides, and more importantly, allowing me to use the form that he likes so much and I liked so much when I saw it on his slides. So we’ll be talking about statistics today. And the topic for the day is assessing risk. My name is Siddharth, and I work at the LV Prasad Eye Institute in Hyderabad. And Cybersight and Orbis have been very, very good friends with us, and I’m happy to play a role in this friendship. So the LV Prasad Eye Institute now has more than 200 places where we can call it our home, and we have presence across multiple countries and continents. And you can find us all around. What I believe is the greatest strength of the LV Prasad Eye Institute is to provide cutting-edge technology to every strata of society, irrespective of where they come from. Along with the medical care and the various aspects of medical care, education has been our forte right from the beginning, and today I’m happy to play a part in it. So let’s just get into it, dive into it. Even before I start talking, I would like to start with the question — I’m sure you all have the opportunity to vote for this question. All these questions will be a multiple choice question, and you will have to choose one of the four answers. So… The question here is: That if the treatment reduces the incidence of glaucoma in patients with ocular hypertension by 50%, so 50% of patients who would develop glaucoma otherwise don’t develop glaucoma if they are treated — so what will you do? Whether you will treat all patients, treat no patients, or you want more information before you decide, or you’ll wait for me to give the answer at the end of the lecture. So I’ll wait for, say, 30 seconds for the answer. I don’t have a clock ticking here, so it will be entirely my hunch that will play a role here. Okay. So I see quite a few answers coming in, and 75%, of the strength of four, has answered that they would want more information. Okay. So let me provide you with more information. But before that, I’ll provide you with another question. The early manifest glaucoma trial showed that treatment reduces the risk of glaucoma progression by 50%. And this is not what I am saying. It’s a sentence from the article. So if the risk of developing glaucoma without treatment is 60%, what is the risk of progression on treatment, if it’s reduced by 50%? 50%, 31%, 45%, or you would want to answer this at the end of the lecture again. Okay. Very nice, very nice. So I see more responses coming in, and it’s nice to see that there is a good division here. But again, we’ll see what the answer is at the end of the lecture, and I hope you will be able to understand better. So risk is all about chance. And it’s been there ever since Frank Sinatra sang the song of the strangers looking at each other and thinking of the chances. And it’s also in the statistics and in our medical course. So let’s start with a very simple question of what are the chances that it will rain today? The best way to know it is to stand outside and wait until the end of the day, and you’ll know it for sure. Other way is to look at your iPhone or your Android mobile and look at the weather. But let’s assume that we will look at the previous forecast from the meteorological department, the weather department. And it tells me that ’til now, on 40% of the days on 1st September, it has rained, and on 60% of the days, it has not rained, over the last 100 years, the data that I accessed. So what do you think is the chance that it will rain today? We will consider that data from the weather department is sacrosanct. So the chance obviously is — if the total number of times it’s been recorded is 100%, and 40% of the times it has rained, it’s actually 40%, or 40 by 100, or 0.4. We can call the 0.4 as a 4 in ten chance, or if you just divide both of these by two, a 2 in 5 chance. But a 40% chance. All these are same. But that’s not the only way you can present it. So you can present it as a probability of it raining against the probability of it not raining. So it can also be presented as a ratio of 40 is to 60. The probability of range is 40%, and probability of not raining is 60%. So the probability when put this way is 2 by 3 or 0.67. So it’s a 4 by 6 chance or 2 to 3 chance. Or other ways to say that there is a risk that you can calculate for every time that you calculate the chance, and you can calculate the odds. So the risk of raining was 4 in 10. Whereas the odds of raining was 4 to 6. What is important to note here: That the numerator in both the equations is the same. 40. But the denominator in the risk includes all the possible days where it could have rained. That is, the total number of days. Whereas the denominator in odds contained the number of days when it had not rained. So the numerator and denominator in odds is exclusive, whereas the numerator is a part of the denominator in risk. And we see here that the same fact that there is a 40% chance of raining is equally presentable as odds also. What is important to understand here is that you can choose how to present data. Though it is much more understandable if you tell a common person that the chances of it raining is 40%, they’ll understand it much better. But for someone who is betting on it, who wants to place a bet on it, he just wants to know: What are the possibilities of the other outcome happening? Of the rain not happening? So for him, odds are better. Just like as you would see in races, if you attend a race, 10 horses in the race, and someone tells you that the chances of this horse winning is, say, 3 by 4, but then he will say what are the odds, which means that: What are the chances of this horse winning compared to this horse not winning? Which may be different from just the presentation of the risk. Let’s see how chance is represented in clinical studies. And I, being a glaucoma person, will be talking about some glaucoma studies. The study that a lot of people consider landmark, as far as the way to conduct a study is concerned, is the ocular hypertension treatment study, and that is what I will talk about first. So in the ocular hypertension study, there were a large number of people involved. 817 in the treatment group and 819 in the control group. These patients were ocular hypertension. That is, the pressures were high, but they did not have glaucoma to start with. They were followed up for a period of five years. At the end of five years, 36 patients in the group of treatment developed glaucoma, compared to 89 patients in the group where there was no treatment offered. Which means that the risk and odds can be calculated from this 2×2 table. This is the simple 2×2 table that you see, and this third column is an extra that is addition of these two, which is the total number of treated population and total number of population in the control group. So the risk is given simply by the number of eyes that develop glaucoma by the total number of eyes that could have developed glaucoma in the treated group. Which means that it is the sum total — the number of eyes that develop glaucoma are a part of the denominator here. The odds, on the other hand, would be the number of eyes that developed glaucoma divided by the number of eyes that did not develop glaucoma. And as I said, the numerator is not a part of the denominator in the odds. So let us see what this means. What is the risk? The risk would be 36 by the total number of patients. That is 36 plus 781 or 817. Which comes from here. Which is 0.460. The odds, on the other hand, is 36 divided by 781. So this 781 is exclusive of the 36 eyes that you see here, and the odds would be 0.0440. So it’s a big number. Basically it’s something like 4.6% and 4.4%. It’s very, very similar to each other. But let’s keep it at decimal values for ease of calculation and understanding. So just as we calculated the risk and odds for the treatment group, we can do the same for the control group. So 89 by 819 gives us the relative risk, whereas 89 by 730 gives us the odds ratio for the control group. Now, what do we know here is this? How to calculate the relative risk of the patient being not treated or being in the control group, and that being a risk factor for glaucoma? Very simple. We’ve divided the risk in the control group by the risk in the treatment group, and that gives us the number of 2.4. For the odds ratio, again, we can simply calculate it by dividing the odds in the control group by the odds in the treatment group, which is 2.8. So very nice. We get the risk of developing glaucoma in ocular hypertension as 2.4 and the odds of developing glaucoma in ocular hypertension as 2.8 over a period of 5 years. So both these numbers are quite similar, and tells you that the risk lies somewhere between 2.5 times to 2.8 times. So quite good. They both agree with each other. Let’s see what the other trial gives us. So we see in the early manifest glaucoma trial, which was the only trial to look at the natural course of glaucoma, and proved that intraocular pressure reduction is actually useful, is the early manifest glaucoma trial. And in that, in the treatment group, there was progression seen in 58 patients. No progression in 71 patients. In the control group, it was slightly reversed. Progression was seen in 78 patients. Versus 48 patients who did not progress at all. This gives us some risk values. So going forward, the relative risk in the treated… In the early manifest glaucoma trial… Would be the relative risk of the control divided by relative risk of the treated patients. And the odds would be odds in the control group divided by the odds in the treated group. Which gives us these values. 1.38 is the relative risk. 1.98 is the odds ratio. Again, slightly different, but not very different numbers here. So this tells us how, from a simple 2×2 table, we can calculate the relative risk and the odds. And we see for both these trials, they are not very different from each other. But how do you compare chance? Let’s see. In the sense: How do you compare odds and risk, and are they actually completely interchangeable? We have a (inaudible) study done to look at incidence of age-related macular degeneration in smokers and non-smokers, and one particular study gave us the incidence of ARMD in smokers as 60 and in the non-smokers at 50 out of a total population of 100. So the risk in the group of smokers is 60 by 100, or 0.6, and in the non-smokers, is 50 by 100, or 0.5. The odds here would be 60 by 40 or 1.5, and in the non-smokers, it would be 50 by 50, or 1. So let’s see what the relative risks and odds ratios are. Very simple. So it’s 1.2 and 1.5. Very similar. So this study can publish anything — the odds ratio or the relative risk — and they both are the same and can be used interchangeably for this particular study. Let’s look at a different scenario, where the incidence has risen from 60% to 80% in the smokers. Whereas the incidence in non-smokers remains the same. Here the risk would become 0.8 in the group of smokers. The odds would be 4. Now, quite surprisingly, when you calculate the relative risk and the odds ratio, you will find a big difference. It becomes 1.6 versus 4. So when the incidence is low, the relative risk and odds ratio are quite interchangeable, as we saw here. But when the incidence becomes more and more common, the event or the disease occurs more and more commonly, the odds ratio starts rising disproportionately in comparison to the relative risk. So odds ratio — what is the role of odds ratio? It is particularly useful when you cannot calculate the incidence of the disease. Or in the case control studies. You cannot calculate the risk, because you don’t know how many patients develop the disease. Because you have not started from patients who had exposure and then went on to look for disease, but you’ve actually chosen 100 patients or so with the disease and 100 patients or so without the disease. That gives you the cases and the controls, and then you look for exposures. So you cannot say how many patients with a particular exposure could have had the disease. So you don’t know the incidence, and you cannot calculate the relative risk. And what has been found — that for incidence less than 30%, the odds and the relative risk are quite interchangeable. But if the incidence rate is more than 30%, the odds are very likely to overestimate the risk in the diseased group or the protective effect of the drug. So one has to be careful, whether there was a valid indication to use the odds or not. Let’s look at another example. So here the incidence is very low. So only 20 patients of the smokers and 10 amongst the non-smokers give a positive result for ARMD when followed up. So you see the relative risk and odds ratio is very close. Let us look at how to interpret risk reduction in clinical studies. We have looked at how risks can be calculated or miscalculated, deliberately. And now let us look at how risk reduction in clinical studies should be looked at. So the early manifest glaucoma trial showed that 58 patients developed glaucoma on treatment, compared to 78 without treatment. So the risk in the treated group was 44.96%, or somewhere around 45%. And the control group was 61.90%. So you see that the reduction is actually 17%, approximately. Because the treatment has reduced the risk from 62% to 45%. Quite simple. So if someone asks you by how much was the risk reduced in this group, because of the treatment, it is only 17%. But let’s see what the authors have published. They have actually published that the treatment reduced the risk of progression by approximately 50%. And this is the article I’m talking about. And this calculation is based on the hazard ratio of 0.50. And not on the absolute risk reduction. So it is very, very important to look at the absolute risk reduction. And not only the hazards ratio-based estimation of protective effect of the drug or treatment. So coming back to this question, quite surprisingly, despite the fact that the risk of developing glaucoma, developing progression in uncontrolled group was 62%, and the study said the risk would be reduced by 50%, the actual risk in the treated group was only 45%. Excuse me. So you see that does not make sense. So the risk reduction was hardly maybe close to 30%, not 50% at all. 17 by 45 would not even be 30%. It would be close to 30%. Now, given the fact that you understand what risk is, what relative risk is, what odds are, and how to interpret the risk reduction, is it sufficient for you to use this guideline to bring about a change in your clinical practice, based on the 2×2 table that the study gives you? So this is what the ocular hypertension treatment study said. And the risk or the incidence of glaucoma in the control group was reduced by 50% when the patients were treated. Right? 9.9% in the control group and 4.4% in the treated group. But that’s not all that you need to look at. You also need to look at the number needed to treat, the potential harm, and the impact of treatment. Now, these are terms which I will describe, and when you start with, they may not make much sense. But let’s look at what these terms mean. And these terms need to be looked at when the risk reduction or relative risk is actually — relative risk reduction is significant. What is important to know here: That in a group of 730, only around 90 people developed glaucoma. So 90% of them did not develop glaucoma over a period of 5 years of follow-up, despite not being treated. So those 90% definitely wouldn’t have benefited with treatment. Let’s look at what number to treat means. Let’s assume that — we start with 20 patients in each group. No treatment group on the top — so treatment group on the top and no treatment group on the bottom. And see how many develop glaucoma. So in the no treatment group, the incidence is around 10%, so 2 out of 20 will develop glaucoma. And in the treatment group, sorry for the label there. The bottom one is the treatment group. The incidence is around 5%. So 1% of patients will develop glaucoma. What would have happened if I had not treated these? How much of a role is the treatment playing? So we see that 1% develops glaucoma anyway. So this person would have not — has not benefited from treatment. We also see that this person or this eye has actually benefited from treatment. It’s jumping happily. But what is quite obvious and quite significantly obvious — that all these 18 patients, or 90% of the patients, have not benefited from treatment at all, over the end of five years. And we have treated them unnecessarily in the treatment group. So that is what the number to treat is. Now, what is potential harm? Obviously, if the treatment has significant side effects, it has probably no role in the clinical field at all. However, most drugs are either not completely harmless or absolutely harmful. They are somewhere in between. In the grey zone. So they have some benefits and some harms. So one has to measure the potential harms and potential benefits. So as far as ocular hypertension is concerned, we use some drugs which have some significant side effects. The glaucoma drugs. But none of them, unless you’re very careless, you give a beta blocker in a patient with bronchial asthma and chronic pulmonary disease, then you are in. Otherwise, you don’t usually do that. So they are not life threatening. We may have some side effects. So what other potential harm can be there? Let’s look at it. So for the five-year treatment, in my country, in India, you can choose multiple number drugs. The most commonly used drugs are the beta blockers. Or the prostaglandin analogs. So what would be the cost of using a drug like Xalatan for five years? Quite a huge amount of… 24 lakhs… 2.4 million rupees, in Indian currency. The same thing when you convert to using a beta blocker. It would be something between 292,000, or 2.9 lakhs. What does this mean? That instead of using Xalatan, if you would have kept the patient in the treatment group, 90% of them could have bought a Mercedes. And if you had stopped them from using Timolet, they could have bought a starting sedan. But they would have had a car. By just not using the drops. And saving that money instead. Putting it anywhere. Even without interest. They would have afforded a significantly beautiful-looking car these days. No matter what category of the car is. So even if there are no systemic life threatening or vision threatening side effects of the drop itself, the cost of treatment should be looked at, because when you add penny to penny over a period of five years, it becomes a huge amount. What about the impact of treatment? Now, we will just look at this visual field. And this is from a glaucoma patient who had an inferior nerve fiber layer defect, with a small excavation, with a disc hemorrhage. How on earth is this considered to be visually significant? This patient, even if the glaucoma progresses for under 40%, is not going to be limited in his actions in any way. He will carry on doing his work, just like as he was earlier, without even knowing that he has glaucoma. Of course, you don’t want him to progress to a 40% VFI, you don’t want him to progress to even 20%. But if the risk is low, you can allow this to happen. It’s not morally killing someone, like if you don’t treat an ocular hypertension. Because if you are wise enough to follow up these patients and see them in time, the maximum that you’re going to get is this. So the impact of treatment is to prevent this, which is not really significant. So when should you treat ocular hypertension? You need more information. What for? To identify high risk patients. So the ocular hypertension treatment study has done that for us. What it will do is, when you identify high risk patients, the number needed to treat will be low. The potential harm will be minimal, because the potential benefit is huge, and because these patients would be very likely to develop and keep on progressing on the glaucoma front, would have a very, very significant impact of treatment. So what becomes very obvious is, if the patients are divided into three groups, the lowest, the middle, and the highest risk, we see in the observation group 40% of patients went on to develop glaucoma. Not quite, disappointingly, because probably the pressure was not reduced significantly. 30% of them developed it despite being treated. Now, this we can take care of, by being more aggressive in treatment. By reducing the intraocular pressure based on target intraocular pressure calculations, and not be misled by just assuming that a 20% reduction will be sufficient. And the pressure of 23 to 24 was also acceptable in the ocular hypertension treatment study. So that kind of reduction is not acceptable. So with more aggressive treatment, we can definitely reduce the incidence here. What this means is, in the high risk group, the number needed to treat is as low as 7. So if in a group of 8, you treat each of these with the drug of your choice, that reduces the intraocular pressure to the OHTS desired levels, one of them will benefit. So that number is low, and will be even lower if you can bring this further down by increasing the control of intraocular pressure. So once you have identified the high risk patients, you can aggressively manage them and give benefit to maximum with causing minimal harm. So the question to the second answer in the quiz is — second question in the quiz — is we will treat only on the basis of high risk characteristics, and we needed more information for this. Now, before we go, this is a very, very popular study. And you can’t stop talking about it in many, many, many places. And you keep hearing about it. This was a very huge study, and 25,920 eyes were included in it. And out of the 25… Sorry, out of the 480,104 eyes, 112 patients whose eyes did not receive intracameral cefuroxime developed endophthalmitis, whereas out of the 25,920 eyes receiving intracameral cefuroxime, none of them developed endophthalmitis. Let’s see what this means. It means that intracameral cefuroxime was 100% effective in preventing postcataract surgery endophthalmitis. So based on this study, what do you think should be the practice pattern? Intracameral cefuroxime with cataract surgery should be used in… All cases? Cases with diabetes? Cases with posterior capsular rupture? Or you will wait for my answer to answer yourself? Okay. So this probably reflects the practice patterns at various places, and a lot of you have chosen all cases. Now, let’s look at why giving it in all cases is not a very, very good idea. So if you look at the fact… At even in the patients with no intracameral cefuroxime, the incidence of endophthalmitis was very, very low. It was 0.025%. Which means that this is a rare disease. And when the disease is rare, you need a huge sample for being able to find a significant difference between the two groups. And on the basis of simple formula, simple guidelines for calculating the sample size, the desired sample size would have been 31,390 patients in each group, which was obviously not met here. But this is just one part of it. The important part here is: That the relative risk reduction, though it’s 100%, because there is no incidence in the cefuroxime group, the absolutely risk reduction is 0.025%. So it’s 0.00025. The absolute risk reduction. So the amount of benefit that you have caused here is very minimal. But the impact of treatment is huge. So if I was going to develop an endophthalmitis in my eye, and I am told that it would have been prevented if I would have received an intracameral cefuroxime, of course — why not give that injection? But the intracameral cefuroxime is not free of adverse effects. If not handled well, it itself can introduce organisms into the body. And on the basis of this absolute risk reduction, the number needed to treat in general population is 4,000. Because the surgery itself, these days, is very, very risk-free. And this being a rare outcome — so you imagine you, in order to prevent one endophthalmitis, will have to give 4,000 injections. The other 3,999 patients would not have had endophthalmitis anyways. So 4,000 injections without any benefit. What would be the better strategy is to identify high risk groups. So what is the number needed to treat in patients with diabetes? It is 1,333. Because they have higher risk of endophthalmitis. But the number needed to treat in cases with vitreous loss is 571. For a disease which is quite serious. So we can quite, quite very well agree that all patients with vitreous loss, because of their higher risk to developing endophthalmitis, should get intracameral cefuroxime. So the number needed to treat gives us an idea here. But this number is huger than 90. But we still treat, because the impact of treatment is huge. And endophthalmitis, once it occurs, it may have a very, very unfavorable outcome. More so in patients with diabetes. So you are definitely justified in choosing all patients with vitreous loss to give cefuroxime. You may not be too wrong or too way off the mark for giving intracameral cefuroxime in patients with diabetes. So the answer would be definitely for posterior capsular rupture, but I think if you give it in patients with diabetes, that should not be too much of a problem. So this is the end of the lecture. But we had have questions. And before you dive into the questions, I would just want to remind of a few things. The relative risk and odds ratio are interchangeable only for diseases with low incidence. Typically less than 30%. So assess odds ratios that are published carefully. Whether it’s been done for a case control study or not. And if it’s not done for a case control study, but a cohort study, then what is the incidence? Because it can be misused to overrepresent the risk or the protective effect of the drug or intervention. Case control studies yield only odds ratio. So if the disease or the outcome is rare, you have to use a case control study in order to get a good sample size. So you should not devalue the odds ratio just because it does not follow the prospective pattern. But look at the reason for which the case control study was done, and odds ratio, when calculated properly, in a well designed case control study, may be a landmark study in itself. And risk reduction per se does not justify inclusion of proposed treatment in practice. You have to look at the number needed to treat, potential harm, and the impact. Thank you very much. And I would like to invite questions, if you have.
>> So it looks like we have one question so far.
DR DIKSHIT: Okay. The question is… Is there any reliable free software… The question will depend on which country are you in.
>> I believe that person was from India.
DR DIKSHIT: Okay. So you need to talk to people around you and see what software — a lot of universities even here that can give you a free copy of many registered softwares. But what you can also do is, for simple calculations, you have Excel. You can calculate mean, standard deviations, confidence intervals, perform the T test, Chi-square test, on Excel itself. And you have a lot of websites which give you a good idea of a whole range of options. For doing various statistical tests.
>> So we do have some time left. If anyone wants to ask a question, feel free to do so. And maybe we can wait about five minutes to see if more questions come in.
DR DIKSHIT: Okay. So there’s one question. With this one question — can you explain again the number needed to treat 4,000 — why we chose to treat patients who have posterior capsular rupture? Okay. Now, the decision to treat is based on four things: One is the absolute risk reduction. The number needed to treat. The potential harm of treatment. And the impact of treatment or a positive result. So what we see is that the absolute risk reduction is quite poor for the whole population. But when you bring it down to patients with posterior capsular rupture, you will realize that the number needed to treat has come down to… Something like 571. In itself, a huge number, but when you have to look at the kind of disease that you’re treating — so if you’re talking about endophthalmitis, this is a very serious disease. Very few patients reach us in time, and may have a very poor outcome, because of lack of this injection. So posterior capsular rupture seems to have at least eight times more risk here in this study, by looking at the number needed to treat. So if you multiply 500 by 8, it’s 4,000. 8 times higher risk than general population to develop endophthalmitis. And there, because of the impact of treatment is huge, despite the relatively higher number to treat than what you would consider for a topical medication, the absolute risk reduction being low, and with low potential harm, you choose to do it in a patient with posterior capsular rupture. That’s the highest risk group here. So how to calculate number needed to treat in patients in small groups? So if you can just make a 2×2 table, and you calculate the risks in both the groups, calculate the absolute risk reduction by subtracting the incidence in the control group with the incidence in the other group, you will get the absolute risk reduction there. And one by absolute risk reduction gives you the number needed to treat. Quite simple. Is it clear? Or you would want me to explain it further? Because we have time, I’ll probably just explain it a little further. If you look here, in the early manifest glaucoma trial, the risk in the control group is 62%. The risk in the treatment group is 45%. So absolute risk reduction is 17%, approximately, or 0.0017. So when you divide 1 by 0.0017, it would give you a result of around — something between 5.5 to 6. Something around 5.25. So that’s the number needed to treat in this study. Okay?
>> So if anyone else has questions, this will be our last call for questions. Maybe we’ll wait a few more minutes.
DR DIKSHIT: Hazard ratio is an entirely different thing. Hazard ratio is a calculation that you make from regression analysis. And it gives you the possible responsibility that a factor would carry amongst many factors in either protecting the disease or precipitating the event, depending on what you are looking at. So hazard ratio is very, very far separated from odds ratio, as far as the actual value is concerned. Hazard ratio would be a different calculation entirely. It’s not related to calculation of the risk at all between two groups. It’s a thing that you calculate based on a regression analysis. And see how much of a risk factor… How much of a risk is a particular factor amongst various risk factors that you have found out.