>> David Luckenbaugh: Hello, my name is David Luckenbaugh. I'm a statistician at the National Institutes of Health in the Office of Equity, Diversity, and Inclusion in the Office of the Director. I want to thank you for listening today and we're going to talk about sensitivity to change. So, our main objective is looking at sensitivity to change within clinical research. Some of you may know sensitivity to change as responsiveness. These are the same concepts that you may know from one -- just depends on the field, whether people talk about it as sensitivity to change or responsiveness. So, I'm going to go back. I'm going to talk about -- a little bit about depression, tremor, and heart disease. In talking about sensitivity to change, one of the things is we have many ways of measuring the same constructs. The problem is that we have to decide which one is our favorite one. In heart disease, you might measure cholesterol, or you might measure C-reactive protein, but the context of the study might determine what, in fact, you want to do. If your interest is looking at brain functioning, probably looking at cholesterol or C-reactive protein is not going to be the best measurement that you have. So, sensitivity to change is the ability to detect -- the ability to detect improvement or worsening. So, one way of measuring this is by looking at effect size. Let's say, for example, that we have two groups that we want to measure. One group is going to be on drug A, one group is going to be on drug B, which is a placebo. What we can do is we can look at the mean of those two groups, divide it by the pooled standard deviation, and we come up with Cohen's d. This measurement is also known as the standardized mean difference. The standard interpretation for this is that an effect size or a Cohen's d of point two is considered a small effect. A moderate effect is about 0.5 and a large effect is about 0.8. Now this value can go up to infinity with still an effect size of 0.8 is considered fairly large. In order to put this in context a little bit, let me say a little bit about my area where I used to work with depression research and the tendency was if you had a significant difference from placebo, you'd have about a moderate difference, usually a Cohen's d somewhere between 0.4 and probably 0.55. Again, to put this in context, here at NIH we were able to do studies with 30, 60, maybe 100 people at most, where as in other places you might be able to look at 10,000 people. In the Physicians Health Study for example, they were able to look at 10,000 people to look at heart attacks and found that aspirin actually helped to prevent heart attacks. Certainly, there has been further research on the use of aspirin, but the effect size in that original study was not 0.2, it was 0.02, so it just goes to show you that even though you have a small effect size, it can actually be very meaningful. In heart disease, you're talking about saving lives, so you could save a lot of lives despite having such a small effect size. So, sensitivity to change again, and responsiveness is the ability to detect change. Sometimes you can have reliability and validity, but you're not able to show a change in measures. So, let's go to looking at an example of why sensitivity to change might be important to you. So, there was a study done back at the end of the '90s looking at the effects of carbamazepine and lithium on mania. The measure -- one measure that was used was a daily rating of manic symptoms, in that with the manic symptoms, they could do one thing, which was take the average severity for every day, so this is the -- taking the average of 100 -- 365 days and coming up with a mean severity. They also looked at the number of days where someone could be considered manic. They clumped these together and said, "Okay, well, if you had a certain number of days together, this could be called an episode." So, they also counted the number of episodes, and finally they used a standardized measure called the Young Mania Rating Scale that's included in here. So, all of these measures were used; three of them derived from the same measure. So, what you see here are the effect sizes in this study for each of the different measurements. Mean severity and time ill actually had a fairly similar effect size. No surprise, they came from the same data, their daily measurements. The Young Mania Rating Scale was done on more of a monthly basis, so that might not be as sensitive to changes, so you have a little bit smaller effect, and then the number of episodes, really you could only get probably four to five maximum values. So, there was a fairly brief range of values that you could get for that, so that probably led to an even smaller sensitivity to change. So, here, okay, so you have effect sizes, you have rating scales, but why should this matter to you? The reason it should matter is because this has implications for the sample size that you would use for your study. So, here we go. If we looked at -- used the number of episodes from this mania study, what we would have is we would need over 400 different patients. In this study actually, the investigators followed -- followed patients on lithium or carbamazepine for a full year. So, having 400 plus people followed for a full year on these drugs could be a difficult challenge. If you go to the Young Mania Scale, that 400 drops to about 120. That's much more manageable, but then, again, if you go with time ill or mean severity, you could actually get down to more like 60 or 70 people for the study. So, it makes a big difference exactly which measure you want to use. The problem, of course, of just looking at sample size is that, in some cases, you actually want to look at something like episodes; maybe that's really your interest, you don't really care about what individual times or what individual days add up to to make a severity. So, the context of your study is important. Even if you need 400 people to do a study, it may be important to do that because that's actually the measure that you're interested. It fits the construct that you want to measure the best. Same thing goes with Young Mania Rating Scale. There may be some items on that scale that you would like to tap into that are not tapped into in the Mean Severity. In this case, for example, the Mean Severity Score comes from just a simple daily rating from one to four -- from zero to four of, "Do you feel manic today?" So, every day, "Do you feel manic today? Do you feel manic today?" and we added those up, took the average, and got a mean severity. So, that is different than looking at a scale that has 11 items that is asking about individual symptoms like the Young Mania Scale does. So, the Young Mania Scale actually may offer you something very different than the Mean Severity Scale. It doesn't look like it's as sensitive as the Mean Severity Scale or the Time Ill Scale, but it may be exactly what you need for your study. And, actually, when you have a scale sometimes you're able to break it down into individual pieces to look at individual symptoms so that's an advantage of that scale that you wouldn't have with the Mean Severity Scale as shown here. So, in summary, sensitivity to change or responsiveness is the ability to detect movement in a measure. Sensitivity to change can be measured with effect sizes and sensitivity to change can alter the sample size needed to conduct a study. A couple questions that may help you in thinking about this information is, "What can be used to measure sensitivity to change?" When planning the number of participants in a study, do you need to worry about sensitivity to change? Thank you very much for listening today. If you have any questions or concerns, please contact the Program Coordinator.