1 00:00:06,441 --> 00:00:10,145 >> GOOD AFTERNOON, EVERYONE. 2 00:00:10,145 --> 00:00:11,746 I'M STATISTICIAN IN CLINICAL 3 00:00:11,746 --> 00:00:13,281 TRIALS UNIT. 4 00:00:13,281 --> 00:00:16,017 I WANT TO FIRST GIVE A BIG THANK 5 00:00:16,017 --> 00:00:18,520 YOU FOR JOINING THE FIFTH PART 6 00:00:18,520 --> 00:00:20,288 OF THE CLINICAL TRIALS 7 00:00:20,288 --> 00:00:22,690 STATISTICAL LECTURE SERIES, 8 00:00:22,690 --> 00:00:26,761 TODAY IS A META-ANALYSIS, SO I 9 00:00:26,761 --> 00:00:30,698 WILL BEGIN WITH AN OVERVIEW OF 10 00:00:30,698 --> 00:00:31,900 THAT METHODOLOGY. 11 00:00:31,900 --> 00:00:38,072 IN SHORT A META-ANALYSIS IS A 12 00:00:38,072 --> 00:00:39,841 STATISTICAL METHOD USED TO 13 00:00:39,841 --> 00:00:46,848 SYNTHESIZE RESULTS FROM A SERIES 14 00:00:46,848 --> 00:00:50,785 OF SCIENTIFIC STUDIESF I PUT A 15 00:00:50,785 --> 00:01:00,461 NUMBER ON IT, IT ENTAILS 16 00:01:00,461 --> 00:01:08,570 CONDUCTING SYSTEMATIC LITERATURE 17 00:01:08,570 --> 00:01:09,103 REVIEW. 18 00:01:09,103 --> 00:01:19,581 THIS ENTAILS CONDUCTING ON A 19 00:01:21,816 --> 00:01:22,684 SYSTEMATIC LITERATURE REVIEW. 20 00:01:22,684 --> 00:01:25,186 WE WILL BE ABLE TO COVER 21 00:01:25,186 --> 00:01:27,488 SECTIONS 2 THROUGH 5 OF THE 22 00:01:27,488 --> 00:01:32,093 PRESENTATION, ALL RELATED TO THE 23 00:01:32,093 --> 00:01:34,896 SYSTEMATIC REVIEW PROCESS AND 24 00:01:34,896 --> 00:01:36,764 SECTION 6 THROUGH 9 FOR THE MOST 25 00:01:36,764 --> 00:01:47,308 PART RELATED TO THE STATISTICAL 26 00:01:58,920 --> 00:01:59,287 SYNTHESIS. 27 00:01:59,287 --> 00:02:03,825 WE WILL DISCUSS THE COMPONENTS. 28 00:02:03,825 --> 00:02:04,392 SYSTEMATIC REVIEWS AND 29 00:02:04,392 --> 00:02:06,461 META-ANALYSIS FORM TOP HIERARCHY 30 00:02:06,461 --> 00:02:15,169 OF EVIDENCE DUE TO OBJECTIVE AND 31 00:02:15,169 --> 00:02:16,738 COMPREHENSIVE NATURE. 32 00:02:16,738 --> 00:02:18,940 SYSTEMATIC REVIEWS ARE 33 00:02:18,940 --> 00:02:21,309 COMPREHENSIVE AND UNBIASED THESE 34 00:02:21,309 --> 00:02:24,779 OF MANY RELEVANT STUDIES USING 35 00:02:24,779 --> 00:02:27,015 RIGOROUS TRANSPARENT METHODS. 36 00:02:27,015 --> 00:02:29,550 THIS PROCESS HELPS RESEARCHERS 37 00:02:29,550 --> 00:02:30,852 MAKE INFORMED DECISIONS AND 38 00:02:30,852 --> 00:02:41,396 IDENTIFY GAPS IN THE LITERATURE. 39 00:02:41,596 --> 00:02:44,098 AS STATED, META-ANALYSIS IS A 40 00:02:44,098 --> 00:02:45,533 STATISTICAL METHOD THAT COMBINES 41 00:02:45,533 --> 00:02:47,302 RESULTS FROM MULTIPLE STUDIES TO 42 00:02:47,302 --> 00:02:52,840 ADDRESS A SIMILAR, THAT IS THE 43 00:02:52,840 --> 00:02:55,777 KEY, WELL-DEFINED RESEARCH 44 00:02:55,777 --> 00:02:56,411 QUESTION. 45 00:02:56,411 --> 00:02:59,948 IT INCREASES STATISTICAL POWER, 46 00:02:59,948 --> 00:03:01,482 PROVIDING MORE ROBUST 47 00:03:01,482 --> 00:03:04,085 CONCLUSIONS, META-ANALYSIS IS 48 00:03:04,085 --> 00:03:05,153 BEING USED INCREASINGLY AND 49 00:03:05,153 --> 00:03:08,323 PLAYS AN IMPORTANT ROLE IN 50 00:03:08,323 --> 00:03:14,696 MEDICAL RESEARCH. 51 00:03:14,696 --> 00:03:21,102 AS SHOWN HERE, THE HIERARCHY OF 52 00:03:21,102 --> 00:03:23,838 SCIENTIFIC STUDIES, SYSTEMATIC 53 00:03:23,838 --> 00:03:24,806 REVIEWS AND META-ANALYSES SIT AT 54 00:03:24,806 --> 00:03:30,545 THE TOP OF THE PYRAMID OF 55 00:03:30,545 --> 00:03:31,546 EVIDENCE-BASED CLINICAL STUDIES. 56 00:03:31,546 --> 00:03:37,618 JUST BELOW IT YOU HAVE 57 00:03:37,618 --> 00:03:39,654 RANDOMIZED CONTROLLED TRIALS, 58 00:03:39,654 --> 00:03:41,389 COHORT STUDIES, CASE-CONTROLLED 59 00:03:41,389 --> 00:03:44,359 STUDIES, CASE AND REPORTS, AND 60 00:03:44,359 --> 00:03:47,295 BASE OF THE PYRAMID IS FORMED BY 61 00:03:47,295 --> 00:03:56,404 BACKGROUND INFORMATION AND 62 00:03:56,404 --> 00:03:57,372 EXPERT OPINION. 63 00:03:57,372 --> 00:03:59,173 THE FLOW DIAGRAM OF 64 00:03:59,173 --> 00:04:04,812 META-ANALYSIS DE DEPICTS FLOW OF 65 00:04:04,812 --> 00:04:09,417 INFORMATION THROUGH PHASES OF 14 66 00:04:09,417 --> 00:04:10,318 STEPS. 67 00:04:10,318 --> 00:04:12,453 SO, THIS DIAGRAM HAS 14 STEPS 68 00:04:12,453 --> 00:04:16,891 BEGINNING WITH DEFINING THE 69 00:04:16,891 --> 00:04:20,962 WELL-DEFINED RESEARCH QUESTIONS. 70 00:04:20,962 --> 00:04:23,965 STEPS 2 THROUGH 9 INVOLVE, MAYBE 71 00:04:23,965 --> 00:04:26,467 HALF OF THE META-ANALYSIS, AN 72 00:04:26,467 --> 00:04:29,437 EXHAUSTIVE AND THOROUGH RESEARCH 73 00:04:29,437 --> 00:04:31,239 OF THE LITERATURE. 74 00:04:31,239 --> 00:04:35,910 STEP 10 IN THIS 14-STEP PROCESS 75 00:04:35,910 --> 00:04:38,146 INVOLVES DATA EXTRACTION ANT 76 00:04:38,146 --> 00:04:39,380 QUALITY ASSESSMENT, AND QUALITY 77 00:04:39,380 --> 00:04:42,517 ASSESSMENT IS ALSO REFERRED TO 78 00:04:42,517 --> 00:04:46,587 AS THE RISK OF BIAS ASSESSMENT. 79 00:04:46,587 --> 00:04:49,590 SO, ALTHOUGH THIS IS THE ENTIRE 80 00:04:49,590 --> 00:04:53,061 FIELD, META-ANALYSIS STILL 81 00:04:53,061 --> 00:04:55,329 FAIRLY NEW, THIS QUALITY 82 00:04:55,329 --> 00:04:57,965 ASSESSMENT I WOULD SAY HAS 83 00:04:57,965 --> 00:04:59,534 BECOME AN ESSENTIAL COMPONENT 84 00:04:59,534 --> 00:05:01,102 WITHIN THE LAST A 5-10 YEARS. 85 00:05:01,102 --> 00:05:08,176 SO WE WILL SEE AN EXAMPLE OF HOW 86 00:05:08,176 --> 00:05:12,780 ONE WAS PERFORMED JUST LOOSELY, 87 00:05:12,780 --> 00:05:16,317 AS AN OVERVIEW. 88 00:05:16,317 --> 00:05:21,956 STEP 11 IS DATA CHECKING TASK. 89 00:05:21,956 --> 00:05:25,793 STEP 12 IS WHEN WE CONDUCT 90 00:05:25,793 --> 00:05:28,129 STATISTICAL ANALYSIS. 91 00:05:28,129 --> 00:05:29,831 SO, SOME OF OUR COLLEAGUES THINK 92 00:05:29,831 --> 00:05:32,533 THE META-ANALYSIS IS THE 93 00:05:32,533 --> 00:05:36,771 STATISTICAL ANALYSIS BUT REALLY 94 00:05:36,771 --> 00:05:39,974 THE META-ANALYSIS IS THIS 95 00:05:39,974 --> 00:05:42,210 COMPLETE 14-STEP PROCESS. 96 00:05:42,210 --> 00:05:44,745 STEP 13, DOUBLE CHECKING AGAIN 97 00:05:44,745 --> 00:05:47,548 DATA. 98 00:05:47,548 --> 00:05:49,217 AND 14 IS DISSEMINATION. 99 00:05:49,217 --> 00:05:53,754 SHARING OF RESULTS, WRITING THE 100 00:05:53,754 --> 00:05:54,655 MANUSCRIPT, REVISIONS, ALSO 101 00:05:54,655 --> 00:05:58,392 SUBMISSION. 102 00:05:58,392 --> 00:06:00,928 HOW DOES IT WORK? 103 00:06:00,928 --> 00:06:03,798 THE PROCESS INVOLVES SEARCHING 104 00:06:03,798 --> 00:06:05,867 VARIOUS DATABASES OF STUDIES, 105 00:06:05,867 --> 00:06:10,972 RELEVANT TO A SPECIFIC QUESTION. 106 00:06:10,972 --> 00:06:16,744 I'M HIGHLIGHTING THAT AGAIN, 107 00:06:16,744 --> 00:06:17,311 VERY IMPORTANT. 108 00:06:17,311 --> 00:06:19,347 CRITERIA APPLY 20 CHOOSE WHICH 109 00:06:19,347 --> 00:06:24,051 STUDIES ARE INCLUDED IN REVIEW, 110 00:06:24,051 --> 00:06:25,520 DATA ARE SUMMARIZED, SYSTEMATIC 111 00:06:25,520 --> 00:06:25,920 REVIEW. 112 00:06:25,920 --> 00:06:29,357 THE VERY FIRST STEP IN THIS 113 00:06:29,357 --> 00:06:32,760 PROCESS IS DEVELOPING A FOCUSED 114 00:06:32,760 --> 00:06:35,563 RESEARCH QUESTION, USING WHAT'S 115 00:06:35,563 --> 00:06:38,266 CALLED THE PICOT FRAMEWORK. 116 00:06:38,266 --> 00:06:41,869 THAT IS AN ACRONYM. 117 00:06:41,869 --> 00:06:44,539 P FOR POPULATION, POPULATION 118 00:06:44,539 --> 00:06:46,674 REFERS TO SAMPLE SUBJECTS THAT 119 00:06:46,674 --> 00:06:53,014 WERE RECRUITED IN A STUDY. 120 00:06:53,014 --> 00:06:54,849 THAT'S AN ESSENTIAL COMPONENT I 121 00:06:54,849 --> 00:06:55,883 FOR INTERVENTION, TREATMENT 122 00:06:55,883 --> 00:06:57,485 PROVIDED TO SUBJECTS ENROLLED IN 123 00:06:57,485 --> 00:06:59,153 YOUR STUDY. 124 00:06:59,153 --> 00:07:01,756 C REFERS TO COMPARISON GROUPS. 125 00:07:01,756 --> 00:07:04,659 O REFERS TO THE OUTCOME 126 00:07:04,659 --> 00:07:05,660 MEASUREMENT. 127 00:07:05,660 --> 00:07:08,162 AND T DESCRIBES THE DURATION FOR 128 00:07:08,162 --> 00:07:09,630 YOUR DATA COLLECTION. 129 00:07:09,630 --> 00:07:12,133 WHEN DEVELOPING THIS VERY 130 00:07:12,133 --> 00:07:13,201 IMPORTANT RESEARCH QUESTION, IT 131 00:07:13,201 --> 00:07:15,970 IS ESSENTIAL TO TAKE INTO 132 00:07:15,970 --> 00:07:20,241 ACCOUNT EACH OF THOSE COMPONENTS 133 00:07:20,241 --> 00:07:30,651 OF THE PICOT FRAMEWORK. 134 00:07:34,422 --> 00:07:37,892 A COMPREHENSIVE LITERATURE 135 00:07:37,892 --> 00:07:44,599 INVOLVES DATABASES LIKE MEDLINE, 136 00:07:44,599 --> 00:07:46,367 EMBAS EE COCHRAN, GRAY 137 00:07:46,367 --> 00:07:47,835 LITERATURE, CLEAR CRITERIA FOR 138 00:07:47,835 --> 00:07:50,938 INCLUSION AND EXCLUSION HELP 139 00:07:50,938 --> 00:07:52,206 MAINTAIN CONSISTENCY AND 140 00:07:52,206 --> 00:07:54,408 RELIABILITY IN SELECTING 141 00:07:54,408 --> 00:07:55,476 RELEVANT STUDIES. 142 00:07:55,476 --> 00:07:57,745 MULTIPLE REVIEWERS MAY BE NEEDED 143 00:07:57,745 --> 00:08:00,014 TO ENSURE OBJECTIVE AND UNBIASED 144 00:08:00,014 --> 00:08:04,151 INCLUSION OF STUDIES IN THE 145 00:08:04,151 --> 00:08:04,518 REVIEW PROCESS. 146 00:08:04,518 --> 00:08:06,354 A COMMON TECHNIQUE FOR THE 147 00:08:06,354 --> 00:08:12,693 RESEARCH TEAMS IS TO USE TWO TO 148 00:08:12,693 --> 00:08:16,297 THREE REVIEWERS TO CONDUCT 149 00:08:16,297 --> 00:08:17,565 INDEPENDENT SYSTEMATIC REVIEWS, 150 00:08:17,565 --> 00:08:23,804 AND THEN RECONCILE THOSE 151 00:08:23,804 --> 00:08:25,673 DIFFERENCES TO MAKE SURE REVIEWS 152 00:08:25,673 --> 00:08:26,774 OF AMOUNT OF LITERATURE 153 00:08:26,774 --> 00:08:36,517 CONSIDERED IS AN EXHAUSTIVE AND 154 00:08:36,517 --> 00:08:37,818 RELEVANT REPRESENTATION OF THE 155 00:08:37,818 --> 00:08:38,919 LITERATURE RELATIVE TO THAT 156 00:08:38,919 --> 00:08:40,755 PARTICULAR RESEARCH QUESTION. 157 00:08:40,755 --> 00:08:42,223 THERE'S SEVERAL GUIDELINES 158 00:08:42,223 --> 00:08:44,091 COMMONLY USED TO PROVIDE WHAT I 159 00:08:44,091 --> 00:08:47,261 WOULD CONSIDER TO BE A BLUEPRINT 160 00:08:47,261 --> 00:08:51,966 FOR HOW TO CONDUCT A 161 00:08:51,966 --> 00:08:52,633 META-ANALYSIS. 162 00:08:52,633 --> 00:08:55,636 MOST RELEVANT FOR RANDOM 163 00:08:55,636 --> 00:08:58,139 CONTROLLED TRIALS IS PRISMA 164 00:08:58,139 --> 00:08:58,539 STATEMENT. 165 00:08:58,539 --> 00:09:02,376 IN 2009, THE PREFERRED REPORTING 166 00:09:02,376 --> 00:09:05,446 ITEMS FOR SYSTEMATIC REVIEWS AND 167 00:09:05,446 --> 00:09:09,617 META-ANALYSES ALSO KNOWN AS 168 00:09:09,617 --> 00:09:13,087 PRISMA STATEMENT WAS LAUNCHED. 169 00:09:13,087 --> 00:09:15,823 PRISMA PROVIDES CLEAR 170 00:09:15,823 --> 00:09:18,659 DOCUMENTATION OF SEARCH 171 00:09:18,659 --> 00:09:19,260 CRITERIA, RECORDS IDENTIFIED, 172 00:09:19,260 --> 00:09:20,795 EXCLUSIONS, AND THEN YOU ALSO 173 00:09:20,795 --> 00:09:24,031 HAVE TO GIVE REASONS FOR THOSE 174 00:09:24,031 --> 00:09:24,398 EXCLUSIONS. 175 00:09:24,398 --> 00:09:28,836 THE PRISMA STATEMENT ITSELF IS A 176 00:09:28,836 --> 00:09:31,372 DOCUMENT, A DOCUMENT CONSISTING 177 00:09:31,372 --> 00:09:40,281 OF A 27-ITEM CHECK LIST AND A 178 00:09:40,281 --> 00:09:42,516 FLOW DIAGRAM. 179 00:09:42,516 --> 00:09:53,027 THIS IS THE ACTUAL COPY OF THE 180 00:09:53,994 --> 00:09:57,198 PRISMA 2020 CHECKLIST, ITEMS 181 00:09:57,198 --> 00:09:59,500 1-15. 182 00:09:59,500 --> 00:10:02,470 AS YOU WORK YOUR WAY THROUGH THE 183 00:10:02,470 --> 00:10:03,871 META-ANALYSIS EACH ITEM HAS TO 184 00:10:03,871 --> 00:10:06,107 BE ADHERED TO. 185 00:10:06,107 --> 00:10:09,977 SECTIONS INCLUDE TITLE, 186 00:10:09,977 --> 00:10:11,212 ABSTRACT, INTRODUCTION, METHODS. 187 00:10:11,212 --> 00:10:14,081 OBVIOUSLY WITHIN THE METHODS YOU 188 00:10:14,081 --> 00:10:19,587 WANT TO PROVIDE ELIGIBILITY 189 00:10:19,587 --> 00:10:21,422 CRITERIA, INFORMATION SOURCES, 190 00:10:21,422 --> 00:10:27,762 SEARCH STRATEGIES, DATA 191 00:10:27,762 --> 00:10:33,200 COLLECTION PROCESSES. 192 00:10:33,200 --> 00:10:37,938 RANDOM ESTIMATION APPROACH. 193 00:10:37,938 --> 00:10:41,709 ITEMS 16-27 CONTAINS THE TOPICS 194 00:10:41,709 --> 00:10:46,947 FOR RESULTS, DISCUSSION, AND 195 00:10:46,947 --> 00:10:47,348 OTHER INFORMATION. 196 00:10:47,348 --> 00:10:48,349 AGAIN, THIS IS THE BLUEPRINT, 197 00:10:48,349 --> 00:10:52,052 THIS WOULD BE THE EXPECTED, LIKE 198 00:10:52,052 --> 00:10:52,820 A TEMPLATE. 199 00:10:52,820 --> 00:10:54,755 WHEN YOU SUBMIT THE ARTICLE TO A 200 00:10:54,755 --> 00:10:58,559 JOURNAL, THIS IS THE EXPECTED 201 00:10:58,559 --> 00:10:58,859 LAYOUT. 202 00:10:58,859 --> 00:11:02,396 IN THE RESULTS SECTION YOU WOULD 203 00:11:02,396 --> 00:11:05,866 DESCRIBE THE STUDY SELECTION, 204 00:11:05,866 --> 00:11:08,969 STUDY CHARACTERISTICS, RISK OF 205 00:11:08,969 --> 00:11:10,604 BIAS IN STUDY, STILL FAIRLY NO 206 00:11:10,604 --> 00:11:14,241 SO IF -- NEW SO IF POSSIBLE TO 207 00:11:14,241 --> 00:11:15,075 DO SO. 208 00:11:15,075 --> 00:11:16,911 RESULTS OF INDIVIDUAL STUDIES. 209 00:11:16,911 --> 00:11:17,945 AND THEN RESULTS OF THE 210 00:11:17,945 --> 00:11:21,315 SYNTHESIS, WHICH IS GOING TO BE 211 00:11:21,315 --> 00:11:24,485 IN MOST CASES THIS SUMMARIZED 212 00:11:24,485 --> 00:11:27,955 EFFECT BASED ON RANDOM EFFECTS 213 00:11:27,955 --> 00:11:34,562 ESTIMATION PROCESS. 214 00:11:34,562 --> 00:11:37,198 THE PRISMA FLOW DIAGRAM DEPICTS 215 00:11:37,198 --> 00:11:39,433 THE FLOW OF INFORMATION THROUGH 216 00:11:39,433 --> 00:11:41,902 THE DIFFERENT PHASES OF 217 00:11:41,902 --> 00:11:43,270 SYSTEMATIC REVIEW, IT MAPS OUT 218 00:11:43,270 --> 00:11:47,374 THE NUMBER OF RECORDS 219 00:11:47,374 --> 00:11:48,275 IDENTIFIES, INCLUDED AND 220 00:11:48,275 --> 00:11:51,178 EXCLUDED, AND THE REASONS FOR 221 00:11:51,178 --> 00:11:53,113 THOSE EXCLUSIONS. 222 00:11:53,113 --> 00:11:56,283 THIS IS OFTEN THE VERY FIRST 223 00:11:56,283 --> 00:11:57,751 FIGURE PROVIDED OR THAT YOU 224 00:11:57,751 --> 00:11:59,820 WOULD ENCOUNTER IN THE 225 00:11:59,820 --> 00:12:04,258 META-ANALYSIS. 226 00:12:04,258 --> 00:12:06,961 I DO WANT TO SAY A FEW WORDS 227 00:12:06,961 --> 00:12:09,430 ABOUT RISK OF BIAS. 228 00:12:09,430 --> 00:12:12,199 BIAS ITSELF IS THE THEORETICAL 229 00:12:12,199 --> 00:12:12,766 MEASUREMENT. 230 00:12:12,766 --> 00:12:14,301 WE KNOW IT EXISTS. 231 00:12:14,301 --> 00:12:18,439 IT'S VERY DIFFICULT TO QUANTIFY 232 00:12:18,439 --> 00:12:19,173 BIAS. 233 00:12:19,173 --> 00:12:25,746 BIAS IS THIS DIFFERENCE BETWEEN 234 00:12:25,746 --> 00:12:26,647 THIS OBSERVED MEASUREMENT OF 235 00:12:26,647 --> 00:12:27,748 ESTIMATE AND TRUE VALUE. 236 00:12:27,748 --> 00:12:29,250 SO IN THE CONTEXT OF A 237 00:12:29,250 --> 00:12:32,386 META-ANALYSIS, IT'S GOING TO BE 238 00:12:32,386 --> 00:12:35,456 THE DIFFERENCE BETWEEN THE 239 00:12:35,456 --> 00:12:38,826 OBSERVED EFFECT VERSUS THE TRUE 240 00:12:38,826 --> 00:12:39,460 EFFECT. 241 00:12:39,460 --> 00:12:41,896 THE RISK OF BIAS ASSESSMENT ON A 242 00:12:41,896 --> 00:12:44,265 QUALITY ASSESSMENT TRIES TO BE 243 00:12:44,265 --> 00:12:45,833 TRANSPARENT. 244 00:12:45,833 --> 00:12:50,271 IT IS A TOOL THAT ALLOWS THE 245 00:12:50,271 --> 00:12:53,908 TEAM TO BE TRANSPARENT ON THE 246 00:12:53,908 --> 00:12:55,042 POSSIBLE SOURCES OF BIAS. 247 00:12:55,042 --> 00:12:58,345 SO, FOR EXAMPLE, BIAS CAN ARISE 248 00:12:58,345 --> 00:13:02,483 AT MULTIPLE STAGES. 249 00:13:02,483 --> 00:13:03,918 SYSTEMATIC REVIEWS AFFECTING 250 00:13:03,918 --> 00:13:05,853 VALIDITY OF THE CONCLUSIONS 251 00:13:05,853 --> 00:13:10,224 DRAWN, TIMES OF BIAS INCLUDE 252 00:13:10,224 --> 00:13:12,793 SELECTION BIAS, PERFORMANCE 253 00:13:12,793 --> 00:13:14,094 BIAS, DETECTION BIAS, 254 00:13:14,094 --> 00:13:15,396 PUBLICATION BIAS. 255 00:13:15,396 --> 00:13:21,936 EACH STUDY SHOULD BE ASSESSED 256 00:13:21,936 --> 00:13:26,307 RIGOROUSLY FOR POTENTIAL BIAS 257 00:13:26,307 --> 00:13:27,341 USING STANDARDIZED TOOLS AND 258 00:13:27,341 --> 00:13:29,143 REPORTING GUIDELINES. 259 00:13:29,143 --> 00:13:30,311 WE'LL LOOK AT AN EXAMPLE WHAT 260 00:13:30,311 --> 00:13:33,514 WAS DONE FOR A META-ANALYSIS 261 00:13:33,514 --> 00:13:36,216 PUBLISHED IN THE COCHRANE 262 00:13:36,216 --> 00:13:42,656 COLLABORATION LIBRARY. 263 00:13:42,656 --> 00:13:46,093 COMMONLY USED TOOLS TO DETECT 264 00:13:46,093 --> 00:13:49,096 BIAS ARE TO CLASSIFY BIAS ON THE 265 00:13:49,096 --> 00:13:56,704 COCHRAN RISK OF BIAS TOOL, THE 266 00:13:56,704 --> 00:14:04,378 NEWCASTLE OTTAWA SCALE, 267 00:14:04,378 --> 00:14:04,645 QUADAS-2. 268 00:14:04,645 --> 00:14:05,446 THESE ARE ALL NEW. 269 00:14:05,446 --> 00:14:08,515 THE GENERAL THOUGHT IS THIS. 270 00:14:08,515 --> 00:14:12,453 THE COCHRAN TOOL HAS ABOUT SIX 271 00:14:12,453 --> 00:14:14,355 TO NINE SECTIONS OR ELEMENTS 272 00:14:14,355 --> 00:14:16,123 THAT HAS TO BE SCORED. 273 00:14:16,123 --> 00:14:18,359 BASED ON THE OVERALL RISK OF 274 00:14:18,359 --> 00:14:22,896 BIAS OR QUALITY ASSESSMENT 275 00:14:22,896 --> 00:14:27,201 SCORE, THE RESEARCH TEAM WOULD 276 00:14:27,201 --> 00:14:29,436 THEN CONSIDER CLASSIFYING A 277 00:14:29,436 --> 00:14:33,540 STUDY AS HAVING LOW RISK OF 278 00:14:33,540 --> 00:14:40,881 BIAS, MODERATE RISK OF BIAS, FOR 279 00:14:40,881 --> 00:14:42,716 EACH INDIVIDUAL STUDY. 280 00:14:42,716 --> 00:14:44,184 THERE WOULD BE THIS OVERALL 281 00:14:44,184 --> 00:14:49,990 ASSESSMENT DONE FOR THE 282 00:14:49,990 --> 00:14:52,259 META-ANALYSIS THIS PROCESS IS 283 00:14:52,259 --> 00:14:54,628 USED WHEN PROBLEM TO HELP 284 00:14:54,628 --> 00:14:56,664 DETERMINE WHICH OF THE FOUR 285 00:14:56,664 --> 00:15:00,601 ARTICLES WILL BE INCLUDED IN THE 286 00:15:00,601 --> 00:15:02,870 META-ANALYSIS. 287 00:15:02,870 --> 00:15:08,642 288 00:15:08,642 --> 00:15:15,049 SO SHOWN HERE IS THE COCHRANE 289 00:15:15,049 --> 00:15:16,517 COLLABORATION TOOL FOR ASSESSING 290 00:15:16,517 --> 00:15:17,084 BIAS. 291 00:15:17,084 --> 00:15:19,386 YOU HAVE DOMAINS IN THE FIRST 292 00:15:19,386 --> 00:15:21,088 COLUMN. 293 00:15:21,088 --> 00:15:28,996 HERE WE HAVE SIX DOMAINS. 294 00:15:28,996 --> 00:15:31,131 SEQUENCE GENERATION, ALLOCATION 295 00:15:31,131 --> 00:15:32,966 CONCEALMENT, POSSIBLE BIAS FROM 296 00:15:32,966 --> 00:15:36,437 BLINDING, POSSIBLE BIAS DUE TO 297 00:15:36,437 --> 00:15:37,838 INCOMPLETE OUTCOME DATA, 298 00:15:37,838 --> 00:15:43,444 POSSIBLE BIAS DUE TO COLLECTIVE 299 00:15:43,444 --> 00:15:43,977 REPORTING. 300 00:15:43,977 --> 00:15:45,345 THE COCHRANE COLLABORATION 301 00:15:45,345 --> 00:15:46,613 TOOLS, SECOND COLUMN SHOWS 302 00:15:46,613 --> 00:15:48,348 SUPPORT OF JUDGMENT, GUIDE FOR 303 00:15:48,348 --> 00:15:49,416 RESEARCH TEAM. 304 00:15:49,416 --> 00:15:52,519 THIRD COLUMN SHOWS REVIEW OF 305 00:15:52,519 --> 00:15:53,253 AUTHOR'S JUDGMENT. 306 00:15:53,253 --> 00:15:55,255 THE AUTHOR HAS TO IN THE 307 00:15:55,255 --> 00:15:58,792 ARTICLE, IN THE META-ANALYSIS, 308 00:15:58,792 --> 00:16:00,461 PROVIDE JUSTIFICATION TO SHOW OR 309 00:16:00,461 --> 00:16:02,930 EXPLAIN, IT'S NOT AN EXACT 310 00:16:02,930 --> 00:16:04,798 SCIENCE, HOW THEY CLASSIFIED AN 311 00:16:04,798 --> 00:16:07,634 ARTICLE AS HAVING LOW RISK, 312 00:16:07,634 --> 00:16:11,004 MODERATE RISK, OR HIGH RISK OF 313 00:16:11,004 --> 00:16:18,746 BIAS. 314 00:16:18,746 --> 00:16:20,013 315 00:16:20,013 --> 00:16:20,714 DATA EXTRACTION. 316 00:16:20,714 --> 00:16:23,584 AGAIN, ALL THIS, AS YOU CAN SEE, 317 00:16:23,584 --> 00:16:25,152 THE META-ANALYSIS FOR THE MOST 318 00:16:25,152 --> 00:16:28,422 PART, I MAY SAY THIS TWO OR 319 00:16:28,422 --> 00:16:32,493 THREE ADDITIONAL TIMES, IS NOT A 320 00:16:32,493 --> 00:16:34,461 COMPLICATED APPROACH BUT DOES 321 00:16:34,461 --> 00:16:38,632 REQUIRE ATTENTION TO DETAIL. 322 00:16:38,632 --> 00:16:39,800 DATA EXTRACTION, I THINK THIS 323 00:16:39,800 --> 00:16:44,471 WAS STEP 10 IN THIS 14-STEP 324 00:16:44,471 --> 00:16:45,706 PROCESS, DATA EXTRACTION 325 00:16:45,706 --> 00:16:48,709 INVOLVES COLLECTING ONLY THE 326 00:16:48,709 --> 00:16:50,344 NECESSARY INFORMATION TO ANSWER 327 00:16:50,344 --> 00:16:53,113 THE RESEARCH QUESTION 328 00:16:53,113 --> 00:16:54,681 EFFECTIVELY AND ACCURATELY. 329 00:16:54,681 --> 00:16:56,917 SO ONE GOAL OF THE EXTRACTION IS 330 00:16:56,917 --> 00:16:59,520 TO COMPUTE A SUMMARY EFFECT, ITS 331 00:16:59,520 --> 00:17:03,123 MEASURE OF PRECISION AND A 332 00:17:03,123 --> 00:17:03,357 P-VALUE. 333 00:17:03,357 --> 00:17:05,659 I'LL SAY IN THEORY BECAUSE IT 334 00:17:05,659 --> 00:17:06,360 SOUNDS SIMPLE. 335 00:17:06,360 --> 00:17:08,195 THAT PART CAN BE VERY 336 00:17:08,195 --> 00:17:09,029 FRUSTRATING BECAUSE WE'RE 337 00:17:09,029 --> 00:17:11,231 WORKING BACKWARDS. 338 00:17:11,231 --> 00:17:15,369 WE HAVE A GOAL IN THIS REVIEW 339 00:17:15,369 --> 00:17:19,439 PROCESS OF BEING ABLE TO EXTRACT 340 00:17:19,439 --> 00:17:21,208 THESE EFFECTS. 341 00:17:21,208 --> 00:17:24,678 AND VERY SPECIFIC MEASURES OF 342 00:17:24,678 --> 00:17:29,149 PRECISION ON A VERY SPECIFIC 343 00:17:29,149 --> 00:17:30,784 MEASURE OF UNCERTAINTY. 344 00:17:30,784 --> 00:17:31,618 HOWEVER, FOR THE INDIVIDUAL 345 00:17:31,618 --> 00:17:33,220 STUDY IT'S VERY POSSIBLE THOSE 346 00:17:33,220 --> 00:17:36,390 MEASURES OF PRECISION COULD BE 347 00:17:36,390 --> 00:17:38,025 DIFFERENT. 348 00:17:38,025 --> 00:17:39,026 THEY COULD BE MISSING. 349 00:17:39,026 --> 00:17:45,265 YOU MAY OR MAY NOT HAVE A 350 00:17:45,265 --> 00:17:45,833 P-VALUE. 351 00:17:45,833 --> 00:17:47,668 SO, THAT'S FRUSTRATING. 352 00:17:47,668 --> 00:17:54,174 IT COULD IMPACT THE ABILITY TO 353 00:17:54,174 --> 00:18:00,147 INCLUDE THOSE STUDIES, AND THE 354 00:18:00,147 --> 00:18:01,281 META-ANALYSIS, STATISTICAL 355 00:18:01,281 --> 00:18:02,216 CALCULATIONS PART OF THIS 356 00:18:02,216 --> 00:18:02,983 PROCESS. 357 00:18:02,983 --> 00:18:03,250 ALL RIGHT. 358 00:18:03,250 --> 00:18:05,886 ARE WE READY FOR THE FIRST 359 00:18:05,886 --> 00:18:09,089 EXAMPLE? 360 00:18:09,089 --> 00:18:09,756 361 00:18:09,756 --> 00:18:10,858 THIS IS EXAMPLE 1. 362 00:18:10,858 --> 00:18:17,064 THIS IS EXAMPLE OF A STUDY FLOW 363 00:18:17,064 --> 00:18:17,297 DIAGRAM. 364 00:18:17,297 --> 00:18:24,838 FROM AN ARTICLE PUBLISHED IN THE 365 00:18:24,838 --> 00:18:29,276 COCHRANE LIBRARY. 366 00:18:29,276 --> 00:18:31,979 IT'S BOTULINUM TOXIN TYPE A 367 00:18:31,979 --> 00:18:33,080 THERAPY, DYSTONIA. 368 00:18:33,080 --> 00:18:34,147 WE'LL CAULK THROUGH DIFFERENT 369 00:18:34,147 --> 00:18:37,417 PARTS OF THE ABSTRACT. 370 00:18:37,417 --> 00:18:39,052 IT IS A SYSTEMATIC APPROACH, SO 371 00:18:39,052 --> 00:18:42,723 YOU CAN SEE REMNANTS OF THOSE 14 372 00:18:42,723 --> 00:18:47,160 STEPS, YOU'LL SEE IT HERE EVEN 373 00:18:47,160 --> 00:18:48,462 IN THE ABSTRACT. 374 00:18:48,462 --> 00:18:51,331 SO BACKGROUND INFORMATION FOR 375 00:18:51,331 --> 00:18:55,669 THIS PARTICULAR STUDY, UPDATE OF 376 00:18:55,669 --> 00:19:00,941 COCHRANE REVIEW PUBLISHED IN 377 00:19:00,941 --> 00:19:04,978 TWIEF, THE MOST COMMON FOLK OF 378 00:19:04,978 --> 00:19:09,049 FOCAL DYSTONIA CHARACTERIZED BY 379 00:19:09,049 --> 00:19:18,325 INVOLUNTARY, USUALLY PAINFUL, 380 00:19:18,325 --> 00:19:19,426 HEAD POSTURING. 381 00:19:19,426 --> 00:19:21,628 BTA IS CONSIDERED FIRST LINE OF 382 00:19:21,628 --> 00:19:22,829 THERAPY FOR THIS CONDITION. 383 00:19:22,829 --> 00:19:24,731 COME DOWN TO THE FIRST METHOD. 384 00:19:24,731 --> 00:19:25,465 THIS IS THE ABSTRACT. 385 00:19:25,465 --> 00:19:29,136 IN THE ABSTRACT YOU HAVE TO 386 00:19:29,136 --> 00:19:31,438 STATE TO IDENTIFY STUDIES FOR 387 00:19:31,438 --> 00:19:36,610 THIS REVIEW WE SEARCH THE 388 00:19:36,610 --> 00:19:40,547 COCHRANE MOVEMENT DISORDERS' 389 00:19:40,547 --> 00:19:43,884 TRIALS REGISTER, CENTRAL 390 00:19:43,884 --> 00:19:45,085 DATABASE, MEDLINE DATABASE, 391 00:19:45,085 --> 00:19:47,521 EMBASE DATABASE, REFERENCE LIST 392 00:19:47,521 --> 00:19:49,456 OF ARTICLES, AND COCHRANE 393 00:19:49,456 --> 00:19:50,991 PROCEEDINGS, THAT'S VERY 394 00:19:50,991 --> 00:19:55,329 IMPORTANT, THAT IS A SOURCE OF 395 00:19:55,329 --> 00:19:57,331 PUBLICATION BIAS THAT OCCURS 396 00:19:57,331 --> 00:20:02,970 WHEN THE RESEARCH TEAM ONLY 397 00:20:02,970 --> 00:20:08,208 SELECTS THOSE STUDIES FROM THE 398 00:20:08,208 --> 00:20:09,576 PEER-REVIEWED JOURNALS, ONLY 399 00:20:09,576 --> 00:20:12,446 SELECT STUDIES WITH SIGNIFICANT 400 00:20:12,446 --> 00:20:15,749 DIFFERENCES, IN THE EFFECT, 401 00:20:15,749 --> 00:20:20,454 COMPARISON BETWEEN TREATMENT 402 00:20:20,454 --> 00:20:23,590 GROUP AND PLACEBO GROUP. 403 00:20:23,590 --> 00:20:24,458 WITHOUT INCLUDING STUDIES, 404 00:20:24,458 --> 00:20:27,661 WHETHER IT DID NOT SHOW A 405 00:20:27,661 --> 00:20:28,595 STATISTICAL SIGNIFICANT 406 00:20:28,595 --> 00:20:28,996 DIFFERENCE. 407 00:20:28,996 --> 00:20:33,000 THAT IS A SOURCE OF PUBLICATION 408 00:20:33,000 --> 00:20:33,266 BIAS. 409 00:20:33,266 --> 00:20:35,602 SO WE CONTINUE HERE. 410 00:20:35,602 --> 00:20:39,039 I HAVE A FEW PARTS OF THIS 411 00:20:39,039 --> 00:20:40,374 ABSTRACT HIGHLIGHTED TO AGAIN 412 00:20:40,374 --> 00:20:46,179 SHOW THAT IN THIS 14-STEP 413 00:20:46,179 --> 00:20:47,614 PROCESS, AT DIFFERENT PHASES OF 414 00:20:47,614 --> 00:20:52,486 THE MANUSCRIPT, YOU HAVE TO 415 00:20:52,486 --> 00:20:54,688 CLEARLY STATE AND JUSTIFY THE 416 00:20:54,688 --> 00:20:57,791 PROCESS THAT WAS USED IN THIS 417 00:20:57,791 --> 00:20:58,325 META-ANALYSIS. 418 00:20:58,325 --> 00:21:02,295 SO HERE, FOR EXAMPLE, DATA 419 00:21:02,295 --> 00:21:03,363 COLLECTION AND ANALYSIS, 420 00:21:03,363 --> 00:21:05,699 HIGHLIGHTS SHOW TWO REVIEW 421 00:21:05,699 --> 00:21:07,034 AUTHORS INDEPENDENTLY ASSESSED 422 00:21:07,034 --> 00:21:10,270 THE RECORDS, SELECTED THE 423 00:21:10,270 --> 00:21:13,373 STUDIES, EXTRACTED DATA USING A 424 00:21:13,373 --> 00:21:17,210 PAPER PRO FORMA, EVALUATED RISK 425 00:21:17,210 --> 00:21:18,345 OF BIAS. 426 00:21:18,345 --> 00:21:23,016 RESULTS ARE HIGHLIGHTED. 427 00:21:23,016 --> 00:21:24,418 WE INCLUDED EIGHT RANDOMIZED 428 00:21:24,418 --> 00:21:26,219 CONTROLS TRIALS OF MODERATE 429 00:21:26,219 --> 00:21:27,254 OVERALL RISK OF BIAS. 430 00:21:27,254 --> 00:21:31,258 ALL OF THIS IS FAIRLY NEW, AND 431 00:21:31,258 --> 00:21:38,698 THIS IS A FAIRLY REPUTABLE 432 00:21:38,698 --> 00:21:39,833 JOURNAL, COCHRANE COLLABORATION 433 00:21:39,833 --> 00:21:40,067 LIBRARY. 434 00:21:40,067 --> 00:21:43,770 IT IS EXPECTED YOU WOULD USE 435 00:21:43,770 --> 00:21:45,739 THIS QUALITY ASSESSMENT TOOL OR 436 00:21:45,739 --> 00:21:49,376 RISK OF BIAS TOOL TO HELP 437 00:21:49,376 --> 00:21:52,245 DETERMINE THE PAPERS THAT WOULD 438 00:21:52,245 --> 00:21:58,185 BE INCLUDED IN THE META-ANALYSIS 439 00:21:58,185 --> 00:21:59,152 OR SYSTEMATIC REVIEW. 440 00:21:59,152 --> 00:22:05,058 HOPEFULLY YOU CAN SEE THIS. 441 00:22:05,058 --> 00:22:06,860 THIS IS THE FIRST FIGURE FROM 442 00:22:06,860 --> 00:22:07,761 THAT META-ANALYSIS. 443 00:22:07,761 --> 00:22:10,564 SO THIS IS THE FLOW DIAGRAM, 444 00:22:10,564 --> 00:22:13,934 HOPEFULLY YOU CAN SEE IT. 445 00:22:13,934 --> 00:22:16,503 IT SHOWS IN TOTAL 208 RECORDS 446 00:22:16,503 --> 00:22:20,607 WERE IDENTIFIED THROUGH THE 447 00:22:20,607 --> 00:22:22,242 CENTRAL DATABASE SEARCH. 448 00:22:22,242 --> 00:22:26,213 182 RECORDS WERE IDENTIFIED 449 00:22:26,213 --> 00:22:27,814 THROUGH MEDLINE. 450 00:22:27,814 --> 00:22:31,384 1256 RECORDS WERE IDENTIFIED 451 00:22:31,384 --> 00:22:38,091 THROUGH EMBASE DATABASE SEARCH. 452 00:22:38,091 --> 00:22:42,095 IN TOTAL 1,624 RECORDS 453 00:22:42,095 --> 00:22:43,096 IDENTIFIED, 1599 RECORDS 454 00:22:43,096 --> 00:22:44,231 INCLUDED AFTER DUPLICATES. 455 00:22:44,231 --> 00:22:48,001 CONTINUE TO DRILL DOWN. 456 00:22:48,001 --> 00:22:52,372 1569 RECORDS WERE IN TOTAL 457 00:22:52,372 --> 00:22:52,906 EXCLUDED. 458 00:22:52,906 --> 00:22:57,777 23 FULL-TEXT ARTICLES WERE 459 00:22:57,777 --> 00:23:00,247 ASSESSED FOR ELIGIBILITY. 460 00:23:00,247 --> 00:23:05,418 THERE'S 23 FROM THE INITIAL 1646 461 00:23:05,418 --> 00:23:06,052 RECORDS. 462 00:23:06,052 --> 00:23:10,490 OF THOSE 23 FULL TEXT ARTICLES, 463 00:23:10,490 --> 00:23:12,726 15 WERE EXCLUDED. 464 00:23:12,726 --> 00:23:17,631 SO THE RESULTING AT THE BOTTOM 465 00:23:17,631 --> 00:23:18,632 RESULTING NUMBER OF STUDIES 466 00:23:18,632 --> 00:23:21,701 INCLUDED FOR THIS META-ANALYSIS 467 00:23:21,701 --> 00:23:21,935 WERE 8. 468 00:23:21,935 --> 00:23:25,138 IT'S VERY IMPORTANT TO SHOW YOU 469 00:23:25,138 --> 00:23:28,842 CAST A WIDE NET TO EVALUATE THE 470 00:23:28,842 --> 00:23:31,077 LITERATURE ALL RELATED TO THAT 471 00:23:31,077 --> 00:23:33,880 PARTICULAR RESEARCH QUESTION. 472 00:23:33,880 --> 00:23:37,384 473 00:23:37,384 --> 00:23:40,854 THIS IS FIGURE 2 IN 474 00:23:40,854 --> 00:23:41,188 META-ANALYSIS. 475 00:23:41,188 --> 00:23:43,790 IN THIS META-ANALYSIS EXAMPLE 476 00:23:43,790 --> 00:23:47,761 RISK OF BIAS SUMMARY OF INCLUDED 477 00:23:47,761 --> 00:23:49,963 STUDIES IT PRESENTS THE AUTHOR'S 478 00:23:49,963 --> 00:23:52,065 JUDGMENT OVERALL ABOUT THE RISK 479 00:23:52,065 --> 00:23:57,904 OF BIAS FOR EACH OF THE DOMAINS. 480 00:23:57,904 --> 00:24:01,708 AT THE BOTTOM YOU SEE THE 481 00:24:01,708 --> 00:24:04,211 LEGEND, COLOR-CODED LEGEND. 482 00:24:04,211 --> 00:24:05,979 THE COLOR CODE OF GREEN 483 00:24:05,979 --> 00:24:09,783 INDICATES LOW RISK OF BIAS. 484 00:24:09,783 --> 00:24:12,485 COLOR CODE OF YELLOW INDICATES 485 00:24:12,485 --> 00:24:14,988 UNCLEAR RISK OF BIAS, WHICH 486 00:24:14,988 --> 00:24:16,823 WOULD HAPPEN QUITE OFTEN, FOR 487 00:24:16,823 --> 00:24:18,458 THE MOST PART. 488 00:24:18,458 --> 00:24:22,128 AND RED, SO THE COLOR CODE OF 489 00:24:22,128 --> 00:24:25,799 RED DENOTES A HIGH RISK OF BIAS. 490 00:24:25,799 --> 00:24:28,401 THIS RISK IS PRESENTED AS A 491 00:24:28,401 --> 00:24:30,036 PERCENTAGE, IN THIS EXAMPLE. 492 00:24:30,036 --> 00:24:33,273 SO 80% OF THE STUDIES WERE 493 00:24:33,273 --> 00:24:38,745 CLASSIFIED AS HAVING A HIGH RISK 494 00:24:38,745 --> 00:24:41,748 OF BIAS, FOR THE FOR HAD BEEN 495 00:24:41,748 --> 00:24:42,382 PROFIT DOMAIN. 496 00:24:42,382 --> 00:24:46,820 OFTEN WITH CLINICAL TRIALS 497 00:24:46,820 --> 00:24:48,355 INVOLVING TREATMENTS, THOSE 498 00:24:48,355 --> 00:24:52,025 TRIALS ARE FUNDED BY THE 499 00:24:52,025 --> 00:24:54,628 INDUSTRY, PHARMA INDUSTRY OR 500 00:24:54,628 --> 00:24:55,428 FOR-PROFIT INDUSTRY, WHICH IS 501 00:24:55,428 --> 00:24:58,231 GREAT BUT IT CAN ALSO BE A 502 00:24:58,231 --> 00:25:01,001 SOURCE OF WHAT'S CALLED 503 00:25:01,001 --> 00:25:08,908 FOR-PROFIT BIAS. 504 00:25:08,908 --> 00:25:09,209 505 00:25:09,209 --> 00:25:11,611 SHOWN ON THIS SLIDE IS FIGURE 506 00:25:11,611 --> 00:25:15,148 NUMBER 3, IN OUR EXAMPLE 507 00:25:15,148 --> 00:25:15,482 META-ANALYSIS. 508 00:25:15,482 --> 00:25:19,519 A RISK OF BIAS SUMMARY IS 509 00:25:19,519 --> 00:25:21,121 PROVIDED FOR EACH INDIVIDUAL 510 00:25:21,121 --> 00:25:22,055 STUDY. 511 00:25:22,055 --> 00:25:30,830 AGAIN USING THE SAME COLOR CODE 512 00:25:30,830 --> 00:25:34,034 HOPEFULLY YOU CAN SEE THIS. 513 00:25:34,034 --> 00:25:37,837 THIS IS ACTUALLY A SUMMARY OF 514 00:25:37,837 --> 00:25:41,041 THE EFFECT SIZE. 515 00:25:41,041 --> 00:25:43,843 USING RANDOM EFFECTS 516 00:25:43,843 --> 00:25:46,446 DESTINATION, THIS RESEARCH TEAM 517 00:25:46,446 --> 00:25:48,882 HERE SHOWS OR ILLUSTRATES FOR 518 00:25:48,882 --> 00:25:50,116 EACH STUDY, AND HERE WE'RE 519 00:25:50,116 --> 00:25:53,687 LOOKING AT JUST SEVEN STUDIES, 520 00:25:53,687 --> 00:25:59,993 THE RESULTS FOR THE COMPARISON 521 00:25:59,993 --> 00:26:03,196 OF BOTULINUM TOXIN TYPE A VERSUS 522 00:26:03,196 --> 00:26:07,033 PLACEBO, OUTCOME MEASURE IS 523 00:26:07,033 --> 00:26:08,201 IMPROVEMENT OF CERVICAL 524 00:26:08,201 --> 00:26:08,668 DYSTONIA. 525 00:26:08,668 --> 00:26:10,437 FOR THESE ANALYSES THE RESEARCH 526 00:26:10,437 --> 00:26:14,441 TEAM USES AN APPROACH CALLED 527 00:26:14,441 --> 00:26:19,279 TWISTERS, AN INDEX THAT CAPTURES 528 00:26:19,279 --> 00:26:20,380 SOME TYPE OF IMPROVEMENT. 529 00:26:20,380 --> 00:26:21,681 ALL RIGHT. 530 00:26:21,681 --> 00:26:25,285 SECOND AND THIRD COLUMNS YOU 531 00:26:25,285 --> 00:26:28,188 HAVE THE ACTUAL NUMBER OF 532 00:26:28,188 --> 00:26:29,889 PARTICIPANTS IN THE TREATMENT 533 00:26:29,889 --> 00:26:31,658 GROUP FOR EACH STUDY, NUMBER OF 534 00:26:31,658 --> 00:26:35,195 PARTICIPANTS THAT WERE IN THE 535 00:26:35,195 --> 00:26:40,300 PLACEBO GROUP, THE FOURTH COLUMN 536 00:26:40,300 --> 00:26:40,900 ILLUSTRATES THE STANDARDIZED 537 00:26:40,900 --> 00:26:43,303 MEAN DIFFERENCE. 538 00:26:43,303 --> 00:26:46,639 THAT'S THE EFFECT SHOWN FOR EACH 539 00:26:46,639 --> 00:26:47,974 INDIVIDUAL STUDY. 540 00:26:47,974 --> 00:26:53,847 THE GOAL HERE WOULD BE TO 541 00:26:53,847 --> 00:26:56,850 SOMEHOW SYNTHESIZE THESE RESULTS 542 00:26:56,850 --> 00:26:58,818 TO CALCULATE AN OVERALL EFFECT. 543 00:26:58,818 --> 00:27:01,287 ALL RIGHT? 544 00:27:01,287 --> 00:27:06,960 FIFTH COLUMN SHOWS THE FOREST 545 00:27:06,960 --> 00:27:08,762 PLOT, EFFECT SIZE, CORRESPONDING 546 00:27:08,762 --> 00:27:09,963 CONFIDENCE INTERVAL. 547 00:27:09,963 --> 00:27:14,601 FOREST PLOTS ARE USED TO SHOW 548 00:27:14,601 --> 00:27:16,102 THE EXTENT OR GAUGE 549 00:27:16,102 --> 00:27:19,139 HETEROGENEITY THAT EXISTS 550 00:27:19,139 --> 00:27:21,875 BETWEEN THESE STUDIES. 551 00:27:21,875 --> 00:27:24,711 IN THEORY, IN THEORY, IF YOU DO 552 00:27:24,711 --> 00:27:27,680 NOT SEE MOVEMENT, IF THE EFFECTS 553 00:27:27,680 --> 00:27:30,750 BETWEEN THE STUDIES, LOOKING AT 554 00:27:30,750 --> 00:27:36,289 THE PLOT, ALL LINES UP, 555 00:27:36,289 --> 00:27:37,891 UNDERNEATH EACH OTHER, AND YOU 556 00:27:37,891 --> 00:27:41,694 DON'T HAVE OVERLAP, THIS IS A 557 00:27:41,694 --> 00:27:44,030 CLEAR SIGN THAT VARIATION 558 00:27:44,030 --> 00:27:46,199 BETWEEN STUDIES IS ALMOST ZERO, 559 00:27:46,199 --> 00:27:47,934 CLOSE TO ZERO. 560 00:27:47,934 --> 00:27:50,804 YOU COULD THEN USE AN APPROACH 561 00:27:50,804 --> 00:27:53,306 CALLED FIXED EFFECTS ANALYSIS, 562 00:27:53,306 --> 00:27:55,275 FIXED EFFECTS MODELS IN MOST 563 00:27:55,275 --> 00:27:56,709 CASES THAT WOULD NOT BE GOOD. 564 00:27:56,709 --> 00:28:00,413 SO EVEN IN THIS CASE YOU DO SEE 565 00:28:00,413 --> 00:28:03,817 THERE'S SOME MOVEMENT, THE 566 00:28:03,817 --> 00:28:06,186 EFFECTS ARE MOVING SLIGHTLY BUT 567 00:28:06,186 --> 00:28:09,189 ENOUGH TO JUSTIFY USING WHAT WE 568 00:28:09,189 --> 00:28:10,156 CALL RANDOM EFFECTS MODELS AND 569 00:28:10,156 --> 00:28:14,327 WE'LL TAKE A CLOSE LOOK AT THAT 570 00:28:14,327 --> 00:28:17,831 IN THE NEXT TWO SLIDES. 571 00:28:17,831 --> 00:28:21,968 ALL RIGHT. 572 00:28:21,968 --> 00:28:26,172 SO, THE CALCULATION OF THIS 573 00:28:26,172 --> 00:28:28,675 OVERALL EFFECT, SUMMARY EFFECT, 574 00:28:28,675 --> 00:28:32,879 IS DONE BY ASSIGNING A WEIGHT, A 575 00:28:32,879 --> 00:28:35,148 WEIGHT TO EACH STUDY, SO THAT IS 576 00:28:35,148 --> 00:28:37,450 SHOWN IN THE FIFTH COLUMN. 577 00:28:37,450 --> 00:28:40,186 WE WILL LOOK AT BACK OF ENVELOPE 578 00:28:40,186 --> 00:28:45,091 CALCULATIONS TO SEE HOW THAT IS 579 00:28:45,091 --> 00:28:50,497 DONE, THIS BUT THIS WEIGHT IS 580 00:28:50,497 --> 00:28:52,098 INVERSELY RELATED TO PRECISION 581 00:28:52,098 --> 00:28:54,901 OF THE EFFECT ON THE ESTIMATE. 582 00:28:54,901 --> 00:28:57,070 THE PRECISION IS LIKE THE 583 00:28:57,070 --> 00:29:00,974 STANDARD ERROR. 584 00:29:00,974 --> 00:29:03,343 SO, STUDIES HAVING SMALL 585 00:29:03,343 --> 00:29:05,178 STANDARD ERRORS CORRESPOND TO 586 00:29:05,178 --> 00:29:08,014 THOSE STUDIES HAVING A LARGE 587 00:29:08,014 --> 00:29:10,884 SAMPLE SIZE, SO THOSE STUDIES 588 00:29:10,884 --> 00:29:12,519 WILL BE GIVEN MORE WEIGHT. 589 00:29:12,519 --> 00:29:14,420 SO THEN WE WOULD THEN USE THOSE 590 00:29:14,420 --> 00:29:16,456 WEIGHTS TO APPLY TO EACH OF THE 591 00:29:16,456 --> 00:29:18,458 EFFECTS, AND THEN TAKE THE 592 00:29:18,458 --> 00:29:19,125 WEIGHTED AVERAGE. 593 00:29:19,125 --> 00:29:21,928 AND THEN THAT'S WHAT YOU HAVE AT 594 00:29:21,928 --> 00:29:25,031 THE BOTTOM. 595 00:29:25,031 --> 00:29:29,435 SHOWN HERE IS THE SUMMARIZED 596 00:29:29,435 --> 00:29:30,937 EFFECT AND CORRESPONDING 597 00:29:30,937 --> 00:29:31,604 CONFIDENCE INTERVAL. 598 00:29:31,604 --> 00:29:34,007 SO, AGAIN, WE'RE GOING TO 599 00:29:34,007 --> 00:29:35,375 ACTUALLY WALK THROUGH AND 600 00:29:35,375 --> 00:29:37,410 REPLICATE THESE ANALYSES, THEY 601 00:29:37,410 --> 00:29:38,244 ARE NOT VERY DIFFICULT. 602 00:29:38,244 --> 00:29:41,781 JUST TO MAKE SURE THAT YOU HAVE 603 00:29:41,781 --> 00:29:44,250 AN UNDERSTANDING OF WHAT IS DONE 604 00:29:44,250 --> 00:29:48,788 IN THE BACKGROUND FOR MOST 605 00:29:48,788 --> 00:29:51,057 SOFTWARE PACKAGES. 606 00:29:51,057 --> 00:29:53,259 SHOWN IN THE BOTTOM LEFT-HAND 607 00:29:53,259 --> 00:29:58,398 CORNER ARE YOUR MEASUREMENTS OF 608 00:29:58,398 --> 00:29:59,265 HETEROGENEITY. 609 00:29:59,265 --> 00:30:01,968 SO HETEROGENEITY OCCURS WHEN 610 00:30:01,968 --> 00:30:05,471 YOU -- THE EFFECTS ARE MOVING. 611 00:30:05,471 --> 00:30:07,507 SO THERE'S SOME RANDOM 612 00:30:07,507 --> 00:30:09,142 DIFFERENCES IN THE POPULATIONS 613 00:30:09,142 --> 00:30:12,946 THAT CAN BE DIFFERENCES IN HOW 614 00:30:12,946 --> 00:30:14,147 THOSE STUDIES WERE CONDUCTED, 615 00:30:14,147 --> 00:30:15,915 ALL OF THOSE THINGS COULD RESULT 616 00:30:15,915 --> 00:30:21,354 IN THE EFFECTS BEING SLIGHTLY 617 00:30:21,354 --> 00:30:21,754 DIFFERENT. 618 00:30:21,754 --> 00:30:24,524 SO WE ARE POOLING, TAKING A 619 00:30:24,524 --> 00:30:26,059 WEIGHTED AVERAGE, OF THESE 620 00:30:26,059 --> 00:30:28,528 EFFECTS OF INDIVIDUAL STUDIES. 621 00:30:28,528 --> 00:30:36,603 WE CAN JUSTIFY DOING THIS, IF 622 00:30:36,603 --> 00:30:38,104 THE HETEROGENEITY IS REASONABLE. 623 00:30:38,104 --> 00:30:38,538 REASONABLE. 624 00:30:38,538 --> 00:30:44,010 SO YOU WANT TO MAKE SURE THAT 625 00:30:44,010 --> 00:30:46,746 YOU ARE POOLING OR TAKING THE 626 00:30:46,746 --> 00:30:49,716 WEIGHTED AVERAGE OF EFFECTS THAT 627 00:30:49,716 --> 00:30:50,383 ARE SOMEWHAT SIMILAR. 628 00:30:50,383 --> 00:30:58,791 AND SO WE'LL LOOK AT SOME OF 629 00:30:58,791 --> 00:31:01,628 THESE MEASUREMENTS TAU SQUARED, 630 00:31:01,628 --> 00:31:03,796 CHI SQUARED, ITEMS SHOWN HERE 631 00:31:03,796 --> 00:31:07,433 ARE EXPECTED IN THE 632 00:31:07,433 --> 00:31:08,267 META-ANALYSIS, THIS IS EXHIBIT 633 00:31:08,267 --> 00:31:08,868 8. 634 00:31:08,868 --> 00:31:13,673 WHAT WOULD BE EXHIBITED WHEN 635 00:31:13,673 --> 00:31:18,845 SUBMITTING YOUR ANALYSES TO 636 00:31:18,845 --> 00:31:20,747 ADJOURN. 637 00:31:20,747 --> 00:31:23,216 BEFORE WE LOOK AT BACK OF THE 638 00:31:23,216 --> 00:31:24,317 ENVELOPE CALCULATIONS I WANT TO 639 00:31:24,317 --> 00:31:27,186 JUST STAY A FEW WORDS ABOUT 640 00:31:27,186 --> 00:31:30,356 HETEROGENEITY. 641 00:31:30,356 --> 00:31:32,892 HETEROGENEITY REFERS TO 642 00:31:32,892 --> 00:31:35,161 VARIABILITY IN STUDY 643 00:31:35,161 --> 00:31:37,964 POPULATIONS, INTERVENTIONS, OR 644 00:31:37,964 --> 00:31:44,237 OUTCOMES IN SYSTEMATIC REVIEWS. 645 00:31:44,237 --> 00:31:50,643 COCHRAN'S Q TEST, TAU SQUARED 646 00:31:50,643 --> 00:31:56,315 USED TO ASSESS HETEROGENEITY. 647 00:31:56,315 --> 00:32:01,254 I SQUARED AND TAU SQUARED ARE 648 00:32:01,254 --> 00:32:06,793 FUNCTIONS OF THE COCHRANE Q 649 00:32:06,793 --> 00:32:08,261 TEST, THE ONLY TRADITIONAL TERT 650 00:32:08,261 --> 00:32:10,763 OF HETEROGENEITY IN THAT 651 00:32:10,763 --> 00:32:12,198 ANALYSIS. 652 00:32:12,198 --> 00:32:14,600 BASED ON TIME SQUARED 653 00:32:14,600 --> 00:32:16,703 DISTRIBUTION GENERATES THE 654 00:32:16,703 --> 00:32:18,171 PROBABILITY WHEN THAT STATISTIC 655 00:32:18,171 --> 00:32:20,606 IS LARGE IT INDICATES LARGE 656 00:32:20,606 --> 00:32:23,810 VARIATION ACROSS STUDIES, RATHER 657 00:32:23,810 --> 00:32:30,016 THAN WITHIN SUBJECTS, OF THE 658 00:32:30,016 --> 00:32:30,950 INDIVIDUAL STUDIES. 659 00:32:30,950 --> 00:32:34,320 UNDERLYING HYPOTHESIS ASSUMES 660 00:32:34,320 --> 00:32:37,190 THE TRUE EFFECT IS THE SAME 661 00:32:37,190 --> 00:32:42,361 ACROSS STUDIES. 662 00:32:42,361 --> 00:32:46,265 BORENSTEIN HAS A VERY GOOD BOOK, 663 00:32:46,265 --> 00:32:47,967 "COMMON MISTAKES WHEN MAKING A 664 00:32:47,967 --> 00:32:48,267 DECISION." 665 00:32:48,267 --> 00:32:50,069 AT SOME POINT THE RESEARCH TEAM 666 00:32:50,069 --> 00:32:54,440 HAS TO MAKE A DECISION TO USE 667 00:32:54,440 --> 00:32:56,109 RANDOM EFFECTS ESTIMATION, 668 00:32:56,109 --> 00:33:00,980 RANDOM EFFECTS MODELS, OR FIXED 669 00:33:00,980 --> 00:33:01,714 EFFECTS MODELS. 670 00:33:01,714 --> 00:33:03,449 IT'S NOT A SETTLED AREA. 671 00:33:03,449 --> 00:33:06,719 THERE'S DIFFERENT THOUGHTS ABOUT 672 00:33:06,719 --> 00:33:07,086 THIS. 673 00:33:07,086 --> 00:33:12,458 SO WE WILL SEE, LOOKING AT THE 674 00:33:12,458 --> 00:33:15,061 ACTUAL ANALYTIC FORMULAS. 675 00:33:15,061 --> 00:33:19,432 IT DOESN'T HURT TO ASSUME A 676 00:33:19,432 --> 00:33:23,102 RANDOM EFFECTS MODEL BECAUSE IN 677 00:33:23,102 --> 00:33:27,240 THE ABSENCE OF HETEROGENEITY, 678 00:33:27,240 --> 00:33:28,975 THOSE ESTIMATES CONVERGE OR MOVE 679 00:33:28,975 --> 00:33:32,044 TOWARDS THE ESTIMATE THAT YOU 680 00:33:32,044 --> 00:33:35,948 WOULD GET WHEN USING THE FIXED 681 00:33:35,948 --> 00:33:37,283 EFFECTS MODEL. 682 00:33:37,283 --> 00:33:39,252 SO, IT'S AN AREA THAT'S NOT 683 00:33:39,252 --> 00:33:42,088 SETTLED BUT I THINK IT'S GOOD TO 684 00:33:42,088 --> 00:33:46,959 NOT OVERTHINK THIS PROCESS. 685 00:33:46,959 --> 00:33:48,628 ALL RIGHT. 686 00:33:48,628 --> 00:33:50,863 ARE WE READY FOR CALCULATIONS? 687 00:33:50,863 --> 00:33:55,701 THESE CALCULATIONS, THIS IS THE 688 00:33:55,701 --> 00:33:57,370 EXAMPLE 3. 689 00:33:57,370 --> 00:34:01,707 THESE CALCULATIONS ARE FAIRLY 690 00:34:01,707 --> 00:34:03,576 SIMPLE. 691 00:34:03,576 --> 00:34:04,977 I REPLICATED THE ANALYSIS, JUST 692 00:34:04,977 --> 00:34:09,081 SHOWN TO YOU IN THE ARTICLE, 693 00:34:09,081 --> 00:34:11,317 PUBLISHED BY THE COCHRANE 694 00:34:11,317 --> 00:34:11,851 COLLABORATION LIBRARY. 695 00:34:11,851 --> 00:34:13,319 BASED ON THE INFORMATION THAT 696 00:34:13,319 --> 00:34:14,287 WAS PROVIDED. 697 00:34:14,287 --> 00:34:17,023 SO THAT'S THE GOAL THAT YOU'RE 698 00:34:17,023 --> 00:34:19,959 PROVIDING ENOUGH INFORMATION IN 699 00:34:19,959 --> 00:34:21,861 THE META-ANALYSIS THAT ANYONE 700 00:34:21,861 --> 00:34:27,600 COULD QUICKLY REPLICATE YOUR 701 00:34:27,600 --> 00:34:27,967 ANALYSES. 702 00:34:27,967 --> 00:34:32,405 SO, IN THIS PARTICULAR SLIDE YOU 703 00:34:32,405 --> 00:34:35,107 SEE THE INDIVIDUAL STUDY, YOU 704 00:34:35,107 --> 00:34:39,946 SEE THE NUMBER OF RESPONDENTS 705 00:34:39,946 --> 00:34:47,420 TREATED WITH BTA. 706 00:34:47,420 --> 00:34:51,858 BOTULINUM TOXIN TYPE A, NUMBER 707 00:34:51,858 --> 00:34:55,962 TREATED WITH PLACEBO, FOURTH 708 00:34:55,962 --> 00:34:58,097 COLUMN SHOWS STANDARDIZED 709 00:34:58,097 --> 00:35:00,466 DIFFERENCE, TWISTER SCORES, 710 00:35:00,466 --> 00:35:01,667 CORRESPONDING ERROR, 711 00:35:01,667 --> 00:35:04,670 STANDARDIZED MEAN DIFFERENCE. 712 00:35:04,670 --> 00:35:10,109 THE SIXTH COLUMN IS THE SQUARE 713 00:35:10,109 --> 00:35:13,045 OF THE STANDARD ERROR. 714 00:35:13,045 --> 00:35:15,147 AND THEN THE BETWEEN-STUDY 715 00:35:15,147 --> 00:35:16,849 VARIANTS, THIS IS TAU SQUARED. 716 00:35:16,849 --> 00:35:19,085 SO WE'RE LOOKING AT CALCULATION 717 00:35:19,085 --> 00:35:19,819 OF TAU SQUARED. 718 00:35:19,819 --> 00:35:24,056 IT IS A FUNCTION OF THE COCHRANE 719 00:35:24,056 --> 00:35:24,924 Q STATISTIC. 720 00:35:24,924 --> 00:35:28,294 ALL OF THIS INFORMATION IS USED 721 00:35:28,294 --> 00:35:30,897 SO WE CAN CALCULATE A WEIGHT, SO 722 00:35:30,897 --> 00:35:35,735 THE WEIGHT HAS TO BE BASED ON A 723 00:35:35,735 --> 00:35:38,504 FUNCTION OF THE VARIANCE, IN 724 00:35:38,504 --> 00:35:40,606 THIS CASE IT IS THE TOTAL 725 00:35:40,606 --> 00:35:41,240 VARIANCE. 726 00:35:41,240 --> 00:35:42,808 TOTAL VARIANCE MEANS THE 727 00:35:42,808 --> 00:35:44,076 VARIANCE WITHIN THE STUDIES AND 728 00:35:44,076 --> 00:35:46,178 VARIANCE BETWEEN THE STUDIES. 729 00:35:46,178 --> 00:35:48,514 TOTAL VARIANCE. 730 00:35:48,514 --> 00:35:51,617 THE NEXT COLUMN YOU HAVE THE 731 00:35:51,617 --> 00:35:52,652 ACTUAL WEIGHTS. 732 00:35:52,652 --> 00:35:56,255 AND THEN THE SECOND TO THE LAST 733 00:35:56,255 --> 00:35:58,291 COLUMN YOU HAVE THE WEIGHTS AS 734 00:35:58,291 --> 00:36:00,726 PERTAINS TO WHAT PERCENTAGE OF 735 00:36:00,726 --> 00:36:05,197 OF THE OVERALL WEIGHT IS 736 00:36:05,197 --> 00:36:06,565 ATTRIBUTED TO EACH STUDY. 737 00:36:06,565 --> 00:36:10,670 THEN THE LAST COLUMN SHOWS THE 738 00:36:10,670 --> 00:36:13,806 PRODUCT OF THE WEIGHT TIMES THE 739 00:36:13,806 --> 00:36:16,943 EFFECT IN EACH STUDY. 740 00:36:16,943 --> 00:36:23,182 YOU SUM THOSE UP. 741 00:36:23,182 --> 00:36:27,353 AND THEN DIVIDE BY THE ACTUAL 742 00:36:27,353 --> 00:36:30,923 POPULATION SIZE TO GET THE 743 00:36:30,923 --> 00:36:32,992 RANDOM EFFECTS ESTIMATION -- 744 00:36:32,992 --> 00:36:33,292 ESTIMATE. 745 00:36:33,292 --> 00:36:37,363 IN THIS CASE, I CAME PRETTY 746 00:36:37,363 --> 00:36:39,865 CLOSE. 747 00:36:39,865 --> 00:36:42,201 SO OUR EFFECT SHOWN HERE IS 748 00:36:42,201 --> 00:36:46,672 ABOUT 6.9, IN -- I THINK IN THE 749 00:36:46,672 --> 00:36:52,878 ARTICLE IT'S 6.7. 750 00:36:52,878 --> 00:36:57,283 STANDARD ERROR .08, AND THEN THE 751 00:36:57,283 --> 00:36:58,651 CONFIDENCE LIMIT, LOW CONFIDENCE 752 00:36:58,651 --> 00:36:59,719 BALANCE OR UPPER CONFIDENCE 753 00:36:59,719 --> 00:37:01,854 BALANCE ARE IDENTICAL TO WHAT 754 00:37:01,854 --> 00:37:05,124 WAS PUBLISHED IN THE 755 00:37:05,124 --> 00:37:06,092 META-ANALYSIS. 756 00:37:06,092 --> 00:37:10,496 757 00:37:10,496 --> 00:37:12,999 THIS SLIDE SHOWS THE ACTUAL 758 00:37:12,999 --> 00:37:15,468 FORMULAS. 759 00:37:15,468 --> 00:37:21,907 760 00:37:21,907 --> 00:37:24,343 761 00:37:24,343 --> 00:37:25,478 SO THE WEIGHTS, INVERSELY 762 00:37:25,478 --> 00:37:27,680 RELATED TO VARIANCE, IN THIS 763 00:37:27,680 --> 00:37:31,017 CASE FIXED EFFECTS. 764 00:37:31,017 --> 00:37:34,553 THOSE WHO ARE NOT FAMILIAR, SOME 765 00:37:34,553 --> 00:37:40,693 OF THIS, SYMBOLS AND NOTATIONS, 766 00:37:40,693 --> 00:37:44,397 SIGMA SUMMATION REFERS TO WE'RE 767 00:37:44,397 --> 00:37:45,097 ADDING. 768 00:37:45,097 --> 00:37:52,338 MATHEMATICAL JARGON FOR ADDING. 769 00:37:52,338 --> 00:37:56,142 I IS STUDY. 770 00:37:56,142 --> 00:37:59,745 SO W IS RATIO OF 1 OVER THE 771 00:37:59,745 --> 00:38:03,382 VARIANTS, WEIGHTED MEAN WILL BE 772 00:38:03,382 --> 00:38:04,683 THE AVERAGE. 773 00:38:04,683 --> 00:38:08,320 THE WEIGHT APPLIES TO THE EFFECT 774 00:38:08,320 --> 00:38:12,992 OF EACH STUDY, DIVIDED BY TOTAL 775 00:38:12,992 --> 00:38:13,559 WEIGHT. 776 00:38:13,559 --> 00:38:17,063 SUM FROM ONE TO K, IN OUR CASE 777 00:38:17,063 --> 00:38:18,697 EXAMPLE THERE WERE ONLY SEVEN 778 00:38:18,697 --> 00:38:20,599 STUDIES, SUM FROM THE FIRST 779 00:38:20,599 --> 00:38:26,038 STUDY THROUGH THE SEVENTH STUDY. 780 00:38:26,038 --> 00:38:32,111 THE THIRD IS VARIANCE OF SUMMARY 781 00:38:32,111 --> 00:38:33,079 EFFECT. 782 00:38:33,079 --> 00:38:35,247 ONE OVER SUMMATION. 783 00:38:35,247 --> 00:38:37,216 FOURTH STANDARD ERROR. 784 00:38:37,216 --> 00:38:38,551 SUMMARY EFFECT. 785 00:38:38,551 --> 00:38:41,487 AND THEN THE CONFIDENCE 786 00:38:41,487 --> 00:38:41,754 INTERVAL. 787 00:38:41,754 --> 00:38:49,462 IT CAN BE SHOWN THAT THE RANDOM 788 00:38:49,462 --> 00:38:51,564 EFFECTS ESTIMATE, THIS OVERALL 789 00:38:51,564 --> 00:38:54,600 EFFECT IS NORMAL DISTRIBUTED, 790 00:38:54,600 --> 00:38:56,202 SHOWN THE CONFIDENCE INTERVAL IS 791 00:38:56,202 --> 00:38:58,871 PROVIDED BY THE POINT ESTIMATE 792 00:38:58,871 --> 00:39:03,342 PLUS OR MINUS MARGIN OF ERROR, 793 00:39:03,342 --> 00:39:07,046 AND 1.96 TIMES STANDARD ERROR OF 794 00:39:07,046 --> 00:39:09,582 THE EFFECT. 795 00:39:09,582 --> 00:39:12,551 796 00:39:12,551 --> 00:39:17,156 IN THE PRESENCE OF 797 00:39:17,156 --> 00:39:18,624 HETEROGENEITY, THE CALCULATIONS 798 00:39:18,624 --> 00:39:21,360 ARE A LITTLE DIFFERENT. 799 00:39:21,360 --> 00:39:22,795 THE MEASUREMENT TAU SQUARED 800 00:39:22,795 --> 00:39:25,998 WHICH MEASURES, THIS IS THE 801 00:39:25,998 --> 00:39:29,635 SCRIPTIVE STATISTICS THAT 802 00:39:29,635 --> 00:39:32,571 MEASURED PERCENTAGE OF VARIATION 803 00:39:32,571 --> 00:39:35,774 DUE TO DIFFERENCES BETWEEN THE 804 00:39:35,774 --> 00:39:40,412 STUDIES, IS THE RATIO OF Q TO 805 00:39:40,412 --> 00:39:42,448 COCHRANE Q MINUS DEGREES OF 806 00:39:42,448 --> 00:39:44,083 FREEDOM DIVIDED BY C. 807 00:39:44,083 --> 00:39:48,721 WE HAVE THE ANALYTIC FORMULA FOR 808 00:39:48,721 --> 00:39:49,088 Q. 809 00:39:49,088 --> 00:39:50,289 IT'S GIVEN. 810 00:39:50,289 --> 00:39:53,726 THE DEGREES OF FREEDOM, NUMBER 811 00:39:53,726 --> 00:39:57,229 OF FEE VAIRALS, NUMBER OF 812 00:39:57,229 --> 00:40:01,066 STUDIES K MINUS 1. 813 00:40:01,066 --> 00:40:02,868 C IS ALSO GIVEN. 814 00:40:02,868 --> 00:40:05,137 SO THE RANDOM EFFECTS 815 00:40:05,137 --> 00:40:08,440 STIMULATION IS GOING TO BE 1 816 00:40:08,440 --> 00:40:10,709 OVER THE WEIGHT, 1 OVER THE 817 00:40:10,709 --> 00:40:12,811 VARIANCE WITHIN STUDIES PLUS TAU 818 00:40:12,811 --> 00:40:18,517 SQUARED, TAU SQUARED IS THE 819 00:40:18,517 --> 00:40:21,353 COMPONENT THAT DESCRIBES THE 820 00:40:21,353 --> 00:40:24,089 VARIANCE DUE TO HOMOGENEITY OR 821 00:40:24,089 --> 00:40:28,727 DIFFERENCES BETWEEN THE STUDIES. 822 00:40:28,727 --> 00:40:35,067 THE LAST ONE SHOWS THE WEIGHTED 823 00:40:35,067 --> 00:40:35,267 MEAN. 824 00:40:35,267 --> 00:40:36,635 AGAIN, COMPLICATIONS ARE NOT 825 00:40:36,635 --> 00:40:38,137 THAT OVERLY COMPLICATED. 826 00:40:38,137 --> 00:40:45,978 YOU CAN DO IT IN EXCEL. 827 00:40:45,978 --> 00:40:47,613 COMPREHENSIVE META-ANALYSIS, IT 828 00:40:47,613 --> 00:40:50,282 IS VERY IMPORTANT FOR THE MOST 829 00:40:50,282 --> 00:40:54,954 PART, I WOULD SAY USE A PACKAGE, 830 00:40:54,954 --> 00:40:57,723 SO CMA WAS DEVELOPED IN PART BY 831 00:40:57,723 --> 00:41:03,262 FUNDING THROUGH CDC, AND NIH. 832 00:41:03,262 --> 00:41:07,299 IT IS COMMONLY USED AS A 833 00:41:07,299 --> 00:41:09,401 PACKAGE, AND SO IF YOU INFORM 834 00:41:09,401 --> 00:41:11,904 SOMEONE YOU USE A PARTICULAR 835 00:41:11,904 --> 00:41:15,007 PACKAGE, IT ISN'T JUST SO EASY 836 00:41:15,007 --> 00:41:20,446 FOR THEM TO REPLICATE YOUR 837 00:41:20,446 --> 00:41:20,679 RESULTS. 838 00:41:20,679 --> 00:41:21,647 ALL RIGHT. 839 00:41:21,647 --> 00:41:28,354 I SQUARED IS SIMILAR TO TAU 840 00:41:28,354 --> 00:41:31,357 SQUARED, DESCRIBES PERCENTAGE OF 841 00:41:31,357 --> 00:41:34,226 VARIATION ACROSS STUDIES, THAT 842 00:41:34,226 --> 00:41:38,297 IS DUE TO HETEROGENEITY, RATHER 843 00:41:38,297 --> 00:41:38,998 THAN CHANCE. 844 00:41:38,998 --> 00:41:46,872 I'M SAYING IT'S NEW, IT'S NEW TO 845 00:41:46,872 --> 00:41:48,807 ME. 846 00:41:48,807 --> 00:41:51,677 2002, 2003, HIGGINS ET AL., IT'S 847 00:41:51,677 --> 00:41:53,779 EXPECTED IN THE META-ANALYSIS 848 00:41:53,779 --> 00:41:59,451 YOU NOT ONLY PROVIDE SUMMARIZED 849 00:41:59,451 --> 00:42:01,654 EFFECT, SUM MEASUREMENT OF 850 00:42:01,654 --> 00:42:04,623 UNCERTAINTY, KV DENSER INTERVAL, 851 00:42:04,623 --> 00:42:07,993 P-VALUE, INCLUDE MEASUREMENTS OF 852 00:42:07,993 --> 00:42:08,961 HETEROGENEITY. 853 00:42:08,961 --> 00:42:16,302 COCHRAN'S Q TEST, TAU SQUARED, 854 00:42:16,302 --> 00:42:20,039 AND ALSO I SQUARED. 855 00:42:20,039 --> 00:42:20,739 I'M PROVIDED REFERENCES I THINK 856 00:42:20,739 --> 00:42:27,313 THAT WOULD BE VERY HELPFUL. 857 00:42:27,313 --> 00:42:28,681 PROFESSOR BORENSTEIN HAS A 858 00:42:28,681 --> 00:42:32,384 SERIES OF VIDEOS ALSO AVAILABLE 859 00:42:32,384 --> 00:42:34,720 ON YouTube. 860 00:42:34,720 --> 00:42:36,088 HE'S WELL PUBLISHED. 861 00:42:36,088 --> 00:42:44,997 ARE THERE ANY QUESTIONS? 862 00:42:44,997 --> 00:42:48,734 SEEING ANY QUESTIONS IN THE 863 00:42:48,734 --> 00:42:50,035 CHAT? 864 00:42:50,035 --> 00:42:53,105 >> NO, I DON'T SEE ANY AT THE 865 00:42:53,105 --> 00:42:53,839 MOMENT. 866 00:42:53,839 --> 00:42:58,477 >> SO THIS IS JUST A QUICK 867 00:42:58,477 --> 00:43:00,312 OVERVIEW OF META-ANALYSIS. 868 00:43:00,312 --> 00:43:02,081 I'M CURRENTLY WORKING THROUGH 869 00:43:02,081 --> 00:43:05,684 TWO META-ANALYSES PROJECTS FOR 870 00:43:05,684 --> 00:43:07,619 OUR COLLEAGUES HERE WITHIN 871 00:43:07,619 --> 00:43:07,820 NINDS. 872 00:43:07,820 --> 00:43:10,189 THEY ARE NOT COMPLICATED BUT IT 873 00:43:10,189 --> 00:43:12,091 INVOLVES A LOT OF DETAILS AND 874 00:43:12,091 --> 00:43:15,527 HAS A LOT OF MOVING PARTS. 875 00:43:15,527 --> 00:43:17,196 IF YOU CAN THINK OF ANY 876 00:43:17,196 --> 00:43:19,264 QUESTIONS, PLEASE DO NOT 877 00:43:19,264 --> 00:43:20,165 HESITATE TO CONTACT ME. 878 00:43:20,165 --> 00:43:22,401 I'LL BE MORE THAN HAPPY TO SHARE 879 00:43:22,401 --> 00:43:25,337 WHATEVER RESOURCES I HAVE WITH 880 00:43:25,337 --> 00:43:35,447 YOU. 881 00:43:36,915 --> 00:43:37,616 882 00:43:37,616 --> 00:43:38,817 >> I SEE ONE QUESTION. 883 00:43:38,817 --> 00:43:40,619 IT SAYS IN EXCLUSION OF RECORDS 884 00:43:40,619 --> 00:43:44,423 IN EXAMPLE HOW IS THAT DONE? 885 00:43:44,423 --> 00:43:48,861 >> EXCLUSION OF RECORDS? 886 00:43:48,861 --> 00:43:51,997 YES, SO THE RECORDS CAN BE 887 00:43:51,997 --> 00:43:55,134 EXCLUDED BASED ON IF THE DESIGN 888 00:43:55,134 --> 00:43:59,772 OF THE STUDY IS NOT IDEAL, SO 889 00:43:59,772 --> 00:44:03,475 PERHAPS RANDOMIZATION IS NOT 890 00:44:03,475 --> 00:44:06,945 IDEAL, THERE'S SOMETHING ABOUT 891 00:44:06,945 --> 00:44:08,580 HOW THAT STUDY WAS CONDUCTED 892 00:44:08,580 --> 00:44:11,984 THAT DOES NOT FIT THE EXCLUSION 893 00:44:11,984 --> 00:44:12,618 CRITERIA. 894 00:44:12,618 --> 00:44:15,354 SO THE RESEARCH TEAM HAS TO 895 00:44:15,354 --> 00:44:18,791 DEVELOP THEIR OWN INCLUSION AND 896 00:44:18,791 --> 00:44:20,125 EXCLUSION CRITERIA. 897 00:44:20,125 --> 00:44:21,527 AND THEN YOU PROVIDE 898 00:44:21,527 --> 00:44:22,127 JUSTIFICATION. 899 00:44:22,127 --> 00:44:25,431 SO FOR THOSE RECORDS THAT 900 00:44:25,431 --> 00:44:26,665 EXCLUDED IN THE META-ANALYSIS, 901 00:44:26,665 --> 00:44:29,601 THAT JUSTIFICATION HAS TO BE 902 00:44:29,601 --> 00:44:29,935 THERE. 903 00:44:29,935 --> 00:44:32,638 WHY WERE THOSE ARTICLES NOT 904 00:44:32,638 --> 00:44:33,105 INCLUDED? 905 00:44:33,105 --> 00:44:35,874 BUT FOR THE MOST PART IT JUST 906 00:44:35,874 --> 00:44:39,545 MEANS THE CONDUCT OF THOSE 907 00:44:39,545 --> 00:44:43,048 STUDIES WAS NOT WHAT WAS 908 00:44:43,048 --> 00:44:45,117 EXPECTED FOR YOUR META-ANALYSIS. 909 00:44:45,117 --> 00:44:48,086 THE RESEARCH QUESTION YOU HAD IN 910 00:44:48,086 --> 00:44:51,924 MIND, IT WAS VERY IMPORTANT THE 911 00:44:51,924 --> 00:44:54,593 TRIAL BE CONDUCTED IN A CERTAIN 912 00:44:54,593 --> 00:45:01,767 WAY. 913 00:45:01,767 --> 00:45:04,837 >> A FOLLOW-UP QUESTION, 1500 914 00:45:04,837 --> 00:45:10,542 EXCLUDED IS THAT TYPICAL? 915 00:45:10,542 --> 00:45:14,379 >> I WOULD SAY IT IS TYPICAL. 916 00:45:14,379 --> 00:45:17,883 I WOULD SAY FOR THE MOST PART IT 917 00:45:17,883 --> 00:45:18,851 IS TYPICAL. 918 00:45:18,851 --> 00:45:19,818 IT'S VERY DIFFICULT. 919 00:45:19,818 --> 00:45:24,556 THERE'S A TON OF LITERATURE IN 920 00:45:24,556 --> 00:45:27,159 THE UNIVERSE, YOU'RE LOOKING FOR 921 00:45:27,159 --> 00:45:32,030 ANSWERS FOR A VERY NARROW, VERY 922 00:45:32,030 --> 00:45:36,401 SPECIFIC RESEARCH QUESTION. 923 00:45:36,401 --> 00:45:37,836 AND SO AMONG ALL THE ARTICLES 924 00:45:37,836 --> 00:45:41,974 THAT ARE AVAILABLE, IT HAS TO BE 925 00:45:41,974 --> 00:45:42,975 RELEVANT, AND CONSISTENT, WITH 926 00:45:42,975 --> 00:45:48,113 THAT PARTICULAR QUESTION. 927 00:45:48,113 --> 00:45:50,115 OTHERWISE WHEN YOU SYNTHESIZE 928 00:45:50,115 --> 00:45:54,019 THE RESULTS, IT COULD BE 929 00:45:54,019 --> 00:46:04,263 MEANINGLESS. 930 00:46:12,404 --> 00:46:12,471 931 00:46:12,471 --> 00:46:22,247 >> ANY FURTHER QUESTIONS? 932 00:46:22,247 --> 00:46:23,348 >> I'M HOPING YOU PROVIDE 933 00:46:23,348 --> 00:46:23,615 FEEDBACK. 934 00:46:23,615 --> 00:46:24,883 THIS IS MY FIRST TIME GIVING 935 00:46:24,883 --> 00:46:28,186 THIS TALK. 936 00:46:28,186 --> 00:46:30,122 SO WE'RE DEVELOPING THESE TALKS 937 00:46:30,122 --> 00:46:31,790 INVOLVING META-ANALYSIS, SO 938 00:46:31,790 --> 00:46:34,393 BASED ON YOUR FEEDBACK, THAT 939 00:46:34,393 --> 00:46:37,896 WILL GIVE ME GUIDANCE ON PERHAPS 940 00:46:37,896 --> 00:46:39,665 TO INCORPORATE ADDITIONAL TOPICS 941 00:46:39,665 --> 00:46:43,335 OR TO REMOVE CERTAIN TOPICS FROM 942 00:46:43,335 --> 00:46:43,969 THE PRESENTATION. 943 00:46:43,969 --> 00:46:45,203 AGAIN, I THANK YOU ALL FOR 944 00:46:45,203 --> 00:46:48,640 JOINING US. 945 00:46:48,640 --> 00:46:49,875 I LOOK FORWARD TO HEARING FROM 946 00:46:49,875 --> 00:46:51,410 YOU OR MEETING YOU IN PERSON. 947 00:46:51,410 --> 00:46:53,946 THANK YOU. 948 00:46:53,946 --> 00:46:59,952 949 00:46:59,952 --> 00:47:01,486 950 00:47:01,486 --> 00:47:04,356 [END OF PROGRAM] 951 00:47:04,356 --> 00:47:10,796 952 00:47:10,796 --> 00:47:11,396 >> THANKS, HENRY. 953 00:47:11,396 --> 00:47:12,664 I THINK THAT'S ALL. 954 00:47:12,664 --> 00:47:15,000 I DON'T SEE ANYTHING ELSE COMING 955 00:47:15,000 --> 00:47:15,601 IN ADDITIONAL. 956 00:47:15,601 --> 00:47:17,202 I'LL GO AHEAD AND END THE 957 00:47:17,202 --> 00:47:21,006 MEETING NOW FOR EVERYONE. 958 00:47:21,006 --> 00:47:22,140 THANKS FOR COMING. 959 00:47:22,140 --> 00:47:23,241 THANKS TO THE BROADCAST TEAM. 960 00:47:23,241 --> 00:47:25,944 [END OF PROGRAM] 961 00:47:25,944 --> 00:47:32,584 962 00:47:32,584 --> 00:47:42,628