1 00:00:05,246 --> 00:00:05,813 HI. 2 00:00:05,813 --> 00:00:07,448 THANKS FOR COMING TO THE 3 00:00:07,448 --> 00:00:09,283 PRESENTATION, THE LECTURE. 4 00:00:09,283 --> 00:00:13,254 MY NAME IS KEN CHEUNG, I'M A 5 00:00:13,254 --> 00:00:14,789 PROFESSOR AT COLUMBIA UNIVERSITY 6 00:00:14,789 --> 00:00:15,823 AND I'VE BEEN WORK WITH NIENDZ 7 00:00:15,823 --> 00:00:20,728 FOR QUITE A 8 00:00:20,728 --> 00:00:22,930 WORKING WITH NINDS FOR QUITE A 9 00:00:22,930 --> 00:00:24,966 WHILE, AND TODAY I'LL TALK ABOUT 10 00:00:24,966 --> 00:00:26,100 MACHINE LEARNING? 11 00:00:26,100 --> 00:00:27,168 MEDICAL TRIALS AS WELL AS 12 00:00:27,168 --> 00:00:30,705 BIOMEDICAL RESEARCH, SO I THINK 13 00:00:30,705 --> 00:00:32,573 WE HAVE TIME FOR QUESTIONS 14 00:00:32,573 --> 00:00:35,576 TOWARDS THE END, BUT IF YOU HAVE 15 00:00:35,576 --> 00:00:39,514 QUESTIONS, YOU CAN SEND IN THE 16 00:00:39,514 --> 00:00:43,017 CHAT BOX. 17 00:00:43,017 --> 00:00:43,217 OKAY. 18 00:00:43,217 --> 00:00:45,653 SO THIS IS A VERY BRIEF AGENDA. 19 00:00:45,653 --> 00:00:47,522 SO I'M GOING TO START WITH A 20 00:00:47,522 --> 00:00:49,290 VERY, VERY BRIEF OVERVIEW OF 21 00:00:49,290 --> 00:00:50,892 MACHINE LEARNING, AND I'M GOING 22 00:00:50,892 --> 00:00:52,427 TO FOCUS ON TWO SPECIFIC TYPES 23 00:00:52,427 --> 00:00:54,295 OF MACHINE LEARNING, SUPERVISED 24 00:00:54,295 --> 00:01:01,869 LEARNING AND UNSUPERVISED 25 00:01:01,869 --> 00:01:04,806 LEARNING. 26 00:01:04,806 --> 00:01:06,240 HOPEFULLY BY NOW PEOPLE HAVE 27 00:01:06,240 --> 00:01:09,077 HEARD OF THE WORD ART OFFICIAL 28 00:01:09,077 --> 00:01:09,410 INTELLIGENCE. 29 00:01:09,410 --> 00:01:12,346 MACHINE LEARNING IS A SUBFIELD 30 00:01:12,346 --> 00:01:14,716 OF ARTIFICIAL INTELLIGENCE WITH 31 00:01:14,716 --> 00:01:15,783 A SPECIFIC STATED TO DO 32 00:01:15,783 --> 00:01:17,985 PREDICTION, TO MAKE 33 00:01:17,985 --> 00:01:20,822 RECOMMENDATION, FOR PATTERN 34 00:01:20,822 --> 00:01:21,856 RECOGNITION, DIMENSION REDUCTION 35 00:01:21,856 --> 00:01:23,758 AND THERE ARE A LOT OF 36 00:01:23,758 --> 00:01:26,427 APPLICATIONS, ROBOTICS AND 37 00:01:26,427 --> 00:01:27,762 GAMING, THE BASIC AREAS THAT 38 00:01:27,762 --> 00:01:29,297 MACHINE LEARNING CAN BE APPLIED 39 00:01:29,297 --> 00:01:29,764 TO. 40 00:01:29,764 --> 00:01:32,600 OBVIOUSLY I'M GOING TO TALK 41 00:01:32,600 --> 00:01:33,901 ABOUT APPLICATIONS AND AREAS 42 00:01:33,901 --> 00:01:37,405 THAT ARE RELEVANT TO THE 43 00:01:37,405 --> 00:01:38,206 BIOMEDICAL RESEARCH AREA. 44 00:01:38,206 --> 00:01:39,607 BUT FIRST I'M GOING TO TALK 45 00:01:39,607 --> 00:01:44,779 ABOUT DIFFERENT TYPES OF MACHINE 46 00:01:44,779 --> 00:01:46,080 LEARNING. 47 00:01:46,080 --> 00:01:47,515 THE FIRST TYPE IN MACHINE 48 00:01:47,515 --> 00:01:49,050 LEARNING IS CALLED SUPERVISED 49 00:01:49,050 --> 00:01:49,450 LEARNING. 50 00:01:49,450 --> 00:01:51,252 THE IDEA OF SUPERVISED LEARNING 51 00:01:51,252 --> 00:01:53,454 IS THAT WE'RE GOING TO TRAIN 52 00:01:53,454 --> 00:01:56,390 USING DATA TO TRAIN AN ALGORITHM 53 00:01:56,390 --> 00:01:57,692 USING DATA THAT IS IN THE FORM 54 00:01:57,692 --> 00:02:07,835 OF FEATURES SOMETIMES CALLED 55 00:02:07,835 --> 00:02:11,639 PREDICTORS, AND WITH A KNOWN 56 00:02:11,639 --> 00:02:12,974 OUTCOME, SOMETIMES CALLED 57 00:02:12,974 --> 00:02:13,274 OUTCOMES. 58 00:02:13,274 --> 00:02:14,909 I'LL GIVE YOU AN EXAMPLE WHEN I 59 00:02:14,909 --> 00:02:16,377 GET TO THAT SECTION OF 60 00:02:16,377 --> 00:02:17,545 SUPERVISED LEARNING, BUT THE 61 00:02:17,545 --> 00:02:19,647 IDEA IS THAT THE TRAINING IS 62 00:02:19,647 --> 00:02:21,282 SUPERVISED BY SOME GROUND TRUTH 63 00:02:21,282 --> 00:02:23,718 AND BASED ON THE TRAINED 64 00:02:23,718 --> 00:02:25,253 ALGORITHM, WE COULD USE THE 65 00:02:25,253 --> 00:02:26,654 ALGORITHM FOR PREDICTION 66 00:02:26,654 --> 00:02:27,221 PURPOSES, THAT IS, WE CAN 67 00:02:27,221 --> 00:02:31,492 PREDICT AN OUTCOME, PREDICT WHAT 68 00:02:31,492 --> 00:02:33,261 WOULD HAPPEN WITH DIFFERENT 69 00:02:33,261 --> 00:02:34,996 INPUT AND SOME FEATURES. 70 00:02:34,996 --> 00:02:37,532 SO THE FIRST GOAL OF SUPERVISED 71 00:02:37,532 --> 00:02:38,633 LEARNING IS TO DO PREDICTION, 72 00:02:38,633 --> 00:02:40,968 AND THAT'S WHAT WE CALL ONE OF 73 00:02:40,968 --> 00:02:43,471 THE INFERENCING GOALS. 74 00:02:43,471 --> 00:02:45,339 ANOTHER INFERENCING GOAL IS TO 75 00:02:45,339 --> 00:02:46,641 MAKE RECOMMENDATION, THAT IS, 76 00:02:46,641 --> 00:02:48,276 MAKE A PREDICTION BASED ON WHAT 77 00:02:48,276 --> 00:02:51,145 IS BEING INPUT TO THE SYSTEM 78 00:02:51,145 --> 00:02:52,547 WITH PREDICTION, AND THEN BASED 79 00:02:52,547 --> 00:02:54,081 ON PREDICTION, WHAT WE SHOULD 80 00:02:54,081 --> 00:02:54,248 DO. 81 00:02:54,248 --> 00:02:57,952 THAT WILL BE THE RECOMMENDATION 82 00:02:57,952 --> 00:02:58,486 PART. 83 00:02:58,486 --> 00:03:00,254 SO ON THE DIAGRAM TO YOUR RIGHT, 84 00:03:00,254 --> 00:03:02,023 YOU SEE THIS IS A DIAGRAM FROM 85 00:03:02,023 --> 00:03:05,493 THE DATA, THE ALGORITHM WE USE 86 00:03:05,493 --> 00:03:06,627 WHAT WE CALL TRIMMING DATA TO 87 00:03:06,627 --> 00:03:10,765 COME UP WITH AN ALGORITHM AND 88 00:03:10,765 --> 00:03:12,166 THEN INFERENCE AND 89 00:03:12,166 --> 00:03:12,633 RECOMMENDATION. 90 00:03:12,633 --> 00:03:14,902 THIS IS SOME OF THE SUPERVISED 91 00:03:14,902 --> 00:03:17,038 LEARNING PARADIGM. 92 00:03:17,038 --> 00:03:19,240 IN CONTRAST WITH SUPERVISED 93 00:03:19,240 --> 00:03:21,976 LEARNING, ANOTHER APPROACH IS 94 00:03:21,976 --> 00:03:23,644 CALLED UNSUPERVISED LEARNING. 95 00:03:23,644 --> 00:03:25,379 A DIFFERENCE BETWEEN SUPERVISED 96 00:03:25,379 --> 00:03:27,748 AND UNSUPERVISED LEARNING IS 97 00:03:27,748 --> 00:03:30,852 THAT IN UNSUPERVISED LEARNING 98 00:03:30,852 --> 00:03:32,720 WE'RE GOING TO TRAIN THE 99 00:03:32,720 --> 00:03:34,522 ALGORITHM USING THE FEATURES 100 00:03:34,522 --> 00:03:34,722 ONLY. 101 00:03:34,722 --> 00:03:36,357 FOR EXAMPLE, IN SUPERVISED 102 00:03:36,357 --> 00:03:38,459 LEARNING WE MAY TRY TO GIVE AN 103 00:03:38,459 --> 00:03:40,294 IMAGE OF A CAT AND GIVE YOU ALL 104 00:03:40,294 --> 00:03:42,496 THE FEE TUFERS A CAT AND THEN 105 00:03:42,496 --> 00:03:44,365 THE ALGORITHM WOULD TRAIN ON 106 00:03:44,365 --> 00:03:45,800 THESE FEATURES SO YOU CAN 107 00:03:45,800 --> 00:03:47,134 PREDICT THAT WHENEVER YOU GIVE 108 00:03:47,134 --> 00:03:49,003 THAT IMAGE, YOU PREDICT IT'S A 109 00:03:49,003 --> 00:03:49,637 CAT. 110 00:03:49,637 --> 00:03:51,172 IN UNSUPERVISED LEARNING, WE'RE 111 00:03:51,172 --> 00:03:54,141 NOT TELLING THE ALGORITHM THAT 112 00:03:54,141 --> 00:03:54,976 THIS IS A CAT. 113 00:03:54,976 --> 00:03:57,178 WE JUST GIVE THE ALGORITHM A 114 00:03:57,178 --> 00:03:58,813 BUNCH OF FEATURES AND THEN TO 115 00:03:58,813 --> 00:04:01,349 TRY TO LEARN ABOUT WHAT THE 116 00:04:01,349 --> 00:04:02,450 PATTERN AND THE FEA TOURS. 117 00:04:02,450 --> 00:04:04,552 AS A RESULT OF THAT -- FEATURES. 118 00:04:04,552 --> 00:04:06,053 AS A RESULT OF THAT, THE MAIN 119 00:04:06,053 --> 00:04:08,890 INFERENCE WE CAN DRAW FROM 120 00:04:08,890 --> 00:04:10,658 UNSUPERVISED LEARNING IS TO 121 00:04:10,658 --> 00:04:12,760 IDENTIFY HOMOGENOUS POPULATIONS 122 00:04:12,760 --> 00:04:15,129 BASED ON THE FEATURES AND ALSO 123 00:04:15,129 --> 00:04:17,031 IN SITUATIONS WHEN WE HAVE A LOT 124 00:04:17,031 --> 00:04:20,001 OF FEATURES AND INPUTS, WE WANT 125 00:04:20,001 --> 00:04:22,603 TO REDUCE THE SET TO A MORE 126 00:04:22,603 --> 00:04:23,604 MANAGEABLE NUMBER OF FEATURES, 127 00:04:23,604 --> 00:04:25,539 THEN WE DO DIMENSION REDUCTION. 128 00:04:25,539 --> 00:04:29,277 SO THE TWO MAIN GOALS IN 129 00:04:29,277 --> 00:04:30,177 UNSUPERVISED LEARNING WOULD BE 130 00:04:30,177 --> 00:04:32,013 TO IDENTIFY HOMOGENOUS 131 00:04:32,013 --> 00:04:33,648 SUBPOPULATIONS AND TO DO 132 00:04:33,648 --> 00:04:34,649 DIMENSION REDUCTION. 133 00:04:34,649 --> 00:04:37,718 AGAIN I'M GOING TO GIVE YOU 134 00:04:37,718 --> 00:04:38,686 EXAMPLES, SPECIFIC EXAMPLES AND 135 00:04:38,686 --> 00:04:40,655 USE CASES ON THESE TWO TOP 136 00:04:40,655 --> 00:04:44,058 LEARNINGS AS WE MOVE ALONG. 137 00:04:44,058 --> 00:04:45,927 THE THIRD TYPE OF LEARNING 138 00:04:45,927 --> 00:04:48,129 METHOD IS WHAT WE CALL ONLINE 139 00:04:48,129 --> 00:04:48,429 LEARNING. 140 00:04:48,429 --> 00:04:55,036 SO THE IDEA IS THAT THE DIAGRAM 141 00:04:55,036 --> 00:04:57,571 ON THE SLIDE, YOU SEE THAT 142 00:04:57,571 --> 00:04:59,307 THERE'S AN ARROW ON THE UPPER 143 00:04:59,307 --> 00:04:59,507 PART. 144 00:04:59,507 --> 00:05:01,609 THAT IS WE'RE GOING TO UPDATE A 145 00:05:01,609 --> 00:05:03,277 TRAINING, UPDATE DATA, COLLECTED 146 00:05:03,277 --> 00:05:04,946 FROM A SYSTEM, AND THEN USE THE 147 00:05:04,946 --> 00:05:08,082 NEW DATA TO UPDATE THE 148 00:05:08,082 --> 00:05:08,416 ALGORITHM. 149 00:05:08,416 --> 00:05:09,650 SO THIS IS REALLY THE BASIC 150 00:05:09,650 --> 00:05:10,918 CONCEPT OF ONLINE LEARNING IS 151 00:05:10,918 --> 00:05:15,323 THAT ONCE WE TRAIN THE 152 00:05:15,323 --> 00:05:16,657 ALGORITHM, IT WILL BE EVOLVING 153 00:05:16,657 --> 00:05:17,658 AND YOU'LL KEEP LEARNING. 154 00:05:17,658 --> 00:05:20,027 THE IDEA IS THAT ONCE IT'S 155 00:05:20,027 --> 00:05:20,995 DEPLOYED IN A RECOMMENDED 156 00:05:20,995 --> 00:05:22,229 SYSTEM, WE COLLECT NEW DATA AND 157 00:05:22,229 --> 00:05:23,831 THEN WE USE THE NEW DATA TO GO 158 00:05:23,831 --> 00:05:27,435 BACK TO THE TRAINING MODE AND TO 159 00:05:27,435 --> 00:05:28,869 IMPROVE THE QUALITY OF THE 160 00:05:28,869 --> 00:05:30,604 PREDICTION AS WELL AS THE 161 00:05:30,604 --> 00:05:33,341 QUALITY OF THE RECOMMENDATION. 162 00:05:33,341 --> 00:05:37,645 AND IN THE CONTEXT OF HUMAN 163 00:05:37,645 --> 00:05:38,980 TRIALS, IT'S ACTUALLY ONE OF THE 164 00:05:38,980 --> 00:05:40,381 SPECIFIC FORMS OF ONLINE 165 00:05:40,381 --> 00:05:40,848 LEARNING. 166 00:05:40,848 --> 00:05:45,086 SO IN TRIALS WHERE WE ENROLL A 167 00:05:45,086 --> 00:05:46,320 FEW PARTICIPANTS AT THE TIME AND 168 00:05:46,320 --> 00:05:48,389 THEN FROM WHAT WE LEARN FROM THE 169 00:05:48,389 --> 00:05:49,924 PATIENT PARTICIPANTS, WE DECIDE 170 00:05:49,924 --> 00:05:51,892 ON WHETHER WE WANT TO CHANGE THE 171 00:05:51,892 --> 00:05:53,327 DOSE OF A TREATMENT, WE WANT TO 172 00:05:53,327 --> 00:05:54,628 CELLS THE DURATION OF A 173 00:05:54,628 --> 00:05:56,263 TREATMENT, WE WANT TO CHANGE THE 174 00:05:56,263 --> 00:05:59,200 ENDPOINT OF A TREATMENT, THIS IS 175 00:05:59,200 --> 00:06:00,968 THE DECISION THAT THE BASIS 176 00:06:00,968 --> 00:06:02,203 POINT DURING THE STUDY. 177 00:06:02,203 --> 00:06:08,209 SO THIS IS ONE FORM OF ONLINE 178 00:06:08,209 --> 00:06:08,809 LEARNING. 179 00:06:08,809 --> 00:06:12,146 SO THIS IS SOMETHING THAT WE'RE 180 00:06:12,146 --> 00:06:13,114 FAMILIAR WITH. 181 00:06:13,114 --> 00:06:14,648 THE FOURTH TYPE OF LEARNING IS 182 00:06:14,648 --> 00:06:16,617 CALLED REINFORCEMENT LEARNING. 183 00:06:16,617 --> 00:06:22,123 THE IDEA IS THAT WE SEND AN 184 00:06:22,123 --> 00:06:23,557 AGENT TO THE FIELD AND THEN 185 00:06:23,557 --> 00:06:25,626 BASED ON THE FEATURES AND THE 186 00:06:25,626 --> 00:06:26,727 FEEDBACK, WE'RE GOING TO SEND IT 187 00:06:26,727 --> 00:06:28,396 BACK TO THE AGENT, AND THEN 188 00:06:28,396 --> 00:06:32,633 BASED ON THAT, WE GIVE THE AGENT 189 00:06:32,633 --> 00:06:33,834 SOME SIGNAL ABOUT THIS IS THE 190 00:06:33,834 --> 00:06:35,803 RIGHT DECISION OR WRONG 191 00:06:35,803 --> 00:06:37,171 DECISION, THE FEEDBACK WILL HELP 192 00:06:37,171 --> 00:06:38,339 THE AGENT TO LEARN. 193 00:06:38,339 --> 00:06:40,341 SO THIS IS A TYPE OF LEARNING 194 00:06:40,341 --> 00:06:42,443 METHOD THAT IS USED A LOT IN 195 00:06:42,443 --> 00:06:44,311 ROBOTICS TRAINING, HOW DO YOU DO 196 00:06:44,311 --> 00:06:48,349 A TASK TO TRAIN WHETHER THEY 197 00:06:48,349 --> 00:06:49,784 KNOW ONE TASK THAT THEY DO IS 198 00:06:49,784 --> 00:06:50,885 CORRECT OR NOT. 199 00:06:50,885 --> 00:06:54,255 SO THIS IS SOMETHING THAT IS 200 00:06:54,255 --> 00:07:00,828 KIND OF -- IS A VERY INTERESTING 201 00:07:00,828 --> 00:07:01,295 APPLICATION. 202 00:07:01,295 --> 00:07:08,536 ANOTHER APPLICATION IS TRAINING 203 00:07:08,536 --> 00:07:11,038 AN ALPHA GOAL TO PLAY CHESS, 204 00:07:11,038 --> 00:07:13,474 THOSE THAT HAVE HEARD ON THE 205 00:07:13,474 --> 00:07:16,410 NEWS, ALPHA GOAL IS TRAINING 206 00:07:16,410 --> 00:07:17,878 BASED ON LEARNING. 207 00:07:17,878 --> 00:07:19,947 BACK TO HUMAN TRIALS, WE HAVE 208 00:07:19,947 --> 00:07:22,450 ACTUALLY USED IT A LOT IN 209 00:07:22,450 --> 00:07:23,684 CLINICAL MANAGEMENT OF PATIENTS. 210 00:07:23,684 --> 00:07:25,653 SO WHEN A PATIENT COMES IN WITH 211 00:07:25,653 --> 00:07:26,954 CERTAIN DIAGNOSIS, WE USUALLY 212 00:07:26,954 --> 00:07:29,256 TREAT THEM WITH SOME SORT OF 213 00:07:29,256 --> 00:07:31,659 PERSONALIZED TREATMENT BASED ON 214 00:07:31,659 --> 00:07:33,928 RECOMMENDATIONS OR APPROVED DRUG 215 00:07:33,928 --> 00:07:36,030 BASED ON RCT. 216 00:07:36,030 --> 00:07:38,032 AND IF THE FIRST RECOMMENDATION 217 00:07:38,032 --> 00:07:40,201 DOESN'T WORK, THEN BASED ON THE 218 00:07:40,201 --> 00:07:41,502 RESPONSE OF THE PATIENT TO 219 00:07:41,502 --> 00:07:42,403 PRESCRIBE THE SECOND LINE 220 00:07:42,403 --> 00:07:43,604 TREATMENT AND SO ON. 221 00:07:43,604 --> 00:07:49,176 SO THIS IS IN SOME WAY 222 00:07:49,176 --> 00:07:50,478 REINFORCEMENT LEARNING THAT IS 223 00:07:50,478 --> 00:07:52,680 BASED ON THE SEQUENCE OF 224 00:07:52,680 --> 00:07:53,747 TREATMENT AND OBSERVATION, WE 225 00:07:53,747 --> 00:07:55,950 MAKE DECISION AND WE LEARN OVER 226 00:07:55,950 --> 00:07:59,019 TIME WHAT WOULD BE THE BEST 227 00:07:59,019 --> 00:08:00,121 SECONDARY TREATMENT IN CASE THE 228 00:08:00,121 --> 00:08:01,989 FIRST LINE DOESN'T WORK. 229 00:08:01,989 --> 00:08:05,059 ONE SPECIFIC SAMPLE THAT I 230 00:08:05,059 --> 00:08:07,661 WORKED ON IS IN THE CONTEXT OF 231 00:08:07,661 --> 00:08:08,996 MANAGING PATIENTS WITH 232 00:08:08,996 --> 00:08:10,131 DEPRESSION SYMPTOMS, AND THE 233 00:08:10,131 --> 00:08:13,367 IDEA IS THAT WHETHER WE GIVE 234 00:08:13,367 --> 00:08:17,872 THEM MEDICATION OR GIVE THEM 235 00:08:17,872 --> 00:08:21,709 BEHAVIORAL THERAPY, SO IF WE 236 00:08:21,709 --> 00:08:23,878 LOOK AT THE VERY, VERY IMMEDIATE 237 00:08:23,878 --> 00:08:25,312 ENDPOINT, WE HAVE DONE A STUDY 238 00:08:25,312 --> 00:08:27,815 WHERE WE SHOW THAT GIVING PEOPLE 239 00:08:27,815 --> 00:08:30,784 MEDICATION IMPROVES THE 240 00:08:30,784 --> 00:08:32,653 SHORT-TERM SYMPTOM BETTER THAN 241 00:08:32,653 --> 00:08:34,655 GIVING THEM CPT. 242 00:08:34,655 --> 00:08:38,092 BUT WHAT'S INTERESTING IS THAT 243 00:08:38,092 --> 00:08:39,793 IF YOU OBSERVE THE PARTICIPANT 244 00:08:39,793 --> 00:08:42,096 FOR A LONGER TERM, THOSE WHO 245 00:08:42,096 --> 00:08:44,798 STARTED OUT WITH CPT FIRST AND 246 00:08:44,798 --> 00:08:48,335 THEN GIVEN MEDICATION END UP 247 00:08:48,335 --> 00:08:49,970 BETTER THAN PARTICIPANTS 248 00:08:49,970 --> 00:08:50,938 RECEIVING MEDICATION ALL THE 249 00:08:50,938 --> 00:08:51,105 WAY. 250 00:08:51,105 --> 00:08:52,907 AND THE REASON IS THAT THE 251 00:08:52,907 --> 00:08:57,444 THEORY IS THAT CPT WILL ACTUALLY 252 00:08:57,444 --> 00:08:59,413 HELP PATIENTS TO PERFORM A LOT 253 00:08:59,413 --> 00:09:03,884 OF PROBLEM-SOLVING AND TO 254 00:09:03,884 --> 00:09:06,854 IMPROVE THE EFFICACY AND IMPROVE 255 00:09:06,854 --> 00:09:08,489 MEDICATION ADHERENCE, SO BIT 256 00:09:08,489 --> 00:09:09,690 TIME THEY'RE GIVEN MEDICATION, 257 00:09:09,690 --> 00:09:11,225 THEY'LL BETTER ADHERE TO THAT. 258 00:09:11,225 --> 00:09:13,527 SO THIS IS ONE FORM OF THE 259 00:09:13,527 --> 00:09:14,128 REINFORCEMENT LEARNING EXAMPLE 260 00:09:14,128 --> 00:09:21,001 THAT WE USE A LOT IN TREATMENT 261 00:09:21,001 --> 00:09:24,371 OR STAFF CARE APPROACH FOR 262 00:09:24,371 --> 00:09:24,872 DEPRESSION. 263 00:09:24,872 --> 00:09:27,241 SO I WON'T BE TOUCHING ON THIS 264 00:09:27,241 --> 00:09:29,210 BECAUSE IT IS QUITE INVOLVING, 265 00:09:29,210 --> 00:09:31,645 BUT THIS IS ONE CONCEPT THAT'S 266 00:09:31,645 --> 00:09:33,414 VERY IMPORTANT IN MANAGEMENT IN 267 00:09:33,414 --> 00:09:36,483 THE CLINIC FOR PATIENTS. 268 00:09:36,483 --> 00:09:41,722 THE LAST TYPE OF LEARNING IS 269 00:09:41,722 --> 00:09:43,791 CALLED ENSEMBLE LEARNING. 270 00:09:43,791 --> 00:09:45,993 SO IT IS NOT A SPECIFIC FORM OF 271 00:09:45,993 --> 00:09:46,660 LEARNING. 272 00:09:46,660 --> 00:09:48,596 RATHER, THIS IS A METHOD THAT 273 00:09:48,596 --> 00:09:50,664 TRIES TO COMBINE DIFFERENT 274 00:09:50,664 --> 00:09:51,165 LEARNERS. 275 00:09:51,165 --> 00:09:53,300 SO THE IDEA IS THAT THERE'S SO 276 00:09:53,300 --> 00:09:56,270 MANY METHODS, EITHER SUPERVISED 277 00:09:56,270 --> 00:09:57,972 LEARNING, UNSUPERVISED LEARNING 278 00:09:57,972 --> 00:09:59,873 IN THE MARKETPLACE, AND THE IDEA 279 00:09:59,873 --> 00:10:05,446 IS THAT EACH METHOD WILL HAVE 280 00:10:05,446 --> 00:10:05,779 MERIT. 281 00:10:05,779 --> 00:10:07,748 SO SOME METHODS ARE BETTER AT 282 00:10:07,748 --> 00:10:09,416 CERTAIN DATASET THAN THE OTHER 283 00:10:09,416 --> 00:10:11,051 METHOD AND VICE VERSA. 284 00:10:11,051 --> 00:10:13,554 THE IDEA OF ENSEMBLE LEARNING IS 285 00:10:13,554 --> 00:10:16,624 THAT INSTEAD OF USING ONLY ONE 286 00:10:16,624 --> 00:10:18,592 LEARNING METHOD, WE'RE GOING TO 287 00:10:18,592 --> 00:10:19,893 PULL ALL THE LEARNING METHODS 288 00:10:19,893 --> 00:10:22,229 OUT THERE, SO IN ONE TRAINING 289 00:10:22,229 --> 00:10:24,431 DATASET WE'LL APPLY, SAY, FIVE 290 00:10:24,431 --> 00:10:25,499 DIFFERENT METHODS, AND THEN 291 00:10:25,499 --> 00:10:26,934 BASED ON THE FIVE DIFFERENT 292 00:10:26,934 --> 00:10:28,135 METHODS, WE'RE GOING TO PULL THE 293 00:10:28,135 --> 00:10:30,971 PREDICTION INTO ONE SINGLE 294 00:10:30,971 --> 00:10:31,305 SUPER-LEARNER. 295 00:10:31,305 --> 00:10:33,274 SO THIS IS THE CONCEPT ON 296 00:10:33,274 --> 00:10:34,208 ENSEMBLE LEARNING. 297 00:10:34,208 --> 00:10:37,778 I'M GOING TO TALK TOO MUCH ABOUT 298 00:10:37,778 --> 00:10:40,414 THIS BECAUSE THIS IS A LOT OF 299 00:10:40,414 --> 00:10:42,916 REALLY DETAILS INTO THE 300 00:10:42,916 --> 00:10:45,886 TECHNIQUE ON HOW PULLING 301 00:10:45,886 --> 00:10:48,188 INFORMATION FROM DIFFERENT 302 00:10:48,188 --> 00:10:48,522 ALGORITHMS. 303 00:10:48,522 --> 00:10:50,457 BUT THE BASIC TERMINOLOGY THAT 304 00:10:50,457 --> 00:10:53,093 YOU MAY HAVE COME ACROSS IN THE 305 00:10:53,093 --> 00:10:54,762 LITERATURE WOULD BE THE BASIC 306 00:10:54,762 --> 00:10:58,465 APPROACH YOU DO ON ENSEMBLE 307 00:10:58,465 --> 00:11:00,234 LEARNING IS BOOSTING, TO AVERAGE 308 00:11:00,234 --> 00:11:01,635 ACROSS DIFFERENT LEARNERS, 309 00:11:01,635 --> 00:11:03,070 BAGGING, ANOTHER WAY OF 310 00:11:03,070 --> 00:11:04,071 AVERAGING ACROSS LEARNER AND 311 00:11:04,071 --> 00:11:05,806 THEN OF COURSE THIS CONCEPT 312 00:11:05,806 --> 00:11:07,908 SUPER LEARNERS, WHICH IS AGAIN A 313 00:11:07,908 --> 00:11:09,009 LEARNER THAT TAKES INFORMATION 314 00:11:09,009 --> 00:11:12,179 FROM ALL SEPARATE LEARNING 315 00:11:12,179 --> 00:11:13,614 ALGORITHMS AND THEN APPLIES THEM 316 00:11:13,614 --> 00:11:15,549 INTO ONE SINGLE RECOMMENDATION. 317 00:11:15,549 --> 00:11:18,285 SO THIS IS THE CONCEPT OF 318 00:11:18,285 --> 00:11:19,620 ENSEMBLE LEARNING. 319 00:11:19,620 --> 00:11:21,255 THERE ARE ACTUALLY MANY, MANY 320 00:11:21,255 --> 00:11:23,991 MORE DIFFERENT WAYS OF AXON MY 321 00:11:23,991 --> 00:11:25,225 OF MACHINE LEARNING METHODS, BUT 322 00:11:25,225 --> 00:11:28,929 THESE ARE THE FIVE TYPES THAT I 323 00:11:28,929 --> 00:11:31,098 SHALL INTRODUCE THE CONCEPT VERY 324 00:11:31,098 --> 00:11:33,067 BRIEFLY, SUPERVISED LEARNING, 325 00:11:33,067 --> 00:11:35,269 UNSUPERVISED LEARNING, ONLINE 326 00:11:35,269 --> 00:11:35,936 LEARNING, REINFORCEMENT LEARNING 327 00:11:35,936 --> 00:11:37,037 AND ENSEMBLE LEARNING. 328 00:11:37,037 --> 00:11:38,238 SO AS I MENTIONED AT THE 329 00:11:38,238 --> 00:11:42,910 BEGINNING RS I'M GOING TO FOLK 330 00:11:42,910 --> 00:11:45,346 FOCUS ON SUPERVISED LEARNING AND 331 00:11:45,346 --> 00:11:46,113 UNSUPERVISED LEARNING WITH 332 00:11:46,113 --> 00:11:47,548 EXAMPLES TO ILLUSTRATE HOW THEY 333 00:11:47,548 --> 00:11:53,921 WORK. 334 00:11:53,921 --> 00:11:54,355 OKAY. 335 00:11:54,355 --> 00:11:56,223 LET'S START WITH SOME USE CASES 336 00:11:56,223 --> 00:11:57,825 OF SUPERVISED LEARNING. 337 00:11:57,825 --> 00:11:59,426 SO THERE ARE MANY OTHER USE 338 00:11:59,426 --> 00:12:00,828 CASES, BUT THESE ARE THE 339 00:12:00,828 --> 00:12:02,896 EXAMPLES THAT ACTUALLY I CAME 340 00:12:02,896 --> 00:12:04,465 ACROSS IN MY COLLABORATION, SO 341 00:12:04,465 --> 00:12:06,100 IT'S EASIER FOR ME TO TALK ABOUT 342 00:12:06,100 --> 00:12:06,433 THEM. 343 00:12:06,433 --> 00:12:10,237 SO THE FIRST APPLICATION THAT WE 344 00:12:10,237 --> 00:12:12,673 USE IN MACHINE LEARNING, 345 00:12:12,673 --> 00:12:15,275 SUPERVISED LEARNING, RATHER, IS 346 00:12:15,275 --> 00:12:18,145 TO DEVELOP AN APP RECOMMENDER 347 00:12:18,145 --> 00:12:18,712 SYSTEM. 348 00:12:18,712 --> 00:12:21,081 SO WE CONDUCTED A STUDY WHERE WE 349 00:12:21,081 --> 00:12:24,017 LOOKED AT ABOUT 13 DIFFERENT 350 00:12:24,017 --> 00:12:26,553 HEALTH APPS FOR USE WITH 351 00:12:26,553 --> 00:12:27,921 DEPRESSION AND ANXIETY, SO WE 352 00:12:27,921 --> 00:12:31,258 WANTED TO MANAGE THE SYMPTOMS. 353 00:12:31,258 --> 00:12:33,560 AND ONE OF THE PURPOSES OF THE 354 00:12:33,560 --> 00:12:36,563 RECOMMENDED SYSTEM IS TO MANAGE 355 00:12:36,563 --> 00:12:38,932 THE USER EXPERIENCE AND TO HELP 356 00:12:38,932 --> 00:12:41,902 THEM TO IMPROVE THE ENGAGEMENT 357 00:12:41,902 --> 00:12:43,303 THROUGH THE HEALTH APPS. 358 00:12:43,303 --> 00:12:45,506 SO THESE ARE SMARTPHONE APPS 359 00:12:45,506 --> 00:12:50,110 THAT YOU CAN ACTUALLY DOWNLOAD 360 00:12:50,110 --> 00:12:51,879 ON THE APP STORE AND GOOGLE PAY. 361 00:12:51,879 --> 00:12:57,451 SO THE IDEA IS THAT WE WANT YOU 362 00:12:57,451 --> 00:12:59,520 TO SEND PUSH NOTIFICATIONS TO 363 00:12:59,520 --> 00:13:01,288 ENHANCE THE RESPONSE RATE OF 364 00:13:01,288 --> 00:13:04,024 PARTICIPANTS SO THAT THEY START 365 00:13:04,024 --> 00:13:05,659 USING APPS IN ORDER TO HELP THEM 366 00:13:05,659 --> 00:13:06,727 MANAGE THE SYMPTOMS. 367 00:13:06,727 --> 00:13:11,765 SO IN THIS CONTEXT, WE ARE IN A 368 00:13:11,765 --> 00:13:15,068 SUPERVISED LEARNING SITUATION 369 00:13:15,068 --> 00:13:18,372 WHERE IN THE SYSTEM THE FEATURE 370 00:13:18,372 --> 00:13:21,141 IS GOING TO SEND THE USERS PUSH 371 00:13:21,141 --> 00:13:22,443 NOTIFICATIONS WITH GIVEN 372 00:13:22,443 --> 00:13:24,044 FEATURES, IN TERM OF THE TYPE OF 373 00:13:24,044 --> 00:13:25,579 PUSH NOTIFICATION, THE TIMING OF 374 00:13:25,579 --> 00:13:27,080 THE PUSH NOTIFICATIONSES, LIKE 375 00:13:27,080 --> 00:13:29,683 WHETHER IT'S A WEEKEND, MORNING, 376 00:13:29,683 --> 00:13:30,884 AFTERNOON, EVENING, AND THE 377 00:13:30,884 --> 00:13:32,186 FREQUENCY OF THE PUSH 378 00:13:32,186 --> 00:13:32,619 NOTIFICATIONS. 379 00:13:32,619 --> 00:13:34,588 SO WE DO HAVE SET FEATURES FOR 380 00:13:34,588 --> 00:13:37,024 THESE PUSH NOTIFICATIONS, AND 381 00:13:37,024 --> 00:13:39,526 THEN ON THE SYSTEM WE CAN 382 00:13:39,526 --> 00:13:40,961 ACTUALLY TRACK WHERE THE USER 383 00:13:40,961 --> 00:13:42,496 WILL ACTUALLY RESPOND TO THE 384 00:13:42,496 --> 00:13:44,598 PUSH NOTIFICATION TO OPEN AN 385 00:13:44,598 --> 00:13:44,798 APP. 386 00:13:44,798 --> 00:13:49,269 SO IN THIS CASE, WE WANT TO 387 00:13:49,269 --> 00:13:50,704 REALLY PREDICT WHAT IS THE BEST 388 00:13:50,704 --> 00:13:52,339 COMBINATION OF FEATURES OF PUSH 389 00:13:52,339 --> 00:13:53,106 NOTIFICATION THAT WILL MAKE 390 00:13:53,106 --> 00:13:54,842 PEOPLE TO USE THE APP. 391 00:13:54,842 --> 00:13:58,445 SO THIS IS A SITUATION WHERE THE 392 00:13:58,445 --> 00:14:00,547 GROUND TRUTH, WHICH IS THE 393 00:14:00,547 --> 00:14:06,086 USER'S RESPONSE, AND SO THIS IS 394 00:14:06,086 --> 00:14:11,692 ONE EXAMPLE THAT WE USE 395 00:14:11,692 --> 00:14:14,528 SUPERVISED LEARNING FOR, SO TO 396 00:14:14,528 --> 00:14:16,296 INDICATE THE OUTCOME FOR THE TWO 397 00:14:16,296 --> 00:14:18,265 DIFFERENT LEARNING METHODS, 398 00:14:18,265 --> 00:14:24,171 SUPERVISED LEARNING METHOD AND 399 00:14:24,171 --> 00:14:29,109 TO BASED ON PREDICT RESPONSE 400 00:14:29,109 --> 00:14:31,278 RATE FOR A DIFFERENT SET OF PUSH 401 00:14:31,278 --> 00:14:31,945 NOTIFICATIONS. 402 00:14:31,945 --> 00:14:33,680 SO THIS IS ONE OF THE EXAMPLES 403 00:14:33,680 --> 00:14:37,351 WE USE SUPERVISED LEARNING. 404 00:14:37,351 --> 00:14:38,385 ANOTHER EXAMPLE THAT WE ARE 405 00:14:38,385 --> 00:14:42,222 CURRENTLY WORKING ON IS WE'VE 406 00:14:42,222 --> 00:14:43,790 CREATED AWE PANEL OF 407 00:14:43,790 --> 00:14:45,626 NEUROLOGISTS AND PHYSICAL 408 00:14:45,626 --> 00:14:55,936 THERAPISTS AND PRACTITIONERS TO 409 00:14:55,936 --> 00:14:57,304 CREATE AN ALGORITHM FOR PATIENTS 410 00:14:57,304 --> 00:14:58,572 WHO JUST HAD A STROKE, WHETHER 411 00:14:58,572 --> 00:15:00,407 THEY SHOULD BE REFERRED TO REHAB 412 00:15:00,407 --> 00:15:05,579 OR WE SHOULD SEND THEM HOME. 413 00:15:05,579 --> 00:15:07,014 SO THIS EXPERT PANEL PROVIDES A 414 00:15:07,014 --> 00:15:09,116 LOT OF USEFUL INSIGHT IN TERMS 415 00:15:09,116 --> 00:15:10,851 OF WHEN WE SHOULD SEND PATIENTS 416 00:15:10,851 --> 00:15:12,185 HOME, WHEN WE SHOULD SEND THE 417 00:15:12,185 --> 00:15:18,091 PATIENT TO REHAB, AND THE USUAL 418 00:15:18,091 --> 00:15:21,028 COURSE IS THAT WE DON'T HAVE 419 00:15:21,028 --> 00:15:22,362 EXPERTISE MANY A LOT OF LOCAL 420 00:15:22,362 --> 00:15:23,664 CLINICS SO THE IDEA IS THAT 421 00:15:23,664 --> 00:15:25,332 WE'RE GOING TO LEVERAGE THIS 422 00:15:25,332 --> 00:15:30,170 PANEL AND ALSO PATIENT TRIALS TO 423 00:15:30,170 --> 00:15:33,740 TRAIN AN ALGORITHM SO THAT THE 424 00:15:33,740 --> 00:15:36,043 ALGORITHM WILL BE MAKING 425 00:15:36,043 --> 00:15:37,778 RECOMMENDATIONS THAT ARE SIMILAR 426 00:15:37,778 --> 00:15:38,912 TO WHAT THIS PANEL IS DOING. 427 00:15:38,912 --> 00:15:41,582 SO IN THIS CASE, IT'S ALSO 428 00:15:41,582 --> 00:15:43,383 SUPERVISED LEARNING BECAUSE THE 429 00:15:43,383 --> 00:15:44,718 EXPERT PANEL WILL GIVE THE GOLD 430 00:15:44,718 --> 00:15:45,819 DECISION OF WHAT THE RIGHT 431 00:15:45,819 --> 00:15:47,654 DECISION IS IN TERMS OF SENDING 432 00:15:47,654 --> 00:15:49,189 PATIENTS TO REHAB OR SENDING 433 00:15:49,189 --> 00:15:49,856 THEM HOME. 434 00:15:49,856 --> 00:15:51,625 THEN THE FEATURE IN THIS EXAMPLE 435 00:15:51,625 --> 00:15:56,830 WILL BE THE PATIENT TRIALS AND 436 00:15:56,830 --> 00:16:00,667 ALSO THE CONTEXT AND THE 437 00:16:00,667 --> 00:16:01,668 BACKGROUND OF THE PATIENT. 438 00:16:01,668 --> 00:16:04,504 SO THIS IS ANOTHER EXAMPLE WHERE 439 00:16:04,504 --> 00:16:06,907 THE USE OF SUPERVISED LEARNING 440 00:16:06,907 --> 00:16:08,775 IS VERY USEFUL. 441 00:16:08,775 --> 00:16:13,480 THE THIRD EXAMPLE IS ON USING 442 00:16:13,480 --> 00:16:15,215 FILE ASSAY FOR CANCER SCREENING. 443 00:16:15,215 --> 00:16:16,617 SO THIS IS AN EXAMPLE THAT I'M 444 00:16:16,617 --> 00:16:19,386 GOING TO SHOW YOU WITH SOME DATA 445 00:16:19,386 --> 00:16:24,791 ON, BUT THE IDEA IS THAT THE 446 00:16:24,791 --> 00:16:25,759 GOLD STANDARD STARTING WITH WE 447 00:16:25,759 --> 00:16:28,528 HAVE A CANCER DIAGNOSIS, BUT 448 00:16:28,528 --> 00:16:30,998 THIS IS NOT INVASIVE, AND IN 449 00:16:30,998 --> 00:16:32,332 MANY CASES THERE'S JUST NO 450 00:16:32,332 --> 00:16:34,301 RESOURCE FOR BIOPSIES, SO IN 451 00:16:34,301 --> 00:16:36,470 THIS STUDY, THE GOAL IS THAT WE 452 00:16:36,470 --> 00:16:43,043 WANT TO USE HPP CHANNEL ASSAYS 453 00:16:43,043 --> 00:16:45,779 ASSAYSFOR SIMPLE CANCER SCREENI. 454 00:16:45,779 --> 00:16:47,781 SO IN THE STUDY WE DO HAVE THE 455 00:16:47,781 --> 00:16:51,718 GOLD STANDARD WHERE THERE ARE 456 00:16:51,718 --> 00:16:53,453 PATIENTS WHEN WE HAVE THEIR 457 00:16:53,453 --> 00:16:55,756 BIOPSY TO DETERMINE WHETHER THE 458 00:16:55,756 --> 00:16:57,858 PATIENT ACTUALLY HAS CERVICAL 459 00:16:57,858 --> 00:16:59,926 CANCER, AT THE SAME TIME WE 460 00:16:59,926 --> 00:17:07,234 OBTAIN THE HPP CHANNELSS 461 00:17:07,234 --> 00:17:08,969 AND THE LEARNING GOAL HERE IS 462 00:17:08,969 --> 00:17:13,140 THAT YOU REALLY USE THE HPV 463 00:17:13,140 --> 00:17:14,207 CHANNELS TO WHETHER IT'S GOING 464 00:17:14,207 --> 00:17:15,876 TO BE A POSITIVE DIAGNOSIS OR 465 00:17:15,876 --> 00:17:16,376 NOT. 466 00:17:16,376 --> 00:17:18,311 AGAIN, WE'LL COME BACK WITH AN 467 00:17:18,311 --> 00:17:19,713 EXAMPLE OF SOME DATA. 468 00:17:19,713 --> 00:17:21,014 OTHER EXAMPLE INCLUDES LOOKING 469 00:17:21,014 --> 00:17:23,850 AT DIFFERENT TREATMENTS, IT CAN 470 00:17:23,850 --> 00:17:26,286 BE IN THE CLINICAL PHASE OR 471 00:17:26,286 --> 00:17:28,455 DISCOVERY PHASE, WHERE WE HAVE 472 00:17:28,455 --> 00:17:29,656 MANY MOLECULES WHERE WE WANT YOU 473 00:17:29,656 --> 00:17:31,291 TO LOOK AT WHETHER THIS MOLECULE 474 00:17:31,291 --> 00:17:35,595 IN COMBINATION WILL PRODUCE A 475 00:17:35,595 --> 00:17:36,263 SYNERGISTIC EFFECTED AND THIS IS 476 00:17:36,263 --> 00:17:37,531 THE TYPE OF SITUATION THAT AGAIN 477 00:17:37,531 --> 00:17:43,503 WE CAN USE SUPERVISED LEARNING 478 00:17:43,503 --> 00:17:45,906 TO DO THE PRIKDZ OF THE OUTCOME 479 00:17:45,906 --> 00:17:47,774 OF THE TREATMENT COMBINATIONS -- 480 00:17:47,774 --> 00:17:49,176 PREDICTION OF THE OUTCOME OF THE 481 00:17:49,176 --> 00:17:50,143 TREATMENT COMBINATIONS. 482 00:17:50,143 --> 00:17:55,215 WE CAN APPLY SUPERVISED 483 00:17:55,215 --> 00:17:56,616 LEARNING, WE HAVE LOTS OF 484 00:17:56,616 --> 00:17:59,486 PATIENT FEATURES IN THE DATA, TO 485 00:17:59,486 --> 00:18:00,654 PREDICT OUTCOMES. 486 00:18:00,654 --> 00:18:01,955 ONE CURRENT PROJECT IS THAT WE 487 00:18:01,955 --> 00:18:07,027 ARE GOING TO USE DATA FROM THE 488 00:18:07,027 --> 00:18:10,197 ER, THERKS HR DATA TO PREDICT 489 00:18:10,197 --> 00:18:11,732 PATIENTS ADMITTED FOR PNEUMONIA 490 00:18:11,732 --> 00:18:13,300 AND TO PREDICT THE OUTCOMES, 491 00:18:13,300 --> 00:18:15,569 BASICALLY TO GET THE PROGNOSIS, 492 00:18:15,569 --> 00:18:20,140 PATIENTS ABOUT 200 CLINICAL 493 00:18:20,140 --> 00:18:21,241 FEATURES, SO ANOTHER EXAMPLE, 494 00:18:21,241 --> 00:18:23,243 VERY, VERY COMMON EXAMPLES OF 495 00:18:23,243 --> 00:18:24,911 SUPERVISED LEARNING TO BUILD 496 00:18:24,911 --> 00:18:26,780 CLINICAL DECISION SUPPORT 497 00:18:26,780 --> 00:18:27,013 SYSTEM. 498 00:18:27,013 --> 00:18:28,782 IN THE CONTEXT OF E-COMMERCE, WE 499 00:18:28,782 --> 00:18:35,288 HAVE A LOT OF DATA IN TERMS OF 500 00:18:35,288 --> 00:18:37,357 BUYING HISTORY, FEATURE OF THE 501 00:18:37,357 --> 00:18:38,792 PRODUCTS TO PREDICT WHETHER A 502 00:18:38,792 --> 00:18:40,193 CUSTOMER WILL ACTUALLY PICK A 503 00:18:40,193 --> 00:18:40,660 PURCHASE. 504 00:18:40,660 --> 00:18:42,829 I'M NOT GOING TO TALK MUCH ABOUT 505 00:18:42,829 --> 00:18:43,029 THIS. 506 00:18:43,029 --> 00:18:44,264 ONE EXAMPLE THAT MAY BE RELEVANT 507 00:18:44,264 --> 00:18:47,868 TO THIS GROUP IN PARTICULAR IS 508 00:18:47,868 --> 00:18:51,471 THE USE OF SUPERVISED LEARNING 509 00:18:51,471 --> 00:18:55,742 TO PREDICT IMAGE. 510 00:18:55,742 --> 00:18:59,146 SO THIS IS A FAIRLY 511 00:18:59,146 --> 00:19:00,013 STRAIGHTFORWARD EXAMPLE. 512 00:19:00,013 --> 00:19:08,421 SO WE COLLECT DATA, HANDWRITTEN 513 00:19:08,421 --> 00:19:10,290 NUMBERS, AND THE IDEA IS THAT WE 514 00:19:10,290 --> 00:19:14,127 WANT TO TRAIN AN ALGORITHM, WHEN 515 00:19:14,127 --> 00:19:15,562 AN ALGORITHM IS PRESENTED WITH A 516 00:19:15,562 --> 00:19:17,831 PICTURE OF A NUMBER, THE 517 00:19:17,831 --> 00:19:18,598 ALGORITHM WILL BE ABLE TO 518 00:19:18,598 --> 00:19:19,800 IDENTIFY THAT THIS IS A ZERO, 519 00:19:19,800 --> 00:19:23,103 THIS IS A ONE, THIS IS A FIVE. 520 00:19:23,103 --> 00:19:25,605 SO THIS IS ANOTHER APPLICATION 521 00:19:25,605 --> 00:19:30,977 OF SUPERVISED LEARNING. 522 00:19:30,977 --> 00:19:32,712 SO IT WAS MENTIONED THAT THIS 523 00:19:32,712 --> 00:19:34,915 CAN BE APPLIED TO A SITUATION 524 00:19:34,915 --> 00:19:36,583 WITH MRI DATA AND WE WANT TO 525 00:19:36,583 --> 00:19:39,085 ASSOCIATE MRI THE IMAGE TO A 526 00:19:39,085 --> 00:19:41,822 DISEASE OUTCOME, AND THIS IS THE 527 00:19:41,822 --> 00:19:43,557 APPLICATION THAT SUPERVISED 528 00:19:43,557 --> 00:19:48,962 SLEARNG VERY HELPFUL. 529 00:19:48,962 --> 00:19:51,565 SO THESE ARE THE DIFFERENT -- 530 00:19:51,565 --> 00:19:52,966 SUPERVISED LEARNING IS VERY 531 00:19:52,966 --> 00:19:53,300 HELPFUL. 532 00:19:53,300 --> 00:19:54,401 SO THESE ARE THE DIFFERENT 533 00:19:54,401 --> 00:19:56,269 EXAMPLES OF SUPERVISED LEARNING. 534 00:19:56,269 --> 00:19:57,671 AGAIN, ONE OF THE GOALS OF 535 00:19:57,671 --> 00:20:00,207 SUPERVISED LEARNING IS TO MAKE 536 00:20:00,207 --> 00:20:00,507 PREDICTION. 537 00:20:00,507 --> 00:20:01,608 THERE ARE TWO TYPES OF 538 00:20:01,608 --> 00:20:03,610 PREDICTION THAT A SUPERVISED 539 00:20:03,610 --> 00:20:04,578 LEARNER CAN DO. 540 00:20:04,578 --> 00:20:06,213 ONE IS CLINICAL CLASSIFICATION. 541 00:20:06,213 --> 00:20:07,647 SO THE IDEA IS THAT IT'S GOING 542 00:20:07,647 --> 00:20:12,018 TO PREDICT A CATEGORICAL 543 00:20:12,018 --> 00:20:13,486 OUTCOME, WHETHER IT IS A CANCER 544 00:20:13,486 --> 00:20:15,789 DIAGNOSIS OR NOT, WHETHER THE 545 00:20:15,789 --> 00:20:18,091 NUMBERS ARE ZERO OR FIVE, 546 00:20:18,091 --> 00:20:21,061 WHETHER A USER RESPONDED TO THE 547 00:20:21,061 --> 00:20:22,362 PUSH NOTIFICATION OR NOT. 548 00:20:22,362 --> 00:20:24,130 SO THESE ARE CATEGORICAL 549 00:20:24,130 --> 00:20:26,199 OUTCOMES THAT A SUPERVISED 550 00:20:26,199 --> 00:20:28,368 LEARNER CAN DO FOR 551 00:20:28,368 --> 00:20:28,735 CLASSIFICATION. 552 00:20:28,735 --> 00:20:32,105 ANOTHER TYPE OF PREDICTION IS 553 00:20:32,105 --> 00:20:32,405 REGRESSION. 554 00:20:32,405 --> 00:20:35,508 THAT IS FOR WHEN THE GRAND TRUTH 555 00:20:35,508 --> 00:20:37,911 IS A CONTINUOUS OUTCOME. 556 00:20:37,911 --> 00:20:40,747 SO FOR EXAMPLE, IN THE 557 00:20:40,747 --> 00:20:43,283 RECOMMENDED SYSTEM EXAMPLE -- 558 00:20:43,283 --> 00:20:44,384 THE APP RECOMMENDATION SYSTEM 559 00:20:44,384 --> 00:20:45,685 EXAMPLE, IN ADDITION TO LOOKING 560 00:20:45,685 --> 00:20:47,787 AT USERS' RESPONSE RATE, WE'RE 561 00:20:47,787 --> 00:20:50,457 ALSO LOOKING AT THE TIME -- 562 00:20:50,457 --> 00:20:51,858 LOOKING AT THE TIME TO RESPOND 563 00:20:51,858 --> 00:20:53,059 TO PUSH NOTIFICATION, SO WE WANT 564 00:20:53,059 --> 00:20:55,228 TO PREDICT HOW SOON A USER WILL 565 00:20:55,228 --> 00:20:57,898 RESPOND TO PUSH NOTIFICATIONS. 566 00:20:57,898 --> 00:20:59,933 IN THE CONTEXT OF CLINICAL 567 00:20:59,933 --> 00:21:04,104 STUDIES, WE ALSO MAY WANT TO USE 568 00:21:04,104 --> 00:21:06,539 FEATURES TO PREDICT A RISK 569 00:21:06,539 --> 00:21:08,275 SCORE, CATEGORY OF OUTCOMES ONCE 570 00:21:08,275 --> 00:21:12,545 YOU SEE ON A SCALE OF ZERO TO 571 00:21:12,545 --> 00:21:13,980 100 WHERE THE RISKS STAND ON 572 00:21:13,980 --> 00:21:14,648 THIS SCALE. 573 00:21:14,648 --> 00:21:16,616 SO THIS IS THE CONTEXT OF 574 00:21:16,616 --> 00:21:18,018 PREDICTING CONTINUOUS OUTCOME, 575 00:21:18,018 --> 00:21:20,520 SO THIS IS A REGRESSION PROBLEM. 576 00:21:20,520 --> 00:21:23,056 SO THESE ARE DIFFERENT TYPES OF 577 00:21:23,056 --> 00:21:24,925 SUPERVISED LEARNER. 578 00:21:24,925 --> 00:21:27,360 SO THERE ARE MANY, MANY, MANY 579 00:21:27,360 --> 00:21:28,228 METHODS OUT THERE. 580 00:21:28,228 --> 00:21:30,463 THIS SLIDE GIVES YOU REALLY THE 581 00:21:30,463 --> 00:21:33,266 MAJOR ONES IN THE LITERATURE, 582 00:21:33,266 --> 00:21:37,237 THEY'RE NOT ALL OF IT, AND SO 583 00:21:37,237 --> 00:21:40,373 I'M GOING TO REVEAL A FEW IN THE 584 00:21:40,373 --> 00:21:42,442 NEXT FEW SLIDES, BUT THE BASIC 585 00:21:42,442 --> 00:21:48,581 IDEA IS THAT THERE'S A SPECTRUM 586 00:21:48,581 --> 00:21:52,252 FOR THE PRINCIPLE FOR LEARNING. 587 00:21:52,252 --> 00:21:54,287 TO THE RIGHT ON THE SLIDE, SO 588 00:21:54,287 --> 00:21:57,991 THESE METHODS TEND TO BE MORE 589 00:21:57,991 --> 00:21:58,858 PARSIMONIOUS FROM THE STANCE 590 00:21:58,858 --> 00:22:04,564 THAT WE UNDERSTAND THESE METHODS 591 00:22:04,564 --> 00:22:07,534 WILL LEVERAGE A STATISTICAL 592 00:22:07,534 --> 00:22:10,003 MODEL TO STRUCTURE THE DATA IN 593 00:22:10,003 --> 00:22:12,539 ORDER TO MAKE PREDICTION. 594 00:22:12,539 --> 00:22:17,277 THE ADVANTAGE OF USING THIS 595 00:22:17,277 --> 00:22:22,015 MODEL LIKE PROVEN METHOD IS THAT 596 00:22:22,015 --> 00:22:24,517 IT'S USUALLY A PRINCIPLE WAY 597 00:22:24,517 --> 00:22:26,486 THAT WE UNDERSTAND WHAT THE 598 00:22:26,486 --> 00:22:28,121 UNDERLYING ASSUMPTION IS SO THAT 599 00:22:28,121 --> 00:22:29,322 WE CAN INTERPRET THE RESULT 600 00:22:29,322 --> 00:22:31,391 FAIRLY EASILY. 601 00:22:31,391 --> 00:22:33,927 SO WHEN WE USE THESE METHODS FOR 602 00:22:33,927 --> 00:22:36,463 TRAINING, AFTER THE TRAINING, WE 603 00:22:36,463 --> 00:22:38,732 CAN AT LEAST EXPLAIN WHY THE 604 00:22:38,732 --> 00:22:41,568 ALGORITHM IS MAKING THAT 605 00:22:41,568 --> 00:22:42,435 PREDICTION. 606 00:22:42,435 --> 00:22:44,637 THE DISADVANTAGE OF THIS MODEL 607 00:22:44,637 --> 00:22:46,406 METHOD IS OF COURSE THAT IF THE 608 00:22:46,406 --> 00:22:47,607 MODEL THAT WE MAKE THE 609 00:22:47,607 --> 00:22:49,442 ASSUMPTION ON IS WRONG, THE 610 00:22:49,442 --> 00:22:51,211 PREDICTION CAN BE OFF. 611 00:22:51,211 --> 00:22:53,313 SO THE PROS AND CONS OF USING 612 00:22:53,313 --> 00:22:54,714 THE MODEL IS THAT IT'S GREAT 613 00:22:54,714 --> 00:22:56,583 WHEN IT'S CORRECT, BUT IF IT'S 614 00:22:56,583 --> 00:22:58,118 INCORRECT, WE MAY NOT BE MAKING 615 00:22:58,118 --> 00:23:00,420 THE RIGHT PREDICTION. 616 00:23:00,420 --> 00:23:02,155 TO THE LEFT SPECTRUM, THAT'S 617 00:23:02,155 --> 00:23:04,891 WHAT WE CALL THE DATA SPEAKS 618 00:23:04,891 --> 00:23:05,492 SPECTRUM. 619 00:23:05,492 --> 00:23:07,193 SO THESE TEND TO BE COMPLETELY 620 00:23:07,193 --> 00:23:09,396 DATA DRIVEN ALGORITHMS, SO THE 621 00:23:09,396 --> 00:23:12,465 ADVANTAGE OF USING DATA DRIVEN 622 00:23:12,465 --> 00:23:13,967 COMPLETELY DRIVEN ALGORITHM IS 623 00:23:13,967 --> 00:23:16,269 THAT WE'RE NOT MAKING ANY MORE 624 00:23:16,269 --> 00:23:19,005 ASSUMPTIONS, SO AS YOU MINIMIZE 625 00:23:19,005 --> 00:23:21,975 THAT, THE PREDICTION CAN BE 626 00:23:21,975 --> 00:23:22,275 WRONG. 627 00:23:22,275 --> 00:23:24,277 BUT ON THE OTHER HAND, THE ISSUE 628 00:23:24,277 --> 00:23:26,446 IS THAT BECAUSE THEY ARE SO DATA 629 00:23:26,446 --> 00:23:28,081 DRIVEN, SOMETIMES THESE 630 00:23:28,081 --> 00:23:29,082 ALGORITHMS WILL MAKE A 631 00:23:29,082 --> 00:23:30,083 PREDICTION THAT APPEARS TO BE 632 00:23:30,083 --> 00:23:34,988 CORRECT BUT IT'S DIFFICULT TO 633 00:23:34,988 --> 00:23:35,555 EXPLAIN. 634 00:23:35,555 --> 00:23:39,359 SO ACCURACY VERSUS INTERPRET 635 00:23:39,359 --> 00:23:42,228 ABILITY ON ALL THESE METHODS. 636 00:23:42,228 --> 00:23:43,763 THERE'S NO SINGLE METHOD THAT 637 00:23:43,763 --> 00:23:45,965 WORKS IN HAUL CASES AND IN 638 00:23:45,965 --> 00:23:47,467 DIFFERENT CONTEXTS, IN THE 639 00:23:47,467 --> 00:23:49,269 CONTEXT OF CLINICAL DECISION WE 640 00:23:49,269 --> 00:23:51,604 MIGHT BE WISE TO USE SOME METHOD 641 00:23:51,604 --> 00:23:53,073 THAT IS MORE INTERPRETABLE, 642 00:23:53,073 --> 00:23:54,574 WHEREAS IN SOME OTHER SITUATION 643 00:23:54,574 --> 00:23:56,443 ACCURACY IS VERY IMPORTANT, 644 00:23:56,443 --> 00:23:59,746 MAYBE DATA DRIVEN MAY BE MORE 645 00:23:59,746 --> 00:24:00,046 APPROPRIATE. 646 00:24:00,046 --> 00:24:03,316 SO THERE'S NO ONE SINGLE METHOD 647 00:24:03,316 --> 00:24:05,018 OUT THERE THAT WILL SOLVE ALL 648 00:24:05,018 --> 00:24:05,752 THE PROBLEMS. 649 00:24:05,752 --> 00:24:07,487 IN MOST CASES, PEOPLE WILL TRY A 650 00:24:07,487 --> 00:24:08,922 NUMBER OF DIFFERENT METHODS 651 00:24:08,922 --> 00:24:09,856 ANYWAY, I THINK THAT'S PROBABLY 652 00:24:09,856 --> 00:24:10,957 ONE OF THE BEST PRACTICES OF 653 00:24:10,957 --> 00:24:11,858 MACHINE LEARNING IS THAT YOU 654 00:24:11,858 --> 00:24:13,893 DON'T WANT TO USE ONE SINGLE 655 00:24:13,893 --> 00:24:15,195 METHOD, YOU WANT TO USE A NUMBER 656 00:24:15,195 --> 00:24:19,432 OF DIFFERENT METHODS. 657 00:24:19,432 --> 00:24:21,034 SO THIS SLIDE IS AN OVERVIEW OF 658 00:24:21,034 --> 00:24:22,669 THE DIFFERENT METHODOLOGIES, SO 659 00:24:22,669 --> 00:24:29,943 I'M GOING TO REVIEW A COUPLE IN 660 00:24:29,943 --> 00:24:32,479 THE NEXT FEW SLIDES. 661 00:24:32,479 --> 00:24:34,114 NEXT SLIDE IS LOGISTIC 662 00:24:34,114 --> 00:24:34,981 REGRESSION, THIS IS FOR 663 00:24:34,981 --> 00:24:36,716 CLASSIFICATION OF A CATEGORICAL 664 00:24:36,716 --> 00:24:37,584 OUTCOME. 665 00:24:37,584 --> 00:24:39,119 THIS METHOD BELONGS TO THE RIGHT 666 00:24:39,119 --> 00:24:40,787 SPECTRUM THAT IS A MODEL BASED 667 00:24:40,787 --> 00:24:41,221 METHOD. 668 00:24:41,221 --> 00:24:42,522 SO THE IDEA IS THAT WE'RE GOING 669 00:24:42,522 --> 00:24:46,459 TO USE REGRESSION TO ESTIMATE 670 00:24:46,459 --> 00:24:48,862 THE PROBABILITY OF A GIVEN 671 00:24:48,862 --> 00:24:51,831 EVENT, GIVEN SOME VALUE. 672 00:24:51,831 --> 00:24:59,305 IN THE EXAMPLE OF USING HPV 673 00:24:59,305 --> 00:25:00,807 ASSAYS TO MAKE CANCER DIAGNOSIS, 674 00:25:00,807 --> 00:25:03,576 SO YOU CAN USE REGRESSION -- YOU 675 00:25:03,576 --> 00:25:04,911 CAN USE LOGISTIC REGRESSION, 676 00:25:04,911 --> 00:25:06,613 WE'RE GOING TO ESTIMATE THE 677 00:25:06,613 --> 00:25:09,782 POSSIBILITY OF A CANCER 678 00:25:09,782 --> 00:25:12,085 DIAGNOSIS GIVEN THE HPV VALUES, 679 00:25:12,085 --> 00:25:12,485 RIGHT? 680 00:25:12,485 --> 00:25:14,587 SO THIS IS WHAT WE CAN ESTIMATE 681 00:25:14,587 --> 00:25:15,655 USING LOGISTIC REGRESSION. 682 00:25:15,655 --> 00:25:18,191 AND THEN THE INFERENCING OF THE 683 00:25:18,191 --> 00:25:19,092 SUPERVISED LEARNING IS THAT 684 00:25:19,092 --> 00:25:23,129 WE'RE GOING TO MAKE A PREDICTION 685 00:25:23,129 --> 00:25:25,298 BASED ON THE DICHOTOMY. 686 00:25:25,298 --> 00:25:27,400 IF THE PROBABILITY IS ABOUT 50%, 687 00:25:27,400 --> 00:25:29,135 THEN IT IS A CANCER DIAGNOSIS. 688 00:25:29,135 --> 00:25:31,471 OTHERWISE, IF IT'S NOT -- 689 00:25:31,471 --> 00:25:32,005 OTHERWISE, IT IS NOT. 690 00:25:32,005 --> 00:25:33,873 SO AGAIN, THE ADVANTAGE OF 691 00:25:33,873 --> 00:25:35,175 LOGISTICS REGRESSION IS THAT 692 00:25:35,175 --> 00:25:37,243 BECAUSE IT'S A SPATIAL MODEL, 693 00:25:37,243 --> 00:25:39,546 IT'S EASY TO INTERPRET, 694 00:25:39,546 --> 00:25:40,780 INTERPRETATION IS SUPER-FAST, 695 00:25:40,780 --> 00:25:46,653 AND IN MANY SITUATIONS WHEN THEY 696 00:25:46,653 --> 00:25:49,455 ARE READ FEATURE INPUT WORKS 697 00:25:49,455 --> 00:25:49,656 WELL. 698 00:25:49,656 --> 00:25:53,326 DISADVANTAGE OF USING LOGISTIC 699 00:25:53,326 --> 00:25:55,962 REGRESSION, WITH MANY FEATURES S 700 00:25:55,962 --> 00:25:58,898 THAT MODEL SPECIFICATION -- 701 00:25:58,898 --> 00:26:00,233 MODELING INTERACTIONS CAN BE A 702 00:26:00,233 --> 00:26:02,302 CHAL TOANG THE ACCURACY, SO 703 00:26:02,302 --> 00:26:03,703 SOMETHING WE NEED TO THINK ABOUT 704 00:26:03,703 --> 00:26:05,438 WHEN USING LOGISTIC REGRESSION. 705 00:26:05,438 --> 00:26:07,040 THERE ARE MANY OTHER METHODS OUT 706 00:26:07,040 --> 00:26:11,077 THERE TO TRY TO EXTEND LOGISTIC 707 00:26:11,077 --> 00:26:14,013 REGRESSIONS TO REDUCE LEE -- FOR 708 00:26:14,013 --> 00:26:15,348 RELIANCE ON MODEL ASSUMPTIONS, 709 00:26:15,348 --> 00:26:17,283 SO THIS METHOD CAN ACTUALLY BE 710 00:26:17,283 --> 00:26:18,718 USED AND THE ADVANTAGE OF USING 711 00:26:18,718 --> 00:26:20,053 THESE REMEDIES IS THAT IT 712 00:26:20,053 --> 00:26:23,022 DOESN'T TAKE AWAY THE 713 00:26:23,022 --> 00:26:24,123 INTERPRETABILITY OF LOGISTIC 714 00:26:24,123 --> 00:26:25,558 REGRESSION WHILE IMPROVING THE 715 00:26:25,558 --> 00:26:25,825 ACCURACY. 716 00:26:25,825 --> 00:26:28,928 SO THIS IS THE FIRST EXAMPLE OF 717 00:26:28,928 --> 00:26:31,364 A SUPERVISED LEARNING METHOD 718 00:26:31,364 --> 00:26:34,234 USING LOGISTIC REGRESSION. 719 00:26:34,234 --> 00:26:38,071 THE SECOND TYPE OF SUPERVISED 720 00:26:38,071 --> 00:26:39,572 LEARNING METHOD ALSO FOR 721 00:26:39,572 --> 00:26:40,773 CLASSIFICATION IS CALLED 722 00:26:40,773 --> 00:26:42,242 CLASSIFICATION TREES. 723 00:26:42,242 --> 00:26:45,378 SO AS YOU CAN SEE IN THE 724 00:26:45,378 --> 00:26:47,447 DIAGRAM, IT'S CREATED WHAT THE 725 00:26:47,447 --> 00:26:48,348 OUTPUT LOOKS LIKE. 726 00:26:48,348 --> 00:26:49,983 SO IN THE CLASSIFICATION TREE, 727 00:26:49,983 --> 00:26:51,417 AGAIN, YOU TAKE ALL THE 728 00:26:51,417 --> 00:26:53,820 FEATURES, IN THIS CASE IN THIS 729 00:26:53,820 --> 00:26:57,790 EXAMPLE, HPV ASSAY TO PREDICT 730 00:26:57,790 --> 00:27:00,393 WHETHER IT'S A CANCER DIAGNOSIS. 731 00:27:00,393 --> 00:27:03,997 SO IN THE OUTPUT OF A 732 00:27:03,997 --> 00:27:06,199 CLASSIFICATION TREE WOULD BE A 733 00:27:06,199 --> 00:27:09,068 DECISION TREE AT THE VERY TOP. 734 00:27:09,068 --> 00:27:11,471 SO FOR EXAMPLE, THE VERY FIRST 735 00:27:11,471 --> 00:27:14,440 DECISION POINT, IF WE LOOK AT 736 00:27:14,440 --> 00:27:22,982 CHANNEL 31, SO CHANNEL 31 IS 737 00:27:22,982 --> 00:27:25,051 LARGE, MEANING THAT CHANNEL 31 738 00:27:25,051 --> 00:27:27,887 IS GREAT THAN .031, THEN IT IS 739 00:27:27,887 --> 00:27:30,256 GOING TO BE A CANCER DIAGNOSIS. 740 00:27:30,256 --> 00:27:33,626 BUT IF CHANNEL 31 IS LESS THAN 741 00:27:33,626 --> 00:27:36,262 .031, WE MOVE TO THE NEXT 742 00:27:36,262 --> 00:27:38,364 DECISION POINT, AND IF YOU LOOK 743 00:27:38,364 --> 00:27:43,036 AT CHANNEL 16, AND IF CHANNEL 16 744 00:27:43,036 --> 00:27:45,938 IS GREATER THAN .029, THEN IT'S 745 00:27:45,938 --> 00:27:48,007 ALSO A CANCER DIAGNOSIS. 746 00:27:48,007 --> 00:27:49,309 OTHERWISE, WE MOVE TO THE NEXT 747 00:27:49,309 --> 00:27:51,311 DECISION POINT, WHICH LOOKS AT 748 00:27:51,311 --> 00:27:53,479 CHANNEL NUMBER 18. 749 00:27:53,479 --> 00:27:55,548 AND AGAIN, IF CHANNEL 18 IS 750 00:27:55,548 --> 00:27:57,650 GREATER THAN .03 1-RBGS IT'S A 751 00:27:57,650 --> 00:28:00,953 CANCER DIAGNOSIS, OTHERWISE WE 752 00:28:00,953 --> 00:28:02,789 MOVE PAST CHANNEL 3 IS. 753 00:28:02,789 --> 00:28:04,657 ONE FEATURE IS THAT YOU CAN USE 754 00:28:04,657 --> 00:28:07,827 THE SAME INPUT FEATURE MULTIPLE 755 00:28:07,827 --> 00:28:14,467 TIMES TO MAKE DECISIONS. 756 00:28:14,467 --> 00:28:17,337 AND THE DECISION ALGORITHM, IF 757 00:28:17,337 --> 00:28:19,872 ALL THESE CHANNELS HAVE LOWER 758 00:28:19,872 --> 00:28:22,375 VALUES THAN THIS THRESHOLD, THEN 759 00:28:22,375 --> 00:28:24,777 YOU'LL BE A NEGATIVE, NONCANCER 760 00:28:24,777 --> 00:28:25,345 DIAGNOSIS. 761 00:28:25,345 --> 00:28:27,080 SO THE ADVANTAGE OF USING 762 00:28:27,080 --> 00:28:27,947 CLASSIFICATION TREES IS THAT IS 763 00:28:27,947 --> 00:28:30,817 VERY CLEAR, SO WE CAN PUT OUR 764 00:28:30,817 --> 00:28:32,885 THUMB IN FRONT OF PEOPLE, PEOPLE 765 00:28:32,885 --> 00:28:34,487 WILL FOLLOW THE CHART, AND THIS 766 00:28:34,487 --> 00:28:37,557 IS EASY TO EXPLAIN, IT'S VERY 767 00:28:37,557 --> 00:28:42,395 INTUITIVE, THAT WHEN HPV CHANNEL 768 00:28:42,395 --> 00:28:45,531 HAS LOW VALUE, IT INCREASES 769 00:28:45,531 --> 00:28:47,133 VALUE IN CANCER DIAGNOSIS, SO 770 00:28:47,133 --> 00:28:51,671 THIS IS REALLY THE MAIN EVENT 771 00:28:51,671 --> 00:28:52,872 USING CLASSIFICATION TREE. 772 00:28:52,872 --> 00:28:54,173 AND THERE'S SOME TECHNIQUE THAT 773 00:28:54,173 --> 00:29:04,050 WE WANT TO PREVENT, OVER FITTING 774 00:29:04,050 --> 00:29:06,552 WHICH IS A CONCEPT IN SUPER 775 00:29:06,552 --> 00:29:08,321 VICED LEARNING ESPECIALLY IN 776 00:29:08,321 --> 00:29:09,856 THOSE METHODS THAT WE DON'T WANT 777 00:29:09,856 --> 00:29:14,660 TO MAKE THE METHOD SO COMPLEX, 778 00:29:14,660 --> 00:29:16,963 WHAT WE CALL OVERFITTING. 779 00:29:16,963 --> 00:29:19,098 AND ONE WAY TO REMOVE THE 780 00:29:19,098 --> 00:29:21,033 OVERFITTING PROBLEM WITH USING 781 00:29:21,033 --> 00:29:22,869 CLASSIFICATION TREE IS THAT WE 782 00:29:22,869 --> 00:29:24,771 USE A FOREST. 783 00:29:24,771 --> 00:29:26,839 SO INSTEAD OF USING ONE TREE, WE 784 00:29:26,839 --> 00:29:31,544 LOOK AT MANY, MANY TREES. 785 00:29:31,544 --> 00:29:33,513 SO FOREST BY DEFINITION ARE AN 786 00:29:33,513 --> 00:29:34,714 ENSEMBLE OF TREES. 787 00:29:34,714 --> 00:29:36,182 SO RANDOM FOREST IS ACTUALLY ONE 788 00:29:36,182 --> 00:29:38,418 OF THOSE ENSEMBLE LEARNING 789 00:29:38,418 --> 00:29:40,186 METHODS I MENTIONED EARLIER IS 790 00:29:40,186 --> 00:29:42,355 THAT THE IDEA IS THAT ON THE 791 00:29:42,355 --> 00:29:44,390 SAME TRAINING DATASET, WE RUN 792 00:29:44,390 --> 00:29:45,958 MANY, MANY TREES AND THEN USE 793 00:29:45,958 --> 00:29:47,293 THE DIFFERENT TREES TO AVERAGE 794 00:29:47,293 --> 00:29:52,432 THEM OUT AND THEN MAKE A 795 00:29:52,432 --> 00:29:52,732 PREDICTION. 796 00:29:52,732 --> 00:29:54,734 THERE ARE SOME EMPIRICAL STUDIES 797 00:29:54,734 --> 00:29:56,269 SUGGESTING THAT RANDOM FOREST 798 00:29:56,269 --> 00:30:02,175 DOES IMPROVE THE ACCURACY OF 799 00:30:02,175 --> 00:30:03,042 PREDICTION, CLASSIFICATION 800 00:30:03,042 --> 00:30:04,811 TREES, SIMPLY BECAUSE YOU REMOVE 801 00:30:04,811 --> 00:30:07,647 THE OVERFITTING PROBLEM AND 802 00:30:07,647 --> 00:30:10,483 REDUCE ERROR IN PREDICTION. 803 00:30:10,483 --> 00:30:11,684 BUT DISADVANTAGE OF USING FOREST 804 00:30:11,684 --> 00:30:14,454 IS THAT IT TAKES AWAY THE 805 00:30:14,454 --> 00:30:15,788 INTERPRETABILITY OF THE TREE, 806 00:30:15,788 --> 00:30:16,956 BECAUSE FOR CLASSIFICATION WE 807 00:30:16,956 --> 00:30:18,491 CAN PRESENT EVERYONE WITH A TREE 808 00:30:18,491 --> 00:30:21,060 LIKE THAT SO THAT THE 809 00:30:21,060 --> 00:30:22,795 DECISION-MAKING PROCESS IS VERY 810 00:30:22,795 --> 00:30:25,298 CLEAR, BUT FOR FOREST IT BECOMES 811 00:30:25,298 --> 00:30:29,368 A BLACK BOX. 812 00:30:29,368 --> 00:30:32,205 SO THIS IS THE SECOND METHOD. 813 00:30:32,205 --> 00:30:34,941 ANOTHER METHOD THAT YOU MAY HAVE 814 00:30:34,941 --> 00:30:37,109 HEARD OF WHICH ACTUALLY IS NOT A 815 00:30:37,109 --> 00:30:38,544 GOOD APPLICATION IN THIS CASE, 816 00:30:38,544 --> 00:30:41,214 BUT I RUN THIS ANYWAY IS A 817 00:30:41,214 --> 00:30:41,948 NEURAL NETWORK. 818 00:30:41,948 --> 00:30:43,483 THE IDEA OF A NEURAL NETWORK IS 819 00:30:43,483 --> 00:30:51,023 THAT YOU TRY TO MODEL THE DATA 820 00:30:51,023 --> 00:30:52,558 AFTER THE WAY THAT WE OBSERVE 821 00:30:52,558 --> 00:30:56,295 HOW LIKE NEURON COMMUNICATE WITH 822 00:30:56,295 --> 00:30:59,232 EACH OTHER IN THE SYNERGISTIC 823 00:30:59,232 --> 00:30:59,465 SYSTEM. 824 00:30:59,465 --> 00:31:02,101 I FIGURED I WOULD TALK ABOUT 825 00:31:02,101 --> 00:31:03,202 THIS SINCE THIS IS A 826 00:31:03,202 --> 00:31:06,005 PRESENTATION IN NINDS ABOUT THE 827 00:31:06,005 --> 00:31:07,807 NEURAL NETWORK, BUT THE IDEA IS 828 00:31:07,807 --> 00:31:09,775 THAT JUST STOPS THERE, SO THIS 829 00:31:09,775 --> 00:31:11,110 CAN BE VIEWED AS A NEURON AND 830 00:31:11,110 --> 00:31:13,279 THEN THE IDEA IS THAT YOU 831 00:31:13,279 --> 00:31:15,147 PROPAGATE, YOU TAKE AN INPUT AND 832 00:31:15,147 --> 00:31:16,782 THEN TRANSFER THE INPUT INTO 833 00:31:16,782 --> 00:31:18,651 SOME NUMBER AND THEN COMMUNICATE 834 00:31:18,651 --> 00:31:21,254 TO THE NEXT LAYER OF NEURONS AND 835 00:31:21,254 --> 00:31:22,922 THEN KEEP PROPAGATING AND THEN 836 00:31:22,922 --> 00:31:32,365 TO MAKE A PREDICTION TO OUTCOME. 837 00:31:32,365 --> 00:31:34,867 SO EACH PROPAGATION IS BASED ON 838 00:31:34,867 --> 00:31:37,503 SWHOOM WE CALL TRANSFER -- BASED 839 00:31:37,503 --> 00:31:38,838 ON WHAT WE CALL TRANSFER 840 00:31:38,838 --> 00:31:40,473 FUNCTION, AND HOW DO WE 841 00:31:40,473 --> 00:31:41,607 DETERMINE THE TRANSFER FUNCTION 842 00:31:41,607 --> 00:31:43,543 IS SOMETHING WE NEED TO SPECIFY, 843 00:31:43,543 --> 00:31:45,444 BUT THE IDEA IS THAT WE'RE GOING 844 00:31:45,444 --> 00:31:47,680 TO USE THE DATA TO DETERMINE THE 845 00:31:47,680 --> 00:31:49,482 SPECIFICS OF THIS TRANSFER 846 00:31:49,482 --> 00:31:52,752 FUNCTION IN ORDER TO MAKE A 847 00:31:52,752 --> 00:31:53,052 PREDICTION. 848 00:31:53,052 --> 00:31:55,755 SO THIS IS AN EXAMPLE WHERE 849 00:31:55,755 --> 00:31:57,590 THERE'S ONE HIDDEN LAYER. 850 00:31:57,590 --> 00:31:59,125 SO THIS HIDDEN LAYER INDICATES 851 00:31:59,125 --> 00:32:02,295 THE FEATURE, SO FOR SIMPLICITY 852 00:32:02,295 --> 00:32:04,830 SAKE I ONLY USED SIX CHANNELS, 853 00:32:04,830 --> 00:32:07,133 SIX FEATURES AS INPUT. 854 00:32:07,133 --> 00:32:09,402 SO ONE LAYER AND THEN WE HAVE 855 00:32:09,402 --> 00:32:12,271 ONE OUTPUT AS A PREDICTION. 856 00:32:12,271 --> 00:32:17,410 THE COMPLEXITY OF THE NEURAL 857 00:32:17,410 --> 00:32:18,945 NETWORK CAN INCREASE 858 00:32:18,945 --> 00:32:20,580 INDEFINITELY BY ADDING NEW 859 00:32:20,580 --> 00:32:20,813 LAYERS. 860 00:32:20,813 --> 00:32:23,049 SO IN THIS EXAMPLE, AGAIN, WE 861 00:32:23,049 --> 00:32:24,684 HAVE THE SIX INPUT FEATURES BUT 862 00:32:24,684 --> 00:32:25,952 WE HAVE TWO LAYERS. 863 00:32:25,952 --> 00:32:30,423 SO AS WE IMPROVE -- AS WE 864 00:32:30,423 --> 00:32:31,424 INCREASE THE NUMBER OF LAYERS, 865 00:32:31,424 --> 00:32:33,492 WE INCREASE THE COMPLEXITY OF 866 00:32:33,492 --> 00:32:35,361 THE NETWORK, THE IDEA IS THAT 867 00:32:35,361 --> 00:32:36,996 YOU'LL EVENTUALLY GET A MUCH 868 00:32:36,996 --> 00:32:39,198 BETTER ACCURACY IF WE HAVE 869 00:32:39,198 --> 00:32:40,166 SUFFICIENT COMPLEXITY TO MODEL. 870 00:32:40,166 --> 00:32:43,803 BUT AGAIN, IT HAS THE ISSUE OF 871 00:32:43,803 --> 00:32:45,104 OVERFITTING IF WE ADD TOO MANY 872 00:32:45,104 --> 00:32:48,941 LAYERS TO IT. 873 00:32:48,941 --> 00:32:52,979 BUT IN TERMS OF ACCURACY, I 874 00:32:52,979 --> 00:32:53,980 THINK NEURAL NETWORK HAS GOT A 875 00:32:53,980 --> 00:32:55,414 LOT OF ATTENTION BECAUSE IT'S 876 00:32:55,414 --> 00:33:00,219 DOING VERY WELL, ESPECIALLY IN 877 00:33:00,219 --> 00:33:02,588 THE APPLICATION OF IMAGING AND 878 00:33:02,588 --> 00:33:06,258 RECOGNITION OF IMAGES. 879 00:33:06,258 --> 00:33:09,295 SO SEVERAL YEARS AGO, I THINK 880 00:33:09,295 --> 00:33:11,631 IBM HAD THIS NEURAL NETWORK TO 881 00:33:11,631 --> 00:33:15,101 REALLY TO TRAIN THE ALGORITHM TO 882 00:33:15,101 --> 00:33:17,436 RECOGNIZE PICTURES, SO THAT WAS 883 00:33:17,436 --> 00:33:19,271 MANY YEARS AGO, IT'S BECOME MUCH 884 00:33:19,271 --> 00:33:21,340 EASIER NOW THESE DAYS. 885 00:33:21,340 --> 00:33:25,311 BUT THAT WAS BASED ON NEURAL 886 00:33:25,311 --> 00:33:25,544 NETWORK. 887 00:33:25,544 --> 00:33:26,946 THERE ARE A NUMBER OF POTENTIAL 888 00:33:26,946 --> 00:33:28,481 ISSUES OF NEURAL NETWORK 889 00:33:28,481 --> 00:33:30,116 ESPECIALLY IN THE CONTEXT OF 890 00:33:30,116 --> 00:33:32,852 CLINICAL APPLICATION. 891 00:33:32,852 --> 00:33:35,621 FIRST, PICKING A NUMBER, WE HAVE 892 00:33:35,621 --> 00:33:37,790 A LOT OF DATA, IT DEPENDS ON HOW 893 00:33:37,790 --> 00:33:39,425 WE SPECIFY THE STARTING VALUES. 894 00:33:39,425 --> 00:33:40,960 THE JOWLT PUT OF A NEURAL 895 00:33:40,960 --> 00:33:43,162 NETWORK CAN BE RANDOM, SO 896 00:33:43,162 --> 00:33:46,098 IT'S -- THE OUTPUT OF A NEURAL 897 00:33:46,098 --> 00:33:48,768 NETWORK CAN BE RANDOM, SO WE MAY 898 00:33:48,768 --> 00:33:50,302 NOT HAVE THAT AMOUNT OF DATA TO 899 00:33:50,302 --> 00:33:53,239 WARRANT USING A NEURAL NETWORK. 900 00:33:53,239 --> 00:33:59,078 AND IT CAN CRASH IF THE DATA IS 901 00:33:59,078 --> 00:34:00,379 TOO SMALL. 902 00:34:00,379 --> 00:34:02,114 THE MAIN DATA IN THE APPLICATION 903 00:34:02,114 --> 00:34:04,583 IS THAT IT'S A COMPLETE BLACK 904 00:34:04,583 --> 00:34:04,750 BOX. 905 00:34:04,750 --> 00:34:10,122 NOT ONLY THAT WE CANNOT 906 00:34:10,122 --> 00:34:11,857 INTERPRET HOW DID THE NETWORK 907 00:34:11,857 --> 00:34:14,727 MAKE THE PREDICTION. 908 00:34:14,727 --> 00:34:19,031 AND ALSO THAT WE DO NOT KNOW 909 00:34:19,031 --> 00:34:20,332 WHETHER THAT PREDICTION -- WHY 910 00:34:20,332 --> 00:34:23,736 THAT PREDICTION WILL WORK. 911 00:34:23,736 --> 00:34:25,071 SO CLINICALLY IT WORKS VERY 912 00:34:25,071 --> 00:34:27,907 WELL, BUT THERE'S NO THEORY TO 913 00:34:27,907 --> 00:34:31,310 SUGGEST WHY NEURAL NETWORK 914 00:34:31,310 --> 00:34:33,279 WORKS. 915 00:34:33,279 --> 00:34:34,680 SO THAT'S THE PART THIS WE NEED 916 00:34:34,680 --> 00:34:37,083 TO BE THINKING ABOUT WHEN WE 917 00:34:37,083 --> 00:34:40,252 WANT TO USE THIS METHOD IN 918 00:34:40,252 --> 00:34:44,523 CLINICAL APPLICATION. 919 00:34:44,523 --> 00:34:48,060 SO WHICH LEADS TO THE NEXT PART 920 00:34:48,060 --> 00:34:51,864 OF THE PRESENTATION IS ABOUT HOW 921 00:34:51,864 --> 00:34:54,266 DO WE EVALUATE AN ALGORITHM. 922 00:34:54,266 --> 00:34:56,469 I MENTIONED THE CONCEPT OF 923 00:34:56,469 --> 00:34:57,937 OVERFITTING, SO FOR ANY METHODS, 924 00:34:57,937 --> 00:34:59,939 IF YOU MAKE IT COMPLEX ENOUGH IN 925 00:34:59,939 --> 00:35:01,440 THE CONTEXT OF NEURAL NETWORK, 926 00:35:01,440 --> 00:35:03,309 IF YOU ADD ENOUGH NUMBER OF 927 00:35:03,309 --> 00:35:04,610 LAYERS IN CLASSIFICATION TREES 928 00:35:04,610 --> 00:35:06,579 AND RANDOM FORESTS, IF YOU ADD 929 00:35:06,579 --> 00:35:08,114 ENOUGH TREES, YOU CAN ALWAYS 930 00:35:08,114 --> 00:35:09,982 IMPROVE THE ACCURACY IN THE 931 00:35:09,982 --> 00:35:13,519 TRAINING DATASET, BUT THAT'S NOT 932 00:35:13,519 --> 00:35:16,255 THE GOAL OF SUPERVISED LEARNING. 933 00:35:16,255 --> 00:35:18,891 THE GOAL OF LEARNING, SUPERVISED 934 00:35:18,891 --> 00:35:20,192 LEARNING OR PREDICTION IS THE 935 00:35:20,192 --> 00:35:23,262 PREDICTION OF WHAT WILL HAPPEN 936 00:35:23,262 --> 00:35:25,030 IN FUTURE INSTANCES, NOT IN THE 937 00:35:25,030 --> 00:35:25,998 TRAINING DATA. 938 00:35:25,998 --> 00:35:32,571 SO THE IDEA IS THAT WHAT MAY 939 00:35:32,571 --> 00:35:34,006 WORK WELL IF WE WANT TO USE THE 940 00:35:34,006 --> 00:35:35,241 TRAINING TO VALIDATE IT. 941 00:35:35,241 --> 00:35:37,843 SO ONE IDEA TO EVALUATE AND 942 00:35:37,843 --> 00:35:42,448 TRAIN THE ALGORITHM IS THAT WE 943 00:35:42,448 --> 00:35:45,284 WANT YOU TO USE ADDITIONAL DATA 944 00:35:45,284 --> 00:35:50,256 WHETHER THE ALGORITHM WILL WORK. 945 00:35:50,256 --> 00:35:52,658 SO CROSS VALIDATION, SO THE 946 00:35:52,658 --> 00:35:54,960 CONCEPT OF CROSS VALIDATION IS 947 00:35:54,960 --> 00:35:56,162 THAT INSTEAD USING THE ENTIRE 948 00:35:56,162 --> 00:35:57,997 DATASET TO TRAIN THE ALGORITHM, 949 00:35:57,997 --> 00:36:00,065 WE SET ASIDE, FOR EXAMPLE, 20% 950 00:36:00,065 --> 00:36:02,835 DATA NOT TO BE USING THE 951 00:36:02,835 --> 00:36:05,004 TRAINING, WE JUST USE THE DATA 952 00:36:05,004 --> 00:36:06,639 FOR TRAINING AND THEN BASED ON 953 00:36:06,639 --> 00:36:09,608 THE ALGORITHM WE USE 20% OF THE 954 00:36:09,608 --> 00:36:13,212 DATA TO EVALUATE THE ACCURACY OF 955 00:36:13,212 --> 00:36:16,081 THE ALGORITHM. 956 00:36:16,081 --> 00:36:18,217 SO IN THE IDEAL WORLD, WHAT WE 957 00:36:18,217 --> 00:36:22,521 CAN DO IS THAT AFTER WE'RE 958 00:36:22,521 --> 00:36:23,522 TRAINING AN ALGORITHM, WE 959 00:36:23,522 --> 00:36:25,057 COLLECT DATA FROM A NEW COHORT 960 00:36:25,057 --> 00:36:26,992 AND THEN USE DATA FROM A NEW 961 00:36:26,992 --> 00:36:28,928 COHORT TO VALIDATE ACCURACY. 962 00:36:28,928 --> 00:36:31,664 SO IT'S IMPORTANT NOT TO ONLY 963 00:36:31,664 --> 00:36:34,533 LOOK AT ACCURACY OF THE 964 00:36:34,533 --> 00:36:39,004 ALGORITHM WITHIN THE TRAINING 965 00:36:39,004 --> 00:36:46,579 DATASET, SO THE FIGURE HERE 966 00:36:46,579 --> 00:36:49,114 INDICATES THAT WE'RE USING TWO 967 00:36:49,114 --> 00:36:51,383 LEARNING OUTCOMES IN THIS 968 00:36:51,383 --> 00:36:53,786 EXAMPLE, AND THE ACCURACY 969 00:36:53,786 --> 00:36:56,755 METRICS IS CALLED AREA UNDER THE 970 00:36:56,755 --> 00:37:02,361 CURVE, WHICH IS A SUMMARY OF 971 00:37:02,361 --> 00:37:02,661 SPECIFICITY. 972 00:37:02,661 --> 00:37:07,299 IF YOU LOOK, AUC, HIGH IS GOOD, 973 00:37:07,299 --> 00:37:08,267 ONE PERFECT DIAGNOSTIC. SO IF 974 00:37:08,267 --> 00:37:10,469 YOU LOOK AT AUC JUST WITHIN THE 975 00:37:10,469 --> 00:37:13,305 TRAINING DATA, THEY BOTH MAP ALL 976 00:37:13,305 --> 00:37:15,908 THE METHODS IN DIFFERENT 977 00:37:15,908 --> 00:37:19,979 POPULATION, ACHIEVE OVER 90% YOU 978 00:37:19,979 --> 00:37:20,145 SEE. 979 00:37:20,145 --> 00:37:24,149 IF YOU USE CROSS VALIDATION, WE 980 00:37:24,149 --> 00:37:26,752 FIND THAT THE CROSS VALIDATION 981 00:37:26,752 --> 00:37:29,388 USES ONLY ABOUT 80%, WHICH IS 982 00:37:29,388 --> 00:37:32,424 ACTUALLY VERY HIGH, BUT IT'S NOT 983 00:37:32,424 --> 00:37:32,992 ABOVE 90%. 984 00:37:32,992 --> 00:37:36,395 SO WHEN WE DO -- WHEN WE APPLY 985 00:37:36,395 --> 00:37:37,496 SUPERVISED LEARNING, IT IS 986 00:37:37,496 --> 00:37:42,534 IMPORTANT FOR US TO LOOK AT THE 987 00:37:42,534 --> 00:37:44,169 CROSS VALIDATION ERROR INSTEAD 988 00:37:44,169 --> 00:37:48,774 OF THE TRAINING ERROR AS A WAY 989 00:37:48,774 --> 00:37:50,843 TO READ THE ALGORITHMS. 990 00:37:50,843 --> 00:37:52,912 LET ME MOVE QUICKLY TO THE AREA 991 00:37:52,912 --> 00:37:55,014 OF UNSUPERVISED LEARNING. 992 00:37:55,014 --> 00:37:56,882 TWO INTERESTING QUESTIONS ABOUT 993 00:37:56,882 --> 00:37:58,317 UNSUPERVISED LEARNING. 994 00:37:58,317 --> 00:38:03,322 ONE IS ON IDENTIFYING HOW MUCH 995 00:38:03,322 --> 00:38:05,624 IS THE POPULATION, THE OTHER IS 996 00:38:05,624 --> 00:38:06,258 DIMENSION REDUCTION. 997 00:38:06,258 --> 00:38:07,793 SO JUST A QUICK EXAMPLE. 998 00:38:07,793 --> 00:38:09,662 WHAT YOU'RE SEEING IN THIS HEAT 999 00:38:09,662 --> 00:38:16,135 MAP IS THAT EACH CORE RESPONDS 1000 00:38:16,135 --> 00:38:19,738 TO ONE CANCER TUMOR AND EACH 1001 00:38:19,738 --> 00:38:21,807 COLUMN IS A GENE, SO THE COLOR 1002 00:38:21,807 --> 00:38:25,110 INDICATES THE EXPRESSION LEVEL 1003 00:38:25,110 --> 00:38:26,545 OF ON THAT GENE. 1004 00:38:26,545 --> 00:38:28,380 FIRST APPLICATION OF CLUSTER 1005 00:38:28,380 --> 00:38:31,583 ANALYSIS IN UNSUPERVISED 1006 00:38:31,583 --> 00:38:33,585 LEARNING IS CLUSTERING OF THE 1007 00:38:33,585 --> 00:38:35,521 SAMPLING UNIT N THIS EXAMPLE WE 1008 00:38:35,521 --> 00:38:36,956 TRIED TO CLUSTER THE CANCER 1009 00:38:36,956 --> 00:38:40,993 TUMORS BY LOOKING AT WHERE THE 1010 00:38:40,993 --> 00:38:42,094 TUMORS ARE SIMILAR TO EACH 1011 00:38:42,094 --> 00:38:42,328 OTHER. 1012 00:38:42,328 --> 00:38:44,863 SO WE TRIED TO IDENTIFY MAYBE A 1013 00:38:44,863 --> 00:38:46,498 HOMOGENOUS GROUP OF THESE CANCER 1014 00:38:46,498 --> 00:38:48,267 TUMORS, SO THAT WILL BE 1015 00:38:48,267 --> 00:38:51,403 CLUSTERING OF THE CANCER TUMOR, 1016 00:38:51,403 --> 00:38:53,372 LOOKING FOR ROW THAT IS ARE 1017 00:38:53,372 --> 00:38:53,739 SIMILAR. 1018 00:38:53,739 --> 00:38:56,342 THAT'S ONE APPLICATION FOR 1019 00:38:56,342 --> 00:38:58,510 INFERENCING OF UNSUPERVISED 1020 00:38:58,510 --> 00:38:58,844 LEARNING. 1021 00:38:58,844 --> 00:39:00,713 THE SECOND INTERESTING GOAL OF 1022 00:39:00,713 --> 00:39:01,513 UNSUPERVISED LEARNING IS THAT WE 1023 00:39:01,513 --> 00:39:03,749 WANT TO LOOK AT WHETHER THERE 1024 00:39:03,749 --> 00:39:05,317 ARE GENES THAT ARE CLUSTERED 1025 00:39:05,317 --> 00:39:06,752 WITH EACH OTHER, IN ORDER TO 1026 00:39:06,752 --> 00:39:08,721 REDUCE THE NUMBER OF GENES, 1027 00:39:08,721 --> 00:39:11,790 BECAUSE WE WANT TO MAKE INFLENS 1028 00:39:11,790 --> 00:39:14,326 OF OVER 6,000 GENES, THIS IS -- 1029 00:39:14,326 --> 00:39:16,161 WE WANT TO MAKE SENSE OF OVER 1030 00:39:16,161 --> 00:39:18,364 6,06,000 GENES, THIS IS A LOT OF 1031 00:39:18,364 --> 00:39:19,698 GENES, WE WANTED TO IDENTIFY 1032 00:39:19,698 --> 00:39:21,433 GENES THAT TBLONG THE SAME 1033 00:39:21,433 --> 00:39:24,370 CLUSTER AND MAKE HELP STUDY IT 1034 00:39:24,370 --> 00:39:26,238 DOWN THE ROAD, SO ANOTHER 1035 00:39:26,238 --> 00:39:29,875 EXAMPLE OF UNSUPERVISED LEARNING 1036 00:39:29,875 --> 00:39:32,544 IS TO PERFORM CLUSTER ANALYSIS 1037 00:39:32,544 --> 00:39:34,580 OF THE GENES IN ORDER TO 1038 00:39:34,580 --> 00:39:35,347 ACCOMPLISH DIMENSION REDUCTION. 1039 00:39:35,347 --> 00:39:39,718 SO THESE ARE THE TWO MAIN GOALS. 1040 00:39:39,718 --> 00:39:43,555 AND THESE ARE THE METHODS OF 1041 00:39:43,555 --> 00:39:44,656 UNSUPERVISED LEARNING. 1042 00:39:44,656 --> 00:39:45,758 I'M GOING TO GO OVER THEM VERY, 1043 00:39:45,758 --> 00:39:46,525 VERY QUICKLY. 1044 00:39:46,525 --> 00:39:48,927 BUT THE MAIN METHOD THAT I'LL 1045 00:39:48,927 --> 00:39:51,563 TALK ABOUT IS K-MEAN CLUSTERING, 1046 00:39:51,563 --> 00:39:52,798 HIERARCHICAL CLUSTERING AND 1047 00:39:52,798 --> 00:39:54,099 MODEL BASED CLUSTERING. 1048 00:39:54,099 --> 00:39:59,004 SO THIS IS AN EXAMPLE WHERE WE 1049 00:39:59,004 --> 00:40:04,977 HAVE DATA OF OVER 20,000 DATA 1050 00:40:04,977 --> 00:40:08,981 AND THEN THE GOAL IS TO IDENTIFY 1051 00:40:08,981 --> 00:40:12,084 SUBPOPULATIONS OF THE PATTERN OF 1052 00:40:12,084 --> 00:40:18,290 THESE INDIVIDUALS WHO TAKE -- 1053 00:40:18,290 --> 00:40:20,392 AND PUT THEM IN THE IDENTIFY OF 1054 00:40:20,392 --> 00:40:21,026 PATTERN BAIMPS. 1055 00:40:21,026 --> 00:40:24,329 THE IDEA OF CLUSTERING IS THAT 1056 00:40:24,329 --> 00:40:25,764 WE IDENTIFY THE NUMBER OF 1057 00:40:25,764 --> 00:40:28,167 CLUSTER AND THEN WE PUT EACH 1058 00:40:28,167 --> 00:40:34,773 GROUP TO A CLUSTER, IF THAT DATA 1059 00:40:34,773 --> 00:40:36,075 POINT IS COEFFICIENT OF THE 1060 00:40:36,075 --> 00:40:38,043 STANDARD OF THAT CLUSTER, THAT'S 1061 00:40:38,043 --> 00:40:40,112 WHAT WE CALL K-MEANS CLUSTERING, 1062 00:40:40,112 --> 00:40:41,780 SO THESE ARE THE DIFFERENT 1063 00:40:41,780 --> 00:40:43,615 METHODS, SO THIS IS A DIFFERENT 1064 00:40:43,615 --> 00:40:45,584 WAY THAT WE USE THE DATA 1065 00:40:45,584 --> 00:40:47,286 SOPHOMORE K-MEANS CLUSTER. 1066 00:40:47,286 --> 00:40:48,754 YOU CAN SEE THE RESULTS ARE 1067 00:40:48,754 --> 00:40:49,655 HIGHLY DIFFERENT. 1068 00:40:49,655 --> 00:40:50,856 SO THEY INDICATE ONE OF THE 1069 00:40:50,856 --> 00:40:52,724 ISSUES WITH USING K-MEANS 1070 00:40:52,724 --> 00:40:54,927 CLUSTERING IS THAT FIRST OF ALL, 1071 00:40:54,927 --> 00:40:58,197 THE SCALING IS CRITICAL. 1072 00:40:58,197 --> 00:41:01,700 SO IN THIS EXAMPLE, ONLY TWO 1073 00:41:01,700 --> 00:41:03,569 INPUT FEATURES TO MAKE IT 1074 00:41:03,569 --> 00:41:04,136 SIMPLE. 1075 00:41:04,136 --> 00:41:07,206 SO I REDUCED THE -- USED THE 1076 00:41:07,206 --> 00:41:08,640 NUMBER OF STEPS AS INPUT 1077 00:41:08,640 --> 00:41:10,809 FEATURES AND THE OTHER IS CALLED 1078 00:41:10,809 --> 00:41:15,547 ACTIVITY INPUT -- IS MIDDAY 1079 00:41:15,547 --> 00:41:18,050 ACTIVITY AS ONE OF THE FEATURES. 1080 00:41:18,050 --> 00:41:20,252 YOU SEE WE GET THESE TWO 1081 00:41:20,252 --> 00:41:21,820 CLUSTERS, BUT THAT IS BASICALLY 1082 00:41:21,820 --> 00:41:23,388 DRIVEN BY THE NUMBER OF STEPS. 1083 00:41:23,388 --> 00:41:24,623 THE REASON IS THAT NUMBER OF 1084 00:41:24,623 --> 00:41:26,692 STEPS HAS A MUCH BIGGER VARIANCE 1085 00:41:26,692 --> 00:41:29,661 SO THAT IT DRIVES BASICALLY 1086 00:41:29,661 --> 00:41:32,164 THAT'S THE DOMINANT POINT, SO WE 1087 00:41:32,164 --> 00:41:37,436 SCALE THE INPUTTED SO THAT -- 1088 00:41:37,436 --> 00:41:40,405 THE INPUT SO THAT THERE'S NO ONE 1089 00:41:40,405 --> 00:41:43,242 SINGLE VARIABLE THAT WILL 1090 00:41:43,242 --> 00:41:44,776 DOMINATE, AND AFTER 1091 00:41:44,776 --> 00:41:47,613 STANDARDIZATION SHES WE GET VERR 1092 00:41:47,613 --> 00:41:48,514 STANDARDIZATION WE GET VERY 1093 00:41:48,514 --> 00:41:49,248 DIFFERENT ANSWERS. 1094 00:41:49,248 --> 00:41:52,851 SO IN GENERAL, SCALING IS VERY 1095 00:41:52,851 --> 00:41:53,819 BENEFICIAL FOR MACHINE LEARNING 1096 00:41:53,819 --> 00:41:57,689 AND PARTICULARLY FOR K-MEANS 1097 00:41:57,689 --> 00:42:00,425 CLUSTERING. 1098 00:42:00,425 --> 00:42:01,527 THE SEKSD EXAMPLE IS 1099 00:42:01,527 --> 00:42:02,761 HIERARCHICAL CLUSTERING. 1100 00:42:02,761 --> 00:42:03,829 AGAIN, SCANNING IS VERY 1101 00:42:03,829 --> 00:42:05,364 IMPORTANT, SO IN THIS CASE WE 1102 00:42:05,364 --> 00:42:07,099 USED THE RIGHT SCALE TO DO IT, 1103 00:42:07,099 --> 00:42:11,537 AND IN HIERARCHICAL CLUSTERING, 1104 00:42:11,537 --> 00:42:17,009 YOU PROFILE, HOW WHAT IS CALLED 1105 00:42:17,009 --> 00:42:20,279 A DENDROGRAM, HOW CLOSE DATA 1106 00:42:20,279 --> 00:42:22,381 POINT TO THE OTHER, IT IS VERY 1107 00:42:22,381 --> 00:42:24,449 INTUITIVE AND LTS VERY CLEAR HOW 1108 00:42:24,449 --> 00:42:25,884 LIKE TWO POINTS WILL BECOME THE 1109 00:42:25,884 --> 00:42:28,854 SAME CLUSTER. 1110 00:42:28,854 --> 00:42:30,923 THE TWO MAIN ISSUES, ONE AGAIN 1111 00:42:30,923 --> 00:42:33,992 IS THAT COMPUTATION, IT IS VERY, 1112 00:42:33,992 --> 00:42:34,760 VERY CHALLENGING. 1113 00:42:34,760 --> 00:42:36,395 THIS IS A SMALL PROBLEM, IT TOOK 1114 00:42:36,395 --> 00:42:38,163 ME A FEW MINUTES. 1115 00:42:38,163 --> 00:42:39,898 FOR LARGER PROBLEMS, YOU WOULD 1116 00:42:39,898 --> 00:42:41,433 NEED A LOT OF COMPUTER POWER. 1117 00:42:41,433 --> 00:42:42,968 THE OTHER ISSUE IS THAT 1118 00:42:42,968 --> 00:42:44,503 SOMETIMES YOU CREATE SMALL 1119 00:42:44,503 --> 00:42:48,640 CLUSTERS, SO WE SEE THAT THIS IS 1120 00:42:48,640 --> 00:42:50,309 BIG, 20,000 DATA POINTS BUT WE 1121 00:42:50,309 --> 00:42:52,477 ONLY HAVE A HANDFUL BELONGING TO 1122 00:42:52,477 --> 00:42:54,012 ONE CLUSTER. 1123 00:42:54,012 --> 00:42:55,514 SO SOMETIMES WE MAY BE SEEING A 1124 00:42:55,514 --> 00:42:58,483 LOT OF THESE SMALL CLUSTER 1125 00:42:58,483 --> 00:43:00,586 PROBLEMS WITH HIERARCHICAL 1126 00:43:00,586 --> 00:43:01,453 CLUSTERING METHOD. 1127 00:43:01,453 --> 00:43:03,755 THE LAST APPROACH IS ON MODEL 1128 00:43:03,755 --> 00:43:07,259 BASED CLUSTERING METHOD, AND IT 1129 00:43:07,259 --> 00:43:10,295 IS ALSO VERY COMPUTATIONALLY 1130 00:43:10,295 --> 00:43:10,596 INTENSIVE. 1131 00:43:10,596 --> 00:43:11,964 THE ADVANTAGE OF USING MODEL 1132 00:43:11,964 --> 00:43:13,265 BASED IS THAT IT IS A PRINCIPAL 1133 00:43:13,265 --> 00:43:18,737 WAY THAT WE MAKE MODEL 1134 00:43:18,737 --> 00:43:21,473 ASSUMPTION TO STRUCTURE DATA SO 1135 00:43:21,473 --> 00:43:23,542 THAT SCALING IS PART OF THE 1136 00:43:23,542 --> 00:43:23,942 MODEL. 1137 00:43:23,942 --> 00:43:26,511 SO THAT PART IS BEING TAKEN CARE 1138 00:43:26,511 --> 00:43:28,914 OF, AND WE ARE APPLYING THIS TO 1139 00:43:28,914 --> 00:43:37,689 THE SAME DATASET, THE DATA, TO 1140 00:43:37,689 --> 00:43:41,393 USE MODEL BASED CLUSTERING TO 1141 00:43:41,393 --> 00:43:43,128 IDENTIFY PATTERNS. 1142 00:43:43,128 --> 00:43:48,433 SO IN THIS EXAMPLE, SO THE 1143 00:43:48,433 --> 00:43:49,501 METHOD IDENTIFIED THREE 1144 00:43:49,501 --> 00:43:51,903 DIFFERENT CLUSTERS, AND EACH 1145 00:43:51,903 --> 00:43:54,740 CURVE IN EACH CLUSTER INDICATES 1146 00:43:54,740 --> 00:43:57,743 THE AVERAGE ACTIVITY DURING, 1147 00:43:57,743 --> 00:43:59,378 OVER DURING THE DAY, FROM 1148 00:43:59,378 --> 00:44:05,083 MIDNIGHT TO MIDNIGHT. 1149 00:44:05,083 --> 00:44:07,052 SO BASICALLY THIS IS UNSUPER 1150 00:44:07,052 --> 00:44:08,453 VICED LEARNING, SO ALL WE GAVE 1151 00:44:08,453 --> 00:44:13,425 TO THE ALGORITHM IS THE FIPPIT 1152 00:44:13,425 --> 00:44:15,294 DATA WITHOUT ANY SUPERVISED 1153 00:44:15,294 --> 00:44:17,362 OUTCOME AND THEN WE JUST COME UP 1154 00:44:17,362 --> 00:44:18,497 WITH EIGHT DIFFERENT PATTERNS, 1155 00:44:18,497 --> 00:44:21,967 AND THEN WE TRY TO ASSOCIATE 1156 00:44:21,967 --> 00:44:23,502 THESE PATTERNS, AND IN 1157 00:44:23,502 --> 00:44:26,138 PARTICULAR, WE IDENTIFY THAT 1158 00:44:26,138 --> 00:44:29,508 CLUSTER 5 IS KIND OF INTERESTING 1159 00:44:29,508 --> 00:44:30,609 WITH WEEKDAYS. 1160 00:44:30,609 --> 00:44:32,277 SO THIS IS A HEAT MAP 1161 00:44:32,277 --> 00:44:36,081 REPRESENTING THE PROPORTION OF 1162 00:44:36,081 --> 00:44:39,017 PARTICIPANTS IN EACH PATTERN ON 1163 00:44:39,017 --> 00:44:40,018 A GIVEN DAY. 1164 00:44:40,018 --> 00:44:44,923 SO ON THE WEEKDAY, WE SEE THAT 1165 00:44:44,923 --> 00:44:46,258 THERE'S A STRONG SIGNAL THAT 1166 00:44:46,258 --> 00:44:49,761 MOST PEOPLE ARE ENGAGING IN 1167 00:44:49,761 --> 00:44:50,529 CLUSTER 5 ACTIVITY. 1168 00:44:50,529 --> 00:44:51,863 WHAT HAPPENS IS THAT IF YOU PAY 1169 00:44:51,863 --> 00:44:55,000 ATTENTION TO THIS CLUSTER 5, 1170 00:44:55,000 --> 00:44:56,335 THERE'S PEAK ACTIVITIES 1171 00:44:56,335 --> 00:44:59,171 OCCURRING AT 8:00, NOON, 5:00, 1172 00:44:59,171 --> 00:45:00,706 BASICALLY TWO COMMUTER TIMES AND 1173 00:45:00,706 --> 00:45:01,340 LUNCH HOUR. 1174 00:45:01,340 --> 00:45:03,108 SO THIS IS ACTUALLY WHAT THE 1175 00:45:03,108 --> 00:45:04,843 CLUSTER ANALYSIS CAN HELP YOU 1176 00:45:04,843 --> 00:45:05,977 IDENTIFY A PATTERN THAT WE ARE 1177 00:45:05,977 --> 00:45:09,881 NOT EVEN AWARE OF THAT MAY BE 1178 00:45:09,881 --> 00:45:10,349 EXISTING. 1179 00:45:10,349 --> 00:45:12,517 SO THIS IS ACTUALLY ONE USEFUL 1180 00:45:12,517 --> 00:45:18,090 WAY TO USE CLUSTER ANALYSIS. 1181 00:45:18,090 --> 00:45:19,658 OUR FINAL SLIDE, JUST TO 1182 00:45:19,658 --> 00:45:22,494 SUMMARIZE WHAT WE TOUCHED ON, 1183 00:45:22,494 --> 00:45:24,696 MACHINE LEARNING IS A VAST AREA 1184 00:45:24,696 --> 00:45:27,432 AND IT'S VERY FAST GROWING, BUT 1185 00:45:27,432 --> 00:45:28,500 HOPEFULLY I'LL WALK THROUGH THE 1186 00:45:28,500 --> 00:45:30,702 MAIN ONE, THE MAIN APPLICATION, 1187 00:45:30,702 --> 00:45:33,572 WHICH IS SUPERVISED AND 1188 00:45:33,572 --> 00:45:34,873 UNSUPERVISED LEARNING AND I WANT 1189 00:45:34,873 --> 00:45:37,609 TO EMPHASIZE THAT THIS IS A 1190 00:45:37,609 --> 00:45:38,710 LEARNING METHOD COMPARED TO 1191 00:45:38,710 --> 00:45:39,878 HYPOTHESIS TESTING. 1192 00:45:39,878 --> 00:45:43,682 SO WE DO NOT HAVE -- TO PLACE IT 1193 00:45:43,682 --> 00:45:45,050 IN THE RIGHT CONTEXT, WE 1194 00:45:45,050 --> 00:45:46,385 PROBABLY WANT TO THINK ABOUT 1195 00:45:46,385 --> 00:45:48,487 THIS AS DISCOVERY, HYPOTHESIS 1196 00:45:48,487 --> 00:45:50,322 GENERATING, SO THAT ONCE WE'RE 1197 00:45:50,322 --> 00:45:51,890 GENERATING A HYPOTHESIS, THIS IS 1198 00:45:51,890 --> 00:45:53,058 SOMETHING THAT NEEDS TO BE 1199 00:45:53,058 --> 00:45:57,696 TESTED DOWN THE ROAD IN A MORE 1200 00:45:57,696 --> 00:45:58,463 RIGOROUS FASHION. 1201 00:45:58,463 --> 00:45:59,998 AND THEN I'LL TOUCH ON A NUMBER 1202 00:45:59,998 --> 00:46:05,670 OF ISSUES IN TERMS OF USING EACH 1203 00:46:05,670 --> 00:46:10,942 METHOD, BUT INCLUDING THE 1204 00:46:10,942 --> 00:46:13,678 IMPORTANCE OF THE SCALE, THE 1205 00:46:13,678 --> 00:46:18,350 FEATURE INPUTS, STARTING VALUES 1206 00:46:18,350 --> 00:46:20,452 OF SOME ALGORITHMS. 1207 00:46:20,452 --> 00:46:21,453 IMPORTANT APPLICATION OF MACHINE 1208 00:46:21,453 --> 00:46:23,622 LEARNING IS THAT WE NEED TO 1209 00:46:23,622 --> 00:46:28,326 REPRESENT DATA IN A PROPER WAY. 1210 00:46:28,326 --> 00:46:30,896 SO THE SUCCESS OF AN ALGORITHM 1211 00:46:30,896 --> 00:46:32,664 REALLY DEPENDS ON WHETHER DATA 1212 00:46:32,664 --> 00:46:34,866 CAN BE REPRESENTED IN A PROPER 1213 00:46:34,866 --> 00:46:35,167 WAY. 1214 00:46:35,167 --> 00:46:38,470 SO THAT'S A CONCEPT, AND 1215 00:46:38,470 --> 00:46:40,138 EMBEDDING AND TOKENNIZATION. 1216 00:46:40,138 --> 00:46:41,406 THIS IS SOMETHING I HAVE NOT 1217 00:46:41,406 --> 00:46:42,841 QUITE TOUCHED UPON, BUT IN THE 1218 00:46:42,841 --> 00:46:44,609 PREVIOUS EXAMPLE, SO THE IDEA 1219 00:46:44,609 --> 00:46:49,848 WHY THIS WORKS SO WELL IS THAT 1220 00:46:49,848 --> 00:46:55,086 WE REPRESENT THE TYPE OF DATA 1221 00:46:55,086 --> 00:46:58,523 USING WHAT WE CALL QCA 1222 00:46:58,523 --> 00:47:00,492 REPRESENTATION TO REDUCE 1223 00:47:00,492 --> 00:47:01,793 DIMENSION. 1224 00:47:01,793 --> 00:47:03,094 SO HELPING -- SO HAVING THE 1225 00:47:03,094 --> 00:47:04,396 RIGHT DATA REPRESENTATION IS 1226 00:47:04,396 --> 00:47:06,465 VERY CRITICAL, AND IT IS VERY 1227 00:47:06,465 --> 00:47:07,999 DATASET SPECIFIC. 1228 00:47:07,999 --> 00:47:12,771 SO FOR EACH LEARNING ALGORITHM, 1229 00:47:12,771 --> 00:47:14,272 TWO PART, ONE IS TRAINING, THE 1230 00:47:14,272 --> 00:47:15,807 OTHER IS INFERENCING. 1231 00:47:15,807 --> 00:47:18,176 SO I THINK I FOCUSED MUCH MORE 1232 00:47:18,176 --> 00:47:19,244 ON THE TRAINING COMPONENT 1233 00:47:19,244 --> 00:47:21,480 ELEMENT IN THIS PROCESS, BUT THE 1234 00:47:21,480 --> 00:47:22,814 INFERENCE IS GETTING MORE AND 1235 00:47:22,814 --> 00:47:24,449 MORE IMPORTANT, ESPECIALLY NOW 1236 00:47:24,449 --> 00:47:26,852 WE'RE TRYING TO USE MACHINE 1237 00:47:26,852 --> 00:47:28,920 LEARNING FOR STUFF THAN JUST 1238 00:47:28,920 --> 00:47:29,821 PREDICTING A NUMBER. 1239 00:47:29,821 --> 00:47:32,858 BUT WITH THAT, I THINK THIS IS 1240 00:47:32,858 --> 00:47:34,826 MY LAST SLIDE, AND THANK YOU FOR 1241 00:47:34,826 --> 00:47:45,103 STAYING WITH ME. 1242 00:47:47,239 --> 00:47:52,644 >> ANY QUESTIONS FROM ANYBODY? 1243 00:47:52,644 --> 00:48:02,787 [PAUSE] 1244 00:48:13,865 --> 00:48:24,342 >> I DON'T SEE ANYTHING IN CHAT. 1245 00:48:32,684 --> 00:48:37,756 >> I SEE ONE QUESTION. 1246 00:48:37,756 --> 00:48:42,227 IS LLM ALSO ML? 1247 00:48:42,227 --> 00:48:44,095 LLM, LARGE LANGUAGE MODEL, 1248 00:48:44,095 --> 00:48:45,730 THAT'S THE BACKBONE OF 1249 00:48:45,730 --> 00:48:47,265 GENERATIVE AI THESE DAYS. 1250 00:48:47,265 --> 00:48:51,002 A LOT OF TRAINING USE THE 1251 00:48:51,002 --> 00:48:52,871 CONCEPT FOR EXAMPLE, LLM IS 1252 00:48:52,871 --> 00:48:54,806 NOTHING BUT WHERE THE DATA 1253 00:48:54,806 --> 00:48:58,209 SOURCE IS AND THEN HOW THEY DO 1254 00:48:58,209 --> 00:49:01,413 THE GENERATIVE AI IS THAT FOR 1255 00:49:01,413 --> 00:49:05,317 THE TRAINING PART I THINK A LOT 1256 00:49:05,317 --> 00:49:07,285 OF THEM USE NEURAL NETWORK FOR 1257 00:49:07,285 --> 00:49:07,686 TRAINING. 1258 00:49:07,686 --> 00:49:09,087 BUT BEFORE THEY GET TO THE 1259 00:49:09,087 --> 00:49:10,555 NEURAL NETWORK, THEY COULD NEED 1260 00:49:10,555 --> 00:49:12,424 TO DO A LOT OF EMBEDDING AND A 1261 00:49:12,424 --> 00:49:13,191 LOT OF ORG. 1262 00:49:13,191 --> 00:49:14,859 THAT'S THE PART I MENTIONED 1263 00:49:14,859 --> 00:49:16,561 TOWARDS THE END OF THE 1264 00:49:16,561 --> 00:49:16,928 PRESENTATION. 1265 00:49:16,928 --> 00:49:19,531 BUT ONE IS DIFFERENT FROM 1266 00:49:19,531 --> 00:49:20,832 GENERATIVE AI AND WHAT I TALK 1267 00:49:20,832 --> 00:49:23,234 ABOUT IS THAT IN THE TRADITIONAL 1268 00:49:23,234 --> 00:49:24,269 MACHINE LEARNING, WE WANT TO DO 1269 00:49:24,269 --> 00:49:25,904 A PREDICTION, RIGHT? 1270 00:49:25,904 --> 00:49:28,640 SO FOR EXAMPLE, WITH THESE 1271 00:49:28,640 --> 00:49:29,975 CLINICAL FEATURES FROM GENERAL 1272 00:49:29,975 --> 00:49:32,644 MEDICAL RECORDS, WE WANT TO 1273 00:49:32,644 --> 00:49:36,448 DEFENS --WE WANT TO GIVE A RISKF 1274 00:49:36,448 --> 00:49:37,949 PATIENTS, THAT'S A PREDICTION. 1275 00:49:37,949 --> 00:49:40,285 IN THE CONTEXT OF GENERATIVE AI 1276 00:49:40,285 --> 00:49:42,020 INFERENCING IS MUCH HARDER 1277 00:49:42,020 --> 00:49:43,121 BECAUSE IT'S NOW NOT ASKING US 1278 00:49:43,121 --> 00:49:44,656 TO PRODUCE NUMBER, IT'S ASKING 1279 00:49:44,656 --> 00:49:46,524 US TO PRODUCE AN IMAGE. 1280 00:49:46,524 --> 00:49:47,959 WE WEREN'T TRYING TO RECOGNIZE 1281 00:49:47,959 --> 00:49:48,493 THE IMAGE. 1282 00:49:48,493 --> 00:49:51,529 WE WANT TO SAY PRODUCE A CAT, 1283 00:49:51,529 --> 00:49:51,730 RIGHT? 1284 00:49:51,730 --> 00:49:52,964 SO RECOGNIZING IT'S A CAT. 1285 00:49:52,964 --> 00:49:55,367 SO THE INFERENCING PART BECOMES 1286 00:49:55,367 --> 00:49:56,134 MUCH MORE COMPLICATED AND THIS 1287 00:49:56,134 --> 00:49:57,769 IS THE PART THAT TRADITIONALLY 1288 00:49:57,769 --> 00:49:59,871 WE HAVE NOT BEEN FOCUSING ON, 1289 00:49:59,871 --> 00:50:01,006 AND STILL MAY NOT BE SOMETHING 1290 00:50:01,006 --> 00:50:05,577 THAT WE WANT TO THINK TOO MUCH 1291 00:50:05,577 --> 00:50:07,312 ABOUT IN CLINICAL RESOURCE, BUT 1292 00:50:07,312 --> 00:50:09,314 THAT'S AN INTERESTING QUESTION. 1293 00:50:09,314 --> 00:50:12,784 I HOPE IT HELPS. 1294 00:50:12,784 --> 00:50:14,285 >> IT DOES. 1295 00:50:14,285 --> 00:50:24,462 THANK YOU. 1296 00:50:30,702 --> 00:50:31,269 [PAUSE] 1297 00:50:31,269 --> 00:50:39,544 >> I'M SURPRISED WE ONLY HAD ONE 1298 00:50:39,544 --> 00:50:41,946 QUESTION ABOUT GENERATIVE AI, I 1299 00:50:41,946 --> 00:50:42,981 THOUGHT PEOPLE WOULD BE ASKING 1300 00:50:42,981 --> 00:50:43,715 THAT QUESTION. 1301 00:50:43,715 --> 00:50:45,116 BUT ANYHOW, THANKS FOR THE 1302 00:50:45,116 --> 00:50:47,919 QUESTION. 1303 00:50:47,919 --> 00:50:48,253 [PAUSE] 1304 00:50:48,253 --> 00:50:48,720 >> THANK YOU, KEN. 1305 00:50:48,720 --> 00:50:51,356 THANK YOU, EVERYONE, FOR JOINING 1306 00:50:51,356 --> 00:50:52,891 TODAY, AND FOR JOINING OUR 1307 00:50:52,891 --> 00:50:54,659 SERIES. 1308 00:50:54,659 --> 00:51:05,070 >> THANK YOU VERY MUCH.