1 00:00:07,607 --> 00:00:10,510 WELL, HELLO, EVERYONE, 2 00:00:10,577 --> 00:00:12,212 I AM REALLY DELIGHTED TO BE HERE 3 00:00:12,278 --> 00:00:15,115 TO KICKOFF THE J. EDWARD RALL 4 00:00:15,181 --> 00:00:18,952 CULTURAL LECTURE AND FIRESIDE CHAT. 5 00:00:19,018 --> 00:00:22,522 WELCOME EVERYONE, IT IS MY 6 00:00:22,589 --> 00:00:25,525 DISTINCT PLEASURE TO INTRODUCE 7 00:00:25,592 --> 00:00:27,527 OUR SPEAKER TODAY 8 00:00:27,594 --> 00:00:30,330 DR. VINT CERF, A TRUE PIONEER 9 00:00:30,397 --> 00:00:32,365 IN THE DIGITAL WORLD. 10 00:00:32,432 --> 00:00:35,535 BUT FIRST I WILL NOTE THAT THE 11 00:00:35,602 --> 00:00:38,171 ANNUAL NIH RALL CULTURAL LECTURE IS 12 00:00:38,238 --> 00:00:41,541 HOLD IN HONOR OF DR. J. EDWARD RALL, 13 00:00:41,608 --> 00:00:42,542 WHO HELPED TO DEFINE THE 14 00:00:42,609 --> 00:00:44,544 MODERN INTRAMURAL RESEARCH 15 00:00:44,611 --> 00:00:47,514 PROGRAM AT NIH. 16 00:00:47,580 --> 00:00:51,518 IN THE 1950S HE ESTABLISHED A 17 00:00:51,584 --> 00:00:53,520 STABLE, ACADEMIC MINDED AND 18 00:00:53,586 --> 00:00:54,521 CULTURAL RICH COMMUNITY WITHIN 19 00:00:54,587 --> 00:00:56,523 OUR RAPIDLY EXPANDING GOVERNMENT 20 00:00:56,589 --> 00:00:59,526 AGENCY. 21 00:00:59,592 --> 00:01:00,527 DR. RALL, WAS KNOWN AS A 22 00:01:00,593 --> 00:01:02,529 CONSUMMATE SCIENTIST, A VERY 23 00:01:02,595 --> 00:01:07,534 CHARISMATIC MENTOR AND A 24 00:01:07,600 --> 00:01:09,536 RENAISSANCE MAN AND IT IS HE WHO 25 00:01:09,602 --> 00:01:11,538 RECOMMENDED IN 1984 THAT NIH ADD 26 00:01:11,604 --> 00:01:13,540 A CULTURAL LECTURE TO ITS 27 00:01:13,606 --> 00:01:18,545 DIRECTOR'S LECTURE SERIES TO 28 00:01:18,611 --> 00:01:21,514 ENRICH OUR SCIENTIFIC COMMUNITY. 29 00:01:21,581 --> 00:01:23,516 I AM TRULY DELIGHTED THAT 30 00:01:23,583 --> 00:01:24,517 DR. CERF AGREED TO BE OUR 31 00:01:24,584 --> 00:01:27,520 SPEAKER THIS YEAR. 32 00:01:27,587 --> 00:01:30,523 HIS TITLE IS VICE PRESIDENT AND 33 00:01:30,590 --> 00:01:32,525 CHIEF INTERNET EVANGELIST FOR 34 00:01:32,592 --> 00:01:33,526 GOOGLE. 35 00:01:33,593 --> 00:01:35,528 HE WILL TALK TO US ON THE TOPIC OF THE 36 00:01:35,595 --> 00:01:37,530 PROMISES AND PERILS OF AI IN 37 00:01:37,597 --> 00:01:38,531 BIOMEDICAL RESEARCH AND 38 00:01:38,598 --> 00:01:41,534 HEALTHCARE DELIVERY. 39 00:01:41,601 --> 00:01:43,536 THIS IS A TOPIC AS MANY OF YOU 40 00:01:43,603 --> 00:01:44,537 KNOW THAT IS VERY CLOSE TO MY 41 00:01:44,604 --> 00:01:46,539 HEART. 42 00:01:46,606 --> 00:01:48,541 BEFORE I CAME TO NIH IN 2022, I 43 00:01:48,608 --> 00:01:51,544 WAS A SURGICAL ONCOLOGIST AND 44 00:01:51,611 --> 00:01:52,512 RESEARCHER IN BOSTON. 45 00:01:52,579 --> 00:01:55,515 OVER TIME, I BECAME INCREASINGLY 46 00:01:55,582 --> 00:01:59,519 INVOLVED IN LEADING CLINICAL 47 00:01:59,586 --> 00:02:00,520 TRIALS AND WORKED ON THE EFFORTS 48 00:02:00,587 --> 00:02:06,526 TO INTEGRATE STANDARDIZED 49 00:02:06,593 --> 00:02:08,528 BIOMARKERS IN TRIALS, A 50 00:02:08,595 --> 00:02:10,530 COMPLICATED ENTERPRISE TRYING TO 51 00:02:10,597 --> 00:02:12,532 MIX BASIC SCIENCE END POINTS 52 00:02:12,599 --> 00:02:15,535 INCLUDING COMPLEX GENOMICS AND 53 00:02:15,602 --> 00:02:16,536 CLINICAL DATA AND NOW WE HAVE 54 00:02:16,603 --> 00:02:18,538 ANOTHER--YET ANOTHER COMPLEX 55 00:02:18,605 --> 00:02:20,540 HORIZON FOR US THAT IS, THAT OF 56 00:02:20,607 --> 00:02:25,512 USING ELECTRONIC HEALTH RECORDS 57 00:02:25,578 --> 00:02:26,513 AS WELL AS MACHINE LEARNING FOR 58 00:02:26,579 --> 00:02:28,515 CLINICAL RESEARCH. 59 00:02:28,581 --> 00:02:31,518 AT NIH I VERY MUCH HOPE FOR ALL 60 00:02:31,584 --> 00:02:32,519 OF US TO WORK TOGETHER TO 61 00:02:32,585 --> 00:02:34,521 DELIVER EVIDENCE-BASED 62 00:02:34,587 --> 00:02:36,523 HEALTHCARE TO ALL PEOPLE. 63 00:02:36,589 --> 00:02:38,525 AND I'M VERY EXCITED ABOUT HOW 64 00:02:38,591 --> 00:02:39,526 NEW TECHNIQUES IN MACHINE 65 00:02:39,592 --> 00:02:41,528 LEARNING AND ARTIFICIAL 66 00:02:41,594 --> 00:02:43,530 INTELLIGENCE CAN HELP US ADVANCE 67 00:02:43,596 --> 00:02:45,532 TOWARD THAT GOAL SO OF COURSE, I 68 00:02:45,598 --> 00:02:49,536 LIKE ALL OF YOU AM VERY MUCH 69 00:02:49,602 --> 00:02:50,537 LOOKING FORWARD TO HEARING 70 00:02:50,603 --> 00:02:51,538 DR. CERF'S INSIGHS ON THIS 71 00:02:51,604 --> 00:02:54,541 TOPIC. 72 00:02:54,607 --> 00:02:56,543 DR. CERF HAS BEEN AT GOOGLE 73 00:02:56,609 --> 00:02:57,544 SINCE 2005 AND CONTRIBUTES TO 74 00:02:57,610 --> 00:02:58,545 TO GLOABAL POLICY DEVELOPMENT AND 75 00:02:58,611 --> 00:03:02,515 CONTINUED STANDARDIZATION AND 76 00:03:02,582 --> 00:03:04,517 SPREAD OF THE INTERNET. 77 00:03:04,584 --> 00:03:06,519 PREVIOUSLY IN HIS CAREER HE 78 00:03:06,586 --> 00:03:07,520 WORKED AT MCI, THE CORPORATION 79 00:03:07,587 --> 00:03:10,523 FOR NATIONAL RESEARCH INITIATIVES AND 80 00:03:10,590 --> 00:03:11,524 DARPA, HE IS WIDELY KNOWN AS THE 81 00:03:11,591 --> 00:03:13,526 FATHER OF THE INTERNET HAVING CO 82 00:03:13,593 --> 00:03:16,529 DESIGNED THE INTERNET'S 83 00:03:16,596 --> 00:03:17,530 ARCHITECTURE AND THE TCP/IP 84 00:03:17,597 --> 00:03:19,532 PROTOCOLS, THE SETS OF RULES FOR 85 00:03:19,599 --> 00:03:22,535 MESSAGE FORMATS AND PROCEDURES 86 00:03:22,602 --> 00:03:24,537 THAT ALLOW MACHINES AND 87 00:03:24,604 --> 00:03:25,538 APPLICATION PROGRAMS TO EXCHANGE 88 00:03:25,605 --> 00:03:27,540 INFORMATION. 89 00:03:27,607 --> 00:03:31,544 HE HAS RECEIVED TOO MANY AWARDS 90 00:03:31,611 --> 00:03:34,514 TO LIST HERE TODAY BUT I WILL 91 00:03:34,581 --> 00:03:36,516 NAME A FEW NOTABLE, THE U.S. 92 00:03:36,583 --> 00:03:39,519 NATIONAL MEDAL OF TECHNOLOGY, 93 00:03:39,586 --> 00:03:40,520 THE ALAN TURING AWARD, THE 94 00:03:40,587 --> 00:03:42,522 PRESIDENTIAL MEDAL OF FREEDOM, 95 00:03:42,589 --> 00:03:45,525 MEMBER OF THE BRITISH ROYAL 96 00:03:45,592 --> 00:03:47,527 SOCIETY AND RECENTLY THE MARCONI SOCIETY 97 00:03:47,594 --> 00:03:49,529 LIFETIME ACHIEVEMENT AWARD. 98 00:03:49,596 --> 00:03:50,530 HE SERVES AS AN ADVISOR TO THE U.S. 99 00:03:50,597 --> 00:03:52,532 GOVERNMENT AGENCIES, INCLUDING THE NSF, NASA, 100 00:03:52,599 --> 00:03:57,537 DEPARTMENT OF DEFENSE, ENERGY 101 00:03:57,604 --> 00:03:58,538 AND COMMERCE, AND I HOPE US. 102 00:03:58,605 --> 00:04:00,540 HE HOLDS BACHELOR OF SCIENCE 103 00:04:00,607 --> 00:04:03,543 DEGREE IN MATHEMATICS FROM STANFORD, 104 00:04:03,610 --> 00:04:04,544 A MASTER IN SCIENCE AND Ph.D. 105 00:04:04,611 --> 00:04:07,513 DEGREES IN COMPUTER SCIENCE 106 00:04:07,580 --> 00:04:08,514 FROM UCLA AS WELL AS MANY, MANY 107 00:04:08,581 --> 00:04:10,516 HONORARY DOCTORATES SO I VERY 108 00:04:10,583 --> 00:04:12,518 MUCH LOOK FORWARD TO HEARING 109 00:04:12,585 --> 00:04:14,520 DR. CERF TODAY AND THEN ALSO TO 110 00:04:14,587 --> 00:04:17,523 OUR CONVERSATION WHICH WILL 111 00:04:17,590 --> 00:04:18,524 OCCUR AFTER THIS TALK, WHICH 112 00:04:18,591 --> 00:04:20,526 WILL ALSO GO OVER SOME OF THE 113 00:04:20,593 --> 00:04:24,530 QUESTIONS THAT ALL OF YOU HAVE 114 00:04:24,597 --> 00:04:26,532 SUBMITTED SO PLEASE JOIN ME 115 00:04:26,599 --> 00:04:34,540 IN WELCOMING 116 00:04:34,607 --> 00:04:38,544 DR. CERF. 117 00:04:38,611 --> 00:04:39,512 [ APPLAUSE ] 118 00:04:39,579 --> 00:04:41,514 >> WELL, GOOD AFTERNOON, 119 00:04:41,581 --> 00:04:43,516 EVERYONE, THANK YOU VERY MUCH FOR JOINING 120 00:04:43,583 --> 00:04:44,517 US TODAY, I CONSIDER THIS AN 121 00:04:44,584 --> 00:04:46,519 EXTRAORDINARY HONOR, I AM NOT 122 00:04:46,586 --> 00:04:47,520 SURE I DESERVE IT BUT YOU KNOW I'LL TAKE IT 123 00:04:47,587 --> 00:04:48,521 ANYWAY. 124 00:04:48,588 --> 00:04:49,522 I'M EXCITED ABOUT WHAT WE ARE 125 00:04:49,589 --> 00:04:51,524 GOING TO TALK ABOUT BUT I NEED 126 00:04:51,591 --> 00:04:55,528 TO WARN YOU AHEAD OF TIME THAT 127 00:04:55,595 --> 00:04:57,530 I'M NOT NECESSARILY THE WORLD'S 128 00:04:57,597 --> 00:04:58,531 BEST EXPERT ON ARTIFICIAL 129 00:04:58,598 --> 00:04:59,532 INTELLIGENCE. 130 00:04:59,599 --> 00:05:01,534 I'M A NETWORKING GUY BUT I WORK 131 00:05:01,601 --> 00:05:03,536 FOR A COMPANY AT GOOGLE WHERE WE 132 00:05:03,603 --> 00:05:05,538 MADE INVESTMESTS IN ARTIFICIAL 133 00:05:05,605 --> 00:05:06,539 INTELLIGENCE FOR QUITE A LONG 134 00:05:06,606 --> 00:05:07,540 TIME. 135 00:05:07,607 --> 00:05:10,543 MAJOR PROGRESS HAS BEEN MADE 136 00:05:10,610 --> 00:05:13,513 ESPECIALLY IN MACHINE LEARNING 137 00:05:13,579 --> 00:05:14,514 WHICH IS A THIRD FORM OF 138 00:05:14,580 --> 00:05:17,517 ARTIFICIAL INTELLIGENCE, THE 139 00:05:17,583 --> 00:05:18,518 TERM WAS DEVELOPED IN THE 1960S 140 00:05:18,584 --> 00:05:22,522 BY JOHN MCCARTHY AND HIS 141 00:05:22,588 --> 00:05:25,525 COLLEAGUES AT MIT AND THEN 142 00:05:25,591 --> 00:05:27,527 THEY STARTED OUT WITH HEURISTIC 143 00:05:27,593 --> 00:05:29,529 PROGRAMMING WHICH ROUGHLY 144 00:05:29,595 --> 00:05:30,530 SPEAKING MEANT SOMETIMES IT 145 00:05:30,596 --> 00:05:33,533 WORKS AND SOMETIMES IT DOESN'T. 146 00:05:33,599 --> 00:05:34,534 IF IT WORKS ALL THE TIME, THAT'S 147 00:05:34,600 --> 00:05:37,537 ENGINEERING, IF IT DOESN'T, 148 00:05:37,603 --> 00:05:38,538 THAT'S ARTIFICIAL INTELLIGENCE. 149 00:05:38,604 --> 00:05:40,540 [LAUGHTER] 150 00:05:40,606 --> 00:05:42,542 THEN THEY HAD SOME MORE 151 00:05:42,608 --> 00:05:43,543 ORGANIZED TECHNOLOGY CALLED 152 00:05:43,609 --> 00:05:47,513 EXPERT SYSTEMS, THESE ARE 153 00:05:47,580 --> 00:05:48,514 ROUGHLY SPEAKING COLLECTIONS OF 154 00:05:48,581 --> 00:05:50,516 IF, THEN, ELSE PHRASES, YOU KNOW 155 00:05:50,583 --> 00:05:51,517 IF THIS, THEN THAT. 156 00:05:51,584 --> 00:05:54,520 THEY WOULD CREATE LARGE NUMBERS 157 00:05:54,587 --> 00:05:55,521 OF THESE IF, THEN, ELSE PHRASES 158 00:05:55,588 --> 00:05:57,523 AND THEY WOULD TRIGGER EACH 159 00:05:57,590 --> 00:05:58,524 OTHER, IF THIS CONDITIONS HOLDS 160 00:05:58,591 --> 00:06:00,526 YOU THEN YOU SHOULD GO DO THAT, 161 00:06:00,593 --> 00:06:01,527 AND IF YOU DID THAT, THEN SOME 162 00:06:01,594 --> 00:06:03,529 OTHER CONDITION WOULD SHOW UP 163 00:06:03,596 --> 00:06:04,530 AND YOU WOULD KEEP TRAINING YOUR 164 00:06:04,597 --> 00:06:05,531 WAY THROUGH. 165 00:06:05,598 --> 00:06:08,534 THE PROBLEM WITH THAT IS THAT AS 166 00:06:08,601 --> 00:06:09,535 THE NUMBER OF IF, THAN ELSE 167 00:06:09,602 --> 00:06:11,537 PHRASES GOT BIG, THE SYSTEM 168 00:06:11,604 --> 00:06:14,540 STARTED TO BE INCONSISTENT AND 169 00:06:14,607 --> 00:06:15,541 IN SOME CASES COLLIDING WITH 170 00:06:15,608 --> 00:06:16,542 EACH OTHER. 171 00:06:16,609 --> 00:06:17,543 SO IT DIDN'T SCALE ALL THAT 172 00:06:17,610 --> 00:06:21,514 WELL, SO THEN THE NEXT THING 173 00:06:21,581 --> 00:06:22,515 THAT HAPPENS IS WHAT WE NOW CALL 174 00:06:22,582 --> 00:06:24,517 NEURAL NETWORKS. 175 00:06:24,584 --> 00:06:25,518 THEY HAD THEIR ORIGINS IN THE 176 00:06:25,585 --> 00:06:29,522 1960S BUT THE NEURAL NETWORK WAS 177 00:06:29,589 --> 00:06:30,523 ONLY ONE LAYER DEEP AND IT DIDN'T 178 00:06:30,590 --> 00:06:32,525 DO VERY MUCH. 179 00:06:32,592 --> 00:06:38,531 AND IN FACT, MCCARTHY AND SOME 180 00:06:38,598 --> 00:06:40,533 OF HIS ACTUALLY, MINSKy AND PAPERT AT MIT WROTE A 181 00:06:40,600 --> 00:06:42,535 BOOK CALLED PERCEPTRONS WHICH 182 00:06:42,602 --> 00:06:43,536 TRASHED THE IDEA OF NEURAL 183 00:06:43,603 --> 00:06:46,539 NETWORKS ON THIS THE BASES OF 184 00:06:46,606 --> 00:06:51,544 THIS ONE LAYER DEEP PERCEPTRON 185 00:06:51,611 --> 00:06:53,513 AND THAT INVOKED A NEW CLEAR 186 00:06:53,579 --> 00:06:54,514 WINNER FOR THE PERIOD OF ABOUT A 187 00:06:54,580 --> 00:06:56,516 DECADE. 188 00:06:56,582 --> 00:06:57,517 BUT THERE SOME PEOPLE WHO HUNG ON AND SAID YOU IF 189 00:06:57,583 --> 00:06:59,519 WE COULD BUILD A DEEPER NETWORK 190 00:06:59,585 --> 00:07:00,520 WE AUGHT TO BE ABLE TO CAPTURE 191 00:07:00,586 --> 00:07:02,522 MORE STATE INFORMATION THAT 192 00:07:02,588 --> 00:07:03,523 WOULD ALLOW US TO DETECT AND 193 00:07:03,589 --> 00:07:07,527 SENSE CORRELATIONS THAT WOULD BE 194 00:07:07,593 --> 00:07:09,529 OF INTEREST AND THAT'S WHERE WE 195 00:07:09,595 --> 00:07:10,530 HAVE COME AS A RESULT IN MACHINE 196 00:07:10,596 --> 00:07:13,533 LEARNING. 197 00:07:13,599 --> 00:07:14,534 NOW ALL OF YOU I'M SURE HAVE 198 00:07:14,600 --> 00:07:19,539 BEEN EXPOSED TO ONE OF THE ASPECTS 199 00:07:19,605 --> 00:07:20,540 OF MACHINE LEARNING CALLED LARGE 200 00:07:20,606 --> 00:07:23,543 LANGUAGE MODELS AND I SUSPECT WE 201 00:07:23,609 --> 00:07:25,545 WILL GET INTO THIS DURING OUR 202 00:07:25,611 --> 00:07:28,514 DISCUSSION BUT I WILL TELL YOU 203 00:07:28,581 --> 00:07:31,517 THAT TODAY'S LARGE LANGUAGE MODELS ARE 204 00:07:31,584 --> 00:07:33,519 SIMULTANEOUSLY UNBELIEVABLY AND 205 00:07:33,586 --> 00:07:35,521 UNXEBTEDLY BRILLIANT AND EQUALLY 206 00:07:35,588 --> 00:07:37,523 CONFUSING. 207 00:07:37,590 --> 00:07:39,525 IN FACT, YOU KNOW I LOOK A 208 00:07:39,592 --> 00:07:41,527 LITTLE BIT LIKE SIGMUND 209 00:07:41,594 --> 00:07:44,530 FREUD SO I'VE BEEN THINKING OF 210 00:07:44,597 --> 00:07:46,532 RECORDING A LITTLE VIDEO. 211 00:07:46,599 --> 00:07:47,533 >> WELL WHAT WE HAVE NOW IS THE 212 00:07:47,600 --> 00:07:50,536 ARTIFICIAL HEAD ON THE 213 00:07:50,603 --> 00:07:53,539 ARTIFICIAL EGO AND WHAT THEY'RE 214 00:07:53,606 --> 00:07:55,541 MISSING ARE THE ARTIFICIAL SUPER 215 00:07:55,608 --> 00:07:57,543 EG0 TO CONTROL THE 216 00:07:57,610 --> 00:08:00,513 UNCONTROLLABLE PULSES OF THE 217 00:08:00,580 --> 00:08:02,515 ARTIFICAL HEAD. 218 00:08:02,582 --> 00:08:03,516 [ APPLAUSE ] 219 00:08:03,583 --> 00:08:08,521 AND [LAUGHTER] 220 00:08:08,588 --> 00:08:10,523 AND IF I HAD ONE I WOULD HOLD UP A 221 00:08:10,590 --> 00:08:11,524 CIGAR AND SAY SOMETIMES A CIGAR 222 00:08:11,591 --> 00:08:12,525 IS JUST A CIGAR. 223 00:08:12,592 --> 00:08:14,527 [LAUGHTER] 224 00:08:14,594 --> 00:08:16,529 SO WHAT IS IMPORTANT THOUGH FOR YOU TO KNOW IS 225 00:08:16,596 --> 00:08:18,531 THAT QUITE APART FROM THE LARGE 226 00:08:18,598 --> 00:08:20,533 LANGUAGE MODELS AND THE KIND OF 227 00:08:20,600 --> 00:08:24,537 HALLUCINATIONS THAT THEY STILL 228 00:08:24,604 --> 00:08:25,538 EXHIBIT THEY STILL SHOW 229 00:08:25,605 --> 00:08:26,539 REMARKABLE CAPABILITY, THERE IS 230 00:08:26,606 --> 00:08:31,544 SOMETHING GOING ON IN THOSE 231 00:08:31,611 --> 00:08:33,512 MODELS THAT IMPLY A LOT OF 232 00:08:33,579 --> 00:08:34,513 SEMANTICS ARE HIDING IN THE 233 00:08:34,580 --> 00:08:35,514 STATISTICS OF WORD 234 00:08:35,581 --> 00:08:36,515 RELATIONSHIPS. 235 00:08:36,582 --> 00:08:37,516 THAT'S HOW THEY WORK. 236 00:08:37,583 --> 00:08:40,519 THEY GENERATE OUTPUT BASED ON 237 00:08:40,586 --> 00:08:42,521 THE STATISTICAL RELATIONSHIP OF 238 00:08:42,588 --> 00:08:44,524 WORDS OCCURRING ADJACENT TO EACH 239 00:08:44,590 --> 00:08:45,524 OTHER OR SOME DISTANCE APART. 240 00:08:45,591 --> 00:08:47,526 BUT THOSE WORDS HAVE COME FROM 241 00:08:47,593 --> 00:08:50,529 SOMETHING THAT HAD MEANING AND 242 00:08:50,596 --> 00:08:51,530 SO THERE IS A GHOST INSIDE THE 243 00:08:51,597 --> 00:08:53,532 LARGE LANGUAGE MODEL AND WE 244 00:08:53,599 --> 00:08:57,536 DON'T FULLY UNDERSTAND OR 245 00:08:57,603 --> 00:08:59,538 APPRECIATE ALL THAT IT CAN DO 246 00:08:59,605 --> 00:09:02,541 BUT WE DO KNOW THERE ARE SOME 247 00:09:02,608 --> 00:09:04,543 THINGS IT DOESN'T DO WELL, SO 248 00:09:04,610 --> 00:09:06,512 THE CONSEQUENCE OF THAT HOWEVER 249 00:09:06,579 --> 00:09:07,513 SHOULD NOT DISTORT YOUR 250 00:09:07,580 --> 00:09:08,514 UNDERSTANDING AND APPRECIATE FOR 251 00:09:08,581 --> 00:09:10,516 WHAT WILL BE POSSIBLE HERE AT 252 00:09:10,583 --> 00:09:12,518 NIH WHEN IT COMES TO GATHERING 253 00:09:12,585 --> 00:09:13,519 LARGE AMOUNTS OF DATA, 254 00:09:13,586 --> 00:09:16,522 UNDERSTANDING CORRELATIONS THAT 255 00:09:16,589 --> 00:09:18,524 ARE EXHIBITED BY THE THAT DATA 256 00:09:18,591 --> 00:09:20,526 AND THEN LOOKING FOR THE CAUSAL 257 00:09:20,593 --> 00:09:22,528 MODEL THAT WILL EXPLAIN WHAT 258 00:09:22,595 --> 00:09:23,529 THOSE CORRELATIONS ARE. 259 00:09:23,596 --> 00:09:25,531 AND THAT'S SOME OF WHAT I HOPE 260 00:09:25,598 --> 00:09:26,532 WE'LL GET TO TALK ABOUT THIS 261 00:09:26,599 --> 00:09:32,538 AFTERNOON. 262 00:09:32,605 --> 00:09:35,541 SO I'M GOING TO STOP HERE WITH 263 00:09:35,608 --> 00:09:36,542 MY LITTLE SERMON WHICH IS WHAT 264 00:09:36,609 --> 00:09:44,517 CHIEF INTERNET EVANGELISTS DO AND 265 00:09:44,583 --> 00:09:45,518 LET'S GET STARTED WITH OUR 266 00:09:45,584 --> 00:09:46,519 DISCUSSION. 267 00:09:46,585 --> 00:09:47,520 [ APPLAUSE ] 268 00:09:47,586 --> 00:09:48,521 I NEED TO WARN EVERYBODY, I WEAR 269 00:09:48,587 --> 00:09:50,523 HEARING AIDS AND I DON'T KNOW 270 00:09:50,589 --> 00:09:51,524 HOW GOOD OR BAD THE ACOUSTICS 271 00:09:51,590 --> 00:09:52,525 ARE HERE SO I'M GOING TO PUT A 272 00:09:52,591 --> 00:09:57,530 HEADSET ON. 273 00:09:57,596 --> 00:09:59,532 I AM NOT LISTENING TO THE LATEST 274 00:09:59,598 --> 00:10:01,534 FOOTBALL GAME; AT LEAST I WON'T 275 00:10:01,600 --> 00:10:03,536 ADMIT IT EVEN IF I WERE. 276 00:10:03,602 --> 00:10:04,537 I AM HOPING TO GET GOOD QUALITY 277 00:10:04,603 --> 00:10:06,539 SOUND BECAUSE I WOULD LIKE TO 278 00:10:06,605 --> 00:10:08,541 MAKE SURE I HEAR THE QUESTIONS. 279 00:10:08,607 --> 00:10:15,514 SO HERE WE GO, AND I HAVE TO 280 00:10:15,581 --> 00:10:17,516 ADJUST MY HEARING AIDS IN THE 281 00:10:17,583 --> 00:10:18,517 PROCESS. 282 00:10:18,584 --> 00:10:19,518 THERE WE GO. 283 00:10:19,585 --> 00:10:21,520 THE THEORY IS I CAN HEAR YOU 284 00:10:21,587 --> 00:10:23,522 WELL. 285 00:10:23,589 --> 00:10:24,523 LET'S TRY THAT. 286 00:10:24,590 --> 00:10:25,524 >> ARE YOU READY? 287 00:10:25,591 --> 00:10:26,525 >> YEAH, WELL I DON'T KNOW IF 288 00:10:26,592 --> 00:10:27,526 I'M READY BUT I CAN HEAR YOU. 289 00:10:27,593 --> 00:10:29,528 >> EXCELLENT. 290 00:10:29,595 --> 00:10:30,529 OKAY, HERE WE GO, WELCOME 291 00:10:30,596 --> 00:10:35,534 DR. CERF AND THANK YOU FOR 292 00:10:35,601 --> 00:10:36,535 JOINING ME FOR THIS 293 00:10:36,602 --> 00:10:38,537 CONVERSATION. 294 00:10:38,604 --> 00:10:39,538 A MAJOR PROBLEM WITH HEALTHCARE 295 00:10:39,605 --> 00:10:43,542 IS THAT ACTUALLY VERY, VERY 296 00:10:43,609 --> 00:10:45,544 LITTLE OF IT IS BASED UPON 297 00:10:45,611 --> 00:10:48,514 RIGOROUS CLINICAL DATA. 298 00:10:48,581 --> 00:10:50,516 >> OH MY GOD, THAT'S SCARY AS 299 00:10:50,583 --> 00:10:53,519 HELL. 300 00:10:53,586 --> 00:10:54,520 >> ISN'T THAT SCARY? 301 00:10:54,587 --> 00:10:56,522 [LAUGHTER] 302 00:10:56,589 --> 00:10:57,523 AND JUST TO GIVE YOU AN IDEA 303 00:10:57,590 --> 00:11:02,528 RIGOROUS ANALYSIS OF AT LEAST I 304 00:11:02,595 --> 00:11:04,530 KNOW FROM THE FIELDS OF ONCOLOGY 305 00:11:04,597 --> 00:11:05,531 AND CARDIOLOGY, ABOUT BETWEEN 5 306 00:11:05,598 --> 00:11:08,534 AND 10% OF THE DECISIONS THAT 307 00:11:08,601 --> 00:11:10,536 ARE MADE IN THE CLINIC ARE 308 00:11:10,603 --> 00:11:11,537 ACTUALLY BASED ON RANDOMIZED 309 00:11:11,604 --> 00:11:12,538 TRIALS. 310 00:11:12,605 --> 00:11:15,541 THAT'S IT. 311 00:11:15,608 --> 00:11:16,542 >> CAN I GET A CHANGE OF 312 00:11:16,609 --> 00:11:17,543 UNDERWEAR NOW? 313 00:11:17,610 --> 00:11:18,544 >> EXACTLY. 314 00:11:18,611 --> 00:11:19,512 SO IT REALLY IS FRIGHTENING AND 315 00:11:19,578 --> 00:11:21,514 IT DOES SAY SOMETHING ABOUT 316 00:11:21,580 --> 00:11:22,515 THE NEURAL NET THAT IS THE 317 00:11:22,581 --> 00:11:26,519 HUMAN BRAIN OF A DOCTOR THAT OUR 318 00:11:26,585 --> 00:11:27,520 CLINICIANS REALLY ARE ABLE TO 319 00:11:27,586 --> 00:11:28,521 PUT TOGETHER THESE REALLY 320 00:11:28,587 --> 00:11:30,523 DIVERSE COLLECTIONS OF DATA 321 00:11:30,589 --> 00:11:32,525 INCLUDING THEIR OWN PERSONAL 322 00:11:32,591 --> 00:11:35,528 EXPERIENCE AND COME UP WITH 323 00:11:35,594 --> 00:11:36,529 SOMETHING THAT REALLY DOES 324 00:11:36,595 --> 00:11:37,530 IMPROVE PEOPLE'S HEALTH TO A 325 00:11:37,596 --> 00:11:38,531 GREAT DEGREE. 326 00:11:38,597 --> 00:11:41,534 SO WE'RE NOT GOING TO DENIGRATE 327 00:11:41,600 --> 00:11:43,536 THE CONTRIBUTION OF THE HUMAN 328 00:11:43,602 --> 00:11:44,537 BRAIN OR THE EXPERIENCE OF A 329 00:11:44,603 --> 00:11:46,539 CLINICIAN, THOSE ARE REALLY 330 00:11:46,605 --> 00:11:47,540 CRITICAL BUT WE ALL THINK WE CAN 331 00:11:47,606 --> 00:11:49,542 DO BETTER. 332 00:11:49,608 --> 00:11:50,543 WE CAN INTRODUCE BETTER DATA TO 333 00:11:50,609 --> 00:11:51,544 MAKE BETTER DECISIONS. 334 00:11:51,610 --> 00:11:53,512 >> YOU KNOW IT JUST OCCURS TO ME 335 00:11:53,579 --> 00:11:54,513 THAT IT'S A GOOD THING THIS IS 336 00:11:54,580 --> 00:11:56,515 CALLED THE NATIONAL INSTITUTES OF 337 00:11:56,582 --> 00:11:57,516 HEALTH AND NOT THE NATIONAL 338 00:11:57,583 --> 00:11:59,518 INSTITUTES OF DISEASE BECAUSE I 339 00:11:59,585 --> 00:12:04,523 DON'T KNOW IF ANYBODY WOULD FUND 340 00:12:04,590 --> 00:12:05,524 THAT, YOU KNOW? 341 00:12:05,591 --> 00:12:06,525 >> EXACTLY. 342 00:12:06,592 --> 00:12:07,526 [LAUGHTER] 343 00:12:07,593 --> 00:12:10,529 SO MY BIGGEST AMBITION AS NIH 344 00:12:10,596 --> 00:12:11,530 DIRECTOR IS TO PRODUCE MORE 345 00:12:11,597 --> 00:12:15,534 EVIDENCE-BASED DATA DRIVEN 346 00:12:15,601 --> 00:12:17,536 HEALTHCARE -- 347 00:12:17,603 --> 00:12:18,537 >> I LOVE THAT. 348 00:12:18,604 --> 00:12:19,538 >> ACROSS THE WORLD 349 00:12:19,605 --> 00:12:20,539 AVAILABLE TO EVERY PATIENT. 350 00:12:20,606 --> 00:12:21,540 AND I WOULD LIKE EVERY DAY 351 00:12:21,607 --> 00:12:23,542 VISITS TO THE DOCTOR, THIS IS 352 00:12:23,609 --> 00:12:25,544 THE IMPORTANT PART, EVERY DAY 353 00:12:25,611 --> 00:12:28,514 VISITS TO THE DOCTOR TO 354 00:12:28,581 --> 00:12:29,515 CONTRIBUTE TO OUR EVIDENCE-BASE. 355 00:12:29,582 --> 00:12:31,517 >> YES, ACTUALLY, ACTUALLY, THIS 356 00:12:31,584 --> 00:12:34,520 IS A CONVERSATION, THIS ISN'T 357 00:12:34,587 --> 00:12:35,521 GOING TO BE JUST Q&A. 358 00:12:35,588 --> 00:12:36,522 THROW AWAY THE QUESTIONS. 359 00:12:36,589 --> 00:12:40,526 OKAY, SO, VISIT TO THE DOCTOR, 360 00:12:40,593 --> 00:12:41,527 IF I WERE A DOCTOR AND MY 361 00:12:41,594 --> 00:12:44,530 PATIENTS ONLY CAME IN WHEN THEY 362 00:12:44,597 --> 00:12:46,532 HAD SYMPTOMS AND THEY WERE SICK, 363 00:12:46,599 --> 00:12:48,534 MY STATISTICS WOULD SAY, MY 364 00:12:48,601 --> 00:12:50,536 PATIENT IS ALWAYS SICK BECAUSE I 365 00:12:50,603 --> 00:12:51,537 NEVER SEE THEM WHEN THEY'RE 366 00:12:51,604 --> 00:12:53,539 WELL, SO ONE OF THE THINGS THAT'S VERY 367 00:12:53,606 --> 00:12:55,541 IMPORTANT I THINK IS FOR US IN 368 00:12:55,608 --> 00:12:57,543 TERMS OF EVIDENCE-BASED MEDICINE 369 00:12:57,610 --> 00:12:58,544 IS TO COLLECT INFORMATION ABOUT 370 00:12:58,611 --> 00:13:00,513 MY PATIENTS WHEN THEY'RE WELL, 371 00:13:00,579 --> 00:13:03,516 ALSO, SO I HAVE SOME SENSE FOR 372 00:13:03,582 --> 00:13:05,518 WHAT'S NORMAL FOR THEM AND THAT 373 00:13:05,584 --> 00:13:07,520 WILL HELP ME FIGURE OUT WHETHER 374 00:13:07,586 --> 00:13:09,522 THEY HAVE GONE IN SOME EXCURSION 375 00:13:09,588 --> 00:13:10,523 AWAY FROM THE NORM. 376 00:13:10,589 --> 00:13:11,524 SO INSTRUMENTING OURSELVES MIGHT 377 00:13:11,590 --> 00:13:14,527 TURN OUT TO BE VERY BENEFICIAL 378 00:13:14,593 --> 00:13:16,529 FOR OURSELVES AND FOR MEDICINE 379 00:13:16,595 --> 00:13:17,530 IN GENERAL AND I'M NOT SURE WHAT 380 00:13:17,596 --> 00:13:19,532 KINDS OF INSTRUMENTATION WE 381 00:13:19,598 --> 00:13:20,533 MIGHT WANT BUT YOU'D KNOW THAT 382 00:13:20,599 --> 00:13:21,534 BETTER THAN I DO. 383 00:13:21,600 --> 00:13:22,535 I DO THINK WE NEED TO THINK 384 00:13:22,601 --> 00:13:23,536 ABOUT THAT. 385 00:13:23,602 --> 00:13:25,538 WE NEED TO HAPPENED PEOPLE'S 386 00:13:25,604 --> 00:13:26,539 STATE OF AFFAIRS WHEN THEY'RE 387 00:13:26,605 --> 00:13:28,541 HEALTHY AS WELL AS WHEN THEY'RE 388 00:13:28,607 --> 00:13:29,542 NOT. 389 00:13:29,608 --> 00:13:32,545 >> COULDN'T AGREE MORE AND SO 390 00:13:32,611 --> 00:13:35,514 MANY OF OUR DISEASES THAT EMERGE 391 00:13:35,581 --> 00:13:38,517 ARE PRESENT FAR, FAR BEFORE EVEN 392 00:13:38,584 --> 00:13:42,521 THE PATIENT IS AWARE OF THEM. 393 00:13:42,588 --> 00:13:43,522 >> NO SYMPTOMS. WE COULD NAME SO MANY. 394 00:13:43,589 --> 00:13:44,523 NO SYMPTOMS. 395 00:13:44,590 --> 00:13:45,524 >> AND THIS IS ANOTHER HOPE THAT 396 00:13:45,591 --> 00:13:49,528 SOMEHOW COLLECTING BETTER DATA 397 00:13:49,595 --> 00:13:53,532 MIGHT HELP US UNCOVER THINGS 398 00:13:53,599 --> 00:13:54,533 THAT MAYBE EVEN THE PATIENT 399 00:13:54,600 --> 00:13:57,536 DOESN'T REALIZE. 400 00:13:57,603 --> 00:13:59,538 >> I LOVE TO HEAR YOU SAY THIS 401 00:13:59,605 --> 00:14:00,539 BECAUSE THINK ABOUT OUR MEDICINE 402 00:14:00,606 --> 00:14:02,541 UP UNTIL THE POINT WHERE WE HAD 403 00:14:02,608 --> 00:14:04,543 THE ABILITY TO DO DNA SEQUENCING 404 00:14:04,610 --> 00:14:08,514 AND IT WAS BASED ON JUST ON OBERVATIONS 405 00:14:08,581 --> 00:14:10,516 AND WE WERE GETTING DATA, BLOOD 406 00:14:10,583 --> 00:14:12,518 TESTS AND WE COULD GET URINE 407 00:14:12,585 --> 00:14:16,522 SAMPLES AND ALL THESE OTHER 408 00:14:16,589 --> 00:14:17,523 KINDS OF THINGS BUT THE DNA 409 00:14:17,590 --> 00:14:19,525 SEQUENCING LED US TO THINK THAT 410 00:14:19,592 --> 00:14:21,527 THAT WAS THE TOP OF THE 411 00:14:21,594 --> 00:14:23,529 MOUNTAIN, YOU KNOW, ONCE WE GOT 412 00:14:23,596 --> 00:14:25,531 THAT, WE KNEW EVERYTHING WE 413 00:14:25,598 --> 00:14:26,532 BEING POSSIBLY KNOW AND THEN WE 414 00:14:26,599 --> 00:14:27,533 DISCOVERED PROTEOMICS AND THEN 415 00:14:27,600 --> 00:14:30,536 AFTER THAT WE DISCOVERED METALLOMICS 416 00:14:30,603 --> 00:14:32,538 AND YOUR POINT THOUGH 417 00:14:32,605 --> 00:14:34,540 WHICH IS 418 00:14:34,607 --> 00:14:36,542 I TAKE VERY 419 00:14:36,609 --> 00:14:39,511 SERIOUSLY IS THAT WE CAN CAPTURE 420 00:14:39,578 --> 00:14:43,515 INFORMATION THAT LATER WILL TELL 421 00:14:43,582 --> 00:14:44,516 US A CORRELATION, WE MIGHT 422 00:14:44,583 --> 00:14:45,517 DISCOVER AFTER WE DISCOVERED THAT 423 00:14:45,584 --> 00:14:47,519 THE PATIENT HAS AN ILLNESS OR 424 00:14:47,586 --> 00:14:49,521 DISEASE, WE CAN LOOK BACK AT THE 425 00:14:49,588 --> 00:14:52,524 DATA THAT WE COLLECTED AND SAY, 426 00:14:52,591 --> 00:14:53,525 IS THERE ANYTHING, ANY 427 00:14:53,592 --> 00:14:55,527 CORRELATION IN THERE OR A CAUSAL 428 00:14:55,594 --> 00:14:59,531 MODEL THAT WE CAN ASSOCIATE WITH 429 00:14:59,598 --> 00:15:00,532 IT 430 00:15:00,599 --> 00:15:02,534 AND THAT WOULD SPEED UP OUR 431 00:15:02,601 --> 00:15:03,535 ABILITY TO RECOGNIZE MEDICAL 432 00:15:03,602 --> 00:15:05,537 CONDITIONS WOULDN'T IT. 433 00:15:05,604 --> 00:15:06,538 >> ABSOLUTELY AND THOSE ARE THE 434 00:15:06,605 --> 00:15:08,540 DATA SETS WE LACK TODAY. 435 00:15:08,607 --> 00:15:10,542 WE DON'T HAVE A LONGITUDINAL 436 00:15:10,609 --> 00:15:12,544 RECORD, WE DO IN SOME, WE DO 437 00:15:12,611 --> 00:15:13,512 HAVE SOME REALLY WONDERFUL 438 00:15:13,579 --> 00:15:15,514 LONG-TERM OBSERVATIONAL COHORTS 439 00:15:15,581 --> 00:15:18,517 BUT WE HAVE IT IN PIECES AND WE 440 00:15:18,584 --> 00:15:19,518 DON'T HAVE THEM FOR EVERYONE IN 441 00:15:19,585 --> 00:15:21,520 THE DIVERSITY THAT WE NEED IT SO 442 00:15:21,587 --> 00:15:22,521 WE'RE NOT EVEN ON TO OUR FIRST 443 00:15:22,588 --> 00:15:23,522 QUESTION YET, OKAY. 444 00:15:23,589 --> 00:15:24,523 SEE WHAT I MEAN. 445 00:15:24,590 --> 00:15:26,525 OKAY, SO LET'S DO OUR FIRST 446 00:15:26,592 --> 00:15:28,527 QUESTION AND THIS FITS WITH WHAT 447 00:15:28,594 --> 00:15:30,529 WE'VE BEEN TALKING ABOUT. 448 00:15:30,596 --> 00:15:33,532 WHAT DO YOU SEE AS THE 449 00:15:33,599 --> 00:15:35,534 CHALLENGES AND OPPORTUNITIES ON 450 00:15:35,601 --> 00:15:38,537 THIS FRONT, BASICALLY FOR MAKING 451 00:15:38,604 --> 00:15:39,538 HEALTHCARE TRULY EVIDENCE-BASED? 452 00:15:39,605 --> 00:15:42,541 >> WELL, WE JUST TALKED A LITTLE 453 00:15:42,608 --> 00:15:46,512 BIT ABOUT THAT AND I THINK AS I 454 00:15:46,578 --> 00:15:48,514 IMAGINED ALL OF US ARE NOT 455 00:15:48,580 --> 00:15:50,516 STATIONARY, AND SO WE MOVE 456 00:15:50,582 --> 00:15:52,518 AROUND, AND AS WE MOVE AROUND WE GO TO DIFFERENT 457 00:15:52,584 --> 00:15:53,519 DOCTORS, DIFFERENT CLINICS 458 00:15:53,585 --> 00:15:54,520 AND HOSPITALS, OUR HEALTH 459 00:15:54,586 --> 00:15:55,521 RECORDS ARE SCATTERED ALL OVER 460 00:15:55,587 --> 00:15:56,522 EVERYWHERE, SOME OF THEM ARE 461 00:15:56,588 --> 00:16:00,526 STILL ON PAPER, IN FACT MAYBE 462 00:16:00,592 --> 00:16:02,528 MOST OF THEM AND SO THE SYSTEM 463 00:16:02,594 --> 00:16:05,531 ISN'T CONTRIBUTING TO EVIDENCE AS 464 00:16:05,597 --> 00:16:06,532 WE WOULD LIKE IT, SO ONE THING WE 465 00:16:06,598 --> 00:16:08,534 MIGHT COLLECTIVELY WORK ON IS 466 00:16:08,600 --> 00:16:12,538 HOW TO CAPTURE THAT EVIDENTIAL 467 00:16:12,604 --> 00:16:13,539 INFORMATION AND MAKE IT COHERENT 468 00:16:13,605 --> 00:16:15,541 AND COLLECTED IN SOME SENSE AND 469 00:16:15,607 --> 00:16:18,544 A LOT OF PEOPLE ARE VERY NERVOUS 470 00:16:18,610 --> 00:16:20,512 ABOUT INFORMATION, HIPAA IS 471 00:16:20,579 --> 00:16:22,514 WITH US AND IN A FUNNY WAY, IT 472 00:16:22,581 --> 00:16:23,515 MAY BE INHIBITING OUR ABILITY TO 473 00:16:23,582 --> 00:16:28,520 DO EVIDENCE-BASED MEDICINE. 474 00:16:28,587 --> 00:16:30,522 SO MAYBE WE SHOULD RETHINK THAT 475 00:16:30,589 --> 00:16:31,523 AND RECOGNIZE THAT EACH OF US 476 00:16:31,590 --> 00:16:39,531 HAS THE OPPORTUNITY TO 477 00:16:39,598 --> 00:16:40,532 CONTRIBUTE TO A LARGER DATABASE 478 00:16:40,599 --> 00:16:42,534 OF HEALTH KNOWLEDGE AND CLINICAL 479 00:16:42,601 --> 00:16:43,535 KNOWLEDGE THAT WOULD HELP 480 00:16:43,602 --> 00:16:48,540 EVERYONE INCLUDING OURSELVES. 481 00:16:48,607 --> 00:16:50,542 AND ONE OF THE THINGS WE TALKED ABOUT AT ONE POINT 482 00:16:50,609 --> 00:16:52,544 IN THE TEAM WAS WHAT IF EVERYTIME SOMEONE 483 00:16:52,611 --> 00:16:53,512 WHO CAME INTO THE DOCTOR'S OFFICE 484 00:16:53,579 --> 00:16:58,517 WITH A PARTICULAR DISEASE, 485 00:16:58,584 --> 00:17:01,520 DIABETES, WE'LL JUST PICK THAT, 486 00:17:01,587 --> 00:17:02,521 THEY COULD BE ASKED TO BE PART 487 00:17:02,588 --> 00:17:05,524 OF A CLINICAL TRIAL. 488 00:17:05,591 --> 00:17:06,525 THEY MAY SAY YES OR SAY NO, BUT 489 00:17:06,592 --> 00:17:07,526 WHAT IF EVERYBODY WHO HAS 490 00:17:07,593 --> 00:17:11,530 DIABETES IS PART OF THE DATA 491 00:17:11,597 --> 00:17:13,532 COLLECTION AND THE ONES WHO--SO 492 00:17:13,599 --> 00:17:16,535 NOW YOU HAVE LARGE OBSERVATIONAL 493 00:17:16,602 --> 00:17:18,537 COHORTS WITH THE CLINICAL TRIALS 494 00:17:18,604 --> 00:17:19,538 NESTED WITHIN THEM AND THE 495 00:17:19,605 --> 00:17:21,540 ABILITY TO LOOK AT THE 496 00:17:21,607 --> 00:17:23,542 DIFFERENCE AND ONE OF THE MOST 497 00:17:23,609 --> 00:17:25,544 CRITICAL THINGS THAT WE THINK 498 00:17:25,611 --> 00:17:27,513 ARE ALL UNDERSTANDING, IS WHO 499 00:17:27,579 --> 00:17:28,514 SAYS YES TO THE TRIAL AND HOW 500 00:17:28,580 --> 00:17:31,517 DIFFERENT THOSE PEOPLE ARE AND 501 00:17:31,583 --> 00:17:32,518 EVEN WHEN PEOPLE HAVE THE SAME 502 00:17:32,584 --> 00:17:33,519 DISEASE, RIGHT? 503 00:17:33,585 --> 00:17:39,525 >> YEAH, BUT THIS IS VERY 504 00:17:39,591 --> 00:17:40,526 RELEVANT TO ME, MY 2 BROTHERS 505 00:17:40,592 --> 00:17:41,527 ARE DIABETIC SO THAT WOULD BE 506 00:17:41,593 --> 00:17:43,529 HELPFUL TO DO. 507 00:17:43,595 --> 00:17:45,531 >> OKAY, HOW DO YOU THINK AI AND 508 00:17:45,597 --> 00:17:46,532 MACHINE LEARNING WILL HELP MAKE 509 00:17:46,598 --> 00:17:49,535 THE CONNECTIONS WE NEED BETWEEN 510 00:17:49,601 --> 00:17:50,536 THE BENCH AND THE BEDSIDE 511 00:17:50,602 --> 00:17:51,537 PRACTICALLY? 512 00:17:51,603 --> 00:17:52,538 >> WELL AS A PRACTICAL MATTER 513 00:17:52,604 --> 00:17:54,540 WHAT WE'VE LEARNED ABOUT MACHINE 514 00:17:54,606 --> 00:17:57,543 LEARNING IS THAT IT'S CAPABLE 515 00:17:57,609 --> 00:18:03,515 OF INGESTING UNBELIEVABLE AMOUNT 516 00:18:03,582 --> 00:18:04,516 OF DATA AND THEN DISCOVERING 517 00:18:04,583 --> 00:18:05,517 RELATIONSHIPS. 518 00:18:05,584 --> 00:18:07,519 NOW SOMETIMES THOSE THEY'RE NOT CAUSAL 519 00:18:07,586 --> 00:18:09,521 AND THEY'RE HAPPEN STANCE, BUT 520 00:18:09,588 --> 00:18:11,523 WHERE WE CAN IDENTIFY A CAUSAL 521 00:18:11,590 --> 00:18:13,525 MODEL THEN HAVING COLLECTED ON 522 00:18:13,592 --> 00:18:17,529 ALL THAT DATA GIVES US 523 00:18:17,596 --> 00:18:19,531 CONFIDENCE THAT THE CAUSAL MODEL 524 00:18:19,598 --> 00:18:22,534 IS LET'S SAY ACCURATE, IT'S 525 00:18:22,601 --> 00:18:23,535 USEFUL FOR PREDICTIONS OR FOR 526 00:18:23,602 --> 00:18:26,538 DIAGNOSIS, BUT WE RIGHT NOW, WE 527 00:18:26,605 --> 00:18:28,540 DO NOT HAVE A REGULAR PRACTICE 528 00:18:28,607 --> 00:18:30,542 OF ACCUMULATING THAT INFORMATION 529 00:18:30,609 --> 00:18:33,512 AND PUTTING IT INTO A FORM WHERE 530 00:18:33,579 --> 00:18:35,514 WE CAN PERFORM THOSE KINDS OF 531 00:18:35,581 --> 00:18:43,522 MACHINE LEARNING ALGORITHMS SO 532 00:18:43,589 --> 00:18:45,524 WE MAY HAVE TO FIND METHODS BY 533 00:18:45,591 --> 00:18:46,525 WHICH PEOPLE ARE--HAVE INCENTIVE 534 00:18:46,592 --> 00:18:48,527 TO PARTICIPATE IN THE DATA 535 00:18:48,594 --> 00:18:49,528 COLLECTION PROGRAM. 536 00:18:49,595 --> 00:18:51,530 YOU JUST SUGGIESTED VERY 537 00:18:51,597 --> 00:18:52,531 INTERESTING WAYS OF DOING THAT. 538 00:18:52,598 --> 00:18:54,533 AND I THINK THE PEOPLE 539 00:18:54,600 --> 00:19:01,540 UNDERSTOOD THAT SHARING OF THAT 540 00:19:01,607 --> 00:19:02,541 DATA IS ALMOST AS IMPORTANT AS 541 00:19:02,608 --> 00:19:07,513 GIVING BLOOD. 542 00:19:07,579 --> 00:19:08,514 KNOWLEDGE IS REALLY IMPORTANT. 543 00:19:08,580 --> 00:19:10,516 >> SO TECHNICALLY DO WE HAVE THE 544 00:19:10,582 --> 00:19:14,520 LEVEL OF TECHNOLOGY THAT ALLOWS 545 00:19:14,586 --> 00:19:15,521 US TO DO THIS WITH ENOUGH 546 00:19:15,587 --> 00:19:17,523 SECURITY AND SAFETY TO CONVINCE 547 00:19:17,589 --> 00:19:18,524 PEOPLE TO CONTRIBUTE THEIR 548 00:19:18,590 --> 00:19:19,525 HEALTH DATA. 549 00:19:19,591 --> 00:19:24,530 LIKE LET'S SAY SOMEBODY HAS A 550 00:19:24,596 --> 00:19:25,531 DISEASE THAT HAS SOME STIGMA 551 00:19:25,597 --> 00:19:26,532 ASSOCIATED WITH IT. 552 00:19:26,598 --> 00:19:28,534 OF COURSE, THAT WOULD GIVE THEM 553 00:19:28,600 --> 00:19:30,536 PAUSE, DO WE HAVE THE TECHNOLOGY 554 00:19:30,602 --> 00:19:32,538 TO ALLOW THAT LEVEL OF SECURITY 555 00:19:32,604 --> 00:19:34,540 AND COULD SOME OF THESE 556 00:19:34,606 --> 00:19:35,541 ALGORITHMS HELP US WITH THAT. 557 00:19:35,607 --> 00:19:39,545 >> I'M PRETTY SURE THE ALGORITHMS 558 00:19:39,611 --> 00:19:41,513 WILL HELP BUT I WILL SAY 559 00:19:41,580 --> 00:19:45,517 THAT IT'S SOFTWARE AND SOFTWARE 560 00:19:45,584 --> 00:19:49,521 HAS BUGS AND BUGS GET EXPLOITED. 561 00:19:49,588 --> 00:19:50,522 AND WHEN PEOPLE WAVE THE 562 00:19:50,589 --> 00:19:52,524 CRYPTOGRAPHY FLAG AROUND, WHICH 563 00:19:52,591 --> 00:19:53,525 IS NOT UNREASONABLE, WE CAN 564 00:19:53,592 --> 00:19:55,527 PROTECT DATA FROM EXPOSURE BUT 565 00:19:55,594 --> 00:19:59,531 SOMETIMES THE BUGS GET PAST SOME 566 00:19:59,598 --> 00:20:01,533 OF THOSE MECHANISMS. 567 00:20:01,600 --> 00:20:02,534 WE SPENT AN AWFUL LOT OF TIME AT 568 00:20:02,601 --> 00:20:05,537 GOOGLE DOING EVERYTHING WE CAN 569 00:20:05,604 --> 00:20:07,539 TO KEEP DATA FROM BEING EXPOSED 570 00:20:07,606 --> 00:20:08,540 OTHER THAN TO AUTHORIZED 571 00:20:08,607 --> 00:20:09,541 PARTIES. 572 00:20:09,608 --> 00:20:11,543 WE MAKE HEAVY USE OF 573 00:20:11,610 --> 00:20:13,512 CRYPTOGRAPHY TO DO THAT, WE 574 00:20:13,579 --> 00:20:14,513 INSIST ON TWO FOCTOR 575 00:20:14,580 --> 00:20:16,515 AUTHENTICATION AND OTHER 576 00:20:16,582 --> 00:20:18,517 METHODS, AND SO I THINK THE 577 00:20:18,584 --> 00:20:19,518 ANSWER IS THAT TECHNOLOGY EXISTS 578 00:20:19,585 --> 00:20:21,520 TO PROTECT INFORMATION BUT IT 579 00:20:21,587 --> 00:20:23,522 HAS TO EXIST EVERYWHERE WHERE 580 00:20:23,589 --> 00:20:26,525 THE PROTECTION AND NEEDED SO 581 00:20:26,592 --> 00:20:27,526 THERE'S STILL WORK TO BE DONE. 582 00:20:27,593 --> 00:20:28,527 >> UNFORTUNATELY. 583 00:20:28,594 --> 00:20:29,528 OKAY. 584 00:20:29,595 --> 00:20:31,530 HERE'S A NEW ONE, YOU'RE KNOWN FOR 585 00:20:31,597 --> 00:20:32,531 STANDARDIZING INFORMATION TO 586 00:20:32,598 --> 00:20:33,532 ALLOW FOR MORE WIDE SPREAD 587 00:20:33,599 --> 00:20:35,534 COMMUNICATION. 588 00:20:35,601 --> 00:20:37,536 WHAT ARE THE SOME OF THE LESSONS 589 00:20:37,603 --> 00:20:39,538 YOU'VE LEARNED FOR THOSE OF US 590 00:20:39,605 --> 00:20:42,541 IN BIOMEDICAL RESEARCH WHERE 591 00:20:42,608 --> 00:20:44,543 STANDARDIZATION IS PRETTY MUCH 592 00:20:44,610 --> 00:20:46,511 NONEXISTENT, UNFORTUNATELY. 593 00:20:46,578 --> 00:20:47,512 >> WELL, IN A FUNNY WAY, THERE'S 594 00:20:47,579 --> 00:20:51,516 HL7 AND MANY OF YOU KNOW ABOUT, 595 00:20:51,583 --> 00:20:54,519 MANY OF YOU MUST KNOW AS WELL, I 596 00:20:54,586 --> 00:20:56,521 WOULD SAY THAT STANDARDIZATION 597 00:20:56,588 --> 00:20:57,522 IS REALLY IMPORTANT NOT ONLY FOR 598 00:20:57,589 --> 00:21:00,525 CLASSIFYING THE INFORMATION THAT 599 00:21:00,592 --> 00:21:03,528 WE HAVE, SO THAT WE KNOW WHEN WE 600 00:21:03,595 --> 00:21:06,531 HAVE PIECES OF INFORMATION THAT 601 00:21:06,598 --> 00:21:11,536 ARE RELATED, IT ALSO CREATES, AS 602 00:21:11,603 --> 00:21:12,537 YOU SUGGESTED INTEROPERABILITY 603 00:21:12,604 --> 00:21:15,540 AMONG SYSTEMS. 604 00:21:15,607 --> 00:21:16,541 STANDARDS ALSO HAVE AN 605 00:21:16,608 --> 00:21:17,542 INTERESTING POTENTIALLY 606 00:21:17,609 --> 00:21:19,544 HAZARDOUS COMPONENT TO IT, LET 607 00:21:19,611 --> 00:21:25,517 ME TRY TO EXPLAIN WHAT THAT IS, 608 00:21:25,584 --> 00:21:28,520 IN OUR HEALTHCARE SYSTEM, WE 609 00:21:28,587 --> 00:21:29,521 HAVE HEALTHCARE HMOs, HEALTH 610 00:21:29,588 --> 00:21:30,522 MANAGEMENT ORGANIZATIONS, WE 611 00:21:30,589 --> 00:21:31,523 HAVE INSURANCE COMPANIES, THE 612 00:21:31,590 --> 00:21:33,525 WAY THE HEALTHCARE SYSTEM IN THE 613 00:21:33,592 --> 00:21:35,527 U.S. WORK SYSTEM THAT WHEN THE 614 00:21:35,594 --> 00:21:37,529 PATIENT COMES TO SEE THE DOCTOR, IF 615 00:21:37,596 --> 00:21:39,531 THE DOCTOR WANTS TO BE COMPENSATED 616 00:21:39,598 --> 00:21:43,535 BY ONE OF THE HEALTHCARE SYSTEMS, 617 00:21:43,602 --> 00:21:46,538 THEY HAVE TO ASSIGN A PARTICULAR 618 00:21:46,605 --> 00:21:48,540 CODE TO EITHER THE VISIT, THE 619 00:21:48,607 --> 00:21:51,543 DIAGNOSIS OR THE TREATMENT. 620 00:21:51,610 --> 00:21:52,544 NOW THAT STANDARDIZATION IS 621 00:21:52,611 --> 00:21:56,515 THERE TO HELP I GUESS ASSESS 622 00:21:56,581 --> 00:22:00,519 WHAT THE VALUE OF THE TREATMENT 623 00:22:00,585 --> 00:22:01,520 WAS OR WHAT THE ACCURACY OF THE 624 00:22:01,586 --> 00:22:04,523 DIAGNOSIS WAS AND WHETHER OR NOT 625 00:22:04,589 --> 00:22:07,526 IT ALL MEETS STANDARDS OF CARE 626 00:22:07,592 --> 00:22:10,529 BUT IT ALSO HAS THIS VERY 627 00:22:10,595 --> 00:22:12,531 PECULIAR PROPERTY THAT IT PIGEON 628 00:22:12,597 --> 00:22:15,534 HOLES AND SHOE HORNS THINGS INTO 629 00:22:15,600 --> 00:22:20,539 PARTICULAR DIAGNOSES AND THAT 630 00:22:20,605 --> 00:22:21,540 SEEMS LIKE IT'S THE ANTITHESIS 631 00:22:21,606 --> 00:22:22,541 OF MEDINE THAT TAKES INTO ACCOUNT THE PERSON AND 632 00:22:22,607 --> 00:22:24,543 THE PERSON'S SPECIFICS. 633 00:22:24,609 --> 00:22:27,512 NOW I'M NOT A DOCTOR AND I COULD 634 00:22:27,579 --> 00:22:30,515 HAVE JUST SAID SOMETHING NOT VERY 635 00:22:30,582 --> 00:22:31,516 PRECISE, BUT IT FEELS TO ME LIKE 636 00:22:31,583 --> 00:22:32,517 THAT STANDARDIZATION WHICH IS 637 00:22:32,584 --> 00:22:33,518 THERE FOR REASONS THAT WE CAN 638 00:22:33,585 --> 00:22:35,520 UNDERSTAND THE FINANCIAL 639 00:22:35,587 --> 00:22:40,525 BUSINESS AND EVERYTHING ELSE, 640 00:22:40,592 --> 00:22:42,527 INTERFERES WITH OUR ABILITY TO 641 00:22:42,594 --> 00:22:44,529 DELIVER HEALTHCARE--I'M GOING TO 642 00:22:44,596 --> 00:22:45,530 USE THE WORDS PERSONALIZED WAY. 643 00:22:45,597 --> 00:22:47,532 SO WE HAVE TO FIGURE THIS OUT 644 00:22:47,599 --> 00:22:50,535 BETTER THAN WE HAVE SO FAR 645 00:22:50,602 --> 00:22:51,536 BECAUSE WE WANT PEOPLE AND WE 646 00:22:51,603 --> 00:22:53,538 WANT DOCTORS SPECIALLY TO 647 00:22:53,605 --> 00:22:56,541 BE ABLE TO TREAT PATIENTS IN 648 00:22:56,608 --> 00:22:57,542 ACCORDANCE TO THE EVIDENCE THAT 649 00:22:57,609 --> 00:22:59,544 THE PATIENT'S CONDITION 650 00:22:59,611 --> 00:23:01,513 EXHIBITS. 651 00:23:01,580 --> 00:23:04,516 AND WE'RE NOT THERE, OUR 652 00:23:04,583 --> 00:23:05,517 STANDARD OF CARE IS TOO 653 00:23:05,584 --> 00:23:07,519 STANDARDIZED. 654 00:23:07,586 --> 00:23:09,521 >> BOY, YOU HAVE HIT PERFECTLY ONE 655 00:23:09,588 --> 00:23:11,523 OF THE MOST DIFFICULT PROBLEMS 656 00:23:11,590 --> 00:23:12,524 IN ALL OF ELECTRONIC HEALTH RECORD 657 00:23:12,591 --> 00:23:16,528 DATA COLLECTION AND THAT IS THE 658 00:23:16,595 --> 00:23:18,530 THING WE CALL THE PROBLEM LIST 659 00:23:18,597 --> 00:23:19,531 WHICH EVERY CLINICIAN OUT THERE 660 00:23:19,598 --> 00:23:23,535 WILL TELL YOU THE PROBLEM LIST 661 00:23:23,602 --> 00:23:24,536 IS A MESS. 662 00:23:24,603 --> 00:23:28,540 YOU KNOW--SOMEONE--SOME 663 00:23:28,607 --> 00:23:30,542 DIAGNOSIS WILL GET PUT ON THAT 664 00:23:30,609 --> 00:23:32,544 LIST AND IT NEVER LEAVES, NEVER 665 00:23:32,611 --> 00:23:34,513 LEAVES THE LIST AND IT MIGHT NOT 666 00:23:34,579 --> 00:23:36,515 EVEN BE TRUE AND IN FACT OFTEN 667 00:23:36,581 --> 00:23:38,517 IS NOT SO IT'S ONE OF THE BIGGEST 668 00:23:38,583 --> 00:23:39,518 ISSUES THAT YOU'VE HIT RIGHT 669 00:23:39,584 --> 00:23:40,519 AWAY. 670 00:23:40,585 --> 00:23:43,522 >> WELL AND YOU WERE TELLING ME 671 00:23:43,588 --> 00:23:44,523 THAT I HIT ON SOMETHING THAT I 672 00:23:44,589 --> 00:23:46,525 DON'T UNDERSTAND BECAUSE WHEN 673 00:23:46,591 --> 00:23:48,527 YOU SAID IT'S ON THE PROBLEM 674 00:23:48,593 --> 00:23:49,528 LIST, HELP ME UNDERSTAND, WHAT 675 00:23:49,594 --> 00:23:52,531 IS IT THAT GETS SOMETHING ON TO 676 00:23:52,597 --> 00:23:54,533 A LIST THAT YOU CALL A PROBLEM 677 00:23:54,599 --> 00:23:55,534 LIST. 678 00:23:55,600 --> 00:23:56,535 WHAT'S THE PROBLEM? 679 00:23:56,601 --> 00:23:59,538 >> SOME CLINICIAN HAS THE PATIENTS 680 00:23:59,604 --> 00:24:03,542 COME IN AND THEY'VE SAID YOU 681 00:24:03,608 --> 00:24:06,545 HAVE HYPERTENSION, IT GOES ON 682 00:24:06,611 --> 00:24:07,512 THE PROBLEM LIST, OH MEAN-- 683 00:24:07,579 --> 00:24:08,513 >> OH THIS IS THE LIST OF 684 00:24:08,580 --> 00:24:10,515 PROBLEMS THAT THE PATIENT HAS 685 00:24:10,582 --> 00:24:12,517 OH, OKAY, THANK YOU. 686 00:24:12,584 --> 00:24:13,518 THANK YOU. 687 00:24:13,585 --> 00:24:15,520 NOW I UNDERSTAND. 688 00:24:15,587 --> 00:24:16,521 AND NOW IT'S STITCHED ON YOUR 689 00:24:16,588 --> 00:24:18,523 FOREHEAD NOW. 690 00:24:18,590 --> 00:24:20,525 >> AND THEN FOREVER THAT PERSON 691 00:24:20,592 --> 00:24:22,527 IS LABELED AND IT MIGHT BE THAT 692 00:24:22,594 --> 00:24:24,529 THEY CAME IN AND THEY GOT REALLY 693 00:24:24,596 --> 00:24:25,530 NERVOUS AND EXCITED BECAUSE THIS 694 00:24:25,597 --> 00:24:27,532 DOCTOR TERRIFIED THEM SO THEIR 695 00:24:27,599 --> 00:24:29,534 BLOOD PRESSURE WAS HIGH. 696 00:24:29,601 --> 00:24:31,536 >> AND THEY HAD SOME SYMPTOM AND 697 00:24:31,603 --> 00:24:33,538 THEIR BLOOD PRESSURE WAS UP AND 698 00:24:33,605 --> 00:24:36,541 NOW HYPERTENSION IS ON THEIR 699 00:24:36,608 --> 00:24:38,543 DIAGNOSIS LIST FOREVER. 700 00:24:38,610 --> 00:24:39,544 WOW, SO YOU TRIGGERED ANOTHER 701 00:24:39,611 --> 00:24:42,514 REACTION FOR ME ANYWAY, I THINK 702 00:24:42,581 --> 00:24:45,517 SOMEDAY I WANT TO WRITE AN ESSAY 703 00:24:45,584 --> 00:24:47,519 CALLED YOU ARE NOT YOUR DISEASE. 704 00:24:47,586 --> 00:24:49,521 YOU KNOW AND I DON'T MEAN TO 705 00:24:49,588 --> 00:24:51,523 OFFEND ANY DOCTORS IN HOUSE HERE 706 00:24:51,590 --> 00:24:52,524 BUT I THINK SOME DOCTORS HAVE A 707 00:24:52,591 --> 00:24:54,526 TENDENCY TO THINK OF THEIR 708 00:24:54,593 --> 00:24:55,527 PATIENTS AS THE DISEASE OPPOSED 709 00:24:55,594 --> 00:25:02,534 TO THE PATIENT AND THAT'S PART 710 00:25:02,601 --> 00:25:03,535 OF THE PROBLEM LIST. 711 00:25:03,602 --> 00:25:04,536 >> PERFECT. 712 00:25:04,603 --> 00:25:06,538 OKAY, WE HAVE ANOTHER QUESTION 713 00:25:06,605 --> 00:25:07,539 FOR SOMEONE. 714 00:25:07,606 --> 00:25:09,541 AT NIH WE DEFINITELY BELIEVE IN 715 00:25:09,608 --> 00:25:10,542 THE POWER OF UNLEASHING 716 00:25:10,609 --> 00:25:12,544 INFORMATION, FOR A LITTLE OVER A 717 00:25:12,611 --> 00:25:13,511 YEAR NOW, WE HAVE HAD A NEW DATA 718 00:25:13,578 --> 00:25:15,513 MANAGEMENT AND SHARING POLICY 719 00:25:15,580 --> 00:25:17,515 THAT REQUIRES EVERY 720 00:25:17,582 --> 00:25:20,518 INVESTIGATOR AND INSTITUTION WHO 721 00:25:20,585 --> 00:25:21,519 RECEIVES FUNDING FROM THE NIH TO 722 00:25:21,586 --> 00:25:23,521 PLAN FOR HOW THEY WILL SHARE 723 00:25:23,588 --> 00:25:25,523 THEIR DATA AND HOW THEY WILL NOT 724 00:25:25,590 --> 00:25:28,526 ONLY SHARE THEIR DATA BUT MANAGE 725 00:25:28,593 --> 00:25:29,527 THEIR DATA SO THAT IT CAN BE 726 00:25:29,594 --> 00:25:30,528 SHARED. 727 00:25:30,595 --> 00:25:32,530 SO WHAT ARE SOME OF THE ROAD 728 00:25:32,597 --> 00:25:33,531 BLOCKS THAT YOU FORSEE IN 729 00:25:33,598 --> 00:25:35,533 GETTING USEABLE DATA FROM 730 00:25:35,600 --> 00:25:37,535 RESEARCHERS, AND HOW DO YOU 731 00:25:37,602 --> 00:25:38,536 RECOMMEND WE OVERCOME THOSE 732 00:25:38,603 --> 00:25:40,538 DIFFICULTIES? 733 00:25:40,605 --> 00:25:41,539 >> MAN, HOW MANY HOURS DO WE 734 00:25:41,606 --> 00:25:44,542 HAVE? 735 00:25:44,609 --> 00:25:46,544 THIS IS A HARD PROBLEM. 736 00:25:46,611 --> 00:25:50,515 FIRST OF ALL, IS THE DATA THAT 737 00:25:50,582 --> 00:25:52,517 THEY HAVE COMMENSURATE WITH ANY 738 00:25:52,584 --> 00:25:55,520 OTHER DATA, HOW COULD WE LABEL 739 00:25:55,587 --> 00:25:56,521 IT IN SUCH A WAY--EVEN THE UNITS 740 00:25:56,588 --> 00:25:57,522 OF DATA, IT'S A NUMBER, OKAY, 741 00:25:57,589 --> 00:26:00,525 25. 742 00:26:00,592 --> 00:26:03,528 WELL IS THAT 25 POUNDS OR 743 00:26:03,595 --> 00:26:05,530 25 OUNCES OR YOU KNOW SOMETHING 744 00:26:05,597 --> 00:26:06,531 ELSE? 745 00:26:06,598 --> 00:26:08,533 25 KILOGRAMS, SO FIRST 746 00:26:08,600 --> 00:26:09,534 OF ALL WE HAVE TO FIGURE OUT HOW 747 00:26:09,601 --> 00:26:11,536 WE HAVE COMMON LABELS FOR THE 748 00:26:11,603 --> 00:26:14,539 DATA OR WE HAVE TO TRANSLATE IT 749 00:26:14,606 --> 00:26:16,541 INTO SOMETHING THAT EVERYBODY WILL 750 00:26:16,608 --> 00:26:17,542 UNDERSTAND. 751 00:26:17,609 --> 00:26:18,543 THAT'S PROBLEM NUMBER ONE. 752 00:26:18,610 --> 00:26:21,513 SECOND, THERE COULD BE 753 00:26:21,579 --> 00:26:23,515 A LOT OF IT. 754 00:26:23,581 --> 00:26:26,518 SO WHEN YOU TELL SOMEBODY, YOU 755 00:26:26,584 --> 00:26:28,520 HAVE TO FIGURE OUT HOW TO SHARE 756 00:26:28,586 --> 00:26:30,522 YOUR DATA, WE BETTER HAVE 757 00:26:30,588 --> 00:26:32,524 IN MIND SOMETHING TO TELL THEM 758 00:26:32,590 --> 00:26:34,526 TO HELP THEM DO THAT. 759 00:26:34,592 --> 00:26:36,528 IF SOMEBODY SAYS I'M HAPPY TO DO BUT I 760 00:26:36,594 --> 00:26:38,530 DON'T KNOW WHERE TO DO IT OR PUT 761 00:26:38,596 --> 00:26:42,534 IT, THERE'S MORE THAN I CAN 762 00:26:42,600 --> 00:26:45,537 AFFORD TO STORE AWAY, SO YOU 763 00:26:45,603 --> 00:26:46,538 HAVE TO OVERCOME SCALE. 764 00:26:46,604 --> 00:26:49,541 YOU HAVE TO EVERYCOME SEMANTICS 765 00:26:49,607 --> 00:26:50,542 OF THE DATA IN THE STANDARDIZED 766 00:26:50,608 --> 00:26:52,544 REPRESENTATIONS AND OH, BY THE 767 00:26:52,610 --> 00:26:55,513 WAY, WE DIDN'T EVEN GET TO THE 768 00:26:55,580 --> 00:26:56,514 OTHER HARD PART WHICH IS 769 00:26:56,581 --> 00:26:58,516 PRESERVING IT OVER PERIODS OF 770 00:26:58,583 --> 00:26:59,517 TIME. 771 00:26:59,584 --> 00:27:00,518 FORTUNATELY OUR LIFETIMES ARE 772 00:27:00,585 --> 00:27:03,521 GETTING LONGER BUT WHAT THAT 773 00:27:03,588 --> 00:27:06,524 MEANS IS THAT THE DATA ABOUT US 774 00:27:06,591 --> 00:27:07,525 OUR HEALTH DATA SHOULD BE 775 00:27:07,592 --> 00:27:10,528 PRESERVED OVER ENTIRE LIFETIME 776 00:27:10,595 --> 00:27:13,531 BECAUSE IT COULD TELL US A GREAT 777 00:27:13,598 --> 00:27:15,533 DEAL TOWARDS OF END OF OUR LIVES 778 00:27:15,600 --> 00:27:16,534 ABOUT WHAT PROBLEMS MAY HAVE 779 00:27:16,601 --> 00:27:19,537 ARISEN THAT THEY MAY HAVE 780 00:27:19,604 --> 00:27:26,544 ORIGINS MUCH EARLIER, MIGHT EVEN 781 00:27:26,611 --> 00:27:27,512 HAVE BEEN ANALYTICALLY 782 00:27:27,579 --> 00:27:32,517 DETECTABLE. 783 00:27:32,584 --> 00:27:33,518 SO HOLDING ONTO THAT DATA OVER A LONG PERIOD OF TIME 784 00:27:33,585 --> 00:27:34,519 FOR DIGITAL PRESERVATION IS A BIG 785 00:27:34,586 --> 00:27:36,521 CHALLENGE. 786 00:27:36,588 --> 00:27:42,527 IT USED TO BE THAT WE WROTE OUR 787 00:27:42,594 --> 00:27:43,528 STUFF ON LITTLE CLAY TABLETS IN 788 00:27:43,595 --> 00:27:44,529 CUNEIFORM THAT WAS 6000 YEARS 789 00:27:44,596 --> 00:27:46,531 AGO AND THE FUNNY THING IS THOSE 790 00:27:46,598 --> 00:27:49,534 TABLETS ARE STILL AROUND IF YOU 791 00:27:49,601 --> 00:27:53,538 CAN READ CUNEIFORM THAT'S NOT 792 00:27:53,605 --> 00:27:55,540 TOO COMMON, BUT THEY WERE 793 00:27:55,607 --> 00:27:57,542 WAREHOUSE RECORD ANDS THE 794 00:27:57,609 --> 00:27:59,544 WAREHOUSES BURNED DOWN AND BAKED 795 00:27:59,611 --> 00:28:01,513 THE CLAY, SO THE CLAY LASTED 796 00:28:01,579 --> 00:28:02,514 6000 YEARS AND THEN THERE WAS 797 00:28:02,580 --> 00:28:06,518 VELUM AND CALF SKIN AND THE LIKE 798 00:28:06,584 --> 00:28:10,522 AND THAT LASTS A COUPLE THOUSAND 799 00:28:10,588 --> 00:28:11,523 YEARS, AMAZINGLY BUT IT'S ONLY A 800 00:28:11,589 --> 00:28:16,528 COUPLE THOUSAND COMPARED TO SIX 801 00:28:16,594 --> 00:28:20,532 AND THEN COMES RAG PAPER, WHICH 802 00:28:20,598 --> 00:28:23,535 LASTS 500, 600 YEARS AND WE'RE 803 00:28:23,601 --> 00:28:24,536 GETTING WORSE AND WORSE AT 804 00:28:24,602 --> 00:28:27,539 STORING OUR INFORMATION. 805 00:28:27,605 --> 00:28:29,541 SO HOW MEAN PEOPLE HAVE 3 AND 806 00:28:29,607 --> 00:28:30,542 HALF INCH FLOPPY DISKS OR 5 AND 807 00:28:30,608 --> 00:28:32,544 A QUARTER INCH FLOPPY DISKS AND 808 00:28:32,610 --> 00:28:33,511 EVEN IF THE BITS ARE STILL 809 00:28:33,578 --> 00:28:34,512 THERE, DO YOU HAVE A READER THAT 810 00:28:34,579 --> 00:28:35,513 CAN READ IT. 811 00:28:35,580 --> 00:28:41,519 SO ONE OF THE BIG PROBLEMS 812 00:28:41,586 --> 00:28:42,520 ASSOCIATE WIDE YOUR QUESTION IS 813 00:28:42,587 --> 00:28:44,522 NOT ONLY THE FORMAT AND 814 00:28:44,589 --> 00:28:46,524 COMMONALITY AND STANDARDIZATION 815 00:28:46,591 --> 00:28:47,525 BUT IT'S ALSO HOW THE HELL DO I 816 00:28:47,592 --> 00:28:48,526 PRESERVE ALL THAT DIGITAL 817 00:28:48,593 --> 00:28:50,528 INFORMATION LONG ENOUGH AND AT 818 00:28:50,595 --> 00:28:55,533 SCALE. SO THAT'S WHERE 819 00:28:55,600 --> 00:28:56,534 COMPANIES LIKE GOOGLE ARE HARD 820 00:28:56,601 --> 00:29:02,540 PRESSED TO INCREASE THE TOTAL 821 00:29:02,607 --> 00:29:03,541 AMOUNT OF ABILITY TO STORE 822 00:29:03,608 --> 00:29:04,542 INFORMATION. 823 00:29:04,609 --> 00:29:05,543 >> IT'S A TOUGH PROBLEM. 824 00:29:05,610 --> 00:29:06,544 >> BUT I LOVE THAT PROBLEM. 825 00:29:06,611 --> 00:29:08,513 SCALE IS FUN TO TACKLE. 826 00:29:08,580 --> 00:29:11,516 YOU KNOW, MAKE IT BIGGER. 827 00:29:11,583 --> 00:29:12,517 WELL, I STILL HAVE MY OLD 828 00:29:12,584 --> 00:29:14,519 LABORATORY NOTEBOOKS WITH ALL 829 00:29:14,586 --> 00:29:16,521 THE WRITING IN THEM BUT I ALSO 830 00:29:16,588 --> 00:29:18,523 HAVE SOME 3-INCH FLOPPY DISKS 831 00:29:18,590 --> 00:29:19,524 AND MY LABORATORY NOTEBOOKS I 832 00:29:19,591 --> 00:29:22,527 CAN STILL READ. 833 00:29:22,594 --> 00:29:25,530 I APPRECIATE THAT PROBLEM. 834 00:29:25,597 --> 00:29:26,531 >> EVERYTHING THAT'S ON THOSE 835 00:29:26,598 --> 00:29:29,534 DISKS ... THE PROBLEM, THE DIFFERENCE THOUGH IS THAT 836 00:29:29,601 --> 00:29:30,535 YOU CAN READ THE LAB NOTEBOOKS 837 00:29:30,602 --> 00:29:31,536 BUT THE COMPUTER MIGHT NOT BE 838 00:29:31,603 --> 00:29:32,537 ABLE TO READ THEM AND IF YOU 839 00:29:32,604 --> 00:29:34,539 WANTED TO EXERCISE THE KIND OF 840 00:29:34,606 --> 00:29:37,542 TOOLS THAT WE'VE BEEN TALKING 841 00:29:37,609 --> 00:29:40,512 ABOUT, WE NEED TO TURN THAT INTO 842 00:29:40,578 --> 00:29:42,514 DIGITAL INFORMATION. 843 00:29:42,580 --> 00:29:45,517 >> OKAY, NIH IS INVESTED IN 844 00:29:45,583 --> 00:29:46,518 ADVANCING POINT OF CARE 845 00:29:46,584 --> 00:29:47,519 TECHNOLOGIES WHICH HELP MEET 846 00:29:47,585 --> 00:29:50,522 PEOPLE WHERE THEY ARE, WHERE 847 00:29:50,588 --> 00:29:52,524 THEY LIVE, WHERE THEY WORK, 848 00:29:52,590 --> 00:29:54,526 INCLUDING IN THEIR OWN HOME ANDS 849 00:29:54,592 --> 00:29:57,529 WE ALSO HAVE A CRITICAL NEED, WE 850 00:29:57,595 --> 00:29:59,531 HAVE TRUST ISSUES WITH OUR 851 00:29:59,597 --> 00:30:00,532 PATIENTS, THE PEOPLE WERE TRYING 852 00:30:00,598 --> 00:30:03,535 TO SERVE SO WE ALSO WANT TO 853 00:30:03,601 --> 00:30:05,537 INVOLVE PEOPLE MORE IN THEIR OWN 854 00:30:05,603 --> 00:30:06,538 DECISION MAKING AND IN SELF-CARE 855 00:30:06,604 --> 00:30:07,539 AS WELL AS IN THE RESEARCH 856 00:30:07,605 --> 00:30:09,541 PROCESS. 857 00:30:09,607 --> 00:30:12,544 SO THIS CAN INCLUDE DEVICES THAT 858 00:30:12,610 --> 00:30:13,545 CONNECT TO THE INTERNET AND 859 00:30:13,611 --> 00:30:15,513 HEALTH SYSTEMS, THE INTERNET OF 860 00:30:15,580 --> 00:30:19,517 THINGS, HOW DO YOU THINK THE 861 00:30:19,584 --> 00:30:20,518 INTERNET OF THINGS DEVICES CAN 862 00:30:20,585 --> 00:30:22,520 IMPROVE HEALTHCARE. 863 00:30:22,587 --> 00:30:23,521 >> WELL, FIRST OF ALL THERE'S NO 864 00:30:23,588 --> 00:30:25,523 DOUBT IN MY MIND THAT THEY CAN. 865 00:30:25,590 --> 00:30:26,524 PART OF THE REASON FOR THIS IS AGAIN 866 00:30:26,591 --> 00:30:28,526 GOES BACK TO MY EARLIER 867 00:30:28,593 --> 00:30:31,529 OBSERVATION THAT BEING ABLE TO 868 00:30:31,596 --> 00:30:34,532 OBSERVE A PATIENT 24 HOURS A 869 00:30:34,599 --> 00:30:35,533 DAY FOR DAYS ON END OR WEEKS OR 870 00:30:35,600 --> 00:30:36,534 MONTHS MEANS THAT WE CAN 871 00:30:36,601 --> 00:30:38,536 CALIBRATE THAT PATIENT'S 872 00:30:38,603 --> 00:30:39,537 CONDITION RELATIVE TO THAT WHICH 873 00:30:39,604 --> 00:30:40,538 IS BEING MEASURED. 874 00:30:40,605 --> 00:30:42,540 THAT'S REALLY IMPORTANT 875 00:30:42,607 --> 00:30:44,542 WE CAN DO ALERTS AND THINGS LIKE 876 00:30:44,609 --> 00:30:48,513 THAT IF WE SEE SOME VARIATION 877 00:30:48,580 --> 00:30:51,516 THAT IS EXTREME FOR EXAMPLE, SO 878 00:30:51,583 --> 00:30:52,517 BUILDING A MODEL OF AN 879 00:30:52,584 --> 00:30:54,519 INDIVIDUAL PATIENT BASED ON THAT 880 00:30:54,586 --> 00:30:55,520 KIND OF DATA IS REALLY QUITE 881 00:30:55,587 --> 00:30:57,522 ATTRACTIVE. 882 00:30:57,589 --> 00:30:58,523 SOME OF YOU PROBABLY WORN 883 00:30:58,590 --> 00:31:01,526 RECORDING DEVICES FOR EXAMPLE, 884 00:31:01,593 --> 00:31:04,529 THAT STRAP TO YOUR WAIST FOR A 885 00:31:04,596 --> 00:31:06,531 PERIOD OF DAYS OR EVEN WEEKS, 886 00:31:06,598 --> 00:31:07,532 THAT INFORMATION COULD BEING 887 00:31:07,599 --> 00:31:10,535 COLLECTED MORE READILY BY WHAT 888 00:31:10,602 --> 00:31:11,536 WE CALL WEARABLES AT GOOGLE, 889 00:31:11,603 --> 00:31:12,537 YOUR WRIST WATCH CAN PICK UP A 890 00:31:12,604 --> 00:31:13,538 LOT. 891 00:31:13,605 --> 00:31:15,540 IT'S AMAZING HOW MUCH YOU CAN 892 00:31:15,607 --> 00:31:17,542 TELL BY STICKING SOMETHING IN 893 00:31:17,609 --> 00:31:18,543 YOUR EAR, IT SOUNDS KINDS OF 894 00:31:18,610 --> 00:31:20,511 SILLY BUT IT'S TRUE. 895 00:31:20,578 --> 00:31:22,513 THERE'S A LOT OF THE BODY'S 896 00:31:22,580 --> 00:31:26,517 CONDITION IS LITERALLY 897 00:31:26,584 --> 00:31:28,519 DETECTABLE INSIDE THE EAR CANAL 898 00:31:28,586 --> 00:31:32,523 SO I CAN EASILY IMAGINE WEARING 899 00:31:32,590 --> 00:31:34,525 DEVICES THAT HELP US, KEEP TRACK 900 00:31:34,592 --> 00:31:35,526 AND WARN US IF SOMETHING'S 901 00:31:35,593 --> 00:31:37,528 HAPPENING. 902 00:31:37,595 --> 00:31:41,532 THOSE OF YOU WHO ARE OR KNOW 903 00:31:41,599 --> 00:31:43,534 ABOUT DIABETICS THAT HAVE 904 00:31:43,601 --> 00:31:44,535 GLUCOSE DETECTORS AND AUTOMATIC DETECTION AND IN SOME 905 00:31:44,602 --> 00:31:46,537 CASES AUTOMATIC INJECTIONS OF 906 00:31:46,604 --> 00:31:49,540 INSULIN IN ORDER TO RESPOND TO 907 00:31:49,607 --> 00:31:50,541 THE CURRENT STATE OF GLUCOSE, 908 00:31:50,608 --> 00:31:51,542 WILL KNOW THAT THOSE DEVICES ARE 909 00:31:51,609 --> 00:31:54,512 REALLY QUITE EFFECTIVE. 910 00:31:54,579 --> 00:31:55,513 NOW WE WORRY A LOT, ANYWAY, 911 00:31:55,580 --> 00:31:58,516 ABOUT THE CORRECT OPERATION OF 912 00:31:58,583 --> 00:32:01,519 THE DEVICE, SO WHATEVER SOFTWARE 913 00:32:01,586 --> 00:32:02,520 IS INVOLVED IN THOSE DEVICES 914 00:32:02,587 --> 00:32:05,523 ABSOLUTELY HAS TO BE RIGIDLY 915 00:32:05,590 --> 00:32:07,525 TESTED AND CHECKED AND VALIDATED 916 00:32:07,592 --> 00:32:09,527 AND EVERYTHING ELSE BUT I SEE, 917 00:32:09,594 --> 00:32:12,530 THE MONITORING OF SENSORY 918 00:32:12,597 --> 00:32:14,532 INFORMATION IS A VERY IMPORTANT 919 00:32:14,599 --> 00:32:17,535 FUTURE FOR OUR ABILITY TO 920 00:32:17,602 --> 00:32:22,540 PROVIDE BETTER HEALTHCARE. 921 00:32:22,607 --> 00:32:23,541 CAN I JUST MENTION THAT I WAS IN 922 00:32:23,608 --> 00:32:24,542 SAUDI ARABIA RECENTLY AND 923 00:32:24,609 --> 00:32:26,544 VISITED A VIRTUAL HOSPITAL WHERE 924 00:32:26,611 --> 00:32:28,513 THE PATIENTS ARE REMOTE BUT THEY 925 00:32:28,579 --> 00:32:32,517 HAVE LOCALLY IN THEIR HOMES AND 926 00:32:32,583 --> 00:32:33,518 ON THEIR PERSON, SENSORY DEVICES 927 00:32:33,584 --> 00:32:35,520 THAT ALLOW A DOCTOR AT THE 928 00:32:35,586 --> 00:32:38,523 CENTRAL VIRTUAL HOSPITAL TO 929 00:32:38,589 --> 00:32:39,524 GATHER THEIR DATA, HAVE 930 00:32:39,590 --> 00:32:41,526 DISCUSSIONS AND VIDEO CALLS AND 931 00:32:41,592 --> 00:32:44,529 SO ON WITH THE PATIENTS SO THAT 932 00:32:44,595 --> 00:32:45,530 THEY CAN GET HEALTHCARE THAT 933 00:32:45,596 --> 00:32:47,532 THEY WOULDN'T OTHERWISE GET 934 00:32:47,598 --> 00:32:48,533 BECAUSE THERE ARE NO CLINICS 935 00:32:48,599 --> 00:32:49,534 LOCALLY AVAILABLE. 936 00:32:49,600 --> 00:32:54,539 AND THERE ARE THINGS THAT ARE 937 00:32:54,605 --> 00:32:55,540 STARTING TO HAPPEN, EXAMPLE 938 00:32:55,606 --> 00:32:57,542 SONOGRAPHY, SONOGRAMS ARE 939 00:32:57,608 --> 00:32:58,543 BECOMING MUCH MORE RELIABLE AS 940 00:32:58,609 --> 00:32:59,544 DIAGNOSTIC TOOLS. 941 00:32:59,610 --> 00:33:03,514 I SOUND LIKE I KNOW WHAT I'M 942 00:33:03,581 --> 00:33:05,516 TALKING ABOUT WHICH IS REALLY 943 00:33:05,583 --> 00:33:07,518 AMAZING, SOUNDS LIKE I'M A LARGE 944 00:33:07,585 --> 00:33:09,520 LANGUAGE MODELING PERSON HERE 945 00:33:09,587 --> 00:33:10,521 SO SONOGRAM RESOLUTION HAS 946 00:33:10,588 --> 00:33:13,524 GOTTEN MUCH, MUCH, BETTER AND 947 00:33:13,591 --> 00:33:16,527 THOSE ARE MUCH LESS EXPENSIVE 948 00:33:16,594 --> 00:33:18,529 THAN AN FMRI, OR MRI-TYPE 949 00:33:18,596 --> 00:33:20,531 MACHINE OR CT SCANNERS. 950 00:33:20,598 --> 00:33:22,533 SO I SEE THESE KINDS OF SMALL 951 00:33:22,600 --> 00:33:24,535 PORTABLE DEVICES AS BEING A 952 00:33:24,602 --> 00:33:26,537 SIGNIFICANT IMPROVEMENT IN OUR 953 00:33:26,604 --> 00:33:31,542 ABILITY TO DIAGNOSE ILLNESS AND 954 00:33:31,609 --> 00:33:35,513 TO TRACK HEALTH STATES. 955 00:33:35,580 --> 00:33:36,514 >> AND SOON. 956 00:33:36,581 --> 00:33:37,515 >> YES. 957 00:33:37,582 --> 00:33:38,516 >> THIS ISN'T THE FUTURE THIS, 958 00:33:38,583 --> 00:33:40,518 IS HERE NOW. THIS IS NOW. 959 00:33:40,585 --> 00:33:42,520 >> ALRIGHT. 960 00:33:42,587 --> 00:33:43,521 THESE ARE GETTING HARDER, 961 00:33:43,588 --> 00:33:45,523 MISINFORMATION, DISINFORMATION 962 00:33:45,590 --> 00:33:47,525 AND DECLINING TRUST REMAIN HUGE 963 00:33:47,592 --> 00:33:50,528 CHALLENGES FOR THE SCIENTIFIC 964 00:33:50,595 --> 00:33:51,529 COMMUNITY ESPECIALLY FOR 965 00:33:51,596 --> 00:33:52,530 GOVERNMENT SCIENTISTS. 966 00:33:52,597 --> 00:33:54,532 DO YOU THINK THIS IS GETTING 967 00:33:54,599 --> 00:33:56,534 BETTER OR WORSE, DO YOU THINK 968 00:33:56,601 --> 00:33:57,535 THIS WILL GET BETTER OR WORSE IN 969 00:33:57,602 --> 00:34:00,538 IN THE FUTURE AND HOW DO YOU 970 00:34:00,605 --> 00:34:04,542 THINK WE CAN COMBAT THE PROBLEM 971 00:34:04,609 --> 00:34:05,543 AND DEVISE SOLUTIONS TO 972 00:34:05,610 --> 00:34:06,544 MISINFORMATION. 973 00:34:06,611 --> 00:34:07,512 I TOLD YOU THEY'RE GETTING 974 00:34:07,578 --> 00:34:08,513 HARDER. 975 00:34:08,579 --> 00:34:10,515 >> YOU DON'T HAPPEN TO HAVE 976 00:34:10,581 --> 00:34:11,516 ANOTHER QUESTION, DO YOU? 977 00:34:11,582 --> 00:34:17,522 >> I DO. 978 00:34:17,588 --> 00:34:18,523 WE CAN SKIP THAT. 979 00:34:18,589 --> 00:34:19,524 >> NO, NO, I WILL NOT DUCK THAT 980 00:34:19,590 --> 00:34:20,525 BECAUSE THAT IS A VERY 981 00:34:20,591 --> 00:34:21,526 IMPORTANT PROBLEM. 982 00:34:21,592 --> 00:34:23,528 SO THERE ARE SEVERAL THINGS THAT WE'RE GOING TO HAVE TO DO, FIRST 983 00:34:23,594 --> 00:34:24,529 OF ALL WITH REGARD TO 984 00:34:24,595 --> 00:34:25,530 INFORMATION, PROVIDENCE OF THE 985 00:34:25,596 --> 00:34:27,532 INFORMATION IS INCREASINGLY 986 00:34:27,598 --> 00:34:28,533 IMPORTANT, WE HAVE TO KNOW WHERE 987 00:34:28,599 --> 00:34:30,535 STUFF COMES FROM, WE HAVEN'T 988 00:34:30,601 --> 00:34:32,537 DONE AS GOOD A JOB AS I THINK WE 989 00:34:32,603 --> 00:34:34,539 WERE ARE GOING TO HAVE TO DO IN ORDER TO 990 00:34:34,605 --> 00:34:35,540 IDENTIFY WHERE THINGS COME FROM. 991 00:34:35,606 --> 00:34:37,542 EVEN IF WE KNOW WHERE THEY COME 992 00:34:37,608 --> 00:34:39,544 FROM, THEN WE HAVE TO ASSESS 993 00:34:39,610 --> 00:34:40,545 WHETHER OR NOT THEY COME FROM A 994 00:34:40,611 --> 00:34:41,512 SOURCE THAT I DON'T CARE TO 995 00:34:41,579 --> 00:34:44,515 TRUST OR NOT. 996 00:34:44,582 --> 00:34:46,517 MAYBE IT'S A SOURCE THAT WE HAVE 997 00:34:46,584 --> 00:34:47,518 TRUSTED AND SUDDENLY WE HAVE TO 998 00:34:47,585 --> 00:34:50,521 DECIDE WE DON'T ANYMORE BECAUSE 999 00:34:50,588 --> 00:34:52,523 THE INFORMATION HAS BECOME POLLUTED. 1000 00:34:52,590 --> 00:34:54,525 FOR SOME KINDS OF MISINFORMATION 1001 00:34:54,592 --> 00:34:56,527 AND DISINFORMATION, THERE IS A 1002 00:34:56,594 --> 00:34:59,530 DEFENSE, IT'S CALLED CRITICAL 1003 00:34:59,597 --> 00:35:00,531 THINKING AND IT TAKES A FAIR 1004 00:35:00,598 --> 00:35:02,533 AMOUNT OF WORK TO DO THAT AND A 1005 00:35:02,600 --> 00:35:04,535 LOT OF PEOPLE ARE UNWILLING TO 1006 00:35:04,602 --> 00:35:06,537 DO THE WORK TO THINK CRITICALLY 1007 00:35:06,604 --> 00:35:07,538 ABOUT WHAT THEY SEE AND HEAR, I 1008 00:35:07,605 --> 00:35:10,541 ACTUALLY THINK THAT YOUNG PEOPLE 1009 00:35:10,608 --> 00:35:14,512 SHOULD BE TRAINED TO THINK 1010 00:35:14,579 --> 00:35:16,514 CRITICALLY, THEY SHOULD BE 1011 00:35:16,581 --> 00:35:17,515 ENCOURAGED AND CONGRATULATED FOR 1012 00:35:17,582 --> 00:35:19,517 THINKING CRITICALLY ABOUT WHAT 1013 00:35:19,584 --> 00:35:20,518 THEY SEE AND HEAR. 1014 00:35:20,585 --> 00:35:22,520 NOW THERE ARE SOME PARENTS THAT 1015 00:35:22,587 --> 00:35:24,522 DON'T LIKE TO HEAR THAT BECAUSE 1016 00:35:24,589 --> 00:35:25,523 THE KID MIGHT COME HOME AND 1017 00:35:25,590 --> 00:35:27,525 DISAGREE WITH THEIR PARENTS AND 1018 00:35:27,592 --> 00:35:28,526 SOME PARENTS DON'T TAKE THAT TOO 1019 00:35:28,593 --> 00:35:30,528 LIGHTLY. 1020 00:35:30,595 --> 00:35:32,530 ON THE OTHER HAND, I THINK 1021 00:35:32,597 --> 00:35:33,531 THAT'S HOW YOU HAVE TO COPE WITH 1022 00:35:33,598 --> 00:35:36,534 A WORLD WHERE SOME INFORMATION 1023 00:35:36,601 --> 00:35:38,536 IS NOT TO BE TRUSTED. 1024 00:35:38,603 --> 00:35:41,539 SO I THINK THERE'S STILL A LOT 1025 00:35:41,606 --> 00:35:45,543 OF WORK TO BE DONE TO LABEL 1026 00:35:45,610 --> 00:35:48,513 INFORMATION, TO IDENTIFY TRUSTED 1027 00:35:48,579 --> 00:35:49,514 SOURCES AND FRANKLY USE CRITICAL 1028 00:35:49,580 --> 00:35:50,515 THINKING. 1029 00:35:50,581 --> 00:35:52,517 YOU KNOW EVERY ONCE IN A WHILE, 1030 00:35:52,583 --> 00:35:56,521 I THINK--YOU KNOW HOW WE HAVE TO 1031 00:35:56,587 --> 00:35:57,522 TAKE A TEST BEFORE YOU GET YOUR 1032 00:35:57,588 --> 00:35:58,523 DRIVER'S LICENSE? 1033 00:35:58,589 --> 00:35:59,524 I'VE OFTEN THOUGHT THAT PEOPLE 1034 00:35:59,590 --> 00:36:01,526 SHOULD HAVE INTERNET DRIVER'S 1035 00:36:01,592 --> 00:36:04,529 LICENSE TEST, WHERE YOU HAVE TO 1036 00:36:04,595 --> 00:36:06,531 LEARN ABOUT THE HAZARDS OF BEING 1037 00:36:06,597 --> 00:36:08,533 ON THE INTERNET ROAD AND THERE 1038 00:36:08,599 --> 00:36:11,536 ARE PARTIES OUT THERE ON THE 1039 00:36:11,602 --> 00:36:13,538 REAL ROADS THAT DON'T KNOW HOW 1040 00:36:13,604 --> 00:36:15,540 TO DRIVE VERY WELL AND THERE ARE 1041 00:36:15,606 --> 00:36:16,541 PEOPLE ON THE VIRTUAL ROADS THAT 1042 00:36:16,607 --> 00:36:17,542 DON'T HAVE THE TRUTH AT THEIR 1043 00:36:17,608 --> 00:36:20,545 FINGERTIPS AND YOU NEED TO 1044 00:36:20,611 --> 00:36:22,513 UNDERSTAND THAT AND BE--FIGURE 1045 00:36:22,580 --> 00:36:23,514 OUT HOW TO BE DEFENSIVE ABOUT 1046 00:36:23,581 --> 00:36:24,515 THAT. 1047 00:36:24,582 --> 00:36:26,517 OUR SCIENTISTS ESPECIALLY 1048 00:36:26,584 --> 00:36:28,519 THOUGH, HAVE TO BE SKILLED IN 1049 00:36:28,586 --> 00:36:29,520 THIS ART BECAUSE OTHERWISE THEY 1050 00:36:29,587 --> 00:36:32,523 WILL INGEST BAD INFORMATION 1051 00:36:32,590 --> 00:36:37,528 WHICH WILL LEAD TO BAD DIAGNOSES 1052 00:36:37,595 --> 00:36:38,529 AND BAD TREATMENTS. 1053 00:36:38,596 --> 00:36:41,532 >> SO WHO REGULATES ALL THIS? 1054 00:36:41,599 --> 00:36:42,533 HOW DO WE MAKE THIS HAPPEN? 1055 00:36:42,600 --> 00:36:44,535 AND WELL THE ME BACK UP TO WHERE 1056 00:36:44,602 --> 00:36:45,536 WE WERE BEFORE ABOUT, YOU KNOW 1057 00:36:45,603 --> 00:36:49,540 THE INTERNET, SOMEONE HAD TO 1058 00:36:49,607 --> 00:36:52,543 DECIDE, THIS IS THE STANDARD 1059 00:36:52,610 --> 00:36:53,544 WE'RE GOING TO USE BECAUSE 1060 00:36:53,611 --> 00:36:56,514 THERE'S ALWAYS COMPETITION FOR 1061 00:36:56,581 --> 00:36:58,516 THESE THING ANDS SOMEBODY HAS TO 1062 00:36:58,583 --> 00:37:01,519 SAY, OKAY ENOUGH SILOS, WE'RE 1063 00:37:01,586 --> 00:37:02,520 GOING WITH THIS ONE. 1064 00:37:02,587 --> 00:37:05,523 YOU KNOW, THINGS LIKE THIS, 1065 00:37:05,590 --> 00:37:06,524 OKAY, WE'VE HAD THIS FREE AND 1066 00:37:06,591 --> 00:37:10,528 THIS OPEN AND NOBODY KNOWS WHERE THIS COMES FROM, NOW WE NEED 1067 00:37:10,595 --> 00:37:13,531 TO SEE AN IDENTIFIER, WHO 1068 00:37:13,598 --> 00:37:16,534 CAN--WHO SHOULD BE SETTING--WHAT 1069 00:37:16,601 --> 00:37:17,535 ORGANIZATION, HOW SHOULD THESE 1070 00:37:17,602 --> 00:37:18,536 STANDARDS COME ABOUT? 1071 00:37:18,603 --> 00:37:23,541 >> OKAY, THAT IS A HARD 1072 00:37:23,608 --> 00:37:24,542 QUESTION. 1073 00:37:24,609 --> 00:37:26,544 >> I KNOW. 1074 00:37:26,611 --> 00:37:27,511 >> WELL, FIRST OF ALL THERE IS 1075 00:37:27,578 --> 00:37:30,514 INCREASING ATTENTION BEING PAID 1076 00:37:30,581 --> 00:37:31,515 TO IDENTITY IN THIS ENVIRONMENT 1077 00:37:31,582 --> 00:37:33,517 AND SOME OF US THINK THAT YOU 1078 00:37:33,584 --> 00:37:37,521 COULD HAVE A DISTRIBUTED 1079 00:37:37,588 --> 00:37:39,523 COLLECTION OF PARTIES WHO WOULD 1080 00:37:39,590 --> 00:37:42,526 VERIFY YOUR IDENTITY, AND LET ME 1081 00:37:42,593 --> 00:37:48,532 USE THAT WORD IN A VERY BROAD 1082 00:37:48,599 --> 00:37:49,533 SENSE TO VERIFY THE ORIGIN OF 1083 00:37:49,600 --> 00:37:51,535 INFORMATION THIS IS BACK TO 1084 00:37:51,602 --> 00:37:53,537 THINKING ABOUT WHERE DO THINGS 1085 00:37:53,604 --> 00:37:54,538 COME FROM, WHAT IS THEIR 1086 00:37:54,605 --> 00:37:55,539 PROVIDENCE? 1087 00:37:55,606 --> 00:37:57,541 SO HAVING A PARTY THAT CAN 1088 00:37:57,608 --> 00:38:00,544 VERIFY A SOURCE AND THEN HAVING A WAY 1089 00:38:00,611 --> 00:38:01,512 FOR THAT SOURCE TO LABEL THE 1090 00:38:01,579 --> 00:38:03,514 INFORMATION AT THE SOURCE IS 1091 00:38:03,581 --> 00:38:07,518 PROVIDED SO WE CAN TRACE IT BACK 1092 00:38:07,585 --> 00:38:08,519 TO THE SOURCE, JUST TO BORROW THAT 1093 00:38:08,586 --> 00:38:10,521 TERM FOR A MOMENT, TRACEABILITY 1094 00:38:10,588 --> 00:38:12,523 IS SOMETHING THAT THE NATIONAL 1095 00:38:12,590 --> 00:38:15,526 INSTITUTES OF HEALTH--I'M SORRY, 1096 00:38:15,593 --> 00:38:16,527 THE NATIONAL INSTITUTES OF 1097 00:38:16,594 --> 00:38:17,528 STANDARDS AND TECHNOLOGY ARE 1098 00:38:17,595 --> 00:38:20,531 VERY BIG ON TRACEABILITY. 1099 00:38:20,598 --> 00:38:22,533 WHAT THEY MEAN BY THAT IS THE 1100 00:38:22,600 --> 00:38:26,537 SCALE AT THE STORE WHERE YOU 1101 00:38:26,604 --> 00:38:27,538 WEIGH A POUND OF BANANAS HAS 1102 00:38:27,605 --> 00:38:29,540 BEEN CALIBRATED AGAINST A STANDARD 1103 00:38:29,607 --> 00:38:31,542 AND YOU CAN TRACE THAT ALL THE 1104 00:38:31,609 --> 00:38:33,544 WAY BACK TO PHYSICAL STANDARDS 1105 00:38:33,611 --> 00:38:35,513 THAT NIST MAINTAINS, SO THAT 1106 00:38:35,579 --> 00:38:36,514 TRACABILITY 1107 00:38:36,580 --> 00:38:39,517 IS OUR FRIEND THERE 1108 00:38:39,583 --> 00:38:40,518 BECAUSE WE CAN TRUST THE 1109 00:38:40,584 --> 00:38:42,520 INSTRUMENTS THAT ARE DOING WEIGHTS ANDS 1110 00:38:42,586 --> 00:38:43,521 MEASURES. 1111 00:38:43,587 --> 00:38:45,523 THAT'S WHAT THE NATIONAL BUREAU STANDARDS ORIGINALLY STARTED, SO 1112 00:38:45,589 --> 00:38:47,525 THE SAME ARGUMENT MIGHT BE MADE 1113 00:38:47,591 --> 00:38:49,527 FOR INFORMATION IF WE CAN 1114 00:38:49,593 --> 00:38:50,528 IDENTIFY THE ORIGIN OF THE 1115 00:38:50,594 --> 00:38:51,529 INFORMATION JUST BECAUSE 1116 00:38:51,595 --> 00:38:52,530 WE KNOW WHERE IT CAME FROM 1117 00:38:52,596 --> 00:38:53,531 DOESN'T NECESARRILY MEAN IT'S CORRECT OR 1118 00:38:53,597 --> 00:38:59,537 ACCURATE, BUT AT LEAST WE KNOW 1119 00:38:59,603 --> 00:39:01,539 WHERE IT CAME FROM. 1120 00:39:01,605 --> 00:39:03,541 AND IN TERMS OF WHO WOULD 1121 00:39:03,607 --> 00:39:06,544 PROVIDE THAT KIND OF VALIDATION, 1122 00:39:06,610 --> 00:39:09,513 YOU CAN IMAGINE JUST AS WE HAVE 1123 00:39:09,580 --> 00:39:10,514 GENERALLY ACCEPTED ACCOUNTING 1124 00:39:10,581 --> 00:39:14,518 PRACTICES, AND WE HAVE COMPANIES 1125 00:39:14,585 --> 00:39:15,519 THAT DO AUDITS, YOU CAN IMAGINE HAVING 1126 00:39:15,586 --> 00:39:17,521 GENERALLY ACCEPTED IDENTITY 1127 00:39:17,588 --> 00:39:21,525 PRACTICES THAT SOME COMPANIES 1128 00:39:21,592 --> 00:39:23,527 WOULD BE CERTIFIED TO PROVIDE 1129 00:39:23,594 --> 00:39:25,529 THAT LABEL SO THAT YOU COULD USE 1130 00:39:25,596 --> 00:39:27,531 A CRYPTOGRAPHIC SIGNATURE SO TO 1131 00:39:27,598 --> 00:39:30,534 SPEAK ON THE CONTENT THAT YOU'RE 1132 00:39:30,601 --> 00:39:33,537 TRYING TO ESTABLISH AS VALID. 1133 00:39:33,604 --> 00:39:34,538 SO I THINK IT DOESN'T HAVE TO BE 1134 00:39:34,605 --> 00:39:36,540 CENTRALIZED NECESSARILY, BUT 1135 00:39:36,607 --> 00:39:37,541 THERE HAVE TO BE STANDARD 1136 00:39:37,608 --> 00:39:39,543 PRACTICES THAT WE ALL AGREE TO 1137 00:39:39,610 --> 00:39:41,512 AND WE HAVE TO AGREE TO BE 1138 00:39:41,579 --> 00:39:44,515 AUDITED WITH REGARD TO OUR 1139 00:39:44,582 --> 00:39:50,521 EXERCISE OF THOSE STANDARDS. 1140 00:39:50,588 --> 00:39:51,522 MAYBE THAT'S A PLACE TO START. 1141 00:39:51,589 --> 00:39:54,525 >> SCIENTIFIC METHOD, GROUNDED 1142 00:39:54,592 --> 00:39:57,528 IN SCIENCE AND GROUNDED IN 1143 00:39:57,595 --> 00:40:01,532 RESEARCH IS CERTAINLY FOR 1144 00:40:01,599 --> 00:40:02,533 MEDICINE, IS OUR STANDARD, 1145 00:40:02,600 --> 00:40:03,534 RIGHT? 1146 00:40:03,601 --> 00:40:04,535 >> YEAH, YOU REMEMBER THAT THE 1147 00:40:04,602 --> 00:40:07,538 SCIENTIFIC METHOD, IT'S 1148 00:40:07,605 --> 00:40:08,539 BEAUTIFUL, I MEAN THERE IS SOMETHING UTTERLY 1149 00:40:08,606 --> 00:40:10,541 ELEGANT ABOUT THIS, IT STARTS 1150 00:40:10,608 --> 00:40:11,542 OUT BY SAYING, YOU HAVE 1151 00:40:11,609 --> 00:40:13,544 OBSERVATIONS AND YOU TRY TO 1152 00:40:13,611 --> 00:40:15,513 FIGURE OUT WHAT THE POSSIBLY 1153 00:40:15,579 --> 00:40:17,515 HAVE PRODUCED THAT OBSERVATION, 1154 00:40:17,581 --> 00:40:20,518 YOU GET A THEORY, AND THEN YOU 1155 00:40:20,584 --> 00:40:23,521 TEST THE THEORY IF YOU'RE A GOOD 1156 00:40:23,587 --> 00:40:24,522 SCIENTIST AND THEN, IF THE DATA 1157 00:40:24,588 --> 00:40:28,526 THAT YOU COLLECT DOESN'T MATCH 1158 00:40:28,592 --> 00:40:29,527 WHAT THE THEORY PREDICTED, THEN 1159 00:40:29,593 --> 00:40:30,528 YOU HAVE TO ADMIT THE 1160 00:40:30,594 --> 00:40:32,530 POSSIBILITY THAT YOUR THEORY IS 1161 00:40:32,596 --> 00:40:33,531 WRONG AND THAT'S REALLY HARD FOR 1162 00:40:33,597 --> 00:40:36,534 SOME SCIENTISTS, THIS IS MY 1163 00:40:36,600 --> 00:40:38,536 THEORY, I'VE BEEN WORKING ON 1164 00:40:38,602 --> 00:40:41,539 THIS, FOR 25 YEARS AND YOU KNOW, 1165 00:40:41,605 --> 00:40:42,540 YOUR INSTRUMENTATION IS BROKEN. 1166 00:40:42,606 --> 00:40:46,544 NOW SOMETIMES THAT'S ACTUALLY 1167 00:40:46,610 --> 00:40:47,545 TRUE, SOME OF YOU MAY REMEMBER 1168 00:40:47,611 --> 00:40:49,513 READING ABOUT AT CERN WHERE SOMEBODY 1169 00:40:49,580 --> 00:40:52,516 THOUGHT THE PHOTONS WERE GOING 1170 00:40:52,583 --> 00:40:54,518 FASTER THAN THE SPEED OF 1171 00:40:54,585 --> 00:40:56,520 NEUTRINOS WE GOING FASTER 1172 00:40:56,587 --> 00:40:57,521 THAN THE SPEED OF LIGHT, WELL IT WAS AN 1173 00:40:57,588 --> 00:40:58,522 INSTRUMENTATION PROBLEM BECAUSE 1174 00:40:58,589 --> 00:40:59,523 THERE WAS A CONNECTOR THAT 1175 00:40:59,590 --> 00:41:01,525 WASN'T CONNECTED PROPERLY AND 1176 00:41:01,592 --> 00:41:03,527 THERE WAS A DELAY THAT WAS 1177 00:41:03,594 --> 00:41:04,528 SUPPOSED TO HAPPEN BUT SOMETIMES THE 1178 00:41:04,595 --> 00:41:06,530 MEASUREMENTS REALLY DO TELL US 1179 00:41:06,597 --> 00:41:09,533 THAT OUR THEORIES ARE WRONG. 1180 00:41:09,600 --> 00:41:10,534 THIS IS THE BEAUTY OF THE 1181 00:41:10,601 --> 00:41:11,535 SCIENTIFIC METHOD IF WE'RE GOOD 1182 00:41:11,602 --> 00:41:13,537 ABOUT IT. 1183 00:41:13,604 --> 00:41:16,540 HERE IS ONE OTHER THING TO OBSERVE HERE, 1184 00:41:16,607 --> 00:41:18,542 AND THAT HAS TO DO WITH 1185 00:41:18,609 --> 00:41:19,543 MEASUREMENT, ITSELF, SOME OF THE 1186 00:41:19,610 --> 00:41:21,512 MOST FUNDAMENTAL THINGS IN 1187 00:41:21,579 --> 00:41:22,513 SCIENCE COME ABOUT BECAUSE WE'VE 1188 00:41:22,580 --> 00:41:24,515 SUDDENLY BEEN ABLE TO MEASURE 1189 00:41:24,582 --> 00:41:26,517 SOMETHING WE COULDN'T MEASURE 1190 00:41:26,584 --> 00:41:27,518 BEFORE, THINK ABOUT DNA 1191 00:41:27,585 --> 00:41:29,520 SEQUENCING, SOMETHING WE COULD 1192 00:41:29,587 --> 00:41:30,521 NOT MEASURE BEFORE, WE CAN NOW 1193 00:41:30,588 --> 00:41:31,522 MEASURE. 1194 00:41:31,589 --> 00:41:32,523 THINK ABOUT SOME OF THE OTHER 1195 00:41:32,590 --> 00:41:34,525 THINGS THAT WE CAN MEASURE NOW 1196 00:41:34,592 --> 00:41:36,527 THAT WE COULDN'T BEFORE. 1197 00:41:36,594 --> 00:41:38,529 ONE OF THE MOST INTERESTING 1198 00:41:38,596 --> 00:41:40,531 QUESTIONS, ANY SCIENTIST AND 1199 00:41:40,598 --> 00:41:42,533 MAYBE EVEN ANY DOCTOR COULD ASK, 1200 00:41:42,600 --> 00:41:44,535 IS WHAT IT THAT I CAN'T MEASURE 1201 00:41:44,602 --> 00:41:47,538 NOW THAT IF I COULD MEASURE 1202 00:41:47,605 --> 00:41:49,540 WOULD MAKE A BIG DIFFERENCE IN 1203 00:41:49,607 --> 00:41:50,541 MY UNDERSTANDING OF DISEASE 1204 00:41:50,608 --> 00:41:52,543 OR IN MY TREATMENT OF 1205 00:41:52,610 --> 00:41:55,512 PATIENTS. 1206 00:41:55,579 --> 00:41:56,513 THERE ARE MILLIONS OF QUESTIONS, 1207 00:41:56,580 --> 00:41:59,516 MILLIONS OF THINGS 1208 00:41:59,583 --> 00:42:03,520 THAT WE CAN'T MEASURE NOW THAT 1209 00:42:03,587 --> 00:42:04,521 ARE EVENTUALLY GOING TO ALLOW US TO TREAT 1210 00:42:04,588 --> 00:42:05,522 PATIENTS BETTER. 1211 00:42:05,589 --> 00:42:06,523 >> SO, WHILE THE IMPORTANT THING 1212 00:42:06,590 --> 00:42:07,524 WOULD BE TO FIGURE OUT WHICH OF 1213 00:42:07,591 --> 00:42:08,525 THOSE THINGS WE DON'T KNOW HOW 1214 00:42:08,592 --> 00:42:12,529 TO MEASURE RIGHT NOW, WE SHOULD 1215 00:42:12,596 --> 00:42:13,530 TRY TO TACKLE FIRST. 1216 00:42:13,597 --> 00:42:16,533 AND TO GIVE AN EXAMPLE OF 1217 00:42:16,600 --> 00:42:21,538 SOMETHING THAT'S REALLY AMAZING 1218 00:42:21,605 --> 00:42:25,542 AND NSF THERE WAS THIS 1219 00:42:25,609 --> 00:42:27,544 LIGO EXPERIMENT, 1220 00:42:27,611 --> 00:42:30,514 LASER INTERFEROMETER GRAVITY 1221 00:42:30,581 --> 00:42:31,515 OBERSERVATION, IT WAS A 40 YEAR 1222 00:42:31,582 --> 00:42:32,516 PROJECT, WHEN IT WAS FIRST 1223 00:42:32,583 --> 00:42:34,518 PROPOSED, THE SCIENTIST WHO 1224 00:42:34,585 --> 00:42:36,520 PROPOSED IT KNEW THAT IF THEY BUILT 1225 00:42:36,587 --> 00:42:38,522 THE MACHINE IT WASN'T GOING TO 1226 00:42:38,589 --> 00:42:39,523 WORK BUT THEY HAD TO BUILD IT TO 1227 00:42:39,590 --> 00:42:41,525 FIGURE OUT HOW FAR OFF THE 1228 00:42:41,592 --> 00:42:43,527 MEASUREMENT WAS TO FIGURE OUT 1229 00:42:43,594 --> 00:42:44,528 WHAT IT WAS TO CHANGE TO GET 1230 00:42:44,595 --> 00:42:46,530 BETTER MEASURES, THEY WENT 1231 00:42:46,597 --> 00:42:47,531 THROUGH THREE CYCLES OF DESIGN AND 1232 00:42:47,598 --> 00:42:49,533 IMPLEMENTATION, EACH ONE GETTING 1233 00:42:49,600 --> 00:42:51,535 CLOSER AND CLOSER AND BETTER 1234 00:42:51,602 --> 00:42:53,537 REFINEMENT AND MORE ACCURATE 1235 00:42:53,604 --> 00:42:55,539 MEASUREMENT UNTIL FINALLY, A FEW 1236 00:42:55,606 --> 00:43:00,544 YEARS AGO, ONE OF THEM WAS IN TEST 1237 00:43:00,611 --> 00:43:01,512 MODE, AND IT DETECTED TWO BLACK HOLES 1238 00:43:01,578 --> 00:43:04,515 CIRCLING EACH OTHER AND 1239 00:43:04,581 --> 00:43:05,516 COLLIDING AND A BILLION LIGHT 1240 00:43:05,582 --> 00:43:08,519 YEARS AWAY. 1241 00:43:08,585 --> 00:43:10,521 AND THAT WAS AN UNMISTAKABLE 1242 00:43:10,587 --> 00:43:12,523 SIGNAL, FORTUNATELY THERE WERE TWO 1243 00:43:12,589 --> 00:43:13,524 SUCH MACHINES AND THE SECOND ONE 1244 00:43:13,590 --> 00:43:15,526 ALSO DETECTED THE SAME SIGNAL 1245 00:43:15,592 --> 00:43:18,529 AND THERE WAS NOTHING MESSY 1246 00:43:18,595 --> 00:43:20,531 ABOUT IT AT ALL, I MEAN IT WAS A 1247 00:43:20,597 --> 00:43:23,534 PERFECT TRACE OF WHAT IT WOULD 1248 00:43:23,600 --> 00:43:25,536 PREDICT FROM THE--IN TERMS OF 1249 00:43:25,602 --> 00:43:26,537 THE GRAVITATIONAL FLUCTUATION. 1250 00:43:26,603 --> 00:43:29,540 SO THAT'S AN EXAMPLE, WHERE WE 1251 00:43:29,606 --> 00:43:30,541 KNEW THAT WE COULDN'T MEASURE 1252 00:43:30,607 --> 00:43:32,543 SOMETHING, BUT WE KNEW THAT WE 1253 00:43:32,609 --> 00:43:33,544 HAD TO BUILD SOMETHING TO FIND 1254 00:43:33,610 --> 00:43:35,512 OUT HOW FAR OFF WE WERE, THAT'S 1255 00:43:35,579 --> 00:43:36,513 PROBABLY GOING TO BE TRUE FOR A 1256 00:43:36,580 --> 00:43:38,515 LOT OF OTHER THINGS. 1257 00:43:38,582 --> 00:43:39,516 >> WELL, IT SOUNDS LIKE 1258 00:43:39,583 --> 00:43:41,518 ARTIFICIAL INTELIENCE TO ME. 1259 00:43:41,585 --> 00:43:42,519 >> WELL, IT'S CERTAINLY A THING, 1260 00:43:42,586 --> 00:43:44,521 WHICH IS NOT WORKING THE WAY WE 1261 00:43:44,588 --> 00:43:47,524 WOULD LIKE IT TO AND SO WE KNOW 1262 00:43:47,591 --> 00:43:49,526 WE HAVE WORK TO DO. 1263 00:43:49,593 --> 00:43:50,527 GOOD POINT. 1264 00:43:50,594 --> 00:43:52,529 >> YEAH, OKAY. 1265 00:43:52,596 --> 00:43:54,531 WHAT ADVICE DO YOU HAVE FOR 1266 00:43:54,598 --> 00:44:00,537 THOSE HERE WHO ARE JUST STARTING 1267 00:44:00,604 --> 00:44:01,538 THEIR SCIENTIFIC CAREERS? 1268 00:44:01,605 --> 00:44:02,539 >> WELL, LET'S SEE THE STOCK 1269 00:44:02,606 --> 00:44:06,543 ANSWER THAT I HAVE TO THAT IS YOU 1270 00:44:06,610 --> 00:44:07,544 SHOULD GO INTO ASTROPHYSICS. 1271 00:44:07,611 --> 00:44:08,512 [LAUGHTER] 1272 00:44:08,579 --> 00:44:10,514 I WILL EXPLAIN. 1273 00:44:10,581 --> 00:44:11,515 THERE'S A RATIONAL FOR THAT. 1274 00:44:11,582 --> 00:44:13,517 YOU KNOW A HUNDRED YEARS AGO, WE 1275 00:44:13,584 --> 00:44:16,520 THOUGHT THAT WE KNEW PRETTY MUCH 1276 00:44:16,587 --> 00:44:18,522 HOW THE UNIVERSE WORKED, WE HAD 1277 00:44:18,589 --> 00:44:20,524 THIS GREAT UTONNIAN MODEL AND IT 1278 00:44:20,591 --> 00:44:22,526 MADE GREAT PREDICTIONS AND WE 1279 00:44:22,593 --> 00:44:24,528 FIGURED WELL IF WE JUST MEASURE 1280 00:44:24,595 --> 00:44:25,529 THE CONSTANTS MORE ACCURATELY, 1281 00:44:25,596 --> 00:44:26,530 WE WILL BE ABLE TO PREDICT 1282 00:44:26,597 --> 00:44:28,532 EVERYTHING AND THEN EINSTEIN 1283 00:44:28,599 --> 00:44:31,535 COMES ALONG AND HE BLOWS THAT 1284 00:44:31,602 --> 00:44:32,536 WHOLE THING UP. 1285 00:44:32,603 --> 00:44:34,538 AND THEN QUANTUM PEOPLE COME 1286 00:44:34,605 --> 00:44:37,541 ALONG AND THEY BLOW UP EINSTEIN 1287 00:44:37,608 --> 00:44:39,543 AND YOU KNOW, AT THIS POINT, NOW 1288 00:44:39,610 --> 00:44:41,512 A HUNDRED YEARS LATER, WE KNOW 1289 00:44:41,578 --> 00:44:42,513 LESS ABOUT THE UNIVERSE THAN WE 1290 00:44:42,579 --> 00:44:45,516 THOUGHT WE DID A HUNDRED YEARS 1291 00:44:45,582 --> 00:44:48,519 AGO. 1292 00:44:48,585 --> 00:44:50,521 WE KNOW ABOUT 5% OF THE UNIVERSE 1293 00:44:50,587 --> 00:44:55,526 IS NORMAL MATTER, THAT INCLUDES 1294 00:44:55,592 --> 00:44:56,527 ANTIMATTER, AND THEN, 70% IS 1295 00:44:56,593 --> 00:44:57,528 DARK ENERGY. 1296 00:44:57,594 --> 00:44:58,529 WHAT'S THAT? 1297 00:44:58,595 --> 00:45:05,536 I DON'T KNOW BUT IT'S CAUSING 1298 00:45:05,602 --> 00:45:06,537 THE EXPANSION OF THE UNIVERSE AND 1299 00:45:06,603 --> 00:45:08,539 IT'S ACCELERATING AND THIS IS 1300 00:45:08,605 --> 00:45:09,540 SORT OF WTF? 1301 00:45:09,606 --> 00:45:12,543 [LAUGHTER] 1302 00:45:12,609 --> 00:45:13,544 AND THEN 25% IS DARK 1303 00:45:13,610 --> 00:45:15,512 MATTER--I'M SORRY,I SAID THAT WRONG IT'S DARK 1304 00:45:15,579 --> 00:45:17,514 ENERGY AND 70% DARK MATTER IS 1305 00:45:17,581 --> 00:45:18,515 OTHER STUFF, WHAT'S THAT, WE 1306 00:45:18,582 --> 00:45:20,517 DON'T KNOW. 1307 00:45:20,584 --> 00:45:22,519 SO WE KNOW ABOUT 5%, SO IF YOU 1308 00:45:22,586 --> 00:45:24,521 GO INTO ASTROPHYSICS AND YOU DO 1309 00:45:24,588 --> 00:45:29,526 ANYTHING, THE CHANCES OF A NOBEL 1310 00:45:29,593 --> 00:45:30,527 PRIZE ARE SKY HIGH. 1311 00:45:30,594 --> 00:45:32,529 [LAUGHTER] 1312 00:45:32,596 --> 00:45:36,533 SO HONESTLY, TO BE FAIR ABOUT 1313 00:45:36,600 --> 00:45:39,536 THE ANSWER HERE, IF I WERE 1314 00:45:39,603 --> 00:45:40,537 YOUNG, I'M NOT OBVIOUSLY, BUT IF 1315 00:45:40,604 --> 00:45:44,541 I WERE YOUNG, I THINK I WOULD 1316 00:45:44,608 --> 00:45:45,542 WANT TO GET DEEPER AND DEEPER 1317 00:45:45,609 --> 00:45:47,544 INTO MICROBIOLOGY. 1318 00:45:47,611 --> 00:45:49,513 I HAVE BECOME UTTERLY FASCINATED 1319 00:45:49,580 --> 00:45:52,516 BY HOW COMPLICATED CELLS ARE AND 1320 00:45:52,583 --> 00:45:53,517 I USED TO THINK THE CELL WAS 1321 00:45:53,584 --> 00:45:55,519 THIS LITTLE BAG FULL OF 1322 00:45:55,586 --> 00:45:58,522 CHEMICALS AND THEY BUMPED INTO 1323 00:45:58,589 --> 00:46:00,524 EACH OTHER AND STUFF HAPPENED. 1324 00:46:00,591 --> 00:46:01,525 WELL THEN I GOT AHOLD OF THE 1325 00:46:01,592 --> 00:46:03,527 MICROCHEMISTRY OF THE CELL, YOU 1326 00:46:03,594 --> 00:46:05,529 KNOW THAT BIG 1,700 PAGE THING 1327 00:46:05,596 --> 00:46:08,532 AND DISCOVERED THAT THE CELL IS 1328 00:46:08,599 --> 00:46:09,533 ABOUT AS COMPLICATED AS DOWN 1329 00:46:09,600 --> 00:46:10,534 TOWN MANHATTAN. 1330 00:46:10,601 --> 00:46:12,536 AND THERE'S ALL KINDS OF STUFF 1331 00:46:12,603 --> 00:46:15,539 GOING ON IN THERE--STUFF GOING 1332 00:46:15,606 --> 00:46:16,540 ON IN THERE EMPLOY THERE'S THESE 1333 00:46:16,607 --> 00:46:18,542 MICROTUBULES AND THESE PROTEIN 1334 00:46:18,609 --> 00:46:19,543 THINGS THAT WALK ALONG THE 1335 00:46:19,610 --> 00:46:20,544 TUBULES CARRYING OTHER PROTEINS 1336 00:46:20,611 --> 00:46:22,513 THAT ARE LABELED JUST 1337 00:46:22,579 --> 00:46:24,515 LIKE PACKETS ON THE INTERNET AND 1338 00:46:24,581 --> 00:46:26,517 THEY GET DELIVERED TO THE RIGHT 1339 00:46:26,583 --> 00:46:28,519 PLACE INSIDE THE CELL. 1340 00:46:28,585 --> 00:46:32,523 I MEAN IT'S--SOMEBODY FROM 1341 00:46:32,589 --> 00:46:34,525 UC SAN DIEGO SENT ME A NOTE THAT SAID, HI, THERE'S 1342 00:46:34,591 --> 00:46:35,526 AN INTERNET INSIDE THE CELL AND 1343 00:46:35,592 --> 00:46:36,527 THEY TALK TO ME ABOUT EXACTLY 1344 00:46:36,593 --> 00:46:39,530 THAT. 1345 00:46:39,596 --> 00:46:40,531 SO MAN, THAT WOULD BE SO MUCH 1346 00:46:40,597 --> 00:46:42,533 DAMN FUN, I WOULD LOVE TO DO 1347 00:46:42,599 --> 00:46:43,534 THAT. 1348 00:46:43,600 --> 00:46:44,535 >> OKAY. 1349 00:46:44,601 --> 00:46:46,537 SO WE'RE NOW GETTING TO THE 1350 00:46:46,603 --> 00:46:50,541 SECOND MOST POPULAR QUESTION 1351 00:46:50,607 --> 00:46:52,543 RECEIVED FROM ALL OF YOU. 1352 00:46:52,609 --> 00:46:56,513 159 UPVOTES FOR THIS ONE. IS THIS ONE 1353 00:46:56,580 --> 00:46:57,514 CALLED WHAT WERE YOU THINKING? 1354 00:46:57,581 --> 00:46:58,515 >> YEAH. 1355 00:46:58,582 --> 00:47:00,517 THIS ONE'S PRETTY PRACTICAL 1356 00:47:00,584 --> 00:47:03,520 THOUGH, HOW DO WE ENSURE, FOR 1357 00:47:03,587 --> 00:47:04,521 BIOMEDICAL RESEARCH, THE BROAD 1358 00:47:04,588 --> 00:47:06,523 AND APPROPRIATE 1359 00:47:06,590 --> 00:47:10,527 REPRESENTATIVENESS OF THE DATA 1360 00:47:10,594 --> 00:47:12,529 BEING USED TO TRAIN OUR MODELS 1361 00:47:12,596 --> 00:47:15,532 GIVEN THE HISTORICAL INEQUITIES 1362 00:47:15,599 --> 00:47:16,533 OF LARGE DATA COLLECTION? 1363 00:47:16,600 --> 00:47:18,535 >> SO THIS IS REALLY HARD 1364 00:47:18,602 --> 00:47:24,541 BECAUSE WE KNOW WE HAVE ENORMOUS 1365 00:47:24,608 --> 00:47:27,544 SDIVERSITY IN THIS WORLD BUT 1366 00:47:27,611 --> 00:47:28,512 NOT EVERYONE IN THIS DIVERSE 1367 00:47:28,579 --> 00:47:30,514 GLOBE HAS ACCESS TO MEDICAL CARE 1368 00:47:30,581 --> 00:47:32,516 THAT WOULD CAPTURE THE DATA IN 1369 00:47:32,583 --> 00:47:33,517 THE FIRST PLACE. 1370 00:47:33,584 --> 00:47:35,519 AND SO, THE IMPORTANT THING IS 1371 00:47:35,586 --> 00:47:38,522 WHAT YOU JUST DID, WHICH IS TO 1372 00:47:38,589 --> 00:47:39,523 RECOGNIZE THAT WE DON'T HAVE 1373 00:47:39,590 --> 00:47:44,528 THAT DATA AND THAT WE SHOULD 1374 00:47:44,595 --> 00:47:46,530 TAKE STEPS IF WE POSSIBLY CAN TO 1375 00:47:46,597 --> 00:47:50,534 GET THAT DATA. IT MAY BE AS 1376 00:47:50,601 --> 00:47:52,536 THE COST OF GATHERING 1377 00:47:52,603 --> 00:47:54,538 INFORMATION ABOUT PEOPLE'S 1378 00:47:54,605 --> 00:47:56,540 HEALTH GETS LESS AND LESS AND 1379 00:47:56,607 --> 00:47:58,542 MORE AFFORDABLE AND MORE 1380 00:47:58,609 --> 00:47:59,543 FEASIBLE AND MORE SUPPORTABLE, 1381 00:47:59,610 --> 00:48:02,512 WE MAY BE ABLE TO GET MORE PEOPLE 1382 00:48:02,579 --> 00:48:05,515 WILLING TO CONTRIBUTE DATA ABOUT 1383 00:48:05,582 --> 00:48:07,517 THEIR--THE WAY THEIR BODIES WORK 1384 00:48:07,584 --> 00:48:10,520 OR ARE WORKING. 1385 00:48:10,587 --> 00:48:11,521 IN FACT, IF I CAN MAKE A GENERAL 1386 00:48:11,588 --> 00:48:12,522 OBSERVATION. 1387 00:48:12,589 --> 00:48:14,524 SOMETIMES YOU TAKE THE BIGGEST 1388 00:48:14,591 --> 00:48:15,525 AND MOST EXPENSIVE TECHNOLOGY 1389 00:48:15,592 --> 00:48:18,528 AND IT'S ONLY USED BY PEOPLE 1390 00:48:18,595 --> 00:48:20,530 THAT CAN AFFORD IT, HOW ABOUT 1391 00:48:20,597 --> 00:48:22,532 COMPUTING IN THE EARLY 1392 00:48:22,599 --> 00:48:24,534 DAYS, ONLY VERY BIG COMPANIES COULD 1393 00:48:24,601 --> 00:48:26,536 AFFORD TO HAVE A COMPUTER AND 1394 00:48:26,603 --> 00:48:28,538 THEY WERE SO PROUD OF IT THEY 1395 00:48:28,605 --> 00:48:30,540 PUT IT IN A GLASS BALLROOM AND 1396 00:48:30,607 --> 00:48:31,541 SAID LOOK, WE HAVE 1397 00:48:31,608 --> 00:48:32,542 COMPUTERS AND YOU DON'T. 1398 00:48:32,609 --> 00:48:33,543 WELL, NOW WE HAVE THEM IN OUR 1399 00:48:33,610 --> 00:48:40,517 POCKETS. 1400 00:48:40,584 --> 00:48:42,519 I HAVE ONE IN NIGH EAR, AND SO IT BECOMES 1401 00:48:42,586 --> 00:48:44,521 SMALLER, LESS ENERGY DEPENDENT AND 1402 00:48:44,588 --> 00:48:46,523 AFFORDABLE SO MANY 1403 00:48:46,590 --> 00:48:47,524 TECHNOLOGIES HAVE THAT 1404 00:48:47,591 --> 00:48:49,526 CHARACTERISTIC. SO ONE THINK WE CAN DO IS TO TRY TO RECOGNIZE EXACTLY THE PROBLEM THAT 1405 00:48:49,593 --> 00:48:51,528 YOU SAID AND LOOK FOR WAYS OF GATHERING THAT DATA 1406 00:48:51,595 --> 00:48:53,530 OF LOWER AND LOWER COST 1407 00:48:53,597 --> 00:48:55,532 TRYING TO PROVIDE BENEFIT TO A LARGER 1408 00:48:55,599 --> 00:48:56,533 AND LARGER POPULATION AND 1409 00:48:56,600 --> 00:48:58,535 ACCUMULATE AS MUCH DATA AS WE 1410 00:48:58,602 --> 00:49:00,537 CAN TO BUILD BETTER MODELS BUT 1411 00:49:00,604 --> 00:49:04,541 IT HAS TO BE SOMETHING WE 1412 00:49:04,608 --> 00:49:05,542 CONSCIOUSLY DECIDE TO DO. 1413 00:49:05,609 --> 00:49:08,512 >> YEAH. 1414 00:49:08,578 --> 00:49:10,514 AND I WILL ADD TO THAT THAT WE 1415 00:49:10,580 --> 00:49:12,516 ALSO RECOGNIZE THAT SEGMENTS OF 1416 00:49:12,582 --> 00:49:14,518 OUR SOCIETY WHO HAVE--FOR WHOM 1417 00:49:14,584 --> 00:49:15,519 WE HAVE THE LEAST DATA ARE 1418 00:49:15,585 --> 00:49:17,521 ALSO THE ONES WITH THE WORST 1419 00:49:17,587 --> 00:49:19,523 OUTCOME WHO ARE ALSO THE ONES THAT 1420 00:49:19,589 --> 00:49:27,531 HAVE LEAST TRUST IN MANY WAYS 1421 00:49:27,597 --> 00:49:28,532 AND SO, WE'VE GOT A SOCIAL 1422 00:49:28,598 --> 00:49:29,533 PROBLEM. 1423 00:49:29,599 --> 00:49:31,535 SCIENCE ISN'T GOING TO FIX THIS. 1424 00:49:31,601 --> 00:49:33,537 IT'S THE ONLY THING THAT'S GOING 1425 00:49:33,603 --> 00:49:39,543 TO FIX THIS IS THE HUMAN TO 1426 00:49:39,609 --> 00:49:40,544 HUMAN INTERACTION AND THE 1427 00:49:40,610 --> 00:49:41,511 EARNING TRUST. 1428 00:49:41,578 --> 00:49:45,515 >> WELL, SOCIAL AND ECONOMIC 1429 00:49:45,582 --> 00:49:47,517 RESPONSE IS NEEDED. 1430 00:49:47,584 --> 00:49:48,518 THIS IS--A LOT OF PROBLEMS WE 1431 00:49:48,585 --> 00:49:53,523 SEE, ARE A CONSEQUENCE OF BEING 1432 00:49:53,590 --> 00:49:55,525 IN A LOWER SOCIOECONOMIC LAYER 1433 00:49:55,592 --> 00:49:57,527 THAN OTHERS AND SO WE HAVE TO 1434 00:49:57,594 --> 00:49:59,529 RECOGNIZE THAT. 1435 00:49:59,596 --> 00:50:02,532 SO YOUR POINT'S EXTREMELY WELL 1436 00:50:02,599 --> 00:50:06,536 TAKEN AND ALL WE CAN DO IS SHOW 1437 00:50:06,603 --> 00:50:07,537 BY OUR DELIBERATE ACTION THAT WE 1438 00:50:07,604 --> 00:50:10,540 ARE TRUSTABLE. 1439 00:50:10,607 --> 00:50:11,541 >> OKAY, SO THIS ONE SHOULD BE 1440 00:50:11,608 --> 00:50:13,543 EASY. 1441 00:50:13,610 --> 00:50:17,514 IN WHAT WAYS COULD TECHNOLOGY 1442 00:50:17,581 --> 00:50:19,516 LIKE CHATGPT AND BARD BE 1443 00:50:19,583 --> 00:50:20,517 LEVERAGED, AND THEY ARE THERE, BE 1444 00:50:20,584 --> 00:50:22,519 LEVERAGED TO IMPROVE MEDICAL 1445 00:50:22,586 --> 00:50:24,521 RESEARCH IN PATIENT CARE AND IF 1446 00:50:24,588 --> 00:50:26,523 THEY ARE USED THERE WHAT 1447 00:50:26,590 --> 00:50:28,525 MEASURES NEED TO BE TAKEN TO 1448 00:50:28,592 --> 00:50:31,528 ENSURE THE ACCURACY AND 1449 00:50:31,595 --> 00:50:32,529 RELIABILITY OF THE INFORMATION 1450 00:50:32,596 --> 00:50:33,530 GENERATED? 1451 00:50:33,597 --> 00:50:37,534 >> WELL I'M GOING TO BORROW FROM 1452 00:50:37,601 --> 00:50:42,539 ONE OF MY DISTANT RELATIVES, AT 1453 00:50:42,606 --> 00:50:45,542 LEAST WE THINK WE WERE RELATED 1454 00:50:45,609 --> 00:50:47,544 BENNETT CERF, AND SOMEBODY WILL REMEMBER BENNETT CERT AND WHAT'S MY LINE 1455 00:50:47,611 --> 00:50:49,513 AND RANDOM HOUSE AND THE FACT THAT HE NEVER 1456 00:50:49,579 --> 00:50:52,516 MET A PUN HE DIDN'T LIKE. AND 1457 00:50:52,582 --> 00:50:54,518 HE WROTE A BOOK CALLED LAUGHTER 1458 00:50:54,584 --> 00:50:56,520 IS THE BEST MEDICINE AND MY 1459 00:50:56,586 --> 00:50:58,522 REACTION RIGHT NOW IS MAYBE WE 1460 00:50:58,588 --> 00:51:01,525 SHOULD USE BARD AND GEMINI 1461 00:51:01,591 --> 00:51:02,526 AND CHATGPT FOR ITS LAUGHTER 1462 00:51:02,592 --> 00:51:04,528 ELEMENT MORE THAN ANYTHING ELSE, 1463 00:51:04,594 --> 00:51:06,530 IT'S GREAT MEDICINE. 1464 00:51:06,596 --> 00:51:07,531 THEY ARE GENERALLY NOT READY FOR 1465 00:51:07,597 --> 00:51:10,534 PRIME TIME DIAGNOSTICS OR 1466 00:51:10,600 --> 00:51:11,535 ANYTHING LIKE THAT OR EVEN 1467 00:51:11,601 --> 00:51:13,537 MEDICAL ADVICE. 1468 00:51:13,603 --> 00:51:14,538 LET ME SET ASIDE THE LARGE 1469 00:51:14,604 --> 00:51:16,540 LANGUAGE MODEL WHICH IS A 1470 00:51:16,606 --> 00:51:19,543 QUESTION YOU ASKED AND ARGUE 1471 00:51:19,609 --> 00:51:21,545 IT'S NOT READY TO BE HELPFUL TO 1472 00:51:21,611 --> 00:51:23,513 US OTHER THAN FOR I WOULD SAY 1473 00:51:23,580 --> 00:51:25,515 ENTERTAINMENT. 1474 00:51:25,582 --> 00:51:27,517 IT DOES DO--LET ME BE FAIR ABOUT 1475 00:51:27,584 --> 00:51:28,518 THIS, THERE IS ONE THING FOR SURE 1476 00:51:28,585 --> 00:51:30,520 THAT THE LARGE LANGUAGE MODELS 1477 00:51:30,587 --> 00:51:34,524 HAVE BEEN ABLE TO DO WHICH I 1478 00:51:34,591 --> 00:51:36,526 FIND ASTONISHING AND THAT'S 1479 00:51:36,593 --> 00:51:38,528 LANGUAGE TRANSLATION, SPEECH 1480 00:51:38,595 --> 00:51:39,529 UNDERSTANDING, SPEECH SYNTHESIS, 1481 00:51:39,596 --> 00:51:41,531 IT'S ASTONISHING HOW WELL THESE 1482 00:51:41,598 --> 00:51:43,533 MODELS WORK AND I USE THEM ALL 1483 00:51:43,600 --> 00:51:44,534 THE TIME BECAUSE I GET E-MAIL 1484 00:51:44,601 --> 00:51:46,536 FROM ALL OVER THE WORLD AND A 1485 00:51:46,603 --> 00:51:48,538 LOT OF IT IS NOT IN A LANGUAGE 1486 00:51:48,605 --> 00:51:51,541 THAT I SPEAK, TRANSLATIONS ARE 1487 00:51:51,608 --> 00:51:54,544 NOT PERFECT BUT THE LARGE LANGUAGE 1488 00:51:54,611 --> 00:51:56,513 MODELS HAVE THEIR UTILITY BUT 1489 00:51:56,580 --> 00:51:58,515 MACHINE LEARNING TOOLS WHICH ARE 1490 00:51:58,582 --> 00:52:00,517 NOT LARGE LANGUAGE MODELS MAY 1491 00:52:00,584 --> 00:52:03,520 VERY WELL BE OUR STRONGEST 1492 00:52:03,587 --> 00:52:05,522 AVENUE TO ANALYZING LARGE 1493 00:52:05,589 --> 00:52:06,523 QUANTITIES OF DATA LOOKING FOR 1494 00:52:06,590 --> 00:52:08,525 CORRELATIONS AND THEN TRYING TO 1495 00:52:08,592 --> 00:52:11,528 UNDERSTAND WHETHER THERE IS AN 1496 00:52:11,595 --> 00:52:12,529 EXPLANATION FOR WHAT THOSE 1497 00:52:12,596 --> 00:52:17,534 CORRELATIONS EXHIBIT. 1498 00:52:17,601 --> 00:52:19,536 AND THIS IS WHY MY FRIEND JUDEA PEARL 1499 00:52:19,603 --> 00:52:20,537 AT UCLA WHO IS 1500 00:52:20,604 --> 00:52:21,538 ANOTHER TURING AWARD WINNER WROTE TWO 1501 00:52:21,605 --> 00:52:22,539 BOOKS. 1502 00:52:22,606 --> 00:52:25,542 ONE OF THEM'S CALLED THE BOOK OF 1503 00:52:25,609 --> 00:52:26,543 WHY, AND THE OTHER ONE WAS CALLED 1504 00:52:26,610 --> 00:52:29,513 CAUSALITY AND HIS POINT IS THAT 1505 00:52:29,579 --> 00:52:30,514 UNLESS YOU HAVE A CAUSAL MODEL, 1506 00:52:30,580 --> 00:52:33,517 HAVING CORRELATIONS DOESN'T GIVE 1507 00:52:33,583 --> 00:52:35,519 YOU MUCH INSIGHT. 1508 00:52:35,585 --> 00:52:40,524 SO I BELIEVE THAT THE MACHINE 1509 00:52:40,590 --> 00:52:43,527 LEARNING TOOLS CAN IN FACT HELP 1510 00:52:43,593 --> 00:52:44,528 US IN JUST--AND LOOK FOR 1511 00:52:44,594 --> 00:52:45,529 CORRELATIONS WHICH POINT WE NOW 1512 00:52:45,595 --> 00:52:48,532 CAN START ASKING OURSLEVES 1513 00:52:48,598 --> 00:52:49,533 SERIOUS QUESTIONS ABOUT WHAT 1514 00:52:49,599 --> 00:52:51,535 POSSIBLE MODEL COULD PRODUCE 1515 00:52:51,601 --> 00:52:54,538 THAT CORRELATION AND DOES THAT 1516 00:52:54,604 --> 00:52:58,542 GIVE US INSIGHT INTO METABOLISM 1517 00:52:58,608 --> 00:53:01,545 AND THE WAY IN WHICH BODIES 1518 00:53:01,611 --> 00:53:02,512 WORK. 1519 00:53:02,579 --> 00:53:05,515 >> OKAY. 1520 00:53:05,582 --> 00:53:07,517 ONE OF THE BIGGEST CHALLENGES IN 1521 00:53:07,584 --> 00:53:09,519 INSURING THE RELIABILITY AND 1522 00:53:09,586 --> 00:53:10,520 VALIDITY OF AI MACHINE LEARNING 1523 00:53:10,587 --> 00:53:12,522 ALGORITHMS HAD HEALTHCARE AND 1524 00:53:12,589 --> 00:53:13,523 HOW CAN WE ADDRESS THOSE 1525 00:53:13,590 --> 00:53:15,525 CHALLENGES? 1526 00:53:15,592 --> 00:53:17,527 >> WELL YOU'VE ALREADY IMPLIED 1527 00:53:17,594 --> 00:53:19,529 SEVERAL WAYS IN WHICH TO DO 1528 00:53:19,596 --> 00:53:20,530 THAT, ONE OF THEM IS TO 1529 00:53:20,597 --> 00:53:26,536 DELIBERATELY SET ABOUT 1530 00:53:26,603 --> 00:53:29,539 COLLECTING THE DATA THAT WE KNOW 1531 00:53:29,606 --> 00:53:31,541 WE NEED AND DON'T HAVE AS WITH 1532 00:53:31,608 --> 00:53:32,542 THE COHORTS THAT DON'T GET 1533 00:53:32,609 --> 00:53:34,544 ADEQUATE ATTENTION FOR EXAMPLE 1534 00:53:34,611 --> 00:53:37,514 AND THEN PROCEED, YOU ALSO 1535 00:53:37,581 --> 00:53:38,515 SUGGESTED FOR EXAMPLE THAT WE 1536 00:53:38,582 --> 00:53:42,519 PROVIDE INCENTIVES NOT ONLY TO 1537 00:53:42,586 --> 00:53:44,521 PATIENTS BUT TO DOCTORS AND TO 1538 00:53:44,588 --> 00:53:45,522 RESEARCHERS FOR COLLECTING AND 1539 00:53:45,589 --> 00:53:46,523 SHARING THEIR DATA, FOR 1540 00:53:46,590 --> 00:53:48,525 PROVOIDING THEM WITH THE TOOLS 1541 00:53:48,592 --> 00:53:50,527 FOR DOING THAT, FOR PROVIDING 1542 00:53:50,594 --> 00:53:53,530 THEM WITH STANDARDS SO THAT THE 1543 00:53:53,597 --> 00:53:54,531 DATA IS COMMENSURATE. 1544 00:53:54,598 --> 00:53:59,536 ALL OF THOSE THINGS ARE STEPS 1545 00:53:59,603 --> 00:54:03,540 THAT NIH IS IN A UNIQUE POSITION 1546 00:54:03,607 --> 00:54:05,542 TO TAKE, I THINK TO SHOW LIGHT 1547 00:54:05,609 --> 00:54:07,544 IN DARK CORNERS. 1548 00:54:07,611 --> 00:54:09,512 AND I HAVE THE SENSE THAT YOU 1549 00:54:09,579 --> 00:54:11,514 WANT TO DO THAT. 1550 00:54:11,581 --> 00:54:13,516 SO I WOULD ENCOURAGE THAT AND I 1551 00:54:13,583 --> 00:54:19,522 HOPE THE LEGISLATORS UNDERSTAND 1552 00:54:19,589 --> 00:54:21,524 THIS IS AN IMPORTANT WAY 1553 00:54:21,591 --> 00:54:22,525 FORWARD. 1554 00:54:22,592 --> 00:54:23,526 >> RELATED. 1555 00:54:23,593 --> 00:54:26,529 GIVEN YOUR EXPERIENCE WITH THE 1556 00:54:26,596 --> 00:54:27,530 EVOLUTION OF THE INTERNET AND 1557 00:54:27,597 --> 00:54:31,534 DIGITAL TECHNOLOGIES, WHAT ARE 1558 00:54:31,601 --> 00:54:32,535 THE ETHICAL CONSIDERATIONS AND 1559 00:54:32,602 --> 00:54:35,538 SECONDARY PRIVACY CONCERNS, DO 1560 00:54:35,605 --> 00:54:38,541 YOU THINK WE SHOULD ALL BE 1561 00:54:38,608 --> 00:54:40,543 THINKING ABOUT AS WE PRIORITIZE 1562 00:54:40,610 --> 00:54:42,512 MACHINE LEARNING TO BECOME 1563 00:54:42,579 --> 00:54:44,514 INTEGRATED INTO HEALTHCARE? 1564 00:54:44,581 --> 00:54:45,515 >> THAT'S AN INTERESTING KIND OF 1565 00:54:45,582 --> 00:54:46,516 SANDWICH YOU JUST PUT TOGETHER 1566 00:54:46,583 --> 00:54:48,518 BECAUSE YOU STARTED OUT WITH THE 1567 00:54:48,585 --> 00:54:49,519 INTERNET AND THEN YOU RAN OFF ON 1568 00:54:49,586 --> 00:54:51,521 THE MACHINE LEARNING STUFF, THE 1569 00:54:51,588 --> 00:54:57,527 INTERNET IS NOT A MACHINE 1570 00:54:57,594 --> 00:54:58,528 LEARNING SYSTEM EXCEPT IN ONE OR TWO 1571 00:54:58,595 --> 00:54:59,529 SENSES. 1572 00:54:59,596 --> 00:55:01,531 THE ROUTING SYSTEM, FOR EXAMPLE, 1573 00:55:01,598 --> 00:55:02,532 ROUTING A PACKAGE GOING THRU THE 1574 00:55:02,599 --> 00:55:04,534 NETWORK COULD BE HELPED WITH 1575 00:55:04,601 --> 00:55:05,535 MACHINE LEARNING TOOLS AND WE 1576 00:55:05,602 --> 00:55:10,540 USE THOSE. 1577 00:55:10,607 --> 00:55:12,542 BUT THE INTERNET ITSELF IS 1578 00:55:12,609 --> 00:55:13,543 BASICALLY, SIMPLY A MECHANISM FOR 1579 00:55:13,610 --> 00:55:14,544 TRANSMITTING DIGITAL INFORMATION FROM ONE 1580 00:55:14,611 --> 00:55:17,514 PLACE TO ANOTHER 1581 00:55:17,580 --> 00:55:19,516 BUT THEN YOU GET TO LAYER A BUNCH OF STUFF ON TOP OF 1582 00:55:19,582 --> 00:55:20,517 THAT, WHEN TIM BURNERS-LEE INVENTED THE 1583 00:55:20,583 --> 00:55:23,520 WORLD WIDE WEB HE PUT A 1584 00:55:23,586 --> 00:55:25,522 A LAYER OF PROTOCOL ON TOP OF THE 1585 00:55:25,588 --> 00:55:26,523 TCP/IP WORK THAT BOB KAHN AND I DID 1586 00:55:26,589 --> 00:55:33,530 IN ORDER TO BUILD WHAT WE NOW 1587 00:55:33,596 --> 00:55:35,532 THINK OF AS THE WORLD WIDE WEB. 1588 00:55:35,598 --> 00:55:39,536 I'M TRYING TO FIGURE OUT 1589 00:55:39,602 --> 00:55:40,537 HOW--OKAY, SO YOU MENTIONED 1590 00:55:40,603 --> 00:55:41,538 ETHICAL CONCERNS. 1591 00:55:41,604 --> 00:55:42,539 THIS IS A PLACE WHERE SOME 1592 00:55:42,605 --> 00:55:43,540 PEOPLE COME UP AND SAY, YOU KNOW 1593 00:55:43,606 --> 00:55:45,542 IT'S ALL YOUR FAULT, AND WHAT 1594 00:55:45,608 --> 00:55:46,543 WERE YOU THINKING, LOOK AT 1595 00:55:46,609 --> 00:55:48,511 ALL THE BAD STUFF THAT HAPPENED 1596 00:55:48,578 --> 00:55:54,517 AND MY REACTION TO THIS IS, 1597 00:55:54,584 --> 00:55:56,519 WELL, WAIT A MINUTE, PEOPLE 1598 00:55:56,586 --> 00:55:57,520 SOMETIMES TAKE TOOLS THAT HAVE 1599 00:55:57,587 --> 00:55:59,522 BEEN INVENTED FOR CONSTRUCTIVE 1600 00:55:59,589 --> 00:56:00,523 REASONS AND THEY USE THEM FOR 1601 00:56:00,590 --> 00:56:02,525 OTHER THAN CONSTRUCTIVE 1602 00:56:02,592 --> 00:56:05,528 PURPOSES, A HAMMER CAN POUND A 1603 00:56:05,595 --> 00:56:11,534 NAIL AND IT CAN POUND A HEAD, SO 1604 00:56:11,601 --> 00:56:14,537 I KIND OF RESIST BEING ACCUSED 1605 00:56:14,604 --> 00:56:19,542 OF ENABLE--WELL, I COULD BE 1606 00:56:19,609 --> 00:56:21,544 ACCUSED ALONG WITH MY COLLEAGUES 1607 00:56:21,611 --> 00:56:22,512 OF ENABLING SOME BAD BEHAVIORS BECAUSE HONESTLY 1608 00:56:22,579 --> 00:56:26,516 WE TRIED TO REDUCE THE BARRIER TO 1609 00:56:26,583 --> 00:56:27,517 ACCESS TO INFORMATION TO A MINIMUM. 1610 00:56:27,584 --> 00:56:29,519 THAT MEANT REDUCING THE BARRIER 1611 00:56:29,586 --> 00:56:31,521 FOR SHARING INFORMATION TO A 1612 00:56:31,588 --> 00:56:32,522 MINIMUM REDUCE THE BARRIER TO 1613 00:56:32,589 --> 00:56:34,524 FINDING INFORMATION TO A 1614 00:56:34,591 --> 00:56:36,526 MINIMUM, ALL OF WHICH HAS HAD 1615 00:56:36,593 --> 00:56:38,528 VERY POSITIVE BENEFIT BUT IT'S 1616 00:56:38,595 --> 00:56:40,530 ALSO CREATED AN OPPORTUNITY TO 1617 00:56:40,597 --> 00:56:42,532 SPREAD MISINFORMATION AND 1618 00:56:42,599 --> 00:56:46,536 DISINFORMATION BUT THE DECISION 1619 00:56:46,603 --> 00:56:48,538 TO DO THAT FOR THE MOST PART IS 1620 00:56:48,605 --> 00:56:49,539 SOMEONE'S DECISION, IT'S NOT THE 1621 00:56:49,606 --> 00:56:52,542 NETWORK DECIDING TO DO THAT. 1622 00:56:52,609 --> 00:56:55,512 SO I THINK WE HAVE TO HOLD 1623 00:56:55,578 --> 00:56:57,514 PARTIES RESPONSIBLE FOR BAD 1624 00:56:57,580 --> 00:56:58,515 BEHAVIOR WHICH TRANSLATES INTO 1625 00:56:58,581 --> 00:57:00,517 ACCOUNTABILITY AND I WOULD SAY 1626 00:57:00,583 --> 00:57:02,519 THAT WHAT IS MISSING FROM THE 1627 00:57:02,585 --> 00:57:05,522 ENVIRONMENT THAT WE'RE IN RIGHT 1628 00:57:05,588 --> 00:57:06,523 NOW IS A SUFFICIENT DEGREE OF 1629 00:57:06,589 --> 00:57:09,526 ACCOUNTABILITY FOR DECISIONS 1630 00:57:09,592 --> 00:57:11,528 THAT ARE MADE, AND IN ORDER TO 1631 00:57:11,594 --> 00:57:14,531 HAVE ACCOUNTABILITY, YOU HAVE TO 1632 00:57:14,597 --> 00:57:15,532 HAVE TO IDENTIFY, ABILITY TO 1633 00:57:15,598 --> 00:57:17,534 IDENTIFY. 1634 00:57:17,600 --> 00:57:18,535 NOW HERE'S WHERE WE GET INTO 1635 00:57:18,601 --> 00:57:19,536 REAL TENSION BECAUSE SOME PEOPLE 1636 00:57:19,602 --> 00:57:22,539 WILL SAY, MY PRIVACY IS 1637 00:57:22,605 --> 00:57:23,540 IMPORTANT TO ME, I DON'T WANT TO 1638 00:57:23,606 --> 00:57:25,542 BE KNOWN WHEN I'M USING THESE 1639 00:57:25,608 --> 00:57:27,544 TOOLS AND YOU CAN UNDERSTAND WHY 1640 00:57:27,610 --> 00:57:29,512 SOME PEOPLE MIGHT SAY THAT, AND 1641 00:57:29,579 --> 00:57:32,515 SOME PEOPLE ARE AT REAL RISK, IF 1642 00:57:32,582 --> 00:57:35,518 THEY WERE IDENTIFIABLE, BUT EVEN 1643 00:57:35,585 --> 00:57:38,521 IF YOU INVOKE THE WHISTLEBLOWER 1644 00:57:38,588 --> 00:57:41,524 SCENARIO WHICH IS A COMMON ONE TO 1645 00:57:41,591 --> 00:57:43,526 JUSTIFY PRIVACY AND ANONYMITY, 1646 00:57:43,593 --> 00:57:44,527 USUALLY THE WHISTLE BLOWER BLOWS 1647 00:57:44,594 --> 00:57:45,528 THE WHISTLE TO SOMEONE WHO HAS 1648 00:57:45,595 --> 00:57:48,531 TO FIGURE OUT WHO THEY ARE IN 1649 00:57:48,598 --> 00:57:50,533 ORDER TO KNOW WHETHER THEY'RE 1650 00:57:50,600 --> 00:57:53,536 CREDIBLE BUT WHO IS RESPONSIBLE 1651 00:57:53,603 --> 00:57:54,537 FOR PROTECTING THEIR IDENTITY. 1652 00:57:54,604 --> 00:57:55,538 REPORTERS GO TO JAIL SOMETIMES 1653 00:57:55,605 --> 00:57:57,540 IN THE PROTECTION OF THESE 1654 00:57:57,607 --> 00:58:00,543 SOURCES THAT THEY CHOOSE. 1655 00:58:00,610 --> 00:58:02,512 SO I THINK IN THE LONG RUN, 1656 00:58:02,579 --> 00:58:03,513 ABSOLUTE ANONYMITY IS NOT OUR 1657 00:58:03,580 --> 00:58:04,514 FRIEND IN THIS ONLINE 1658 00:58:04,581 --> 00:58:06,516 ENVIRONMENT AND WE HAVE TO HAVE 1659 00:58:06,583 --> 00:58:08,518 A WAY OF PIERCING THE VEIL UNDER 1660 00:58:08,585 --> 00:58:10,520 THE RIGHT CIRCUMSTANCES, WITH 1661 00:58:10,587 --> 00:58:13,523 THE RIGHT AUTHORITIES. 1662 00:58:13,590 --> 00:58:17,527 JUST LIKE LICENSE PLATES THAT 1663 00:58:17,594 --> 00:58:20,530 LOOK LIKE GOBBLEDY-GOOK TO US 1664 00:58:20,597 --> 00:58:21,531 BUT SOME PEOPLE HAVE THE 1665 00:58:21,598 --> 00:58:22,532 ABILITY, THE POLICE, HAVE THE ABILITY TO FIND OUT WHICH OWNER IS 1666 00:58:22,599 --> 00:58:23,533 ASSOCIATED WITH THAT LICENSE 1667 00:58:23,600 --> 00:58:24,534 PLATE. 1668 00:58:24,601 --> 00:58:25,535 THAT IS NOT NECESSARILY THE 1669 00:58:25,602 --> 00:58:26,536 DRIVER OF THE CAR THAT JUST WENT 1670 00:58:26,603 --> 00:58:28,538 120-MILES AN HOUR DOWN THE ROAD 1671 00:58:28,605 --> 00:58:29,539 BUT AT LEAST YOU IDENTIFY THE 1672 00:58:29,606 --> 00:58:30,540 OWNER OF THE CAR AND THAT'S A 1673 00:58:30,607 --> 00:58:33,543 START. 1674 00:58:33,610 --> 00:58:34,544 SO I THINK THESE ARE ALL THINGS 1675 00:58:34,611 --> 00:58:35,511 ACCOUNTABILITY IS 1676 00:58:35,578 --> 00:58:36,512 REALLY IMPORTANT NOW 1677 00:58:36,579 --> 00:58:38,514 THERE'S ONE OTHER THING THAT'S 1678 00:58:38,581 --> 00:58:40,516 EQUALLY IMPORTANT AND THAT'S 1679 00:58:40,583 --> 00:58:43,519 AGENCY IN THE SENSE THAT WE NEED 1680 00:58:43,586 --> 00:58:44,520 TO GIVE PEOPLE THE TOOLS TO 1681 00:58:44,587 --> 00:58:47,523 PROTECT THEMSELVES, TO PROTECT 1682 00:58:47,590 --> 00:58:48,524 THEIR PRIVACY, TO USE THESE 1683 00:58:48,591 --> 00:58:49,525 SYSTEMS WITHOUT REVEALING 1684 00:58:49,592 --> 00:58:52,528 ANYMORE THAN THEY ABSOLUTELY 1685 00:58:52,595 --> 00:58:53,529 HAVE TO IN ORDER TO LEGITIMATELY 1686 00:58:53,596 --> 00:58:54,530 USE IT. 1687 00:58:54,597 --> 00:58:55,531 LIKE SOMETIMES YOU HAVE TO PAY 1688 00:58:55,598 --> 00:58:57,533 FOR THINGS SO THAT MEANS YOU 1689 00:58:57,600 --> 00:58:58,534 HAVE TO SHARE YOUR CREDIT CARD 1690 00:58:58,601 --> 00:58:59,535 INFORMATION. 1691 00:58:59,602 --> 00:59:01,537 IF YOU DON'T WANT TO DO THAT, 1692 00:59:01,604 --> 00:59:04,540 YOU MAY NOT GET ACCESS TO WHAT 1693 00:59:04,607 --> 00:59:05,541 IT IS YOU'RE TRYING TO GET. 1694 00:59:05,608 --> 00:59:06,542 SO AGENCY AND ACCOUNTABILITY, I 1695 00:59:06,609 --> 00:59:10,513 THINK ARE GOING TO BE OUR 1696 00:59:10,580 --> 00:59:11,514 FRIENDS HERE AS WE TRY TO MOVE 1697 00:59:11,581 --> 00:59:12,515 FORWARD. 1698 00:59:12,582 --> 00:59:13,516 SO THIS TAKES THAT IN A 1699 00:59:13,583 --> 00:59:15,518 LITTLE BIT OF A DIFFERENT 1700 00:59:15,585 --> 00:59:18,521 DIRECTION BUT I THINK IT'S AN 1701 00:59:18,588 --> 00:59:24,527 IMPORTANT QUESTION, SO, WE HAVE 1702 00:59:24,594 --> 00:59:25,528 LAB TESTS, A LAB TEST, THERE'S A 1703 00:59:25,595 --> 00:59:28,531 GOVERNING APPROVING AGENCY THAT 1704 00:59:28,598 --> 00:59:29,532 SAYS THIS TEST IS SAFE, THIS 1705 00:59:29,599 --> 00:59:32,535 TEST IS REPRODUCABLE, IT PERFORMS 1706 00:59:32,602 --> 00:59:34,537 THE SAME ON SAMPLES EVERY TIME, 1707 00:59:34,604 --> 00:59:36,539 IT CAN SAFELY BE USED IN A 1708 00:59:36,606 --> 00:59:39,542 CLINICAL ENVIRONMENT TO MAKE 1709 00:59:39,609 --> 00:59:40,543 DECISIONS FOR PEOPLE, CLIA, 1710 00:59:40,610 --> 00:59:45,515 APPROVAL OF LABS, ET CETERA. 1711 00:59:45,581 --> 00:59:47,517 DO WE NEED TO THE SAME KIND OF 1712 00:59:47,583 --> 00:59:50,520 KNOWLEDGEABLE REGULATORY AGENCY 1713 00:59:50,586 --> 00:59:55,525 OR BODY FOR MACHINE LEARNING AND 1714 00:59:55,591 --> 00:59:56,526 AI BUILT TOOLS AND TECHNOLOGIES? 1715 00:59:56,592 --> 00:59:59,529 >> SO, I ACTUALLY WOULD SUGGEST 1716 00:59:59,595 --> 01:00:00,530 THAT WE SHOULD EXPAND RATHER 1717 01:00:00,596 --> 01:00:02,532 THAN CONTRACT HERE. 1718 01:00:02,598 --> 01:00:04,534 AND I WOULD SAY GENERALLY 1719 01:00:04,600 --> 01:00:06,536 SPEAKING FOR SOFTWARE, WHETHER 1720 01:00:06,602 --> 01:00:09,539 IT'S AI OR YOU KNOW MACHINE 1721 01:00:09,605 --> 01:00:13,543 LEARNING OR ANY OTHER KIND OF 1722 01:00:13,609 --> 01:00:15,511 SOFTWARE, THERE SHOULD BE 1723 01:00:15,578 --> 01:00:16,512 THOUGHTFUL ABILITY TO AUDIT THE 1724 01:00:16,579 --> 01:00:18,514 SAFETY AND SECURITY OF THAT 1725 01:00:18,581 --> 01:00:20,516 SOFTWARE, WHETHER IT WAS AI OR 1726 01:00:20,583 --> 01:00:22,518 MACHINE LEARNING OR ANYTHING 1727 01:00:22,585 --> 01:00:24,520 ELSE BECAUSE WE'RE SO DARN 1728 01:00:24,587 --> 01:00:25,521 RELIANT ON SOFTWARE NOW. 1729 01:00:25,588 --> 01:00:28,524 I MEAN THE PHONE IN YOUR POCKET, 1730 01:00:28,591 --> 01:00:30,526 THE MOBILE, WHEN IT DOESN'T 1731 01:00:30,593 --> 01:00:33,529 WORK, IT'S CASCADE FAILURE TIME, 1732 01:00:33,596 --> 01:00:35,531 ALL KINDS OF THINGS CAUSE YOU 1733 01:00:35,598 --> 01:00:38,534 TROUBLE, SO THE IDEA OF HAVING 1734 01:00:38,601 --> 01:00:40,536 STANDARDS FOR DEVELOPING 1735 01:00:40,603 --> 01:00:42,538 RELIABLE SAFE AND SECURE 1736 01:00:42,605 --> 01:00:44,540 SOFTWARE, INCLUDING AI AND 1737 01:00:44,607 --> 01:00:45,541 MACHINE LEARNING MAKES A LOT OF 1738 01:00:45,608 --> 01:00:49,512 SENSE. 1739 01:00:49,579 --> 01:00:55,518 NOW IN THE ACCOUNTING WORLD 1740 01:00:55,585 --> 01:00:56,519 THERE'S SOMETHING CALLED GENERAL ACCEPTED 1741 01:00:56,586 --> 01:00:58,521 ACCOUNTING PRINCIPLES, MAYBE WE NEED TO HAVE GENERALLY ACCEPTED SOFTWARE 1742 01:00:58,588 --> 01:01:00,523 PRINCIPLES WHERE 1743 01:01:00,590 --> 01:01:02,525 MAYBE GENERALLY ACCEPTED, IT'S 1744 01:01:02,592 --> 01:01:03,526 GAS, GENERALLY ACCEPTED AI 1745 01:01:03,593 --> 01:01:05,528 PRINCIPLES WHICH DOESN'T 1746 01:01:05,595 --> 01:01:07,530 PRONOUNCE WELL BUT I THINK WE 1747 01:01:07,597 --> 01:01:09,532 SHOULD BE LOOKING FOR PRINCIPLES 1748 01:01:09,599 --> 01:01:11,534 THAT WE CAN ALL ADOPT AND APPLY 1749 01:01:11,601 --> 01:01:13,536 IN THE DEVELOPMENT AND USE OF 1750 01:01:13,603 --> 01:01:20,543 THESE KINDS OF TECHNOLOGIES. 1751 01:01:20,610 --> 01:01:22,512 BUT I WANT TO EMPHASIZE 1752 01:01:22,578 --> 01:01:23,513 SOMETHING HERE. IT'S THE USE THAT'S IMPORTANT HERE. THE RISK 1753 01:01:23,579 --> 01:01:25,515 FACTORS HAVE TO DO WITH THE WAY THESE 1754 01:01:25,581 --> 01:01:28,518 TECHNOLOGIES ARE APPLIED SO IT'S 1755 01:01:28,584 --> 01:01:29,519 NOT THE SPECIFIC DETAIL OF THE 1756 01:01:29,585 --> 01:01:31,521 NEURAL NETWORK THAT IS AT ISSUE 1757 01:01:31,587 --> 01:01:36,526 HERE AS MUCH AS IT IS THE 1758 01:01:36,592 --> 01:01:37,527 DECISIONS TO USE THAT SYSTEM FOR 1759 01:01:37,593 --> 01:01:40,530 PARTICULAR APPLICATION. 1760 01:01:40,596 --> 01:01:41,531 SO IF I USE A CHATBOT, JUST FOR 1761 01:01:41,597 --> 01:01:44,534 ENTERTAINMENT, THAT'S A 1762 01:01:44,600 --> 01:01:46,536 RELATIVELY LOW RISK THING, 1763 01:01:46,602 --> 01:01:47,537 UNLESS YOU HAVE A BIG ARGUMENT 1764 01:01:47,603 --> 01:01:48,538 OVER INTELLECTUAL PROPERTY AND 1765 01:01:48,604 --> 01:01:50,540 WHERE DID THE INFORMATION COME 1766 01:01:50,606 --> 01:01:53,543 FROM FOR TRAINING THE MODEL THAT 1767 01:01:53,609 --> 01:01:54,544 WE CAN HAVE QUITE A LONG 1768 01:01:54,610 --> 01:01:56,512 CONVERSATION ABOUT THAT BUT WHEN 1769 01:01:56,579 --> 01:01:57,513 YOU START TO GET INTO 1770 01:01:57,580 --> 01:01:59,515 APPLICATIONS WHERE WE'RE SAYING, 1771 01:01:59,582 --> 01:02:00,516 WE'RE GOING TO USE THIS THING TO 1772 01:02:00,583 --> 01:02:04,520 GIVE YOU FINANCIAL ADVICE, OR 1773 01:02:04,587 --> 01:02:06,522 WE'RE GOING TO DO DIAGINOSIS OF 1774 01:02:06,589 --> 01:02:08,524 YOUR SYMPTOMS OR WE'RE GOING TO 1775 01:02:08,591 --> 01:02:11,527 MAKE RECOMMENDATIONS FOR 1776 01:02:11,594 --> 01:02:12,528 TREATMENT, BOY OH BOY ARE YOU 1777 01:02:12,595 --> 01:02:19,535 EVER IN HIGH RISK TERRITORY 1778 01:02:19,602 --> 01:02:21,537 THERE SO HOW DO YOU DEMONSTRATE 1779 01:02:21,604 --> 01:02:22,538 TO A BODY THAT'S GOING TO AUDIT 1780 01:02:22,605 --> 01:02:23,539 YOUR PRACTICE THAT HAVE YOU 1781 01:02:23,606 --> 01:02:26,542 TAKEN PRUDENT STEPS TO PROTECT 1782 01:02:26,609 --> 01:02:28,544 THE SAFETY AND INTEREST OF THE 1783 01:02:28,611 --> 01:02:30,513 PARTY WHO'S GOING TO BE 1784 01:02:30,580 --> 01:02:31,514 FOLLOWING THE ADVICE, THE 1785 01:02:31,581 --> 01:02:34,517 DIAGNOSIS AND THE TREATMENT. 1786 01:02:34,584 --> 01:02:37,520 SO WE NEED TO HAVE THAT KIND OF 1787 01:02:37,587 --> 01:02:40,523 AUDIT, I DON'T KNOW THAT WE ARE 1788 01:02:40,590 --> 01:02:43,526 ALREADY PREPARED TO LAY OUT WHAT 1789 01:02:43,593 --> 01:02:45,528 THOSE AUDIT TERMS SHOULD BE 1790 01:02:45,595 --> 01:02:48,531 BUT WE'VE HAD THE LONG TIME TO 1791 01:02:48,598 --> 01:02:49,532 DEVELOP THAT FOR ACCOUNTING. 1792 01:02:49,599 --> 01:02:52,535 MAYBE IT'S TIME TO GET STARTED 1793 01:02:52,602 --> 01:02:54,537 ON THE SAME THING FOR SOFTWARE 1794 01:02:54,604 --> 01:02:56,539 AND FOR MACHINE LEARNING AND FOR 1795 01:02:56,606 --> 01:03:00,543 LARGE LANGUAGE MODELS, AND 1796 01:03:00,610 --> 01:03:02,512 GENERALLY MACHINE LEARNING AND 1797 01:03:02,578 --> 01:03:04,514 AI, AND WE JUST NEED TO START 1798 01:03:04,580 --> 01:03:06,516 WORKING ON THAT. 1799 01:03:06,582 --> 01:03:08,518 I SUSPECT WE WILL NEED TO HAVE 1800 01:03:08,584 --> 01:03:10,520 SOME OVERSIGHT FOR THESE JUST 1801 01:03:10,586 --> 01:03:12,522 LIKE WE DO FOR EVERYTHING IF 1802 01:03:12,588 --> 01:03:13,523 THEY END UP HELPING US MAKE 1803 01:03:13,589 --> 01:03:14,524 CLINICAL DECISIONS. 1804 01:03:14,590 --> 01:03:17,527 THESE ARE LIFE AND DEATH 1805 01:03:17,593 --> 01:03:19,529 DECISIONS SO 1806 01:03:19,595 --> 01:03:20,530 WE NEED TO UNDERSTAND AND HAVE 1807 01:03:20,596 --> 01:03:22,532 SOME KIND OF QUALIFICATION 1808 01:03:22,598 --> 01:03:25,535 SYSTEM SO IT'S NOT THE WILD WEST 1809 01:03:25,601 --> 01:03:27,537 SO AN INTERESTING QUESTION IS 1810 01:03:27,603 --> 01:03:28,538 WHETHER WE HAVE ANY WORKED 1811 01:03:28,604 --> 01:03:29,539 EXAMPLES. 1812 01:03:29,605 --> 01:03:30,540 WE HAVE THE FEDERAL DRUG 1813 01:03:30,606 --> 01:03:31,541 ADMINITRATION FOR EXAMPLE, AND 1814 01:03:31,607 --> 01:03:37,513 THEY HAVE RULES ABOUT HOW WE 1815 01:03:37,580 --> 01:03:39,515 DECIDE WHETHER OR NOT A DRUG 1816 01:03:39,582 --> 01:03:41,517 TREATMENT IS SAFE, OR A TEST IS 1817 01:03:41,584 --> 01:03:42,518 ACCURATE OR NIST HELPS 1818 01:03:42,585 --> 01:03:44,520 US WITH SOME OF THOSE THING TOO BECAUSE THEY 1819 01:03:44,587 --> 01:03:46,522 HELP US CALIBRATE WHETHER THE 1820 01:03:46,589 --> 01:03:47,523 TESTS ARE PRODUCING RELIABLE 1821 01:03:47,590 --> 01:03:49,525 RESULTS, SO WE NEED SOMETHING 1822 01:03:49,592 --> 01:03:51,527 LIKE THAT, I DON'T KNOW IF IT 1823 01:03:51,594 --> 01:03:55,531 NEEDS TO BE CENTRALIZED EXACTLY 1824 01:03:55,598 --> 01:03:59,535 BUT WE DO HAVE TO HAVE A 1825 01:03:59,602 --> 01:04:01,537 COMMON SET OF AGREED PRACTICES 1826 01:04:01,604 --> 01:04:03,539 BECAUSE WE WOULD LIKE TO BE ABLE 1827 01:04:03,606 --> 01:04:04,540 TO RELY ON WHOEVER IS PROVIDING 1828 01:04:04,607 --> 01:04:07,543 THOSE PRODUCTS AND SERVICES TO 1829 01:04:07,610 --> 01:04:08,544 HAVE MET THE SAME CERTIFICATION 1830 01:04:08,611 --> 01:04:09,512 STANDARDS. 1831 01:04:09,579 --> 01:04:10,513 >> GREAT. 1832 01:04:10,580 --> 01:04:11,514 AND IF THESE THINGS ARE 1833 01:04:11,581 --> 01:04:17,520 REGULATED AS DEVICES IT WILL BE 1834 01:04:17,587 --> 01:04:19,522 A VERY DIFFERENT SITUATION--SOME 1835 01:04:19,589 --> 01:04:21,524 OF THE CHALLENGES WE HAVE IN THE 1836 01:04:21,591 --> 01:04:22,525 DEVICE WORLD IT HASN'T BEEN 1837 01:04:22,592 --> 01:04:23,526 SOLVED THERE. 1838 01:04:23,593 --> 01:04:25,528 >> AND THE INTERESTING THING 1839 01:04:25,595 --> 01:04:26,529 ABOUT DEVICES THESE DAYS IS THAT 1840 01:04:26,596 --> 01:04:28,531 MOST OF THEM HAVE A LOT OF 1841 01:04:28,598 --> 01:04:29,532 SOFTWARE IN THEM. 1842 01:04:29,599 --> 01:04:30,533 SO THAT'S WHY I GET NERVOUS 1843 01:04:30,600 --> 01:04:31,534 ABOUT SOFTWARE IN GENERAL IS 1844 01:04:31,601 --> 01:04:34,537 THAT WE REALLY NEED TO PAY 1845 01:04:34,604 --> 01:04:36,539 ATTENTION TO HOW DOES THIS STUFF 1846 01:04:36,606 --> 01:04:37,540 WORK AND WHAT IF IT DOESN'T? 1847 01:04:37,607 --> 01:04:42,511 WHAT ARE THE EDGE CASES THAT 1848 01:04:42,578 --> 01:04:44,513 HAVE TO BE CATERED TOO? 1849 01:04:44,580 --> 01:04:47,516 >> SO WE'RE OUT OF TIME. 1850 01:04:47,583 --> 01:04:51,520 WOW, THAT WAS JUST WONDERFUL. 1851 01:04:51,587 --> 01:04:52,521 >> GEEZ, THAT WENT FAST. 1852 01:04:52,588 --> 01:04:54,523 OOF. 1853 01:04:54,590 --> 01:04:56,525 >> CAN YOU BELIEVE THAT? 1854 01:04:56,592 --> 01:04:57,526 >> NOPE, THERE'S A TIME WARP IN 1855 01:04:57,593 --> 01:04:58,527 HERE. 1856 01:04:58,594 --> 01:05:00,529 >> BUT WE CAN'T END TALKING 1857 01:05:00,596 --> 01:05:01,530 ABOUT REGULATORY PROCEDURES, 1858 01:05:01,597 --> 01:05:02,531 THAT'S NOT FAIR. 1859 01:05:02,598 --> 01:05:04,533 SO I WILL GIVE THE FLOOR OVER TO 1860 01:05:04,600 --> 01:05:08,537 YOU ONE LAST--FIRST OF ALL I'M 1861 01:05:08,604 --> 01:05:09,538 GOING TO SAY THANK YOU FOR THIS 1862 01:05:09,605 --> 01:05:10,539 INCREDIBLE CONVERSATION, WE'RE 1863 01:05:10,606 --> 01:05:12,541 SO GRATEFUL YOU ARE HERE FOR 1864 01:05:12,608 --> 01:05:16,512 THIS VISIT, I COULDN'T THINK OF 1865 01:05:16,579 --> 01:05:18,514 A MORE SPECTACULAR RALL 1866 01:05:18,581 --> 01:05:20,516 CULTURAL LECTURE THAT YOU ARE 1867 01:05:20,583 --> 01:05:21,517 PROVIDING FOR ALL OF US. 1868 01:05:21,584 --> 01:05:22,518 >> WELL, THANK YOU. 1869 01:05:22,585 --> 01:05:23,519 >> THIS HAS BEEN SO THOUGHTFUL 1870 01:05:23,586 --> 01:05:25,521 AND INSPIRING BUT I WILL TURN IT 1871 01:05:25,588 --> 01:05:27,523 OVER TO YOU BECAUSE WE WOULD 1872 01:05:27,590 --> 01:05:30,526 LOVE TO HEAR IF YOU HAVE ANY 1873 01:05:30,593 --> 01:05:31,527 PARTING IDEAS FOR US? 1874 01:05:31,594 --> 01:05:33,529 >> WELL, THANK YOU FIRST OF ALL 1875 01:05:33,596 --> 01:05:34,530 FOR THE OPPORTUNITY TO HAVE THIS 1876 01:05:34,597 --> 01:05:36,532 CONVERSATION. 1877 01:05:36,599 --> 01:05:37,533 I WILL TAKE AWAY A BUNCH OF 1878 01:05:37,600 --> 01:05:38,534 IDEAS THAT I MIGHT NOT HAVE 1879 01:05:38,601 --> 01:05:42,538 OTHERWISE THOUGHT ABOUT. 1880 01:05:42,605 --> 01:05:43,539 SO I APPRECIATE THE OPPORTUNITY. 1881 01:05:43,606 --> 01:05:46,542 I HOPE THOSE OF YOU SITTING HERE 1882 01:05:46,609 --> 01:05:49,512 IN THE AUDIENCE RECOGNIZE WHAT 1883 01:05:49,578 --> 01:05:50,513 AN OPPORTUNITY NIH HAS TO 1884 01:05:50,579 --> 01:05:52,515 IMPROVE THE STATE OF HEALTHCARE, 1885 01:05:52,581 --> 01:05:53,516 NOT ONLY HERE IN THE U.S. BUT 1886 01:05:53,582 --> 01:05:56,519 ELSEWHERE, WITH THE KIND OF 1887 01:05:56,585 --> 01:05:59,522 RESEARCH THAT YOU'RE DOING AND 1888 01:05:59,588 --> 01:06:02,525 THE ATTEMPT TO EXERCISE THESE 1889 01:06:02,591 --> 01:06:03,526 POWERFUL COMPUTING TOOLS THAT 1890 01:06:03,592 --> 01:06:04,527 ARE EMERGING. 1891 01:06:04,593 --> 01:06:06,529 YOU KNOW PEOPLE, YOU ASKED 1892 01:06:06,595 --> 01:06:08,531 EARLIER, YOU KNOW WHAT WOULD I 1893 01:06:08,597 --> 01:06:10,533 HAVE SUGGESTED PEOPLE STUDYING 1894 01:06:10,599 --> 01:06:11,534 GO INTO, AND I SAID 1895 01:06:11,600 --> 01:06:13,536 ASTROPHYSICS BUT GENERALLY SPEAKING 1896 01:06:13,602 --> 01:06:14,537 SOMETHING THAT'S HAPPENED OVER 1897 01:06:14,603 --> 01:06:16,539 THE PAST 20 YEARS AND I WILL 1898 01:06:16,605 --> 01:06:19,542 CALL IT COMPUTATIONAL X FOR 1899 01:06:19,608 --> 01:06:24,513 EVERY VALUE OF X, YOU CAN THINK 1900 01:06:24,580 --> 01:06:25,514 OF LINGUISTICS, CHEMISTRY, 1901 01:06:25,581 --> 01:06:28,517 PHYSICS, ECONOMICS, WE ARE USING 1902 01:06:28,584 --> 01:06:29,518 COMPUTING TOOLS INCREASINGLY TO 1903 01:06:29,585 --> 01:06:30,519 BUILD MODELS, TO TEST THE 1904 01:06:30,586 --> 01:06:34,523 MODELS, TO TRY TO UNDERSTAND HOW 1905 01:06:34,590 --> 01:06:37,526 THE WORLD WORKS, MAKE 1906 01:06:37,593 --> 01:06:39,528 PREDICTIONS AND NIH IS INVESTED 1907 01:06:39,595 --> 01:06:42,531 HEAVILY IN TRYING TO DEVELOP 1908 01:06:42,598 --> 01:06:44,533 THOSE KINDS OF TOOLS AND YOU 1909 01:06:44,600 --> 01:06:46,535 HAVE AN OPPORTUNITY TO MAKE A 1910 01:06:46,602 --> 01:06:50,539 LOT OF PROGRESS, I THINK BECAUSE 1911 01:06:50,606 --> 01:06:53,542 YOU'RE THE SOURCE OF A FABULOUS 1912 01:06:53,609 --> 01:06:55,544 RESEARCH, GREAT DATA AND NOW, 1913 01:06:55,611 --> 01:06:56,512 ALSO POTENTIAL FOR USING 1914 01:06:56,579 --> 01:07:00,516 COMPUTER BASED TOOLS TO MAKE 1915 01:07:00,583 --> 01:07:01,517 YOUR PREDICTIONS EVEN MORE 1916 01:07:01,584 --> 01:07:02,518 PRECISE. 1917 01:07:02,585 --> 01:07:04,520 SO IF I WERE IN YOUR SHOES, I 1918 01:07:04,587 --> 01:07:08,524 WOULD BE SITTING HERE THINKING, 1919 01:07:08,591 --> 01:07:11,527 BOY, DO WE HAVE A LOT OF WORK TO 1920 01:07:11,594 --> 01:07:12,528 DO, AND A HUGE AMOUNT OF 1921 01:07:12,595 --> 01:07:16,532 OPPORTUNITY TO DO GOOD, SO THANK 1922 01:07:16,599 --> 01:07:20,536 YOU FOR RUNNING THE SHOW. 1923 01:07:20,603 --> 01:07:24,540 >> THANK YOU. 1924 01:07:24,607 --> 01:07:28,544 >> THANK YOU. 1925 01:07:28,611 --> 01:07:29,512 [ APPLAUSE ] 1926 01:07:29,578 --> 01:07:30,579 [APPLAUSE ]