1
00:00:04,822 --> 00:00:06,057
ALL RIGHT.

2
00:00:06,057 --> 00:00:06,557
WELCOME, EVERYONE.

3
00:00:06,557 --> 00:00:10,228
WE'LL GO AHEAD AND GET STARTED.

4
00:00:10,228 --> 00:00:12,296
SO, WELCOME TO THE NIH 

5
00:00:12,296 --> 00:00:13,998
DIRECTOR'S LECTURE.

6
00:00:13,998 --> 00:00:16,934
THIS IS A VERY SPECIAL LECTURE 

7
00:00:16,934 --> 00:00:19,403
AND IT FEATURES LEADING 

8
00:00:19,403 --> 00:00:20,137
RESEARCHERS FROM AROUND THE 

9
00:00:20,137 --> 00:00:21,172
GLOBE.

10
00:00:21,172 --> 00:00:23,875
NOMINATIONS ARE TAKEN FROM 

11
00:00:23,875 --> 00:00:28,079
ACROSS NIH AND TYPICALLY ONLY 

12
00:00:28,079 --> 00:00:33,718
ABOUT THREE DIRECTOR'S LECTURERS

13
00:00:33,718 --> 00:00:35,887
ARE CHOSEN EACH YEAR AND IT'S 

14
00:00:35,887 --> 00:00:38,189
OUR HONOR TO WELCOME EDDIE CHANG

15
00:00:38,189 --> 00:00:40,191
TODAY TO SPEND THIS TIME WITH 

16
00:00:40,191 --> 00:00:40,424
US.

17
00:00:40,424 --> 00:00:45,630
I'M PERSONALLY VERY GRATEFUL.

18
00:00:45,630 --> 00:00:48,933
EDDIE IS SO WELL KNOWN TO US I 

19
00:00:48,933 --> 00:00:51,569
WAS THINKING IN NEUROSCIENCE 

20
00:00:51,569 --> 00:00:54,005
CIRCLES AT NIH YOU JUST SAY 

21
00:00:54,005 --> 00:00:55,306
EDDIE AND EVERYONE KNOWS WHO YOU

22
00:00:55,306 --> 00:01:00,311
ARE SO IT'S LIKE MADONNA TO US.

23
00:01:00,311 --> 00:01:03,748
BEFORE I INTRODUCE, DR. CHANG, A

24
00:01:03,748 --> 00:01:06,083
FEW HOUSEKEEPING ITEMS.

25
00:01:06,083 --> 00:01:09,287
IF YOU'RE ATTENDING THIS LECTURE

26
00:01:09,287 --> 00:01:14,025
FOR CME CREDIT THE CODE IS 

27
00:01:14,025 --> 00:01:14,425
57680.

28
00:01:14,425 --> 00:01:15,593
IF YOU'RE WATCHING ONLINE AND 

29
00:01:15,593 --> 00:01:17,929
WOULD LIKE TO SUBMIT QUESTIONS, 

30
00:01:17,929 --> 00:01:20,264
PLEASE CLICK THE LINK FOR SEND 

31
00:01:20,264 --> 00:01:21,599
LIVE FEEDBACK AND YOUR QUESTION 

32
00:01:21,599 --> 00:01:24,902
WILL BE RELAYED TO US AND WE'LL 

33
00:01:24,902 --> 00:01:26,037
PRESENT THAT TO THE SPEAKER AT 

34
00:01:26,037 --> 00:01:27,505
THE END OF THE LECTURE.

35
00:01:27,505 --> 00:01:29,173
AND YOU CAN SUBMIT QUESTIONS AT 

36
00:01:29,173 --> 00:01:32,843
ANY TIME DURING THE LECTURE.

37
00:01:32,843 --> 00:01:38,215
SO, DR. CHANG IS THE CHAIR AND 

38
00:01:38,215 --> 00:01:39,951
JEANIE ROBERTSON DISTINGUISHED 

39
00:01:39,951 --> 00:01:41,352
PROFESSOR OF NEUROLOGICAL 

40
00:01:41,352 --> 00:01:44,055
SURGERY AT THE UNIVERSITY OF 

41
00:01:44,055 --> 00:01:45,323
CALIFORNIA IN SAN FRANCISCO.

42
00:01:45,323 --> 00:01:47,992
HIS CLINICAL EXPERTISE IS IN 

43
00:01:47,992 --> 00:01:49,627
SURGICAL THERAPIES FOR EPILEPSY,

44
00:01:49,627 --> 00:01:51,662
PAIN AND BRAIN TUMORS.

45
00:01:51,662 --> 00:01:56,801
HE SPECIALIZES IN ADVANCED 

46
00:01:56,801 --> 00:01:59,036
NEUROPHYSIOLOGIC BRAIN MAPPING 

47
00:01:59,036 --> 00:02:02,473
METHODS AND AWAKE SPEECH AND 

48
00:02:02,473 --> 00:02:05,142
MOTOR MAPPING TO SAFELY PERFORM 

49
00:02:05,142 --> 00:02:07,578
PROCEDURES IN THE ELOQUENT OF 

50
00:02:07,578 --> 00:02:08,446
THE BRAINS.

51
00:02:08,446 --> 00:02:10,982
HE'S THE CONSUMMATE SURGEON 

52
00:02:10,982 --> 00:02:11,315
SCIENTIST.

53
00:02:11,315 --> 00:02:13,250
HE'S A PRACTICING SURGEON.

54
00:02:13,250 --> 00:02:16,387
HE CHAIRS THE DEPARTMENT AND HE 

55
00:02:16,387 --> 00:02:18,422
DOES ALL OF THIS INCREDIBLE 

56
00:02:18,422 --> 00:02:20,257
RESEARCH THAT YOU'LL HEAR ABOUT 

57
00:02:20,257 --> 00:02:20,491
SHORTLY.

58
00:02:20,491 --> 00:02:23,327
HIS LAB HAS DEMONSTRATED THE 

59
00:02:23,327 --> 00:02:24,362
DETAILED FUNCTIONAL ORGANIZATION

60
00:02:24,362 --> 00:02:26,998
OF THE HUMAN SPEECH CORTEX AND 

61
00:02:26,998 --> 00:02:29,600
HAS TRANSLATED THESE DISCOVERIES

62
00:02:29,600 --> 00:02:32,370
TOWARDS THE DEVELOPMENT MUCH A 

63
00:02:32,370 --> 00:02:34,505
SPEECH PROSTHETIC DEVICE TO 

64
00:02:34,505 --> 00:02:35,172
RESTORE COMMUNICATION FOR PEOPLE

65
00:02:35,172 --> 00:02:37,274
LIVING WITH PARALYSIS AND SPEECH

66
00:02:37,274 --> 00:02:37,541
DISORDERS.

67
00:02:37,541 --> 00:02:40,945
 HE DIRECTS OR CO-DIRECTS THE 

68
00:02:40,945 --> 00:02:43,814
CENTER FOR NEURAL ENGINEERING 

69
00:02:43,814 --> 00:02:47,618
AND PROSTHESIS AND THAT'S A 

70
00:02:47,618 --> 00:02:51,689
COLLABORATIVE ENTERPRISE OF UCSF

71
00:02:51,689 --> 00:02:55,860
AND BERKELEY AND BRINGS TOGETHER

72
00:02:55,860 --> 00:02:58,863
EXPERTS TO DEVELOP STATE OF THE 

73
00:02:58,863 --> 00:03:07,505
ART BIO MEDICAL TECHNOLOGIES FOR

74
00:03:07,505 --> 00:03:08,539
SPEECH DISORDERS.

75
00:03:08,539 --> 00:03:10,307
HE'LL DESCRIBE CRITICAL ADVANCES

76
00:03:10,307 --> 00:03:13,678
IN THE PAST DECADE MADE BY HIS 

77
00:03:13,678 --> 00:03:18,749
LAB IN DECIPHERING THE CORTICAL 

78
00:03:18,749 --> 00:03:20,217
SIGNALS THAT UNDER LIE OUR 

79
00:03:20,217 --> 00:03:21,085
ABILITY TO SPEAK AND UNDERSTAND 

80
00:03:21,085 --> 00:03:22,053
WORDS.

81
00:03:22,053 --> 00:03:23,688
HE'S THE RECIPIENT OF MANY 

82
00:03:23,688 --> 00:03:26,157
AWARDS AND WAS ELECTED TO THE 

83
00:03:26,157 --> 00:03:26,791
NATIONAL ACADEMY OF MEDICINE IN 

84
00:03:26,791 --> 00:03:30,895
2020.

85
00:03:30,895 --> 00:03:32,930
I AM SO EXCITED TO WELCOME 

86
00:03:32,930 --> 00:03:33,664
DR. CHANG TODAY.

87
00:03:33,664 --> 00:03:43,441
WE LOOK FORWARD TO YOUR TALK.

88
00:03:43,441 --> 00:03:45,676
>> THANK YOU FOR THE 

89
00:03:45,676 --> 00:03:46,777
INTRODUCTION AND TO THANK 

90
00:03:46,777 --> 00:03:48,145
DR. SHOER FOR THE HONOR TO BE 

91
00:03:48,145 --> 00:03:48,479
HERE.

92
00:03:48,479 --> 00:03:50,915
IT'S A PRIVILEGE TO BE AT THE 

93
00:03:50,915 --> 00:03:51,182
NIH.

94
00:03:51,182 --> 00:03:55,820
I MEAN THAT FROM THE DEEPEST 

95
00:03:55,820 --> 00:03:56,854
PART OF MY HEART AND WANT TO 

96
00:03:56,854 --> 00:03:58,856
THANK YOU FOR THE WORK YOU DO IN

97
00:03:58,856 --> 00:04:01,559
PARTICULAR FOR THE COLLABORATION

98
00:04:01,559 --> 00:04:06,030
WE'VE HAD OVER THE LAST TWO 

99
00:04:06,030 --> 00:04:06,263
DECADES.

100
00:04:06,263 --> 00:04:08,599
WE'RE JUST GOING TO SWITCH THE 

101
00:04:08,599 --> 00:04:18,542
SLIDES HERE.

102
00:04:18,542 --> 00:04:20,177
THE FOCUS OF THE LECTURE IS PART

103
00:04:20,177 --> 00:04:22,580
OF WHAT MY LAB STUDIES.

104
00:04:22,580 --> 00:04:24,949
WE STUDY HOW THE BRAIN PROCESSES

105
00:04:24,949 --> 00:04:27,218
WHAT WE HEAR AND HOW WE CREATE 

106
00:04:27,218 --> 00:04:27,451
SPEECH.

107
00:04:27,451 --> 00:04:29,620
I'M FOCUS ON THE LATTER.

108
00:04:29,620 --> 00:04:31,222
THE PROCESS OF THE CORTICAL 

109
00:04:31,222 --> 00:04:32,523
CONTROL OF SPEAKING.

110
00:04:32,523 --> 00:04:34,425
HOW WE USE THAT KNOWLEDGE TO 

111
00:04:34,425 --> 00:04:36,393
BUILD A TRANSLATIONAL DEVICE FOR

112
00:04:36,393 --> 00:04:38,929
PEOPLE WHO HAVE LOST THE ABILITY

113
00:04:38,929 --> 00:04:40,965
TO COMMUNICATE.

114
00:04:40,965 --> 00:04:42,500
ONE PERSPECTIVE THAT IS REALLY 

115
00:04:42,500 --> 00:04:45,669
IMPORTANT TO UNDERSTANDING HOW 

116
00:04:45,669 --> 00:04:47,004
WE SPEAK IS THE FUNDAMENTAL 

117
00:04:47,004 --> 00:04:50,341
PROCESS OF SPEAKING IS ABOUT HOW

118
00:04:50,341 --> 00:04:54,211
WE SHAPE THE BREATH.

119
00:04:54,211 --> 00:04:57,348
SPEECH STARTS WITH THE AIR FROM 

120
00:04:57,348 --> 00:04:57,815
THE LUNGS.

121
00:04:57,815 --> 00:05:00,050
WHEN IT GETS TO THE LEVEL OF 

122
00:05:00,050 --> 00:05:01,485
YOUR VOCAL FOLDS IT STARTS 

123
00:05:01,485 --> 00:05:04,822
VIBRATIONS THERE AND THAT IS THE

124
00:05:04,822 --> 00:05:06,791
VOICE ENERGY OF OUR WORDS.

125
00:05:06,791 --> 00:05:08,726
THAT VOICE ENERGY THEN GOES UP 

126
00:05:08,726 --> 00:05:10,327
THROUGH THE UPPER PART OF THE 

127
00:05:10,327 --> 00:05:12,329
VOCAL TRAP, THE LIPS, THE JAW, 

128
00:05:12,329 --> 00:05:14,765
THE TONGUE WHERE THE MOVEMENTS 

129
00:05:14,765 --> 00:05:17,635
CREATE A FILTER ON THAT SOUND 

130
00:05:17,635 --> 00:05:19,870
AND THAT WHAT GIVES RISE TO ALL 

131
00:05:19,870 --> 00:05:20,504
THE CONSONANTS AND VOWELS THAT 

132
00:05:20,504 --> 00:05:22,940
WE SPEAK.

133
00:05:22,940 --> 00:05:23,841
SO, A LOT OF WHAT I'M GOING TO 

134
00:05:23,841 --> 00:05:27,978
BE FOCUSSING ON IS THE CONTROL 

135
00:05:27,978 --> 00:05:28,979
OF THE VOCAL TRACK.

136
00:05:28,979 --> 00:05:31,115
HOW DOES THE BRAIN ALLOW US TO 

137
00:05:31,115 --> 00:05:33,083
CONTROL THAT ABILITY TO SHAPE 

138
00:05:33,083 --> 00:05:35,352
THE BREATH AND IT'S REALLY A 

139
00:05:35,352 --> 00:05:38,055
REMARKABLE PROCESS ONE UNIQUE TO

140
00:05:38,055 --> 00:05:41,125
OUR SPECIES TO CREATE WORDS AND 

141
00:05:41,125 --> 00:05:44,328
SOMETHING QUITE SPECIALIZED.

142
00:05:44,328 --> 00:05:47,031
WHEN I STARTED THIS WORK, IT WAS

143
00:05:47,031 --> 00:05:51,735
ABOUT 15 YEARS AGO STARTED IN MY

144
00:05:51,735 --> 00:05:52,236
LAB.

145
00:05:52,236 --> 00:05:53,637
IT BECAME IMMEDIATELY OBVIOUS TO

146
00:05:53,637 --> 00:05:55,306
ME WE DID NOT UNDERSTAND VERY 

147
00:05:55,306 --> 00:05:57,541
MUCH ABOUT THE ORGANIZATION OR 

148
00:05:57,541 --> 00:06:00,044
THE BASIC PRINCIPLES BY WHICH 

149
00:06:00,044 --> 00:06:01,078
THE HUMAN BRAIN PROCESSES WORDS.

150
00:06:01,078 --> 00:06:02,746
SO IN ADDITION TO STUDYING HOW 

151
00:06:02,746 --> 00:06:05,015
WE HEAR, WE DECIDED TO DO 

152
00:06:05,015 --> 00:06:06,584
STUDIES THAT WERE GOING TO BE 

153
00:06:06,584 --> 00:06:07,918
DIRECTLY RELATED TO HOW WE 

154
00:06:07,918 --> 00:06:10,254
PRODUCE AND ARTICULATE.

155
00:06:10,254 --> 00:06:14,225
AND 99% OF MOTOR NEUROSCIENCE IS

156
00:06:14,225 --> 00:06:17,261
REALLY FOCUSSED ON THE ARMS AND 

157
00:06:17,261 --> 00:06:18,896
LIMBS, THE ARM AND HAND.

158
00:06:18,896 --> 00:06:20,130
THE VAST MAJORITY.

159
00:06:20,130 --> 00:06:23,167
BUT HOW MUCH WE ACTUALLY STUDY 

160
00:06:23,167 --> 00:06:24,768
THE CENTRAL CONTROL OF SPEAKING 

161
00:06:24,768 --> 00:06:26,637
IS VERY UNDER STUDIED.

162
00:06:26,637 --> 00:06:32,276
SO THIS IMAGE IS ACTUALLY AN 

163
00:06:32,276 --> 00:06:34,545
ILLUSTRATED FROM HARVEY CUSHING 

164
00:06:34,545 --> 00:06:36,780
CONSIDERED THE FATHER OF 

165
00:06:36,780 --> 00:06:37,114
NEUROSURGERY.

166
00:06:37,114 --> 00:06:38,782
A STUDENT GAVE ME THIS 

167
00:06:38,782 --> 00:06:39,917
ILLUSTRATION WHICH HANGS IN THE 

168
00:06:39,917 --> 00:06:40,885
WALL IN MY LAB.

169
00:06:40,885 --> 00:06:42,253
HE WAS DESCRIBING A PATIENT THAT

170
00:06:42,253 --> 00:06:46,323
WAS SHOT IN THE HEAD WITH A 

171
00:06:46,323 --> 00:06:48,959
BULLET ENTRY IN THE FOREHEAD AND

172
00:06:48,959 --> 00:06:50,694
DEVELOPED SEIZURES FROM PART OF 

173
00:06:50,694 --> 00:06:51,562
THE TEMPORAL LOBE.

174
00:06:51,562 --> 00:06:53,430
THE REASON I'M SHOWING THIS IS 

175
00:06:53,430 --> 00:07:02,506
NOT RELATED TO HIS INJURY BUT 

176
00:07:02,506 --> 00:07:03,407
IT'S ONE OF THE FIRST 

177
00:07:03,407 --> 00:07:06,010
DESCRIPTIONS OF THE BODY 

178
00:07:06,010 --> 00:07:09,713
ORGANIZATION OR SOMATIC TOPIC OF

179
00:07:09,713 --> 00:07:20,257
THE HUE  PART OF THE BRAIN THAT 

180
00:07:26,864 --> 00:07:28,933
CORRESPONDS TO MOVEMENT OF THE 

181
00:07:28,933 --> 00:07:31,635
FACE, CORNER OF THE LIPS, THE 

182
00:07:31,635 --> 00:07:34,271
JAW, TONGUE, PALLET AND BACK OF 

183
00:07:34,271 --> 00:07:35,973
OUR THROAT AND VOCAL FOLDS.

184
00:07:35,973 --> 00:07:38,142
SO BASICALLY THERE'S A MAP IN 

185
00:07:38,142 --> 00:07:40,678
ALL OF OUR BRAINS THAT 

186
00:07:40,678 --> 00:07:41,879
RECAPITULATES THE SPATIAL 

187
00:07:41,879 --> 00:07:46,116
ORGANIZATION FROM FRONT TO BACK 

188
00:07:46,116 --> 00:07:46,583
OF THE VOCAL TRACK.

189
00:07:46,583 --> 00:07:57,061
THAT WAS THE STARTING POINT.

190
00:07:57,828 --> 00:07:59,897
THIS WAS IN THE CONTEXT OF 

191
00:07:59,897 --> 00:08:00,931
MOVING THE AREAS AND I WAS 

192
00:08:00,931 --> 00:08:03,000
INTERESTED IN HOW TO RELATE THE 

193
00:08:03,000 --> 00:08:04,935
PHENOMENON IN UNDERSTANDING 

194
00:08:04,935 --> 00:08:06,370
SPEECH.

195
00:08:06,370 --> 00:08:09,340
SO THE MAIN TECHNIQUE THAT WE 

196
00:08:09,340 --> 00:08:11,308
USE WAS A TECHNIQUE AND WE STILL

197
00:08:11,308 --> 00:08:20,784
USE THIS CALLED ELECTROCO

198
00:08:20,784 --> 00:08:21,452
ELECTROCORTIOGRAPHY AND WE 

199
00:08:21,452 --> 00:08:23,454
INCREASED THE DENSITY OF THE 

200
00:08:23,454 --> 00:08:24,955
RAYS ON THE SURFACE OF THE 

201
00:08:24,955 --> 00:08:27,725
BRAINS ALLOWING US TO ASK DEEPER

202
00:08:27,725 --> 00:08:31,028
QUESTIONS ABOUT THE NEURAL 

203
00:08:31,028 --> 00:08:32,796
CODING WITHOUT INTRODUCING 

204
00:08:32,796 --> 00:08:33,330
ADDITIONAL RISK.

205
00:08:33,330 --> 00:08:36,967
MORE ELECTRODES BUT ON THE 

206
00:08:36,967 --> 00:08:40,237
SURFACE MEANT MORE DATA AND THIS

207
00:08:40,237 --> 00:08:45,776
IS A 16X16, ARRAY.

208
00:08:45,776 --> 00:08:47,578
STILL IS THE MOST HIGHLY 

209
00:08:47,578 --> 00:08:50,681
RESOLVED METHOD IN NEUROSCIENCE 

210
00:08:50,681 --> 00:08:54,785
IN SPATIAL AND SIMULTANEOUSLY 

211
00:08:54,785 --> 00:08:55,786
TEMPORAL RESOLUTION TO 

212
00:08:55,786 --> 00:08:59,490
UNDERSTANDING WHAT IS HAPPENING 

213
00:08:59,490 --> 00:09:02,292
IN THE CORTEX AND YOU'LL SEE THE

214
00:09:02,292 --> 00:09:04,194
PRE AND POST CENTRAL GYRUS WHEN 

215
00:09:04,194 --> 00:09:04,928
SOMEONE IS SPEAKING.

216
00:09:04,928 --> 00:09:07,431
>> SHIP BUILDING IS A 

217
00:09:07,431 --> 00:09:07,865
FASCINATING PROCESS.

218
00:09:07,865 --> 00:09:09,767
>> SO THAT PATTERN OF
ACTIVITY 

219
00:09:09,767 --> 00:09:12,936
IS WHAT WE REFER TO IS THE 

220
00:09:12,936 --> 00:09:13,771
NEURAL CODE.

221
00:09:13,771 --> 00:09:16,673
AND WHAT WE HAVE DEEPLY, DEEPLY 

222
00:09:16,673 --> 00:09:18,642
FOCUSSED ON FOR THE LAST 15 

223
00:09:18,642 --> 00:09:20,878
YEARS IS TRYING TO UNDERSTAND 

224
00:09:20,878 --> 00:09:23,981
HOW THAT NEURAL CODE WORKS.

225
00:09:23,981 --> 00:09:26,650
HOW DOES ELECTRICAL ACTIVITY 

226
00:09:26,650 --> 00:09:28,252
REFLECTED IN WHAT YOU JUST SAW 

227
00:09:28,252 --> 00:09:30,554
IN THAT PARTICULAR PART OF THE 

228
00:09:30,554 --> 00:09:30,754
BRAIN.

229
00:09:30,754 --> 00:09:32,156
HOW DOES THAT CORRESPOND TO 

230
00:09:32,156 --> 00:09:33,824
DIFFERENT ASPECTS OF SPEECH 

231
00:09:33,824 --> 00:09:34,058
CONTROL.

232
00:09:34,058 --> 00:09:35,592
I'M GOING TO TALK A LITTLE BIT 

233
00:09:35,592 --> 00:09:37,628
ABOUT HOW WE DECIPHERED THAT 

234
00:09:37,628 --> 00:09:38,929
CODE, WHAT ARE THE THINGS WE 

235
00:09:38,929 --> 00:09:41,765
LEARNED FROM IT AND THEN HOW WE 

236
00:09:41,765 --> 00:09:44,902
LEVERAGE THAT CODE FOR SOMETHING

237
00:09:44,902 --> 00:09:55,145
TRANSLATIONAL.

238
00:09:56,613 --> 00:09:59,716
WE HAD PARTICIPANTS READING A 

239
00:09:59,716 --> 00:10:02,252
LIST OF SYLLABLES, KIND OF 

240
00:10:02,252 --> 00:10:02,586
BORING, RIGHT?

241
00:10:02,586 --> 00:10:12,796
VERY SIMPLE.

242
00:10:15,599 --> 00:10:17,634
IF YOU LOOK AT THE SYLLABLES 

243
00:10:17,634 --> 00:10:21,672
WHEN YOU MAKE THE BA SOUND YOUR 

244
00:10:21,672 --> 00:10:26,143
LIPS COME TO CLOSURE THAT CREATE

245
00:10:26,143 --> 00:10:29,980
THE SOUND THE TOP TIP FROM THE 

246
00:10:29,980 --> 00:10:33,584
UPPER TOOTH THAT CREATES THE BA 

247
00:10:33,584 --> 00:10:35,986
SOUND AND THE GA SOUND IS THE 

248
00:10:35,986 --> 00:10:38,355
BACK OF THE TONGUE GOING UP.

249
00:10:38,355 --> 00:10:39,957
THAT'S QUITE INCREDIBLE.

250
00:10:39,957 --> 00:10:42,126
WHEN I LEARNED ALL THIS I WAS 

251
00:10:42,126 --> 00:10:42,392
ASTOUNDED.

252
00:10:42,392 --> 00:10:44,928
I HAD NO IDEA HOW WE WERE SPEAK 

253
00:10:44,928 --> 00:10:48,932
AND THIS IS SOMETHING WE ALL 

254
00:10:48,932 --> 00:10:51,001
LEARN WITHOUT EXPLICIT KNOWLEDGE

255
00:10:51,001 --> 00:10:53,036
WHICH IS QUITE INCREDIBLE.

256
00:10:53,036 --> 00:10:55,205
NONETHELESS, THIS IS HOW WE 

257
00:10:55,205 --> 00:10:56,206
PRODUCE THE SOUNDS AND IF YOU 

258
00:10:56,206 --> 00:10:58,709
LOOK AT THE ELECTRODES SORTED 

259
00:10:58,709 --> 00:11:00,644
FROM TOP TO BOTTOM HERE, THESE 

260
00:11:00,644 --> 00:11:03,080
ARE THE SAME SET OF ELECTRODES 

261
00:11:03,080 --> 00:11:05,749
ACROSS THE THREE UTTERANCES.

262
00:11:05,749 --> 00:11:07,518
YOU CAN SEE THEY ARE HAVE 

263
00:11:07,518 --> 00:11:08,852
OVERLAPPING AND DISTINCT 

264
00:11:08,852 --> 00:11:10,587
PATTERNS OF ACTIVATION.

265
00:11:10,587 --> 00:11:12,623
FOR EXAMPLE, IN THE BA SOUND, 

266
00:11:12,623 --> 00:11:15,125
THE B SOUND YOU SEE THE 

267
00:11:15,125 --> 00:11:17,327
ELECTRODES ACTIVATED HERE BUT 

268
00:11:17,327 --> 00:11:20,664
NOT SO MUCH IN THE "D" AND GA 

269
00:11:20,664 --> 00:11:22,232
SOUND WHERE IN THIS SOUND THE 

270
00:11:22,232 --> 00:11:23,800
SOUND IS ACTIVATED BUT NOT 

271
00:11:23,800 --> 00:11:27,137
OTHERS AND THIS GA SOUND HAS AN 

272
00:11:27,137 --> 00:11:27,604
ACTIVATION DOWN HERE.

273
00:11:27,604 --> 00:11:29,106
AND THEN THERE'S SOME THAT ARE 

274
00:11:29,106 --> 00:11:32,042
OVERLAPPING ACROSS ALL OF THESE.

275
00:11:32,042 --> 00:11:33,577
SO IT'S NOT AS SIMPLE AS THIS 

276
00:11:33,577 --> 00:11:36,980
ONE AREA IS ONE SYLLABLE, 

277
00:11:36,980 --> 00:11:37,948
ETCETERA.

278
00:11:37,948 --> 00:11:40,551
THERE'S A COMPLEX REPRESENTATION

279
00:11:40,551 --> 00:11:42,452
THAT HAS DISTRIBUTED 

280
00:11:42,452 --> 00:11:42,753
INFORMATION.

281
00:11:42,753 --> 00:11:44,588
SO ONE WAY THAT WE TOOK TO TRY 

282
00:11:44,588 --> 00:11:47,124
TO UNDERSTAND THIS WAS TO TAKE 

283
00:11:47,124 --> 00:11:49,126
ALL THE DIFFERENT SOUNDS, FOR 

284
00:11:49,126 --> 00:11:51,361
EXAMPLE, ALL THE LIP SOUNDS LIKE

285
00:11:51,361 --> 00:11:54,398
THE BA, FA, MA, PA, THEY'RE WHAT

286
00:11:54,398 --> 00:11:56,033
WE CALL LABIALS.

287
00:11:56,033 --> 00:11:56,934
WE'RE ABLE TO CORRELATE THOSE 

288
00:11:56,934 --> 00:12:00,237
WITH THE ELECTRODES ACROSS 

289
00:12:00,237 --> 00:12:02,239
PATIENTS THAT ARE CORRELATED 

290
00:12:02,239 --> 00:12:04,007
WITH THAT MOVEMENT.

291
00:12:04,007 --> 00:12:06,577
SO, WHAT THIS IS IS BASICALLY A 

292
00:12:06,577 --> 00:12:16,887
SPATIAL PLOT FROM TOP TO BOTTOM 

293
00:12:16,887 --> 00:12:17,821
AND THESE ARE ASSOCIATED WITH 

294
00:12:17,821 --> 00:12:20,090
THE LABIAL MOVEMENTS WHILE 

295
00:12:20,090 --> 00:12:21,124
PEOPLE ARE SPEAKING.

296
00:12:21,124 --> 00:12:22,793
THEN IF YOU LOOK AT A DIFFERENT 

297
00:12:22,793 --> 00:12:27,497
SET OF ARTICULATORS, ARTICULATED

298
00:12:27,497 --> 00:12:30,701
SOUND FOR THE LARYNX WE FOUND 

299
00:12:30,701 --> 00:12:32,703
ONE AT THE BOTTOM WHICH I SHOWED

300
00:12:32,703 --> 00:12:36,607
YOU EARLIER IN THE CUSHING 

301
00:12:36,607 --> 00:12:36,940
ILLUSTRATION.

302
00:12:36,940 --> 00:12:43,380
THERE'S THIS VOCAL FOLD AT THE 

303
00:12:43,380 --> 00:12:47,618
HOMUNCULUS AND THEN FOUND AN 

304
00:12:47,618 --> 00:12:48,952
ADDITIONAL LARYNGEAL AREA THAT 

305
00:12:48,952 --> 00:12:50,053
WAS HARD TO UNDERSTAND.

306
00:12:50,053 --> 00:12:53,657
WE DID MULTIPLE STUDIES TO TRY 

307
00:12:53,657 --> 00:12:55,626
TO MAKE SURE THIS IS REAL.

308
00:12:55,626 --> 00:12:58,295
AND I'LL DESCRIBE ONE OF THEM.

309
00:12:58,295 --> 00:12:59,329
BUT WHEN YOU LOOK ACROSS 

310
00:12:59,329 --> 00:13:02,266
MULTIPLE PATIENTS WITH HE SEE 

311
00:13:02,266 --> 00:13:08,538
THE SEE THE LIP AREA, JAW, 

312
00:13:08,538 --> 00:13:11,108
TONGUE, LARYNX AND IT'S 

313
00:13:11,108 --> 00:13:12,943
ACTIVATED WHEN WE SPEAK.

314
00:13:12,943 --> 00:13:15,245
THIS IS THE FIRST FUNCTIONAL 

315
00:13:15,245 --> 00:13:20,150
ORGANIZATION OF THE SPEECH 

316
00:13:20,150 --> 00:13:21,985
MORTAR CORTEX AND RECAPITULATED 

317
00:13:21,985 --> 00:13:24,721
THINGS WE THROUGH BEFORE AND 

318
00:13:24,721 --> 00:13:26,690
THERE WAS A NEW THING, AN 

319
00:13:26,690 --> 00:13:30,060
ADDITIONAL AREA OF THE LARYNX.

320
00:13:30,060 --> 00:13:31,928
ONE OF MY STUDENTS WAS VERY 

321
00:13:31,928 --> 00:13:34,831
INTERESTED IN THE FINDING AND 

322
00:13:34,831 --> 00:13:39,002
DID HIS WHOLE Ph.D. IN TRYING TO

323
00:13:39,002 --> 00:13:41,038
UNDERSTAND HOW THE VOCAL FOLD 

324
00:13:41,038 --> 00:13:42,239
CONTROLLED BY THE BRAINS.

325
00:13:42,239 --> 00:13:44,941
THE VOCAL FOLDS IN ADDITION TO 

326
00:13:44,941 --> 00:13:46,176
LIKE PRODUCING THE ENERGY OF THE

327
00:13:46,176 --> 00:13:49,846
VOICE, IT'S WHAT MODULATES THE 

328
00:13:49,846 --> 00:13:50,914
PITCH OF MY VOICE.

329
00:13:50,914 --> 00:13:53,984
SO WHEN I STRESS A PARTICULAR 

330
00:13:53,984 --> 00:13:56,353
WORD, THAT'S PRIMARILY BECAUSE 

331
00:13:56,353 --> 00:13:57,888
I'M CHANGING THE PITCH OF THAT 

332
00:13:57,888 --> 00:14:02,359
AND THAT'S BY TENSING THE VOCAL 

333
00:14:02,359 --> 00:14:02,592
FOLDS.

334
00:14:02,592 --> 00:14:05,262
SO WHAT BEN DISCOVERED IN A 

335
00:14:05,262 --> 00:14:08,532
PAPER IN CELL IN 2018 IS THIS 

336
00:14:08,532 --> 00:14:11,134
AREA, IN PARTICULAR THIS 

337
00:14:11,134 --> 00:14:12,936
PREVIOUSLY UNKNOWN LARYNGEAL 

338
00:14:12,936 --> 00:14:17,974
MOTOR CORTEX IS DIRECTLY 

339
00:14:17,974 --> 00:14:20,744
RESPONSIBLE FOR THE VOCAL FOLDS.

340
00:14:20,744 --> 00:14:22,946
HE DID AN EXPERIMENT WHERE 

341
00:14:22,946 --> 00:14:24,948
PEOPLE BASICALLY SAID THE SAME 

342
00:14:24,948 --> 00:14:26,249
SENTENCE OVER MULTIPLE TRIALS 

343
00:14:26,249 --> 00:14:28,785
BUT ON A GIVEN TRIAL THEY STRESS

344
00:14:28,785 --> 00:14:31,822
ONE WORD AND WE DID THESE KIND 

345
00:14:31,822 --> 00:14:32,456
OF RANDOMLY.

346
00:14:32,456 --> 00:14:38,762
IN ONE INSTANCE I NEVER SAID SHE

347
00:14:38,762 --> 00:14:41,698
STOLE MY MONEY, I NEVER SAID SHE

348
00:14:41,698 --> 00:14:46,203
STOLE MY MONEY AND CHANGE THIS 

349
00:14:46,203 --> 00:14:47,904
AND WE USE THIS IN ENGLISH AND 

350
00:14:47,904 --> 00:14:50,607
YOU CAN CHANGE DIRECTLY THE WORD

351
00:14:50,607 --> 00:14:50,841
MEANING.

352
00:14:50,841 --> 00:14:52,943
WHAT BEN FOUND WAS FINDING THIS 

353
00:14:52,943 --> 00:14:58,415
CLUSTER OF ELECTRODES ON THE 

354
00:14:58,415 --> 00:15:00,650
RIGHT AND LEFT HEMISPHERE 

355
00:15:00,650 --> 00:15:02,219
CORRELATED TO THE PITCH OF THE 

356
00:15:02,219 --> 00:15:02,786
VOICE.

357
00:15:02,786 --> 00:15:04,955
STRONGLY CORRELATED AND TIGHTLY 

358
00:15:04,955 --> 00:15:07,324
CLUSTERED AND CONFIRMED WHAT WE 

359
00:15:07,324 --> 00:15:08,925
FOUND FROM OUR EARLIER STUDIES 

360
00:15:08,925 --> 00:15:09,326
USING SYLLABLES.

361
00:15:09,326 --> 00:15:12,929
IF YOU LOOK AT ALL THE TRIALS 

362
00:15:12,929 --> 00:15:20,070
AND LOOKED AND THE PITCH TRA TRG

363
00:15:20,070 --> 00:15:25,208
OF THE UTTERANCES AND LOOKING AT

364
00:15:25,208 --> 00:15:27,711
THE HIGH FREQUENCY COMPONENT, WE

365
00:15:27,711 --> 00:15:29,946
CALL HIGH GAMMA, THERE'S SINGLE 

366
00:15:29,946 --> 00:15:34,951
TRIAL TIME LINE RESPONSE TO WHAT

367
00:15:34,951 --> 00:15:40,757
WORD WHAT  -- WAS STRESSED.

368
00:15:40,757 --> 00:15:45,595
THE VOCAL FOLDS ARE TENSING AND 

369
00:15:45,595 --> 00:15:47,697
WE VERIFIED.

370
00:15:47,697 --> 00:15:51,435
BEN ALSO SHOWED WHEN PEOPLE ARE 

371
00:15:51,435 --> 00:15:55,739
SENDING MALADIES THE DOE RAY ME 

372
00:15:55,739 --> 00:15:58,842
KIND OF THING IT'S NOT SPECIFIC 

373
00:15:58,842 --> 00:16:00,944
TO SPEECH IT'S SOMETHING MORE 

374
00:16:00,944 --> 00:16:03,880
RUDIMENTARY LIKE THE VOCAL FOLD 

375
00:16:03,880 --> 00:16:04,581
TENSION.

376
00:16:04,581 --> 00:16:07,284
SOMETHING UNEXPECTED BUT QUITE 

377
00:16:07,284 --> 00:16:09,386
INTERESTING WAS THAT WHEN WE 

378
00:16:09,386 --> 00:16:11,188
PLAYED BACK PEOPLE'S SPEECH 

379
00:16:11,188 --> 00:16:13,757
THROUGH A SPEAKER, WE FOUND 

380
00:16:13,757 --> 00:16:15,459
THESE AREAS WERE ALSO ACTIVATED 

381
00:16:15,459 --> 00:16:20,130
AND CODING WHAT THEY HEARD.

382
00:16:20,130 --> 00:16:22,766
SO THIS MEANS THERE'S AUDITORY 

383
00:16:22,766 --> 00:16:24,935
INFORMATION IN THIS PART OF THE 

384
00:16:24,935 --> 00:16:27,471
BRAIN AS WELL.

385
00:16:27,471 --> 00:16:29,906
IT'S ENCODING THE PITCH OF WHAT 

386
00:16:29,906 --> 00:16:30,507
PEOPLE HEARD.

387
00:16:30,507 --> 00:16:32,275
IN A SEPARATE SET OF EXPERIMENTS

388
00:16:32,275 --> 00:16:33,977
AND PEOPLE UNDERGOING BRAIN 

389
00:16:33,977 --> 00:16:35,412
MAPPING AS PART OF THEIR SLEEP 

390
00:16:35,412 --> 00:16:38,949
SURGERIES, WHAT WE DID WAS WE 

391
00:16:38,949 --> 00:16:42,486
USED A BREATHING TUBE MY 

392
00:16:42,486 --> 00:16:44,588
COLLEAGUES IN E.N.T. USE FOR 

393
00:16:44,588 --> 00:16:46,122
MAPPING THE NERVES IN THE NECK 

394
00:16:46,122 --> 00:16:47,491
WHEN THEY DISSECT TUMORS HERE.

395
00:16:47,491 --> 00:16:50,927
BUT HERE WE WERE USING THE 

396
00:16:50,927 --> 00:16:55,131
ENDOTRACHEAL TUBE THAT HAS WIRES

397
00:16:55,131 --> 00:17:00,103
THAT NON-INVASIVELY TRACK THE 

398
00:17:00,103 --> 00:17:05,475
FOLDS AND SEE WHAT PART OF WERE 

399
00:17:05,475 --> 00:17:11,214
EVOKING MOVEMENT AND THERE WERE 

400
00:17:11,214 --> 00:17:13,450
PARTS THAT EVOKED MOVEMENT.

401
00:17:13,450 --> 00:17:16,353
WE CAN PROVE THIS WAS A MOTOR 

402
00:17:16,353 --> 00:17:18,188
AREA NOT JUST A SENSORY AUDITORY

403
00:17:18,188 --> 00:17:20,457
ONE BUT IN FACT A MOTOR AND 

404
00:17:20,457 --> 00:17:21,024
SENSORY.

405
00:17:21,024 --> 00:17:23,260
IN FACT, IF YOU INCREASE THE 

406
00:17:23,260 --> 00:17:25,595
MAGNITUDE OF THE STIMULATION WE 

407
00:17:25,595 --> 00:17:27,764
SEE AN INCREASE IN MAGNITUDE OF 

408
00:17:27,764 --> 00:17:28,465
THE ENG.

409
00:17:28,465 --> 00:17:31,001
SO THERE'S A PART OF OUR BRAIN 

410
00:17:31,001 --> 00:17:34,371
ON BOTH SIDES THAT CONTROLS THE 

411
00:17:34,371 --> 00:17:36,373
VOCAL FOLDS IN A VERY DIRECT WAY

412
00:17:36,373 --> 00:17:37,007
AND WE THINK IT'S THE 

413
00:17:37,007 --> 00:17:38,608
CONNECTION.

414
00:17:38,608 --> 00:17:41,411
WHAT ABOUT WHAT'S HAPPENING 

415
00:17:41,411 --> 00:17:43,413
ABOVE THE VOCAL FOLDS?

416
00:17:43,413 --> 00:17:46,550
THE MOVEMENT OF SPEECH ARE 

417
00:17:46,550 --> 00:17:47,217
COMPLEX COMPARED TO THE MOVEMENT

418
00:17:47,217 --> 00:17:52,756
OF THE AROUND AND YOU CAN MODEL 

419
00:17:52,756 --> 00:17:55,025
THE MOVEMENTS.

420
00:17:55,025 --> 00:17:58,495
THE MUSCLES OF THE LIPS, FOR 

421
00:17:58,495 --> 00:18:01,498
EXAMPLE HAVE A TOLDLY DIFFERENT 

422
00:18:01,498 --> 00:18:08,939
SET OF PROPERTIES MORE LIKE AN 

423
00:18:08,939 --> 00:18:19,482
IT  SPHINCTER AND CAN STUDY IT 

424
00:18:20,650 --> 00:18:21,518
USING MICROSOUND AND IF YOU LOOK

425
00:18:21,518 --> 00:18:27,457
AT ONE ELECTRODE, THE SITES IN 

426
00:18:27,457 --> 00:18:30,393
THE MOTOR CORTEX ARE ENCODING 

427
00:18:30,393 --> 00:18:31,361
MOVEMENT TRAJECTORIES AND 

428
00:18:31,361 --> 00:18:34,598
ENCODING MOVEMENTS OF OUR VOCAL 

429
00:18:34,598 --> 00:18:35,498
TRACT WHEN WE SPEAK.

430
00:18:35,498 --> 00:18:36,967
BY THEMSELVES THEY HAVE NO 

431
00:18:36,967 --> 00:18:39,336
MEANING BUT WHEN YOU PUT THEM 

432
00:18:39,336 --> 00:18:43,707
TOGETHER AND GIVE RISE TO ALL 

433
00:18:43,707 --> 00:18:44,307
CONSON

434
00:18:44,307 --> 00:18:44,608
CONSONANTS.

435
00:18:44,608 --> 00:18:45,775
THIS IS AN EXAMPLE THAT IS VERY 

436
00:18:45,775 --> 00:18:50,580
IMPORTANT FOR ENCODING THE DA 

437
00:18:50,580 --> 00:18:50,847
SOUND.

438
00:18:50,847 --> 00:18:52,983
THIS PARTICULAR TRAJECTORY WHERE

439
00:18:52,983 --> 00:18:56,953
THE TONGUE TIP GOES TO THE BACK 

440
00:18:56,953 --> 00:18:59,723
AND THE JAW GOES UP AND DOWN 

441
00:18:59,723 --> 00:19:01,458
THAT ONE PARTICULAR ELECTRODE IS

442
00:19:01,458 --> 00:19:07,263
ENCODED BY A VIDEO AND THE 

443
00:19:07,263 --> 00:19:10,000
TONGUE TIP GOES FORWARD AND THE 

444
00:19:10,000 --> 00:19:12,068
JAW OPENS AND CLOSES.

445
00:19:12,068 --> 00:19:15,572
IT'S A COORDINATED MOVEMENT 

446
00:19:15,572 --> 00:19:17,340
ACROSS ARTICULATORS ALL ENCODED 

447
00:19:17,340 --> 00:19:18,408
IN THIS ONE PARTICULAR SPOT.

448
00:19:18,408 --> 00:19:20,710
ALL OF US HAVE THIS.

449
00:19:20,710 --> 00:19:23,313
THIS ENCODING OF THIS SOUND.

450
00:19:23,313 --> 00:19:25,215
THAT'S HOW WE MAKE THE DA SOUND.

451
00:19:25,215 --> 00:19:28,785
IF YOU COMPARE IT TO ANOTHER 

452
00:19:28,785 --> 00:19:30,220
ELECTRODE A CENTIMETER AWAY IT'S

453
00:19:30,220 --> 00:19:31,888
IMPORTANT FOR THE LABIAL 

454
00:19:31,888 --> 00:19:34,724
TRAJECTORY AND YOU'LL SEE THE 

455
00:19:34,724 --> 00:19:38,361
MAIN MOVEMENTS HERE ARE AROUND 

456
00:19:38,361 --> 00:19:39,429
CLOSURE OF THE LIPS.

457
00:19:39,429 --> 00:19:42,532
SO WE HAVE THESE DIFFERENT SPOTS

458
00:19:42,532 --> 00:19:44,968
IN THE MOTOR CORTEX THAT ENCODE 

459
00:19:44,968 --> 00:19:47,003
DIFFERENT TRAJECTORIES OF 

460
00:19:47,003 --> 00:19:48,438
MOVEMENT THAT CONTROL DIFFERENT 

461
00:19:48,438 --> 00:19:48,805
PARTS.

462
00:19:48,805 --> 00:19:50,807
AGAIN, BY THEMSELVES THEY HAVE 

463
00:19:50,807 --> 00:19:51,841
NO MEANING BUT WHEN YOU SEQUENCE

464
00:19:51,841 --> 00:19:54,110
THEM YOU PUT THEM TOGETHER AND 

465
00:19:54,110 --> 00:19:56,980
THEY'RE WORKING IN A COORDINATED

466
00:19:56,980 --> 00:19:59,949
WAY IT GIVES RISE TO ALL 

467
00:19:59,949 --> 00:20:02,786
CONSONANTS AND VOWELS WHICH GIVE

468
00:20:02,786 --> 00:20:04,954
RISE TO ALL WORDS AND POSSIBLE 

469
00:20:04,954 --> 00:20:05,555
MEANINGS.

470
00:20:05,555 --> 00:20:08,658
THINKING ABOUT THESE IS 

471
00:20:08,658 --> 00:20:10,794
FUNDAMENTAL FOR COMMUNICATION IN

472
00:20:10,794 --> 00:20:11,728
BUILDING BLOCKS FOR SPEECH.

473
00:20:11,728 --> 00:20:14,431
ONE THING WE CAN DO IS WE CAN 

474
00:20:14,431 --> 00:20:16,533
LOOK AT THE DATA ACROSS 

475
00:20:16,533 --> 00:20:16,900
ELECTRODES.

476
00:20:16,900 --> 00:20:19,202
EACH COLUMN CORRESPONDS TO A 

477
00:20:19,202 --> 00:20:22,439
DIFFERENT SENSOR AND EACH ROW 

478
00:20:22,439 --> 00:20:24,874
CORRESPONDS TO DIFFERENT PARTS 

479
00:20:24,874 --> 00:20:27,744
OF THE VOCAL TRACT AND BELOW 

480
00:20:27,744 --> 00:20:36,986
THAT ARE THE PHONYMS IN ENGLISH 

481
00:20:36,986 --> 00:20:39,522
AND THIS IS ALL THE ELECTRODES 

482
00:20:39,522 --> 00:20:41,524
AND HOW THEY'RE ACTIVATED BY 

483
00:20:41,524 --> 00:20:43,727
DIFFERENT PRODUCTIONS.

484
00:20:43,727 --> 00:20:48,898
ONE REASON I SHOW THIS IS AS 

485
00:20:48,898 --> 00:20:50,934
EARLY AS EIGHT YEARS AGO WE HAD 

486
00:20:50,934 --> 00:20:55,071
THIS INTUITION WE COULD ACCESS 

487
00:20:55,071 --> 00:20:56,973
THE ENTIRE FRENETIC ALPHABET, 

488
00:20:56,973 --> 00:21:00,977
THE ENCODING, A DICTIONARY IN 

489
00:21:00,977 --> 00:21:06,850
SOME WAYS OF THE ELECTRICAL 

490
00:21:06,850 --> 00:21:10,286
PATTERNS IN ENGLISH AND THAT WAS

491
00:21:10,286 --> 00:21:12,322
IMPORTANT BECAUSE THEN WE 

492
00:21:12,322 --> 00:21:15,158
STARTED OF TRANSLATING THIS 

493
00:21:15,158 --> 00:21:16,593
TECHNOLOGY FOR PEOPLE WHO ARE 

494
00:21:16,593 --> 00:21:17,160
PARALYZED.

495
00:21:17,160 --> 00:21:19,129
I'LL TALK ABOUT WHAT WE'VE DONE 

496
00:21:19,129 --> 00:21:21,564
WITH THAT KIND OF INSIGHT TO 

497
00:21:21,564 --> 00:21:24,200
BUILD A SPEECH PROSTHETIC.

498
00:21:24,200 --> 00:21:30,306
THIS QUOTE FROM STEVEN HAWKINGS 

499
00:21:30,306 --> 00:21:33,009
THOUGH I HAVE TO SPEAK THROUGH A

500
00:21:33,009 --> 00:21:35,712
COMPUTER, IN MY MIND I AM FREE.

501
00:21:35,712 --> 00:21:36,646
THIS ENCAPSULATES PART OF THE 

502
00:21:36,646 --> 00:21:38,414
PROBLEM WE'RE TRYING TO SOLVE 

503
00:21:38,414 --> 00:21:42,185
FOR PEOPLE WHO ARE LOCKED IN 

504
00:21:42,185 --> 00:21:47,657
WHERE ESSENTIAL LY AND HAVE TO 

505
00:21:47,657 --> 00:21:51,294
SEND SIGNALS OUT TO THE MAP.

506
00:21:51,294 --> 00:21:59,369
THERE'S WE'VE ENDEAVORED TO 

507
00:21:59,369 --> 00:22:00,970
BUILD A DEVICE, A TECHNOLOGY 

508
00:22:00,970 --> 00:22:03,006
THAT CAN TRANSLATE THAT CODE 

509
00:22:03,006 --> 00:22:03,573
I'VE BEEN DESCRIBING INTO 

510
00:22:03,573 --> 00:22:05,441
SOMETHING THAT CAN DIRECTLY READ

511
00:22:05,441 --> 00:22:08,578
OUT AND PRODUCE SPEECH YOU CAN 

512
00:22:08,578 --> 00:22:08,778
LEAR.

513
00:22:08,778 --> 00:22:12,048
SO THE BASIC QUESTION FRAMED BY 

514
00:22:12,048 --> 00:22:15,318
THIS M.I.T. TECH REVIEW COVERING

515
00:22:15,318 --> 00:22:19,088
THE EARLY WORK IS COULD A PERSON

516
00:22:19,088 --> 00:22:20,990
UNABLE TO SPEAK LIKE STEVEN 

517
00:22:20,990 --> 00:22:24,227
HAWKING USE A BRAIN IMPLANT TO 

518
00:22:24,227 --> 00:22:25,061
CARRY ON A CONVERSATION?

519
00:22:25,061 --> 00:22:25,895
THAT WAS THE BASIC QUESTION WE 

520
00:22:25,895 --> 00:22:36,206
WANTED TO PURSUE.

521
00:22:37,307 --> 00:22:39,542
AND ONE QUESTION IS WHY SPEECH?

522
00:22:39,542 --> 00:22:44,981
WHY NOT HAVE PEOPLE COMMUNICATE 

523
00:22:44,981 --> 00:22:48,718
THROUGH HANDWRITING?

524
00:22:48,718 --> 00:22:50,119
BRAIN-INTERFACE BY WHAT PEOPLE 

525
00:22:50,119 --> 00:22:51,020
ARE TRYING TO EXPRESS THROUGH 

526
00:22:51,020 --> 00:22:54,190
THEIR WRITING OR TYPE OR TEXT?

527
00:22:54,190 --> 00:22:56,092
THOSE ARE ALL EFFECTIVE WAYS.

528
00:22:56,092 --> 00:23:00,396
IN FACT THE WHOLE FIELD WAS 

529
00:23:00,396 --> 00:23:02,098
FOCUSSED ON GETTING PATIENTS 

530
00:23:02,098 --> 00:23:05,535
PARALYZED ABLE TO COMMUNICATE 

531
00:23:05,535 --> 00:23:08,471
THROUGH MOVING A COMPUTER CURSER

532
00:23:08,471 --> 00:23:10,640
QUICKLY TO DIFFERENT PARTS OF 

533
00:23:10,640 --> 00:23:11,541
THE KEY BOARD.

534
00:23:11,541 --> 00:23:12,876
WE WANTED TO DO SOMETHING 

535
00:23:12,876 --> 00:23:17,914
DIFFERENT BECAUSE SPEECH IS I A 

536
00:23:17,914 --> 00:23:19,249
DIFFERENT PART OF COMMUNICATE 

537
00:23:19,249 --> 00:23:21,818
AND ALLOWS US TO COMMUNE UP TO 

538
00:23:21,818 --> 00:23:23,019
200 WORDS PER MINUTE.

539
00:23:23,019 --> 00:23:24,954
THAT'S SUCH A SPECIAL BEHAVIOR 

540
00:23:24,954 --> 00:23:26,356
FOR OUR SPECIES.

541
00:23:26,356 --> 00:23:28,324
IT ALLOWS ME TO TRANSMIT 

542
00:23:28,324 --> 00:23:31,527
INFORMATION FROM MY MIND TO 

543
00:23:31,527 --> 00:23:33,363
YOURS AND BACK IN A VERY FLUENT 

544
00:23:33,363 --> 00:23:33,696
WAY.

545
00:23:33,696 --> 00:23:36,466
IT ALLOWS US TO BUILD 

546
00:23:36,466 --> 00:23:37,000
CIVILIZATION.

547
00:23:37,000 --> 00:23:39,302
IT ALLOWS US AS SOCIAL BEINGS TO

548
00:23:39,302 --> 00:23:43,206
TRANSMIT IDEAS TO BUILD, TO 

549
00:23:43,206 --> 00:23:44,307
LEARN AND FOUNDATIONAL.

550
00:23:44,307 --> 00:23:48,211
SUCH AN IMPORTANT PART OF BEING 

551
00:23:48,211 --> 00:23:48,845
HUMAN.

552
00:23:48,845 --> 00:23:51,848
THIS IS WHY WE WENT STRAIGHT TO 

553
00:23:51,848 --> 00:23:54,550
THIS AMBITIOUS IDEA OF TRYING TO

554
00:23:54,550 --> 00:23:57,186
DO A SPEECH PROSTHETIC.

555
00:23:57,186 --> 00:23:59,188
EARLY ON A LOT OF MY CLOSEST 

556
00:23:59,188 --> 00:24:00,957
FRIENDS AND COLLEAGUES SAID THIS

557
00:24:00,957 --> 00:24:01,991
IS IMPOSSIBLE.

558
00:24:01,991 --> 00:24:03,026
TOO COMPLEX.

559
00:24:03,026 --> 00:24:06,262
SPEECH IS TOO ABSTRACT.

560
00:24:06,262 --> 00:24:08,364
I KIND OF AGREE WITH THEM BUT AT

561
00:24:08,364 --> 00:24:12,335
THE SAME TIME WE WERE SEEING 

562
00:24:12,335 --> 00:24:13,136
THIS ELECTRICAL CODE LAID OUT IN

563
00:24:13,136 --> 00:24:15,838
FRONT OF US SO WE SAID, OKAY, 

564
00:24:15,838 --> 00:24:19,776
WHY DON'T WE GIVE IT A TRY?

565
00:24:19,776 --> 00:24:22,345
ONE OF THE FIRST THINGS WE DID 

566
00:24:22,345 --> 00:24:24,981
WAS TRY IT IN EPILEPSY PATIENTS 

567
00:24:24,981 --> 00:24:27,083
AS PROOF OF PRINCIPLE.

568
00:24:27,083 --> 00:24:29,018
THESE ARE PEOPLE SPEAKING.

569
00:24:29,018 --> 00:24:31,020
NORMALLY SPEAKING.

570
00:24:31,020 --> 00:24:36,726
WE COULD RECORD ABOUT WHAT THEY 

571
00:24:36,726 --> 00:24:40,563
ARE WERE SPEAKING AND DECODE 

572
00:24:40,563 --> 00:24:42,665
THEIR BRAIN ACTIVITY FROM WHAT 

573
00:24:42,665 --> 00:24:43,166
THEY SAID.

574
00:24:43,166 --> 00:24:45,668
A GREAT WAY TO START BECAUSE WE 

575
00:24:45,668 --> 00:24:48,304
KNEW WHAT THEY SAID AND TRY TO 

576
00:24:48,304 --> 00:24:49,639
BUILD A COMPUTER MODEL TO DECODE

577
00:24:49,639 --> 00:24:51,407
WHAT THEY WERE SAYING FROM THE 

578
00:24:51,407 --> 00:24:51,808
BRAIN ACTIVITY.

579
00:24:51,808 --> 00:24:55,545
THAT'S WHAT WE DID IN THE STUDY 

580
00:24:55,545 --> 00:25:00,016
LED BY OUR GRADUATE STUDENT AND 

581
00:25:00,016 --> 00:25:00,984
POSTDOC.

582
00:25:00,984 --> 00:25:02,452
IN THIS PARTICULAR APPROACH WE 

583
00:25:02,452 --> 00:25:04,087
TRANSLATED THE BRAIN ACTIVITY TO

584
00:25:04,087 --> 00:25:08,624
A SIMULATION OF THE VOCAL TRACT 

585
00:25:08,624 --> 00:25:12,261
INTO A COMPUTER AND HAD THAT 

586
00:25:12,261 --> 00:25:13,463
CREATE WORDS YOU COULD HEAR.

587
00:25:13,463 --> 00:25:16,265
IT WAS A TWO-PART DECODER.

588
00:25:16,265 --> 00:25:18,668
ONE PART IS TRANSLATING BRAIN 

589
00:25:18,668 --> 00:25:19,469
ACTIVITY INTO MOVEMENTS.

590
00:25:19,469 --> 00:25:22,472
A SIMULATION OF THE VOCAL TRACT 

591
00:25:22,472 --> 00:25:24,674
AND THEN TRANSLATING THOSE 

592
00:25:24,674 --> 00:25:26,342
SIMULATED MOVEMENTS INTO SOUND.

593
00:25:26,342 --> 00:25:28,978
AND IT IS POSSIBLE TO GO FROM 

594
00:25:28,978 --> 00:25:31,214
BRAIN ACTIVITY TO THE SOUNDS BUT

595
00:25:31,214 --> 00:25:32,682
ONE OF THE REASONS WE CHOSE THIS

596
00:25:32,682 --> 00:25:33,983
METHOD IS BECAUSE WE KNEW THIS 

597
00:25:33,983 --> 00:25:37,987
PART OF THE BRAIN IS CONTROLLING

598
00:25:37,987 --> 00:25:40,957
MOVEMENT OF THE VOCAL TRACT.

599
00:25:40,957 --> 00:25:48,965
WE WANTED TO BE SORT OF AS BIO 

600
00:25:48,965 --> 00:25:49,899
MIMMENIC AS POSSIBLE AND MATCH 

601
00:25:49,899 --> 00:25:52,001
THE MODEL AND THIS IS AN EXAMPLE

602
00:25:52,001 --> 00:25:53,970
OF THE SPEECH.

603
00:25:53,970 --> 00:26:00,610
HERE'S A SPECTROGRAM RECORDED ON

604
00:26:00,610 --> 00:26:03,546
TOP AND WHAT WAS GENERATED BY 

605
00:26:03,546 --> 00:26:07,316
OUR FIRST DECODER OF THOSE TWO 

606
00:26:07,316 --> 00:26:08,918
SENTENCES JUST FROM THE NEURAL 

607
00:26:08,918 --> 00:26:09,252
ACTIVITY.

608
00:26:09,252 --> 00:26:11,654
YOU CAN SEE IT HAS A LOT OF THE 

609
00:26:11,654 --> 00:26:16,059
SAME ENERGY CONTOURS AND GROSS 

610
00:26:16,059 --> 00:26:19,262
SPECTRAL AND TEMPORAL 

611
00:26:19,262 --> 00:26:20,963
INFORMATION THERE WAS STRUCTURE 

612
00:26:20,963 --> 00:26:24,567
OF THE SPEECH WAS MISSING.

613
00:26:24,567 --> 00:26:29,672
WE WERE EXCITED TO SEE IT HAD 

614
00:26:29,672 --> 00:26:30,973
THIS VISUAL APPEARANCE THAT WAS 

615
00:26:30,973 --> 00:26:32,975
CLOSE BUT WHAT WAS EXCITING IS 

616
00:26:32,975 --> 00:26:43,519
SOME SENTENCES WERE INTELIGABLE 

617
00:26:47,523 --> 00:26:48,791
AND TO PROVE THEY'RE POSSIBLE 

618
00:26:48,791 --> 00:26:50,193
WITH THE ELECTRODES NEVER DESIGN

619
00:26:50,193 --> 00:27:00,670
TO DO THIS KIND OF PURPOSE.

620
00:27:01,637 --> 00:27:12,115
AND SEE THE VOCAL TRACT AND THEN

621
00:27:13,382 --> 00:27:16,052
HEAR WHAT THE PERSON SAID TO 

622
00:27:16,052 --> 00:27:26,195
COMPARE.

623
00:27:28,431 --> 00:27:32,969
>> THE APPROACH YOU ARE
SEEKING 

624
00:27:32,969 --> 00:27:35,171
IS NOT AVAILABLE IN BOOKS.

625
00:27:35,171 --> 00:27:37,006
THE PROOF YOU ARE SEEKING IS NOT

626
00:27:37,006 --> 00:27:46,249
AVAILABLE IN BOOKS.

627
00:27:46,249 --> 00:27:48,951
SHIP BUILDING IS A MOST 

628
00:27:48,951 --> 00:27:55,892
FASCINATING PROCESS.

629
00:27:55,892 --> 00:27:58,694
>> SO, WHAT WAS EXCITING
WAS FOR

630
00:27:58,694 --> 00:28:03,232
THE FIRST TIME HEARING A VOICE 

631
00:28:03,232 --> 00:28:04,901
IN THIS SYNTHESIS.

632
00:28:04,901 --> 00:28:15,311
IT'S RASPY, IT'S NOISY.

633
00:28:15,511 --> 00:28:18,214
AND IF YOU HALLUCINATE YOU'LL 

634
00:28:18,214 --> 00:28:20,082
HEAR VOICES AT THE BORDERLINE 

635
00:28:20,082 --> 00:28:21,350
BUT THAT WAS EXCITING TO HEAR 

636
00:28:21,350 --> 00:28:23,152
AND USE THAT WITH THE LEARNING 

637
00:28:23,152 --> 00:28:23,386
MODELS.

638
00:28:23,386 --> 00:28:26,789
THIS IS JUST AN EXAMPLE OF WHY 

639
00:28:26,789 --> 00:28:29,592
WE DID THIS WITH THIS KINEMATIC 

640
00:28:29,592 --> 00:28:29,792
MODEL.

641
00:28:29,792 --> 00:28:32,795
THE MCD IS A COEFFICIENT THAT 

642
00:28:32,795 --> 00:28:35,965
EXPRESSES HOW SIMILAR THE 

643
00:28:35,965 --> 00:28:39,936
ORIGINAL SPOKEN SPECTROGRAM IS 

644
00:28:39,936 --> 00:28:43,806
TO WHAT IS DECODED AND GOING 

645
00:28:43,806 --> 00:28:44,740
STRAIGHT FROM BRAIN ACTIVITY 

646
00:28:44,740 --> 00:28:46,909
SOUND YOU CAN DO IT WITH ENOUGH 

647
00:28:46,909 --> 00:28:47,276
DATA.

648
00:28:47,276 --> 00:28:55,685
BUT BUILDING AN ARTICULARY MODEL

649
00:28:55,685 --> 00:28:56,686
REQUIRES MORE DATA.

650
00:28:56,686 --> 00:28:58,854
WITH BRAIN ACTIVITY TO SOUND HAS

651
00:28:58,854 --> 00:29:00,656
TO DEVELOP A COMPLEX MAPPING.

652
00:29:00,656 --> 00:29:02,792
THE STRATEGY HERE WAS LET'S 

653
00:29:02,792 --> 00:29:04,860
REDUCE HOW MUCH THE NETWORK HAS 

654
00:29:04,860 --> 00:29:07,663
TO LEARN BY IMPOSING STRUCTURE 

655
00:29:07,663 --> 00:29:08,965
WE KNOW TO BE TRUE.

656
00:29:08,965 --> 00:29:11,234
THE VOCAL TRACT MOVEMENTS ARE A 

657
00:29:11,234 --> 00:29:12,568
CAUSAL MECHANISM OF SOUND 

658
00:29:12,568 --> 00:29:14,470
GENERATION AND PUT THAT IN AND 

659
00:29:14,470 --> 00:29:15,972
THAT CAN REDUCE AND REDUCING THE

660
00:29:15,972 --> 00:29:17,873
TRAINING TIME IS VERY IMPORTANT 

661
00:29:17,873 --> 00:29:20,776
FOR PEOPLE WHO ARE PARALYZED OF 

662
00:29:20,776 --> 00:29:21,377
COURSE BECAUSE YOU CAN'T TAKE 

663
00:29:21,377 --> 00:29:25,348
FOREVER.

664
00:29:25,348 --> 00:29:27,850
SO WE BASICALLY CONVINCE 

665
00:29:27,850 --> 00:29:31,887
OURSELVES THAT WE COULD TRY 

666
00:29:31,887 --> 00:29:33,055
THIS.

667
00:29:33,055 --> 00:29:36,125
IN SOMEONE PARALYZED.

668
00:29:36,125 --> 00:29:41,697
SO WE CREATED THE BRAVO TRIAL AT

669
00:29:41,697 --> 00:29:44,166
UCSF IT STANDS FOR THE BCI 

670
00:29:44,166 --> 00:29:46,168
RESTORATION OF ARM AND VOICE.

671
00:29:46,168 --> 00:29:50,373
MY LAB AND A REHABILITATION 

672
00:29:50,373 --> 00:29:52,975
NEUROLOGIST STUDYING HOW 

673
00:29:52,975 --> 00:29:53,876
CORTICAL REPRESENTATIONS AROUND 

674
00:29:53,876 --> 00:29:55,945
THE HAND ARE USED TO CONTROL A 

675
00:29:55,945 --> 00:29:57,380
CURSER OR ROBOTIC ARM.

676
00:29:57,380 --> 00:30:00,750
THE PART MY LAB WAS FOCUSSING ON

677
00:30:00,750 --> 00:30:10,459
WAS THE PART OF THE SPEECH.

678
00:30:10,459 --> 00:30:16,899
WE HAVE PARALLEL BUT 

679
00:30:16,899 --> 00:30:19,001
COMPLIMENTARY STREAMS AND 

680
00:30:19,001 --> 00:30:24,073
THEY'RE FOR PEOPLE WITH SEVERE 

681
00:30:24,073 --> 00:30:26,876
PARALYSIS AND LIMB MOVEMENT.

682
00:30:26,876 --> 00:30:31,213
THIS WAS APPROVED WITHIN I.D.E.

683
00:30:31,213 --> 00:30:32,214
THE FIRST PARTICIPANT THAT 

684
00:30:32,214 --> 00:30:34,550
UNDERWENT THE STUDY WAS 

685
00:30:34,550 --> 00:30:38,020
DESCRIBED IN THIS PAPER IN THE 

686
00:30:38,020 --> 00:30:39,021
NEW ENGLAND JOURNAL IT DESCRIBES

687
00:30:39,021 --> 00:30:40,623
THE FIRST SUCCESSFUL DECODING OF

688
00:30:40,623 --> 00:30:43,259
WORDS AND SENTENCES FROM SOMEONE

689
00:30:43,259 --> 00:30:44,093
PARALYZED.

690
00:30:44,093 --> 00:30:45,995
I'LL TELL YOU A LITTLE BIT ABOUT

691
00:30:45,995 --> 00:30:47,096
HIS CASE.

692
00:30:47,096 --> 00:30:50,933
HE WAS THE FIRST IN THE WORLD 

693
00:30:50,933 --> 00:30:52,368
AND IMPLANTED ABOUT SIX AND A 

694
00:30:52,368 --> 00:30:53,069
HALF YEARS AFTER THE ORIGINAL 

695
00:30:53,069 --> 00:31:03,212
IMPLANT.

696
00:31:03,512 --> 00:31:06,816
AND THE WAY HE NORMALLY 

697
00:31:06,816 --> 00:31:10,186
COMMUNICATED IS WITH A STITCH 

698
00:31:10,186 --> 00:31:12,588
ATTACHED TO HIS HAT AND TYPES 

699
00:31:12,588 --> 00:31:15,991
THINGS ON THIS ASSISTIVE DEVICE 

700
00:31:15,991 --> 00:31:17,860
MANY PEOPLE USE WITH A 

701
00:31:17,860 --> 00:31:18,694
COMMUNICATION DISORDER CALLED 

702
00:31:18,694 --> 00:31:19,061
AAC.

703
00:31:19,061 --> 00:31:25,701
THE GOAL IS TO ALLOW PONCHO TO 

704
00:31:25,701 --> 00:31:27,036
COMMUNICATE FASTER.

705
00:31:27,036 --> 00:31:30,706
THIS IS WHAT HIS MRI LOOKS LIKE.

706
00:31:30,706 --> 00:31:32,541
THIS IS AN IMPORTANT CONCEPT.

707
00:31:32,541 --> 00:31:35,344
WHERE HIS STROKE WAS HERE IN THE

708
00:31:35,344 --> 00:31:36,712
PART CALLED THE BRAIN STEM AND 

709
00:31:36,712 --> 00:31:38,981
THE BRAIN STEM IS WHAT CONNECTS 

710
00:31:38,981 --> 00:31:42,651
THE BRAIN TO THE SPINAL CORD AND

711
00:31:42,651 --> 00:31:43,552
CRANIAL NERVES.

712
00:31:43,552 --> 00:31:47,857
SO HE CAN'T GET THE SIGNALS OUT 

713
00:31:47,857 --> 00:31:51,927
OF HIS BRAIN BECAUSE HAD A HUGE 

714
00:31:51,927 --> 00:31:54,597
STROKE THAT OCCURRED IN THE POS 

715
00:31:54,597 --> 00:31:56,799
BUT EVERYTHING ABOVE THAT IS 

716
00:31:56,799 --> 00:31:57,333
WORKING NORMALLY.

717
00:31:57,333 --> 00:31:59,702
HIS THOUGHTS, HIS IDEAS ARE ALL 

718
00:31:59,702 --> 00:31:59,902
THERE.

719
00:31:59,902 --> 00:32:03,406
HE JUST CAN'T GET THEM OUT 

720
00:32:03,406 --> 00:32:07,009
BECAUSE THE COMMUNICATION TO THE

721
00:32:07,009 --> 00:32:07,943
EXTERNAL WORLD HAS BEEN SEVERED.

722
00:32:07,943 --> 00:32:12,047
SO WHAT WE DID WAS WE BUILT AN 

723
00:32:12,047 --> 00:32:15,951
ARRAY -- WE TOOK THE ARRAYS WE 

724
00:32:15,951 --> 00:32:19,655
WERE USING FOR EPILEPSY AND 

725
00:32:19,655 --> 00:32:22,358
WORKED WITH SOME OF THE 

726
00:32:22,358 --> 00:32:25,327
COMPANIES, IN PARTICULAR, 

727
00:32:25,327 --> 00:32:28,264
BLACKROCK THAT HAD THIS PORT 

728
00:32:28,264 --> 00:32:30,232
APPROVED WITH MICROELECTRODE 

729
00:32:30,232 --> 00:32:32,968
ARRAY WE SAID WHY NOT ATTACH IT 

730
00:32:32,968 --> 00:32:36,005
TO AN ECOG BECAUSE I WANT TO 

731
00:32:36,005 --> 00:32:37,373
JUST RECORD FROM THE BRAIN 

732
00:32:37,373 --> 00:32:38,607
SURFACE AND WE WANTED TO DO THAT

733
00:32:38,607 --> 00:32:40,376
AND THE FDA WAS OKAY WITH THAT 

734
00:32:40,376 --> 00:32:44,947
BECAUSE AFTER TESTING WE COULD 

735
00:32:44,947 --> 00:32:48,784
PROVE IT HAD THE SAME SAFETY AS 

736
00:32:48,784 --> 00:32:51,620
OTHER FORMS IT WITH WAS ALREADY 

737
00:32:51,620 --> 00:32:53,289
APPROVED FOR AND ADDITIONAL 

738
00:32:53,289 --> 00:32:54,423
SAFETY TO PROVE THE NEW FORM 

739
00:32:54,423 --> 00:32:57,126
WOULD BE OKAY.

740
00:32:57,126 --> 00:33:01,030
THIS IS THE FIRST ECOG AND 

741
00:33:01,030 --> 00:33:02,198
RECORDS FROM HIS BRAIN AND 

742
00:33:02,198 --> 00:33:04,967
SIGNALS ARE TAKEN OUT AND 

743
00:33:04,967 --> 00:33:08,137
TRANSLATED AND DIGITIZED FROM 

744
00:33:08,137 --> 00:33:09,505
ANALOG BRAIN WAVES INTO DIGITAL 

745
00:33:09,505 --> 00:33:12,074
NEURAL SIGNALS.

746
00:33:12,074 --> 00:33:14,243
THE COMPUTER IS DOING SIGNAL 

747
00:33:14,243 --> 00:33:16,445
PROCESSING AT THE FIRST STEP.

748
00:33:16,445 --> 00:33:19,081
PICKING OUT THE PARTS OF THE E 

749
00:33:19,081 --> 00:33:21,250
COG MOST INFORMATIVE.

750
00:33:21,250 --> 00:33:24,954
THE SECOND IS AN A.I. ALGORITHM 

751
00:33:24,954 --> 00:33:26,422
DETECTING WHEN HE'S TRYING TO 

752
00:33:26,422 --> 00:33:26,655
SPEAK.

753
00:33:26,655 --> 00:33:29,325
THE THIRD PART IS DETERMINING 

754
00:33:29,325 --> 00:33:31,961
WHAT WORD HE'S TRYING TO SAY AND

755
00:33:31,961 --> 00:33:34,730
IN THE FIRST EXPERIMENT WAS 50 

756
00:33:34,730 --> 00:33:38,968
WORDS AND RANKED PROBABILITY OF 

757
00:33:38,968 --> 00:33:41,337
WORDS AND DISPLAYING THE FIRST 

758
00:33:41,337 --> 00:33:42,438
ON THE SCREEN.

759
00:33:42,438 --> 00:33:44,039
SO IT'S THE FIRST ATTEMPT AT 

760
00:33:44,039 --> 00:33:44,240
THIS.

761
00:33:44,240 --> 00:33:45,875
IT'S FOR PEOPLE TO UNDERSTAND 

762
00:33:45,875 --> 00:33:46,709
WHAT THIS IS ABOUT.

763
00:33:46,709 --> 00:33:49,712
IT'S ABOUT A PERSON WHO HAS 

764
00:33:49,712 --> 00:33:50,880
VOLUNTEERED TO BE PART OF AN 

765
00:33:50,880 --> 00:33:51,914
EXPERIMENTAL STUDY.

766
00:33:51,914 --> 00:33:55,518
THE FIRST TIME IT WAS EVER DONE 

767
00:33:55,518 --> 00:33:57,119
AND IS NOW LIVING WITH THIS PORT

768
00:33:57,119 --> 00:34:03,959
ON HIS HEAD AND THIS IS WHAT 

769
00:34:03,959 --> 00:34:04,760
THAT CONNECTOR LOOKS LIKE.

770
00:34:04,760 --> 00:34:08,797
YOU MIGHT BE ABLE TO APPRECIATE 

771
00:34:08,797 --> 00:34:15,137
IT BUT IT HAS 128 LITTLE GOD  --

772
00:34:15,137 --> 00:34:18,874
GOLD CONTACT AND EACH HAS A WIRE

773
00:34:18,874 --> 00:34:20,910
GOING TO A DIFFERENT PART OF HIS

774
00:34:20,910 --> 00:34:21,710
BRAIN.

775
00:34:21,710 --> 00:34:26,949
SO WE'RE LITERALLY TAPPING INTO 

776
00:34:26,949 --> 00:34:27,383
PON

777
00:34:27,383 --> 00:34:28,751
PONCHO'S BRAIN USING THIS PORT 

778
00:34:28,751 --> 00:34:31,320
AND HE'S HAD THIS FOR QUITE SOME

779
00:34:31,320 --> 00:34:33,055
TIME BUT WE'RE MOVING TOWARDS 

780
00:34:33,055 --> 00:34:33,389
WIRELESS.

781
00:34:33,389 --> 00:34:36,992
THE SURGERY TAKES ABOUT FOUR 

782
00:34:36,992 --> 00:34:41,830
HOURS TO DO THIS BUT WE SPENT 

783
00:34:41,830 --> 00:34:44,967
HOURS, DAYS, MONTHS, YEARS 

784
00:34:44,967 --> 00:34:48,237
TRYING TO UNDERSTAND THE SIGNAL 

785
00:34:48,237 --> 00:34:48,737
GENERATED FROM IT.

786
00:34:48,737 --> 00:34:50,940
THE CHALLENGE IS TO TRANSLATE 

787
00:34:50,940 --> 00:34:56,679
THIS KIND OF SIGNAL INTO WORDS.

788
00:34:56,679 --> 00:34:59,214
IT'S IN THERE BUT ANY OF US WHO 

789
00:34:59,214 --> 00:35:00,950
STUDY THE BRAIN IN NEURAL CODE 

790
00:35:00,950 --> 00:35:02,184
IS ESSENTIALLY WHAT WE ARE 

791
00:35:02,184 --> 00:35:07,289
TRYING TO DO WHICH IS TRANSLATE 

792
00:35:07,289 --> 00:35:08,490
THE ELECTRICAL CODE INTO 

793
00:35:08,490 --> 00:35:08,991
SOMETHING INTERPRETABLE.

794
00:35:08,991 --> 00:35:19,168
THIS IS AN EXAMPLE OF WHICH 

795
00:35:19,168 --> 00:35:20,970
ELECTRODES WERE FOR TRYING TO 

796
00:35:20,970 --> 00:35:22,504
SPEECH AND THE AREA SHADED IS 

797
00:35:22,504 --> 00:35:25,307
THE ENTIRE MOTOR CORTEX HERE.

798
00:35:25,307 --> 00:35:27,710
THEN THE AREAS THAT WERE 

799
00:35:27,710 --> 00:35:28,510
IMPORTANT FOR CLASSIFICATION 

800
00:35:28,510 --> 00:35:30,079
WERE DOWN HERE IN THE CENTRAL 

801
00:35:30,079 --> 00:35:32,948
PART OF THIS MOTOR CORTEX IN 

802
00:35:32,948 --> 00:35:36,852
THIS AREA IN THE MIDDLE.

803
00:35:36,852 --> 00:35:40,422
THE FIRST 50 WORDS WE CHOSE TO 

804
00:35:40,422 --> 00:35:42,558
WORK ON SHOWED DECENT NOT 

805
00:35:42,558 --> 00:35:44,960
PERFECT, FAR FROM PERFECT 

806
00:35:44,960 --> 00:35:47,396
DECODING ACCURACY BUT IT 

807
00:35:47,396 --> 00:35:48,330
BASICALLY WORKED.

808
00:35:48,330 --> 00:35:50,165
WE COULD DISCRIMINATE DIFFERENT 

809
00:35:50,165 --> 00:35:52,801
WORDS FROM ONE ANOTHER AND NOT 

810
00:35:52,801 --> 00:35:54,603
SURPRISINGLY THE WORDS THAT ARE 

811
00:35:54,603 --> 00:35:58,507
SIMILAR LIKE WHAT AND IS ARE 

812
00:35:58,507 --> 00:36:02,244
CONFUSABLE AND THOSE DISSIMILAR 

813
00:36:02,244 --> 00:36:03,646
WERE -- THAT SHOULD MAKE SENSE 

814
00:36:03,646 --> 00:36:04,947
BASED ON THE PREVIOUS WORK WE 

815
00:36:04,947 --> 00:36:09,151
SHOWED HOW THE INFORMATION IS 

816
00:36:09,151 --> 00:36:10,653
CODED IN THAT PART OF THE BRAIN.

817
00:36:10,653 --> 00:36:13,922
WE WERE ABLE TO SEE OVER WEEKS, 

818
00:36:13,922 --> 00:36:19,962
MONTHS, NOW YEARS A STABLE 

819
00:36:19,962 --> 00:36:21,797
ACTIVATION OF THIS OVER TIME 

820
00:36:21,797 --> 00:36:24,199
BECAUSE THIS IS WHY WE CHOSE IT.

821
00:36:24,199 --> 00:36:25,934
SOMETHING CLINICALLY STABLE OVER

822
00:36:25,934 --> 00:36:28,937
SOME TIME TO BE ABLE TO SHOW 

823
00:36:28,937 --> 00:36:30,472
IT'S SOMETHING SOMEONE CAN USE 

824
00:36:30,472 --> 00:36:32,441
OVER A LONG TIME PERIOD.

825
00:36:32,441 --> 00:36:35,944
THIS IS THE DEMO VIDEO FROM 

826
00:36:35,944 --> 00:36:36,178
PONCHO.

827
00:36:36,178 --> 00:36:37,646
SOMEONE WILL WRITE TO HIM AND 

828
00:36:37,646 --> 00:36:39,281
SEE WHAT'S DECODED FROM HIS 

829
00:36:39,281 --> 00:36:40,349
BRAIN ACTIVITY.

830
00:36:40,349 --> 00:36:44,720
THIS IS THE PORT WITH THE HDMI 

831
00:36:44,720 --> 00:36:44,953
CABLE.

832
00:36:44,953 --> 00:36:48,557
THE BREAKING DOTS MEANS THIS 

833
00:36:48,557 --> 00:36:50,225
BLINKING DOTS HERE CORRESPONDS 

834
00:36:50,225 --> 00:36:51,326
SO THE ALGORITHMS DETECTED.

835
00:36:51,326 --> 00:36:53,629
HE'S TRYING TO SAY SOMETHING.

836
00:36:53,629 --> 00:36:55,364
THAT'S THE SPEECH DETECTION.

837
00:36:55,364 --> 00:36:57,032
THE REST OF THE TIME HE'S TRYING

838
00:36:57,032 --> 00:36:59,468
TO FIGURE OUT WHAT WORD HE SAID 

839
00:36:59,468 --> 00:37:03,739
AND ONCE IT FIGURES OUT THE 

840
00:37:03,739 --> 00:37:03,939
PLOT.

841
00:37:03,939 --> 00:37:06,942
NOW, IT'S TRIED TO RANK ALL 50 

842
00:37:06,942 --> 00:37:07,142
WORDS.

843
00:37:07,142 --> 00:37:10,412
IT'S JUST GOING TO SHOW THE TOP 

844
00:37:10,412 --> 00:37:20,956
RANK BUT KEEPS TRACK OF ALL 50.

845
00:37:24,760 --> 00:37:29,865
SO THIS IS DOING WORD BY WORD 

846
00:37:29,865 --> 00:37:30,766
ANALYSIS.

847
00:37:30,766 --> 00:37:34,536
CHUNK IT BY WORDS AND THE LAST 

848
00:37:34,536 --> 00:37:37,005
ONE I INCLUDED BECAUSE IT'S GOT 

849
00:37:37,005 --> 00:37:40,542
A COUPLE ERRORS BUT USES THE 

850
00:37:40,542 --> 00:37:42,578
AUTOCORRECT LANGUAGE MODEL.

851
00:37:42,578 --> 00:37:45,447
BECAUSE IT KEEPS TRACT OF THE 

852
00:37:45,447 --> 00:37:46,515
RANKED PROBABILITY IT CAN USE 

853
00:37:46,515 --> 00:37:48,450
THE CONTEXT OF THE WORDS BEFORE 

854
00:37:48,450 --> 00:37:52,454
TO UPDATE EITHER THE SPELLING OR

855
00:37:52,454 --> 00:37:54,423
SO WE'RE USING A LOT OF THESE 

856
00:37:54,423 --> 00:37:56,959
THINGS PEOPLE ARE FAMILIAR WITH 

857
00:37:56,959 --> 00:38:00,696
FROM TEXTING AND ING LIKE 

858
00:38:00,696 --> 00:38:01,697
AUTOCORRECT TO ACCELERATE CALLED

859
00:38:01,697 --> 00:38:11,974
LANGUAGE MODEL.

860
00:38:16,578 --> 00:38:19,248
AND THERE'S SIMILAR SITUATION 

861
00:38:19,248 --> 00:38:20,449
BRAIN STEM STROKE EIGHT YEARS 

862
00:38:20,449 --> 00:38:22,584
PRIOR AND IS THE COULD NOT MOVE 

863
00:38:22,584 --> 00:38:33,095
HER LIMBS AND COULD NOT SPEAK.

864
00:38:37,132 --> 00:38:38,467
>> TELL ME WHERE YOUR
LIVING 

865
00:38:38,467 --> 00:38:48,644
RIGHT NOW.

866
00:38:56,318 --> 00:39:00,489
>> ANN USES EYE TRACKING 

867
00:39:00,489 --> 00:39:01,924
TECHNOLOGY IN ORDER FOR HER TO 

868
00:39:01,924 --> 00:39:02,958
COMMUNICATE.

869
00:39:02,958 --> 00:39:05,227
SHE'S AN INCREDIBLE PERSON AND 

870
00:39:05,227 --> 00:39:07,362
RAISED TWO INCREDIBLE DAUGHTERS 

871
00:39:07,362 --> 00:39:08,797
THIS WHOLE TIME SHE WAS 

872
00:39:08,797 --> 00:39:10,699
PARALYZED.

873
00:39:10,699 --> 00:39:14,136
AN INCREDIBLE STORY.

874
00:39:14,136 --> 00:39:15,504
AND BRAIN STEM STROKE BY PART OF

875
00:39:15,504 --> 00:39:17,439
THE BRAIN AND WE WENT BACK TO 

876
00:39:17,439 --> 00:39:20,943
THE FDA AND ASKED IF WE COULD 

877
00:39:20,943 --> 00:39:27,616
INCREASE THE -- DOUBLE IT FROM 

878
00:39:27,616 --> 00:39:30,886
128 TO 153 CHANNELS.

879
00:39:30,886 --> 00:39:33,555
THIS IS REPORTING FROM THE BRAIN

880
00:39:33,555 --> 00:39:34,756
AND WE SAW BETTER PERFORMANCE 

881
00:39:34,756 --> 00:39:35,824
WITH ANN.

882
00:39:35,824 --> 00:39:39,895
ONE OF OUR INTERNAL METRICS IS 

883
00:39:39,895 --> 00:39:42,331
HAVING PEOPLE SPEAK NATO CODE 

884
00:39:42,331 --> 00:39:42,698
WORDS.

885
00:39:42,698 --> 00:39:45,367
IT'S A STANDARD TASK WE CAN 

886
00:39:45,367 --> 00:39:48,837
QUANTIFY AND COMPARE ACROSS 

887
00:39:48,837 --> 00:39:56,245
INDIVIDUALS.

888
00:39:56,245 --> 00:40:00,182
AND ANN WAS ABLE TO INCREASE 

889
00:40:00,182 --> 00:40:00,449
ACCURACY.

890
00:40:00,449 --> 00:40:04,853
WE FROZE THE CLASSIFY IER 

891
00:40:04,853 --> 00:40:08,257
ANALYZING THE DATA TWO WEEKS 

892
00:40:08,257 --> 00:40:10,259
AFTER STARTING THIS AND USED THE

893
00:40:10,259 --> 00:40:12,961
SAME ALGORITHM FOR WEEKS AND 

894
00:40:12,961 --> 00:40:15,030
MONTHS LATER AND STILL COULD GET

895
00:40:15,030 --> 00:40:15,731
GOOD PERFORMANCE.

896
00:40:15,731 --> 00:40:18,066
IT MEANS THE DECODER IS STABLE 

897
00:40:18,066 --> 00:40:20,168
OVER TIME AND THEREFORE COULD BE

898
00:40:20,168 --> 00:40:22,037
PRACTICAL FOR SOMEONE TO DO 

899
00:40:22,037 --> 00:40:25,073
WITHOUT HAVING TO TRAIN EVERY 

900
00:40:25,073 --> 00:40:35,150
DAY.

901
00:40:41,757 --> 00:40:43,992
AND WE HAD SEPARATE PARALLEL 

902
00:40:43,992 --> 00:40:46,328
A.I. TRANSLATING HER BRAIN 

903
00:40:46,328 --> 00:40:48,764
ACTIVITY INTO WORDS YOU COULD 

904
00:40:48,764 --> 00:40:50,365
HERE CALLED SPEECH SYNTHESIS.

905
00:40:50,365 --> 00:40:52,935
THEN ANOTHER ONE THAT IS 

906
00:40:52,935 --> 00:40:55,971
DECODING FACIAL MOVEMENTS 

907
00:40:55,971 --> 00:41:01,143
ASSOCIATED WITH SPEECH THROUGH 

908
00:41:01,143 --> 00:41:05,948
AN AVATAR.

909
00:41:05,948 --> 00:41:11,486
ONE WAY IS TEXT, ONE IS SPEECH 

910
00:41:11,486 --> 00:41:12,688
SYNTHESIS AND THE THIRD IS 

911
00:41:12,688 --> 00:41:13,288
EXPRESSIVE AND WHAT IT LOOKS 

912
00:41:13,288 --> 00:41:23,398
LIKE.

913
00:41:23,732 --> 00:41:27,336
WE'RE INTERESTED IN HOW HAVING 

914
00:41:27,336 --> 00:41:29,304
EMBODIMENT OF SPEECH CAN GIVE 

915
00:41:29,304 --> 00:41:31,173
FEEDBACK TO GIVE ARTICULATION.

916
00:41:31,173 --> 00:41:34,443
WHAT'S IT LIKE TO HAVE AN AVATAR

917
00:41:34,443 --> 00:41:35,777
OF YOURSELF AND SEE AND 

918
00:41:35,777 --> 00:41:37,112
UNDERSTAND WHAT'S WORKING OR NOT

919
00:41:37,112 --> 00:41:41,783
WORKING.

920
00:41:41,783 --> 00:41:45,687
WHAT WE LEARNED FROM THEM IS IT 

921
00:41:45,687 --> 00:41:46,455
GOES BEYOND THAT.

922
00:41:46,455 --> 00:41:48,523
BEING ABLE TO CHOOSE THE EYE 

923
00:41:48,523 --> 00:41:51,193
COLOR OR HAIR COLOR, FEATURES OF

924
00:41:51,193 --> 00:41:56,631
THEIR AVATAR WAS RESTORING 

925
00:41:56,631 --> 00:41:56,898
IDENTITY.

926
00:41:56,898 --> 00:42:00,969
PEOPLE WHO HAVE LOST ABILITY TO 

927
00:42:00,969 --> 00:42:02,371
COMMUNICATE HAVE NOT JUST LOST 

928
00:42:02,371 --> 00:42:12,881
THEIR VOICE BUT IDENTITY AND 

929
00:42:21,023 --> 00:42:23,291
THIS IS THE MODEL WE BUILT AND 

930
00:42:23,291 --> 00:42:24,926
HAVE A NETWORK FROM BRAIN 

931
00:42:24,926 --> 00:42:29,931
ACTIVITY TO CODING THE DIFFERENT

932
00:42:29,931 --> 00:42:31,033
PHONETIC ELEMENTS.

933
00:42:31,033 --> 00:42:34,870
WE ARRANGED THE PHONETIC 

934
00:42:34,870 --> 00:42:37,539
ELEMENTS INTO WORD FORMS AND 

935
00:42:37,539 --> 00:42:48,016
THEN WE PROCESS THIS INTO A TEXT

936
00:42:48,016 --> 00:42:49,051
AND GO FROM BRAIN ACTIVITY TO 

937
00:42:49,051 --> 00:42:59,161
TEXT.

938
00:43:01,696 --> 00:43:04,566
WE'RE GIVING A COUNT DOWN AND 

939
00:43:04,566 --> 00:43:06,468
ASKING HER TO ARTICULATE THE 

940
00:43:06,468 --> 00:43:07,702
WORDS AND SENTENCES.

941
00:43:07,702 --> 00:43:12,941
BECAUSE WITH HAVE MORE DATA AND 

942
00:43:12,941 --> 00:43:23,485
INFORMATION WE ASKED HER SPEAK 

943
00:43:29,458 --> 00:43:31,093
AND THE ANALYSIS IS NOW 

944
00:43:31,093 --> 00:43:32,494
ANALYZING THE WHOLE PHRASE IN 

945
00:43:32,494 --> 00:43:41,937
THE SENTENCE NOT WORD BY WORD.

946
00:43:41,937 --> 00:43:44,606
YOU CAN SEE IT'S A LOT FASTER 

947
00:43:44,606 --> 00:43:50,078
WITH 75% ACCURACY.

948
00:43:50,078 --> 00:43:53,849
FASTER WITH VOCABULARY AND 1500 

949
00:43:53,849 --> 00:43:54,416
WORDS WHICH CAN EXPAND TO 

950
00:43:54,416 --> 00:44:04,259
SEVERAL THOUSAND.

951
00:44:04,259 --> 00:44:06,661
245 WAS SUPER EXCITING TO SEE.

952
00:44:06,661 --> 00:44:14,402
WHAT ABOUT THE SPEECH SYNTHESIS?

953
00:44:14,402 --> 00:44:16,905
WE TRANSLATED TO UNITS OF 

954
00:44:16,905 --> 00:44:19,741
SPEECH.

955
00:44:19,741 --> 00:44:22,911
LATE ENT UNITS LEARNED FROM A 

956
00:44:22,911 --> 00:44:24,279
SPEECH RECOGNITION MODEL 

957
00:44:24,279 --> 00:44:27,048
PRODUCED BY META.

958
00:44:27,048 --> 00:44:28,517
WE WERE ABLE TO TRANSLATE THOSE 

959
00:44:28,517 --> 00:44:31,953
TO SPEECH YOU CAN HEAR.

960
00:44:31,953 --> 00:44:34,222
THIS PARTICULAR CASE WE USED 

961
00:44:34,222 --> 00:44:39,194
ANN'S ORIGINAL VOICE FROM HER 

962
00:44:39,194 --> 00:44:43,832
WEDDING WE HAD ACCESS TO AND 

963
00:44:43,832 --> 00:44:47,936
QUASI-FIT IT TO HER AND USING AN

964
00:44:47,936 --> 00:44:51,740
AVATAR MACHINE LEARNING METHOD.

965
00:44:51,740 --> 00:44:52,874
>> YOU ARE WONDERFUL.

966
00:44:52,874 --> 00:45:03,385
>> SO IT'S DECODING TEXT
AND 

967
00:45:05,020 --> 00:45:12,761
MOVEMENTS SIMULTANEOUSLY.

968
00:45:12,761 --> 00:45:13,395
UNCANNY, RIGHT?

969
00:45:13,395 --> 00:45:17,699
KIND OF STRANGE TO SEE SOMEONE 

970
00:45:17,699 --> 00:45:22,637
COMMUNICATING IN THAT WAY.

971
00:45:22,637 --> 00:45:24,639
VERY SURREAL TOO.

972
00:45:24,639 --> 00:45:27,409
BECAUSE WE BUILT AN AVATAR I 

973
00:45:27,409 --> 00:45:30,378
ASKED THE TEAM LET'S USE FACIAL 

974
00:45:30,378 --> 00:45:30,679
EXPRESSIONS.

975
00:45:30,679 --> 00:45:33,615
THAT WORKS WELL.

976
00:45:33,615 --> 00:45:36,051
YOU CAN CLASSIFY EMOTIONAL 

977
00:45:36,051 --> 00:45:36,718
EXPRESSIONS.

978
00:45:36,718 --> 00:45:40,188
HERE'S EXAMPLES OF ANN 

979
00:45:40,188 --> 00:45:48,396
EXPRESSING A FAKE SMILE.

980
00:45:48,396 --> 00:45:49,998
>> ALL THESE LINK BACK.

981
00:45:49,998 --> 00:45:54,135
AS SHE IS MAKING THESE ATTEMPTED

982
00:45:54,135 --> 00:45:56,037
SPEECH MOVEMENTS WE CAN GO BACK 

983
00:45:56,037 --> 00:45:59,007
TO THE FIRST PRINCIPLES WE 

984
00:45:59,007 --> 00:46:02,477
DISCOVERED 10, 15 YEARS AGO 

985
00:46:02,477 --> 00:46:03,445
ABOUT THIS.

986
00:46:03,445 --> 00:46:05,714
THE CLEARANCE, THE HAND AREA, 

987
00:46:05,714 --> 00:46:07,649
THE LIPS, TONGUE, ETCETERA.

988
00:46:07,649 --> 00:46:10,685
IT'S ALL RELATED.

989
00:46:10,685 --> 00:46:21,363
THE SAME PRINCIPLES INVOLVED AND

990
00:46:25,867 --> 00:46:29,738
WHEN PONCHO HAD HIS ACCIDENT HE 

991
00:46:29,738 --> 00:46:34,442
WAS A SPANISH SPEAKER AND WE 

992
00:46:34,442 --> 00:46:37,779
BUILT A BILINGUAL DECODER AND 

993
00:46:37,779 --> 00:46:39,714
HAS A LANGUAGE MODEL SPECIFIC TO

994
00:46:39,714 --> 00:46:43,685
ENGLISH OR SPANISH AND ONLY 

995
00:46:43,685 --> 00:46:45,687
PRESENTS THE ONE THAT DETERMINES

996
00:46:45,687 --> 00:46:48,556
ENGLISH OR SPANISH.

997
00:46:48,556 --> 00:46:51,960
SO IT'S THE BRAIN AREA BUT 

998
00:46:51,960 --> 00:46:57,599
DETECTING IF THE SEQUENCE OF 

999
00:46:57,599 --> 00:46:59,034
ELEMENTS CORRESPONDS AND 

1000
00:46:59,034 --> 00:47:09,277
DISTINGUISHES.

1001
00:47:17,719 --> 00:47:19,387
SO THAT'S AN EXAMPLE OF ENGLISH.

1002
00:47:19,387 --> 00:47:23,591
I'LL GIVE ANOTHER IN SPANISH.

1003
00:47:23,591 --> 00:47:26,761
WE DON'T TELL THE DECODER.

1004
00:47:26,761 --> 00:47:30,632
IT HAS TO MOVE FORWARD WITH 

1005
00:47:30,632 --> 00:47:35,470
THESE KINDS OF SEQUENCES IN 

1006
00:47:35,470 --> 00:47:36,104
PHONETIC ELEMENTS IN THE BRAIN 

1007
00:47:36,104 --> 00:47:46,281
ACTIVITY.

1008
00:47:47,415 --> 00:47:47,549
OKAY.

1009
00:47:47,549 --> 00:47:51,419
ANOTHER THING NOT PUBLISHED YET 

1010
00:47:51,419 --> 00:47:54,289
BUT ALREADY IN PRESS FROM TWO 

1011
00:47:54,289 --> 00:47:57,192
GRADUATE STUDENTS BUT HERE WE'RE

1012
00:47:57,192 --> 00:47:59,494
TRYING TO MAKE THE SYNTHESIS 

1013
00:47:59,494 --> 00:48:00,328
FASTER.

1014
00:48:00,328 --> 00:48:04,666
IF YOU NOTICE IN THIS EXAMPLE 

1015
00:48:04,666 --> 00:48:06,901
THERE'S A LONG LATENCY.

1016
00:48:06,901 --> 00:48:09,704
WE'VE TRIED TO REDUCE THAT TO A 

1017
00:48:09,704 --> 00:48:11,539
SECOND AND WE'RE DOING REAL-TIME

1018
00:48:11,539 --> 00:48:11,806
SYNTHESIS.

1019
00:48:11,806 --> 00:48:22,250
HERE'S AN EXAMPLE OF THAT.

1020
00:48:22,450 --> 00:48:32,894
>> I HAVE AN APPOINTMENT
NOW.

1021
00:48:34,529 --> 00:48:36,498
>> LET ME TELL YOU -- 

1022
00:48:36,498 --> 00:48:40,435
>> THAT'S WHAT WE CALL
EXTREMING

1023
00:48:40,435 --> 00:48:40,702
SYNTHESIS.

1024
00:48:40,702 --> 00:48:45,407
WE CAN TAKE IT OUT EVERY MONTH, 

1025
00:48:45,407 --> 00:48:55,350
WEEK, WHATEVER.

1026
00:48:55,350 --> 00:49:01,122
THIS FIELD IS EXPLODING AND 

1027
00:49:01,122 --> 00:49:08,797
THERE'S INCREDIBLE RESULTS A.

1028
00:49:08,797 --> 00:49:10,031
OTHER GROUPS AS WELL THE NEW 

1029
00:49:10,031 --> 00:49:11,099
ENGLAND JOURNAL ASKED ME TO 

1030
00:49:11,099 --> 00:49:14,636
WRITE A REVIEW AND I PUT FORWARD

1031
00:49:14,636 --> 00:49:16,204
SOME CRITERIA WHAT TO LOOK FOR.

1032
00:49:16,204 --> 00:49:17,772
WE WANT IT TO BE HIGH 

1033
00:49:17,772 --> 00:49:21,609
PERFORMANCE, FAST, ACCURATE, 

1034
00:49:21,609 --> 00:49:23,978
SOMETHING EASY TO USE.

1035
00:49:23,978 --> 00:49:26,915
WE WANT IT ROBUST.

1036
00:49:26,915 --> 00:49:32,420
WE DON'T WANT TO SPEND HOURS 

1037
00:49:32,420 --> 00:49:32,720
CALIBRATING.

1038
00:49:32,720 --> 00:49:35,690
WE WANT IT STABLE AND USE 

1039
00:49:35,690 --> 00:49:38,359
DIFFERENT SETTINGS AND DISEASE 

1040
00:49:38,359 --> 00:49:39,260
STAGES.

1041
00:49:39,260 --> 00:49:44,732
WE WANT IT TO BE HIGH LATENCY.

1042
00:49:44,732 --> 00:49:51,473
WE WANT IT TO BE FULLY 

1043
00:49:51,473 --> 00:49:52,941
IMPLEMENTABLE AND WIRELESS AND 

1044
00:49:52,941 --> 00:49:54,375
SURGICALLY AS EASY AS POSSIBLE 

1045
00:49:54,375 --> 00:49:56,678
NOT ONLY TO IMPLANT BUT ALSO TO 

1046
00:49:56,678 --> 00:49:57,312
REMOVE.

1047
00:49:57,312 --> 00:50:02,283
SO THIS IS THE WISH LIST.

1048
00:50:02,283 --> 00:50:04,786
HERE'S SOME OPTIMIZATIONS.

1049
00:50:04,786 --> 00:50:08,656
WE CAN PREDICT HOW TO GET HIGH 

1050
00:50:08,656 --> 00:50:11,092
ACCURACY FROM PAPERS AND OTHERS 

1051
00:50:11,092 --> 00:50:16,631
AND PROJECTIONS ARE VERY SIMILAR

1052
00:50:16,631 --> 00:50:18,967
THE NUMBER OF SENSORS MATTERS 

1053
00:50:18,967 --> 00:50:21,002
AND DENSITY MATTERS.

1054
00:50:21,002 --> 00:50:23,238
WE NOW HAVE THE DATA TO MAKE 

1055
00:50:23,238 --> 00:50:25,406
THOSE KINDS OF PROJECTIONS AND 

1056
00:50:25,406 --> 00:50:28,443
MAYBE OPTIMIZATION.

1057
00:50:28,443 --> 00:50:30,411
I'LL SUMMARIZE HERE AND GET TO 

1058
00:50:30,411 --> 00:50:32,313
BIG PICTURE.

1059
00:50:32,313 --> 00:50:34,449
SO WHAT I'VE DESCRIBED HERE IS A

1060
00:50:34,449 --> 00:50:38,553
PATH THAT WE'VE TAKEN FROM THE 

1061
00:50:38,553 --> 00:50:41,256
BASIC SCIENCE OF THE CORTICAL 

1062
00:50:41,256 --> 00:50:42,757
PHYSIOLOGY OF THE WAY WE SPEAK.

1063
00:50:42,757 --> 00:50:44,392
IT GOES DEEP.

1064
00:50:44,392 --> 00:50:46,294
I'VE ONLY SHOWN SOME HIGHLIGHTS 

1065
00:50:46,294 --> 00:50:51,900
RELATED TO THIS MAP OF AUDITORY 

1066
00:50:51,900 --> 00:50:53,268
MOVEMENTS AND THE LARYNGEAL 

1067
00:50:53,268 --> 00:50:55,270
CORTEX THAT WAS UNDER 

1068
00:50:55,270 --> 00:50:55,570
APPRECIATED.

1069
00:50:55,570 --> 00:50:57,739
HOW MOVEMENTS ARE ENCODED BY 

1070
00:50:57,739 --> 00:50:58,439
CORTICAL ACTIVITY IN THIS PART 

1071
00:50:58,439 --> 00:50:59,674
OF OUR BRAIN.

1072
00:50:59,674 --> 00:51:04,946
WHEN HAVE YOU A STROKE OR 

1073
00:51:04,946 --> 00:51:08,883
NORTH -- INJURY YOU HAVE THE 

1074
00:51:08,883 --> 00:51:15,323
LOSS OF THE ABILITY TO TRANSLATE

1075
00:51:15,323 --> 00:51:23,264
AND THIS IS THE ERA OF A.I. AND 

1076
00:51:23,264 --> 00:51:29,604
WE CAN EMBODY FORMS FOR VERBAL 

1077
00:51:29,604 --> 00:51:33,274
AND NON-VERBAL COMMUNICATION.

1078
00:51:33,274 --> 00:51:39,547
WE NOW BASIC PARAMETERS TO 

1079
00:51:39,547 --> 00:51:39,814
OPTIMIZE.

1080
00:51:39,814 --> 00:51:42,617
IT'S NOW A QUESTION OF HOW GOOD 

1081
00:51:42,617 --> 00:51:43,885
CAN WE MAKE IT.

1082
00:51:43,885 --> 00:51:47,822
IT'S NOT ABOUT IF IT'S POSSIBLE 

1083
00:51:47,822 --> 00:51:51,192
AND WE'RE VERY FOCUSSED ON THAT.

1084
00:51:51,192 --> 00:51:56,764
SO, BIG PICTURE, I WANTED -- I 

1085
00:51:56,764 --> 00:52:00,935
THINK IT'S VERY IMPORTANT TO ME 

1086
00:52:00,935 --> 00:52:04,672
PERSONALLY TO BE HERE TODAY 

1087
00:52:04,672 --> 00:52:05,540
BECAUSE NONE OF THIS WOULD HAVE 

1088
00:52:05,540 --> 00:52:08,943
BEEN POSSIBLE WITHOUT THE 

1089
00:52:08,943 --> 00:52:09,377
SUPPORT OF THE NIH.

1090
00:52:09,377 --> 00:52:15,717
I FEEL EXTREMELY LUCKY TO BE A 

1091
00:52:15,717 --> 00:52:18,786
BENEFACTOR TO COLLABORATE WITH 

1092
00:52:18,786 --> 00:52:21,589
NIH AT LARGE AND NICHD AND ONE 

1093
00:52:21,589 --> 00:52:23,291
OF THE LUCKIEST PEOPLE IN THE 

1094
00:52:23,291 --> 00:52:24,592
WORLD TO DO THIS WORK THAT'S 

1095
00:52:24,592 --> 00:52:26,961
BEEN SO FULFILLING AND 

1096
00:52:26,961 --> 00:52:27,261
PRODUCTIVE.

1097
00:52:27,261 --> 00:52:31,199
I DO WANT TO CALL OUT ROGER 

1098
00:52:31,199 --> 00:52:40,208
MILLER WHO IS HERE.

1099
00:52:40,208 --> 00:52:48,149
THERE WAS A NIDC WORKSHOP AND I 

1100
00:52:48,149 --> 00:52:50,051
REMEMBER READING THE SUMMARY AND

1101
00:52:50,051 --> 00:52:52,120
I WASN'T ELIGIBLE TO ATTEND BUT 

1102
00:52:52,120 --> 00:52:54,322
I REMEMBER READING THE ARCHIVE 

1103
00:52:54,322 --> 00:52:56,157
SUMMARY AND THINKING, WOW, THIS 

1104
00:52:56,157 --> 00:52:58,660
IS SUPER EXCITING THIS IS EVEN 

1105
00:52:58,660 --> 00:53:00,128
POSSIBLE AND IT PLANTED A SEED 

1106
00:53:00,128 --> 00:53:02,196
IN MY MIND LIKE I WANT TO WORK 

1107
00:53:02,196 --> 00:53:06,901
ON SOMETHING RELATE TO THAT AND 

1108
00:53:06,901 --> 00:53:09,070
IT TOOK A WHILE BUT IF YOU 

1109
00:53:09,070 --> 00:53:11,205
NOTICE THIS IS 2006.

1110
00:53:11,205 --> 00:53:16,477
THAT'S ALMOST 20 YEARS AGO.

1111
00:53:16,477 --> 00:53:21,349
SO THANK YOU FOR THE VISION AND 

1112
00:53:21,349 --> 00:53:21,716
FORESIGHT.

1113
00:53:21,716 --> 00:53:23,284
A DECADE AND A HALF LATER WE 

1114
00:53:23,284 --> 00:53:26,387
WERE ABLE TO GET SUPPORTED BY A 

1115
00:53:26,387 --> 00:53:29,524
CLINICAL TRIAL PROGRAM THROUGH 

1116
00:53:29,524 --> 00:53:31,726
NIDCD AND SUPPORT OUR PROGRAM.

1117
00:53:31,726 --> 00:53:34,595
IT WAS THE FIRST PILOT CLINICAL 

1118
00:53:34,595 --> 00:53:36,130
TRIAL FOR SPEECH PROSTHESIS.

1119
00:53:36,130 --> 00:53:39,000
WHAT I JUST SHOWED YOU IN THE 

1120
00:53:39,000 --> 00:53:41,169
TRANSLATION WAS REALLY SPONSORED

1121
00:53:41,169 --> 00:53:46,307
BY THE WORK OF NIDCD AND NIH.

1122
00:53:46,307 --> 00:53:48,943
SO THIS TIME LINE IS MY 

1123
00:53:48,943 --> 00:53:52,146
REFLECTION OF WHAT MY CAREER AND

1124
00:53:52,146 --> 00:53:54,582
OUR WORK HAS DONE AND THERE'S 

1125
00:53:54,582 --> 00:53:56,884
MANY OTHER LABS THAT HAVE THIS 

1126
00:53:56,884 --> 00:53:58,786
PARALLEL KIND OF TRAJECTORY.

1127
00:53:58,786 --> 00:54:01,823
THE FIRST DESCRIPTION OF THE 

1128
00:54:01,823 --> 00:54:04,792
SPEECH AND MOTOR CORTEX WAS IN 

1129
00:54:04,792 --> 00:54:07,095
2013 SPONSORED BY AN INNOVATOR 

1130
00:54:07,095 --> 00:54:10,098
AWARD FROM THE DIRECTOR'S OFFICE

1131
00:54:10,098 --> 00:54:12,233
CALLED FUNCTIONAL ARCHITECTURE 

1132
00:54:12,233 --> 00:54:13,901
OF SPEECH MOTOR CORTEX AND US 

1133
00:54:13,901 --> 00:54:16,070
WITH ABLE TO DOUBLE DOWN ON THAT

1134
00:54:16,070 --> 00:54:19,107
WITH THE NIH BRAIN INITIATIVE 

1135
00:54:19,107 --> 00:54:24,946
AND GAVE US CODING FOR CONSO

1136
00:54:24,946 --> 00:54:28,649
CONSONANTS AND VOWELS AND WERE 

1137
00:54:28,649 --> 00:54:32,053
ABLE TO APPLY MORE A.I. AND THEN

1138
00:54:32,053 --> 00:54:35,289
ABLE TO BUILD THE EVIDENCE AND 

1139
00:54:35,289 --> 00:54:39,927
CASE FOR MOVING TO A PATIENT 

1140
00:54:39,927 --> 00:54:43,898
AREA AND TAKING IT OUT OF 

1141
00:54:43,898 --> 00:54:45,333
EPILEPSY MONITORING AND MOVING 

1142
00:54:45,333 --> 00:54:48,936
IT TO PEOPLE WHO ARE SUFFERING 

1143
00:54:48,936 --> 00:54:54,408
AND GET SYNTHESIS SUPPORTED AND 

1144
00:54:54,408 --> 00:54:57,645
NOW A NEW UGA-3 SUPPORTING 

1145
00:54:57,645 --> 00:54:59,180
DEVELOPMENT OF THE WILL LAFRD 

1146
00:54:59,180 --> 00:54:59,380
WEAR.

1147
00:54:59,380 --> 00:55:00,948
A TIRELESS SYSTEM SO WE DON'T 

1148
00:55:00,948 --> 00:55:04,919
HAVE TO DEAL WITH THIS CRAZY 

1149
00:55:04,919 --> 00:55:05,720
PORT.

1150
00:55:05,720 --> 00:55:07,522
SUPER IMPORTANT TO DO THE 

1151
00:55:07,522 --> 00:55:09,557
INITIAL DERISKING BUT NOW WE 

1152
00:55:09,557 --> 00:55:11,259
HAVE TO USE BETTER TECHNOLOGY.

1153
00:55:11,259 --> 00:55:14,128
WE'RE HOPING FOR THE FIRST IN 

1154
00:55:14,128 --> 00:55:15,930
HUMAN DEVICE IN THE NEXT TWO 

1155
00:55:15,930 --> 00:55:16,130
YEARS.

1156
00:55:16,130 --> 00:55:17,799
KNOWLEDGE OF THIS WOULD HAVE 

1157
00:55:17,799 --> 00:55:20,401
BEEN POSSIBLE WITHOUT THE 

1158
00:55:20,401 --> 00:55:26,140
SUPPORT OF THE NIH OR NIDCD AND 

1159
00:55:26,140 --> 00:55:26,741
BRAIN INITIATIVE.

1160
00:55:26,741 --> 00:55:27,975
THE LAST THING I WANT TO 

1161
00:55:27,975 --> 00:55:29,177
DESCRIBE IS WHERE WE'RE GOING 

1162
00:55:29,177 --> 00:55:32,146
WITH THE SCIENCE, FULL CIRCLE.

1163
00:55:32,146 --> 00:55:32,947
TALKING ABOUT SCIENCE TO 

1164
00:55:32,947 --> 00:55:34,782
TRANSLATION BACK TO SCIENCE.

1165
00:55:34,782 --> 00:55:40,054
WE'RE USING A NEW TOOL CALLED 

1166
00:55:40,054 --> 00:55:44,926
NEURAL PIXELS A WAY TO STUDY 

1167
00:55:44,926 --> 00:55:47,228
NEURONAL ACTIVITY AND A FRIEND 

1168
00:55:47,228 --> 00:55:48,863
IS A PIONEER IN RECORDING THIS 

1169
00:55:48,863 --> 00:55:51,165
IN HUMAN USING A SLIGHTLY 

1170
00:55:51,165 --> 00:55:53,134
DIFFERENT TECHNOLOGY ALONG THE 

1171
00:55:53,134 --> 00:55:55,436
LINEAR ARRAY.

1172
00:55:55,436 --> 00:55:57,305
THIS AREA WE DESCRIBED A LOT IN 

1173
00:55:57,305 --> 00:56:05,913
THE LAST DAY AND A HALF USING 

1174
00:56:05,913 --> 00:56:07,882
ELECTROCARDIOGRAPHY FOR RARE 

1175
00:56:07,882 --> 00:56:09,317
CASES WHERE THERE'S ACCESS BUT 

1176
00:56:09,317 --> 00:56:10,718
OVER THE YEARS WE'VE LAD RAY 

1177
00:56:10,718 --> 00:56:11,118
COUPLE.

1178
00:56:11,118 --> 00:56:14,522
WHAT YOU'RE LOOKING AT IS 

1179
00:56:14,522 --> 00:56:17,391
UNPUBLISHED WORK WHERE FROM A 

1180
00:56:17,391 --> 00:56:22,430
SINGLE NEURAL PIXEL WE CAN PULL 

1181
00:56:22,430 --> 00:56:25,766
NEURONS AND THIS IS WHAT THEY'RE

1182
00:56:25,766 --> 00:56:27,668
DOING IN A TASK WHERE PEOPLE 

1183
00:56:27,668 --> 00:56:35,042
HAEFR A WO -- HEAR A WORD AND TN

1184
00:56:35,042 --> 00:56:38,246
GIVEN A PROMPT TO STREAK 37 IN 

1185
00:56:38,246 --> 00:56:39,780
THE CORTEX IT'S NOT JUST 

1186
00:56:39,780 --> 00:56:41,983
ACTIVATED DURING SPEAKING BUT A 

1187
00:56:41,983 --> 00:56:44,585
SUBSET OF THE CELLS ARE 

1188
00:56:44,585 --> 00:56:45,219
PRIMARILY RESPONDING WHEN THEY 

1189
00:56:45,219 --> 00:56:47,822
HEAR THE WORK.

1190
00:56:47,822 --> 00:56:51,559
SO THIS IS NOW THE CELLULAR 

1191
00:56:51,559 --> 00:56:54,295
LEVEL SO THERE ARE CELLULAR SUB 

1192
00:56:54,295 --> 00:56:55,696
POPULATIONS THAT ARE VERY MUCH 

1193
00:56:55,696 --> 00:56:57,665
RESPONSIVE WHEN YOU HEAR A WORD 

1194
00:56:57,665 --> 00:56:59,867
AND OTHER POPULATIONS PREPARING 

1195
00:56:59,867 --> 00:57:03,070
TO SPEAK THE WORD AND THEN 

1196
00:57:03,070 --> 00:57:03,604
EXECUTING.

1197
00:57:03,604 --> 00:57:05,840
YOU CAN LOOK AT OTHER PARTS OF 

1198
00:57:05,840 --> 00:57:08,276
THE GYRUS AND SEE SOMETHING 

1199
00:57:08,276 --> 00:57:08,609
SIMILAR.

1200
00:57:08,609 --> 00:57:13,314
IF YOU LOOK AT THIS AREA, IT 

1201
00:57:13,314 --> 00:57:14,715
DOESN'T LOOK LIKE THERE'S MUCH 

1202
00:57:14,715 --> 00:57:16,951
INTERESTING GOING ON THERE AT 

1203
00:57:16,951 --> 00:57:17,251
ALL.

1204
00:57:17,251 --> 00:57:18,753
LOOKING AT THE AUDITORY CORTEX 

1205
00:57:18,753 --> 00:57:21,889
YOU HEAR RESPONSES TO HEARING 

1206
00:57:21,889 --> 00:57:23,591
THE WORDS AND STRONG RESPONSES 

1207
00:57:23,591 --> 00:57:28,930
TO HEARING THE WORD.

1208
00:57:28,930 --> 00:57:32,667
THE CELLULAR LEVEL HAS DOZENS OF

1209
00:57:32,667 --> 00:57:35,603
NEURONS RECORDED SIMULTANEOUSLY.

1210
00:57:35,603 --> 00:57:36,170
IT'S INCREDIBLE TECHNOLOGY.

1211
00:57:36,170 --> 00:57:40,942
AND IF YOU LOOK AT THIS OVER A 

1212
00:57:40,942 --> 00:57:42,910
MULTIPLE DATA SETS WE'RE TALKING

1213
00:57:42,910 --> 00:57:44,078
ABOUT SINGLE NEURON AT SCALE.

1214
00:57:44,078 --> 00:57:45,846
FOR EXAMPLE, THIS IS AN EXAMPLE 

1215
00:57:45,846 --> 00:57:49,150
OF A COUPLE OF DATA SETS 

1216
00:57:49,150 --> 00:57:51,819
TOGETHER, 500 UNITS ALONG THE 

1217
00:57:51,819 --> 00:57:52,954
DIFFERENT PHASES ACROSS THE 

1218
00:57:52,954 --> 00:57:53,788
DIFFERENT BRAIN AREAS.

1219
00:57:53,788 --> 00:57:56,757
SO WE'RE NOW STARTING TO ENTER 

1220
00:57:56,757 --> 00:57:58,392
THIS NEW PHASE OF HUMAN 

1221
00:57:58,392 --> 00:57:59,293
NEUROSCIENCE WHERE WE CANNOT 

1222
00:57:59,293 --> 00:58:01,128
ONLY DO THE SAME BUT AT SCALE.

1223
00:58:01,128 --> 00:58:04,966
WE CAN GET A LARGE POPULATION TO

1224
00:58:04,966 --> 00:58:08,369
UNDERSTAND MORE HOLISTICALLY HOW

1225
00:58:08,369 --> 00:58:10,071
THE POPULATION REPRESENTS 

1226
00:58:10,071 --> 00:58:10,371
INFORMATION.

1227
00:58:10,371 --> 00:58:14,075
ONE THING SUPER EXCITING IS NOT 

1228
00:58:14,075 --> 00:58:19,046
JUST IN THE BUT  POPULATION BUT 

1229
00:58:19,046 --> 00:58:20,948
DIFFERENT EXAMPLES AND NEURONS 

1230
00:58:20,948 --> 00:58:23,484
IN THE GYRUS AND WE'VE BEEN 

1231
00:58:23,484 --> 00:58:26,554
DESCRIBING THE DECODING AND IT'S

1232
00:58:26,554 --> 00:58:29,156
MORE DIVERSE AT THE CELLULAR 

1233
00:58:29,156 --> 00:58:29,590
LEVEL.

1234
00:58:29,590 --> 00:58:32,960
ONE NEURON IS ONLY ACTIVE IN 

1235
00:58:32,960 --> 00:58:35,496
LISTENING TO THIS AND HERE'S 

1236
00:58:35,496 --> 00:58:36,597
ANOTHER NEURON ACTIVATED WITH 

1237
00:58:36,597 --> 00:58:38,766
THE SPEAKING.

1238
00:58:38,766 --> 00:58:40,434
THIS IS ANOTHER SUPER 

1239
00:58:40,434 --> 00:58:42,570
INTERESTING ONE WHERE IT'S 

1240
00:58:42,570 --> 00:58:44,939
ACTIVE DURING BOTH LISTENING AND

1241
00:58:44,939 --> 00:58:48,409
SPEAKING.

1242
00:58:48,409 --> 00:58:50,177
BUT FOR THE SAME WORD.

1243
00:58:50,177 --> 00:58:55,916
SO IT'S ACTIVATED WHEN YOU HEAR 

1244
00:58:55,916 --> 00:58:58,986
A WORD AND REACTIVATED AND THEN 

1245
00:58:58,986 --> 00:59:00,254
ANOTHER SET ACTIVATE THE WHEN 

1246
00:59:00,254 --> 00:59:04,225
THEY GET THE GO WITH QUEUE AND 

1247
00:59:04,225 --> 00:59:04,992
ANOTHER ACTIVATED WHEN THEY HEAR

1248
00:59:04,992 --> 00:59:11,999
A WORD AND LISTEN AND STAYS ACT 

1249
00:59:11,999 --> 00:59:12,967
VA

1250
00:59:12,967 --> 00:59:16,203
 -- ACTIVATED UNTIL YOU SAY IT.

1251
00:59:16,203 --> 00:59:18,439
IT'S ALL HAPPENING IN THE 

1252
00:59:18,439 --> 00:59:21,142
CELLULAR POPULATION.

1253
00:59:21,142 --> 00:59:24,478
THESE ARE WHAT WE CALL THE 

1254
00:59:24,478 --> 00:59:25,746
MIRROR NEURONS AND FIGURE OUT 

1255
00:59:25,746 --> 00:59:30,651
WHAT WORD THEY'RE ENCODING.

1256
00:59:30,651 --> 00:59:34,789
HERE THEY'RE RESPONDING AND HERE

1257
00:59:34,789 --> 00:59:36,724
RESPONDING TO THE PRODUCTION OF 

1258
00:59:36,724 --> 00:59:37,258
THE WORD.

1259
00:59:37,258 --> 00:59:38,959
HERE'S AN EXAMPLE OF WHAT WE 

1260
00:59:38,959 --> 00:59:39,894
CALL BRIDGE NEURONS.

1261
00:59:39,894 --> 00:59:41,996
THEY'RE ACTIVATED BY HEARING 

1262
00:59:41,996 --> 00:59:43,264
WORD AND THEY STAY ACTIVATED 

1263
00:59:43,264 --> 00:59:49,336
UNTIL YOU SAY THE WORD.

1264
00:59:49,336 --> 00:59:51,372
REALLY INTERESTING PHENOMENON 

1265
00:59:51,372 --> 00:59:53,374
AND PROBABLY HAS A LOT OF 

1266
00:59:53,374 --> 00:59:54,241
IMPLICATIONS FOR UNDERSTANDING 

1267
00:59:54,241 --> 00:59:58,279
HOW WE LEARN TO SPEAK AND OTHER 

1268
00:59:58,279 --> 00:59:58,579
PHENOMENON.

1269
00:59:58,579 --> 01:00:01,849
INTEGRATION HOW WE HEAR AND 

1270
01:00:01,849 --> 01:00:04,485
SPEAK AT A CELLULAR LEVEL AT THE

1271
01:00:04,485 --> 01:00:05,453
PREMOTOR CORTEX.

1272
01:00:05,453 --> 01:00:09,523
I JUST WANT TO FINISH HERE AND 

1273
01:00:09,523 --> 01:00:11,725
THANK THE PEOPLE IN MY LAB AND 

1274
01:00:11,725 --> 01:00:14,061
THANK ALL OF YOU AT NIH.

1275
01:00:14,061 --> 01:00:16,464
IT'S BEEN AN AMAZING 

1276
01:00:16,464 --> 01:00:18,666
COLLABORATION WITH THE INSTITUTE

1277
01:00:18,666 --> 01:00:23,804
AND BRAIN INITIATIVE AND NIDCD.

1278
01:00:23,804 --> 01:00:24,738
THANK YOU SO MUCH FOR ENABLING 

1279
01:00:24,738 --> 01:00:32,379
ALL THIS WORK.

1280
01:00:32,379 --> 01:00:36,517
 HAPPY TO TAKE SOME QUESTIONS.

1281
01:00:36,517 --> 01:00:42,690
WE HAVE A FEW ONLINE.

1282
01:00:42,690 --> 01:00:44,258
>> HI, THANK YOU FOR THE
TALK.

1283
01:00:44,258 --> 01:00:46,627
I HAVE TWO QUESTIONS IF THAT'S 

1284
01:00:46,627 --> 01:00:46,827
OKAY.

1285
01:00:46,827 --> 01:00:49,063
THE FIRST ONE IS LIKE WHEN I'M 

1286
01:00:49,063 --> 01:00:51,232
TRYING TO SAY SOMETHING TO 

1287
01:00:51,232 --> 01:00:53,200
SOMEBODY BUT HAVE PASSING 

1288
01:00:53,200 --> 01:00:55,202
THOUGHTS I DON'T MEAN TO 

1289
01:00:55,202 --> 01:00:58,038
VOCALIZE LIKE I'M TIRED OR 

1290
01:00:58,038 --> 01:01:00,074
SOMETHING IF SOMEONE ASKS HOW 

1291
01:01:00,074 --> 01:01:02,443
I'M DOING WELL I'LL SAY WELL BUT

1292
01:01:02,443 --> 01:01:04,478
I'M SLEEPY.

1293
01:01:04,478 --> 01:01:05,846
WITH THESE PATIENTS SINCE 

1294
01:01:05,846 --> 01:01:09,550
THEY'RE NOT TRYING TO VOCALIZE 

1295
01:01:09,550 --> 01:01:11,418
THAT THOUGHT WOULD IT NOT COME 

1296
01:01:11,418 --> 01:01:11,585
OUT.

1297
01:01:11,585 --> 01:01:13,420
>> YEAH, WE'VE ACTUALLY
TRIED 

1298
01:01:13,420 --> 01:01:13,654
THAT.

1299
01:01:13,654 --> 01:01:16,290
WE TRIED TO DECODE WHAT THINGS 

1300
01:01:16,290 --> 01:01:19,660
PEOPLE ARE THINKING ABOUT TO SEE

1301
01:01:19,660 --> 01:01:24,865
IF THEY ARE COULD AND FOR THAT 

1302
01:01:24,865 --> 01:01:26,534
WE NEED TO RECORD FROM DIFFERENT

1303
01:01:26,534 --> 01:01:27,601
BRAIN AREAS.

1304
01:01:27,601 --> 01:01:28,102
>> OKAY.

1305
01:01:28,102 --> 01:01:32,840
>> THIS IS NOT REALLY
ABOUT 

1306
01:01:32,840 --> 01:01:35,142
DECODING THE THOUGHT IT'S ABOUT 

1307
01:01:35,142 --> 01:01:39,847
DECODING WHAT SOMEONE IS TRYING 

1308
01:01:39,847 --> 01:01:40,481
TO VOCALIZE.

1309
01:01:40,481 --> 01:01:43,384
>> WHERE WOULD THOSE
THOUGHTS BE

1310
01:01:43,384 --> 01:01:44,151
STORED IN THE BRAIN?

1311
01:01:44,151 --> 01:01:47,154
>> THAT'S THE BILLION
DOLLAR 

1312
01:01:47,154 --> 01:01:48,689
QUESTION.

1313
01:01:48,689 --> 01:01:50,324
ACTUALLY THROUGHOUT THE BRAIN 

1314
01:01:50,324 --> 01:01:51,759
REALLY DISTRIBUTED.

1315
01:01:51,759 --> 01:01:54,962
THERE'S NOT A LOT OF PLACES WE 

1316
01:01:54,962 --> 01:01:58,799
CAN DO SURGERY ON THAT 

1317
01:01:58,799 --> 01:01:59,667
FUNDAMENTALLY CHANGE SOMEONE'S 

1318
01:01:59,667 --> 01:02:00,768
ABILITY TO THINK.

1319
01:02:00,768 --> 01:02:03,571
WE THINK THIS KIND OF 

1320
01:02:03,571 --> 01:02:04,572
INFORMATION IS DISTRIBUTED.

1321
01:02:04,572 --> 01:02:06,440
>> THANK YOU.

1322
01:02:06,440 --> 01:02:08,943
SECOND QUESTION IS SIMILAR.

1323
01:02:08,943 --> 01:02:11,745
THE A.I. HAPPINESS, SADNESS THAT

1324
01:02:11,745 --> 01:02:15,082
COMES OUT -- I'M ASSUMING THAT'S

1325
01:02:15,082 --> 01:02:16,951
ALSO FROM A DIFFERENT BRAIN 

1326
01:02:16,951 --> 01:02:18,118
REGION FROM THE MOTOR SPEAK SO 

1327
01:02:18,118 --> 01:02:20,921
HOW IS THAT GENERATED?

1328
01:02:20,921 --> 01:02:23,924
>> THAT'S THE OPPOSITE IN
WHICH 

1329
01:02:23,924 --> 01:02:26,227
ALL THESE THINGS OPERATE AT THE 

1330
01:02:26,227 --> 01:02:30,564
MUSCLES OF OUR FACE OR FACIAL 

1331
01:02:30,564 --> 01:02:32,433
STRUCTURES SO OUR BRAIN HAS ONE 

1332
01:02:32,433 --> 01:02:35,569
MODE OF MOVING THOSE TO CREATE 

1333
01:02:35,569 --> 01:02:37,271
THE WORDS AND SOUNDS AND THE 

1334
01:02:37,271 --> 01:02:39,540
OTHER MODE IS JUST FOR 

1335
01:02:39,540 --> 01:02:39,840
EXPRESSION.

1336
01:02:39,840 --> 01:02:42,343
WE DON'T JUST USE THE VOCAL 

1337
01:02:42,343 --> 01:02:48,749
TRACT TO SPEAK, WE USE IT FOR 

1338
01:02:48,749 --> 01:02:50,818
DIFFERENT THINGS IT'S THE SAME 

1339
01:02:50,818 --> 01:02:52,853
BRAIN AREA BUT DIFFERENT FOR 

1340
01:02:52,853 --> 01:02:53,220
THOSE GESTURES.

1341
01:02:53,220 --> 01:02:54,455
>> OKAY.

1342
01:02:54,455 --> 01:02:56,156
THANK YOU VERY MUCH.

1343
01:02:56,156 --> 01:02:58,158
>> EDDIE, THAT WAS A
FANTASTIC 

1344
01:02:58,158 --> 01:02:58,425
TALK.

1345
01:02:58,425 --> 01:03:00,861
I HAVE A LOT OF QUESTIONS BUT 

1346
01:03:00,861 --> 01:03:02,963
I'LL ONLY ASK ONE.

1347
01:03:02,963 --> 01:03:06,934
THE ONE I THINK ABOUT WHEN YOU 

1348
01:03:06,934 --> 01:03:09,069
TALK ABOUT IMPLANTING PEOPLE SO 

1349
01:03:09,069 --> 01:03:11,305
LONG AFTER THEY'VE SPOKEN YOU 

1350
01:03:11,305 --> 01:03:12,439
MIGHT EXPECT THERE'S DEGRADATION

1351
01:03:12,439 --> 01:03:16,043
OF THE BRAIN'S ABILITY TO 

1352
01:03:16,043 --> 01:03:18,412
ACTUALLY PRODUCE THE NEURAL 

1353
01:03:18,412 --> 01:03:20,080
ACTIVITY BECAUSE THEY HAVEN'T 

1354
01:03:20,080 --> 01:03:20,447
SPOKEN.

1355
01:03:20,447 --> 01:03:24,585
I WONDER IF ONE THING THAT 

1356
01:03:24,585 --> 01:03:29,456
MAINTAINS THAT FUNCTION IS THE 

1357
01:03:29,456 --> 01:03:30,624
LAST COUPLE OF SLIDES OF THE 

1358
01:03:30,624 --> 01:03:35,763
INTEGRATION OF THE HEARING AND 

1359
01:03:35,763 --> 01:03:36,597
SPEECH.

1360
01:03:36,597 --> 01:03:39,466
WE THINK OF COCHLEAR 

1361
01:03:39,466 --> 01:03:42,770
IMPLANTATION AND HAVE PLASTICITY

1362
01:03:42,770 --> 01:03:45,072
PREVENTS PEOPLE LONG DEAFENED 

1363
01:03:45,072 --> 01:03:49,343
FROM BENEFITING BUT YOU'RE NOT 

1364
01:03:49,343 --> 01:03:51,612
SEEING THAT HERE SO WONDERING 

1365
01:03:51,612 --> 01:03:52,112
YOUR THOUGHTS.

1366
01:03:52,112 --> 01:03:54,214
>> THANK YOU.

1367
01:03:54,214 --> 01:03:55,783
THE QUESTION IS AFTER NOT 

1368
01:03:55,783 --> 01:03:57,451
SPEAKING FOR OVER A DECADE, IS 

1369
01:03:57,451 --> 01:03:59,386
THAT PART OF TO THE BRAIN STILL 

1370
01:03:59,386 --> 01:03:59,953
WORKING?

1371
01:03:59,953 --> 01:04:01,155
THAT WAS THE BIGGEST QUESTION IN

1372
01:04:01,155 --> 01:04:04,024
MY MIND BEFORE WE STARTED THIS 

1373
01:04:04,024 --> 01:04:07,261
IS THAT PART OF THE BRAIN DOES 

1374
01:04:07,261 --> 01:04:09,229
IT BECOME DORMANT.

1375
01:04:09,229 --> 01:04:10,831
IT'S REALLY CLEAR THAT'S NOT THE

1376
01:04:10,831 --> 01:04:13,200
CASE.

1377
01:04:13,200 --> 01:04:19,840
LIKE YOU SAID, PEOPLE HEARING 

1378
01:04:19,840 --> 01:04:20,941
WORDS AND UNDERSTANDING WORDS 

1379
01:04:20,941 --> 01:04:23,944
AND I DIDN'T SHOW THAT THERE'S 

1380
01:04:23,944 --> 01:04:24,945
READING RESPONSES IN THAT PART 

1381
01:04:24,945 --> 01:04:26,814
WE THOUGHT WAS MOTOR CONTROL HAS

1382
01:04:26,814 --> 01:04:28,315
MORE SENSORY PROCESSING 

1383
01:04:28,315 --> 01:04:30,451
INCLUDING BREATHING AND 

1384
01:04:30,451 --> 01:04:30,851
LISTENING.

1385
01:04:30,851 --> 01:04:33,654
I THINK WHEN PEOPLE INTERNALLY 

1386
01:04:33,654 --> 01:04:35,689
HAVE INNER SPEECH OR TRYING TO 

1387
01:04:35,689 --> 01:04:36,890
EXPRESS THEMSELVES WITH THE 

1388
01:04:36,890 --> 01:04:39,226
POINTER OR WHAT NOT, THEY'RE 

1389
01:04:39,226 --> 01:04:41,595
USING THEIR INTERNAL SPEECH TO 

1390
01:04:41,595 --> 01:04:42,096
GENERATE WORDS.

1391
01:04:42,096 --> 01:04:45,833
IT GOES BEYOND THE MOTOR 

1392
01:04:45,833 --> 01:04:48,902
CONTROL.

1393
01:04:48,902 --> 01:04:52,072
I THINK THERE'S A LOT OF HOPE 

1394
01:04:52,072 --> 01:04:54,341
THOSE CIRCUITS ARE STILL 

1395
01:04:54,341 --> 01:04:56,076
FUNCTIONAL AND WE CAN TAP INTO 

1396
01:04:56,076 --> 01:04:59,213
THEM AT THE RIGHT RESOLUTION WE 

1397
01:04:59,213 --> 01:05:00,080
CAN UNLOCK WHAT EXPRESSION IS 

1398
01:05:00,080 --> 01:05:04,785
WITH THEM.

1399
01:05:04,785 --> 01:05:06,787
>> MY QUESTION IS SIMILAR
TO 

1400
01:05:06,787 --> 01:05:07,788
WHAT WAS JUST ASKED.

1401
01:05:07,788 --> 01:05:09,656
I WAS CURIOUS IF THERE IS ANY 

1402
01:05:09,656 --> 01:05:11,191
DIFFERENCE IN THE DATA YOU GET 

1403
01:05:11,191 --> 01:05:15,129
FROM PEOPLE WHO ARE ABLE TO 

1404
01:05:15,129 --> 01:05:20,134
UTILIZE THEIR ARTICULATORY 

1405
01:05:20,134 --> 01:05:22,436
KINEMATICS IS THERE MORE NOISE 

1406
01:05:22,436 --> 01:05:23,904
IN THOSE NOT ABLE TO FORM THE 

1407
01:05:23,904 --> 01:05:25,572
WORDS WITH THEIR MOUTH OR DO YOU

1408
01:05:25,572 --> 01:05:27,040
SEE ANYTHING LIKE THAT?

1409
01:05:27,040 --> 01:05:29,576
>> THEY'RE DECODING FROM
SOMEONE

1410
01:05:29,576 --> 01:05:32,112
WHO HAS SEVERE PARALYSIS IS 

1411
01:05:32,112 --> 01:05:35,215
NOWHERE CLOSE TO SOMEONE WHO 

1412
01:05:35,215 --> 01:05:38,919
SPEAKS FLUENT.

1413
01:05:38,919 --> 01:05:42,089
SO, WHILE IT'S THERE AND LATENT 

1414
01:05:42,089 --> 01:05:44,358
IT'S NOT AS CLEAR AS THOSE WHO 

1415
01:05:44,358 --> 01:05:44,992
ARE SPEAKING EVERY DAY ALL THE 

1416
01:05:44,992 --> 01:05:50,197
TIME.

1417
01:05:50,197 --> 01:05:52,933
ONE MUCH THE THINGS WE'VE SEEN 

1418
01:05:52,933 --> 01:05:54,835
ANECDOTALLY IS THE SIGNALLING 

1419
01:05:54,835 --> 01:05:57,604
CHANGES OVER THE WEEKS, DAYS AND

1420
01:05:57,604 --> 01:05:59,673
MONTHS WE DO THE TRAINING.

1421
01:05:59,673 --> 01:06:02,743
THE SIGNALS BECOME CLEARER.

1422
01:06:02,743 --> 01:06:05,646
THE ANALYSIS BECOMES A LITTLE 

1423
01:06:05,646 --> 01:06:06,380
BIT MORE STRAIGHTFORWARD HOWEVER

1424
01:06:06,380 --> 01:06:10,050
IT'S NOT JUST THAT.

1425
01:06:10,050 --> 01:06:12,119
WE SEE PEOPLE ARE HAVING 

1426
01:06:12,119 --> 01:06:15,889
INCREASE IN THE MUSCLE TONE AND 

1427
01:06:15,889 --> 01:06:22,963
THEY STILL CAN'T SPEAK BUT FOR 

1428
01:06:22,963 --> 01:06:24,865
THE FIRST TIME THEY'RE TRYING TO

1429
01:06:24,865 --> 01:06:26,767
SPEAK FOR HOURS THROUGH 

1430
01:06:26,767 --> 01:06:33,440
PROSTHETIC BUT SEEING THE 

1431
01:06:33,440 --> 01:06:35,742
BYPRODUCT.

1432
01:06:35,742 --> 01:06:36,610
>> TWO MORE.

1433
01:06:36,610 --> 01:06:38,645
>> HI.

1434
01:06:38,645 --> 01:06:42,115
MY QUESTION WAS REGARDING THE 

1435
01:06:42,115 --> 01:06:46,787
SECOND PATIENT WHERE THEY'D GO 

1436
01:06:46,787 --> 01:06:47,888
AHEAD AND READ THE QUESTION BACK

1437
01:06:47,888 --> 01:06:58,232
THE ENTIRE SENTENCE.

1438
01:07:01,001 --> 01:07:11,011
>> 

1439
01:08:08,468 --> 01:08:09,169
>> THERE'S AN IMPROVEMENT
IN 

1440
01:08:09,169 --> 01:08:16,043
MOTOR FUNCTION.

1441
01:08:16,043 --> 01:08:19,146
>> THERE'S FIBERS
REGROWING IN 

1442
01:08:19,146 --> 01:08:25,919
THE SPINAL CORD.

1443
01:08:25,919 --> 01:08:30,123
MAYBE THE DEGREE OF PARALYSIS 

1444
01:08:30,123 --> 01:08:40,400
DIMINISHES MUCH 

1445
01:08:44,171 --> 01:08:46,540
>> WE OBSERVED THE
STRENGTHENING

1446
01:08:46,540 --> 01:08:47,541
OF MUSCLE TONE AND MOVEMENT.

1447
01:08:47,541 --> 01:08:51,645
I DON'T KNOW IF IT'S BECAUSE OF 

1448
01:08:51,645 --> 01:08:55,782
NEW CELLS 

1449
01:08:55,782 --> 01:08:59,219
>> PROBABLY SYNAPTICALLY
THE 

1450
01:08:59,219 --> 01:09:04,958
CIRCUITS ARE BEING STRENGTHENED.

1451
01:09:04,958 --> 01:09:09,663
IN THE WHERE IT'S FIXED IS WHERE

1452
01:09:09,663 --> 01:09:11,098
IT'S DECLARED.

1453
01:09:11,098 --> 01:09:14,701
WHERE WE'RE STUDYING IS IN 

1454
01:09:14,701 --> 01:09:19,373
COMPLICATED REGIME WHERE THE 

1455
01:09:19,373 --> 01:09:20,941
PARALYSIS IS DEGENERATION.

1456
01:09:20,941 --> 01:09:21,975
AT THE SAME TIME WE'RE TRYING TO

1457
01:09:21,975 --> 01:09:25,078
GET THE DECODER TO WORK.

1458
01:09:25,078 --> 01:09:27,981
IT'S A LITTLE BIT OF MOVING 

1459
01:09:27,981 --> 01:09:34,488
TARGET AND DEGENERATION OVER 

1460
01:09:34,488 --> 01:09:35,722
TIME.

1461
01:09:35,722 --> 01:09:36,223
IT'S QUITE IMPORTANT.

1462
01:09:36,223 --> 01:09:38,792
>> IMPORTANT.

1463
01:09:38,792 --> 01:09:39,993
>> ALL RIGHT.

1464
01:09:39,993 --> 01:09:40,661
THANK YOU SO MUCH AGAIN.

1465
01:09:40,661 --> 01:09:44,965
THAT WAS ABSOLUTELY WONDERFUL.

1466
01:09:44,965 --> 01:09:46,900
EVERYONE, PLEASE JOIN US FOR A 

1467
01:09:46,900 --> 01:09:48,935
RECEPTION IN THE BACK AND GIVE 

1468
01:09:48,935 --> 01:09:49,503
DR. CHANG ANOTHER ROUND OF 

1469
01:09:49,503 --> 01:09:49,703
APPLAUDS.