>> OKAY, WELL LET'S KICK THIS
OFF.
GOOD MORNING EVERYONE AND THANK
YOU SO MUCH FOR COMING IN PERSON
OR ATTENDING IN THE VIRTUAL
AUDIENCE TO THE BRAIN NEUROAI WORKSHOP.
WE PUT TOGETHER A GREAT PROGRAM
HERE.
AND YOU'LL HEAR A LOT MORE ABOUT
IT IN THIS OPENING SESSION BUT
FIRST OFF I WANT TO PASS THE
MIC TO ANDREA BECKEL-MITCHENER, THE DEPUTY DIRECTOR OF
THE NIH BRAIN INITIATIVE.
WELCOME, ANDREA.
>> THANKS, JOE.
THANKS EVERYONE FOR BEING HERE.
I UNDERSTAND WE HAVE ABOUT 300
PLUS IN PERSON REGISTRATIONS AND
OVER 1,000 JOINING US VIRTUALLY.
SO WELCOME TO ONE AND ALL.
WE'RE REALLY EXCITED ABOUT THIS.
I WANTED TO THANK THE ORGANIZING
COMMITTEE INCLUDING JOE AS WELL
AS THE SCIENTIFIC PROGRAM
COMMITTEE WHO HAS PUT TOGETHER
WHAT LOOKS LIKE A VERY INTERESTING
WORKSHOP.
IT'S VERY IMPORTANT TO THE BRAIN
INITIATIVE AND WE'RE EXCITED TO
LEARN MORE.
SO AS JOE SAID I AM ANDREA THE
DEPUTY DIRECTOR OF THE BRAIN
INITIATIVE HERE AT NIH.
I THOUGHT I'D TAKE A FEW MINUTES
TO DESCRIBE THE BRAIN INITIATIVE
AT DIFFERENT LEVELS AND WALK YOU
THROUGH A FEW THINGS THAT WE
THINK ARE EXTRA IMPORTANT TO PAY
ATTENTION TO OVER THESE NEXT TWO
DAYS.
SO THE BRAIN INITIATIVE IS
BROADER THAN NIH AND WAS ANNOUNCED
BACK IN 2013.
THE NIH MADE OUR FIRST AWARDS IN
2014 AND REALLY THE BRAIN
INITIATIVE IS A PARTNERSHIP.
A PUBLIC-PRIVATE PARTNERSHIP
AMONG SEVERAL U.S. AGENCIES AND
FOUNDATIONS AND YOU'LL SEE THEIR ICONS
AT THE BOTTOM OF THE SLIDE.
WE'VE ENJOYED A LOT OF SUPPORT
FROM CONGRESS SINCE ITS
INCEPTION IN 2014.
WE FUNDED OVER 1600 PROJECTS AT
A BUDGET OF $3.5 BILLION.
THE NIH BRAIN INITIATIVE IS WHO
WE ARE AND WE ARE SPONSORING THIS
WORKSHOP. WE ARE MADE UP OF 10
INSTITUTES AND CENTERS HERE AT
THE NIH.
THE TWO LEAD INSTITUTES BEING
NINDS THE NATIONAL INSTITUTE OF
NEUROLOGICAL DISORDERS AND
STROKE AND MY HOME INSTITUTE THE
NATIONAL INSTITUTE OF MENTAL
HEALTH.
BUT WE RELY ON THE WORK AND
EXPERTISE AND SUPPORT OF EIGHT
OTHER INSTITUTES AND WE'VE BEEN
FAIRLY SUCCESSFUL WE THINK USING
THIS MODEL OVER THE LAST 10
YEARS.
OUR MISSION IS TO REVOLUTIONIZE
OUR UNDERSTANDING OF THE HUMAN
BRAIN BY ACCELERATING
DEVELOPMENT AND APPLICATION OF
INNOVATIVE TECHNOLOGIES.
AGAIN, SO 10 NIH INSTITUTES AND
CENTERS CONTRIBUTING BUT THIS IS
THE REAL ENGINE.
I KNOW YOU CAN'T READ THESE
NAMES BUT THIS IS A LIST OF ALL
THE STAFF MEMBERS ACROSS THE NIH
INSTITUTE AS WELL AS BEYOND THE
INSTITUTES THAT ARE
CONTRIBUTORS.
WE COULDN'T DO WHAT WE'RE DOING
WITHOUT THEIR EXPERTISE AND
DEDICATION. WE'RE GRATEFUL FOR IT.
NOW A WALK THROUGH HISTORY OF
OUR BUDGET.
I'M NOT GOING TO SPEND A LOT OF
TIME ON THIS BUT IT COMES UP IN
CONVERSATION A LOT. THE BRAIN
INITIATIVE IS FUNDED THROUGH TWO
STREAMS. OUR BASE APPROPRIATION
THE BASE BUDGET AT NIH ALL
INSTITUTES AND CENTERS GET FROM
CONGRESS. WE ALSO HAVE A CHUNK OF
MONEY FROM ANOTHER AUTHORIZED
LEGISLATION CALLED THE CURES
ACT.
AND YOU'LL SEE OUR BASE BUDGET
HAS GROWN STEADILY FOR THE MOST
PART FROM FY14 THROUGH FY22.
AND THE CURES ACT WHICH WAS
ENACTED IN FY17 IS ANOTHER TYPE
OF FUNDING THAT FLUCTUATES YEAR
TO YEAR.
IT HAS TO BE APPROPRIATED EVERY
YEAR AND I'M USING A LOT OF
TECHNICAL TERMS BUT WE ARE
AUTHORIZED THROUGH 2026 WITH
ADDITIONAL CURES MONEY.
SO YOU'LL NOTE -- THIS
IS THE WAY THIS BUDGET LOOKED
THROUGH 2022 WITH THE
COMBINATION OF THE BLUE BAR, THE
BASE MONEY AND THE REDDISH-PINK
BAR THE CURES MONEY AND WHAT
HAPPENED IN 2023 IS OUR BASE MONEY
DROPPED BECAUSE OUR CURES MONEY
WAS SO HIGH THAT YEAR.
AND IF YOU RECALL ANCIENT
HISTORY WHEN FY24 BUDGET WAS
PASSED THEY KEPT THE NIH
FLAT SO THE BRAIN BUDGET WAS
ALSO FLAT. BUT THAT YEAR WHEN
THE CURES MONEY DIPPED, WE ENDED
UP HAVING ABOUT A $278 MILLION
DECREASE IN OUR OVER ALL BUDGET.
WE PLANNED FOR THIS.
WE WEREN'T SURE IT WAS GOING TO
HAPPEN BUT MODELLED OUR BUDGET
IN DIFFERENT WAYS.
IT WAS UNFORTUNATE WE HAD TO
LOSE SOME OF OUR FUNDING
ANNOUNCEMENTS AT LEAST
TEMPORARILY AND HAD HOPE SOME
MIGHT BE RESTORED IN THE FUTURE
BUT THAT'S STILL $400 MILLION
OF A BUDGET.
THAT'S STILL A PRETTY HEALTHY
BUDGET TO MAKE INVESTMENTS IN
THE KINDS OF SCIENCE WE THINK IS IMPORTANT.
SO HERE WE ARE, THE BRAIN
INITIATIVE AT NIH AND HERE'S A
LIST OF SOME OF THE THINGS WE
SUPPORT IN BROAD STROKES.
CELL AND CIRCUIT TECHNOLOGIES,
RECORDING AND NEURAL MODULATION,
IMAGING TECHNOLOGIES OF ALL
KINDS AND DIFFERENT RESOLUTIONS,
SYSTEMS NEUROSCIENCE AND WHAT WE
CALL HUMAN NEUROSCIENCE.
ON THE RIGHT-HAND SIDE OF THIS
SLIDE IS SOME OVERARCHING THEMES
WE THINK ARE VERY IMPORTANT.
THE DATA SCIENCE AND INFORMATICS
COMPONENT.
WE HAVE A STRONG PROGRAM AND
TRAINING, FOCUSSING ON INCLUSION
AND EQUITY.
WE ALSO SUPPORT NEUROETHICS
DISSEMINATION AND COMMERCIALIZATION
THROUGH DIFFERENT AVENUES.
I'LL TALK LATER IN THE TALK ON
THE DATA SCIENCE AND NEUROETHICS
PIECES JUST TO INTRODUCE THOSE
IN MORE DETAIL.
WE THINK BRAIN -- WE LIKE TO
THINK OF OURSELVES AS UNIQUE AND
FOUNDATIONAL.
THE TYPES OF PROJECTS WE SUPPORT
ARE INNOVATIVE.
WE SUPPORT A LOT OF TEAM SCIENCE IN
ALL DIFFERENT AREAS. AGAIN,
BEING INCLUSIVE, GROWING THE NEW
GENERATION OF SCIENTISTS ENGINEERS
COMPUTATIONAL BIOLOGISTS AND
MATHEMATICIANS AND PHYSICISTS TO
SUPPORT THE SCIENCE OF THE
BRAIN.
WE HAVE A VERY OPEN ASSERTIVE
SHARING TRADITION, I GUESS
AND WE CONTINUE TO DO THAT WHERE
WE THINK THE SCIENCE SHOULD BE
OPEN AND ACCESSIBLE AND
AVAILABLE TO ALL.
AND OF COURSE AS I MENTIONED
NEUROETHICS AND ETHICS IN
GENERAL IS VERY IMPORTANT.
THERE'S A PUBLICATION THAT CAME
OUT A MONTH OR TWO AGO.
JOHN NGAI OUR DIRECTOR IN NEURON
IMPORTANT TO REFLECT ON THE LAST
10 YEARS OF INNOVATION.
I RECOMMEND YOU ACCESS THAT.
IT WILL TELL US MORE WHERE WE'VE
BEEN AND WHERE WE HOPE TO GO.
AND OF THOSE 1600 PROJECTS THAT
I MENTIONED THAT WE FUNDED, WE
HAVE AN EMPHASIS ON FOUR MAIN
PROJECTS THAT WE'VE SUPPORTED IN
GENERAL WE'RE CALLING THESE THE
PARTS LIST. ON THE LEFT HAND
SIDE IS THE CELL ATLASING PROGRAM
LOOKING AT THE DIFFERENT CELLS IN
THE BRAIN OF DIFFERENT SPECIES.
WE'RE SUPPORTING WIRING DIAGRAMS
AGAIN OF DIFFERENT SPECIES.
WE CALL THIS OUR BRAIN CONNECTS
NETWORK.
THAT'S ANOTHER LARGE PROGRAM
WE'RE SUPPORTING.
WE ALSO HAVE AN INTEREST IN
USING THE DATA FROM THE CELL
ATLASING AND WIRING PROGRAMS TO
UNDERSTAND HOW WE CAN INTERVENE.
SO PRECISION CELL ACCESS WE CALL
THAT THIS ARMAMENTARIUM AND OUR
NEWEST PROGRAM BRAIN BEHAVIOR
QUANTIFICATION AND SYNCHRONIZATION
LOOKING AT REALTIME SIGNALS, BRAIN ACTIVITY
AND HOW THOSE RELATE TEMPORALLY TO BEHAVIORS OF DIFFERENT organisms including humans
I'M GOING TO TALK A LITTLE BIT,
THAT I HOPE ALL OF YOU WERE
ACCESSING IN THE LAST MONTH OR
SO THIS WAS FLYWIRE.
A HUGE UNDERTAKING, A REMARKABLE
UNDERTAKING EXAMINING AND
DEVELOPING THE TOOLS AND FINALLY
THE OUTPUT OF THIS PROJECT THAT
WAS PUBLISHED IN NATURE A COUPLE
MONTHS AGO AND IT'S THE COMPLETE
WIRING DIAGRAM OF THE
DROSOPHILA MELANOGASTER COMMON
FRUITFLY - IT'S AMAZING.
I WISH THERE WAS SOME ORCHESTRAL
MUSIC THAT WOULD PLAY WHILE THIS
VIDEO PLAYS BECAUSE IT'S
ABSOLUTELY GORGEOUS.
SO WHAT WAS SUPPORTED HERE OR
WAS DISCOVERED, I SHOULD SAY, IS
CONNECTIVITY AT SYNAPSE LEVEL IN
A FLY BRAIN. S0 139000 NEURONS,
4500 OR SO UNIQUE CELL TYPES AND
50 MILLION APPROXIMATELY, GIVE
OR TAKE, SYNAPSES ACROSS ALL
THOSE.
THERE WERE NINE PUBLICATIONS IN
NATURE, JOINT PUBLICATIONS AND
WHAT WE'RE REALLY EXCITED ABOUT
NOT ONLY WILL THIS ENABLE ADDITIONAL
STUDIES AND SETS THE FOUNDATION
FOR DOING THIS IN LARGER
ORGANISMS LIKE THE MOUSE AND NON
HUMAN PRIMATES AND HUMANS WHICH
ARE PART OF THE CONNECTS PROGRAM I
I MENTIONED EARLIER.
SO THE OTHER PART I WANT TO
EMPHASIZE IS THIS WAS AN
INTERNATIONAL EFFORT.
OVER 50 INSTITUTIONS FUNDED BY
MULTIPLE AGENCIES.
MANY AGENCIES I MENTIONED
EARLIER.
AND SOME NONPROFITS.
THE POWER OF CITIZEN SCIENCE.
ONE OF THE COOL THINGS ABOUT THE
PROJECT WAS IT WAS CROWD SOURCED
PROOF READING THE CONNECTIONS
WOULD HAVE TAKEN 33 PERSON
YEARS.
BECAUSE OF THIS INTERESTING
APPROACH OF CROWD SOURCING IT
TOOK ONLY A FEW YEARS.
AND FINALLY AS I MENTIONED THE
POWER OF OPEN SCIENCE.
ALL THESE DATA ARE AVAILABLE
THROUGH FREE WEB TOOLS TO ENABLE
USERS TO DELVE INTO THE DATA SET
AND WE THINK IT WILL BE A REAL
ENGINE FOR DISCOVERY.
JUST A PEEK INTO ANOTHER
IMPORTANT PAPER THAT CAME OUT
THIS YEAR FROM THE LICHTMAN LAB
RECONSTRUCTED A 1 CUBIC MILLIMETER
HUMAN TEMPORAL CORTEX ACROSS ALL
LAYERS. AGAIN, A HUGE EFFORT AND
UNDERTAKING BUT THE SCALE AND
RESOLUTION NEEDED TO LOOK AT
THESE DIFFERENT CELL TYPES
ACROSS THESE LAYERS. SO AT ULTRA
STRUCTURAL RESOLUTION ONE CUBIC
MILLIMETER (MM), THEY WERE ABLE TO
RECONSTRUCT 57,000 CELLS, 230 MM OF
BLOOD VESSELS, 150 MILLION SYNAPSES
AT JUST REMARKABLE DETAIL. ALL
THESE ARE AVAILABLE FOR RESEARCHERS
TO EXAMINE AND ASK OWN QUESTIONS.
THOSE ARE A COUPLE OF EXAMPLES
OF THE SCIENCE WE SUPPORT.
I WANT TO SPEND A COUPLE MINUTES
NOW AT THE END OF THIS INTRO TO
TALK ABOUT TWO OF THE CORE
PRINCIPLES THAT ARE VERY
IMPORTANT TO THE BRAIN
INITIATIVE.
THIS IS DATA AND DATA SHARING
AND THE PLATFORMS TO ENABLE THAT
AS WELL AS THE ETHICAL
IMPLICATIONS AND CONSIDERATIONS
OF THE KIND OF WORK THE BRAIN
INITIATIVE SUPPORTS.
SO ONE OF OUR PROGRAMS IS
ACTUALLY TO SUPPORT A LIBRARY OF
DATA ARCHIVES AT DIFFERENT DATA
TYPES.
WE HAVE EIGHT NOW AND A COUPLE
OR FIVE ARE REPRESENTED HERE.
THREE ARE NOT ON THIS SLIDE.
YOU'LL BE HEARING A BIT ABOUT
DABI, ANOTHER ONE FROM
DOMINIQUE DUNCAN A LITTLE LATER.
WE SET THEM UP A FEW YEARS BACK
AND ALL THE DATA FROM THE BRAIN
INITIATIVE PROJECTS ARE
SUBMITTED TO THE DATA ARCHIVES
THEY ARE CLEANED AND AVAILABLE
AND FOR USE BY THE
COMMUNITY AND IT'S BEEN AN
INCREDIBLE UNDERTAKING BUT
ALSO VERY IMPORTANT TO THE BRAIN
INITIATIVE.
NEUROETHICS IS ANOTHER PIECE
YOU'LL HEAR MORE ABOUT. DOCTOR
KAREN ROMMELFANGER WILL TALK IN
SESSION 2 AND GIVE YOU INTRO ON
NEUROETHICS.
I WANT TO MENTION IT HERE BECAUSE
IT'S NOT SOMETHING THAT GETS
A LOT OF AIR TIME OUTSIDE THE
BRAIN INITIATIVE. IT IS IMPORTANT
AND BRAIN IS LEADING THE WAY IN
THIS.
THE STUDY OF ETHICAL ISSUES
RAISED BY THE TECHNOLOGIES THAT
ARE BEING DISCOVERED AND
POTENTIALLY USED BOTH IN
RESEARCH LABS AS WELL AS
POTENTIALLY IN THE CLINIC. WE ARE
PROACTIVE. ONGOING ASSESSMENT OF
THE NEUROETHICAL IMPLICATIONS OF
THE RESEARCH THAT'S DONE.
WE INTEGRATE NEUROETHICS INTO OUR
DECISION MAKING. A WORKING GROUP
DR ROMMELFANGER IS PART OF HAVE
PUBLISHED GUIDING PRINCIPLES
FOR SCIENTISTS AND THE PUBLIC TO
UNDERSTAND THE KINDS OF
QUESTIONS THAT SHOULD BE ASKED
AND THE THINKING THAT GOES
AROUND THE ETHICAL
CONSIDERATIONS.
WE HOLD TOPICAL WORKSHOPS ABOUT
ONE A YEAR ON SOME OF THESE
ISSUES THAT ARE RAISED WHERE WE
GET BROADER INPUT AND BROADER
DISCUSSION.
INTERNALLY WE HAVE A TEAM OF NIH
PROGRAM OFFICERS THAT MEET ONCE A
MONTH TO THINK OF THE NEUROETHICAL
CONSIDERATIONS AND SCIENCE OF UPCOMING
AND PROJECTS ALREADY FUNDED. AND
FINALLY WE HAVE RELEASED FUNDING
ANNOUNCEMENTS WHERE WE FUND
NEUROETHICS RESEARCH IN AREAS
THAT ARE IMPORTANT TO THE BRAIN.
AND I MENTION THESE HERE. WE HAVE
AN RO1 AND NEWLY ADDED R21 PROGRAM
TO EXAMINE ETHICAL IMPLICATIONS
OF NEUROTECHNOLOGIES. A.I. AND
NEUROAI WILL BE COMPONENT OF THIS.
WE ENVISION SEEING APPLICATIONS
ON TOPICS WE WILL DISUCSS TODAY
AND TOMORROW. SO AGAIN, ETHICAL
IMPERATIVES ABOUT A.I.
I WON'T SPEND A LOT OF TIME.
KAREN WILL LEAD THIS DISCUSSION
ON OUR BEHALF OF THE BRAIN
INITIATIVE. THE A.I. TOOLS THROUGH
THE LENS OF NEUROETHIC.
THIS IDEA IT CAN BE IMPEDED BY BLACK
BOX MODELS. ROBUST TRACKING OF
METADATA INCLUDING DATA PROVIDENCE
CONSENT AND WE WANT NEUROETHICS TO
BE PERSON CENTRIC. WITH THAT
FOUNDATION WE HOPE EVERY SESSION HAS
SOME DISCUSSION AROUND NEUROETHICAL
IMPLICATIONS.
AND SO FINALLY FOR ALL OF YOU,
BOTH IN PERSON AND ONLINE, WE
INVITE YOU TO STAY UP TO DATE ON
WHAT'S HAPPENING IN BRAIN
INITIATIVE.
YOU CAN SUBSCRIBE TO OUR BLOG,
WEBSITE IS FULL OF INFORMATION.
MONITOR THE NIH GUIDE NOTICES
CONNECT WITH PROGRAM OFFICERS
IN I.C. OF INTEREST
PAUSE ON THE FUNDING OPPORTUNITIES
PAGE. TAKE PICTURES. WE HAVE ABOUT
20 OR SO ACTIVE FUNDING
OPPORTUNITIES BETWEEN 20 AND 30
USUALLY AT ANY GIVEN TIME.
AND YOU CAN SUBSCRIBE TO THE
GUIDE OR BLOG POST TO GET
NOTIFICATIONS OF THE FUNDING
OPPORTUNITIES.
ALL THIS CAN BE FOUND ON OUR
WEBSITE BRAIN.GOV
AND I WILL TURN IT BACK OVER TO
JOE.
>> THANK YOU ANDREA
THAT WAS EXCELLENT.
>> I'M JOE MONACO A SCIENTIFIC
MANAGER WITH THE BRAIN
INITIATIVE.
BEFORE WE GET TO A COUPLE KEY
NOTES THIS MORNING I WANT TO
GIVE A BRIEF INTRODUCTION TO THE
WORKSHOP, GIVE YOU POINTERS AND
MATERIALS WE PRODUCED THAT
INCORPORATE, MOTIVATE THE
QUESTIONS WE LOOK TO DISCUSS
AND EXPLORE AT THE WORKSHOP.
I WANT TO START WITH THIS HAS
BEEN A LARGE TEAM EFFORT.
THIS IS ALL THE INTERNAL WORKING
GROUP MEMBERS THAT ATTRIBUTED TO
PLANNING THIS WORKSHOP.
I'VE CO-ORGANIZED THIS WITH GRACE
HWANG A PROGRAM DIRECTOR AT NINDS.
I PARTICULARLY WANT TO POINT OUT
COURTNEY PINARD AND JESSICA MOLLICK
WHO DID EXCELLENT WORK LEADING OUR
EARLY-CAREER SCHOLAR EFFORTS AND THE
POSTER SESSION AND COMPETITION.
IT'S BEEN A BIG TEAM EFFORT
WORKING WITH A SET OF EXCELLENT
EXTERNAL ADVISERS AND COMMITTEE
MEMBERS TONY ZADOR, BRAD AIMONE,
DORIS TSAO, GINA ADAM, BLAKE RICHARDS
WE'VE BEEN WORKING TOGETHER FOR
QUITE A WHILE TO PUT TOGETHER THIS
PROGRAM FOR YOU.
IN THE LONGER RUN AS ANDREA
TALKED ABOUT, THE BRAIN INITIATIVE
IS CELEBRATING ITS FIRST DECADE
WE ARE 10 YEARS INTO THE BRAIN
INITIATIVE.
YOU CAN ALSO THINK WE ARE 10
YEARS INTO THE NEW MODERN ERA OF
A.I. AROUND 2012, 2014 AND A.I.
COMPUTING TECHNOLOGY REVOLUTION
HIT ITS INFLECTION POINT AND
ENABLED NEW KINDS OF SCALE AND
DATA DRIVEN COMPUTATION AND SO
THIS IS A MAJOR POINT OF
CONVERGENCE WITH THE BRAIN
INITIATIVE AND A.I. AND IT'S A
GREAT TIME TO THINK BROADLY ABOUT
WHAT WE CAN LEARN FROM THESE DOMAINS
I WANT TO POINT OUT ON A MORE
PERSONAL STORY BRIEFLY HOW I AND
GRACE GOT HERE, THE US BRAIN
INITIATIVE WAS LAUNCHED. NSF
HAD A PORTFOLIO OF PROGRAMS
INCLUDING THE NEURAL AND COGNITIVE
SYSTEMS OR NCS. AND SO A FEW YEARS BACK
WE WERE PART OF A PROJECT TEAM
AND WE TOOK SOME IDEAS FROM THEORETICAL
NEUROSCIENCE AND HIPPOCAMPAL FUNCTION
APPLIED THEM IN AN INTERESTING WAY TO KEY
PROBLEMS IN MULTIAGENT CONTROL AND
AUTONOMOUS SYSTEMS. SO WE FORMED THIS
INTERDISCIPLINARY COLLABORATION WITH
JOHNS HOPKINS APPLIED PHYSICS LAB AND
SAW THE POWER OF RESEARCH THERE.
AND THAT LED TO US MOSTLY GRACE
BUT I HELPED OUT A LITTLE BIT
CO-ORGANIZING A SYMPOSIUM AT THE 2020
BRAIN INITIATIVE INVESTIGATORS MEETING
AROUND THESE ISSUES. HOW DYNAMICAL SYSTEMS IDEAS IN NEUROSCIENCE
CONTRIBUTE TO MACHINE LEARNING.
A.I. WASN'T THE BUZZWORD BACK THEN BUT THAT'S WHAT WE WERE TALKING ABOUT.
THERE'S A QR CODE THAT LEADS TO VIDEO.
HERE'S A SCREEN SHOT OF MY TALK
AND YOU CAN SEE HERE'S TWO COLUMNS
WE ARE ABLE TO SEE ARTIFICIAL
INTELLIGENCE NOW AND START TO COMPARE
THE CAPABILITIES, THE SIMILARITIES
AND DIFFERENCES IN BIOLOGY AND
ABOUT WHAT WE'RE SEEING IN A.I.
AND NETWORK MODELS.
IT'S EASY TO COME UP WITH A LIST
LIKE THIS. MODULARITY, RECURRENCE, TEMPORAL COMPLEXITY
AND THERE'S ALL SORTS OF WAYS
THESE SYSTEMS ARE DIFFERENT.
THERE'S A LOT OF SCIENTIFIC
OPPORTUNITIES AND SO THIS JUNE
AT THE BRAIN CONFERENCE WE HAD A
SPECIALTY SESSION ON EMBODIED
NEUROAI
A SHORT ONE-HOUR PANEL TO GET THE
CONVERSATION STARTED. THERE'S THE
QR CODE THERE TO WATCH THE VIDEO
OF THAT SESSION. SO WE'RE EXCITED
THAT A LOT OF THE FOLKS WE'VE MET
IN THE LAST YEAR IN THE ROOM AND
IN THE VIRTUAL AUDIENCE ARE
HAVING THESE CONVERSATIONS. I DO WANT
TO POINT OUT THIS GAP 2020-2024.
LOT OF INTEREST IN THE FIELD EXPLORING
THESE QUESTIONS. WHY ARE WE SEEING
AMAZING PROGRESS IN THE A.I.
COMPUTING TECHNOLOGY? WHAT DOES
IT MEAN FOR UNDERSTANDING THE BRAIN?
HOW MUCH PROGRESS IS DUE TO WHAT
WE KNOW ABOUT THE BRAIN?
IS THERE A DISCOVERY LOOP THAT
WE SHOULD BE EMPHASIZING HERE IN
ORDER TO DRIVE HOME THE SCORE?
WE WANT TO BRING THESE QUESTIONS
TOGETHER AND A LOT OF THE PEOPLE
PUTTING THESE IDEAS OUT THERE
ARE IN THE ROOM AND IN OUR PANELS.
A FIGURE FROM A PAPER BY BRAD AIMONE
LAID OUT BASICALLY POST BEHAVIORISM
LOOK AT THE TOP TIME LINE. THESE
ARE MAJOR INNOVATIONS IN NEUROSCIENCE
COMING IN THAT HAVE IMPACTED OR
INFLUENCED THE EVOLUTION OF WHAT
WE NOW CALL A.I.: REPERTOIRE
OF APPROACHES OF LEARNING FROM DATA.
SOME OF THIS HAS LED TO A RECENT
NOBEL PRIZE OF COURSE.
BUT HOW DO THESE LINES CONVERGE?
THAT'S THE QUESTION FOR THE WORKSHOP.
WHERE'S IT GO FOR NEUROSCIENCE?
AND HOW CAN THEY BOTH DRIVE EACH OTHER
FORWARD?
ONE OF THE PLACES IT HAS GONE IS THIS NEW SET OF AMAZING A.I. BASED TOOLS THAT WE NOW HAVE IN
NEUROSCIENCE FOR TRACKING BEHAVIOR AND INFORMING LARGE-SCALE POPULATION ANALYSIS, GEOMETRY,
TOPOLOGICAL DATA ANALYSIS OR THE
USE OF RNNS TO UNPACK TIME-SERIES.
THERE'S A LOT OF GREAT TOOLS.
THAT'S NOT WHAT THE WORKSHOP IS
ABOUT.
PEOPLE ARE DOING THIS KIND OF
WORK AND IT WILL CONTINUE TO DO
AMAZING THINGS TO HELP US
UNDERSTAND OUR DATA BUT THIS
WORKSHOP IS ABOUT MORE
FUNDAMENTAL SCIENTIFIC
QUESTIONS.
ARTIFICIAL NEURAL NETWORKS ARE TRAINED.
YOU SPECIFY ARCHITECTURE AND HAVE AN
OBJECTIVE FUNCTION AND YOU HAVE
SOME PROCESS THAT INSTANTIATES THE
LEARNING RULE. THIS IS ALSO A
REVIEW FROM OUR COMMITTEE MEMBER
BLAKE RICHARDS. THOSE THREE
COMPONENTS WE GET ALL THIS DATA
DRIVEN POWER FOR LEARNING AS COMPUTING.
BUT IT'S A DOUBLE-EDGED SWORD
BECAUSE THERE'S NO GUARDRAILS
BUILT INTO THAT. YOU CAN'T PROGRAM
THEM AND EVERYTHING BECOMES DATA
CURATION AND THEN DEALING WITH
THINGS IN THE BACK END OF THE
SYSTEMS PERSPECTIVE.
THIS IS WHERE THE ETHICAL
CONCERN IS AND WHAT WE'LL LEARN
ABOUT IN THE WORKSHOP AND I JUST
WANT TO CLARIFY WE HAVE A LOT OF
AMAZING A.I. TOOLS FOR NEUROSCIENCE.
THE FOCUS OF THE MEETING IS
LOOKING FORWARD TO A SET OF WIDE
RANGING SCIENTIFIC OPPORTUNITIES
TO ADVANCE NEW THEORIES AND
THINK ABOUT NEW COMPLEMENTARY
MODELS AND NEW DISCOVERY LOOPS
TO PUSH ALL THIS FORWARD. THIS
COULD BE A SIMULTANEOUS ADVANCE
IN COMPUTING TECHNOLOGY AND TOOLS
FOR UNDERSTANDING THE BRAIN.
THAT'S REALLY EXCITING.
SO HOW CAN WE BRING THE TROVES OF
DATA BRAIN HAS COLLECTED, LIKE
FLYWIRE, BICAN, CELL ATLASES, AND
BRING THAT TO BEAR INTO THIS PROCESS?
I'M NOT GOING TO GO THROUGH ALL THIS.
THIS IS A SUMMARY FROM THE POSITION
PAPER WHICH IS PART OF THE PROGRAM
BOOK. I ENCOURAGE EVERYONE TO CHECK
OUT THE PROGRAM BOOK. THERE ARE
TRADEOFFS AND WE THINK OF
COMPUTATIONAL NEUROSCIENCE AND HOW
TO SCALE UP. THINK ABOUT THE
LARGE SCALE TRANSFORMERS GETTING
RID OF BIOLOGICAL DETAIL TO SCALE
UP MODEL SIZES BUT WHAT DO YOU LOSE?
SO LET'S THINK OF FUNDAMENTAL
TRADEOFFS. HOW DO WE BREAK THIS
TRIANGLE OF TRADE-OFFS OF SPACE,
TIME AND BIOLOGICAL REALISM. HOW
CAN WE USE BRAIN DATA TO BREAK
THESE BARRIERS TO SCALE, LIKE A.I.
TECHNOLOGIES HAVE BROKEN SCALE
BARRIERS? THAT'S THE BASIS OF THE
DISCUSSIONS WE WANT TO HAVE HERE.
SO TO HEAR MORE ABOUT THE
RECIPROCITY AT THE HEART OF
NEUROAI, LET'S TURN IT OVER TO
TONY ZADOR.
>> HELLO.
FIRST, LET ME THANK THE
ORGANIZERS AND ESPECIALLY JOE
AND GRACE FOR PUTTING TOGETHER
WHAT I THINK IS A REALLY
EXCITING TWO-DAY WORKSHOP.
I'M GOING TO SORT OF TAKE THE
VIEW FROM 30,000 FEET AND GIVE
AN OVERVIEW OF WHAT I THINK THE
WHOLE FIELD OF NEUROAI IS ABOUT
AND IT'S BASICALLY I THINK
SHARING THE INSIGHT FROM
NEUROSCIENCE TO AFFECT A.I. AND
A.I. TO INFLUENCE NEUROSCIENCE.
I THINK HISTORICALLY THAT'S BEEN
A REALLY FRUITFUL TWO-WAY STREET
THAT IS SOMETIMES IN THE MODERN
ERA FORGOTTEN ABOUT.
I THINK WE'RE ALL FAMILIAR WITH
THE TREMENDOUS ADVANCES OF A.I.
IN THE LAST DECADE AND
ESPECIALLY THE LAST FEW YEARS.
WHEN I WAS A GRADUATE STUDENT
THE IDEA WE WOULD EVER HAVE AN
A.I. SYSTEM THAT PASSED THE TEST
SEEMED FAR FETCHED.
AND THE FACT IS WE EFFECTIVELY
DO HAVE ONE THAT PASSES THE TURING
TEST. CHATBOTS CAN MIMIC THE
ANSWERS THAT A HUMAN WOULD GIVE.
SO I ASKED CHATGPT WHETHER IT WOULD
PASS THE TURING TEST AND IT GAVE
A LONG ANSWER BUT BASICALLY IT
WAS BEING MODEST.
IT'S CLEAR IT CAN AND DOES.
WE'RE FAMILIAR WITH THE TREMENDOUS
ADVANCES OF A.I. THIS IS
PROTEIN FOLDING WHICH WAS AWARDED
THE NOBEL PRIZE A MONTH AGO.
AS WE LOOK AT THE HISTORY OF
NEUROSCIENCE AND A.I., YOU CAN
TAKE IT BACK TO THE BEGINNING OF
THE COMPUTING ERA. I DON'T HAVE
TIME TO GO THROUGH ALL OF THESE
BUT I WILL SAY THE FIRST PAPER
OUTLINING THE DESIGN OF THE MODERN
COMPUTER BY VON NEUMANN CITED A
PAPER BY MCCULLOCH AND PITTS WHICH
LAYS THE FOUNDATION OF NEURAL
NETWORKS. THE TWO FIELDS,
COMPUTATION AND NEUROSCIENCE HAVE
BEEN INTERTWINED SINCE THEIR
INCEPTION.
AND OF COURSE, THIS YEAR THE
NOBEL PRIZE WAS A DIFFERENT
NOBEL PRIZE WAS AWARDED FOR
NEURAL NETWORKS.
SO NEUROSCIENCE AND A.I. HAVE
HAD A VIRTUOUS CIRCLE WHERE
NEUROSCIENCE INFLUENCES A.I. AND
THE A.I. PROVIDES PROBABLY THE
BEST MODELS FOR HOW TO THINK
ABOUT HOW BRAINS COMPUTE.
SO THAT VIRTUOUS CIRCLE HAS BEEN
GOING ON FOR DECADES AND HAS
BEEN SELF-CONSUMER AND USEFUL
PRODUCTS LIKE CHATGPT AND DALLE
AND SOMETIMES THE TOOLS CAN BE
USED FOR SCIENTIFIC RESEARCH.
SO THE WAY THAT I THINK ABOUT
THE MODELS THAT WE HAVE IN
NEURAL NETWORKS IN A.I. CAN BE
TRACED BACK TO THIS QUOTE BY
RICHARD FEYNMAN WHO WROTE WHAT I
CANNOT CREATE I DO NOT
UNDERSTAND.
IT TURNS OUT WHEN WE TRY TO
UNDERSTAND HOW BRAINS COMPUTE
AND SOLVE PROBLEMS. WE CAN HAVE
ALL SORTS OF IDEAS BUT
IMPLEMENTING THEM IS WHERE THE
RUBBER MEETS THE ROADS. WHERE
WE DISCOVER WHETHER THE IDEAS WE
HAVE FOR HOW BRAINS COMPUTE. THAT'S
WHERE WE DISCOVER IF THEY WORK
WHETHER THEY SCALE OR MAKE
SENSE. OFTEN WE'RE CONFRONTED
WITH MORAVEC'S PARADOX
WRITTEN 40 YEARS AGO. I WON'T
READ THE WHOLE QUOTE. WHAT ANIMALS
DO IS HARD. WHAT WE THINK AS THE
HARDEST PART, COGNITION AND
REASONING ARE THE EASIEST PART.
THE THINGS A.I. ARE GOOD AT ARE
UNIQUELY HUMAN OR THINGS WE
PRIDE OURSELVES AT BEING
PARTICULARLY GOOD AT. THINGS WE
THINK SMART PEOPLE ARE GOOD AT
THESE THINGS AND I'M SURE YOU
ALL KNOW A.I. SYSTEMS BEAT THE
BEST CHEST AND GO PLAYERS. THEY'RE
GETTING BETTER AT MATH AND ABLE TO
PRODUCE GREAT SENTENCES. BUT THE
THING THEY CAN'T DO ARE SENSING,
INTERACTING, PLANNING, AND USING A
WORLD MODEL, ACTING AUTONOMOUSLY
FOR AN ENTIRE LIFE TIME.
MOST A.I. SYSTEMS CANNOT DO
THESE THINGS OR DO THEM WELL.
AND SO THAT LED US A FEW YEARS
AGO TO PROPOSE A NEW CHALLENGE
WHICH IS THE EMBODIED TURING TEST
THAT REQUIRES NOT ONLY THE A.I.
SYSTEMS BE ABLE TO INTERACT WITH
THE SENSORIMOTOR WORLD BUT BEYOND
THAT THEY SHOULD BE ABLE
TO ACT AUTONOMOUSLY AND PLAN AND
THESE ARE EXAMPLES OF SCIENCE
FICTION DEPICTIONS OF A.I.
SYSTEMS THAT CAN DO THAT AND
WE ARE STILL FAR FROM THAT.
WHAT IS NEUROAI? NEUROAI IS THE
MEETING OF THESE TWO STREAMS.
NEUROSCIENCE HAS INSPIRED AI
AND WILL LIKELY DRIVE PROGRESS.
I'LL TOUCH UPON A FEW OF WHAT I
SEE AS THE CHALLENGES AND
OPPORTUNITIES DISCUSSED IN
GREATER DETAIL IN THE VARIOUS
PANELS.
SO THE FIRST IS THE MORAVEC'S
PARADOX LEARNING THE WAY ANIMALS
DO -- I'LL TALK BRIEFLY ABOUT A
SECOND IMPORTANT OPPORTUNITY AND
CHALLENGE WHICH IS ENERGY USE.
LOWEST HANGING FRUIT. A.I. SYSTEMS
USE TREMENDOUS AMOUNTS OF ENERGY.
THE AMOUNTS OF ENERGY REQUIRED
FOR THESE SYSTEMS TO BE TRAINED
UP NOW IS A SIGNIFICANT ITEM ON
THE WORLD'S ENERGY BUDGET AND
IT'S GOING UP RAPIDLY.
SO THERE'S DIFFERENT WAYS OF
ESTIMATING THIS. BY SOME MEASURES
BRAINS ARE FOUR ORDERS OF
MAGNITUDE MORE ENERGY EFFICIENT
THAN A.I. SYSTEMS.
THE AMOUNT OF ENERGY IT TAKES TO
TRAIN A LARGE LANGUAGE MODEL IS
NOT THERE BECAUSE IT'S
PROPRIETARY BUT ON THE ORDER OF
ENERGY USED TO RUN A MEDIUM
SIZED CITY FOR A WEEK -- IT
DOESN'T TAKE THAT MUCH ENERGY TO
TRAIN A CHILD TO SPEAK.
IT LEARNS WITH THE AMOUNT OF
FOOD IT HAS IN THE FIRST TWO
YEARS OF LIFE SO WHAT
CONTRIBUTES TO THE ENERGY
EFFICIENCY?
PART IS THE HARDWARE AND OTHER
STUFF.
WE CAN TALK ABOUT THE HARDWARE.
SO THE REASON A.I. SYSTEMS WE
USE TODAY RUN ON THE KINDS OF
HARDWARE THAT WE USE TODAY CAN
BE ATTRIBUTED IN PART TO
SOMETHING THAT SARAH HOOKER IN A
BEAUTIFUL ARTICLE REFERS TO AS
THE HARDWARE LOTTERY.
THE ALGORITHMS WE DEVELOP, THE
WAY WE DEVELOP AND THE ONES THAT
WORK AND DON'T ARE NOT
INDEPENDENT OF THE HARDWARE THEY
RUN ON.
IT TURNS OUT MOST MODERN A.I.
RUNS ON GPUs ORIGINALLY
DEVELOPED TO PROCESS IMAGES.
ONE CAN IMAGINE A DIFFERENT
WORLD IN WHICH DIFFERENT KINDS
OF HARDWARE WOULD BE USEFUL OR
DEVELOP THAT WOULD PROMOTE THE
USE OF MORE BIO INSPIRED TYPE
ALGORITHMS.
WE KIND OF KNOW WHAT THE
DIFFERENCES ARE.
WE DON'T YET KNOW HOW TO BUILD
THESE HARDWARE AND DON'T KNOW
HOW TO DEVELOP ALGORITHMS
BECAUSE WE DON'T YET HAVE THE
HARDWARE ON WHICH TO TEST THESE
IDEAS. I'LL MENTION A THIRD
OPPORTUNITY AND CHALLENGE
REFERRED TO AS THE ALIGNMENT
PROBLEM.
HOW DO WE GET A.I. SYSTEMS TO DO
WHAT WE WANT THEM TO? THE EXTREME
VERSION OF THIS IS CALLED THE
PAPER CLIP PROBLEM. IT'S A
THHOUGHT EXPERIMENT PROPOSED
BY NICK BOSTROM ABOUT 20
YEARS AGO IN WHICH SOMEBODY
TRAINS AN A.I. SYSTEM TO MAXIMIZE
THE NUMBERS OF PAPER CLIPS AND THE
A.I. GETS OUT OF CONTROL AND IT
REALIZES THE WAY TO MAXIMIZE IS
TURN ALL MATTER ON EARTH TO
PAPER CLIPS.
THAT'S AN EXAMPLE OF AN A.I.
SYSTEM NOT ALIGNED -- DOING WHAT
WE ASKED IT TO DO BUT NOT WHAT
WE WANTED IT TO DO.
THERE ARE MANY POSSIBLE
EXPLANATIONS, WAYS AROUND THIS.
ONE INSPIRATION ONE WAY AROUND
THIS FROM NEUROSCIENCE COMES
FROM THE OBSERVATION THAT
ANIMALS DON'T PURSUE A SINGLE ROLE
THEY PURSUE MULTIPLE ROLES.
REFERRED TO AS THE FOUR Fs:
FEEDING, FLEEING, FIGHTING AND
FAMILY. WE DON'T KNOW HOW TO BUILD
SYSTEMS THAT PURSUE MULTIPLE GOALS.
I WILL CLOSE BY SUMMARIZING WHAT WE
MIGHT WANT OUR FUTURE A.I. SYSTEMS
TO DO. TO RESOLVE MORAVEC'S PARADOX
TO BE MORE ENERGY EFFICIENT AND
BETTER ALIGNED TO OUR GOALS.
I'LL CLOSE WITH A SHAMELESS PLUG
NOW THAT I HAVE A PLATFORM. AT
COLD SPRING HARBOR WE HAVE A
NEUROAI PROGRAM FOR GRADUATE
STUDENTS ALREADY INVOLVED IN A.I.
TO SPEND THE SUMMER THERE.
THIS IS THE NEUROAI INTERN
PROGRAM AND AN INDEPENDENT
SCHOLARS PROGRAM FOR THOSE WHO
FINISHED THEIR TRAINING IN A.I.
AND WOULD LIKE TO LEARN MORE
ABOUT NEUROSCIENCE.
WITH THAT I'D LIKE TO TURN THE
FLOOR BACK OVER TO
JOE.
>> I'D LIKE TO TURN THE FLOOR
OVER TO DIMITRI YATSENKO CEO
AND CO-FOUNDER OF DATA JOINT
>> GOOD MORNING IT'S GREAT TO BE
HERE TO TALK ABOUT NEUROSCIENCE
AND A.I. AND WHAT IT MEANS TO DO
NEUROAI TOGETHER AND TALK ABOUT
INFRASTRUCTURE OPERATIONS AND
COLLABORATION. WE'VE COLLABORATED
WITH VARIOUS PROJECTS TO DEFINE A
UNIFIED FRAMEWORK FOR MANAGING
DATA AND COMPUTATIONS AND CODE
AND INTEGRATING THEM INTO
STANDARDIZED PIPELINES FOR
EXPERIMENTS.
WHAT MAKES NEUROSCIENCE SO
COMPLEX MORE THAN ANY OTHER
FIELD IS IT'S THE MOST MODALITY
RICH FIELD. WE COMBINE GENETIC AND
NEUROPHYSIOLOGY, ANATOMICAL DATA,
AND MOLECULAR METHODS.
AND A.I. METHODS RIGHT
NOW ARE BECOMING NOWHERE CLOSE
TO THE RICHNESS OF THE DATA WE
HAVE IN NEUROSCIENCE.
SO HOW DO WE HANDLE THIS?
RIGHT NOW THE APPROACH HAS BEEN
AS A COMMUNITY DEVELOPING A LOT
OF NEUROSCIENCE TOOLS. YOU WOULD
SEE THESE MULTIMODEL EXPERIMENTS.
THERE'S A SLIDE HERE FROM ODIN
OPEN DATA NEUROSCIENCE INITIATIVE
FROM MY TEAM AT McGOVERN INSTITUTE
AND HAS A LOT OF OPEN SOURCE
TOOLS AROUND THE DANDI ARCHIVE
BUT DATA KEEPS GROWING AT A VERY
FAST RATE. HARD TO ESTIMATE.
WE FOLLOW MOORE'S LAW IN DATA
IN NEUROSCIENCE DOUBLING EVERY
18 MONTHS OR SO. BUT THE NUMBER
OF FINDINGS DOESN'T GROW AS
FAST.
DOUBLING EVERY 10, 15 YEARS.
THAT MEANS EVERY STUDY REQUIRES
MORE DATA AND OPERATIONS.
WE NEED TO SHIFT GEARS. SOMETIMES
I THINK WE'RE LIVING IN SOMEBODY
ELSE'S DREAM. WE WOULD THINK SO
WE'RE LIVING IN A FUTURE WHERE THE
TYPES OF EXPERIMENTS FRANCIS CRICK
THOUGHT WERE IMPOSSIBLE ARE
NOW BECOMING THE VERY SUBJECT
OF OUR STUDIES.
SO, WITH THIS COMBINATION OF
NEUROSCIENCE AND A.I., NEUROAI
ALLOWS TRANSFER BETWEEN DOMAINS
WE CAN LEARN THE LESSONS FROM A.I.
AND APPLY THEM TO NEUROSCIENCE
EVEN TRANSFER THE APPROACH OF THE
SCIENTIFIC METHOD. SO CLOSING THE
DISCOVERY LOOP IS BECOMING
POSSIBLE WITH MACHINE LEARNNG IN
THE EXPERIMENT. NOT JUST LEARNING
SEPARATELY BUT COMBINING STUDIES
WHERE NEUROSCIENCE AND A.I. ARE
WORKING TOGETHER.
IT BECOMES POSSIBLE TO CLOSE THE
DISCOVERY LOOP COVERING SCIENCE
OR PORTIONS OF SCIENCE THAT WERE
EXCLUSIVELY HUMAN. THINGS LIKE
OBSERVING PHENOMENA, FORMULATING
HYPOTHESIS ARE BECOMING PART OF THE
AUTOMATED AND FORMALIZED APPROACH.
ONE EXAMPLE IS FROM ANDREAS TOLIAS'
LAB. USING MACHINE LEARNING IN THE LOOP WHERE A NEURAL NETWORK
APPROXIMATES ACTIVITY OF A NEURAL CIRCUIT TO DEFINE THE NEXT EXPERIMENT
SO THIS REQUIRES WE FIGURE OUT
WHAT MADE THE A.I. REVOLUTION
POSSIBLE. NOT JUST ALGORITHMS BUT
INCREASED OPERATIONAL EFFICIENCY.
WHAT WILL IT TAKE FOR US TO DO SAME
IN NEUROSCIENCE? ARE WE GOING TO DO SAME TYPE OF OPERATIONS AS THE TECH GIANTS?
WHEN THEY WORK WITH A.I.?
AND DEV OPS FOR SOFTWARE AND
DATA OPS AND ML OPS. THESE ARE
TECHNICAL FIELDS THAT REQUIRE
MANAGING INFRASTRUCTURE THAT RELY
ON TECHNOLOGY, CONTAINERIZATION,
VERSION CONTROL. FAST EVOLVING FOR
SCALE UP. THE FACT THAT WE'RE NOW
LIMITED BY ENERGY IN A.I. IS AN
ACCOMPLISHMENT. WE'RE NOT LIMITED
BY OUR ABILITY TO ORGANIZE, BUT WE
ARE HITTING PHYSICAL LIMIT. IN
NEUROSCIENCE WE'RE NOT THERE YET.
WE'RE NOT LIMITED BY OUR
ELECTRICITY BILL AS MUCH AS
A.I. COMPANIES.
SO CAN WE JUST TAKE WHAT A.I.
COMPANIES HAVE DONE AND APPLY IT
AND START USING IT IN OUR RESEARCH?
IT MAY OR MAY NOT BE POSSIBLE.
MAYBE THROUGH SOME COMBINATION
OF COLLABORATING BETWEEN
INDUSTRY AND SCIENCE.
SO WE ASKED THESE QUESTIONS
EXACTLY ABOUT TWO YEARS AGO.
WE HAD A GRANT TO DEVELOP
PLATFORMS FOR NEUROSCIENCE
OPERATIONS AND THE PAPER GOT
ACCEPTED FOR PEER REVIEW
WE STARTED THINKING ABOUT HOW TO
DEFINE SCIOPS LIKE ML OPS AND
DATA OPS HAVE BEEN FORMALIZED IN
THE A.I. INDUSTRY. WE SURMISE IT
WILL PROGRESS TOWARD OPERATIONAL
MATURITY AND ADOPT STANDARDS, OPEN
SOURCE, AND FOLLOW FAIR WORKFLOWS
THEY BECOME ABLE TO HAVE BIGGER
COLLABORATIONS.
AND THEN TO DO THE TYPES OF
SCIENCE WE'RE DESCRIBING HERE
REQUIRES ENTERING INTO THE NEW
PHASES OF SCI OPS AT ADOPTING
TECHNOLOGIES FOR MANAGING DATA
PIPELINES AND AUTOMATING TASKS
AND SCALING UP. AND THEN IMPLEMENT
A.I. IN THE DISCOVERY LOOP.
WHAT DOES IT MEAN IN PRACTICE?
ONE OF THE CONCLUSIONS WE MADE IS
THAT DATA, CODE, AND PROCESSES
HAVE TO BE MANAGED IN A UNIFIED
UNIFIED FRAME. IT'S REALLY HARD
TO DO THIS SEPARATELY. AND NEED
INFRASTRUCTURE THAT SEPARATES
THE SCIENTIFIC PROCESS FROM THE
INFRASTRUCTURE IMPLEMENTATION
SO THAT IT SCALES, CAN TRANSFER.
DATAJOINT FRAMEWORK BEGAN IN
ANDREAS LAB. I WORKED THERE ON THIS
PROBLEM HOW TO EXPRESS
SCIENTIFIC STUDIES AS A
COMPUTATIONAL SCHEMA AND DEPLOY
DIVERSE INFRASTRUCTURE. WITH
FUNDING FROM THE BRAIN
INITIATIVE AND COLLABORATING
WITH DOZENS OF NEUROSCIENCE
PROJECTS, WE EXPANDED TO A
PLATFORM THAT PROVIDES A COMMON
LANGUAGE FOR LABS TO EXPRESS
THEIR DATA ANALYSIS PIPELINES
FROM THE INSTRUMENT TO THE PAPER
AND SUBMITTED TO ARCHIVE AND
RELYING ON COMMON LANGUAGE
DESIGNS. OTHER PLATFORMS DEVELOPED
SIMILAR APPROACHES DATA AND
COMPUTATION MANAGED ON TOP OF THIS.
THIS IS AN EXAMPLE FROM VIRTUAL
BRAIN RUNNING ON EBRAINS PLATFORM
BY PETRA RITTER THAT COMBINE THESE
PRINCIPLES AND ALLOWS MANY COLLABORATORS TO MODEL BRAIN INPUTS.BRAINLIFE ALLOWS SIMILAR
APPROACH IN NEUROIMAGING. OTHER GATEWAYS AT UCSD ALLOWS ACCESS
TO HIGH PERFORMANCE COMPUTING TO
PERFORM ANALYSIS AT SCALE.
MOST ARE DONE IN THE LAB AND
WILL BE DONE IN THE LAB.
WE'RE WORKING MORE CLOSELY WITH
INSTITUTIONS TO PROVIDE A NEW
SERVICE. UNIVERSITIES PROVIDE I.T.
INFRASTRUCTURE BUT NOT
NECESSARILY THE DATA OPERATION
AND REQUIRES NEW LANGUAGE. A LOT
OF OPERATIONS ARE TIED TO THE GRANT
CYCLE AND IT REQUIRES A LITTLE
BIT OF LEADERSHIP WORKING TO
PROVIDE THIS AS A GENERAL THING.
WHAT DO WE SEE IN THE FUTURE?
IT'S HARD TO BUILD A.I.
COMBINATION ON TOP OF STICKY
NOTES AND SPREADSHEETS THAT
FLUCTUATES.
WE NEED TO IMPLEMENT THEM AT THE
INSTITUTIONAL LEVEL THAT ALLOWS
RESEARCHERS TO DEFINE THE LOGIC
AND STRUCTURE OF THEIR EXPERIMENTS
IN SUCH A WAY TO BE INTEGRATED
WITH COLLABORATIVE GROUPS WORKING
ON A.I.. NEEDS TO BE REPRODUCIBLE,
RELIABLE AND MANAGED WELL IN A
SUSTAINABLE WAY. THIS WILL LEAD TO
THE RECOGNITION OF TECHNOLOGY AND
ATTRACTING FUNDING AND TALENT.
>> IT'S A DANGEROUS OPERATION.
>> THANK YOU DIMITRI.
WITH THE FEW MINUTES REMAINING I
WANT TO GIVE A HIGH-LEVEL
OVERVIEW OF THE WORKSHOP AND
PARTICIPANT GUIDANCE WHAT THE
PANEL SESSION DISCUSSIONS LOOK
LIKE.
AND SHOW WHERE THE EXITS ARE SO
HOPEFULLY MANY HAVE SEEN THE
AGENDA.
IF YOU HAVEN'T SEEN THE AGENDA
IT'S ON THE WEB AT THIS QR CODE.
TODAY WE ARE THE OPENING REMARKS
AND ABOUT TO HIT INTO COFFEE
BREAK.
WE HAVE TWO SESSIONS.
MORNING SESSION, SESSION 1 IS
CONTINUING THIS DISCUSSION ABOUT
DEFINING NEUROAI FOR BRAIN AND
WHAT IS THE SCOPE THAT MATTERS
FOR THE FUTURE VISION AND NEARTERM
GAPS. WHAT ARE THE OPPORTUNITIES?
HELP US IDENTIFY PRIORITIES.
WE'LL HAVE LUNCH AND THEN END OF
LUNCH WE'LL HAVE A FUNDERS PANEL
REPRESENTATIVES FROM FUNDING
AGENCIES AND INVESTORS AND THEY
WILL HAVE A DISCUSSION MODERATED.
BY TERRY SEJNOWSKI THAT WILL HELP
LAY OUT THE FUNDING LANDSCAPE AND
FOLLOWING THAT WE'LL HAVE SESSION
TWO DIGGING INTO NEUROAI METHODS
METHODS OF MAPPING ARTIFICIAL
NEURAL NETWORK MODELS ONTO BRAIN
STRUCTURE, FUNCTION, AND ANATOMY.
WHAT CAN WE LEARN FROM METRICS AND
THEN WE'LL CLOSE OUT THE DAY
WITH A MODERATED DISCUSSION
BRINGING ALL THESE THINGS TOGETHER
FOR THE FUTURE.
SO DAY TWO FOLLOWS A SIMILAR
STRUCTURE.
WE START WITH EARLY CAREER
SESSION POSTER BLITZ STARTING AT
8:45 TOMORROW MORNING. SESSION 3
WILL INCLUDE ASPECTS OF EMBODIMENT,
PHYSICAL INTELLIGENCE, BIOMECHANICAL
FEEDBACK, THINKING ABOUT ALL THE PROBLEMS BIOLOGY
HAVE SOLVED THAT WE DON'T SEE THAT IN A.I. SYSTEMS c
AND HOW TO INCORPORATE THOSE INTO FUTURE VISIONS OF NEUROAI
SESSION 4 IS ABOUT TRANSITIONS IDEAS
INTO THEORIES AND TECHNOLOGIES THAT CAN MAKE
REAL IMPACT INTO HOW WEDO SCIENCE AND HOW
WE IMPROVE
BRAIN HEALTH.
WE ARE EXCITED TO HAVE EVERYONE HERE
FOR THIS AGENDA.
AS WE GO THROUGH THE SESSIONS,
WE'LL PUT UP A SLIDE THAT LOOKS
LIKE THIS.
THIS TELLS US ROUGHLY WHAT THE
GOALS ARE AND THERE WILL BE
QUESTIONS DURING THE DISCUSSION
PERIOD AND THINK ABOUT THE
QUESTIONS. THERE WILL BE A QR CODE
ON THE LEFT. IF YOU SCAN THAT DURING
THE SESSION IT WILL SEND AN
E-MAIL AND IF YOU'RE A VIRTUAL
AUDIENCE WE'LL READ YOUR QUESTIONS.
THERE ARE THREE MAIN PARTS TO EACH
SESSION. SHORT TALKS BY SPEAKERS,
QUESTIONS FROM INVITED DISCUSSANTS,
AND WE WILL OPEN IT UP TO AUDIENCE
Q&A AND IF YOU HAVE YOU
A CLEAR CONCRETE QUESTION READY
TO GO FOR A SPECIFIC SPEAKER
DURING THE TRANSITION, STEP UP
TO THE MIC OR SEND IN A
QUESTION AND WE'LL ASK THAT FOR
THE SPEAKERS OTHERWISE PLEASE
HOLD YOUR OPEN ENDED QUESTIONS
FOR AUDIENCE Q&A.
IF YOU'RE A PANELIST ON ZOOM, REMAIN
MUTED UNLESS YOU'RE PROMOTED TO
SPEAK. IF YOU ARE ON VIDEOCAST
AND HAVE A QUESTION HIT THE
Q&A BUTTON ON THE BOTTOM OF YOUR
SCREEN OR SEND USE THE SEND LIVE
FEEDBACK FORM. SO YOU KNOW WHERE
ARE. THERE ARE STAIRS GOING UP TO
THE ATRIUM WHERE THE POSTER SESSION
WILL BE TOMORROW. WITH THAT I WANT
TO THANK A SPONSORS SUPPORTING THE
NEUROAI WORKSHOP: THE KAVLI
FOUNDATION, JOHNS HOPKINS' KAVLI
NEUROSCIENCE DISCOVERY INSTITUTE,
WU TSAI INSTITUTE AT YALE AND DARPA.
WE'RE BEGINNING SESSION 1.
OUR FIRST SPEAKER IS ALI MINAI.
WE WILL HAVE A SEQUENCE OF SHORT
TALKS FOLLOWED BY A DISCUSSION
ALL RIGHT.
SO THANK YOU ALL FOR BEING HERE.
IT'S A GREAT PLEASURE TO BE AT
THIS WONDERFUL WORKSHOP. MY TOPIC
IS DEEP INTELLIGENCE.WHY A.I. MUST
LEARN FROM NATURE'S IMAGINATION.
THAT TERM COMES FROM THIS
WONDERFUL QUOTE FROM FEYNMAN THE
IMAGINATION OF NATURE IS FAR FAR
GREATER THAN THE IMAGINATION OF
MAN.
SO THE THING IS WHY SHOULD WE
NOT LEARN FROM THAT IMAGINATION?
HERE IS THE NEUROAI LOOP TONY
SHOWED AND OTHERS TOO.
I AM VERY MUCH GOING TO FOCUS ON
THE UPPER BRANCH ON THIS LOOP AND
ASKING MY A.I. COLLEAGUES TO PAY
MORE ATTENTION TO NEUROSCIENCE.
THERE'LL BE A LOT OF TALKS WHICH
WILL DEAL WITH THE OTHER
DIRECTION I'M SURE.
THE GOAL IS TO BUILD A MORE
NATURAL A.I. BY TAKING ADVANTAGE
OF WHAT WE HAVE LEARNED FROM
NEUROSCIENCE.
THE QUESTION IS DO WE WANT TO
BUILD A MORE NATURAL A.I.?
THAT IS NOT A VERY HOT IDEA IN ML
WHERE THE FOCUS IS ON BUILDING
HUMAN-LIKE SYSTEMS TO SOLVE
HUMAN-LIKE PROBLEMS.
A.I. AS TOOLS FOR HUMAN USE.
SOME OF US ARE HERE TO BUILD
REAL INTELLIGENCE WHICH IS MORE
LIKE THE INTELLIGENCE IN THE
ANIMAL WORLD. AND THE QUESTION IS
WHETHER WE'LL HAVE A STAR TREK
OR STAR WARS FUTURE.
I'M VERY MUCH TALKING ABOUT A
STAR WARS FUTURE WITH LOTS OF
INTELLIGENT THINGS GOING AROUND
AND MANY INTELLIGENT BEINGS.
HOW WOULD WE DEFINE NATURAL
INTELLIGENCE?
IT'S INTELLIGENCE ANIMALS HAVE.
WHAT ARE CHARACTERISTICS?
HERE ARE SOME.
IT'S BIOLOGICAL AND EMBODIED.
IT'S EMERGENT DEPENDS ON EMERGENCE.
IT'S HETEROGENOUS AND REQUIRES
HETEROGENOUS STRUCTURES AND
PROCESSES. NOT JUST A BLOB.
IT HAS MULTISCALE ORGANIZATION
IT HAS MULTISCALE AND ADAPTATION
AND IT'S AUTONOMOUS AND ALWAYS
REAL TIME AND IT'S ENVIRONMENT
SPECIFIC SO IT IS INTELLIGENT IN
ITS ENVIRONMENT.
IT'S INHERENTLY INTEGRATED UNLIKE
THE MACHINE LEARNING A.I. WE BUILD
NOW FOCUSSING ON INDIVIDUAL
TASKS AND HOPEFULLY WE'LL BE
ABLE OR HOPE TO PUT THEM ALL
TOGETHER. REAL INTELLIGENCE IS
ALWAYS INTEGRATED. THE INTELLIGENCE
OF A MOUSE OR SHRIMP IS JUST AS
INTEGRATED AS THE INTELLIGENCE OF
A HUMAN AND FINALLY IT'S EVOLVABLE.
THE COMPUTATIONAL A.I. IS VERY
DIFFERENT AND SELDOM EMBODIES
ANY OF THESE CHARACTERISTICS.
IF WE COMPARE THE TWO OF THESE
ARTIFICIAL AND NATURAL
INTELLIGENCE, OFTEN OUR MACHINE
LEARNING MODELS BEGIN WITH A
GENERAL UNDIFFERENTIATED REGULAR
STRUCTURE. THEN WE HOPE AND USES
LOTS OF DATA AND COMPUTATION AND
SOMEHOW THE ORGANIZATION NEEDED
TO ACCOMPLISH THE TASK WE'RE
TRAINING IT ON WILL APPEAR.
WE HOPE THE RESULTING SYSTEM
WILL BE USEFUL NOT JUST FOR THE
TASK YOU TRAIN IT ON BUT USEFUL
FOR LOTS OF OTHER THINGS LIKE
GENERATING TEXT AND REASONING
AND PLANNING AND ALL THE THINGS
WE TRY TO USE THE SYSTEMS FOR.
THE PROBLEM IS ONE, THE MODELS
ARE TASK SPECIFIC.
TWO, WE'RE GOING FROM A SYSTEM
THAT'S NAIVE TO A SYSTEM WITH
FUNCTIONALITY, DATA INTENSIVE AND
DIFFICULT IN GENERAL.
WE DON'T HAVE ANY CONTROL IN
THAT PROCESS OVER WHAT KIND OF
SYSTEM WILL EMERGE.
SO IN PRINCIPLE A WEIRD MODEL
COULD EMERGE.
IN SOME CASES, FOR EXAMPLE,
IN VISION THERE'S BEEN WORK WHICH
SHOWS THERE IS A CORRESPONDENCE
BETWEEN THE MODELS THAT EMERGE AND
THE MODELS BRAINS MAY BE USING.
DOES THIS HAPPEN IN LARGE LANGUAGE
MODELS? BECAUSE WE DON'T HAVE
CONTROL OVER THE MODEL, EVEN IF IT
WORKS WELL ON EVERYTHING WE TEST,
WE DON'T KNOW WHAT KIND OF HIDDEN
FAILURE MODES IT HAS AND WHEN
THEY MIGHT SHOW UP.
SO THIS SORT OF THING -- THE
QUESTION EXISTS WITHIN THE
MACHINE LEARNING COMMUNITY AND
THERE'S VARIOUS WAYS TO TRY TO
ADDRESS THAT WHICH I WILL PUT UP.
ML TODAY USES SHALLOW ADAPTATION.
USE DATA AND TRAIN THE SYSTEM ON
LOTS OF DATA AND HAVE A SYSTEM
THAT PERFORMS WELL ON ALL THE
TASKS WE PUT IT THROUGH.
IT'S UNLIKE HOW ANIMALS LEARN
WHICH IS THIS PROCESS WHICH I'M
CALLING DEEP ADAPTATION.
DEEP IN EVOLUTIONARY HISTORY AND
SOME SIMPLE ANIMALS EMERGED AND
THEY DEVELOPED OR EVOLVED INTO
MORE COMPLEX ANIMALS AND WHEN WE
GET TO A PARTICULAR ANIMAL WHETHER
A RAT CAT OR HUMAN, WE HAVE
WE HAVE 500 MILLION YEAR OF
EVOLUTIONARY ENGINEERING THAT
EXISTS.
THAT'S WHY SO LITTLE LEARNING
IS OFTEN NEEDED FOR ANIMALS TO
LEARN TO ACCOMPLISH COMPLICATED
TASKS AND WHY SO LITTLE ENERGY
IS NEEDED AND ALL SORTS OF OTHER
THINGS. AND EVEN AFTER THAT, THE
LEARNING IS NOT JUST DOING
GRADIENT DESCENT PROCESS OR
SUPERVISED LEARNING. IT IS GOING
THRU A DEVELOPMENT PROCESS. THE BRAIN AND BODY BECOMES COMPLEX TOGETHER AND LEARN IN STAGES
BUILDING UPON PREVIOUS STAGES. AND THEN OF COURSE DEVELOPMENT
AND NEURAL LEARNING GO HAND IN
HAND BASE ON EXPERIENCE OF WORLD.
WE SHOULD NOT JUST LEARN FROM
NEUROSCIENCE BUT DEVELOPMENTAL
AND EVOLUTIONARY BIOLOGY AS WELL.
WHAT SHOULD WE LEARN FROM NATURE'S
IMAGINATION? FROM NEUROSCIENCE?
ARCHITECTURES, MODULES, LEARNING
RULES. THINGS WE ALREADY LEARNED
FROM NEUROSCIENCE BUT ALSO FOCUS
ON HOW EVOLUTION BUILDS COMPLEXITY
IN ANIMALS AND BRAINS AND BODIES
HOW ONTOGENESIS DEVELOPS FROM A
SIMPLE CELL TO A COMPLEX ORGANISM.
DEVELOPMENTAL LEARNING,
UNSUPERVISED AND REINFORCEMENT
LEARNING THEN AND AT END OF THAT
SUPERVISED LEARNING ON A SYSTEM
READY FOR SUPERVISED LEARNING.
RIGHT NOW WE'RE TRYING TO GET
ALMOST EVERYTHING FROM THE LAST
KIND OF LEARNING.
AND WE SKIP ALL THE OTHER
THINGS.
SO INSTEAD OF A NEUROAI TO HAVE
A EVO DEVO NEUROAI. NOT SAYING WE
SHOULD DO COMPUTATIONAL EVOLUTION
BUT I'M SUGGESTING WE SHOULD
LEARN THE LESSONS FROM EVOLUTION
AND MINE WHAT WE KNOW FROM
EVOLUTION. SO WE HAVE
SIMPLE AND MORE COMPLEX SPECIES
AND WE SHOULD LOOK AT ALL OF
THOSE AND THEIR BODIES AND HOW
THEY WORK AND INCORPORATE INTO
OUR A.I. MODELS. TO DEVELOP A CANONICAL SET OF STRUCTURAL MODULES, FUNCTIONAL PRIMITIVES AND
COMPLEXIFICATION RULES THAT WOULD
ENABLE US TO BUILD MORE COMPLEX
A.I. SYSTEMS IN A MORE BIOLOGICAL
AND SYSTEMATIC WAY.
HERE'S COMPLEXIFICATION GOING ON
ONE EXAMPLE DEVELOPMENTAL LEARNING.
THE POINT IS WHEN WE DO ALIGNMENT
AND IF WE DID THAT IN A MORE
DEVELOPMENT WAY WE'D BE ABLE TO
DO IT BETTER AND TO HAVE IT
BE MORE NATURAL.
ALL RIGHT.
THANK YOU VERY MUCH.
>> I'M SAD TO HEAR I ONLY HAVE
59 MINUTES AND 12 SECONDS TO
SPEAK.
I LOOK FORWARD TO DISCUSSING THE
POTENTIAL ROLE OF ASTROCYTES IN
BIOCOMPUTING. I'VE BEEN PART OF
THE BRAIN INITIATIVE CENTER TO
STUDY CROWD CODING IN THE BRAIN
IN THE DATA SCIENCE CORE.
AND LOOKING AT NATURE BRAINS CAN
WE LOOK AT ASTROCYTES AND EXPLORE
THEIR ROLES IN MAKING SENSE OF
THEIR SURROUNDING WORLD AND
FUNDAMENTALLY IN BIOCOMPUTING.
THE REASON I MIGHT BE WORTH
LOOKING AT OTHER MODES OF
COMPUTING IS THOUGH A.I. HAS
GOTTEN INSPIRATION FROM
NEUROSCIENCE, HUMANS STILL
OUTPERFORM MACHINES IN MINIMAL
SUPERVISION AND TRAINING,
LIMITED DATA.
IT WOULD BE VERY SAD IF HUMANS
COULD LEARN ALL THE KNOWLEDGE IN
THE WORLD AND THEN NOT DECIDE
WHAT TO DO WITH IT. A HAWK WITH
1 WATT OF POWER AND 15 GRAMS OF
BRAIN MASS CAN RECEIVE DATA FROM
THOUSAND OF SENSORS AND LEARN
RAPIDLY. THE QUESTION IS DO WE
NEED NEW ARCHITECTURES TO HARNESS
THE COMPLEXITY OF THE HUMAN BRAIN?
I WANT TO TAKE SIMPLER APPROACH
AND ASK THE QUESTION
WHAT ABOUT ASTROCYTES?
50% OF OUR BRAIN MASS OR SO ARE
ASTROCYTES.
WHAT ROLE COULD THEY PLAY?
THE OTHER QUESTION I'M GOING TO
ASK IS WHAT ABOUT NON-ELECTRICAL
SIGNALS IN THE BRAIN?
HOW DO THEY CONTRIBUTE TO NEUROAI?
I WANT TO THINK ABOUT NEUROAI AS A
BIOCOMPUTING SYSTEM. WE HAVE
NEURONS AND ASTROCYTES. THEY'RE
DIFFERENT FROM HOW A QUANTUM
COMPUTER IS EMERGING AS A NEW
WAY OF PROCESSING INFORMATION
WITH ATOMS, PHOTONS, ELECTRONS.
WHAT'S AMAZING ABOUT
BIOCOMPUTING WITH ONLY 139,000
NEURONS A FLY CAN ALREADY CARRY
OUT VERY COMPLEX TASKS, DO
SOMETHING INTELLIGENT. THAT'S
SIMILAR TO A QUANTUM COMPUTER
REACHING THE THRESHOLD FOR
INTELLIGENCE AND VERY DIFFERENT
FROM THE BILLIONS OF NODES THAT
HAVE BEEN NEEDED FOR CHATGPT 4.
THE ADVANTAGES OF THE DIFFERENT
MODALITIES MAY BE DISTINCT.
CHATGPT 4 IS AMAZING TO HAVE
DATA STORED BUT BIOCOMPUTERS ARE
POWERFUL MAKING DECISIONS FROM
LIMITED DYNAMIC DATA, LEARNING,
EXTRAPOLATING TO NEW SITUATIONS.
ARE THERE OTHER SIGNALS?
AND WE'VE BEEN LOOKING AT
CHEMICAL SIGNALS OF ASTROCYTE.
THEY'RE NOT JUST PASSIVE HOMEO-
STATIC PROVIDERS OF NUTRITION TO
THE BRAIN BUT ALSO YIELDING THEIR
OWN INTRINSIC RHYTHM AND IF YOU
LOOK AT THE CYTOSKELETON OF
INDIVIDUAL CELLS, ACTIN POLYMERIZATION, IT'S NOT JUST FORMING SYNAPSES ON DEMAND
BUT HAS IT'S OWN INTRINSIC
RHYTHM ON THE MINUTE SCALE.
WHAT EMERGES IS A PICTURE OF THE
BRAIN WHERE YOU HAVE DIFFERENT
SIGNALS, ELECTRICAL ON THE MS
SCALE, CHEMICAL SIGNALS FROM
ASTROCYTES ON 10-20 SECOND SCALE,
SLOWER MINUTE TIME SCALE RHYTHMS
OF THE CYTOSKELETON. DO SLOW
RHYTHMS CONTRIBUTE OR A SUBSTRATE
THAT ELECTRICAL SIGNALS OPERATE?
AND THERE'S BEEN MECHANICAL
SIGNALS OF SYNAPSE AND EVIDENCE OF
DIRECT COUPLING. AND WE ASKED THE
SIMPLE QUESTION, IF YOU HAVE
MECHANICAL TRANSMISSION THROUGH
SYNAPSES WHAT IF THE MECHANICAL
SYSTEM IS NOT A STEADY SYSTEM.
IT'S AN EXCITABLE SYSTEM IN ITS
OWN RIGHT.
RUNNING ON THE MINUTE TIME SCALE
WHAT WOULD IT DO?
WE LOOKED AT A SIMPLE ALGORITHM.
WHAT IF WE HAD MULTI MODAL
SIGNAL AND WHAT IF YOU VARIED IT
AND THE QUESTION IS IT SUCKS.
IT GETS WORSE BECAUSE YOU'RE
VARING IT AND YOUR FORCED TO
STORE IN THE SYSTEM AND MORE
ROBUST AS TIME GOES ON THEN ASK
YOURSELF HOW DO YOU LEARN IN A
SEESAWING RHYTHMIC SYSTEM.
WE DISCOVERED YOU CAN COME UP
WITH A NEW LEARNING ALGORITHM
BASED ON SYNCHRONIZING AND THE
SIMPLE ALGORITHM WE'RE ABLE TO
GENERATE DYNAMIC TWINS THAT
RAPIDLY EXTRAPOLATE DYNAMICS AND
PROMISING TO GO IN THE RIGHT
DIRECTION OF EXTRAPOLATION AND
RAPID LEARNING. THIS IS A SMALL
EXAMPLE OF LOOKING BEYOND THE
ELECTRICAL SIGNAL OF NEURONS
AND EXPLORING THE BIOLOGICAL
RHYTHMS AT MULTIPLE LENGTHS AND
TIME SCALE. NOW WITH THAT, WHAT
WE WOULD NEED IS TECHNOLOGY TO
SYSTEMICALLY LOOK AT THESE RHYTHMS
ON THE MILLISECOND TIME SCALE
AND IMAGING ON SECOND TO MINUTE
TIME SCALE AND SPATIAL SCALE.
AND TO ACCOMPLISH THIS I WANTED
TO HIGHLIGHT THE BRAIN
INITIATIVE HAS BEEN GREAT IN
SUPPORTING INTERDISCIPLINARY
WORK AT THE BOUNDARIES BETWEEN
BIOENGINEERING AND BIOLOGY AND
NEUROSCIENCE AND PHYSICS AND I
WANT TO ACKNOWLEDGE SUPPORT FROM
AFOSR AND ARL AND THANK YOU.
>> I'LL SET MY TIMER.
THANKS FOR HAVING ME HERE.
SO MY NAME IS PATRICK MINEAULT
AND WORKING ON ISSUES WITH A.I.,
NEUROSCIENCE AND A.I. SAFETY.
NOW, SUCH AN AMAZING OCCASION TO
MEET UP WITH YOU INVOLVED IN THE
BRAIN INITIATIVE.
WE HAVE LEARNED SO MUCH ABOUT
THE BRAIN IN THE LAST 10 YEARS
FROM THE CONNECTOME WITH FLIES
AND HAVING THE ABILITY TO HAVE A
LOT OF INFORMATION ABOUT CELL
TYPES AND FUNCTIONAL ACTIVITY AND
APPLICATIONS IN HUMAN NEUROSCIENCE.
MEANWHILE A.I. HAS ADVANCED
TREMENDOUSLY DURING IN THE SAME
TIME PERIOD.
OF COURSE, I DON'T NEED TO TELL
YOU THAT WHEN YOU LOOK AT
BENCHMARKS IN A.I., THERE'S
TREMENDOUS IMPROVEMENTS IN THE
LAST COUPLE OF YEARS.
IT SEEMS EVERY TIME THERE'S A
NEW BENCHMARK THAT COMES OUT IT
ONLY TAKES A FEW YEARS AND
SOMETIMES MONTHS TO ACTUALLY
SATURATE IT.
WE'VE NOW INTEGRATED LARGE
LANGUAGE MODELS TO EVERYDAY
WORK.
WE'VE SEEN THE NOBEL PRIZE TO
ALPHAFOLD FOR APPLICATIONS IN
PROTEIN FOLDING. GENERATE PHOTOS
THE ASPIRATION OF NEUROAI IS TO
BRING TWO FIELDS TOGETHER AND I'D
BE REMISSED NOT TO HAVE A FIGURE
LIKE THIS. WE'RE LEARNING FROM
NEUROSCIENCE TO BRING INTO A.I.
BUT WE ARE GETTING A LOT MORE FROM
A.I. THAN A.I. IS GIVING US.
I WANT TO TRY AND DIAGNOSE WHY
THIS IS THE CASE. THE FACT THAT
WE HAVE TO DO EXPERIMENTS IN THE
REAL WORLD ON BIOLOGICAL SYSTEMS
MEANS NEUROSCIENCE IS ALWAYS
GOING TO BE SLOWER THAN
ARTIFICIAL INTELLIGENCE.
AS A POINT HERE, IF YOU LOOK AT
THE PAPER SUBMITTED IN JULY 2023
IT NOW HAS 9,500 CITATIONS. THIS
IS AN A.I. PAPER BUT I DON'T KNOW
OF A SINGLE PAPER IN
NEUROSCIENCE THAT WOULD HAVE
THAT NUMBER OF CITATIONS IN A
YEAR AND A HALF SINCE THE
PUBLICATION. THINGS GO A LOT
FASTER IN A.I. BECAUSE
WE HAVE THE ABILITY TO DUPLICATE
MODELS AND DON'T HAVE TO GO
THROUGH THE BIOLOGICAL LOOP.
THE RATE OF PUBLICATION IS
HIGHER AND HOW CAN WE CONTINUE
TO INSPIRE A.I.? AND THERE'S GOOD
REASONS WHY WE WANT TO CONTINUE
TO IN SPIRE A.I.
ONE IS HUMAN BRAIN IS THE ONLY
EXAMPLE OF TRUE GENERAL
INTELLIGENCE THAT'S FLEXIBLE AND
COMPOSABLE AND EMBODIED.
THE HUMAN BRAIN IS THE ONLY
TRUE SAFE GENERAL INTELLIGENCE
SO WE'RE ABLE TO NEGOTIATE,
COOPERATE AND UNDERSTAND EACH
OTHER'S ACTIONS. OUR BRAINS ARE
ENERGY EFFICIENT. TO ACCELERATE
NEUROSCIENCE TO CONTINUE BUILDING
THE BRIDGE TOWARDS A.I.
WHAT I ADVOCATE FOR IS TO
LEVERAGE VIRTUAL NEUROSCIENCE.
SO IF WE'RE ABLE TO STUDY
VIRTUAL BRAINS RATHER THAN REAL
BRAINS WE'D HAVE ACCESS TO A
FULLY OBSERVABLE STATE AND
REPRODUCIBLE AND HAVE CAUSAL
ABLATIONS, INTERVENTIONS, EDITS AND
BE FASTER THAN THE CURRENT MO.
AND REALLY THE TEMPLATE I'M
THINKING ABOUT FOR THIS IS WHAT
WE'VE SEEN IN FLY NEUROSCIENCE.
IT IS FAR FROM SYSTEMS NEUROSCIENCE.
BUT I'M ASTOUNDED BY THE
ADVANCES THAT WE'VE BEEN ABLE TO
DO FROM CREATING THE CONNECTOME
OF A FLY AND ABLE TO RECONSTRUCT
TRACKS AND VIRTUALIZE THE BRAINS
AND BODIES AND FIGURE OUT
ESSENTIALLY THE ENTIRE TRACK OF
TRANSFORMATIONS FROM THE FIRST
SYNAPSE TO THE MORE CENTRAL
LOCATION INSIDE THE FLY'S BRAIN.
IF WE HAD SOMETHING LIKE THAT
FOR MAMMAL NEUROSCIENCE I THINK IT
WOULD BE TRULY TRANSFORMATIVE. SO
WHAT DOES THAT MEAN TO HAVE ACCESS
TO VIRTUAL NEUROSCIENCE?
THERE'S MANY THINGS TO TAKE INTO
ACCOUNT INCLUDING A WORLD MODEL.
WE WOULD NEED TO TAKE INTO ACCOUNT
THE VIRTUALIZATION OF THE WORLD
AND INPUTS OF THE
SYSTEM.
WE WOULD NEED TO TAKE INTO
ACCOUNT THE BRAIN AND
MEASUREMENTS WE TAKE OUT OF IT
AND THE POSITION OF THE BODY
WITHIN ITS ENVIRONMENT.
BUT WE'VE SEEN A LOT OF THE
BASIC BUILDING BLOCKS BEING
BUILT.
THAT INCLUDES THIS MAPPING FROM
INPUTS TO MEASUREMENT. WE NOW HAVE
ACCESS TO NEURAL NETWORKS THAT
CAN ACCOUNT FOR PROCESSING
THAT WE SEE IN THE VENTRAL VISUAL
STREAM. AND THERE'S DATA DRIVEN
NEURAL NETWORKS AND WE WILL HEAR
HOW TO BUILD DIGITAL TWINS OF THE
SYSTEMS. WE HAVE ACCESS TO
AMAZING TECHNOLOGY TO TRACK THE
POSITION OF THE BODIES OF
ANIMALS AND VIRTUALIZATION
TECHNOLOGIES.
AND FINALLY WE'RE SEEING A
REVOLUTION IN OUR ABILITY TO
AGGREGATE INFORMATION THRU
FOUNDATION MODELS. THERE'S OVER
100,000 HOURS IN DATA ARCHIVES
MANY SUPPORTED BY THE BRAIN
INITIATIVE AND THE NIH.
SO IF WE COULD BRING ALL THESE
DATA TOGETHER I THINK WE'D HAVE
A FAST PATH TOWARDS VIRTUAL
NEUROSCIENCE.
WHAT'S THE RECIPE?
WE'RE TALKING ABOUT DIGITAL
TWINS, AUTO REGRESSIVE FOUNDATION
MODELS, VIRTUAL BODIES, AND
VIRTUAL ENVIRONMENTS.
TO BUILD THE VIRTUAL BRAINS WE
WANT TO LEVERAGE THE EXISTING
DATA ARCHIVES.
BRING CLOSED DATA IN ADDITION TO
THAT. ACCELERATE COST-CUTTING DATA
ACQUISITION TECHNOLOGIES AND
CONNECTOMICS, WIRELESS
ELECTROPHYSIOLOGY, AND POOL DATA
ACQUISITION ACROSS MULTIPLE LABS
UNDER HIGH-ENTROPY CONDITIONS.
AND POOLED RESOURCES TO BUILD
TWINS, FOUNDATION MODELS AND
VIRTUAL BODIES AND DISSEMINATE
TO THE RESEARCH COMMUNITY.
THE REALITY OF NEUROAI MAY SEEM
LIKE THERE'S MORE WE'RE GETTING
OUT OF A.I. BUT THROUGH THE PATH
I THINK WE COULD POTENTIALLY
MAKE THE FUTURE OF NEUROAI WHERE
THE ARROW AT THE TOP IS LARGER.
THANK YOU VERY MUCH.
>> THANK YOU.
THANK YOU FOR THE ORGANIZERS FOR
THIS GREAT WORKSHOP.
SO, ARE THESE DOGS OR MUFFINS?
WHEN YOU ASSEMBLE THEM AND SHOW
THEM TO THE MODELS IN A.I., THEY
FAIL AND EVEN SOME CANNOT EVEN
PARSE THE IMAGES TO GET THE
RIGHT NUMBER OF ROWS AND COLUMNS.
THE REASON THEY FAIL IS BECAUSE
OF GENERALIZATION.
IT'S THE KEY INGREDIENT OF
INTELLIGENCE OF ARTIFICIAL AND
NATURAL AND WHAT I MEAN BY
GENERALIZATIONS IS THE ABILITY OF
AGENTS, HUMANS, A.I. SYSTEMS TO
DRAW INFERENCES AND MAKE
PREDICTIONS OF NEW DATA YOU'VE
NEVER SEEN BEFORE OR ON DATA
OUTSIDE THE TRAINING
DISTRIBUTION. AND BIG TECH'S
ANSWER TO THIS GENERALIZATION IS
ILLUSTRATED HERE WHERE FREDDY ASKED
JOHNNY DEPP MY MODEL CANNOT DO IT.
ANSWER IS TRAIN IT UNTIL THERE IS
NOTHING LEFT OUT OF DISTRIBUTION.
THIS IS WHY WE NEED
LARGE COMPUTE AND MORE LARGE
DATA TO BUILD THESE MODELS.
AND WHILE THIS IS GOING TO
CONTINUE HAPPENING AND WE'LL SEE
VERY IMPORTANT IMPROVEMENTS WE
MAY BE RUNNING OUT OF DATA.
FOR EXAMPLE, THERE'S EVIDENCE
WE'RE RUNNING OUT OF HIGH
QUALITY TEXT DATA AND IN MANY
CASES WE JUST DON'T HAVE ENOUGH
DATA TO BUILD POWERFUL A.I.
MODELS.
SO WE NEED NEW ALGORITHMS.
AS WE HEARD IN THE PREVIOUS
TALKS WE NEED TO DRAW
INSPIRATION FROM THE BRAIN.
NEUROSCIENCE IS ALL ABOUT FINDING
CORRESPONDENCES, IN SOME CASES,
ISOMORPHISMS BETWEEN THE EXTERNAL
WORLD OUR PERCEPTION OUR ACTIONS
OUR NEUROCOGNITIVE STATES AT
DIFFERENT LEVELS OF ANALYSIS.
A PARTICULAR LEVEL THAT'S BEEN
FRUITFUL, AT THE REPRESENTATIONAL
LEVEL, WE FOUND CORRESPONDENCES
BETWEEN OUR PERCEPTIONS OUR ACTIONS
OUR LOCATIONS IN SPACE AND
NEURAL ACTIVITY IN THE BRAIN.
AT THIS LEVEL THERE'S AN
INTERESTING FACT OF NEURAL
CODING THERE ARE PRINCIPLES THAT
TRANSCEND THE SUBSTRATE.
WHAT DO I MEAN BY THIS IS IF YOU
TAKE ARTIFICIAL NEURAL NETWORK AND
TRAIN IT IN OBJECT RECOGNITION AND
LOOK INSIDE THE BLACK BOX YOU'LL
FIND REPRESENTATIONS THAT LOOK
SIMILAR TO WHAT WAS STUDIED IN
THE BRAIN.
HERE'S DIFFERENT ARTIFICIAL
NEURAL NETWORKS AND YOU SEE IF
YOU LOOK AT THE CERTAIN LAYER
WITHIN THE ARTIFICIAL NEURAL
NETWORK YOU HAVE DETECTED
CURVATURE OR DIFFERENT PATTERNS
AND IF YOU DO THE SAME FOR THE
AREA BEFORE A MID LEVEL AREA
IN THE MACAQUE BRAIN YOU START
FINDING SIMILAR FEATURES.
HERE'S ANOTHER EXAMPLE OF THE GRID
CELLS IN THE ENTORHINAL CORTEX
THAT ENCODE SPATIAL INFORMATION
IN VISION YOU FIND SIMILAR
NOW WE HAVE A WAY TO COMPARE
ARTIFICIAL NEURAL NETWORKS IN
BRAINS IN THE SAME WAY AND THEN
LOOK FOR SIMILARITIES AND
DIFFERENCES.
THIS IS A FRUITFUL APPROACH TO
TRAIN ADVANCED A.I. AND
DIFFERENCES IN THE PRESENTATION
IS WHAT ENABLES A.I. SYSTEMS
IF YOU LOOK AT THE HISTORY OF
STUDYING REPRESENTATION AND
INFORMATION PROCESSING OF NEURAL
CODE, IT'S MOSTLY SERENDIPITOUS.
HERE'S AN EXAMPLE OF HUBEL AND
WEISEL WHERE THEY DISCOVERED
ORIENTATION SELECTIVITY AND WON
THE NOBEL PRIZE FOR IT. THIS 60-MINUTE CLIP DESCRIBED THE SERENDIPITOUS DISCOVERY THAT ONE DAY THE SLIDE
THEY WERE USING PRODUCED THIS THIN LINE. AND THEY FOUND THAT THIS WAS DRIVING THE NEURONS.
NOW BELIEVE IT OR NOT, THIS INSPIRED FUKUSHIMA AND GAVE RISE
TO CONVOLUTIONAL NEURAL NETWORKS.
A HALLMARK OF THE A.I. REVOLUTION.
IF YOU LOOK AT THE BRAIN OF THE
MACAQUE THERE'S THOUSANDS OF VISUAL
AREAS, MANY CORTICAL LAYERS,
HUNDREDS OF CELL TYPES. SO HOW TO
DECIPHER REPRESENTATIONS? FOLLOW
FOLLOW THIS SERENDIPITOUS PROCESS.
IF YOU HAVE TO MONITOR, 256 X 256
PIXELS AND JUST WANT TO STUDY THE
SYSTEM YOU CAN GENERATE TWO TO
65,000 POSSIBLE PATTERNS AND IF YOU
TAKE INTO ACCOUNT BEHAVIORAL
STATES AND DIFFERENT INTERNAL
STATES AND MODALITIES IT BECOMES
A VERY COMPLEX PROBLEM. WE ARE IN
A GOLDERN ERA BECAUSE OF COLLECTING
DATA AT SCALE. THANKS TO THE BRAIN
INITIATIVE IN THE LAST DECADE WE
HAVE THESE AMAZING TECHNOLOGIES.
AND THE OTHER IS THAT WE HAVE
THE A.I REVOLUTION WITH THE
TECHNOLOGY TO ENABLE TO INGEST
LARGE SCALE DATA AND FIND
RELATIONSHIPS AND MAKE PREDICTIONS.
WE CREATE PLATFORMS TO COLLECT
LARGE SCALE DATA, USE MACHINE
LEARINGTO BUILD PREDICTIVE MODELS,
USE LATENT AND WORLD VARIABLES,
BRAIN ACIVITY AND BEHAVIOR
THEN ONCE YOU HAVE THE DIGITAL
TWINS, NOW YOU CAN IMAGINE YOU
CAN SCALE THEM AND RUN WHAT IT
WOULD TAKE HUNDREDS OF YEARS AND
SOMETIMES THOUSAND OF YEARS. IF
YOU HAVE A LARGE ENOUGH
INFRASTRUCTURE, YOU CAN RUN IT IN
2 DAYS. MAKE DISCOVERIES, GENERATE
HYPOTHESIS THAT YOU CAN TEST.
HERE'S AN EXAMPLE OF A STUDY WE
DID A COUPLE YEARS AGO WHERE WE
WANTED TO STUDY HOW THE
SENSITIVITY OF THE MOUSE VISUAL
SYSTEM TUNING CHANGES DURING THE
CHANGE OF COGNITIVE STATES.
WE SHOWED NATURAL IMAGES WITH
COLOR AND BUILD A DIGITAL TWIN OF
THIS MODEL. YOU CAN SEE HERE WE
DISCOVERED WHEN THE PUPIL
DILATES THE COLOR SELECTIVITY
CHANGES TO UV BUT SPATIAL STRUCTURE
DIDN'T. THIS WASN'T A HYPOTHESIS
DRIVEN DISCOVERY. IT WAS GENERATED
BY THE MODEL AND COULD BE VERIFIED
INSILICO. IN THE NEXT DECADE WE'RE
IN THIS TIME IN NEUROSCIENCE AND
A.I. WHERE IF YOU THINK ABOUT IT,
A.I. IS ALL ABOUT GETTING HUMAN
BEHAVIORAL DATA AND TRAINING MODELS
TO MEET HUMANS INTELLIGENCE SYSTEMS.
NOW WE HAVE OPPORTUNITIES TO GET
NEURAL DATA AT LARGE SCALE AND
PRODUCE A.I. SYSTEMS THAT WILL
NOT JUST HAVE BEHAVIORAL
ALIGNMENT WITH OUR BRAINS BUT ALSO
NEURAL ALIGNMENT.
I'D LIKE TO CLOSE WITH THIS QUOTE
FROM DANIEL DENNETT: WE HOPE TO
BUILD A.I. SYSTEMS AND NOT JUST BE
COMPETENT BUT WILL COMPREHEND THE
WORLD. HE'S USING AN EXAMPLE OF
TERMITE MOUNDS. THEY ARE BEAUTIFUL
STRUCTURE AND ONE BUILT FROM SIMPLE
RULES NOT COMPREHENDING BUILDING
THE MASTER PIECE. THANK YOU TO THE
BRAIN INITIATIVE FOR PROVIDING
FUNDING.
>> ALL RIGHT.
HELLO, EVERYONE. EXCITING
WORKSHOP. GLAD TO BE HERE.
THANKS TO THE ORGANIZERS FOR
INVITING ME.
SO, WHAT I WOULD LIKE TO DO IS
TO MAKE A PITCH FOR AN IDEA OF
WHAT WE AS A COMMUNITY COULD
STUDY AMONG OTHER THINGS OF
COURSE.
IN THE NEXT DECADE OR SO AND
NAMELY WHAT I WANT TO ARGUE FOR
IS HIGHLY DETAILED BIOLOGICALLY
REALISTIC SIMULATIONS OF BRAINS AT
THE WHOLE BRAIN LEVEL OF MAMMALS
IN THE NEXT 10 YEARS AND WHY?
WE CARE ABOUT UNDERSTANDING THE
BRAINS AND UNDERSTANDING HOW
THEY WORK AND FIXING THAT.
AND I THINK SUCH SIMULATIONS
COULD CONSTITUTE EXCELLENT
PLATFORMS FOR DISCOVERY AND
IN SILICO EXPERIMENTATION.
I'LL SAY MORE WORDS ABOUT THAT
AT THE END BUT FIRST LET'S THINK
ABOUT WHAT'S POSSIBLE NOW.
I WANTED TO SHOW THE MOVIE WHICH
ILLUSTRATES THE MODEL OF THE
MOUSE PRIMARY VISUAL CORTEX WE
DEVELOPED AT THE ALLEN INSTITUTE
AND PUBLISHED A FEW YEARS AGO.
THIS INTEGRATED WORK FROM OTHER
SOURCES FROM THE CELL TYPES AND
DATA PROPERTIES, CONNECTIVITY,
IN VIVO ACTIVITY. THIS MODEL COMES
WITH VISUAL INPUT SO WE CAN STUDY
STUDY NEUROPHYSIOLOGICAL ACTIVITY
IN THIS SIMULATION.
WE MADE THE MODELS PUBLICLY
AVAILABLE. WE HAVE TWO VERSIONS.
WE HAVE A DETAILED VERSION AND A
POINT-NEURON VERSION. FOR THE
METRICS WE CAN COMPUTE SPIKING
ACTIVITIES IT REPRODUCES WELL.
BOTH MODELS HAVE A QUARTER
MILLON NEURONS AND A LOT OF
COMPLEXITY.
OTHER GROUPS CAME OUT WITH
SIMILAR MODELS AND I FEEL LIKE
WE'RE AT THE POINT WHERE THIS IS
POSSIBLE AND CAN MAKE LARGER
SIMULATIONS AND MODELS.
MULTIPLE GROUPS ARE USING OTHER
OUR AND OTHER MODELS AVAILABLE.
I WANT TO HIGHLIGHT THE WORK OF
A COLLABORATOR WHO PORTED OUR
MODEL TO TENSOR FLOW AND WERE ABLE
TO TRAIN IT. ONE EXAMPLE HERE
THE CURVE IN BLUE SHOWS THE BIO-
REALISTIC MODEL IS ORDERS OF
MAGNITUDE MORE ROBUST TO NOISE
IN THE HANDWRITTEN DIGIT
RECOGNITION TASK. A LOT CAN BE DONE
ON THIS MODEL STILL KEEPING THE
BIOLOGICAL CONSTRAINTS.
WHAT I BELIEVE IS POSSIBLE IN
THE NEXT DECADE IS GOING BIGGER
AND GOING TO THE WHOLE BRAIN.
WHAT'S ALREADY POSSIBLE NOW ARE
SIMULATIONS OF THE BRAINS OF A
A WORM, A FLY AS WE'VE SEEN IN
BEAUTIFUL PUBLICATIONS IN RECENT
MONTHS.
I BELIEVE ZEBRAFISH IS GETTING
THERE. WHAT CAN BE POSSIBLE SOON
IS A SIMULATION OF THE WHOLE MOUSE
BRAIN OR SUBSTANTIAL PIECE, 1 CC,
OF A HUMAN OR MONKEY BRAIN
AT CELLULAR RESOLUTION.
WE HAVE THE TOOLS TO DO THAT AND
THERE HAVE BEEN PREPRINTS WHERE
SIMULATIONS ON THE SCALE OF A
HUMAN BRAIN WAS DEMONSTRATED.
TOOLS SHOWN HERE AND FOR FAST
SIMULATIONS, DENDRITIC HIERARCHICAL
SCHEDULING, OR JAXLEY TOOL THAT
SHOW EXCITING RESULTS OF BIO-
PHYSICALLY DETAILED SIMULATIONS
OF GPUs AND OF COURSE WE NEED
THE DATA TO MAKE IT REALITY AND
A LOT OF THE DATA AS A BEEN
COLLECTED ALREADY AS PART OF THE
BRAIN INITIATIVE.
ON ONE HAND ONE KEY COMPONENT IS
GOING TO BE THE CONNECTOME OF
THE WHOLE MOUSE BRAIN.
THERE'S GROUPS WORKING ON THIS
ALREADY EITHER USING ELECTRON
OR EXPANSION LIGHT MICROSCOPY.
AND THAT'S NOT ENOUGH.
WE NEED TO IMBUE THE STRUCTURE
ONE LIFE BY COLLECTING DATA ON
ELECTROPHYSIOLOGY AND BIOCHEMICAL
PROPERTIES OF CELLS AND CONNECTION
AND DATA CONNECTIVITY IN VIVO.
THAT'S IMPRACTICAL TO ACHIEVE AT
THE LEVEL OF THE WHOLE BRAIN FOR
EVERY SINGLE CELL IN THE NEXT 10
YEARS.
BUT WHAT CAN BE DONE, I BELIEVE,
IS FOCUSING EXPERIMENTS ON
TRAINING A.I. TO PREDICT FROM
DATA WHAT THE PROPERTIES OF
THOSE BRAIN CELLS AND
CONNECTIONS ARE GOING TO BE
BASED ON A SERIES OF EXPERIMENTS
IN VARIOUS PARTS OF THE BRAIN.
ULTIMATELY, I BELIEVE THERE'S A
SPACE HERE FOR A PIPELINE THAT
CREATES BLUEPRINTS FOR
SIMULATIONS REGARDING WHAT
PROPERTIES THOSE COMPONENTS
SHOULD HAVE.
THIS IS FEASIBLE AND WE MAY BE
ABLE TO ACHIEVE WHOLE BRAIN AT
LEAST FOR THE MOUSE SIMULATIONS
IN THE NEXT TEN YEARS OR SO. TO
CONCLUDE I WANT TO ECHO WHY IT'S
IMPORTANT.
TO GIVE A COUPLE EXAMPLES, WE
CAN PRODUCE HUGE AMOUNTS OF
ENERGY BUT STILL CANNOT REALLY
PUT TOGETHER MITOCHONDRIA ON
DESIGN. WE CAN MOVE WITH CARS BUT
IF SOMETHING GOES WRONG IT'S NOT
STRAIGHTFORWARD TO FIX. A.I. IS
AMAZING GROWING FAST AND PROBABLY
WILL DEVELOP SUBSTANTIALLY WITHOUT
TOO MUCH INPUT FROM THE
NEUROSCIENCE BUT WE CAN
LEVERAGE IT ALSO AND LEARN A
LOT OF FROM THE BRAIN. BUT
WE CAN LEVERAGE THE DEVELOPMENT
OF A.I. TO CREATE DETAILED
SIMULATIONS OF THE BRAINS TO
LEARN ABOUT THE MECHANISMS OF THE
BRAIN THAT GIVE RISE TO THE
REPRESENTATIONS THAT ANDREAS
MENTIONED AND STUDY THAT
COMPREHENSIVELY IN HEALTHY
CONDITIONS AS WELL AS IN DISEASE
AND TRY TO FIX WHAT'S GOING ON
BASED ON SIMULATIONS.
SO WITH THAT, THANK YOU FOR
YOUR ATTENTION.
>> AS A BIOLOGIST AND
COMPUTATIONAL SCIENTIST --
[AUDIO DIGITIZING]
>> WE HAVE A VIRTUAL
PRESENTATION AND THAT IS WHAT IS
BEING SET UP NOW.
WELCOME BING, VIRTUALLY.
THANK YOU.
AS A BIOLOGIST AND COMPUTATIONAL
SCIENTIST THE NERVOUS SYSTEMS
GIVES US THE AUTONOMY TO DECIDE
WHERE TO GO AND THE AGENCY.
YET NEUROSCIENCE HAS BEEN
CONSIDERING THE BRAIN IN
ISOLATION FROM THE BIOMECHANICS
OF THE BODY THOUGH WE KNOW THE
BRAIN IS NOT DIRECTLY CONNECTED
TO THE WORLD.
IT RECEIVES NO SENSORY INPUT AND
HAS NO ACTION OUTPUTS EXCEPT
THROUGH THE BODY.
SO MY RESEARCH PROGRAM IS
FOCUSSED ON EMBODIED
INTELLIGENCE INTEGRATING MODELS
OF THE BRAIN AND NERVOUS SYSTEM
WITH THE BODY TO UNDERSTAND THE
NEURAL BASIS OF NATURAL
BEHAVIOR.
IT MAY SEEM THE MOST MYSTERIOUS
THING ABOUT THE BRAIN IS
COGNITION AND INTELLIGENCE.
WHERE MOST TAKE FOR GRANTED IT'S
RELATIVELY EASY TO WALK UP AND
DOWN THE STAIRS AND DOING THAT
CARRYING A LAPTOP AND A CUP OF
COFFEE, SOMETHING I DO ALL THE
TIME.
BUT I CONTEND THE COMPLEXITY OF
CONTROLLING MY BODY EXCEEDS THE
COMPLEXITY TO SOLVING CALCULUS
AND WE NEUROSCIENTISTS STUDY A
SYSTEM THAT EVOLVED OVER 500 MILLION
YEARS AGO. SO ANIMALS CAN SENSE
AND DECIDE WHERE TO GO. TO UNDER-
STAND ARCHITECTURE, FUNCTION, AND
DYSFUNCTION OF THE NERVOUS SYSTEM
THU NEURAL COMPUTATION. ONE MUST
EMBRACE THAT THE BRAIN AND BODY
HAVE BEEN INEXTRICABLE IN SOLVING
THE CHALLENGES FACED BY THE ANIMAL
FROM MOMENT TO MOMENT AND
PLANNING FOR THE FUTURE.
TO UNDERLINE THE POINT WE SEE A
HUNGRY EAGLE HUNTING INTEGRATING
VISUAL AND SOMATOSENSORY INPUTS
PREDICTING THE FUTURE POSITION OF
THE FISH, COORDINATED MOVEMENT TO
EXECUTE THIS BANKING MANEUVER AND
COLLIDE HIS CLAWS WITH THE FISH.
ALL THIS IS HAPPENING WITH
SENSORS AND ACTUATORS EMBEDDED
IN THIS BODY PLAN.
TO TACKLE THIS INTEGRATION WE'VE
BUILT BODY MODELS OF THE BRAIN
WHERE A NERVOUS SYSTEM INTERFACES
WITH MODELS OF THE BODY AND
KINEMATICS, DYNAMICS, AND BIO-
MECHANICS. OUR MODELS ARE AGENT-
BASED AND CAN PREDICT RESPONSES TO
MANIPULATIONS NEVER BEFORE SEEN.
THE LAYERS OF THE BRAIN, BODY
AND WORLD AND MANY INTERACTIONS
ARE THE MOST ESSENTIAL LAYERS TO
MAKE SURE WE'RE NOT COMPROMISING
ON OUR CORE PRINCIPLES.
FIRST, FEEDBACK IS ESSENTIAL AND
UBIQUITOUS.
WE'RE TALKING ABOUT MORE THAN ME
PUSHING ON THE TABLE AND TABLE
PUSHES BACK.
THE FEEDBACK LOOP GOES BACK TO
CONNECTION BETWEEN BRAIN REGIONS
AND SYNAPTIC PLASTICITY. SECOND,
BODY MATTERS. WE SEE A TROUT THAT'S
SWIMMING UP STREAM AND WILL GO
UP TOWARDS THE ROCK.
IT LOOKS LIKE BEAUTIFUL BEHAVIOR
BUT THAT TROUT IS ALREADY DEAD.
THERE'S NO NEURONS AND NO
CONTROL.
JUST THE BODY INTERACTING WITH
THE WATER.
WE AS NEUROSCIENTISTS CANNOT
IGNORE THIS. THIRD, LOTS OF PARTS
CAN BE BROKEN DOWN INTO MODULES
THAT INTERFACE THRU INFORMATIONAL
BOTTLENECKS. EXAMPLE INCLUDE
RETINAL GANGLION CELLS ARE ONLY
CELL TYPE THAT CONNECT TO VISUAL
SYSTEM. ENTIRE MOTOR SYSTEM CONVERGES ON A FEW MOTOR NEURONS.
I EMPHASIZE MODULARITY BECAUSE IT
POINTS TO A TRACTABLE SOLUTION OF
MODELLING EMBODIED INTELLIGENCE.
AND RIGHT KNOW IN 2024 IS A
SPECIAL TIME IN COMPUTATIONAL
NEUROSCIENCE BECAUSE I SEE A
UNIQUE CONVERGENCE OF IDEAS,
DATA AND TECHNOLOGY AND CULTURE
ALL COMING TOGETHER TO MOTIVATE
THE TIMELINESS OF MY APPROACH
WHILE UNDERSTANDING NEURAL
COMPUTATION THROUGH BUILDING
EMBODIED SYSTEM OF SYSTEMS MODEL
CAPABLE OF MIMICKING THE NATURAL
BEHAVIOR OF ANIMALS IN A VIRTUAL
FACSIMILE OF REALITY. AND LOOKING AT
EXTENSIONS ALL THESE ARE READY
TO BE INTEGRATED IN A SIMULATION
FRAMEWORK GROUNDED IN REAL DATA
WITH MODULES THAT WILL LEARN FROM
DATA SUCH AS THE MANY SENSORY
CAPABILITY OF ANIMALS, MEMORY,
AROUSAL STATE, MOTOR PLANNING,
EXECUTION AND MUSCLES AND
SKELETONS. TO MAKE AN ANALOGY,
THE MODELS ARE FOR ANIMALS
CALLED DIGITAL TWIN MODELS IN
ENGINEERING.
IT'S A SYSTEM OF SYSTEMS
STIMULATION WITH MANY PARTS.
NOW VIRTUAL COMPUTATIONAL
SIMULATION IS CRITICAL.
BY LOOKING AT A WIRING DIAGRAM
OF AN AIRPLANE ONLY MAKES US
MORE CONFUSED HOW A PLANE WORKS.
TO PREDICT ANYTHING WE MUST KNOW
ABOUT THE BODY TOO.
WHICH WIRES GOES TO THE FUEL
PUMPS AND WHICH GOES TO THE TAIL
FLAPS?
>> IT'S THE FEEDBACK
INTERACTIONS THAT ACCOMPLISH FIGHT.
IN NEUROSCIENCE, WE ARE WEALTHY
IN DATA.
YET THIS WEALTH THREATENS TO
DRIVE US FURTHER APART BECAUSE
OUR FAVORITE TOOLS AND PREFERRED
ANALYSES ARE NOT READILY
TRANSFERABLE BETWEEN THE DATA
SETS.
FOR THIS REASON I SEE A PRESSING
NEED FOR EMBODIED MODELS TO BE
DEVELOPED AS COMPUTATIONAL FRAME-
WORKS AND INTEGRATING ALL THIS
DATA TO MAKE QUANTITATIVE
PREDICTIONS AND GUIDE THE DESIGN
OF INCREASINGLY EXPENSIVE AND
COMBINATORIALLY COMPLEX
EXPERIMENTS. WHEN PREDICTIONS
DEVIATE FROM EXPERIMENTS, WE WILL
REFINE OUR MODELS. WE NEVER
COMPROMISE OUR CORE PRINCIPLES.
THE BRAIN AND BODY CANNOT BE
UNDERSTAND IN ISOLATION FROM
EACH OTHER AND THE GOAL OF THE
OVERALL AGENT IS ALWAYS THE
COORDINATED MOVEMENT OF ITS BODY
TO TACKLE THE CHALLENGES OF
SURVIVAL.
FINALLY, THIS IS THE ONLY WAY
WE'LL BE ABLE TO GAIN HOLISTIC
UNDERSTANDING OF THE EMBODIED
INTELLIGENCE OF BRAIN AND BEHAVIOR
MODELS ARE GENERALIZABLE BECAUSE
THEY'RE MODULAR AND THE MODULE
REFLECTS THE PHYSICS OF THE BODY.
I'M EXCITED ABOUT WHERE WE'RE
GOING WITH THIS BECAUSE THIS IS
MY PROPOSED MODELLING APPROACH
TO TACKLE MORE INTERESTING
NATURALISTIC BEHAVIORS AND
COMPLEXITY OF NATURAL BEHAVIORS
OF ANIMALS INCLUDING HUMANS AS
WE DO IN THE REAL WORLD.
WE'LL DO THIS BY EXTENDING THE
MODEL WITH MORE MODULES,
IMPROVING THE FIDELITY OF THE
MODULES AND WORK WITH
COLLABORATORS TO INTEGRATE ON
MAKING TESTABLE PREDICTIONS AND
REFINE THE UNDERSTANDING OF HOW
THE COMPLEX SYSTEMS MAY BE
WORKING TOGETHER.
NOW, FOR REAL WELL STUDIED MODEL
ORGANISMS LIKE FLIES AND
RODENTS, THE NEXT COUPLE YEARS
WILL BE EXCITING BECAUSE OF ALL
THE COMPREHENSIVE DATA SETS
PUBLISHED OR WILL BE PUBLISHED
SOON.
AND FOR EACH MODEL ORGANISM THE
WORK WE'RE DOING LEANS ON THE
STRENGTH OF THE AVAILABLE DATA
SETS. AND BIOPHYSICAL KNOWLEDGE
IS AVAILABLE: THE FLY CONNECTOME
OF THE BRAIN AND VENTRAL NERVOUS
SYSTEM AND WE'LL INCORPORATE
THAT INTO OUR DYNAMIC MODELS.
WHERE NEEDED, WE'LL APPROXIMATE
WITH MACHINE LEARNING TO MATCH THE
INPUTS AND OUTPUTS OF THOSE
MODULES. FOR THE RODENT AND HUMAN
WE'LL MODEL AT THE MESO SCALE BUT
GROUND MODELS IN THE BEHAVIOR OF
ANIMALS TRACKED BY COMPUTER VISION
FURTHERMORE I ENVISION THE
FRAMEWORK AS A WAY TO ASK
QUESTIONS TO CONNECT SHORT TERM
BIOPHYSICAL MECHANISMS WITH
LARGER AND LONGER TERM SCALE
IDEAS.
FOR EXAMPLE, THE NERVOUS SYSTEM
MODIFIES ITSELF THROUGH PLASTICITY
AND LIFELONG LEARNING.
THE BODY CHANGES DURING
DEVELOPMENT AND AGING AS WELL AS
ACTIVITY DEPENDENT CHANGES WITH
EXERCISE OR THE CHANGING
ENVIRONMENT, SOCIETY AND
CLIMATE.
TO SUMMARIZE, THIS ENTIRE
RESEARCH PLAN IS BUILT ON THE
IDEA BRAINS AND BODIES CANNOT BE
UNDERSTOOD IN ISOLATION AND
HOLISTIC MODELS MUST SERVE THE
GOALS OF THE ANIMAL.
I FEEL INCREDIBLY FORTUNATE TO
BE A MEMBER OF THIS COMMUNITY
RIGHT NOW AND OPTIMISTIC ABOUT THE
SIGNIFICANCE OF WHAT WE'LL BE
BUILDING TOGETHER.
I THINK THE POTENTIAL IMPACT OF
MY PROPOSED WORK AND AS I'VE
DESCRIBED TODAY IS NOTHING SHORT
OF CHANGING THE WAY
NEUROSCIENCE IS DONE IN THE NEXT
YEARS TO DECADES.
I WOULD LIKE TO GIVE MY SINCERE
THANKS TO EVERYONE IN MY LAB,
PAST AND PRESENT
AND ALL OF OUR
COLLABORATORS.
I'M GRATEFUL TO YOU AND THANK
YOU FOR TRUSTING ME ENOUGH TO
WORK WITH ME.
>> I'M DOMINIQUE DUNCAN AND
WANTED TO TALK ABOUT NEUROAI
FROM A DATA SHARING PERSPECTIVE
AND HOW IMPORTANT IT IS TO SHARE
DATA AND THE CONSIDERATIONS
AROUND THAT AND CHALLENGES AND
OPPORTUNITIES.
THE BRAIN INITIATIVE FUNDED DATA
ARCHIVES AND SOME ARE OPENNEURO,
NEMAR, NEMO, BRAIN IMAGE LIBRARY,
DANDI AND DABI. I'VE BEEN WORKING
ON DABI. I'LL GO OVER THAT
INFRASTRUCTURE IN A LITTLE BIT.
MORE INFORMATION ABOUT THE DATA
STORED IN THE DIFFERENT DATA
ARCHIVES.
THIS IS A LIST OF THOSE BUT SOME
DATA ARCHIVES HAVE ALSO EXPANDED
INTO OTHER DATA TYPES.
SO THERE'S MANY CHALLENGES IN
LARGE SCALE DATA REPOSITORIES.
THESE ARE SOME OF THEM.
SO DATA ARE HETEROGENEOUS AND
CAN BE DIFFICULT TO COMPARE
ACROSS SITES.
WE'RE DEALING WITH LARGE FILE
SIZES.
SOME OF THE DATA ARE HIGHLY
SPECIFIC.
PEOPLE USE DIFFERENT FORMATS.
THERE CAN BE PRIVACY CONSTRAINTS
AND NEED TO SECURE RESOURCE TO
SUSTAIN THE ARCHIVE EFFORT
BEYOND THE INITIAL FUNDING
PERIOD IN CASE FUNDING ENDS OR
THE PROJECT IS NOT RENEWED.
SO HERE ON THE LEFT I WANTED TO
BRING UP THE APPROACH WE TAKE
FOR DABI IN TERMS OF DATA
STORAGE MODELS.
WE'RE AGNOSTIC TO THE WAY THE
DATA ARE STORED.
SO WE GIVE FLEXIBILITY TO OUR
DATA PROVIDERS.
AND ONE OPTION IS CENTRALIZED
DATA STORAGE MODEL.
IN THAT CASE THE DATA ARE STORED
AT OUR SITE AT USC. THE OTHER DATA
STORAGE MODEL IS FEDERATED.
THAT MEANS WE HAVE THE METADATA
AND RAW DATA AT THE SITE WHERE
THEY'RE COLLECTED OR CLOUD
BASED.
CURRENTLY WE'RE MOVING OUR DATA
TO AWS. THE NEXT FEW SLIDES
I WANTED TO TALK ABOUT THE
INFRASTRUCTURE. DABI IS FOR
INVASIVE HUMAN NEUROPHYSIOLOGY.
WE GIVE FLEXIBILITY TO OUR DATA
PROVIDERS IN TERMS OF STANDARDS
USED.
WE ENCOURAGE DATA PROVIDERS TO
USE EITHER BIDS OR NEURODATA
WITHOUT BORDERS OR HAVE OUR OWN
DABI FRAMEWORK THE DATA
PROVIDERS CAN USE. WE WORK CLOSELY
WITH THE NEUROSURGEON DATA
PROVIDERS TO FIGURE OUT WHAT
THEIR NEEDS ARE, WHAT WOULD MAKE
THEIR LIVES EASIER IN TERMS OF
SHARING DATA. FOR ANALYSIS WE CAN
HELP THEM DEVELOP ANALYTIC TOOLS
INCLUDING VISUALIZATION.
WE HAVE MOSTLY EEG. WE HAVE ECOG
AND DBS AND OTHER DATA TYPES,
CLINICAL, IMAGING, PATHOLOGY,
DEMOGRAPHIC AND BEHAVIORIAL.
USE JSON TO HARMONIZE METADATA
JSON WAS CHOSEN BY ITS USE OF BIDS
WE'VE WORKED CLOSELY WITH THE DATA
PROVIDERS TO FIGURE OUT WHAT
WOULD MAKE THEIR LIVES EASY. IF
THERE'S REQUESTS FOR ACCESS AND
ADJUSTMENTS TO SHARE THE DATA.
IF YOU GET ON DABI ANYONE CAN
CREATE A FREE ACCOUNT.
IF YOU WANT TO BUILD A COHORT
YOU CAN QUERY THE DATA WE HAVE
ON DABI AND LOOK AT CRITERIA YOU
MIGHT BE INTERESTED IN.
SELECT THE CRITERIA IF THEY'RE
DATA MODALITIES, BRAIN REGIONS,
DIAGNOSIS, ETCETERA.
AND YOU CAN LOOK AT THE IMAGING
AND THE SIGNAL AND DECIDE IF YOU
WANT TO DISCARD ANY OF THE
SUBJECTS FROM YOUR COHORT.
SO ONCE YOU DO THAT, THEN YOU
CAN SAVE YOUR COHORT AND MOVE ON
TO THE NEXT STEP OF DOING SOME
ANALYSIS.
AND WE HAVE A LIBRARY OF TOOLS
AND PROCESSING TOOLS AND YOU
CAN SELECT THE ONES YOU WANT TO
USE FOR YOUR COHORT AND PULL OUT
THE MODULES AND CONNECT THEM AND
RUN YOUR COHORT THROUGH THAT AND
AT EACH STEP CAN DO A QUICK
VISUAL INSPECTION.
SAY YOU DO FILTERING AND DO A
QUICK INSPECTION TO SEE HOW THAT
CHANGED THE SIGNAL.
ONE OF THE TOOLS WAS IEEG SYNC.
IT'S LIKE GOOGLE DOCS FOR IEEG
WHERE YOU CAN COLLABORATIVELY
MARK UP DATA AND MARK UP ANY
EVENTS OR RANGES OF INTEREST AND
PERFORM FURTHER ANALYSIS.
WE CAN INTEGRATE THE VARIOUS
DATA ARCHIVES AND ALLOW USERS TO
QUERY ACROSS THE ARCHIVES AND
LEVERAGE THESE RICH DATA SETS
AND BE ABLE TO PERFORM A.I.
METHODS AND ANALYSIS ON THE
DIFFERENT DATA SETS.
SO BRAIN INITIATIVE AS YOU HEARD
FROM MANY PEOPLE ENCOURAGED
MULTIDISCIPLINARY TEAMS AND MY
BACKGROUND IS ELECTRICAL
ENGINEERING AND MATH. I WORK WITH
NEUROSURGEONS, CLINICIAN, PEOPLE
WITH VARIOUS TYPES OF EXPERTISE.
WE NEED STANDARDIZED CROSS MODAL DATA INTEGRATION,
COMMON THEORETICAL FRAMEWORKS,
METRICS TO COMPARE BIOLOGICAL AND
ARTIFICIAL NEURAL NETWORKS AND
IF NEW ARCHIVES ARE NEEDED TO
INCORPORATE ADDITIONAL DATA
TYPES. WE ALSO LINK TO BRAIN TOOLS
DABI HAS INTEGRATED RAVE AND A
TOOL FOR VISUALIZATION OF IEEG
I WANTED TO END WITH THE
NEWEST BRAIN DATA ARCHIVE FOR
THE BBQS PROGRAM. IT LEVERAGES
EXISTING DATA ARCHIVES SUCH AS
DABI, DANDI, AND STANDARDS LIKE
BIDS AND NWB. IT'LL USE A
CLOUD-BASED ENVIRONMENT SO
PEOPLE CAN PERFORM A.I. ANALYSIS.
SO THANK YOU.
AND THESE ARE SOME OF MY LAB
MEMBERS AND WE'RE GRATEFUL FOR
THE FUNDING FROM NIH AND NSF.
I'LL END THERE.
>> THANK YOU TO ALL OF OUR
SPEAKERS.
TONY'S UP.
AT THIS POINT I WANTED TO INVITE
ALL OUR INVITED DISCUSSANTS TO
THE STAGE. FRANCES CHANCE, SUEYEON
CHUNG AND PAUL MIDDLEBROOKS IF
YOU CAN JOIN US ON THE STAGE.
WE SHOULD HAVE THE GUIDING
QUESTION SLIDE DISPLAYED.
YOU CAN SEE THE GOAL FOR THE
SESSION WITH THE TALKS WE'VE
HEARD IS TO HELP ESTABLISH A
VISION FOR NEUROAI, A CONVERGENT
FRAMEWORK TO DRIVE RECIPROCAL
ADVANCES THRU SHARED PRINCIPLES
AND WE NEED TO IDENTIFY CONCRETE
OPPORTUNITIES FOR BRAIN TO BRIDGE
BIOLOGICAL AND ARTIFICIAL
SYSTEMS AND WE HEARD GREAT
PERSPECTIVES ABOUT THAT AND THE
AUDIENCE MEMBERS CAN SEE THE
QUESTIONS WE'VE PUT TOGETHER
BEFORE THE WORKSHOP TO GUIDE
YOUR THINKING BUT WE WANT TO
START OUR PANEL DISCUSSION WITH
FRANCES.
>>
>> ONE THING I APPRECIATED ABOUT
THE SESSION IS WE'RE SEEING AN
APPRECIATION OF THE BIODIVERSITY
THAT EXISTS IN NEURAL SYSTEMS
FROM THE PEOPLE COMING FROM MORE
THE A.I. CAMP. AND NEUROSCIENCE
CAMP GETTING DIVERSITY IN DATASETS
INTO A.I. MODELS AND FACILITATING
THAT.
THE QUESTION I'D LIKE TO ASK IS
HOW DO YOU CHOOSE YOUR
CONSTRAINTS?
ON ONE HAND IF YOU'RE JUST --
THAT'S HOW I'LL LEAVE IT.
MAYBE IF SOMEONE FROM THE A.I.
SIDE, ALI, WOLFGANG FOR PATRICK,
HOW DO YOU CHOOSE WHAT YOU TRY
TO REPLICATE FROM A BRAIN?
OTHERWISE YOU'RE JUST TRYING TO
RECREATE A BRAIN IN A SIMULATION
AND IF THERE'S TIME WE CAN MOVE
TO ANDREAS, ANTONE, DOMINIQUE ON
HOW YOU'LL FACILITATE A.I. LOOKING
TO THE LARGE DATA SETS FOR
SPECIFIC ASPECTS OF HOW
BIOLOGICAL NEURONS ARE DOING
PROCESSING.
>> I RECENTLY UNEARTH THE
QUOTE IT'S A SHORT STORY ABOUT
A KINGDOM MAP THE SAME SIZE AND
RESOLUTION AS THE KINGDOM ITSELF
AND IT TALKS ABOUT HOW IN THE
DESERT THEY HAVE A MAP THAT'S
NOW CRUMBLING BECAUSE IT'S ISO
MORPHIC AND I THINK THAT'S MADE
SOME REALLY INTERESTING POINTS
ABOUT THE FACT THAT WE NOW HAVE
3457Z FROM THE REAL WORLD AT THE
SAME RESOLUTION AS THE REAL
THING AND IT'S HELPFUL.
SO IS THERE SCALE SEPARATION IN
THE BRAIN?
WE CAN SAY ANYTHING BELOW THE
SCALE DOESN'T MATTER OR THERE'S
A CLEAR DIFFERENCE OF THE KINDS
OF STUFF THAT YOU'RE FOCUSING
ON.
THERE'S A CLEAR SEPARATION OF
SCALE IN THE SENSE THAT NEURONS
HAVE CELL WALLS AND THERE'S A
CLEAR INSIDE AND OUTSIDE AND IT
SEEMS LIKE A NATURAL SCALE TO
FOCUS ON AT THE LEVEL OF NEURONS
THEIR CONNECTIONS AND ACTIVITY.
THE MESO AND MICROSCOPIC SCALES
ARE SUBSERVIENT TO THIS LEVEL.
NATURAL SCALE LEVEL IS NEURONS.
>> MOVING ON TO WOLFGANG AND
ALI.
YOU WERE ADVOCATING FOR BRINGING
EVOLUTION TO YOUR MODELS.
>> RIGHT.
NOT TO EXPLICITLY PUT EVOLUTION
IN THE MODEL BUT TO MINE WHAT
EVOLUTION HAS DONE TO INFORM OUR
MODELS.
NUMBER ONE I DON'T THINK THE
GOAL OF A.I. SHOULD BE TO
REPLICATE ANY BIOLOGICAL SYSTEM
AND I THINK A FLY IS JUST AS
GENERALLY INTELLIGENT IN IT'S
OWN ENVIRONMENT AND NICHE AS
A HUMAN AS IN THEIRS.
I THINK WHEN WE BUILD THE A.I.
SYSTEMS WE SHOULD BE THINKING
ABOUT BUILDING NEW SPECIES OF
WHICH HAVE INTELLIGENCES
EQUIVALENT TO OR THAT WOULD FIT
INTO THE SAME SCALE OF
BEHAVIORAL COMPLEXITY AND
COGNITIVE COMPLEXITY WE SEE IN
LIVING SYSTEMS.
AND AS WE SUCCEED IN DOING THAT
WE'RE CREATING ARTIFICIAL
INTELLIGENCES.
THAT'S THE WAY I WOULD LOOK AT
THAT.
>> AND I'M HAPPY TO AGREE THAT
FLIES HAVE BEHAVIORS WE WANT TO
BRING INTO OUR A.I. MODELS BUT
HOW ARE WE GOING TO LOOK
AT THE DATA FROM NEUROSCIENCE TO
RECREATE THESE INVALUABLE
BEHAVIORS WE GET?
>> IT'S A GREAT QUESTION AND
ALREADY A.I. HAS PROVEN WHAT IT
CAN DO WITH STRUCTURAL DATA
ABOUT THE WIRING OF THE BRAIN.
WHAT I WOULD ARGUE IS WE FOCUS
ON NEURONS AND STRUCTURAL
CONNECTIONS. EXPLORE NON-NEURONAL
CELLS, NON-ELECTRICAL SIGNALS AND
DYNAMICS AS INFORMATIONAL CARRIER.
THERE'S ROOM FOR THE EVO-DEVO SIDE
AND BACK TO SIGNALS IN THE BRAIN
AND QUESTIONS ABOUT HOW THEY CAN BE
INTEGRATED IN AN A.I. MODEL
>> IF I CAN PULL ON THAT MORE
AND ONE MORE QUESTION, HOW DO
YOU KNOW WHEN YOU'RE DONE?
WHEN YOU'VE RECREATED THE RIGHT
DYNAMICS OR AS MANY NEURONAL
SUBTYPES OBSERVES IN THE DATASETS
OR WHEN YOU GET THE RIGHT
DYNAMICS IN THE CONNECTIONS?
>> I'M ASKING YOU BECAUSE YOUR
MIC IS ON.
>> I ALWAYS ASK STUDENTS TO DRAW
A PROTEIN AND EACH ONE DRAWS A
DIFFERENT MODEL OF A PROTEIN.
COMPLEXITY OF THE MODEL HAS TO
MATCH WHAT WE'RE TRYING TO ACHIEVE
THAT'S AN OVERLY SIMPLISTIC
ANSWER TO YOUR QUESTION.
>> I WOULD LIKE TO ASK A
QUESTION TO THE SPEAKERS AND
ANYONE AT THE TABLE FROM THE
PERSPECTIVE OF A THEORIST. I LIKE
THE ARROW OF NEUROSCIENCE TO A.I.
AND ALSO WANT THE VIRTUAL LOOP TO
HAVE BETTER THEORIES ABOUT
THE BRAIN AND HOW IT COMPUTES.
HOW SHOULD WE EXPAND OUR
UNDERSTANDING OF NEURAL
COMPUTATION AND COMPUTING IN THE
BRAIN TO INCORPORATE MANY
DIFFERENT TYPES OF BIOLOGICAL
SYSTEMS LIKE GLIA, NON-NEURON
COMPUTATIONS OR NEUROMODULATION,
EVO-DEVO, TO INFORM THEORETICAL
ADVANCES AND PRACTICAL NEUROAI
IMPLICATIONS
WHAT QUESTIONS SHOULD WE HAVE IN
THINKING ABOUT THIS TYPE OF
NEUROAI AND IF YOU'RE A THEORIST
IF YOU HAVE THOUGHTS ON THAT.
TOWARDS THE END OF A TALK AND
ANDREAS WAS TALKING ABOUT
REPRESENTATIONS SO MAYBE I
COULD POINT THE QUESTION TO ANDREAS
IF YOU LOOK AT THE NEXT FIVE TO
TEN YEARS AT THE LEVEL OF NEURAL
REPRESENTATIONS AND NEURAL
DYNAMICS, WE HAVE A DIRECT WAY
TO MAKE LINKS TO A.I. IN THEORY
I THINK IT'S ONE OF THE MOST
FRUITFUL DIRECTION AND AT THAT
LEVEL WE CAN ANALYZE NEURAL
DYNAMICS, MANIFOLDS, BRINGING
THEORY AND WE CAN DO THE SAME
WITH ARTIFICIAL NEURAL NETWORKS.
WE CAN DO THIS LEVEL OF ANALYSIS
WOULD LIKE TO SEE THEORY NOT JUST
WITH STATISTICAL TOOLS IN ANALYZING THE GEOMETRY OF NEURAL MANIFOLDS
BUT TRY TO RELATE THEORETICAL WAYS
TO UNDERSTAND DIGITAL TWINS IN THE NEURAL MANIFOLDS IN RELATIONSHIP TO WORLD MODELS.
OUR BRAINS HAVE CAUSAL UNDERSTANDING
THE WORLD MODEL IS INCORPORATED IN
NEURAL DYNAMICS.
IF WE UNDERSTAND THESE PRINCIPLES
THEN WE'LL BE ABLE TO HAVE
IMPACT IN A.I.
AT LEAST THAT'S ONE AREA I
THINK WE SHOULD PUT EFFORT IN
BECAUSE IT'S AN AREA WE SEE EVIDENCE
THAT SOMETHING IS POSSIBLE. I WAS ALSO WONDERING PATRICK IF YOU HAVE
THOUGHTS ON ALIGNMENT OR INTERPRETABILITY?
THERE'S A REVOLUTION IN MECHANISTIC
INTERPRETABILITY. IF YOU'VE BEEN
FOLLOWING THE LITERATURE THE LAST
FIVE YEARS, THE LAST YEAR LOOKS VERY
DIFFERENT FROM THE FIRST FOUR.
MECHANISTIC INTERPRETABILITY IS
LIKE NEUROSCIENCE FOR A.I.
PEOPLE ARE DOING LOOKING AT
REPRESENTATION AND ABLATIONS OF
SINGLE NEURONS IN A.I. MODELS AND
MAPPING OUT RECEPTIVE FIELDS
MECHANISTIC INTERPRETABILITY HAS
MOVED FROM SINGLE NEURON VIEWS
TO POPULATION-BASED VIEWS
WHETHER IT'S WORK ON SPARSE AUTO-
ENCODERS TO EXPLAIN LLM TO DOING
POPULATION ABLATIONS AND
ACTIVATION VECTORS IN ANOTHER
SUBPART OF THAT FIELD.
IF THERE'S ANY ARROW OF
INFLUENCE IN THAT FIELD, TAKING
THOSE TECHNIQUES BACK INTO
NEUROSCIENCE WOULD SEEM VERY
FRUITFUL.
THE KIND OF THINGS THEY CAN
DO IS AMAZING.
YOU CAN TAKE THE REPRESENTATIONS
FROM ONE NETWORK LOOKING AT A
PARTICULAR STIMULATION AND THEN
GRAFT IT ONTO ANOTHER ONE AND
SWAP OUT LAYERS AND DO ABLATIONS
ZAP SOME NEURONS. IF WE CAN DO
IN NEUROSCIENCE, WE'D BE GOLDEN.
IS THERE A MINIMUM SET OF TOOLS
STRUCTURED OPTOGENETICS, STIMULA-
TION OR THE ABILITY TO GROW NET-
WORKS AND BRING INTO NEUROSCIENCE?
EVEN IN A.I., PHYSICAL INTELLIGENCE
AND EMBODIED A.I. ARE THE HARDER
QUESTIONS COMPARED TO VISUAL ANN.
WHAT ARE SOME TYPES OF DATASETS
THAT MAY BE NEEDED FOR OR TYPES
OF THEORIES OR CONCEPTUAL
FRAMEWORKS NEEDED IN THIS DOMAIN
IF ANYBODY HAS ANYTHING TO ADD.
EMBRACE HIGH-ENTROPY DATA.
NEED ANIMALS TO INTERACT IN
NATURALISTIC ENVIRONMENT.
NEUROSCIENTISTS ARE TRAINED TO
COLLECT HYPOTHESIS-DRIVEN DATA.
WE NEED TO GET AWAY FROM THAT AND
DO NATURALISTIC DATASET COLLECTION
>> GREAT TO SEE THOSE TYPES OF DATASETS OPEN TO PUBLIC TO
DEVELOP MODELS AND THEORIES TO TEST IDEAS, ANYONE?
>> MAY I SAY SOMETHING ABOUT
ALIGNMENT?
THE WAY WE ARE THINKING ABOUT
ALIGNMENT IS WE TRAIN THESE
LARGE MODELS WHICH ARE UNALIGNED.
AND THEN WE ALIGN THEM USING
RLHF OR
SOMETHING LIKE THAT WHICH IS
VERY UNNATURAL.
THE REASON YOU AND I ARE ALIGNED
IS BECAUSE WE WENT THROUGH A
LONG DEVELOPMENTAL PROCESS AND
LEARNED IN THE EARLY STAGE IN
OUR LIFE HOW TO BEHAVE AND WHAT
ARE GOOD THINGS TO THINK ABOUT
AND BAD WAYS.
THE ALIGNMENT PROFESSION HAS TO
BE INTEGRATED WITH THE LEARNING
PROCESS.
THE SYSTEMS AS THEY LEARN ARE
INHERENTLY ASSIGNED. MAYBE NOT
PERFECTLY. SOME OF US NEED TUNING
BUT IT'S NOT AN AFTERTHOUGHT.
WE DON'T FIRST TRAIN BY WOLVES AND
THEN TRY TO CIVILIZE IT. THIS IS
A.I. PEOPLE SHOULD THINK ABOUT.
>> WHAT WOULD BE GOOD METRICS TO
MEASURE SUCH ALIGNMENT?
>> THAT'S THE MAIN TOPIC OF
SESSION 2 AND NEED TO STAY ON
TIME WE'LL NEED TO MOVE TO PAUL
AS AN INVITED DISCUSSANT BUT
WE'LL HAVE MORE ON METRICS FOR
SURE.
>> A.I. IS SUPERFAST.
SCIENCE IS SUPER SLOW AND
DEPENDENT ON LUCK. GIVEN THE
DELUGE OF DATA AND PIPELINES,
HOW DO WE MAKE TRAINEES MORE
EFFICIENT?
IF YOU HAD A PERFECT AVATAR OF
THE SKILLS NECESSARY TO
IMPLEMENT WHAT YOU'RE TALKING
ABOUT, WHAT WOULD THAT LOOK LIKE?
SO TRAINEES ARE PART OF THE
INFRASTRUCTURE WE NEED. WHAT DO
YOU NEED IN TERMS OF SKILLS,
ET CETERA?
I THINK THE MENTAL MODELS FOR
HOW WE'LL DO SCIENCE ARE IN THE
NEAR FUTURE MIGHT BE A LITTLE
BIT OFF.
THE WAY I DID SCIENCE IN GRAD
SCHOOL AND THE WAY MY
FOREFATHERS DID SCIENCE IS
SIMILAR BUT MIGHT CHANGE A
LITTLE BIT.
WE'RE SEEING FUNDAMENTAL ADVANCES
IN A.I. AND WE'RE STARTING TO SEE
USING A.I. FOR SCIENCE IN RADICAL
WAYS. TRAINEES WILL HAVE ACCESS TO
MORE KNOWLEDGEABLE, POWERFUL TOOLS
THE WAY WE TRAIN TRAINEES IT'S
HARD TO BUILD A ROAD GIVEN WE
DON'T KNOW WHERE IT'S GOING TO
END UP THE ADVANCE OF A.I. BUT WE
SHOULD TRY TO INCORPORATE IT AND
THINK ABOUT IT AND ADAPT.
>> WE'RE STILL BETTER AT
EXPLORING THAN A.I. AND ONE OF
THE THINGS THERE'S ALSO THE NEED
TO COLLABORATE ACROSS DIFFERENT
LEVELS OF EXPERTISE.
AND HAVE TO FOLLOW GUIDANCE AND
EXPLORE THINGS IN A RESEARCH
LAB.
WE CAN'T GO TO LIKE A
MATHEMATICIANS MODEL TO EXPLORE
WHATEVER YOU WANT FOR YOUR Ph.D.
BUT HAS TO BE MORE ABILITY TO BE
CURIOUS AT THE TRAINING LEVEL.
>> CURIOSITY WOULD DRIVE YOU TO
A MILLION THINGS SO ONE QUESTION
IS HOW TO FOCUS THE EXPLORATION
I SUPPOSE.
>> HOW TO FOCUS THE EXPLORATION
IS THE ROLE OF THE MENTOR TO
HELP DIRECT TRAINEES TO WHAT IS
MOST FRUITFUL, INTERESTING, AND
DOABLE. MORE GENERALLY WE NEED
TO GIVE PEOPLE THE TOOLS EARLY
TO FLOURISH AND BECOME GOOD
SCIENTISTS. WE ARE TALKING ABOUT
SOPHISTICATED SYSTEMS
WHETHER THAT'S A.I. SYSTEMS. SO
WHAT WE'RE DOING IN MY GROUP IS
BIOREALISTIC SIMULATIONS OF BRAIN
CIRCUITS.
IT'S SUCH A COMPLICATED THING TO
BE PRODUCTIVE. YOU HAVE TO LEARN
A LOT OF AND A LOT OF THE STEPS
IN GETTING THERE ARE TAKEN CARE
OF BY THE DEVELOPING TOOLS.
NEEDS TO TRAIN PEOPLE IN PHYSICS
AND USING THE TOOLS EARLY ON.
>> I WANT TO MAKE A COMMENT AND
>> A.I. HAS RELIED ON HUMANS
THOUSANDS OF YEARS PRODUCING
DATA AND IS THE TECHNOLOGY THAT'S
ACCELERATED THE PROGRESS.
NOW WE'RE IN A SPECIAL ERA IN
NEUROSCIENCE WHERE IF WE CREATE
LARGE CENTERS THAT WILL OUTPACE
THE AMOUNT OF DATA EVEN THE LARGE
LANGUAGE MODELS GET TRAINED ON.
IN THAT CASE IT'S POSSIBLE THAT
WE MAY BE IN A REGIME WE MAY BE
ABLE TO MOVE MAYBE EVEN FASTER
POSSIBLY THAN CURRENT A.I.
RESEARCH AND THE DATA BANDWIDTH
WILL BE HIGH LIKE RECORDING NEURAL
DATA VS BEHAVIORAL WHICH IS LOW
BIT, WHETHER YOUR WRITE TEXT OR
OR LABEL YOUR DATA. POSSIBLE FUTURE.
>> FOR ANTOINE AND ANDREAS WHAT
IS THE BALANCE FOR MORE DATA AND
ANALYZING DATA WE ALREADY HAVE FOR
A.I. SO WE STAY ON TARGET?
>> 70% FOR THE NEW DATA, 30% FOR
ANALYZING THE EXISTING ONES.
THERE'S A LOT OF TO BE DONE WITH
EXISTING ONES.
I'M VERY EXCITED WITH WHAT'S
COMING AND I AGREE WITH ANDREAS
THE BANDWIDTH IS GOING TO BE
TREMENDOUS.
>> CAN WE ACCELERATE
NEUROSCIENCE CONTRIBUTIONS BY
LOOKING AT EXISTING DATA?
>> I'M MORE PESSIMISTIC THERE.
A LOT OF THE DATA WE'VE
COLLECTED SO FAR ARE NOT THE
RIGHT KIND OF DATA FOR A.I.
THEY'RE BASICALLY VERY
HYPOTHESIS DRIVEN DATA.
THEY COULDN'T GENERALIZE AND I
ACTUALLY BELIEVE THE LAST DECADE
IS THE NEURAL TECHNOLOGY THAT
WAS THE BIG THING AND NOT THE
DATA WE'VE COLLECTED SO FAR.
I THINK ALSO THE WAY WE COLLECT
THE DATA MAY NEED TO CHANGE.
I'M NOT SURE IF THE LAB BASED
ORGANIZATION IS THE RIGHT WAY
FORWARD. WE HAVE TO THINK LIKE
PHYSICISTS DO. I THINK WE NEED
NEW TYPES OF DATA ACTUALLY.
>> GREAT AND WITH THAT I
WOULD LIKE TO OPEN IT UP TO
AUDIENCE Q&A AT THIS POINT.
>> I SEE A FACE I RECOGNIZE.
>> KURT THOROUGHMAN FROM
WASHINGTON UNIVERSITY IN ST. LOUIS
I WANT TO ASK ABOUT EXPERIMENTAL
MANIPULATION. IN NEUROSCIENCE
THERE'S VOLTAGE CLAMPS AND
OPTOGENETIC MANIPULATION AND IT
SEEMS LIKE A.I. SHOULD BE A
SYSTEM WHERE YOU CAN USE THAT
TO CONTROL COMPLEX SYSTEMS IN
WAY THAT WILL PAY BENEFITS AS
NEUROSCIENCE. WHAT NEWFANGALED WAYS
DOES NEUROSCIENCE LEARN ABOUT
TO BE ABLE TO UNDERSTAND A.I. AND
HOW CAN AI THEN USE VOLTAGE CLAMPS
TO FIGURE OUT THEIR OWN SYSTEMS?
>> I CAN ANSWER THAT A LITTLE
BIT.
>> MOST OF A.I. MODELS WHICH ARE
CURRENT ARE NOT RECURRENT
MODELS.
I THINK THERE WAS A LOT OF
REALLY INTERESTING WORK FROM
PEOPLE ON TRYING TO UNDERSTAND
WHAT'S GOING ON INSIDE OF
RECURRENT NEURAL NETWORKS AND
GIVEN THE SHIFTED TOWARDS
TRANSFORMERS THEY DON'T HAVE
AUTO REGRESSION. IT'S DIFFICULT
TO APPLY THE SAME TOOLS. VOLTAGE
CLAMP WOULD BE EMBEDDED IN CONTROL
THEORY AND MAKE MORE SENSE FOR RNN
PEOPLE ARE LOOKING AT STATE-SPACE
MODELS, A NEWFANGLED VERSION OF
RNN, REACHING INTERESTING PRO-
PERTIES AND WORKING WELL AND MAY
BE TIME TO UNEARTH THOSE PRIOR
TOOLS AND APPLY THEM TO THE NEW
SYSTEMS.
>> I'LL PUSH BACK BECAUSE
TRANSFORMERS WE BUILT.
IT'S OUR FAULT THEY LOOK THIS
WAY. WHEN I TEACH HODGKIN AND
HUXLEY THERE ARE SPACE CLAMPS,
ADJUSTMENTS OF EXTRACELLULAR FLUID
TO BALANCE IONIC MIX. WE SHOULD BE
ABLE TO DO THIS FOR A.I. WE
BUILT THE DARN THING. HOW CAN WE
MOVE BACK TOWARDS THOSE SORTS
OF MANIPULATABILITIES?
>> LIKE HH MODEL FOR A.I.?
>> OR LOOK AT WHAT'S HAPPENING
UNDER THE HOOD AND PUT THE
CONNECTIONS IN THE A.I. TO MAKE
THAT MORE INTERPRETABLE THE WAY
THAT THEY BUILT VOLTAGE MAPS.
>> IT DEPENDS ON THE QUESTION.
AND PREDICTING NEURAL ACTIVITY
DYNAMICS BY ADDING MORE
PARAMETERS FROM A PRACTICAL
POINT OF VIEW IT'S NOT HELPING
BECAUSE THE NEURAL NETWORKS ARE
A FUNCTION OF APPROXIMATORS.
WE HAVE BETTER WAYS TO OPTIMIZE
THEM, AS PATRICK IS SAYING, BEING
GENERIC ARCHITECTURES.
THERE'S A MATCH BETWEEN THE
OPTIMIZATION METHOD IN MACHINE
LEARNING AND THE ARCHITECTURE.
BECAUSE OF HOW THINGS EVOLVED WE
HAVE METHODS WORKING BEST WITH
EXISTING ARCHITECTURE AND GIVES
THE BEST MODELS.
WHEN WE ADD HIGHER LEVEL
COMPLEXITIES THEY GET HARDER TO
FIT TO COMPLEX DATA AND MANY
NEURONS.
>> THANK YOU.
TERRY.
>> THIS HAS BEEN A WONDERFUL
DISCUSSION AND
LOTS OF ISSUES
BROUGHT UP.
WHEN THE WRIGHT BROTHERS WERE
DESIGNING THE PLANE POWERED WITH
MAN FLIGHT WHAT DID THEY LOOK AT
FROM BIRDS?
WHAT PRINCIPLES DID THEY
EXTRACT?
THEY EXTRACTED HOW BIRDS COULD
GLIDE LONG DISTANCES WITHOUT
MUCH ENERGY AND THEN LOOKED AT
TO THE WINGS AND THE
AERODYNAMICS OF WINGS.
THEY LOOKED AT THE FEATHER.
THE DIDN'T TAKE AWAY THE DETAILS
OF HOW NATURE BUILT THE
MATERIALS BUT THEY TOOK AWAY
SOMETHING STIFF THAT HAD A LOT
OF AREA AND SO THEY BUILT WINGS
THAT HAD WOOD SPARS AND CANVAS
LIGHTWEIGHT BETWEEN IT.
IT WAS CLEAR THE GOAL WAS TO
EXTRACT PRINCIPLES FROM ALL
THOSE DETAILS. HERE'S THE
QUESTION.
WHAT ADDITIONAL PRINCIPLES CAN
WE EXTRACT FROM ALL THE
FANTASTIC DISCOVERIES MADE IN
NEUROSCIENCES AND THE DATABASES
WE'RE BUILDING THAT COULD BE OF
USE IN A.I.? WE'VE HEARD A
FEW. I AGREE LARGE LANGUAGE MODELS
HAVE TO BE ALIGNED WHILE THEY'RE
BEING BUILT.
THERE'S ONLY TWO TIME SCALES IN
THEM. TRAINING AT THE BEGINNING
AND THEN THE INFERENCE.
THAT'S LEAVING OUT A TREMENDOUS
NUMBER OF TIME SCALES IN BETWEEN
WHAT ELSE CAN WE LEARN ABOUT
TIME SCALES FROM NEUROSCIENCE
GENERAL PRINCIPLES?
THEY EXTRACTED LIFT, NOT FLIGHT
TONY MAKES THIS EXPLICIT. PLANES
DON'T FLY THE WAY BIRDS FLY SO THE
TARGET IN THAT INSTANCE -- SO
THE PRINCIPLE IS LIFT NOT FLIGHT.
>> IT'S AIR FOILS.
THAT'S WHAT THEY EXTRACTED.
THEY LOOKED AT THE AIR FOIL OF
THE BIRD WING. IN FACT THE
REAL PROBLEM GETTING PLANES SAFE
WAS CONTROL AND IT WAS MENTIONED
IN WHAT NATURE GOT RIGHT.
HOW TO DO CONTROL SAFELY. WE
DON'T KNOW HOW TO DO THAT IN A.I.
WHAT OTHER PRINCIPLES CAN WE
EXTRACT FROM NATURE FROM ALL THE
FACTS AND WE HAVE HUGE
DATABASES.
WE WANT TO EXTRACT PRINCIPLES.
BY THE WAY NOT ONLY TO HELP A.I.
BUT HELP OURSELVES UNDERSTAND
HOW BRAINS ARE ORGANIZED.
>> A COUPLE THINGS THAT COME TO
MIND ARE LEARNING, LOCAL LEARNING
RULES VS. HOW A.I. SYSTEMS WORK.
NEUROMODULATOR CHANGING STATE
AND CONTRIBUTING TO LEARNING.
THERE'S PROBABLY MORE BUT ALSO I
WANT TO SAY FOR BROADER CONTEXT
MY PERSONAL OPINION IS PROBABLY
A.I. CAN DEVELOP MUCH FASTER NOT
NECESSARILY BORROWING FROM THE
BRAIN. BUT THE HOPE IS THAT WE
CAN LEARN SOMETHING THAT'LL BE
USEFUL FOR THE A.I. MY WORK IS
ABOUT USING A.I. TO LEARN ABOUT
THE BRAIN, HOW THE BRAIN WORKS,
WE WANT TO UNDERSTAND THE BRAIN.
>> IT GOES IN BOTH DIRECTION.
WE SHOULD BE ABLE TO EXTRACT
PRINCIPLES FROM THE LARGE
LANGUAGE MODELS TO HELP ASK NEW
QUESTIONS ABOUT THE BRAIN.
I AGREE WITH THAT.
WE HAVE LEARNED A LOT OF THINGS
FROM THE BRAIN THAT A.I. IS NOT
NOT USING ANYMORE. FOR EXAMPLE,
ATTRACTOR NETWORKS, DENDRITIC
COMPUTATION, SUPERVISED LEARNING
FROM MAP FORMATION.
THEY'RE OLD METHOD THAT WE LEARN
FROM THE BRAIN AND PUT INTO A.I.
AND THEN A.I. HAS ABANDONED FOR
NOW BUT THEY MIGHT NEED TO COME
BACK.
AND THERE'S A WHOLE LOT OF
THINGS A.I. HAS NOT
INCORPORATED.
I MENTIONED DEVELOPMENTAL LEARN
AND THINGS AT OTHER SCALES WE
COULD TAKE FROM BIOLOGY NOT JUST
NEUROSCIENCE BUT BIOLOGY IN
GENERAL AND INCORPORATE INTO
A.I.
>> I'D LIKE TO ADD ANOTHER
PRINCIPLE THAT WAS LEARNED FROM
THE BRAIN NOW BEING USED IN A.I.
WHICH IS SELF-SUPERVISION AND
REWARD PREDICTION ERROR THAT CAME
FIRST FROM TEMPORAL DIFFERENCE
LEARNING IN BASAL GANGLIA AND
DOPAMINE AND THE IDEA THOUGH YOU
COULD LEARN A LOT OF BY
PREDICTING THE NEXT SENSORY
INPUT OR PREDICTING IF YOU MAKE
A MOTOR ACTION WHAT'S THE
CONSEQUENCE IN THE CEREBELLUM?
>> THEY'RE IMPORTANT IDEAS AND
WE WANT TO GET TO A ZOOM
QUESTION AND WE MAY GO A FEW
MINUTES LATE BEFORE WE SEND YOU
TO LUNCH BUT A ZOOM QUESTION
NOW.
THANK YOU.
>> THIS QUESTION COMES FROM
ONLINE.
AND THE QUESTION IS I WONDER HOW
THE DATA COLLECTION COULD BE
ORGANIZED BEST IF NOT LAB-BASED
AND HOW TO INCREASE NEURO
DATA INTEROPERABILITY AND MAKE
IT DESCRIBE A NATURAL STATE.
>> WHAT WAS WITH THE SECOND
QUESTION?
>> HOW TO INCREASE NEUROSCIENTIFIC
DATA INTRAOPERABILITY AND INCREASE
THE NATURAL STATE.
>> THE FIRST QUESTION WAS HOW TO
COLLECT DATA NOT BASED ON
INDIVIDUAL LABS.
THIS WOULD BE THE CREATION OF
NATIONAL LABS WOULD BE ONE WAY
IN THE U.S. WE HAVE
EXPERIENCED THIS AND THAT WOULD
BE ONE APPROACH. CREATE A
NATIONAL LAB FOR THE LARGE-SCALE
DATA COLLECTION EXPERIMENTS AND
DATA AND EVERYTHING WOULD BE
MANAGED AT THAT LEVEL.
AND THEN ORGANIZATIONAL SYSTEM
WHERE PEOPLE HAVE ACCESS TO THE
DATA AND MAYBE THEY PROPOSE
EXPERIMENTS AFTER THEY ANALYZE
AND RUN INSILICO SIMULATIONS.
I THINK THIS IS A WAY FORWARD
JUST NEEDS TO RETHINK THE WHOLE
INFRASTRUCTURE OF HOW WE DO
THINGS RIGHT NOW.
>> DO WE HAVE A QUICK QUESTION
IN THE ROOM?
>> YEAH, FOLLOWING UP ON THAT MY
QUESTION ORIENTED TOWARDS
DOMINIQUE AND DIMITRI
ESPECIALLY.
I'M CURIOUS WHEN THINKING ABOUT
THE MASSIVE HETEROGENEITY OF
DATA SETS AND INCONSISTENT DATA
FILE FORMATS, METADATA, SCHEMAS,
WHAT OPPORTUNITIES ARE THERE TO
LEVERAGE LARGE SCALE LLMs TO
HELP FACILITATE THAT
HARMONIZATION AND CURATION
PROCESS IN ORDER TO PULL THE
DATA INTO THE TYPES OF LARGE
MODELS WE'RE TALKING ABOUT.
>> YES, OPPORTUNITIES ARE THERE
FOR REUSING THE DATA.
WE'RE FOCUSING MORE ON THE
QUESTIONS OF THE APPLICATION
THAT'S EXISTING. THE MAIN
PURPOSE OF THIS DISCUSSION IS
ABOUT DESIGNING STUDIES, COST
FUNCTIONS TO LEARN AND EXTRACT THE
MAXIMUM AMOUNT OF INFORMATION
AND DESIGN CLOSED-LOOP
EXPERIMENTS AND MAKING SENSE OF
DIVERSE EXPERIMENTS.
IT'S A SLIGHTLY DIFFERENT
PROBLEM.
THAT'S MORE OF THE TECHNOLOGICAL
SIDE OF THINGS AND WORKING
ACROSS LABS AND YES EVERY LAB
UNLIKE OTHER RELATES DISCIPLINES
LIKE BIOINFORMATICS, NEUROSCIENCE
WILL LIKELY NOT CONVERGE TO A
SET OF STANDARD EXPERIMENTS.
WE'LL ALWAYS BE EXPLORING DIVERSE
PARADIGMS FOR EXPERIMENTS.
THESE APPROACHES BECOME
NECESSARY.
>> THAT'S A REAL CHALLENGE
IN DABI PEOPLE HAD THEIR OWN
WAYS OF USING FORMATS AND
STANDARDS AND NOT EVERYONE WANTS
TO USE BIDS FOR EXAMPLE.
IT'S A HUGE EFFORT TO CONVERT
THEIR DATA INTO BIDS.
IT'S A TIME AND RESOURCES AND
THAT MAY NOT BE AVAILABLE.
FINDING WAYS TO HELP IN THAT
CONVERSION AND HAVE CONVERTERS
THAT MAKE IT EASIER TO CONVERT
TO A STANDARDIZED FORMAT AND
STRUCTURE WOULD BE BENEFICIAL.
>> AL XIN, I'M A TRAINEE AT THE
MACHINE LEARNING CORE AT NIMH
SEVERAL PRESENTERS MENTIONED THE
DESIDERATA OF NATURAL INTELLIGENCE
REQUIRE LESS TRAINING ON LESS
DATA BUT I WANTED TO INTERROGATE
THE ROBUSTNESS OF THIS CLAIM.
IN TERMS OF DATA NATURAL
INTELLIGENCE IS TRAINED ON. IT'S
VERY DIFFERENT FROM THE TYPE OF
DATA SETS WE CAN PRODUCE IN TERMS
OF TEXT DATA AND IMAGE DATA AND
EVEN VIDEO DATA. I WAS WONDERING
IF ANYONE CAN COMMENT ON EFFORTS
TO CREATE DATA SETS THAT CAPTURE
THE COMPLEXITY OF NATURAL
INTELLIGENT TRAINED DATA IN ORDER
TO CREATE MODELS THAT HAVE THESE
CHARACTERISTICS. HOW ROBUST IS THE
CLAIM THAT IT REQUIRES LESS
TRAINING TIME ON LESS DATA?
IS THERE EFFORTS TO CURATE
NATURALLY INTELLIGENT DATA SETS?
>> YEAH, THERE'S A DATA SET
COLLECTED BY PEOPLE AT
STANFORD WHERE THEY PUT
HEAD CAMERAS IN CHILDREN AND
COLLECTED THIS KIND OF DATA.
WHERE BASICALLY YOU COLLECT HOW
CHILDREN INTERACT WITH THE WORLD
AND USE THE DATA TO TRAIN A.I.
SYSTEMS.
YOU RAISE A VERY GOOD QUESTION.
I THINK IT'S MOSTLY LIKE WHEN
PEOPLE SAY THAT WE GET TRAINED
ON THE DATA, A LOT OF IT HAS TO
DO WITH LANGUAGE.
THAT'S A CLEAR CASE.
LIKE NOBODY AND THERE'S EVIDENCE
WE HUMANS ARE MUCH BETTER AT
GENERALIZING OUR TRAINING
DISTRIBUTION WHICH IS THE HALL-
MARK OF INTELLIGENCE IN MANY
WAYS. AND THERE'S PSYCHOPHYSICAL
AND BEHAVIORAL EVIDENCE FOR
THAT.
>> THAT'S GREAT.
I THINK WE'VE GOT A LOT OF
MATERIAL TO CONTINUE OUR
CONVERSATIONS OVER LUNCH AND
OVER ALL THE REMAINING SESSIONS
OF THE WORKSHOP.
SO ONCE AGAIN I WANT TO THANK
ALL OF OUR SPEAKERS AND INVITED
DISCUSSANTS AND ALL THE
PANELISTS AND AUDIENCE MEMBERS
IN THE ROOM AND WATCHING ON ZOOM
AND VIDEOCAST.
PANELISTS AND ORGANIZERS, PLEASE
STAND BY FOR A GROUP PHOTO BY
THE STAIRWELL AFTERWARDS BUT
EVERYONE ELSE IS FREE TO FIND LUNCH.
THANK YOU SO MUCH.
All right.
It's one 115.
So I'd like to welcome everyone back to the first program
in the afternoon of the NEURO AI workshop.
We're delighted to have gathered a bunch
of different types of funders, investors and sponsors.
And many of you may have noticed the carousel slides
earlier with lots of funding opportunities.
So, we will not repeat those slides
unless one of our panelists need to reference to them.
During the course of this discussion.
So let me introduce myself.
I am Grace Hwang.
I am a Program Director
at the National Institute
of Neurological Disorders and Stroke,
and I support the BRAIN Initiative,
where I manage a portfolio of projects on neuromodulation
recording and stimulation technologies.
And I'm honored to introduce our moderator
today, Doctor TERRENCE SEJNOWSKI
He is a professor and laboratory
head of the Computational Neurobiology Laboratory.
HE IS A PIONEER IN
COMPUTATIONAL NEUROSCIENCE, HE HAS MANY ACCOLADES
that you can read about IN THE PROGRAM BOOK.
HIS MOST RECENT RECOGNITION IS the 2024
LUNDBEck FOUNDATION’s BRAIN PRIZE FOR HIS
CONTRIBUTIONS TO THEORETICAL AND COMPUTATIONAL NEUROSCIENCE.
HIS COLLABORATIVE WORK WITH GEOFF HINTON ON LEARNING ALGORITHMS
FOR BOLTZMANN MACHINES FROM 1985 WHICH WAS FEATURED
IN TONY ZADOR’S TALK EARLIER TODAY, ALSO SUBSTANTIALLY
CONTRIBUTED TO THE RECENT NOBEL PRIZE IN PHYSICS
FOR FOUNDATIONAL DISCOVERIES AND INVENTIONS THAT
ENABLE MACHINE LEARNING WITH ARTIFICIAL NEURAL NETWORKS.
I'M GOING TO GO OVER MINOR
HOUSEKEEPING HERE.
AND LET THE AUDIENCE KNOW WE'LL
FIRST DO INTRODUCTIONS AND GO TO
THE FUNDER PANEL GUIDING
QUESTIONS AND DISCUSSION SLIDE.
ANYBODY WHO WANTS TO SEND THEIR
RESPONSE IS WELCOME TO SCAN THE
QR CODE, WHETHER YOU'RE WATCHING
THIS PROGRAM LIVE OR AS A
RECORDING BECAUSE WE WILL BE
MONITORING RESPONSES IN E-MAIL
FOR SOME WEEKS TO COME.
AND WITH THAT, TERRY, WHY DON'T
WE GO BACK TO YOU.
>> I'D LIKE TO BEGIN BY ASKING
ALL OF THE PANEL DISCUSSANTS TO
INTRODUCE THEMSELVES.
NAME, RANK, SERIAL NUMBER, SO WE
DON'T TAKE TOO MUCH TIME FROM
OUR DISCUSSION.
>> I CAN HEAR MY VOICE.
I'M STEVEN ZENDER, NATIONAL
SCIENCE FOUNDATION.
>> I'M ALYSSA SCHAFER, SENIOR
SCIENTIST AND VICE PRESIDENT,
SIMONS FOUNDATION, WHERE I
DIRECT THE NEUROSCIENCE
COLLABORATION.
>> GOOD AFTERNOON, STEPHANIE
GAGE, ASSOCIATE PROGRAM
DIRECTOR, NATIONAL SCIENCE
FOUNDATION, WORK IN THE
COMPUTER SCIENCE DIRECTORATE IN
THE COMPUTING AND COMMUNICATIONS
FOUNDATIONS DIVISION.
>> OKAY.
THANK YOU.
GOOD AFTERNOON, ROBINSON PINO,
PROGRAM DIRECTOR, DEPARTMENT OF
ENERGY.
THANK YOU.
>> HAL GREENWALD, PROGRAM
OFFICER AT THE AIR FORCE OFFICE
OF SCIENTIFIC RESEARCH, WHICH IS
AIR FORCE'S RESEARCH FUNDING ARM.
I FUND HIGH RISK HIGH IMPACT
RESEARCH IN THE COGNITIVE AND
COMPUTATIONAL NEUROSCIENCE
PROGRAM WHICH FUNDS WORK ON THE
MECHANISMS OF PERCEPTION,
COGNITION, BEHAVIOR, ALSO AT THE
INTERSECTION OF NEUROSCIENCE AND
A.I., I ALSO MANAGE THE
COMPUTATIONAL COGNITION PROGRAM,
AN A.I. PROGRAM.
>> GOOD AFTERNOON, THANK YOU,
GRACE, FOR THE INVITATION AND
OPPORTUNITY TO BE HERE.
I'M CHRISTINE EDWARDS, NATIONAL
SECURITY AGENCY, I'VE BEEN THERE
FOR ABOUT 30 YEARS.
MY MOST CURRENT ROLE IS DARPA,
I'M A NSA REPRESENTATIVE AT DARPA,
A NEW POSITION FOR THEM SO IT'S A
LIAISON TYPE ROLE.
SO I GET TO WORK WITH PEOPLE
ACROSS THE DIFFERENT DARPA
TECHNICAL OFFICES, THERE'S SIX
OF THEM TOTAL.
AND I SIT WITHIN THE INFORMATION
INNOVATION OFFICE.
I LOOK FORWARD TO TALKING WITH
PEOPLE DURING THIS SESSION BUT
ALSO AFTERWARDS. THERE'S A TABLE
WITH HANDOUTS IF YOU'RE
INTERESTED.
>> I'M CHOU HUNG, PROGRAM
MANAGER WITH ARMY RESEARCH OFFICE.
EXTRAMURAL ARM OF ARMY RESEARCH
LABORATORY. MY FUNDING TOPIC AREA
IS NEUROPHYSIOLOGY OF COGNITION,
AREAS THAT COVER INCLUDES
EVOLUTIONARY BIOLOGY,
INTERACTIONS WITH REAL AND
AUGMENTED WORLD, BIO-INSPIRED
AND NEUROMORPHIC COMPUTING.
THANK YOU.
>> DAVID WYATT, THIS IS MY
SECOND NIH CONFERENCE.
I'M A RECOVERING ENGINEER, SO I
USED TO WORK IN SILICON VALLEY,
AT NVIDIA.
AND THESE DAYS I'M RUNNING THE
OPENGPU FOUNDATION AS WELL AS
OTHER START-UPS IN AUSTIN WHICH
IS NOW SILICON HILLS.
>> WE'RE LIVING IN EXTRAORDINARY
TIMES.
EVOLUTION WAS BROUGHT UP IN THE
FIRST SESSION.
AND JUDGING BY THE TIME SCALE OF
EVOLUTION, WHAT WE'RE
EXPERIENCING RIGHT NOW OVER THE
LAST DECADE IS LIKE AN
EXPLOSION.
AND EVEN JUDGING BY THE TIME
SCALE OF CULTURAL CHANGE, THE
INDUSTRIAL REVOLUTION TOOK
HUNDREDS OF YEARS TO PERMEATE
THROUGH SOCIETY, AND NOW A.I. IS
SPREADING ON THE TIME SCALE OF
DECADES.
SO, THIS IS WITHIN, YOU KNOW,
LIFETIMES OF PEOPLE WHO ARE
GOING TO BE BOTH USERS AND
DEVELOPERS SO THIS IS AN
EXTRAORDINARY GROUP OF FUNDERS
WE HAVE.
WE HAVE THE CHANCE TO FIND OUT A
LITTLE BIT ABOUT WHAT
OPPORTUNITIES ARE GOING TO BE
LIKE IN THE FUTURE.
I'LL START BY A QUESTION, WHY IS
YOUR AGENCY OR ORGANIZATION
INTERESTED IN NEUROAI?
ANYONE WANT TO TAKE THAT
QUESTION?
ANYBODY CAN JUMP IN.
>> YOU KNOW, WHY NOT?
OH, GO AHEAD.
>> THANK YOU.
SO, I'VE BEEN AT DARPA FOR ABOUT
A YEAR SO FAR, TWO MORE TO GO AT
LEAST.
BUT DARPA AS MANY OF YOU KNOW IS
ABOUT HIGH-RISK HIGH-PAYOFF
RESEARCH.
I'VE BEEN IN NEUROSCIENCE AND
A.I. FOR DECADES NOW, THIS IS A
REALLY EXCITING TIME, AS YOU
STATED.
AND SO DARPA IS LOOKING AT THE
BIG IDEAL THAT ARE COMING.
MY PERSONAL PERSPECTIVE, I THINK
OTHERS AT DARPA SHARE THAT,
GROWING MOMENTUM IN THE FACE OF
NEUROSCIENCE AND A.I.,
NEUROMORPHIC COMPUTING,
HUMAN-MACHINE COLLABORATION,
THAT TIGHTLY COUPLED LOOP. THE
OODA LOOP, AND THE A.I. POWERED
OODA LOOP. AND HOW DO YOU DO THAT
IN A TRUSTWORTHY WAY.
TO GET TO NEXTGEN A.I. WE CAN
BENEFIT BY UNDERSTANDING HOW
BRAINS WORK, NEUROAI IS PERFECT
FOR THAT. SCALING IS GREAT, BUT
CAN ONLY GO SO FAR. SO WHAT'S
BEYOND THAT? ALGORITHMS, ARCHI-
TECTURE, HARDWARE, THE FULLSTACK.
>> WHICH PARTS OF DARPA?
>> SO, I SIT WITHIN I2O, CARES
ABOUT NEXT GENERATION A.I.,
CYBERSECURITY APPLICATIONS,
RESILIENCY.
THAT'S ANOTHER THING THAT WE'RE
THINKING ABOUT AS WELL AS THE
SECURITY OF LET'S SAY DEVICES
THAT ARE BASED ON NEUROMORPHIC
HARDWARE, WE SHOULD BE THINKING
ABOUT ARE THERE BENEFITS FROM A
SECURITY PERSPECTIVE TO MAKE
THINGS PRIVATE AND MORE SECURE
GOING FORWARD.
I THINK NOW IS THE TIME TO THINK
ABOUT THAT WHEN WE'RE THINKING
ABOUT NEUROMORPHIC COMPUTE
STACK.
AND THEN MTO IS THE MICROSYSTEMS
TECHNOLOGY OFFICE, THEY THINK
ABOUT DIFFERENT TYPES OF
HARDWARE, THAT INCLUDES QUANTUM,
CAN INCLUDE BIO, NEUROMORPHIC
COMPUTING, ANALOG, NEURAL
NETWORK HARDWARE.
>> IT'S ACROSS MANY --
>> YEAH, AND THERE'S BTO,
BIOLOGICAL TECHNOLOGY OFFICE, A
NATURAL FIT.
>> GREAT.
HAL WAS GOING TO RESPOND TOO.
>> SURE.
SO, IN GENERAL, SO THE FIRST
QUESTION IS WHY IS THE AIR FORCE
INTERESTED IN NEUROSCIENCE.
AND THERE ARE A NUMBER OF
REASONS FOR THAT.
WHEN YOU THINK ABOUT THE AIR
FORCE, YOU MAY THINK FIRST ABOUT
AIRCRAFT BUT THE AIR FORCE
REALLY DEPENDS FIRST AND
FOREMOST ON ITS PEOPLE, NOT JUST
TO PILOT THE AIRCRAFT CURRENTLY
BUT FOR ALL THE OTHER
OPERATIONS, THE OTHER ASPECTS OF
OPERATIONS THE AIR FORCE AND NOW
THE SPACE FORCE ARE INVOLVED IN.
SO ONE OBJECTIVE IS TO
UNDERSTAND HOW HUMANS PROCESS
THE INFORMATION AROUND US, TO
MAKE HUMANSS SMARTER, MORE
CAPABLE, FITS WITH THE IDEAS TO
MAKE COMPUTERS SMARTER AND MORE
CAPABLE.
AND AS THE SPEAKERS THIS MORNING
DISCUSSED, IT MAKES SENSE TO
HAVE RESEARCH IN HUMAN --
NEUROSCIENCE, WHETHER HUMANS OR
ANIMAL MODELS, COMPUTATIONAL
MODELS AS WELL, INVESTMENTS IN
A.I. LOOKING AT PRINCIPLES FROM
THE BRAIN, COMPUTATIONS THE
BRAIN PERFORMS, TO HAVE NICE
CYCLES BETWEEN THEM SO
PREDICTIONS FROM ONE PART CAN BE
TESTED IN THE OTHER.
>> GO AHEAD.
>> THANK YOU, TERRY.
THE SIMONS FOUNDATION'S MISSION
IS TO ADVANCE THE FRONTIERS OF
RESEARCH IN MATHEMATICS AND THE
BASIC SCIENCES.
I THINK OUR INTEREST IN A.I.,
LIKE YOU GUYS AT DARPA, LIES
ACROSS A BUNCH OF OUR DIVISIONS.
SO WE HAVE AN INTRAMURAL
DIVISION, THE FLATIRON
INSTITUTE, WHICH IS AN IN-HOUSE
COMPUTATIONAL RESEARCH INSTITUTE
THAT HOUSES OUR FIVE CENTERS,
COMPUTATIONAL NEUROSCIENCE,
COMPUTATIONAL BIOLOGY,
COMPUTATIONAL MATHEMATICS, QUANTUM
CHEMISTRY, ASTROPHYSICS
THEN ON THE EXTRAMURAL SIDE, OUR
OUTWARD-FACING GRANTING
DIVISION, I SIT IN THE
NEUROSCIENCE DIVISION, OUR
NEUROSCIENCE COLLABORATIONS THAT
I DIRECT ARE FOCUSED ON
UNCOVERING FUNDAMENTAL
PRINCIPLES OF BRAIN FUNCTION.
MY COLLEAGUES IN MY SISTER
DIVISION IN AUTISM ARE FOCUSED
ON BASIC AUTISM RESEARCH.
AND SO OUR INTEREST IN NEUROAI
AND A.I. IS SPREAD ACROSS THESE
DIVISIONS.
SO BOTH NEUROSCIENCE
COLLABORATIONS AND FLATIRON
INSTITUTE ON FOCUSED ON USING
NEUROAI TO UNDERSTAND THE BRAIN
AND HOW THE BRAIN WORKS, WHETHER
STUDYING BIOLOGICAL CONSTRAINED
NEURAL NETS AS MODEL SYSTEMS
UNDERSTAND THE BRAIN OR HARNESSING
THE POWER OF NEUROINSPIRED A.I.
AND A.I. MODELS TO ANALYZE DATA.
IN THE AUTISM RESEARCH
INITIATIVE, OUR INTEREST IN A.I.
EXTENDS SPECIFICALLY TO
AUTISM-RELATED WORK, USING A.I.
TO SPOTLIGHT PATHWAYS FROM
GENOMICS TO BIOLOGY.
FLATIRON INSTITUTE, THE WORK ON
MODERN DEEP NETWORK THEORY, ALSO IN
OUR CENTER FOR COMPUTATIONAL
BIOLOGY WE HAVE RESEARCHERS WITH
PROTEIN PROTEIN INTERACTIONS.
WE HAVE A LARGE INTEREST IN
INVESTMENT IN A.I. AT THE SIMONS
FOUNDATION, FROM LOTS OF
DIFFERENT ANGLES.
>> THAT'S ONE OF THE THINGS
ABOUT A.I., IT SEEMS TO PERMEATE
EVERYTHING, RIGHT?
EVERYTHING FROM ENHANCING HUMAN
PERFORMANCE, COMPUTERS, AND
ESPECIALLY PEOPLE WHO SUFFER
FROM MENTAL DISORDERS,
NEUROLOGICAL DISEASES.
THIS IS REALLY EXCITING.
OKAY.
OTHERS THAT WANT TO JUMP IN?
STEVE?
>> OKAY, THANKS.
TERRY, WHAT YOU JUST SAID STOLE
MY ANSWER.
A.I. PERMEATES EVERYTHING.
AT THE NATIONAL SCIENCE
FOUNDATION, WE ARE, YOU KNOW,
OUR MISSION IS TO ADVANCE
SCIENCE FOR THE BENEFIT OF THE
NATION, FULL STOP.
WE'RE TRYING TO POSITION
OURSELVES TO BE A LEADER IN THE
FUTURE OF A.I., PREMIER FEDERAL
FUNDING, FUTURE OF A.I., VARIOUS
AREAS OF BIOTECHNOLOGIES ALIGNED
HERE.
THE PROBLEMS WE'VE BEEN TALKING
ABOUT TODAY, YOU KNOW, LIKE THE
POWER ISSUES WITH A.I., RIGHT?
YOU KNOW, MAYBE WE NEED HARDWARE
SYSTEMS, WE NEED FUNDAMENTAL
RESEARCH INTO HARDWARE SYSTEMS
FASTER, CHEAPER, UNDERSTANDING
INTERACTION AND CONTROL OF
PHYSICAL SYSTEMS THAT INTERFACE
WITH A.I., ALL OF THESE SORTS OF
ISSUES COULD GENERATE
FUNDAMENTAL SCIENCE PROPOSALS
THAT COULD BE SUBMITTED TO THE
NSF.
I DON'T KNOW, STEPHANIE, IF YOU
WANT TO ADD ANYTHING?
IT REALLY WAS, TERRY SUMMED IT
UP.
I DON'T THINK THERE'S AN AREA OF
NSF THAT ISN'T INTERESTED IN
NEUROAI. WE HAVE A NEW
UNDERSTANDING THE BRAIN WEBSITE
TO HELP THE PUBLIC THINK ABOUT
HOW NSF THINKS ABOUT NEUROSCIENCE
AND BRAIN RESEARCH. FROM NEURAL
MECHANISMS AND PROCESSING TO
UNDERSTANDING INTELLIGENCE,
BRAIN-INSPIRED CONCEPTS OF
DESIGN, THEORY, COMPUTATION,
ANALYSIS, MODELING OF THE BRAIN,
HUMAN COGNITION OF SOCIETY.
I'M SURE I'M MISSING OTHERS.
NEUROTECHNOLOGY AND
INFRASTRUCTURE.
EVERY TALK THAT I'VE HEARD SO
FAR HAS A PLACE AT NSF FOR
FUNDING THOSE BASIC QUESTIONS.
>> YES, GO AHEAD.
>> OKAY.
SO, IN ADDITION TO OVERALL
INTEREST IN A.I., THERE'S A LOT
OF INTEREST, ENHANCEMENTS, BUT
LOOKING INTO THE FUTURE, ONE OF
THE ISSUES, WE'RE NOT REALLY
EVOLVED FOR THE WORLD WE'RE
LIVING IN NOW.
ALL THE A.I., AUGMENTED REALITY,
YOU KNOW.
AND WHAT THEY FORESEE IN THE
FUTURE WARFARE IS MORE AUTONOMY,
MORE A.I.-ENABLED DECISION
SUPPORT.
FOR THAT YOU NEED A.I. THAT'S
SMART ENOUGH TO BE MORE
AUTONOMOUS.
YOU KNOW, WITH THE LIMITATIONS
WITH EXISTING DATA, EXISTING
A.I., WE NEED SOME NEW
PRINCIPLES, NEW THEORIES, NEW
APPROACHES
>> AUTONOMY CAME UP IN THE
PREVIOUS SESSION.
A.I. RIGHT NOW CAN TALK THE TALK
BUT CAN'T WALK THE WALK BECAUSE
IT DOESN'T HAVE A BODY.
BUT AUTONOMY MEANS MORE THAN
JUST A BODY.
IT MEANS IT HAS ITS OWN SELF,
IT'S SELF-GENERATED ACTIVITY.
WHEN YOU STOP TALKING, YOU KNOW,
TO ChatGPT, YOU PRESS THE
BUTTON, IT COMPLETELY
DISAPPEARS.
ChatGPT JUST GOES BLANK.
THERE'S NO INTERNAL THOUGHT
THERE.
RIGHT?
AND SO WE HAVE A LONG WAY TO GO
BEFORE WE GET ANY PLACE NEAR TO
AUTONOMY.
GO AHEAD.
>> I'D LIKE TO RECAST THE
QUESTION SO INSTEAD OF THE WORD
NEUROAI IN THERE, LET'S GO BACK
TO THE, SAY, EARLY INDUSTRIAL
REVOLUTION AND REPLACE THE WORD
WITH THE RAILWAY, TRAINS.
WHY ARE WE INTERESTED IN TRAINS?
WELL, PERHAPS WE'D LIKE TO CROSS
THE CONTINENT, BUILD A COUNTRY,
LET'S CALL IT AMERICA.
WE NEED TO GET FROM ONE SIDE TO
THE OTHER, EFFECTIVELY.
IT'S GOING TO AFFECT EVERY
VERTICAL, WHETHER THE ARMY,
WHETHER IT'S PASSENGER
MOTIVATION.
A.I. IS HORIZONTAL TECHNOLOGY
THAT AFFECTS EVERY VERTICAL.
WE HAVE TO FIGURE OUT HOW TO GET
INVOLVED EARLY TO MAKE IT MORE
EFFICIENT, MORE PROLIFIC.
>> YEAH, THIS IS OFTEN THE CASE
WITH I MEAN THE RAILROADS ARE A
GOOD EXAMPLE.
THE INTERNET IS AN EVEN BETTER
EXAMPLE.
YOU REMEMBER THERE WAS A TIME
WHEN THEY WERE PUTTING IN
FIBEROPTICS EVERYWHERE, WHEN OF
COURSE THERE WAS THE INTERNET
BUBBLE BURST.
THAT ALL DISAPPEARED.
IT WENT SILENT, DARK, THEY CALL
IT DARK.
NOTHING GOING THROUGH THE FIBER.
THIS MIGHT HAPPEN RIGHT NOW,
IT'S THE CURRENT A.I., IT'S JUST
ACCELERATING SO MUCH.
$100 BILLION, LARGE COMPANIES
ARE PUTTING IN $100 BILLION
INVESTMENTS IN DATA CENTERS.
SO MUCH POWER, GIGAWATTS OF
POWER THEY NEED TO BUILD NUCLEAR
POWER PLANTS NEXT TO THEM.
SO I DON'T THINK THAT'S
SUSTAINABLE BUT MAYBE I'M WRONG.
>> I'D LIKE TO BRING THIS
CONVERSATION BACK TO NEUROAI,
WHICH IS NOT A.I., RIGHT?
>> NO, NO, RIGHT NOW A.I. IS
PULLING THE LOAD, RIGHT?
AND WHERE WE GO AS NEUROAI IS
COUPLED, YOU KNOW. WE'RE ACTUALLY
IN TERMS OF AMOUNT OF FUNDING
INVOLVED, I DON'T
KNOW WHAT THAT NUMBER IS, BUT
WE'RE THE CABOOSE.
OKAY?
>> YEAH, OKAY.
SORRY, I BROUGHT UP THE RAILWAY
ANALOGY.
JUMP TO CARS AND MOTORCYCLES.
INTERNET ANALOGY, MORE RECENT,
SURE, WE ALL STARTED WITH
MODEMS.
I THINK GRACE'S POINT IS WE'RE
LOOKING FORWARD, PERHAPS TO A
DAY WE CAN HAVE VIDEO
TELECONFERENCE AS PORTRAYED IN
THE MOVIE 2001 A SPACE ODYSSEY,
RIGHT?
IT'S GOING TO TAKE FIBER.
PEOPLE KNEW, GRACE IS POINTING
OUT NEUROAI THERE'S AN IMPLICIT
UNDERSTANDING WE'VE GOT TO DO IT
BETTER, EARLIER, AS FAST AS
POSSIBLE, TO YOUR POINT WITH
EFFICIENCY.
>> IT'S TRUE.
IT'S HEADING IN THAT DIRECTION.
I THINK THAT IN THE LONG TERM
WE'LL GET THERE BUT IT'S GOING
TO BE INCREMENTAL.
MAYBE THERE WILL BE SOME HUGE
BREAKTHROUGH, I DON'T THINK YOU
CAN PREDICT THAT.
>> EVEN BETTER ANSWER TO THE
FIRST QUESTION, BECAUSE I WANT
TO MAKE IT HAPPEN FASTER.
>> OKAY, GOOD.
NO, THAT'S A GREAT ANSWER.
BECAUSE WHAT WE'RE TRYING TO DO.
>> NIH IS INTERESTED IN NEUROAI
OR BRAIN WOULDN'T BE HOSTING
THIS WORKSHOP.
NEUROAI REQUIRES TRUE
INTERDISCIPLINARY COLLABORATION.
BEYOND WHAT BRAIN HAS ACHIEVED.
NIH BRAIN IS A GREAT EXAMPLE OF
CONVERGENT SCIENCE, WE BROUGHT
TOGETHER ENGINEERS, CLINICIANS,
NEUROSCIENTISTS TO BUILD AMAZING
TECHNOLOGY, ALL IN THE SERVICE
OF NEUROSCIENCE.
NEUROAI IS DIFFERENT.
IT'S ABOUT TWO TO THREE FIELDS
COMING TOGETHER TO TEACH EACH
OTHER, THAT RECIPROCAL VIRTUOUS
LOOP HAL MENTIONED.
NEUROAI WILL GIVE US NEW KNOWLEDGE
ABOUT ROBOTICS, NEUROSCIENCE,
THAT'S THE PROMISE.
>> WE HAD SLIDES SHOWING THAT
VIRTUOUS CIRCLE, BUT ALSO
POINTED OUT THAT IN FACT MODERN
A.I. WAS BUILT FROM 20th
CENTURY NEUROSCIENCE, AND NOW
THE QUESTION IS -- THAT THIN
STRAND ON THE TOP, HOW IS THAT
GOING TO BE FATTENED UP?
BUT, YOU KNOW, I TAKE YOUR
POINT, IT'S A TWO-WAY STREET.
BUT REALLY THINK IT'S NOT JUST
NEUROSCIENCE AND A.I.
WE'RE TALKING ABOUT, YOU KNOW,
THE FUTURE, AND IT'S GOING HAVE
AN INFLUENCE ON EVERY PERSON ON
THE PLANET.
THIS IS GOING TO HAVE HUGE
IMPACT.
SO REGULATORY ISSUES, WE JUST
TOUCHED ON THOSE.
BUT, YOU KNOW, WE'RE REALLY JUST
BEGINNING THAT PROCESS AND OF
COURSE THIS HAS HAPPENED MANY
TIMES WITH DIFFERENT
TECHNOLOGIES.
DOES ANYONE ELSE WANT TO FINISH
THIS BEFORE WE GO TO THE NEXT
TOPIC?
OKAY.
NEXT TOPIC IS WHAT MULTI-AGENCY
EFFORTS ALREADY EXISTS THAT COULD
BE A MODEL FOR THE COLLABORATION
IN NEUROAI?
>> I'M NOT AWARE OF ANY.
>> WHY IS THAT?
WHY IS THAT?
>> I THINK IT'S, AGAIN, A CHANCE
TO REPLACE THE WORD WITH
SOMETHING ELSE.
AND THEN REFLECT ON THE
SITUATION.
OKAY?
SO, I'LL TAKE LIBERTIES.
REPLACE THE WORD NEUROAI WITH
USB.
WE ALL USE IT.
I'VE GOT IT HERE.
WHAT MULTIAGENCY EFFORTS EXIST
THAT COULD BE A MODEL FOR
COLLABORATION IN USB?
>> WELL, WE ACTUALLY HAVE
EXISTING MULTIAGENCY
COLLABORATION IN PLACE FOR
NEUROAI, SCH, FOLKS IN THIS
ROOM PROBABLY FAMILIAR WITH
SMART-CONNECTED HEALTH PROGRAM,
COLLABORATIVE RESEARCH AND
COMPUTATION NEUROSCIENCE
PROGRAM.
I REALIZE YOU'RE NOT AN NSF NIH
GRANTEE, BUT THERE'S OBVIOUS
ANSWERS TO NUMBER 2, AND MORE
RECENTLY NSF AND OUR COLLEAGUES
AT NIBIB CREATED A THIRD MODEL
THAT'S REALLY INTERESTING.
>> OKAY.
I THINK WHEN I READ THE WORD AGENCY
I INTERPRETED IT-PRIVATE SECTOR
>> BY THE WAY, IEEE HAD A LOT TO
DO WITH THE STANDARDS.
>> ACTUALLY THEY HAD NOTHING TO
DO WITH IT.
>> REALLY?
>> I TAKE THAT BACK.
IEEE 3094 WAS COMPETING STANDARD TO
USB, WANTED TO CHARGE A ROYALTY,
SEVERAL SAID NO.
MY FRIEND IN THE FOLSOM FACTORY AT
INTEL SAID LET'S DO IT FOR FREE.
>> THIS IS REALITY.
WE'RE TALKING ABOUT IDEAL --
OKAY.
>> YEAH.
THE PROBLEM WITH MULTIAGENCY
WAS I READ PRIVATE SECTOR AS
WELL INTO THAT.
BUT YOU'RE RIGHT, IT COULD BE AN
ENTIRE GROUP OF AGENCY
COLLABORATIONS THAT WE KNOW
NOTHING ABOUT.
PERHAPS TURN THAT AROUND, HOW
COULD WE FIX THAT SO THE PRIVATE
SECTOR KNOWS MORE ABOUT IT?
>> I KNOW NSF AND NIH ALWAYS
COLLABORATE WITH EACH OTHER,
RIGHT?
>> THAT'S TWO AGENCIES, NOT WITH
PRIVATE
>> I WAS HOPING THIS QUESTION
WOULD GO THAT WE LOOK AT WHAT WE
KNOW ABOUT SCH AND CRCNS TODAY,
DO WE THINK IT WOULD BE A GOOD
MODEL FOR SUPPORTING FUTURE
NEUROAI RESEARCH THAT'S
INCLUSIVE OF EXTERNAL
PARTNERSHIPS WITH INDUSTRY.
>> BETWEEN GOVERNMENT AND
INDUSTRY.
>> RIGHT.
BECAUSE CRNCS AND SCH TODAY IS
PURELY GOVERNMENT FUNDING AGENCIES,
I DON'T BELIEVE THERE'S A COMMERCIAL
PARTNER ON THOSE OPPORTUNITIES.
>> THAT MIGHT BE AN OPPORTUNITY
TO GO BACK TO FIRST PRINCIPLES.
WHY DO WE WANT TO INCREASE
COLLABORATION AND PERHAPS THE
REASON IS BECAUSE WE WANT TO
PROLIFERATE IT MORE EFFICIENTS,
COST EFFICIENT.
>> CYBERSECURITY, IF YOU THINK
ABOUT OUR MISSION SPACE WE MIGHT
HAVE DIFFERENT MISSION SPACE BUT
THERE'S CORE COMMONALITIES,
SECURITY IS ONE.
POWER EFFICIENCY, DATA
EFFICIENCY.
ASPECTS OF ETHICS.
I'M MISSING ONE MORE
COMMONALITY.
>> PROBABLY COST.
>> COST, YES.
LEVERAGED RESOURCES.
>> I THOUGHT I WAS IN
WASHINGTON.
>> THERE'S AN EXAMPLE, META
RELEASED LLAMA 3, OPEN SOURCE,
ONE OF THEIR BIGGEST LARGE
LANGUAGE MODELS.
IT'S SPURRING A LOT OF START-UPS
AND ACADEMICS WHO COULDN'T
AFFORD TO BUILD THEIR OWN,
RIGHT?
THAT'S AN EXAMPLE WHERE, YOU
KNOW, BIG -- YOU KNOW, HIGH TECH
INDUSTRY IS HELPING ALL OF US TO
EXPLORE THIS NEW DIRECTION.
>> SORRY TO PUT A DOSE OF
REALITY BACK IN THAT ONE AGAIN.
AS AN IEEE VERSUS USB, REALITY IS
THEY ARE NOT SHARING SOME OF THE
CORE TRAINING -- NOT SHARING
NEAR ENOUGH OF THE CORE TRAINING
DATA TO MAKE IT ACTUALLY USEFUL.
>> OKAY.
YEAH, IT DEPENDS ON WHAT THEY
ARE RELEASING, YOU'RE RIGHT.
BUT THEY HAVE RELEASED ACTUAL
NETWORK.
>> YEAH.
>> PEOPLE CAN FINE TUNE IT FOR
THEIR PURPOSES AND A LOT OF
PEOPLE ARE DOING THAT, FOR
SPECIFIC APPLICATIONS
>> WELL, THE TRAINING.
AGAIN, WITH RESPECT TO TRAINING
TO CREATE WHAT THE WEIGHTS ARE
IN THAT MODEL IS WHAT REQUIRES
DATA CENTERS GPUs, BILLIONS OF
DOLLARS.
>> YEAH, BUT -- OKAY.
ONCE YOU HAVE THAT INVESTMENT IN
THIS LARGE LANGUAGE MODEL, FINE
TUNING IS ONLY A SMALL FRACTION
OF THAT BECAUSE YOU'RE NOW FINE
TUNING WITH A MUCH SMALLER
CORPUS OF DATA.
WE'RE DIVERTING OURSELVES FROM
THE LARGER QUESTION THAT GRACE
ASKED.
>> MAY I ADD A COMMENT?
>> GO AHEAD.
>> I CAN THINK OF SEVERAL
DIFFERENT THINGS THAT WOULD BE
GOOD EXAMPLES OF COLLABORATION
ACROSS DIFFERENT DISCIPLINES
THAT WOULD BENEFIT NEUROAI.
ONE OBVIOUS TO ME IS NSF A.I.
INSTITUTE, ESPECIALLY THE ARNI
ARTIFICIAL AND NATURAL
INTELLIGENCE -- DID I GET THAT
RIGHT?
THERE'S ALSO AT MY HOME AGENCY,
FOR EXAMPLE, WE HAVE SCIENCE OF
SECURITY.
IT'S A COLLABORATION ACROSS
DIFFERENT LABS THAT INCLUDE
INDUSTRY, ACADEMIA, ALSO UNDER
THE WHITE HOUSE OFFICE OF
SCIENCE TECHNOLOGY POLICY
THERE'S THE NITRD PROGRAM, STANDS
FOR NETWORKING INFORMATION
TECHNOLOGY RESEARCH AND
DEVELOPMENT.
SO THAT HELPS WITH
COLLABORATIONS ACROSS ALL OF
FEDERAL GOVERNMENT, NOT JUST
DEPARTMENT OF DEFENSE OR NIH,
IT'S ACROSS THE FEDERAL
GOVERNMENT.
THAT'S ANOTHER EXAMPLE.
THERE'S LOTS OF OPTIONS.
>> IS THIS ADVERTISED?
>> SO IT IS ADVERTISED.
NITRD, PEOPLE PRONOUNCE THE
ACRONYM DIFFERENT WAYS,
NITRD. IT'S NITRD.GOV
OR JUST TO THE WHITE HOUSE
>> I WANTED TO ADD GRACE HWANG
HAD THE BRAID EFRI WHEN SHE WAS AT
THE NSF A FEW YEAR AO, THE OF TWO US
PARTNERED AND SO WE WORKED TOGETHER
TO SHAPE SOLICITATION AND THEN GO
THROUGH THE SELECTION PROCESS AS
WELL.
I ALSO WORKED WITH CHOU.
WE'RE ALWAYS LOOKING FOR
OPPORTUNITIES TO WORK ACROSS THE
GOVERNMENT.
IT HAS A LOT OF BENEFITS IN
TERMS OF JUST YOU HAVE MORE
PEOPLE THINKING ABOUT A PROBLEM.
BUT ALSO YOU'RE AWARE OF WHAT'S
GOING ON ACROSS THESE AGENCIES.
WE TRY TO KEEP UP.
BUT IT'S NOT EASY.
AND SO WE WANT TO MAKE SURE
WE'RE NOT DUPLICATING EFFORTS
AND THINGS WE DO CAN BENEFIT
EACH OTHER.
>> AS A FOLLOW-UP QUESTION, HOW
DIFFICULT IS IT TO DO THAT KIND
OF INTERAGENCY COOPERATION?
>> SO TALKING IS EASY.
EXCHANGING MONEY IS HARDER
BECAUSE THERE ARE AGREEMENTS
THAT HAVE TO BE PUT IN PLACE.
IT'S NOT EVEN -- JUST BECAUSE
YOU'RE BOTH GOVERNMENT AGENCIES
YOU CAN EXCHANGE THINGS, BUT YOU
HAVE TO MEMORANDA AND THOSE HAVE
TO BE APPROVED.
AND IT CAN BE TRICKY.
BUT IT'S POSSIBLE TO DO.
BUT, YEAH, SO MONEY IS ONE WAY,
BUT EXCHANGING IDEAS, THERE'S
VARIOUS WAYS.
>> I ALSO HAVE A PERSONAL
EXAMPLE, WHICH IS IT USED TO BE
THE CASE THAT NIH STUDY SECTIONS
WOULD THROW OUT ANY
COMPUTATIONAL GRANTS BECAUSE
THEY COULDN'T JUDGE IT.
BIOLOGISTS.
BUT THE NSF WAS ASKED TO DO THE
REVIEWS.
AND IT WAS A CRCNS, MUST HAVE
BEEN SOME AGREEMENT, BUT THE
BEAUTY IS THEY HAVE
MATHEMATICIANS TO GO THROUGH THE
PROPOSAL, PICK OUT ONES THE
HIGHEST RANK, SEND THEM BACK TO
NIH AND THOSE WERE FUNDED.
IT WAS REALLY A GREAT PROGRAM.
JUMP STARTED OUR WHOLE FIELD,
COMPUTATIONAL SCIENCE.
STEVEN HAS BEEN STANDING THERE.
>> YES, GRACE WAS TRYING TO GO
TO ME FIVE MINUTES AGO.
CRCNS IS STILL ONGOING.
YOU'RE SPEAKING LIGHT OF LIKE IT
DOESN'T EXIST.
NSF IS STILL RUNNING THE REVIEW,
WORKING CLOSELY WITH NIH, IT'S A
FANTASTIC PROGRAM.
IF YOU CAN FIND KEN WHANG IN THE
AUDIENCE, YEAH, HE JUST WAVED.
>> HE STOOD UP.
>> SO I WANTED TO TALK ABOUT
THAT WE'RE ALWAYS LOOKING FOR
NEW WAYS TO WORK WITH OUR OTHER
FEDERAL PARTNERS.
HAL BROUGHT UP REALLY WELL HOW
DIFFICULT THAT CAN BE, TO HAVE
MONEY CHANGING HANDS, OR TO WORK
TOGETHER.
WE HAVE MODELS LIKE CRCNS.
WE HAVE NEW COLLABORATIONS WITH
NIH WHERE WE'RE THINKING ABOUT
WELL, HOW CAN IT BE THAT NSF
COULD MAYBE DO ONE REVIEW SO
THERE'S NOT DOUBLE JEOPARDY AND
WE COULD HAVE A PIPELINE FROM
INITIAL NSF FUNDING FOR A COUPLE
YEARS WITH A SORT OF INTERIM
REVIEW BY NIH, NOT BY A PANEL,
ADDITIONAL TWO YEARS OF FUNDING
BY NIH, LEVERAGING UNIQUE
MISSIONS, WHAT WE'RE LOOKING TO
SUPPORT TO SUPPORT SOMETHING
BIGGER THAN WHAT WE COULD DO
ALONE.
SO YEAH, NOT BE A DOUBLE TAX,
DOUBLE REVIEWED.
YEAH, NO, YOU KNOW, SO THE THING
I WANT TO SAY THAT WE WANT TO
HEAR FROM YOU.
THAT CAME FROM LISTENING TO THE
SYNTHETIC BIOLOGY COMMUNITY,
COMING TO US AND SAYING, HEY,
WHY ISN'T THERE A MECHANISM LIKE
THIS?
SO TELL US.
PLEASE, EVERYBODY IN THE
AUDIENCE, WHAT ARE YOUR ACTUAL
NEEDS IN TERMS OF FUNDING?
WE'LL TRY TO FIGURE OUT A WAY.
>> STEPHANIE HAS BEEN WAITING.
>> I WANTED TO ADD ONE POINT
ABOUT CRCNS, THERE'S ALSO
INTERNATIONAL AGENCIES ALSO
PARTNERS SO IN ADDITION TO OUR
FEDERAL AGENCIES WE DO HAVE
COLLABORATION WITH THOSE
INTERNATIONAL PARTNERS WHICH
GIVEN THE IMPORTANCE OF THIS
TOPIC IT'S HELPFUL TO GET THOSE
OPINIONS.
>> FROM THE PRIVATE FOUNDATION
PERSPECTIVE, THE SIMONS
FOUNDATION, IT'S MOSTLY
ASTROPHYSICS BUT WE'VE PUT OUT
JOINT CALLS WITH NSF,
SPECIFICALLY NSF AND SIMONS
FOUNDATION LAUNCHED TWO A.I.
INSTITUTE TO HELP ASTRONOMERS
UNDERSTAND DATA, WE HAVE BLUE-
PRINTS BETWEEN THE TWO AGENCIES.
AND WE COULD COLLABORATE WITH OTHER
FOUNDATIONS ON SIMILAR CAUSE AND
USE THOSE MECHANISMS AS WELL.
>> GREAT.
>> SO, WE HAVE STANDARDIZATION
FOR DATA EFFICIENCY FOR TESTBEDS,
THAT'S GREAT. WE HAVE THOSE
SHARED HPC RESOURCES
ACROSS THE DOD.
BUT IF YOU HAVE IDEAS WHAT OTHER
KINDS OF TESTBEDS WE NEED, HOW TO
MAKE THOSE AVAILABLE FOR
RESEARCH.
>> CHRISTINE.
>> TWO THINGS.
I WANTED TO MENTION DARPA SINCE
I'M REPRESENTING DARPA TODAY.
DARPA, MY EXPERIENCE, PROBABLY
THE EXPERIENCE OF PEOPLE IN THE
ROOM, IT'S COLLABORATIVE,
MULTIDISCIPLINARY PROGRAMS THEY
PULL TOGETHER.
BUT ONE I WANTED TO MENTION WAS
THEIR A.I. CYBER CHALLENGE THEY
HAD RECENTLY.
SO THEY BROUGHT TOGETHER THE BIG
FOUNDATION COMPANIES LIKE
GOOGLE, ANTHROPIC, OPENAI,
THEY SUPPORTED THIS CHALLENGE.
SEMIFINAL COMPETITION AT DEVCON
THIS YEAR, FINAL COMPETITION
NEXT YEAR, BUT IT'S A REALLY
GOOD EXAMPLE OF DARPA WORKING
TOGETHER WITH ARPA-H AND ALSO
INDUSTRY PARTNERS.
SO THAT'S ONE REALLY GOOD
EXAMPLE OF A DARPA CHALLENGE
PROBLEM THAT ENDED UP BRINGING
IN ARPA-H TO USE A.I. TO DETECT
AND FIX SOFTWARE VULNERABILITIES
IN CRITICAL SOFTWARE.
>> COMPETITION AND STANDARDS.
>> RIGHT.
>> AND TESTBEDS.
SO, I THINK THIS ILLUSTRATES
THERE'S A LOT OF OPPORTUNITIES,
A LOT OF THINGS WE PROBABLY
HAVEN'T EVEN COVERED YET, BUT I
THINK WE COULD PROBABLY BENEFIT
IF THERE WERE MORE COOPERATION.
NOW WE'RE GOING TO GO OVER TO
OUR ZOOM WEBINAR QUESTIONS.
I THINK THAT JOE IS GOING TO --
IS GOING TO SHOOT THE BALL UP.
>> HINTING WE GO SHOULD OPEN
UP TO AUDIENCE Q&A, EITHER IN
THE ROOM OR ZOOM.
UNLESS --
>> I DON'T SEE ANYBODY OUT THERE
ASKING QUESTIONS.
WE HAVE ONE A VOLUNTEER.
>> ALL RIGHT.
I'LL START OFF.
SO, ONE OF THE THINGS ABOUT THIS
MEETING THAT WILL CONTINUES TO
BE MORE CLEAR AS WE GO FORWARD
IS THAT THIS IS A VERY -- GOING
TO BE VERY INTERDISCIPLINARY,
WE'RE TALKING ABOUT BIOLOGISTS,
NEUROSCIENTISTS, COMPUTER
SCIENTISTS, MATHEMATICIANS, YOU
NAME IT.
ONE OF THE CHALLENGES OFTEN WITH
FUNDING IS THAT IT'S RELATIVELY
EASY, IN THE FUNDING SENSE, TO
GET MONEY IN A SWIM LANE.
BUT WHEN YOU CROSS SWIM LANES,
WHERE YOU NEED TO BRIDGE MONEY
FOR COMPUTER SCIENTISTS AND
NEUROSCIENTISTS, SAY, IT BECOMES
HARD.
UNLESS THERE'S A SPECIAL CALL.
BUT THOSE SPECIAL CALLS USUALLY
HAVE LIKE A TIME FRAME LIKE
THREE YEARS OR SOMETHING.
YET WE'RE TRYING TO BUILD THESE
COLLABORATIONS, IF THEY WORK,
LAST DECADES, HOW DO WE GET PAST
THIS SILOED SWIM LANE SHORT FUSED
FUNDING CYCLES OPPOSITE WHAT IS
NEEDED FOR BIG INTERDISCIPLINARY
EFFORTS.
>> GOOD QUESTION, ALWAYS A
PROBLEM.
>> PRIVATE SECTOR.
TURN IT INTO A PRODUCT
>> FOR PROFIT.
>> WITHIN THE DOD FIRST THING
OBVIOUS IS MULTIDISCIPLINARY
UNIVERSITY RESEARCH INITIATIVE.
SO THAT HAS TOPICS THAT CHANGE
EVERY YEAR.
THE PROGRAM OFFICERS ACROSS THE
THREE SERVICES COMPETE FOR
TOPICS.
BUT ONE OF THE KEY ASPECTS IS
THOSE PROJECTS WHICH ARE UP TO
$1.5 MILLION A YEAR FOR UP TO
FIVE YEARS, THEY ARE FAIRLY BIG,
BUT THEY HAVE TO BE FOR BASIC
RESEARCH, IT HAS TO BE
MULTIDISCIPLINARY
IN TERMS OF THE TEAM MAKEUP,
APPROACH, SO WE SPECIFICALLY
LOOK FOR THAT.
OUTSIDE OF THAT WE'RE ALWAYS
LOOKING FOR MULTIDISCIPLINARY
ACTIVITIES, AND THERE'S -- WE
DON'T HAVE THOSE SORTS OF SILOS
WITHIN OUR PROGRAMS.
NEUROSCIENCE IS A
MULTIDISCIPLINARY FIELD.
I WOULD SAY WITHIN MY PROGRAM I
HAVE PEOPLE REPRESENTING VARIOUS
DISCIPLINES.
BUT SOMETIMES YOU'LL FIND THERE
ARE EFFORTS THAT -- WHERE THE
IDEAS WILL SPAN MULTIPLE
PROGRAMS WITHIN AN AGENCY, AND
SO I'M ABLE TO COLLABORATE
FAIRLY EASILY WITH MY COLLEAGUES
BECAUSE WE DON'T HAVE MEMORANDA
IN PLACE, IF THINGS FIT MULTIPLE
PORTFOLIOS WE CAN CO-FUND ACROSS
PROGRAMS.
I DON'T THINK THAT'S AN ISSUE.
WE'RE TRYING TO ENCOURAGE
MULTIDISCIPLINARY ACTIVITIES.
I DON'T THINK WE HAVE BARRIERS
TO THAT.
>> NSF HAS THE A.I. CENTERS
PROGRAM, MEANT TO BE LONGER
LASTING LIKE FIVE YEARS
RENEWABLE, IS THAT RIGHT?
YOU'RE RIGHT.
IT IS POSSIBLE.
WITH CENTERS PROGRAMS TO HAVE
SOMETHING MORE EXTENDED BUT
THERE'S STILL AN ISSUE, I THINK,
VERY FEW PEOPLE CAN TAKE
ADVANTAGE OF THAT, RIGHT?
IT'S NOT OPEN TO EVERYBODY.
>> THE MURIES ARE LIMITED, FOR
U.S. INSTITUTIONS.
TRADITIONAL GRANTS ARE OPEN, WE
FUND RESEARCH THAT'S OPEN TO
ANYONE.
>> NO DEADLINES, NO DATA
REQUIREMENTS, TRYING TO BE
INTERDISCIPLINARY.
>> THESE PROGRAMS ARE ONGOING.
SO IT'S NOT LIKE THEY ARE --
VERY GOOD, OKAY.
NEXT QUESTION.
>> SO PIGGYBACKING ON THAT
THERE'S TWO FACETS OF
INTERDISCIPLINARY, RIGHT?
THERE'S INTERDISCIPLINARY WORK
AND THERE'S ALSO
INTERDISCIPLINARY OF TRAINING,
RIGHT?
SO ONE WAY TO DEAL WITH THIS IS
TRAINING NEW GENERATIONS OF
ENGINEERS AND SCIENTISTS THAT
ARE COMFORTABLE IN BOTH WORLDS.
DO ANY FUNDERS HAVE INSIGHTS OR
PERSPECTIVES ON HOW TO
INCENTIVIZE FUNDING AND TRAINING
OF INTERDISCIPLINARY RESEARCHERS
BETWEEN THESE FIELDS?
>> I THINK THE BRAIN INITIATIVE
IS DOING A PRETTY GOOD JOB.
>> IT'S A REALLY IMPORTANT
POINT.
WE'RE AT A PLACE READY TO TRAIN
THE NEXT GENERATION OF
NEUROSCIENTISTS THROUGH
NEUROAI.
YOU KNOW, EARLY CAREER SCHOLARS
ARE NOT AFRAID OF COMPUTERS.
THEY ARE NOT AFRAID OF DATA, TO
WORK WITH ANIMALS.
IT'S LIKE THREE IN ONE GOING
FORWARD.
>> I WAS TALKING TO PEOPLE AT
LUNCH, THEY NORMALLY TEACH A
COURSE, THEY GET A COUPLE DOZEN
STUDENTS .
AS SOON AS A.I. IS THROWN INTO
THE POT THEY HAD 50, 100.
THE YOUNG GENERATION IS REALLY
EXCITED ABOUT THIS NEUROAI.
THEY ARE READY. THEY WANT TO BE
INTERDISCIPLINARY. SO I THINK
THAT'S WHERE THE FUTURE IS.
>> THE OVERWHELMING RESPONSE FOR
EARLY CAREER SCHOLAR PROGRAM THAT
JESSICA MOLLICK AND COURTNEY PINARD
PUT TOGETHER SPEAKS TO THAT.
YOU'LL HEAR MORE TOMORROW.
>> OKAY.
ANOTHER QUESTION?
WAIT, DO YOU WANT TO SAY
SOMETHING?
>> REAL QUICK.
ONE OF THE THINGS I MENTIONED
FOR THE DSO, A PROGRAM CALLED
INNOVATION FELLOWSSHIP PROGRAM, IT
IS A 2-YEAR ASSIGNMENT FOR PEOPLE
WITHIN -- WHO HAVE THEIR
Ph.D., LIKE A POSTDOC
POSITION, BUT IT'S A GREAT
OPPORTUNITY FOR PEOPLE TO GET
EXPERIENCE OVERSEEING PORTFOLIO
OF HIGH RISK, HIGH REWARD
RESEARCH, SO I WANT TO PUT A
PLUG, POSTCARD ON THE TABLE DESCRIBE
A CALL FOR THEIR - IT'S CALLED
INSPIRE, ACRONYM, BUT ONE OF THE
INNOVATION FELLOWS IS HERE
TODAY.
HER NAME IS ALISSA LING.
I DON'T SEE HER BECAUSE THE
LIGHTS ARE BLINDING ME.
THERE SHE IS.
SHE WOULD BE HAPPY TO TELL YOU
MORE ABOUT THAT PROGRAM AND
TOPIC.
THAT'S A REALLY GOOD EXAMPLE OF
CROSS-TRAINING.
AND MY HOME AGENCY WE REALLY
VALUE GETTING THE INTELLIGENCE
ANALYST PERSPECTIVE, WOULD HAVE
CROSS-TRAINING WITHIN OUR OWN
AGENCY BUT I THINK THAT CONCEPT
OF THE RESEARCHERS GETTING
INVOLVED IN END USER SPACE, VICE
VERSA, IS REALLY VALUABLE.
>> I WANT TO MENTION THE ARMY
HAS A PROGRAM, A FUNDING MECHANISM,
$60,000 FOR NINE MONTHS. POSTDOC
GETS A CRAZY IDEA, NO PILOT DATA,
HIGH RISK, IF IT HAS A CHANCE TO
OPEN A NEW FIELD, GO FOR IT.
>> IT'S A POSTDOC.
>> YES, A POSTDOC OR FACULTY
COULD APPLY FOR IT HIMSELF.
>> I CAN BRING IN A QUESTION
FROM ZOOM.
WELL, NOTING THAT THE R&D BUDGET
OF BIG TECH COMPANIES ARE SUPER
HIGH, WHAT DO WE THINK ABOUT
CREATING SMALLER SCALE COMPANIES
FOR INDIVIDUAL DOMAINS AND
APPLICATIONS LIKE BCIs, BASED
ON NEUROAI MODELS OR THEORIES,
AND HOW DO YOU SUPPORT DIVERSE
ECOSYSTEM OF SMALLER COMPANIES? DO
THEY GO TO VC OR IS THERE PUBLIC
SECTOR SUPPORT OR OTHER PRIVATE
SECTOR MECHANISMS?
HOW DOES THIS FALL OUT AGAINST
THE DIFFERENT LINES AND SECTORS
OF SUPPORT ACROSS AGENCIES?
>> OKAY, I COULD TALK TO THE
PRIVATE SECTOR ASPECT.
SO, PART OF OPENGPU FOUNDATION,
IS A SPINOFF OPENGPU VENTURES,
WHICH IS SPECIFICALLY FOCUSED AT
FUNDING STARTUPS DOING AND
SUPPORTING OPEN STANDARDS WORK.
ONE OF THE CHALLENGES AS A SMALL
STARTUP YOU CAN FUND MANY POINT
EFFORTS BUT THEY TEND NOT TO BE
EXTENSIBLE, IT'S DIFFICULT TO
STAND ON SHOULDERS WHEN EVERY
ONE IS AT KNEE LEVEL.
PART OF THE EFFORT I THINK TO
GROW THIS HAS TO INVOLVE SOME
LEVEL OF, YEAH, STARTUPS CAN
PROLIFERATE WITH SOME POINT
SOLUTION BUT IT'S GOT TO BE
SOMETHING OPEN, REUSABLE,
LEVERAGABLE, EXTENSIBLE INTO
FUTURE EFFORTS.
>> I WOULD POINT TO THE SBIR AND
STTR PROGRAMS, SMALL BUSINESS
INNOVATIVE RESEARCH AND THESE
ARE AWARDS, CONTRACTS, GIVEN TO
SMALL BUSINESSES UNDER CERTAIN
CONDITIONS TO INVESTIGATE
TOPICS OF INTEREST.
STTRs ARE LIKE SBIRs, BUT IT'S A
PARTNERSHIP BETWEEN A SMALL
BUSINESS AND A UNIVERSITY.
AND SO THEY -- THESE TYPES OF
OPPORTUNITIES MAY BE OF INTEREST
TO THIS COMMUNITY.
>> MY QUESTION WAS ON QUESTION
2, WHAT MULTI-AGENCIES EFFORTS
ALREADY EXIST. AND
IT HAD MORE TO DO WITH
ESTABLISHING STANDARDS BECAUSE
WE'RE ALL HAVING TO LIKE PROCESS
REFORMAT ALL THIS DIFFERENT
DATA.
AND I WAS WONDERING IF AN
EXAMPLE LIKE THE APACHE
FOUNDATION, WHERE MULTIPLE
COMPANIES ARE COMING TOGETHER TO
STANDARDIZE DATABASE FORMATS,
FILE TYPES, WHERE THEY VOTE,
THEY ALL CONTRIBUTE TO A LARGER
BODY OF WORK WOULD BE A GOOD
EXAMPLE.
>> I DIDN'T QUITE CATCH THE
QUESTION IN THERE.
>> LIKE IS THE WAY THAT OPERATES
AN EXAMPLE OF A MODEL FOR
COLLABORATION IN NEUROAI?
>> I DELIBERATELY PICKED OUT THE
USB EXAMPLE BEFORE BECAUSE JUST
LIKE BLUETOOTH THERE ARE ZERO
GOVERNMENT AGENCIES INVOLVED IN
THE USB STANDARD.
AND IN BLUETOOTH.
BUT IT'S EVERYWHERE, RIGHT?
SO I THINK THE CORE POINT, YOU
KNOW, ESPECIALLY WITH THE GROWTH
IN INDUSTRY, WITH ABILITY TO
INCREASE, YOU KNOW, TO BRAD'S
QUESTION, MORE PEOPLE ONLINE,
AND TO TRAIN THE NEXT GENERATION
PART HAS TO BE, YOU KNOW, WHAT
ARE THE STANDARDS, WHAT COULD
YOU GIVE A TEENAGER AS A BOOK TO
GET HIM EXCITED, TO GO TO
UNIVERSITY, TO WANT TO DO AND
APPLY FOR THE GRANT.
THAT'S GREAT BUT WE HAVE TO GET
IT MUCH EARLIER.
THE WAY THAT'S HAPPENED IN THE
INDUSTRY IS THROUGH CREATING
OPEN STANDARDS.
YEAH, IT'S ABSOLUTELY ESSENTIAL.
>> THANK YOU.
>> OH, SORRY.
I FORGOT TO DO THE PLUG.
BY THE WAY, IT'S CALLED OPENGPU
BECAUSE IT IS AN OPEN STANDARD.
THAT'S WHAT I REPRESENT HERE
TODAY.
>> SINCE WE'RE PLUGGING THINGS.
I HAVE A NEW BOOK.
THIS WAS LAUNCHED LAST MONTH.
ChatGPT AND THE FUTURE OF
A.I., IT'S WRITTEN FOR YOU.
ALL THESE ISSUES THAT HAVE COME
UP ARE ONES THAT I HAVE TALKED
ABOUT.
OKAY.
ON THIS SIDE?
>> JUST SORT OF AN OBSERVATION
AND PERHAPS SORT OF REQUEST
REACTION FROM PANELISTS.
THE FIELD OF A.I. GROWS
EXTREMELY FAST.
YOU KNOW, THERE ARE SORT OF NeRF
WAS THE BIG THING IN GRAPHICS. IT'S
NO LONGER THE BIG THING, IT'S
CALLED SPLATTING.
PAPERS PUBLISHED A YEAR AGO HAVE
10,000 CITATIONS AND SO ON.
THE REASON FOR THAT IS
CODE AND DATA ARE EASILY
AVAILABLE AND PEOPLE CAN ITERATE
ON IT.
ON THE OTHER HAND NEUROSCIENCE
HAS A DIFFERENT CULTURE, PEOPLE
SPEND YEARS AND CAREFUL
EXPERIMENTATION TO GET DATA, AND
THEY WANT TO EXTRACT VALUE OUT
OF THE DATA AS THEY SHOULD FOR
THE AMOUNT OF EFFORT THEY HAVE
PUT IN.
HOW DO WE BRIDGE THIS CULTURAL
DIVIDE, IF YOU WOULD?
>> ANYONE WANT TO BRIDGE THE
DIVIDE?
>> BETWEEN WHAT AND WHAT?
>> BETWEEN THE DESIRE FOR
RESEARCHER TO MAXIMIZE THEIR
BENEFIT FROM THE PAINSTAKINGLY
COLLECTED DATA ON THE ONE SIDE,
AND THE DESIRE OF FAST
DEVELOPING FIELD TO QUICKLY HAVE
MANY, MANY PEOPLE EXTRACT THE
JUICE FROM THE NEW CODE AND DATA.
>> I THINK IN MANY WAYS THE
NEUROSCIENCE FIELD HAS WEIGHED
IN ON THAT WITH THE PUSH TO OPEN
DATA AND OPEN SCIENCE, RIGHT?
I THINK MOST OF NEUROSCIENCE IS
MOVING IN THAT DIRECTION WHICH I
THINK LEANS TOWARDS LESS SORT OF
HOARDING DATA, EXTRACTING ALL
THE JUICE AS YOU SAID, VERSUS
GETTING IT OUT THERE FOR THE
WORLD TO ALLOW OTHERS TO
INTERACT WITH THE DATA.
SO I DON'T KNOW THAT THERE'S
MUCH MORE THAT WE CAN DO AT
LEAST FROM THE FOUNDATION
PERSPECTIVE.
WE ALL HAVE DIFFERENT IDEAS.
I DON'T KNOW.
NEUROSCIENCE HAS ALREADY MOVED
IN THAT DIRECTION.
>> THAT WAS NOT THE TRADITION,
BEFORE THE BRAIN INITIATIVE.
AND I KNOW BECAUSE I WROTE THAT
SECTION.
AND THE REASON I WROTE THAT
SECTION, BRAIN 2025, WAS THAT
TRADITIONALLY, YOU KNOW, YOU DO
AN EXPERIMENT IN YOUR LAB,
COLLECT THE DATA, IT NEVER
LEAVES THE LAB.
NOW THERE'S SO MUCH DATA, OTHER
PEOPLE SHOULD HAVE AN
OPPORTUNITY TO ANALYZE, OTHER
TOOLS AND TECHNIQUES.
NOW THAT'S BECOME THE STANDARD.
IT'S REALLY CHANGED COMPLETELY;
YOU'RE ABSOLUTELY RIGHT ABOUT
THAT.
>> A QUESTION FROM THE VIDEOCAST
MONITOR, DANA.
>> YES.
THERE'S A QUESTION THAT IS
ASKING IF THERE ARE ANY GRANTS
THAT STUDENTS, PhDs CAN
APPLY FOR IN ORDER TO GAIN
ACCESS TO NEUROMORPHIC HARDWARE
IN ORDER TO TRAIN THESE NEUROAI
MODEL.
>> MORE GENERALLY, WHAT IS THE
LANDSCAPE SUPPORT LOOKING LIKE
FOR BREAKING BARRIERS TO ACCESS
TECHNOLOGY?
CERTAINLY IN NEUROMORPHIC
COMPUTING WHICH WE'LL HEAR MORE
ABOUT TOMORROW, PEOPLE HAVE
TROUBLE ACCESSING TESTBEDS AND
DEVICES IN ORDER TO TRY OUT NEW
DESIGNS SO THAT'S GOING TO BE AN
IMPORTANT PART OF TRAINING AS
WELL AS EARLY STAGE DEVELOPMENT
OF THE SCIENTIFIC MODELS AND AS
TECHNOLOGICAL PRODUCTS.
SO IT WOULD BE GREAT TO HEAR
FROM MANY OF YOU WITH NEUROMORPHIC
PROGRAMS WHAT THAT LOOKS LIKE
GETTING THINGS OUT AT THESE
EARLY STAGES FOR STUDENTS AND
OTHERS.
>> THERE'S A PRECEDENT IN
SEMICONDUCTORS. SO MOSIS WAS
SET UP, THESE BIG COMPANIES, VERY
EXPENSIVE, BUT CARVER MEAD WAS
INVOLVED IN GETTING THEM TO TAKE
ACADEMIC CHIPS, A WAY FOR
TEACHING STUDENTS HOW TO DESIGN
CHIPS, THE COMPANIES BENEFIT
BECAUSE NOW THEY HAVE PEOPLE
THAT WERE COMING IN, ALREADY
TRAINED, HOW TO DO THAT.
MAYBE WE COULD DO SOMETHING
ALONG THOSE LINES.
>> SO, I'M GOING TO REFLECT ON
MY EXPERIENCE AT NVIDIA. THERE
WERE MORE THAN 10 YEARS OF PATH-
FINDING AND SEEDING RESEARCHERS
IN INDUSTRY, UNIVERSITIES WITH GPUs
AND CUDA. JUST GO FOR IT. GIVE
US BACK THE IP LICENSE, BUT GO FOR
IT, DO WHATEVER YOU WANT.
THE FACT AS A RESEARCHER YOU
COULD BUY A GPU FROM BEST BUY
AND DOWNLOAD THE CUDA LIBRARIES,
THE LATEST ONES AND JUST GO.
I THINK THE QUESTION WAS HOW TO
GET ACCESS TO NEUROMORPHIC HARDWARE
NOT HOW TO DESIGN A CHIP.
SURE, WE WOULD LOVE HAVE MORE
PEOPLE AWARE OF SEMICONDUCTORS
BUT THE CORE PROBLEM IS GETTING
PROLIFIC ACCESS TO NEXTGEN HARDWARE
IN A.I. TO GET OFF THE GROUND.
I DON'T HAVE AN ANSWER TO HOW TO
GET TO NEUROMORPHIC HARDWARE
TODAY.
THAT'S KIND OF WHY WE DO OPENGPU
VENTURES, BECAUSE WE WANT TO
CREATE PROLIFIC HARDWARE, AS
CHEAP AS GPU.
RIGHT?
SO MY ANSWER TO THE QUESTION
WOULD BE I CAN'T HELP YOU GET
ACCESS TO SPINNAKER, I
UNDERSTAND IT'S IN EUROPE AND
VERY EXPENSIVE, CAN'T HELP YOU
GET ACCESS TO SANDIA LABS' LOIHI
INSTALLATION, BUT MAYBE WE COULD
TALK ABOUT HOW TO MAKE PROLIFIC
HARDWARE THAT'S EVERYWHERE AND
EASILY ACCESSIBLE AND CHEAP.
>> YEAH, I THINK ABSOLUTELY.
OKAY.
NEXT QUESTION HERE?
>> JUST A PRACTICAL QUESTION
SINCE WE HAVE INDIVIDUALS FROM
FUNDING AGENCIES, WHEN WE DO
MULTIDISCIPLINARY RESEARCH IT
REQUIRES A MULTITUDE OF SKILLS,
AND MANY THINGS WE DO IN THE
FIELD HAVE BIOLOGISTS, PEOPLE
THAT ARE EITHER EXPERTS IN
COMPUTER SCIENCE OR MATHEMATICS
OR PHYSICS AND CLINICIANS.
BUT THE WAY EACH OF US ARE
REINFORCED WHEN WE NEED TO GO UP
FOR TENURE REVIEW, THESE KINDS
OF THINGS, DEPENDS ON HAVING
SPECIFIC LABELS FROM THE FUNDING
AGENCIES, NAMELY ARE YOU A P.I.,
CO-P.I.
WOULD THERE BE A ROLE FOR GRANT
CALLS FOR INDIVIDUALS WHERE THERE'S
MULTI-P.I. STRUCTURE BUILT IN?
PI CLINICIAN, ENGINEER, BIOLOGIST,
THAT KIND OF THING IS JUST GOING
TO BE HARDBAKED INTO ANY CALLS?
DOES THAT MAKE SENSE?
>> NSF, NIH?
>> I THINK NIH HAS SOMETHING
LIKE THAT ON RM1, IS KAREN DAVID
IN THE AUDIENCE? MAYBE YOU
COULD SPEAK TO THAT. SHE PIONEERED
THE COLLABORATIVE OPPORTUNITIES
FOR MULTI-DISCIPLINARY, BOLD,
AND INNOVATIVE NEUROSCIENCE.
>> THE INSPIRATION WAS OUR BRAIN
INITIATIVE TEAM SCIENCE PROGRAM,
THE BRAIN CIRCUITS PORTFOLIO.
WE HAVE DIFFERENT SCALES AND
SIZES, ONE IS FOR INDIVIDUAL LABS,
THE OTHER SIDE IS PURELY FOR
TEAM SCIENCE WHERE WE REQUIRE AT
LEAST THREE UP TO SIX
MULTI-P.I.s, AND BY DESIGN THE
SCIENTIST WOULD NEED AN
EXPERIMENTALIST, A COMPUTATIONAL
NEUROBIOLOGIST, AND OTHER
EXPERTISE THAT'S NEEDED TO
ANSWER THE PROPOSED QUESTIONS OR
PROJECT IN MIND.
SO THE SCIENCE NECESSITATES THE
CONVERGENCE OF THE EXPERTISE
THAT'S NEEDED TO SOLVE THAT.
AND THAT SERVES AS INSPIRATION
FOR NINDS TEAM SCIENCE PROGRAM,
WHICH IS THE RM1 COMBINE
PROGRAM, MORE GENERAL, NOT
PRESCRIBED WHAT TYPE OF
COLLABORATION OR TYPE OF
SCIENCE, IT'S MORE WHATEVER IS
IN THE NINDS SPACE.
THAT ONE WE REQUIRE SIMILAR,
THREE TO SIX P.I.s, HAS TO BE
MULTIDISCIPLINARY OR
TRANSDISCIPLINARY, CONVERGENC OF
EXPERTISE IS NECESSARY TO SOLVE
THE PROPOSED QUESTION.
TEAM SCIENCE BY DESIGN, WE
REQUIRE THEM TO THINK ABOUT
GLUES THAT SERVE TO BRIDGE
EXPERTISE AT LEVEL OF SCIENCE,
THEN AT THE LEVEL OF THE PEOPLE,
LEVEL OF THE APPROACHES.
I THINK SOCIAL ASPECTS OF THE
TEAM MANAGEMENT PLAN IS AS
IMPORTANT AS THE SCIENCE BEING
PROPOSED.
>> LET'S GO TO THE RIGHT SIDE
HERE.
WE HAVE TO FINISH IN FIVE
MINUTES.
GO AHEAD.
>> THANK YOU.
I FULLY UNDERSTAND FOR THE
NEUROMORPHIC COMPUTING AND
NEUROAI COMMUNITY, MANY PEOPLE
FROM HARDWARE, SOFTWARE AND BIOLOGY
I'M WONDERING FOR THE AGENCY
THAT, FOR EXAMPLE, FOR THE NSF
TO BUILD A NEW PROGRAM SPECIFIC
TO NEUROAI, AND MAYBE FOR
EXAMPLE LIKE DIFFERENT THINGS
LIKE THAT HAVE THEIR OWN
FOUNDATION, SO IF WE ARE ALL
INTERESTED IN THESE KIND OF
NEUROAI OR NEUROMORPHIC APPROACH,
POSSIBLE ALL AGENCIES TO WORK
WITH THE COMMUNITY TO BUILD KIND
OF THE FOUNDATION FOR THIS.
THANK YOU.
>> THAT WOULD BE WONDERFUL.
BUT UNFORTUNATELY IT TAKES A LOT
OF RESOURCES, AND A LOT OF
POLITICS IN ORDER TO GET
SOMETHING LIKE THAT STARTED.
THE BIGGER THE THING YOU TRY TO
START, THE LONGER IT'S GOING
TO TAKE.
WE'RE TRYING TO START HERE FROM
GRASSROOTS.
IN OTHER WORDS, WE HAVE -- IT IS
INTERESTING.
THERE SHOULD BE PROBABLY
SOMEWHERE A SPECIFIC PROGRAM FOR
NEUROAI.
THAT WOULD BE SOME AGENCY, MAYBE
FOUNDATION, SHOULD ACTUALLY TAKE
THAT ON SO THE KERNEL IS THERE
FOR -- YOU KNOW, MAYBE IT'S
ALREADY THERE, WE HAVEN'T
FERTILIZED IT YET.
>> COULD I JUMP IN?
YOU KNOW, GOING BACK TO
SOMETHING WHERE I PLEADED WITH
THE AUDIENCE WE NEED TO HEAR
WHAT YOU WANT, RIGHT?
EARLIER THERE WAS A TALK ABOUT
THESE SMALLER TARGETED PROGRAMS
THAT DON'T LAST FOREVER.
A LOT OF THOSE SMALLER TARGETED
PROGRAMS CAN LEAD TO BIGGER
THINGS.
THERE'S A PIPELINE, YOU AS A
COMMUNITY THINK THIS IS AN AREA
WORTH INVESTING IN, REACHING
OUT, TALKING TO PEOPLE, IN
FUNDING PROGRAMS, WHETHER IN THE
GOVERNMENT OR IN PRIVATE
FUNDERS, SAY THIS IS IMPORTANT.
THERE'S A PATH TO EVENTUALLY
GENERATING SOMETHING.
SO REACH OUT TO ALL OF US,
SAYING THIS IS IMPORTANT,
SOMETHING WE WANT, THINGS MOVE
SLOWLY BUT THEY DO MOVE.
>> SCAN THE QR CODE AND SEND US
YOUR THOUGHTS.
>> YES.
>> WE HAVE ONE MORE QUESTION IN
THE ROOM BEFORE WE HAVE TO WRAP
UP.
>> JULIE, COMING TO LONDON.
YOU'VE MENTIONED INTERDISCIPLINARY
AND INTERNATIONAL COLLABORATION.
THE BRAIN INITIATIVE PIVOTAL IN
DEVELOPING MY RESEARCH QUESTION
AND DOING WELL, MY QUESTION IS WHAT
SORT OF FUNDING YOU HAVE FOR AN
INTERNATIONAL SCHOLAR, AND WHAT
CAN YOU DO TO ACTUALLY HELP
PEOPLE ABROAD TO EXPAND RESEARCH
IF THEY ARE ACTUALLY DRAWING ON
SOME OF THE DATA THAT THE BRAIN
INITIATIVE IS ACTUALLY
DISSEMINATING.
THANK YOU.
>> WITHIN THE DEPARTMENT OF
DEFENSE, SO ALL THREE SERVICES,
ARMY, NAVY, AIR FORCE, HAVE
INTERNATIONAL OFFICES.
AND SO I THINK ALL THREE, YOU
TALKED ABOUT THINGS IN LONDON,
ALL THREE HAVE OFFICES IN
LONDON, BUT I KNOW FOR THE AIR
FORCE WE'VE GOT OFFICES IN TOKYO
SANTIAGO, CHILE, SAO PAULO, BRAZIL
MELBOURNE, AUSTRALIA. ARMY HAS
OWN LOCATIONS SOME OVERLAP.
SAME WITH ONR GLOBAL FOR THE NAVY.
SO, WE FUND SOME INTERNATIONAL
WORK FROM WHERE WE'RE BASED IN
THE U.S.
BUT ALSO THE INTERNATIONAL
PROGRAM OFFICERS ARE OUR EYES
AND EARS ABROAD, FOLLOWING
TRENDS, SCIENCE IN THEIR REGION,
HELPING TO INFORM US BUT THEY
ALSO HAVE THEIR OWN BUDGET SO
THEY ARE ABLE TO FUND WORK FROM
THERE BUT THEY CAN HELP DIRECT
YOU TO THE RIGHT PLACES AND MAKE
SURE THAT YOU'RE ABLE TO
NAVIGATE OUR PROCESSES.
>> THANK YOU.
>> SIMONS FOUNDATION FUNDS
INTERNATIONALLY BY DESIGN AND
MANY PRIVATE FOUNDATIONS DO AS
WELL.
>> SO TO FINISH, THE DEPARTMENT
OF ENERGY DOES IT HAVE
INTERNATIONAL PROGRAMS?
>> RELATED TO COMPUTATIONAL
NEUROSCIENCE, WE ARE PART OF CRCNS
WITH NSF, NIH AND DOE, SPAIN, GERMANY, ISRAEL, JAPAN, AND FRANCE.
>> ONE LAST THING IF I COULD.
WE'RE BY NATURE INTERNATIONAL,
OPENGPU IS TRADEMARKED
INTERNATIONAL.
WE BELIEVE INSIGHTS AND
DEVELOPMENT COMES FROM
EVERYWHERE.
>> THIS IS A GOOD WAY TO END.
THANKS TO THE PARTICIPANTS HERE
AND THE AUDIENCE FOR THE
QUESTIONS.
THANK YOU.
[APPLAUSE]
>> LET'S TRANSITION TO SESSION
2.
IT IS MY GREAT PLEASURE TO
INTRODUCE DR. KAREN
ROMMELFANGER, FOUNDER AND
DIRECTOR OF THE INSTITUTE OF
NEUROETHICS THINK AND DO TANK,
SHE'S ALSO A FOUNDING MEMBER OF
THE NIH BRAIN NEUROETHICS
WORKING GROUP, NEWG, SINCE 2015.
SHE WILL TEACH US ABOUT
NEUROETHICS TODAY AND WHAT SHE
HAS TO SAY ABOUT APPLY TO ALL
FOUR SESSIONS.
>> THANK YOU, GRACE.
THANK YOU FOR HAVING ME HERE
TODAY.
THANK YOU, GRACE AND JOE FOR
YOUR EXCELLENT ORGANIZATION AND
TO THE EXTERNAL ADVISORS AS
WELL, WITH THE BIG NIH TEAM THAT
MADE THIS HAPPEN.
I'LL THRILLED TO HAVE YOUR
ATTENTION.
WE'RE IN THE PART OF THE DAY
WHERE YOU NEED A LITTLE I.V.
DOSE OF CAFFEINE BUT I'LL TRY MY
BEST TO GET YOU THROUGH HERE.
OKAY.
SO, WHAT I'M HOPING YOU GET BY
THE END OF THIS IS THAT YOU
UNDERSTAND SOMETHING WE'VE HAD
AS A CORE VALUE IN THE
NEUROETHICS WORKING GROUP FROM
ITS INCEPTION.
THAT IS GOOD SCIENCE IS
ETHICAL NEUROSCIENCE.
THAT MEANS THAT ETHICS IS
INTEGRATED THROUGHOUT THE LIFE
CYCLE OF DESIGNING AN IDEA
IDEA, CARRYING OUT THE RESEARCH,
AND ALSO THROUGH DISSEMINATION
AND TRANSLATION OF FINDINGS.
SO I WON'T SHOW THE SAME
VIRTUOUS CYCLE THING EVERYONE
ELSE HAS SHOWN BUT IN A
DIFFERENT WAY, WHICH IS THE
REASON WHY THIS SPACE AND WHY
NEUROETHICS AS A FIELD EXISTS IS
BECAUSE THE BRAIN IS NOT JUST
BIOLOGICALLY SPECIAL.
IT'S CULTURALLY SPECIAL.
SO, AT LEAST FOR OVER 2500 YEARS
WE'VE THOUGHT OF THE BRAIN AS
THE PLACE FROM WHICH
INTELLIGENCE DERIVES, A PLACE
TODAY THAT'S THE HOME OF
PERSONAL IDENTITY, OF THOUGHTS,
EMOTIONS, HUMAN EXPERIENCE, OF
THE THINGS THAT WE PRIZE MOST IN
LIFE.
AND THE MOST ADVANCED TECHNOLOGY
THROUGHOUT HISTORY OF TIME HAS
ALWAYS BEEN COMPARED TO THE
BRAIN.
SO, ONE EXAMPLE, THE BRAIN WAS A
TELEGRAPH.
NOW IT'S A COMPUTER.
NOW THE TWIST IS THAT THE MOST
SOPHISTICATED TECHNOLOGY MIGHT
BE THE BRAIN AND COMPUTER NEEDS
TO BE MORE LIKE THE BRAIN.
THAT MEANS ANYTHING INTERVENING
WITH THIS BIOLOGICALLY AND
CULTURALLY SPECIAL ORGAN MEANS
THAT IT BECOMES EXTRA ETHICALLY
COMPLEX.
AND HERE IN NEUROAI WE BRING
TOGETHER TWO FIELDS, ACTUALLY,
SURPRISINGLY THAT DON'T REALLY
WORK TOGETHER THAT MUCH, MAYBE
IN WAYS THAT OFTEN WE DON'T SEE
NEUROSCIENCE AND A.I. SCIENCES
WORK TOGETHER.
NEUROETHICS BROADLY EXPLORES
ETHICAL, LEGAL, SOCIAL
IMPLICATIONS OF NEUROSCIENCE IN
APPLICATION.
A.I. ETHICS DOES SIMILAR THINGS
AROUND DEVELOPMENT OF A.I. AND
APPLICATIONS AND IT'S A FIELD 30
YEARS OLDER THAN NEUROETHICS.
IT HAS ROUTES IN COMPUTER ETHICS
ROBOT ETHICS. MOST OF YOU FAMILIAR
WITH TOPICS IN THE BLUE THAT
ADDRESS A.I. ETHICS.
THIS IS NOT EXHAUSTIVE BUT
THINGS LIKE FAIRNESS AND
EXPLAINABILITY. FOR NEUROETHICS
RELATED TO INDIVIDUAL EVENT,
FREE WILL, MENTAL PRIVACY,
PERSONAL IDENTITY.
THE INTERSECTION WE SEE THINGS
RELATED TO SAFETY, HUMAN RIGHTS,
CHANGES TO THE STATUS QUO,
CHALLENGES TO HUMAN NATURE, AND
AGENCY AND AUTONOMY WHICH WE'VE
HEARD MANY WORDS THROWN AROUND
TODAY.
AS WE THINK ABOUT THESE ISSUES
THERE'S MANY FLAVORS OF ETHICAL
ISSUES LUMPED IN THE VENN
DIAGRAM, CATEGORIZED IN A NUMBER
OF WAYS.
THE WAY WE CATEGORIZE, AND THEY
OVERLAP BY THE WAY, WHAT BUCKET
WE PUT THEM IN MIGHT DETERMINE
THE KIND OF INTERVENTION WE
WOULD HAVE TO ADDRESS THAT
ETHICAL ISSUE.
SO THERE'S ETHICAL ISSUES
RELATED TO HOW TECHNOLOGY WORKS.
WE MIGHT BE TALKING ABOUT
ENVIRONMENT IMPACT,
TRANSPARENCY, AND THERE ARE
THOSE THAT ARE ETHICAL-LEGAL,
HOW NEUROAI IS GOVERNED,
THERE ARE THOSE THAT ARE HOW
NEUROAI IS APPLIED, INDIVIDUAL
MEDICAL CONTEXT OR MAKING
ASSUMPTIONS ABOUT BROAD GROUPS
OF PEOPLE AND DEFINING DISEASE.
THEN THERE'S RELATIONAL
METAPHYSICAL, THESE ARE MORE
AROUND HOW WE CONCEPTUALIZE
OURSELVES, CHANGING WAYS WE
INTERACT WITH ONE ANOTHER, HOW
WE MIGHT EVEN THINK ABOUT
TECHNOLOGY ITSELF.
SO AS AN ETHICIST, WHO HAS
WORKED AS A FORMER SCIENTIST
TURNED ETHICIST AND HAVING
WORKED WITH MANY SCIENTISTS,
IT'S EASY, IT'S TEMPTING TO ASK
CAN YOU GIVE ME A LIST OF THING
I CAN DO, A CHECKLIST, CAN'T
TELL YOU HOW MANY TIMES I'VE
HEARD THAT.
I WISH IT WERE THAT SIMPLE, I
WOULDN'T EXIST IF IT WERE BLACK
AND WHITE LIKE THAT.
SO MY INSTITUTE, ROOTS ARE IN
INTERNATIONAL BRAIN INITIATIVE,
A LARGE SCALE COLLABORATION
BETWEEN SEVEN EXISTING AND
EMERGING NATIONAL LEVEL BRAIN
RESEARCH PROJECTS AROUND THE
GLOBE.
AND AT THAT TIME THAT GROUP
WANTED TO LEVERAGE ITS RESOURCES
AND ITS DATA TO MAKE SURE THAT
THEY MOVE THE FIELD FORWARD,
PERHAPS ACCELERATE WITH
COLLABORATION.
AND A KEY FOUNDATIONAL PIECE
THAT WAS NECESSARY IN BRIDGING
CULTURES WAS GOING TO BE ETHICS,
THE VALUES UNDERLYING IT.
WE WORKED WITH THE SCIENTISTS,
ETHICISTS, CULTURAL SCHOLARS IN
THAT SPACE AND DEVELOPED A
SERIES OF QUESTIONS THAT COULD
BE USED TO GUIDE EACH OF THE
RESEARCH PROJECTS, SCIENCE AND
ETHICAL QUESTIONS. WE HAVE A
SPECIAL SERIES IN NEURON, BRAIN
PROJECTS TALK ABOUT HOW THEY
IMPLEMENT THESE. FEATURED IN THE
BRAIN 2.0 NEUROETHICS ROADMAP.
I'LL GO THROUGH WHAT I CAN,
EXAMPLES, AND I'LL GIVE ANYONE
WHO WANTS ACCESS TO THESE SLIDES
BECAUSE WE WON'T HAVE TIME TO GO
THROUGH IT ALL TODAY.
ONE AREA THAT WE'VE TALKED ABOUT
THIS MORNING WAS DIGITAL TWINS.
THERE'S PROJECTS RELATED TO HOW
BRAIN WORKS AND THOSE APPLIED TO
PRECISION MEDICINE IN BODY SYSTEMS.
QUESTIONS WE FACE WITH DIGITAL
TWINS ARE RELATED TO CERTAINLY
HOW IT WORKS, SO BIAS, WHAT
CONSTITUTES ACCURATE
REPRESENTATION, IF DATA ISOLATED
IN THESE VARIOUS EXPERIMENTS
ACTUALLY ARE SITUATED IN THE
CONTEXT OF PUBLIC HEALTH
FEATURES AND OTHER DATA THAT
MAKE THEN MEANINGFUL IN REAL
LIFE.
THERE ARE OTHER NEW CONCERNS
THAT ARISE AROUND WHAT
SAFEGUARDS, FOR EXAMPLE, MIGHT
BE IN PLACE WITH EVEN WHAT
PATIENTS AND PARTICIPANTS MIGHT
CONSENT TO IN OWNING OR GIVING
AWAY RIGHTS TO A DIGITAL TWIN.
THEN WE SEE EXAMPLES IN
NEUROMORPHIC INSPIRED BCIs.
SOMEBODY IS DRIVING MY SCREEN.
NEUROMORPHIC-BCI RELATING TO
CHALLENGES TO AUTONOMY AND
AGENCIES.
A QUESTION WE MIGHT ASK IS THERE
ARE MEANINGFUL DIFFERENCES IN
RISKS AND BENEFITS BETWEEN
NEUROMORPHIC BCI AND TRADITIONAL
BCI, IS THERE SOMETHING
DIFFERENT ABOUT NATIVE
COMMUNICATION TO THE BRAIN THAT
WARRANTS A DIFFERENT PROCESS HOW
WE MIGHT GOVERN AND HAVE FUTURE
CIRCUMSTANCES FOR INTEGRATING
TECHNOLOGIES.
AND THEN OF COURSE ONGOING
CONVERSATIONS ABOUT LONG-TERM
RESPONSIBILITIES TO DEVICES
PARTICULARLY IMPLANTED WHICH THE
BRAIN NEUROETHICS WORKING GROUP
HAS WORKED ON QUITE A BIT.
THE LAST QUESTION, THIS IS
PROBABLY ONE OF THE TOUGHEST TO
CRACK, THIS ONE AROUND EMBODIED
NEUROAI.
WE'VE TALKED ABOUT THE VALUE OF
THAT, VALUE OF THAT CREATING
MORE LEGITIMATE SYSTEMS TO
BETTER UNDERSTAND THE BRAIN,
CREATE A BETTER MODEL, SOMETHING
THAT HAS MOTIVATION, MAYBE
SOMETHING THAT'S BETTER
REPRESENTED ACROSS SCALES.
A FUNDAMENTAL THING HERE IS WE
HAVE VERY LOOSELY IN THE FIELD
AND OTHERS BEEN USING WORDS LIKE
INTELLIGENCE, BEHAVIOR,
MOTIVATION, CONSCIOUSNESS, AND
MIND, AS IF WE MEAN THE SAME
THING WHEN WE SAY THAT.
I DON'T MEAN ACROSS SCIENTISTS,
I MEAN ACROSS SOCIETY.
AND WE HAVE ALREADY SEEN THIS
BORNE OUT IN A SCIENTIFIC
COMMUNITY IN MANY WAYS. RECENT
EXAMPLE NOT A NEUROAI-RELATED
ONE, FOR THOSE THAT MAY BE AWARE
WITH THE BRAIN, DISH BRAIN,
CELLS IN A DISH THAT WERE ABLE
TO PLAY PONG, SAID TO BE
EMBODIED IN A VIRTUAL WORLD,
DEMONSTRATE SENTIENCE, AND WE
SAW CALLS FROM THE ETHICISTS.
A PATHWAY TO EXPLORE NEUROAI ETHICS
ITERATE ISSUES, PILOT SOLUTIONS
THRU THE RESEARCH LIFE CYCLE, NOT
JUST ONCE AT THE STAGE OF A
PROTOTYPE, HAVE EMBEDDED ETHICS
RESEARCH THAT CAN FACILITATE
THAT, TOOLS AND FRAMEWORKS. WE
HAVE THE NEUROETHICS PRINCIPLES,
THE QUESTIONS I SHOWED. I SUSPECT
MORE WOULD BE NEEDED IN THE NIH
FOR NEUROAI AND NEUROAI ENGAGEMENT
EXPERIENCES INVOLVING BROAD
GROUPS, NOT JUST SCIENTISTS,
ACROSS DISPRESENCE TO HAVE
BETTER UNDERSTANDING HOW TO
REPRESENT THE SCIENCE AND
DIRECTION THE SCIENCE SHOULD GO.
WITH THAT, THANK YOU FOR YOUR
TIME, I WILL PASS TO THE NEXT
PANEL.
>> I'D LIKE TO INTRODUCE BLAKE
RICHARDS FROM MILA, KICKING OFF
THE FIRST SESSION ON EXPLORING
THE STRUCTURAL AND FUNCTIONAL
CONVERGENCE OF DEEP NEURAL
NETWORKS AND BRAINS.
>> I WANTED TO DO SOMETHING LIKE
WHAT TONY DID, AND TALK ABOUT
WHAT THIS FIELD THAT WE HAVE
COME TOGETHER TO DISCUSS TODAY
ACTUALLY IS BECAUSE I THINK IT'S
NOT ALWAYS CLEAR TO EVERYONE.
SO, I'M GOING TO DO LIKE
EVERYONE ELSE DID AND PUT THIS
CYCLE UP.
BUT I'M GOING TO DO SOMETHING
WHICH IS MAYBE MEAN AND SAY THAT
IT'S VACUOUS.
I DON'T THINK THIS ANSWER
PROVIDES A GREAT DEAL OF MEANING
TO US.
SURE, THIS SOUNDS NICE, BUT IN
PRACTICE WHAT DOES THIS MEAN?
I USUALLY GIVE A STORY ABOUT HOW
A.I. HELPED US INTERPRET AND
ANALYZE DATA, NEUROSCIENCE
INSPIRES SOLUTIONS BUT THAT'S
HAND WAVY.
PRACTICAL ANSWER IS THAT IT'S
GENERALLY RESEARCH THAT USES ANN
TO MODEL THE BRAIN OR WHICH USES
NEURO IDEAS TO IMPROVE ANNs AND
A.I.
THAT MIGHT SEEM A BORING ANSWER.
BUT I THINK IT HIDES MUCH MORE
INTERESTING AND DEEPER
DISCIPLINARY TERM, WHAT I WANT
TO DESCRIBE FOR YOU TODAY.
SO IN THE POST WAR ERA, THERE
WAS A FUNDAMENTAL QUESTION.
SHOULD WE ENGINEER SYSTEMS BASED
ON INSPIRATION FROM THE BRAIN
ITSELF OR FROM LOGIC.
AT THE START IDEAS WERE UNIFIED
WITH THE MCCULLEN PITTS MODEL,
PEOPLE THOUGHT YOU COULD DO BOTH
AT THE SAME TIME.
BUT AS THE FIELD PROGRESSED, A
BATTLE BETWEEN TWO IDEOLOGICAL
CAMPS EMERGED. IN MY EDUCATION
IN COGNITIVE SCIENCE, I LEARNED
THESE CAMPS WERE REFERRED TO AS
GOOD OLD-FASHIONED A.I. OR
GOFAI, AND CYBERNETICS OR
CONNECTIONISM.
THESE TWO CAMPS HAD VERY
DIFFERENT ANSWERS TO HOW EXACTLY
YOU SHOULD TRY TO UNDERSTAND OR
MODEL INTELLIGENCE. GOFAI
FOLKS SAID LOOK INTELLIGENCE IS
ABOUT LOGICAL THINKING, SO CLEAR
SYMBOLIC STRUCTURES, LOGICAL
REASONING PROGRAMS, THESE ARE
THE WAYS TO GO.
EMPHASIS WAS ON PRE-PROGRAMMED,
HIGHLY INTERPRETABLE SYSTEMS.
IN CONTRAST THE IDEA IT HAD TO
BE DONE IN THE SAME WAY AS THE
BRAIN, THAT IS USING PARALLEL
DISTRIBUTED PROCESSING SYSTEMS.
AND THESE RESEARCHERS GENERALLY
TENDED TO EMPHASIZE LEARNING
FROM FEEDBACK AND CONTROL LOOPS,
AND SORT OF LOW LEVEL THINGS
LIKE MOTOR CONTROL THAT A LOT OF
US HAVE TALKED ABOUT TODAY.
SO REALLY IT COULD BE BOILED
DOWN TO ONE QUESTION.
DO WE NEED THE BRAIN TO
UNDERSTAND OR ENGINEER
INTELLIGENCE?
AND THE ORIGINAL KIND OF LIKE
GODFATHERS OF A.I., NOT MODERN
ONES, OLD SCHOOL ONES, LET'S
CALL THEM THE FOUNDERS OF A.I.,
HAD A CLEAR ANSWER.
THE ANSWER WAS NO.
YOU DID NOT NEED THE BRAIN TO
ENGINEER OR UNDERSTAND
INTELLIGENCE.
THIS IS A PICTURE OF A GROUP OF
RESEARCHERS AT THE DARTMOUTH
WORKSHOP, THE FIRST WORKSHOP I
THINK 54 OR 55, WHERE THEY
COINED THE TERM ARTIFICIAL
INTELLIGENCE, IN FACT.
AND NOTABLY, THEY MADE A POINT
OF NOT INVITING ANY OF THE
CYBERNETICISTS OR CONNECTIONISTS
TO THEIR WORKSHOP.
I WAS AT A CFAR MEETING WITH
TERRY THE OTHER DAY, DOING A
CELEBRATION FOR GEOFF HINTON'S
NOBEL PRIZE.
GEOFF HAD AN INTERESTING
ANECDOTE. IN HIS OPINION
HE FELT THAT HAD TURING AND
VON NEUMANN SURVIVED TO
OLDER AGE THIS WOULD NOT HAVE
HAPPENED BECAUSE THEY WOULD HAVE
BEEN ABLE TO ADVOCATE BRAIN
SHOULD BE INCLUDED IN
UNDERSTANDING INTELLIGENCE, AND
THEY BOTH THOUGHT THAT AND
UNDERSTOOD IT AND HAD GOOD
LOGICAL REASONS BUT DID DIE
YOUNG.
AS A RESULT THERE WAS NO ONE TO
CORRECT THESE FOLKS AND SAY, NO,
NO, YOU HAVE TO BRING BRAINS IN.
SO, THEY BASICALLY IGNORED THE
BRAIN FOR A COUPLE DECADES, AND,
WELL, WE ALL KNOW WHERE THAT GOT
THEM. NOW, PART OF THE REASON WHY
IS THAT THE CYBERNETICS PEOPLE
AND CONNECTIONISTS PEOPLE HAVE
A TENDENCY TO OVERPROMISE AND
UNDERDELIVER. WEINER'S THE
HUMAN USE OF HUMAN BEINGS
IS A BRILLIANT BOOK, BUT
IT'S FAR OUT, EVERYTHING IS
GOING TO BE SOLVED FROM THE
ECONOMY TO CHEMISTRY, ELECTRICAL
POWER PLANTS, THE BRAIN, A.I.,
EVERYTHING, CYBERNETIC WILL
SOLVE IT ALL, CONNECTIONS WILL
SOLVE IT ALL.
THAT WAS HARD TO SUPPORT IN THE
FACE OF SMART AND CLEAR THINKERS
LIKE MINKSY, PAPERT, CHOMSKY,
ET CETERA.
BUT THE STORY HAS CHANGED OVER
THE LAST COUPLE DECADES.
NOW A.I. IS ALMOST ENTIRELY
ASSOCIATED WITH PARALLEL
DISTRIBUTED PROCESSING MODELS.
THE GOFAI APPROACH HAS BEEN LEFT
BY THE WAYSIDE BOTH IN
ARTIFICIAL INTELLIGENCE
ITSELF AND ARGUABLY IN PSYCHOLOGY
AND NEUROSCIENCE. SO THIS BRINGS
ME TO WHAT I SAY NEUROAI IS,
THE REALIZATION OF THE ORIGINAL
PROMISE OF CYBERNETICS AND
CONNECTIONISM, IT IS THE FIELD
THAT ATTEMPTS TO UNDERSTAND
INTELLIGENCE USING PARALLEL
DISTRIBUTED PROCESSING SYSTEMS
WITH A FOCUS ON CONTROL LOOPS
AND LEARNING FROM FEEDBACK.
RATHER THAN CONTINUING TO BATTLE
WITH FOLKS IN A.I. AND COGNITIVE
SCIENCE AND OTHER PLACES, WHAT
MANY OF US IN THE NEUROAI FIELD
HAVE DONE IS SIMPLY BYPASS IT.
WE SAID, OKAY, WE'RE DONE WITH
YOU GUYS.
WE DON'T WANT THIS DISCUSSION
ANYMORE.
WE'RE GOING TO DO OUR OWN THING
AND WE'VE CREATED A NEW FIELD.
REALLY, THIS WAS A LESSON WE
LEARNED FROM THE QUOTE/UNQUOTE
NOW GODFATHERS OF A.I., LIKE
GEOFF HINTON WHO SAID, WELL,
OKAY, I'M GOING TO STOP HAVING
DEBATES WITH THESE FOLKS AND
JUST GOING TO ENGINEER STUFF
THAT WORKS.
AND THAT WILL WIN THE DAY.
>> ONE MINUTE WARNING.
>> YEP.
THANKS.
SO, WITHIN THIS NEUROAI PROGRAM
WE HAVE I WOULD ARGUE A LAKATOSIAN
RESEARCH PROGRAM, AN IDEA MY
COLLEAGUES ADVOCATED FOR IN THIS
PAPER ON NEUROCONNECTIONISM,
IT'S A BEAUTIFUL CONCEPT.
THE IDEA WITH THE LAKATOSIAN
RESEARCH PROGRAM IS THAT UNLIKE
THE EXTREME FALSIFIABILITY OF
KARL POPPER PERSPECTIVE OR IT'S
ABOUT PARADIGMS LIKE KUNIAN
SCIENCE EVOLVES WITH CORE
ASSUMPTIONS CRITICAL FOR MAKING
PROCESS, A SERIES OF AUXILIARY
HYPOTHESES THAT ARE OPEN FOR
TESTING AND FALSIFICATION, YES,
INTELLIGENCE IS BEST MODELED WITH
PARALLEL DISTRIBUTED PROCESSING
SYSTEMS AND FOCUS ON CONTROL AND
LEARNING.
AND THEN THERE'S A SERIES OF
HYPOTHESES AROUND THAT WE CAN
TEST AND FALSIFY.
SO RAPIDLY WE CAN START WITH
THAT CORE AND BUILD UP THINGS
LIKE NEOCOGNITRON OR CNNs TO MODEL
THE VENTRAL VISUAL STREAM BUT
WE'LL DISCOVER PROBLEMS AS WE
HAVE WITH CNNs. WE CAN FALSIFY
AND MODIFY AND MOVE THE FIELD
FORWARD WITHOUT DITCHING THAT
CORE ESSENCE OF THE PARADIGM.
SO I SHOULD END NOW BECAUSE I
HAVE FOUR SECONDS.
THANK YOU VERY MUCH.
>> I'D LIKE TO INTRODUCE THE
NEXT SPEAKER, DORIS TSAO FROM
UC-BERKELEY.
>> GREAT TO BE HERE.
SO, TODAY I WANT TO TALK ABOUT
THE CURRENT STATE OF NEUROAI.
SORRY ABOUT THAT.
IS THAT BETTER?
ALL RIGHT.
YES I FIRST WANT TO TALK ABOUT
CURRENT STATE OF NEUROAI BY SHARING
HIGHLIGHTS FROM OUR WORK IN THE
SPACE, AND THEN I WANT TO TELL
YOU ABOUT MY VISION FOR THE
FUTURE OF NEUROAI.
DEEP NETWORKS ARE POWERFUL
BECAUSE THEY LET US DESCRIBE
THINGS HARD TO DESCRIBE.
ORIENTATION OF AN EDGE,
SIMULATE DIFFUSION OF SMOKE BUT
CAN'T DERIVE A FORMULA FROM FIRST
PRINCIPLES FOR A TREE. HARD FOR
CHARLIE GROSS TO GET TRACTION.
WE CAN DESCRIBE COMPLEX OBJECTS
PREVIOUSLY INNEFABLE WITH THESE
DEEP NETWORKS.
THIS TRANSFORMED OUR
UNDERSTANDING OF MECHANISMS FOR
OBJECT RECOGNITION IN I.T.
CORTEX.
NEURONS REPRESENT FACIAL
IDENTITY USING SO-CALLED
SHAPE-APPEARANCE CODE THAT COMES
FROM CLASSIC COMPUTER VISION.
AND THIS IS A WAY OF
PARAMETERIZING FACES THAT SEPARATE
FROM TEXTURE. SINGLE FACE CELLS
PROJECTING TO SPECIFIC PREFERRED
AXIS, ENCODING A GENERATIVE
MODEL THAT LETS YOU RECONSTRUCT
FACES FROM NEURAL ACTIVITY.
A BEAUTIFUL INSIGHT THAT DEEP
NETWORKS GAVE US IS ALL I.T.
CORTEX WORKS IN THE SAME WAY.
CELLS IN IT CORTEX ARE USING AXIS
TO ENCODE, PASSING THROUGH A DEEP
NETWORK TO PRODUCE COMPRESSED
REPRESENTATION IN THE DEEP
LAYER, RECONSTRUCTING ARBITRARY
OBJECTS FROM NEURAL ACTIVITY.
FIRST TWO DIMENSIONS DERIVE FROM
DEEP NETWORKS SPAN A 2D SPACE.
AMAZINGLY WE FOUND PRECISE
CORRESPONDENCE BETWEEN LAYOUT OF
THE 2D OBJECT SPACE AND TOPO-
GRAPHIC LAYOUT. EACH QUADRANT OF
SPACE MAP TO OWN NETWORK OF I.T.
ORGANIZATION OF THE FACE PATCH SYSTEM GENERALIZES TO THE REST OF I.T. CORTEX.
AND WE INITIALLY MADE THIS
DISCOVER USING ALEXNET, DOESN'T
MATTER WHICH NETWORK YOU USE,
THEY ALL REPRESENT THE SAME
OBJECT SPACE, AND MULTIPLE OTHER
GROUPS MADE RELATED FINDINGS
ABOUT CONVERGENCE ACROSS
DIFFERENT NETWORKS.
AFTER I.T. CORTEX, VISUAL
INFORMATION ABOUT FACES GOES TO
MULTIPLE BRAIN REGIONS FOR PROCES-
SING HIGHER LEVEL SOCIAL STIMULI.
IT TURNS OUT OF ACTIVITY IN THESE
REGIONS CAN'T BE EXPLAINED BY
ALEXNET BUT INSTEAD CAN BE WELL
EXPLAINED BY GENERATIVE VIDEO
NETWORK. INDEED USING LATENCE
OF THE VIDEO NETWORK WE COULD
RECONSTRUCT SOCIAL STIMULI FROM
NEURAL ACTIVITY IN FRONTAL
CORTEX AND INSULA, THE LEVEL OF
DETAIL WAS ASTONISHING TO US AND
WE COULD NEVER HAVE DISCOVERED
THIS WITHOUT HELP OF DEEP
NETWORK.
WE'RE MAKING EXCITING PROCESS,
THE WHOLE FIELD OF NEUROAI
BUT I BELIEVE THIS IS THE
BEGINNING OF WHAT'S POSSIBLE.
CURRENT A.I. NETWORK IS BEHIND.
COMPARING START OF THE ART
MULTIMODAL LLMs TO HUMANS.
FOR EXAMPLE, ANSWERING A
MULTIPLE CHOICE QUESTION WHY DID
THE CHILD MOVE THE STEP TOOL
NEAR THE KITCHEN COUNTERTOP?
HUMANS ARE WAY BETTER.
85% COMPARED TO 37%.
AND TO TAKE EXAMPLE FROM MY OWN
WORK, MY PANDEMIC PROJECT, WAS TO
COMPUTATIONAL IMPLEMENT THEORY OF
VISUAL SEGMENTATION TRACKING
DERIVED FROM FIRST PRINCIPLES ABOUT
TOPOLOGY OF VISUAL SURFACES.
MY PROGRAM WAS PAINFULLY SLOW,
RUNNING A THOUSAND COPIES IN
PARALLEL, BUT WAS ABLE TO SHOW
IT COULD SUCCEED ON THIS
SYNTHETIC VIDEO ON THE LEFT, AS
PROOF OF CONCEPT.
WHEN I FED THIS EXACT SAME VIDEO
TO SAM, A DEEPNET FOR SEGMENTATION
TRACKING DEVELOPED BY META, IT
COMPLELTELY FAILED.
LIKELY BECAUSE THIS VIDEO WAS
OUTSIDE THE TRAINING DISTRIBUTION
SO WHATEVER HACKS THE NETWORK
WAS USING DIDN'T WORK.
IT'S NOT POPULAR TO SAY THIS IN
OUR AGE OF BIG DATA BUT I TRULY
BELIEVE WE WANT NETWORKS THAT
INCORPORATE PRINCIPLED
UNDERSTANDING.
I BELIEVE THIS IS THE ONLY PATH
TO REPLICATE NATURAL
INTELLIGENCE AND TO THEREBY
UNDERSTAND THE BRAIN.
WE HAVE TO REMEMBER THAT
NEUROSCIENCE AND A.I. HAVE
FUNDAMENTALLY DIFFERENT GOALS.
NEUROSCIENCE IS FOCUSED ON
UNDERSTANDING.
A.I. IS FOCUSED ON PERFORMANCE.
THE TITLE OF OUR SESSION HERE IS
EXPLORING THE STRUCTURE AND
STRUCTURAL CONVERGENCE CAPTURING
WHAT'S HAPPENING NOW, OFF THE
SHELF NETWORKS ARE COMPARED TO
THE BRAIN, WE MUST ACTIVELY
CREATE THIS CONVERGENCE.
I BELIEVE THE PATH FORWARD IS TO
INTENTIONALLY ALIGN INSIGHTS
FROM A.I., NEUROSCIENCE, AND
COGNITIVE SCIENCE.
SO AS A VISION SCIENTIST, SOME
IDEAS FROM COGNITIVE SCIENCE ARE
DEEP, NEED TO BE INCORPORATED
INTO A.I. VISION SYSTEMS, THE
CONCEPT OF SURFACES AS
FUNDAMENTAL UNITS OF VISUAL
REPRESENTATION, IDEA OF EXPLICIT
OBJECT FILES AS CRITICAL
STRUCTURE FOR ORGANIZING SENSORY
INFORMATION. INVARIANCE IS NOT
PERCEIVED BY POOLING BUT BY
COMPUTING OBJECT TRANSFORMATION.
THESE CONCEPTS ARE SUPPORTED BY
WEALTH OF BEHAVIORAL DATA, ALL
MISSING FROM CURRENT A.I.
AND THE FACT THEY ARE MISSING
HIGHLIGHTS A WEAKNESS OF CURRENT
A.I.
I THINK WE SHOULDN'T JUST WAIT
FOR CONCEPTS TO MAGICALLY APPEAR
IN THE DEEP NETWORKS ENGINEERED
IN SILICON VALLEY BUT SHOULD
INCORPORATE STRUCTORAL PRIORS INTO
OUR NEURAL NETWORKS.
IT'S TRICKY TO FIGURE OUT HOW TO
INCORPORATE THESE PRIORS BUT
NEUROSCIENCE COMES IN HERE.
INSIGHTS FROM NEUROSCIENCE GUIDES
US TO IMPLEMENT THESE PRIORS IN
THESE DISTRIBUTED SYSTEMS.
FOR EXAMPLE, WORK ON SURFACE
REPRESENTATION LED TO THE
DISCOVERY OF BORDER-OWNERSHIP
CELLS THAT CODE WHICH SIDE OF A
FIGURE OWNS A BORDER. WE COULD
INCORPORATE BORDER-OWNERSHIP
CELLS INTO A VISUAL SURFACE
REPRESENTATION NETWORK.
SO THERE'S A SAYING, RENDER TO
CAESAR THE THINGS THAT ARE
CAESAR'S, TO GOD THE THINGS THAT
ARE GOD'S.
AND HERE IS MY PARAPHRASE.
EVERYTHING RIGHT NOW IN A.I. IS
ABOUT DATA.
I THINK NEUROSCIENCE AND
COGNITIVE SCIENCE CAN HELP US
DISCOVER INNATE STRUCTURE OF THE
BRAIN AND RESTORE THE BALANCE.
AND I WANT TO NOTE THESE THREE
STRUCTURES IN PARTICULAR REALLY
BECOME CRITICAL FOR HANDLING
SENSORY INPUT IN DYNAMIC 3D
WORLD, LOTS OF OBJECT GENERATING
OCCLUSIONS AND PERSPECTIVE
CHANGES IN ACTIVE EXPLORATION.
THIS SCENARIO YOU NEED REAL
UNDERSTANDING OF 3D WORLD WHERE
CURRENT DEEP NETWORKS FAIL
MISERABLY AS WE ALREADY
DISCUSSED.
>> TWO-MINUTE WARNING.
>> I'M LUCKY TO TACKLE THIS
CHALLENGE OF REAL 3D WORLD
UNDERSTANDING AS PART OF THE
BRAIN INITIATIVE GRANT ON INTERNAL
MODELS LED BY ADRIENNE FAIRHALL.
THE MONKEYS LOVE PLAYING THIS
GAME. WITH NEUROPIXELS PROBES
WE ARE RECORDING ACROSS THE
MONKEY BRAIN TO UNDERSTAND HOW
A DYNAMIC SPATIAL ENVIRONMENT OF
OBJECTS IS REPRESENTED.
I HOPE THE DATA CAN CONTRIBUTE
TO FOUNDATION MODELS AND AID IN
THIS QUEST TO ALIGN
NEUROSCIENCE, COGNITIVE SCIENCE,
AND A.I.
WE NEED A NEW CROSS-DISCIPLINARY
TEAM GRANTS TO SUPPORT THIS
EFFORT, ALSO COMPUTING RESOURCES
SO TEAMS CAN IMPLEMENT IDEAS AT
SCALE TO COMPETE WITH MODELS IN
SILICON VALLEY.
THANK YOU.
>> I'D LIKE TO INTRODUCE THE
NEXT SPEAKER YIOTA POIRAZI FROM
IMBB-FORTH.
>> THANK YOU VERY MUCH FOR THIS
INVITATION.
GREAT TO BE HERE.
THANKS FOR THE WONDERFUL
INTRODUCTION, THANKS TO DORIS.
I'LL TALK ABOUT HOW TO INCORPORATE
NEW PRINCIPLES INTO NEUROAI.
I WILL TALK ABOUT DENDRITES, MY
FAVORITE SUBJECT.
WHY DENDRITES? FIRST, DENDRITIC
AND MORPHOLOGICAL COMPLEXITY
CORRELATES WITH OUR COGNITIVE
ABILITIES.
AS YOU CAN SEE IN THE SLIDE WHEN
WE'RE YOUNG CHILDREN, DENDRITIC
MORPHOLOGY IS SIMPLE, PEAKS
AROUND THE SAME TIME THE
COGNITIVE ABILITIES PEAK, STARTS
TO DECLINE WITH AGING AND
NEURODEGENERATION.
IT'S NOT ONLY MORPHOLOGY THAT
CORRELATES WITH COGNITIVE
ABILITIES.
THERE'S A NUMBER OF COMPUTATIONS
KNOWN TO TAKE PLACE WITHIN
DENDRITES THAT ARE IMPORTANT.
FOR EXAMPLE, PLASTICITY
CONSIDERED TO BE THE MAIN
SUBSTRATE OF LEARNING TAKES
PLACE AT SYNAPSES HOSTED WITHIN
DENDRITES, IT'S FACILITATED BY A
NUMBER OF NONLINEAR EVENTS,
DENDRITIC SPIKES THAT OPERATE AT
VERY DIFFERENT TIMESCALES.
WE AND OTHERS HAVE SHOWN THAT
THERE ARE STRUCTURAL PLASTICITY
LEARNING RULES THAT CAN HELP
NEURONS ORGANIZE THEIR INPUTS IN A
WAY THAT STRONGLY GENERATE SPIKES
GROUPING THEIR SYNAPTIC
INPUTS INTO NEARBY LOCATIONS IN
SPACE AND TIME. AND BY DRIVING
SPIKES UNDER SPECIFIC CONDITIONS
DENDRITES CAN ACT AS SMALL
COMPUTING UNITS HIGHLY
NONLINEAR AND PRODUCING LOCAL
SPIKES, SPATIAL ARRANGEMENT
BECOMES IMPORTANT READOUT IN
THESE NONLINEAR MODELS.
IF DENDRITES ACT AS SMALL
COMPUTING UNITS THEN NEURONS
ACT AS NEURAL NETWORKS, EVERY
DENDRITIC UNIT IS OFFERING ANOTHER
LEVEL OF NONLINEAR COMPUTATION.
THIS MEANS DENDRITES ALLOW
BIOLOGICAL BRAINS TO BECOME MUCH
MORE COMPUTATIONALLY POWERFUL
AND ALSO EFFICIENT AND LESS
ENERGY COSTLY BY PACKING
EVERYTHING INTO A SIMPLE NEURON.
AND THERE ARE MANY OTHER REASONS
WHY WE'RE INTERESTED IN DENDRITES.
THEY UNDERLIE A LOT OF COMPLEX
FUNCTIONS LIKE SENSORY PERCEPTION,
MEMORY, COINCIDENCE DETECTION,
TEMPORAL SEQUENCE, AND SO FORTH
WE LOVE DENDRITES BUT THIS IS
ABOUT BIOLOGICAL INTELLIGENCE.
WHAT IS THE ROLE OF DENDRITES IN
ARTIFICIAL INTELLIGENCE?
WELL, THEY HAVE SEEMED TO BE
QUITE PROMISING THERE AS WELL,
A NUMBER OF STUDIES ADOPT FOR A.I.
ABILITY TO SEGREGATE INFORMATION
INPUT INTO DIFFERENT DENDRITIC
COMPARTMENTS, ONLY INTEGRATING
TWO INFORMATION SOURCES UNDER
SPECIFIC CONDITIONS THROUGH THE
DENDRITIC SPIKES HAS BEEN USED
TO IMPLEMENT MULTIPLEXING OR
CREDIT ASSIGNMENT INTO SYNAPSES
RESPONSIBLE FOR LEARNING.
UNFORTUNATELY, I DON'T HAVE TIME
TO GO INTO THE FINDINGS BUT WE
HAVE RECENTLY WRITTEN A REVIEW YOU
CAN READ ABOUT IT IF YOU WANT.
IN ESSENCE, HAVING DENDRITES
MEANS THAT INDIVIDUAL NEURONS
ARE EMPOWERED WITH POWERFUL
MULTI-TOOL.
THEY CAN IMPLEMENT DIFFERENT
FUNCTIONS LIKE CONTEXT-GATING,
LOCAL LEARNING, THRESHOLDING,
MULTIPLEXING, AND SO ON, NOT
ALLOWED BY USING POINT NEURONS
WITHOUT DENDRITIC STRUCTURES
WHILE MAINTAINING A VERY LOW
ACTIVITY LEVEL, HIGH SPARSITY IN
THE BRAIN THUS MAINTAINING A
VERY LOWER POWER CONSUMPTION.
LET ME GIVE AN EXAMPLE OF HOW
INCORPORATING DENDRITES INTO
CLASSICAL ARTIFICIAL NEURAL
NETWORKS CAN IMPROVE EFFICIENCY.
TAKE A CLASSICAL FULLY CONNECTED
NEURAL NETWORK, INCORPORATE TWO
DENDRITIC PROPERTIES, ONE IS
STRUCTURE OF THE CONNECTIVITY,
SO SOME OF THE NODES OF THE
NETWORK ONLY TALK TO THEIR CELL
BODIES, SO IT'S SPARSELY CONNECTED.
LET DENDRITES SAMPLE INFORMATION
FROM THE INPUT IN A RESTRICTED
MANNER USING RECEPTIVE FIELDS.
THEN WE COMPARE HOW DO THESE
NETWORKS BEHAVE IF WE PUT THEM
NEXT TO A FULLY CONNECTED NEURAL
NETWORK.
WHAT YOU SEE HERE IS
PERFORMANCE, GRAY IS FULLY
CONNECTED, COLORED ARE THE
DENDRITIC ONES.
YOU NEED ABOUT A MILLION
PARAMETERS TO ACHIEVE THE BEST
PERFORMANCE OF A FULLY CONNECTED
NETWORK, WHEREAS YOU NEED 100
TIMES FEWER PARAMETERS TO
ACHIEVE THE SAME PERFORMANCE IF
YOU INCORPORATE DENDRITES INTO
THESE NETWORKS.
THIS IS ONE KEY EXAMPLE OF HOW
DENDRITIC COMPUTATION CAN ALLOW
PERFORMANCE IMPROVEMENT EVEN IN
CLASSICAL NEURAL NETWORKS THAT
RAN ON GPUs. NOT NEUROMORPHIC
HARDWARE OR MORE BIOLOGICAL
HARDWARE. THERE'S ALSO AN
IMPORTANT BENEFIT WITH RESPECT
TO OVERFITTING. CLASSICAL FULLY
CONNECTED NETWORKS TEND TO INCREASE
LOSS ONCE THEY GET LARGER,
DENDRITIC DO NOT SEEM TO SUFFER
SO MUCH. BY HAVING DENDRITES
INTO FULLY CONNECTED NEURAL
NETWORKS YOU CAN MAKE THEM MORE
EFFICIENT, MORE RESILIENT TO OVER-
FITTING AND EVEN MORE ACCURATE.
WE CAN STUDY HOW THESE NETWORKS
LEARN AND SEE THAT THEY USE A
VERY DIFFERENT LEARNING
STRATEGY.
GRAY IS FULLY CONNECTED.
WHAT NETWORKS TRY TO DO IS EACH
ONE OF THEIR NODES TRY TO LEARN
ONE CLASS, TO BE CLASS SPECIFIC.
DENDRITIC ONES, COLORED ONES,
REVERSE DISTRIBUTION, STRIVE TO
ENCODE INFORMATION ABOUT ALL
CLASSES. BY INCORPORATING THESE
PRINCIPLES IN THESE NETWORKS NOT
ONLY WE CAN MAKE THEM MORE
EFFICIENT BUT PUSH THEM IN A
COMPLETELY DIFFERENT LEARNING
REGIME THAT MAYBE TELLS US THAT
THE BRAIN IS USING A COMPLETELY
DIFFERENT LEARNING STRATEGY TO
SOLVE THE SAME PROBLEM.
AND WE CAN DO THIS ALSO IN
SPIKING NEURAL NETWORKS, ADDING
BIOLOGICAL RULES, SYNAPTIC
TURNOVER, AND LOOKING HOW
THESE PROPERTIES CONTRIBUTE TO
LEARNING IN THESE NETWORKS.
I DON'T HAVE TIME TO GO INTO
DETAILS BUT YOU CAN READ THE
PAPER. IF WE ADD SYNAPTIC
TURNOVER INTO SNN WE CAN GET
HIGHER PERFORMANCE, USING FEWER
EPOCHS RATHER THAN IF WE DON'T
HAVE IT THERE.
SO AGAIN ENSURING MORE ACCURATE
AND FASTER LEARNING HAVING
BIOLOGICAL PROPERTIES.
WE STILL DON'T UNDERSTAND EXACTLY
HOW DENDRITES CONTRIBUTE TO
CIRCUITS FOR LEARNING, THIS IS
REALLY IMPORTANT TO BUILD A.I.
SYSTEMS. TO DO THAT
WE NEED TO BRIDGE GAPS.
WE NEED TO HAVE MORE INFORMATION
ABOUT HOW DENDRITIC PROPERTIES
LOOK LIKE, AND UNFORTUNATELY
PEOPLE DON'T RECORD FROM
DENDRITES ANYMORE SO WE DON'T
KNOW FOR DIFFERENT CELL TYPES.
WE NEED TO DESCRIBE INPUT-OUTPUT
FUNCTION, IMPACT AT NEURONAL AND
BEHAVIORAL LEVELS. EXPERIMENTAL
TECHNIQUES TO HELP US WITH
DETAILED BIOPHYSICAL MODELS CAN
SHED NEW LIGHT.
AND WITH RESPECT TO A.I. FIELD
WE NEED TO IDENTIFY WHICH OF
THESE DENDRITIC PROPERTIES ARE
LIKELY TO BRING THE MOST IMPORTANT
BENEFITS, DESCRIBE THEM
MATHEMATICALLY, PUT THEM INTO
NEW A.I. SYSTEMS AND COME UP
WITH METRICS TO MEASURE ADVANTAGE
WE NEED COLLABORATION BETWEEN
DISCIPLINES TO ACHIEVE THIS GOAL
FROM .
>> TWO-MINUTE WARNING.
>> WE'VE BUILT TOOLS, FIRST
CALLED DENDROTWEAKS TO UPLOAD
EXISTING PREVIOUSLY DEVELOPED
BIOPHYSICAL MODEL AND PLAY WITH
ITS PARAMETERS IN A VISUALLY
EASY TO MANIPULATE SETTING.
YOU CAN CHANGE PARAMETERS AND
LOOK AT RESPONSES AT DIFFERENT
LEVELS OF THE MODEL, STANDARDIZE
ION CHANNELS THAT ARE THERE, YOU
CAN REDUCE MORPHOLOGY AT
DIFFERENT STAGES, KEEPING THE
RESPONSE SIMILAR TO ORIGINAL ONE
AND YOU CAN USE SCRIPTS TO
VALIDATE THE MODEL THAT YOU'RE
INTERESTED IN.
AND THIS IS REALLY IMPORTANT FOR
ADVANCING OUR UNDERSTANDING
BECAUSE YOU CAN CHANGE
PARAMETERS AND LOOK AT THE
IMPACT AT THE SINGLE NEURON
LEVEL AT LEAST. YOU CAN GO TO
USE DENDRIFY, THE OTHER TOOL TO
ADD ONLY THE MOST IMPORTANT
PROPERTIES OF DENDRITES TO POINT
NEURONS. LIF NEURONS ADD COMPARTMENT
OF DENDRITIC PROCESSES IN A
SIMPLE MANNER, REALLY FAST,
RUNNING ON BRIAN, CAN CAPTURE
KEY PROPERTIES, DENDRITIC NON-
LINEARITIES, BACKPROPAGATING
ACTION POTENTIAL, NONLINEAR
INTEGRATION EFFICIENTLY SO
YOU CAN RUN IT ON LARGE NETWORKS
AND ASSESS THE BENEFITS OF
DENDRITES. WITH THAT I'D LIKE
TO THANK YOU VERY MUCH FOR YOUR
ATTENTION.
[APPLAUSE]
>> I'D LIKE TO INTRODUCE CARINA
CURTO FROM BROWN UNIVERSITY.
>> IS THE SOUND OKAY?
NOW IT'S ON?
OKAY.
GREAT.
ALL RIGHT, GREAT.
LET'S GET STARTED.
SO, I'D LIKE TO FOCUS ON
SOMETHING SLIGHTLY DIFFERENT
THAN WHAT THE PREVIOUS TALKS
TALKED ABOUT, THE ROLE OF MATH
IN SMALL NETWORKS.
I'M A MATHEMATICIAN BY TRAINING,
TRAINED IN MATHEMATICAL PHYSICS.
I THROUGH MY Ph.D., SWITCHED
TO NEUROSCIENCE, A VERY EXCITING
FIELD, I DID NOT REGRET IT.
MORE EXCITING THAN EVER RIGHT
NOW.
AND I WANT TO THINK ABOUT THE
ROLE THAT DEVELOPING MORE
ADVANCED MATHEMATICS CAN PLAY IN
HELPING WITH NEUROAI.
SO, HERE I, YOU KNOW, ASKED
GOOGLE THE A.I. OVERVIEW, HOW
BIG THE NETWORKS ARE.
I GOT SOME NUMBERS.
NOTHING NEW.
TRILLIONS OF WEIGHTS HAVE TO BE
STORED FOR SOME OF THESE MODELS,
VERY LARGE DATASETS.
HIGH ENERGY USAGE.
WE'RE TALKING ABOUT NUCLEAR
REACTORS NOW FOR SOME STUFF.
AND THERE'S A HOPE
THAT WE CAN DO BETTER BY GETTING
INSPIRATION FROM NEUROSCIENCE
AND I TOTALLY AGREE WITH THAT.
SOME INGREDIENTS THAT A LOT OF
CURRENT A.I. SYSTEMS SEEM TO
IGNORE WE JUST HEARD A LOVELY
TALK ABOUT DENDRITES AND OTHER
ASPECTS, NEUROMODULATION IS
REALLY IMPORTANT BECAUSE IT
ALLOWS YOU TO BASICALLY ENCODE
MULTIPLE NETWORKS
IN ONE.
BECAUSE VIA NEUROMODULATION THE
SAME NETWORK CAN HAVE MULTIPLE
DYNAMIC REGIMES.
RHYTHMS AND OSCILLATIONS ARE
ALSO A REALLY IMPORTANT ASPECT
OF WHAT NETWORKS IN THE BRAIN
ACTUALLY DO.
WHEN YOU THINK ABOUT NEURAL
CIRCUITS THAT MAINTAIN CENTRAL
PATTERN GENERATORS, THESE CIRCUITS,
MAINTAIN DIFFERENT PATTERNS OF
ACTIVITY FOR LOCOMOTION AND SO
ON WHEN TALKING ABOUT EMBODIED
A.I., THESE ARE THINGS WE NEED
TO THINK ABOUT.
IT IS VERY INSPIRING TO THINK
ABOUT THE CONNECTIVITY
STRUCTURES WE'VE ALREADY WORKED
OUT, THE FLY COMPASS SYSTEM A
LOVELY PART OF THE FLY BRAIN,
WHICH HAS BEEN MENTIONED ALREADY
SEVERAL TIMES TODAY.
I THINK OTHER SMALL CREATURES,
ZEBRAFISH IS ONE I WORK ON WITH
THE CRCNS GRANT THROUGH THE BRAIN
INITIATIVE, THAT'S BEEN GREAT.
THERE ARE LOTS OF DIFFERENT
CIRCUITS THAT I THINK ARE VERY
INTERESTING TO STUDY, HOW THESE
SMALL CREATURES DO INCREDIBLY
COMPLEX BEHAVIORS, MAKE
DECISIONS AND SO ON AND DON'T
NEED TRILLIONS OF PARAMETERS TO
DO IT.
MATHEMATICAL ISSUES, IT'S WHERE
I WANT TO GET TO IN THIS TALK.
I THINK A LOT OF THE POTENTIAL
THAT WE'VE SEEN IN THE PAST AS
WELL OF GETTING NEUROSCIENCE
IDEAS INTO THE A.I. SPACE GOES
THROUGH KIND OF THE STEP OF
MATHEMATICAL ABSTRACTION. IF YOU
THINK ABOUT MCCULLOCH-PITTS AND
THESE EXAMPLES WE'VE TALKED ABOUT
THE HOPFIELD MODEL, BOLTZMANN
MACHINES, THERE ARE INSPIRATIONS
FROM NEUROSCIENCE, THERE'S THIS
MATHEMATICAL ABSTRACTION, THEN THE
MOST USEFUL MODELS TEND TO BE THE
ONES THAT WE CAN UNDERSTAND.
I WANT TO TAKE ONE MOMENT TO
IMAGINE THE FOLLOWING SCENARIO.
IMAGINE A WORLD WHERE WE DIDN'T
HAVE LINEAR ALGEBRA.
OKAY?
SO IMAGINE YOU HAVE A LINEAR
SYSTEM OF DIFFERENTIAL EQUATIONS
CAN WRITE IT DOWN, WE HAVE
CALCULUS, WE KNOW ABOUT RATES OF
CHANGE, AND WE HAVE SOME MATRIX
A OF INTERACTION, AND LOTS AND
LOTS OF -- LOTS AN LOTS OF
VARIABLES INTERACTING IN COMPLEX
WAYS, ARBITRARILY COMPLEX AND
DIFFICULT TO UNDERSTAND.
YET IF THE SYSTEM IS LINEAR, WE
KNOW WHAT TO DO, RIGHT?
EVEN OUR SECOND YEAR
UNDERGRADS CAN STUDY THAT
MATRIX, WE KNOW HOW TO COMPUTE
VALUES, EIGENVECTORS, THIS
MATHEMATICS HAS BEEN USED TO
UNDERSTAND MARKOV CHAINS,
ENGINEERING APPLICATIONS, SO ON.
WE UNDERSTAND LINEAR SYSTEMS.
VERY, VERY WELL.
THAT HAS LED TO REMARKABLE
PROGRESS IN ENGINEERING,
PHYSICS, EVEN IN BIOLOGY.
NOW THINKING ABOUT
NEUROSCIENCE, FUNDAMENTALLY THE
NEURONS TELL US WE NEED TO LOOK
AT NONLINEARITIES.
WE HEARD ABOUT IT IN THE
DENDRITE TALK AS WELL.
WHEN YOU ADD NONLINEARITY EVEN
IN THE MOST SIMPLIFIED VERSION
OF THE PROBLEM IT CHANGES
EVERYTHING.
OKAY?
NOW SUDDENLY WE DO NOT HAVE THE
HANDLE THAT WE HAD ON THE
LEFT-HAND SIDE IN THE LINEAR
CASE.
NOW SUDDENLY WE DON'T KNOW WHAT
THE ASYMPTOTIC DYNAMICS WILL DO.
WE DON'T KNOW HOW MANY EQUILIBRIUM
POINTS. IN NONLINEAR DYNAMICS,
THERE CAN BE MULTIPLE, LIMIT CYCLES
PERIODIC PATTERNS OF ACTIVITY,
LOTS OF STUFF CAN GO ON, AND
WITHOUT A BETTER THEORY IT'S VERY
DIFFICULT TO UNDERSTAND WHAT'S
POSSIBLE IN THIS PARAMETER SPACE
EVEN WITH THE SAME NUMBER OF
PARAMETERS.
THAT W MATRIX AND B VECTOR CAN
HAVE THE SAME SIZE AS THE A AND
B ON THE LEFT BUT BECAUSE OF THE
NONLINEARITY SUDDENLY IT'S
INTRACTABLE.
OKAY.
SO, ONE OF THE THINGS THAT I
GUESS I WANT TO ADVOCATE IS THAT
NEUROSCIENCE AND NEURAL CIRCUITS
THAT WE SEE IN THE BEHAVIOR AND
INTERACTIONS WE SEE CAN LEAD US
TO VERSIONS OF THIS WHERE EVEN
THOUGH WE HAVE NONLINEARITIES
THEY ARE SURPRISINGLY
TRACTABLE.
ONE EXAMPLE FROM MY WORK, ONE
VERSION OF THE FIRING RATE MODEL
WITH A SIMPLE RELU NONLINEARITY,
MACHINE LEARNING PEOPLE LIKE
THAT ONE TOO.
IN THIS SITUATION THERE ARE --
THERE'S A FULL RICHNESS OF
NONLINEAR DYNAMICS, NETWORKS
USED SUCCESSFULLY FOR LOTS OF
MODELING APPLICATIONS IN
NEUROSCIENCE.
WE DIDN'T INVENT THEM.
THEY HAVE BEEN AROUND AT LEAST
40 YEARS. FIRST MATHEMATICAL
RESULTS IN EARNEST MORE THAN
20 YEARS FROM SEUNG, HAHNLOSER,
SLOTINE. YOU CAN THINK OF THEM
AS SORT OF A 21ST CENTURY HOPFIELD
MODEL THAT HAS ALL OF THE SORT OF
STATIC, MULTI-STABILITY AND
MULTIPLE-STATIC ATTRACTORS LIKE
HOPFIELD MODEL DOES, CAN ENCODE
DYNAMIC ATTRACTORS, RICH
PATTERNS OF ACTIVITY AND DO LOTS
OF THINGS.
SO THE PROBLEM OF COURSE IS
BECAUSE OF THE NONLINEARITY
IT'S HARD TO UNDERSTAND.
SO IN MY GROUP WE HAVE BEEN
WORKING ON THESE AS LIKE KIND OF
THE BABY STEPS TOWARDS
UNDERSTANDING HIGH DIMENSIONAL
NONLINEAR DYNAMICAL SYSTEMS,
FOR NEUROSCIENCE INSPIRED
NETWORKS AND WE'VE BEEN ABLE TO
PROVE A BUNCH OF THEOREMS TO
CONNECT GRAPH STRUCTURE AND
ARCHITECTURE TO THE DYNAMICS.
THIS GIVES AN EXAMPLE OF WHERE A
LITTLE BIT OF MATHEMATICS CAN GO
A LONG WAY. FOR EXAMPLE, MY FORMER
STUDENT, JULIANA LONDONO ALVAREZ
WHO'S HERE GIVING A POSTER
TOMORROW MORNING I BELIEVE --
>> TWO-MINUTE WARNING.
>> OKAY.
SHE HAD THE VERY INTERESTING
PROJECT AS A GRADUATE STUDENT OF
TRYING TO DESIGN A NETWORK THAT
ENCODED MULTIPLE QUADRUPED GAITS.
SO LIMIT CYCLES, DYNAMIC ATTRACTORS
INVOLVED THE SAME CONTROL
NEURONS FOR THE LIMB AND COULD
ENCODE MULTIPLE GAITS AT ONCE, A
PROBLEM MANY GROUPS TACKLED, AND
TYPICALLY THE SOLUTIONS INVOLVE
A FAMILY OF MODELS WHERE YOU
HAVE TO CHANGE SOME PARAMETER IN
ORDER TO GO FROM ONE PATTERN TO
ANOTHER, SYNAPTIC PLASTICITY OR
EXTERNAL DRIVE, SOMETHING HAS TO
CHANGE.
AND BECAUSE OF THE
SMALL AMOUNT OF MATHEMATICAL
UNDERSTANDING THAT WE HAVE AND
SPECIAL FAMILY OF THRESHOLD
LINEAR NETWORKS SHE IS ABLE TO
QUICKLY DESIGN A NETWORK
ENCODING FIVE DIFFERENT KINDS OF
GAITS, ALL SIMULTANEOUSLY, NO
CHANGE OF PARAMETERS, REALLY IS
FIVE ATTRACTORS IN THE NETWORK
AND DIFFERENT INITIAL CONDITIONS
WILL DROP YOU IN EACH ONE.
THIS IS JUST AN ADVERTISEMENT
FOR HER WORK, BUT POINT IS
SOMETHING COMPLICATED, WE DID NOT NEED A NETWORK OF 20,000 TO 100,000 NEURONS WITH LOTS OF TRAINING.
WE NEEDED A LITTLE BIT OF MATHEMATICAL
INSIGHT, AND I THINK THAT, YOU
KNOW, IN THE LONG RUN THERE WILL
BE MANY OPPORTUNITIES TO REALLY
TAKE ADVANTAGE OF A LITTLE BIT
OF EXTRA MATHEMATICAL INSIGHT IN
ORDER TO SEARCH BETTER THE
PARAMETER SPACE.
SAME MODELS, RIGHT?
SAME MODELS THAT WE'RE USING
THAT FIND BETTER
PLACES IN THE PARAMETER SPACE
WHERE WE CAN START BETTER
ARCHITECTURES THAT LEAD US TO
THE KIND OF DYNAMICS WE WANT.
I THINK WITH THAT I WILL END.
THANK YOU VERY MUCH.
>> NEXT SPEAKER IS EV
FEDORENKO FROM M.I.T., AND USE
PODIUM MICS, BETTER SOUND.
>> THANKS FOR HAVING ME HERE.
ANIMAL MODELS HAVE BEEN
INVALUABLE IN BOTH BASIC
NEUROSCIENCE AND MEDICAL
RESEARCH.
OF COURSE IN MANY CASES THE HOPE
IS TO UNDERSTAND SOMETHING ABOUT
HOW THE HUMAN BRAIN WORKS.
THE REASON WE USE ANIMAL MODELS
IS BECAUSE THE
HARDWARE IS SIMILAR ENOUGH, WE
CAN USE APPROACHES NOT AVAILABLE
IN HUMANS. WE RECORD FROM
HUNDREDS OF CELLS AT ONCE, USE
OPTOGENETICS TO CONTROL CELLS,
SO ON.
WHAT IF THE BEHAVIOR OF INTEREST
IS LANGUAGE, SOCIAL REASONING,
OR MATHEMATICAL COGNITION?
WE DON'T DO WELL WITH A MOUSE OR
MACAQUE AND WE'RE STUCK WITH
HUMANS.
THE REASON IT'S SO CRITICAL TO
UNDERSTAND HOW LANGUAGE WORKS IS
THAT IT'S SUCH A FUNDAMENTAL
PART OF THE HUMAN EXPERIENCE.
USING LANGUAGE WE BUILD DEEP
INTERPERSONAL RELATIONSHIPS AND
ESTABLISH INTERNATIONAL
POLITICAL ORDER, SEND ROCKETS TO
SPACE, SUBMARINES TO THE OCEAN,
CURE DISEASES, ENACT SOCIAL AND
POLITICAL CHANGE, AND MAKE A
RECORD OF THE HISTORY OF OUR
PLANET AND OUR SPECIES.
LOSING LANGUAGE EARLY ON IN LIFE
DUE TO DEVELOPMENTAL
COMMUNICATIVE DISORDERS IS
DEVASTATING FOR OUR ABILITIES TO
LEARN ABOUT THE WORLD, AND
LOSING LANGUAGE DUE TO STROKE OR
DEGENERATION IS INCREDIBLY
PAINFUL FOR BOTH PATIENTS AND
THEIR LOVED ONES.
SO OF COURSE IF WE'RE STUCK WITH
HUMANS WE'RE LIMITED TO THE
KINDS OF APPROACHES THAT WE CAN
USE.
BUT STILL WE'VE BEEN ABLE TO
MAKE SOME PROGRESS, I'LL TELL
YOU IN ONE MINUTE, SO WE'VE
LEARNED THAT THERE'S A NETWORK
IN THE LEFT HEMISPHERE THAT
SUPPORTS LANGUAGE PROCESSING.
THESE AREAS ARE ENGAGED WHEN WE
UNDERSTAND LANGUAGE, REGARDLESS
OF WHETHER IT'S SPOKEN, WRITTEN,
SIGNED.
AND WHEN WE PRODUCE LANGUAGE
ALSO ACROSS MODALITIES, WHICH
SUGGESTS THAT THESE AREAS LIKELY
STORE OUR KNOWLEDGE OF LANGUAGE,
WHAT WORDS MEAN, HOW TO PUT THEM
TOGETHER TO MAKE COMPLEX
MEANINGS, AND OF COURSE WE NEED
THIS KNOWLEDGE TO CONVERT OUR
THOUGHTS INTO SEQUENCES OF WORDS
AND EXTRACT MEANING.
WHAT I TOLD YOU SHOULD MAKE IT
CLEAR THAT THESE AREAS ARE
DISTINCT FROM THE AREAS THAT
HAVE BEEN HISTORICALLY DISCUSSED
BROCA'S AREA IN RED AND WERNICKE'S
AREA IN BLUE AND ARE INSENSITIVE
TO THE MEANING OF LANGUAGE.
WORK IN MY GROUP ESTABLISHED
LANGUAGE SYSTEM IS DISTINCT FROM
SYSTEMS OF KNOWLEDGE AND
REASONING, I'M SHOWING THREE
SUCH SYSTEMS BUT THERE ARE MORE.
OF COURSE LANGUAGE HAS TO WORK
CLOSELY WITH THESE SYSTEMS BUT
IT'S NOT THE CASE THAT YOUR
LANGUAGE SYSTEM IS DOING THE
THINKING. YOU CAN STILL THINK IF
YOU LOSE YOUR LANGUAGE SYSTEM.
IT'S THERE BY AGE 3 BUT WE CAN'T
MEASURE IN 2-YEAR-OLDS, MUCH
HARDER. IT'S SIMILAR ACROSS
TYPOLOGICALLY DIVERSE
LANGUAGES, MANY OF US IN THE
AUDIENCE HAVE TWO LANGUAGES
SYSTEMS, TWO OR MORE LIVING IN
THEIR BRAIN, THEY ALL LIVE IN
THE SAME PLACE.
DAMAGE TO THESE REGIONS IN
ADULTHOOD LEADS TO LOSS OF
FUNCTION SO THEY ARE CRITICALLY
IMPORTANT FOR LANGUAGE.
FINALLY BASED ON DOZENS OF
STUDIES WE HAVE A BASIC
UNDERSTANDING OF WHAT THE
REGIONS DO WHICH IS RELATED TO
RETRIEVING WORDS FROM MEMORY AND
PUTTING THEM TOGETHER TO MAKE
NEW COMPLEX MEANINGS. BUT WHAT
DATA STRUCTURES SUPPORT THESE
COMPUTATIONS IN COMPREHENSION AND
PRODUCTION? AND HOW ARE THEY
INSTANTIATED IN NEURONS?
WELL, UNTIL RECENTLY IT WAS NOT
CLEAR HOW TO MAKE PROGRESS BUT
THEN EVERYTHING
CHANGED.
NEURAL NETWORK LANGUAGE MODELS
EMERGED PRODUCING BEHAVIOR THAT
WAS HUMAN-LIKE, LINGUISTIC
BEHAVIOR THAT WAS HUMAN-LIKE.
JUST LIKE THAT WE HAD A MODEL
ORGANISM ADMITTEDLY
IN SILICO, NOT BIOLOGICAL, BUT
FOR THE FIRST TIME IN THE
HISTORY OF HUMANKIND WE HAD A
SYSTEM OTHER THAN THE HUMAN
BRAIN THAT WOULD PRODUCE
BEHAVIOR OF INTEREST, MEANING
OUR LANGUAGE SCIENTIST BEHAVIOR
OF INTEREST. INSPIRED BY THESE
MODELS, MARTIN SCHRIMPF WHO
WILL WHO TALK AFTER ME WANTED TO
SEE IF SIMILARITY IN BEHAVIORS
EXPANDS TO REPRESENTATIONS.
HE PRESENTED SENTENCES TO BOTH
MODELS IN HUMANS FROM THE BRAIN,
EXTRACTED RESPONSES FROM
LANGUAGE AREAS, FROM THE MODELS
INTERNAL REPRESENTATIONS AS THEY
PROCESSED SIMILARLY, TESTING
SIMILARITY BETWEEN THEM.
EVALUATED A LARGE NUMBER OF
LANGUAGE MODELS, FOUND QUITE A
FEW OF THEM DO PRETTY WELL IN
ALIGNING WITH HUMAN NEURAL
REPRESENTATION, EACH BAR IS A
MODEL, THE GRAY LINE IS THE CEILING BASED ON THE INTERNAL RELIABILITY OF THE DATASET.
NOW WE AS LANGUAGE SCIENTISTS
HAVE A MODEL SYSTEM, IN SILICO
SPECIES, THAT PRODUCES THE SAME
BEHAVIORS AS HUMANS,
QUOTE/UNQUOTE BRAIN CAN BE
ANALYZED AT THE CELLULAR AND
CIRCUIT LEVELS USING THE
EVER-GROWING SUITE OF MODEL
INTERPRETABILITY TOOLS WHICH
CAN BE TAKEN APART LIKE WE CAN'T
WITH BIOLOGICAL BRAINS.
WE CAN NOW UNDERSTAND HOW
LANGUAGE WORKS WITH THE
PRECISION AND SCALE COULD NOT
IMAGINE POSSIBLE IN MY
LIFETIME.
WE CAN UNDERSTAND HOW IT WORKS.
WE RECENTLY REVIEWED EFFORTS IN
THE SPACE IN THE RECENT PAPER IN
HER WORK GRETA TUCKUTE SHOWED YOU
CAN MODULATE RESPONSE IN CLOSED
LOOP MANNER. EGHBAL HOSSEINI IS
DISCOVERING THE CORE CODING ACCESS
USING CONTROVERSIAL STIMULI.
WE USE INTRACRANIAL RECORDINGS TO
EVALUATE ALGORITHMS FOR HOW
REPRESENTATIONS EVOLVE AS NEW WORDS
COME IN. WE ALSO ASK QUESTIONS ABOUT
HOW LANGUAGE SYSTEM DEVELOPS AND
CAN BREAK. ON DEVELOPMENT, WE USE
CONTROLLED REARING APPROACH TO
ISOLATE DIFFERENT VARIABLES THAT
AFFECT TRAJECTORY AND ENDPOINT
OF LANGUAGE LEARNING, INCLUDING
USING MODELS THAT LEARN IN MORE
HUMAN-LIKE WAYS.
WITH RESPECT TO BEHAVIOR,
WE CAN RELAY REPRESENTATIONS
FROM BABY LANGUAGE MODELS TO
NEURAL DATA ACROSS DEVELOPMENTAL
TRAJECTORY.
WITH RESPECT TO LINGUISTIC
DISORDERS, WE CAN BREAK MODELS IN
VARIOUS WAYS AND TEST THEIR
BEHAVIOR TO SEE IF WE MAY
SIMULATE PATTERNS OF IMPAIRMENT
AND TRY TO UNDERSTAND WAYS THEY
BREAK SUCH THAT THEY LEAD TO
TO PARTICULAR KINDS OF
LINGUISTIC DEFICITS.
FINALLY WE CAN START ASKING HOW
DIFFERENT PIECES OF LANGUAGE
PROCESSING PIPELINE WORK
TOGETHER.
FOR EXAMPLE HOW DOES LANGUAGE
SYSTEM WORK WITH REASONING
SYSTEMS.
HOW DOES INFORMATION GO FROM THE
LANGUAGE SYSTEM TO DOWNSTREAM
SYSTEM WHEN WE HAVE TO REASON ON
THE INFORMATION THAT WE GET
THROUGH LANGUAGE FOR EXAMPLE TO
SOLVE A VERBAL MATH PROBLEM OR
UPDATE OUR KNOWLEDGE OF SOME
SOCIAL AGENT? ALMOST DONE.
SO WHAT DO WE NEED TO MAKE ALL
THESE COOL THINGS?
THERE'S FOUR PIECES HERE.
I THINK WE NEED MUCH BETTER DATA
THAN WHAT WE HAVE.
I WOULD ARGUE THE DATA THAT WE
NEED TO UNDERSTAND HUMAN
LANGUAGE AND OTHER HIGH LEVEL
COGNITION DOESN'T EXIST HARDLY
AT ALL YET.
WE NEED MORE BIOLOGICALLY AND
COGNITIVELY PLAUSIBLE MODELS,
THERE'S QUITE A FEW MORE THAN
WHAT I LISTED.
WE NEED SOPHISTICATED MODELS TO
METRICS, CURRENT METRICS ATTEMPT
TO MAKE CONNECTIONS WHICH LINK
BETTER TO IMPLEMENTATION LEVEL
COMPARED TO REPRESENTATIONAL LEVEL.
AND THESE MODELS ARE POWERFUL,
AND THAT'S FINE BUT THAT MEANS
THE BURDEN IS ON US TO BE GOOD
EXPERIMENTALISTS AND DRAW THE
RIGHT CONCLUSIONS AND NOT
OVERINTERPRET WHAT WE CAN FIND
FROM MODELS WITH A LOT OF
EXCITING DISCOVERIES THAT LAY
AHEAD.
THANK YOU.
NEXT SPEAKER IS MARTIN SC
HRIMPF FROM EPFL.
>> THANKS TO THE ORGANIZERS,
USUALLY, TO GIVE BACKGROUND WHY
WE WANT MODELS, I THOUGHT WE
WOULD ALL AGREE MODELS ARE
GREAT.
I DON'T THINK THE GOAL IS TO BE
BUILD AGI, JUST BUILDING A MODEL
OF THE BRAIN IS USEFUL.
IT'S USEFUL FROM ONE
PERSPECTIVE, BUT ALSO FROM THE
PERSPECTIVE THIS COULD LEAD TO
FUTURE CLINICAL APPLICATIONS, I
WANT TO TALK ABOUT WHAT WE CAN
DO ON THE MORE BRAIN SIDE OF
THINGS IF WE HAD A REALLY GOOD
MODEL OF THE BRAIN.
FIRST BACKGROUND, ONE OF THE
CORE FINDINGS THAT HAS COME OUT,
WHEN WE SHOW THE SAME STIMULI TO MODELS THAT WERE SHOWN TO HUMANS OR ANIMALS AND MEASURE
INTERNAL ACTIVITY, THAT JUST TENDS TO ALIGN, TRUE
EVEN THOUGH A.I. MODELS WERE NOT
BUILT FOR NEUROSCIENCE THE LAST
10-20 YEARS OR SO. A.I. DOES NOT
BUILD THINGS IN A BRAIN-LIKE
WAY BUT THE WAY THEY OPTIMIZE
MODELS LEADS TO REPRESENTATIONS THAT ARE SURPRISINGLY BRAIN LIKE.
THIS IS AN EXAMPLE FROM VISION.
Y AXIS IS ALIGNMENT TO A SET OF
DIFFERENT NEURAL RECORDINGS FROM
PRIMATES AS WELL AS BEHAVIOR.
AND ONE DOT IS ONE MODEL.
TURNS OUT IF YOU MEASURE THEIR
ALIGNMENT, THAT'S PREDICTED BY
THEIR TASK PERFORMANCE.
THAT IS WHAT MACHINE LEARNING
CARES ABOUT, AND THIS ALIGNS
WITH THE BRAIN LIKENESS OF THE
MODEL. THIS IS NOT JUST IN
VISION. RECENTLY THERE HAS BEEN
A PAPER FOR MOTOR.
THIS SEEMS TO BE A THEME WHERE
MODELS TEND TO BE GOOD AT
PERFORMING A TASK ARE THE ONES
THAT ARE BRAIN-LIKE IN THEIR
INTERNAL REPRESENTATIONS.
THIS IS A MORE RECENT FINDING.
IF YOU LOOK AT THEM, MOST RECENT
MODELS, ONES GETTING BETTER AT
MACHINE LEARNING THE LAST COUPLE
YEARS, THEY ARE NOT TURNING OUT
TO BE MORE BRAIN-LIKE.
IF ANYTHING WE'RE SEEING
INVERSE, THE MODELS STARTING TO
APPROACH 100% ARE LESS AND LESS
BRAIN-LIKE.
RECENTLY WE FIND THAT IF YOU
LOOK AT THE SCALE OF MODELS,
THIS IS WHAT MACHINE LEARNING
OPTIMIZES FOR VERSUS AVERAGE
ALIGNMENT OF MODELS THEN THERE'S
A BIT OF DISTINCTION. THE GRAY
LINE IS A RANDOM HIGH DIMENSIONAL
EMBEDDING. MANY FEATURES ALONE DO NOT HELP. THEY DO NOT GIVE YOU HIGH ALIGNMENT.
IF YOU LOOK AT THE ALIGNMENT OF MODELS WITH RESPECT TO BEHAVIOR AND TO NEURONS SEPARATELY,
IT DOES SEEM TO CONTINUE TO SCALE
AT LEAST IN VISION AGAIN.
FOR INTERNAL ALIGNMENT OF MODELS
IT DOES SEEM WE'RE NOT GOING TO
GET TO A MODEL THAT'S REALLY
LIKE THE BRAIN.
I THINK THERE'S SOMETHING THAT
NEEDS TO BE DONE AND TO ME THAT
MEANS THIS IS A FRONTIER FOR OUR
FIELD OF ACTUALLY BUILDING
BRAINLIKE MODELS TRULY
RESPECTING BREADTH AND DEPTH OF
THE BRAIN MECHANISMS, I'LL SPEAK
TO THAT IN MORE DETAIL BUT I
WANT TO SAY OUR FIELD NEEDS TO
BUILD MODELS THAT DO NOT ABANDON
WHAT WE'VE LEARNED FROM A.I. BUT
ENHANCE IT WITH BRAIN INTERNALS.
I'LL WALK THROUGH A FEW THINGS
WE NEED FOR THAT.
ONE IS ACCESSIBLE HIGH-FIDELITY
DATA.
BRAIN AND OTHER INITIATIVES HAVE
BEEN GREAT IN GIVING LARGE SCALE
DATA.
THERE'S STILL CHALLENGES IN EVEN
THOUGH DATA IS SUPPOSED TO BE
OPEN, WE DON'T HAVE STANDARDS.
ONES WE DO DON'T REINFORCE
METADATA, OFTEN FROM PERSONAL
PAINSTAKING EXPERIENCE, IT'S
EXTREMELY HARD TO REALLY GET
DATA USEFUL, TO BE USEFUL, OFTEN
NOT COLLECTED WITH MODELS IN
MIND.
IT'S SO LIMITED IN ITS SCALE OR
IN STORABILITY YOU CAN'T RELEASE
IT OR TEST MODELS AT SCALE.
THERE ARE EFFORTS TO TRY AND FIX
THAT.
ALL THE POWER TO THEM.
ALSO IF YOU HAVE DATA THAT'S
SMALL SCALE, OFTEN THAT DOES NOT
HELP TO DIFFERENTIATE BETWEEN
MODELS, THIS IS ABOUT RAISING
THE CEILING.
CURRENTLY IN LANGUAGE I THINK
ALSO OTHER FIELDS, SUBFIELDS,
WE'RE GETTING TO A POINT FOR A
LOT OF DATASETS, WHICH MODELS
ARE BETTER, WHICH WORSE,
EVERYTHING LOOKS TO BE THE SAME.
WE NEED TO DEVELOP MORE DATA
WITH MODELS IN MIND AND SEPARATE
BETWEEN MODEL TYPES.
THE OPPORTUNITY TO ME IN THIS,
WE SHOULD COLLECT NEW DATA, THAT
IS MAYBE NOT JUST FOCUS ON A A
TINY CIRCUIT OR ASPECT OF BRAIN
PROCESSING BUT IS MULTI-SYSTEM
FROM VISION TO LANGUAGE TO MAYBE
ACTUAL REASONING, THIS IS ALSO
MULTI-DOMAIN, IT SPANS DIFFERENT
BRAIN AREAS. ALSO DATA THAT WE
SHOULD COLLECT INVOLVE
CAUSAL MANIPULATION TO GET
US AT SOMETHING TO TURN TO
TREATMENT FOR PEOPLE WITH
DISORDERS.
THE SECOND POINT THAT I THINK WE
NEED IS WE NEED TO HAVE SOME
REFERENCE POINT FOR WHETHER OR
NOT CHANGES TO THE MODELS WILL
MAKE THEM BETTER.
WE'RE PROBABLY ALL DISAGREE ON
METRICS BUT TESTING MODELS IN
COMPARABLE WAY AND SEEING IF
MODEL UPDATE X IS BETTER IS
USEFUL.
FROM EXPERIENCE, VERY EFFICIENT
WAY TO MAKE PROGRESS.
BRAIN SCORE PLATFORM HAS A LOT
OF DATA AND ONE THEME THAT I
WANT TO HIGHLIGHT RECENTLY HAD A
COMPETITION WHERE WE TURNED
AROUND AND ASKED
EXPERIMENTALISTS TO SUBMIT
BENCHMARKS THAT FALSIFY THE
MODELS.
THAT IS INCENTIVE WE NEED MORE,
NOT JUST ABOUT MODELERS, BUT
PRODUCING DATA THAT CAN FALSIFY
MODELS AND SHOW MODELERS AGAIN
WHERE WE NEED TO PUT MORE
RESOURCES.
AND THEN FINAL POINT WE NEED IS
MODELS THAT RESPECT THE
INTRICACIES OF THE BRAIN MORE, I
WANT TO GIVE EXAMPLES.
AGAIN THE THEME OF BREADTH AND
DEPTH.
BREADTH BUILDING ON FEDORENKO'S
REVIEWS, MULTIPLE SYSTEMS, WILL
HELP US TO CONNECT FROM SENSORY
PROCESSING THROUGH REPRESENTATIONS
TO REASONING BEHAVIORS.
I THINK WE SHOULD ALSO RESPECT
MORE THE BIOLOGIC DETAIL THAT
COMES FROM THE BRAIN AND REFERS
TO DIFFERENT THEMES THAT COULD
BE.
ONE THEME WOULD BE TOPOGRAPHY,
THE BRAIN IS SPATIALLY
ORGANIZED, RATHER THAN A CLUMP
OF FACTORS.
BUT I THINK FOR WHAT DEPTH
REALLY MEANS, HOW TO BUILD
MODELS THAT ARE FUNCTION ALIGNED
SHOULD BE A THEME BUT WE SHOULD
AGREE ON TESTING, AT LEAST TELL
WHICH OF THE DIFFERENT CHANGES
LEADS TO IMPROVEMENT IN THE
MODEL SPACE.
AND SO THEN IF WE FOLLOW THROUGH
ON THE THREE POINTS I THINK
THERE'S AN EXCITING TIME FOR THE
FIELD. WE CAN TURN MODELS WE'RE
BUILDING INTO CLINICAL APPLICATIONS
TO ME, THERE'S BEEN TALK OF
FOUNDATION MODELS BUT IF WE HAD
A TRUE FOUNDATION MODEL IN OUR
FIELD IT WOULD ACCEPT DIFFERENT
SENSORY INPUT. VISUAL BEHAVIOR AND
CORRESPONDING ACTIVITY, SINCE WE
WANT TO GO BROADER, WE SHOULD
ALSO INCLUDE A LOT OF DIFFERENT
BEHAVIORS, LANGUAGE ABILITIES THAT
ACCEPTS DIFFERENT SENSORY INPUTS.
DEPTH WOULD ENABLE NEW LEVERS,
FOR INSTANCE NEURAL INTERVENTIONS
THAT PREDICT BEHAVIORAL EFFECT
OF MUSCIMOL OR MICROSTIMULATION
AS YOU STIMULATE SOMEONE'S BRAIN.
>> TWO-MINUTE WARNING.
>> TO GIVE TWO EXAMPLES, LET'S
SAY WE HAVE A PERFECT MODEL,
TURN IT AROUND AND ASK WHAT KIND
OF STIMULI CAN WE PRESENT TO DRIVE
OR SUPPRESS NEURAL ACTIVITY, AND
WE'VE DONE A PRELIMINARY VERSION
IN LANGUAGE, WHERE IF YOU TAKE
THE BEST MODEL THAT'S MOST
PREDICTIVE OF BRAIN DATA IN THE
LANGUAGE SYSTEM AND ASK IT TO
PRODUCE STIMULI THAT DRIVE OR
SUPPRESS ACTIVITY, RUN IN HUMANS,
THAT WORKS WEKK. THIS IS A
FORM OF NON-INVASIVE NEURAL
CONTROL THAT I THINK WITHOUT
THESE MODELS WOULD NOT
BE POSSIBLE. FINALLY THINK OF
MODELS THAT PREDICT NOT JUST NEURAL
ACTIVITY BUT EFFECT OF CAUSAL
INTERVENTION. WHAT IF YOU MICRO-
STIMULATE, APPLY OPTOGENETICS?
WE'VE DONE THIS FOR TOPOGRAPHIC
MODELS IN VISION, APPLY MICRO-
STIMULATION AND IT PREDICTS
BIOLOGICAL DATA QUITE NICELY.
SO TO ME I THINK THE 10-YEAR
PROCESS OF OUR FIELD WILL
CONTINUE TO NEED LARGE SCALE
DATA, WE'LL NEED DETAILED DATA,
STILL DISCUSSING METRICS, SHARED
SET OF REFERENCE POINTS. WE
CAN BUILD ON A.I. BY BRINGING OUR
OWN EXPERTISE IN BUILDING MODELS
TOWARD CLINICAL APPLICATIONS TO
WHERE WE CAN DRIVE A LOT OF
VALUE FOR US AND HUMANS IN
GENERAL.
>> I'D LIKE TO INVITE SESSION 2
DISCUSSANTS TO THE STAGE.
>> THIS IS A GOOD TIME TO SAY IF
YOU'RE NOT SPEAKING YOUR
MICROPHONE SHOULD BE MUTED.
ONLY SO MANY CAN BE ACTIVATED AT
THE SAME TIME.
WE'RE GOING TO HAVE EACH
DISCUSSANT ASK THEIR
QUESTION.
I'LL START WITH MARK HISTED FROM
THE NIMH INTRAMURAL PROGRAM.
>> THANK YOU.
THANKS TO ALL THE SPEAKERS.
THAT WAS VERY I THINK A LOT OF
REALLY INTERESTING IDEAS
INTRODUCED.
I HAVE A QUESTION THAT'S REALLY
FOR THE WHOLE PANEL, INSPIRED BY
A COUPLE DIFFERENT COMMENTS, A
COUPLE DIFFERENT TALKS THAT HAD
SORT OF A COMMON THREAD RUNNING
THROUGH ABOUT THIS KIND OF LIKE
WORSE IS BETTER APPROACH BLAKE
REFERRED TO, TRYING TO BUILD
MODELS THAT WORK AND SEE WHERE
WE GET TO.
I THINK I'M SYMPATHETIC TO THE
IDEA OF CONTROL LOOPS, LEARNING
FROM FEEDBACK, RECURRENT
NETWORKS.
AND WHEN IT COMES TO USING THESE
APPROACHES, WE THINK ABOUT USING
NEUROAI TO UNDERSTAND THE
BRAIN, FROM MY PLACE
IN THE MENTAL HEALTH INSTITUTE,
I'LL SAY WE'VE HAD A LONG EFFORT
OVER THE PAST TWO DECADES TO USE
GENETIC DATA TO UNDERSTAND
MENTAL DISEASE AND BEHAVIOR.
WE HAVE APPLIED A SORT OF DATA-
BASED APPROACH TO COLLECT
GENETIC DATA AND TRY TOPREDICT
DISEASE PATHOLOGY, LET'S SAY,
PREDICT WHEN DISEASE WILL
HAPPEN, ALSO TRY TO DEVELOP WAYS
TO TREAT THEM.
I THINK THERE'S WIDESPREAD
FEELING THAT THAT HASN'T
PRODUCED EFFECTS THAT PEOPLE
WANTED.
I THINK -- SO WHEN WE THINK --
ONE CAN THINK ABOUT THAT FROM
LIKE DATA-CENTERED APPROACH
WHERE THEY JUST TRY TO BRUTE
FORCE THROUGH GENETIC DATA AT
THINGS BUT YOU CAN ALSO THINK
ABOUT THERE'S A LOT OF THEORIES
THAT WERE DEVELOPED AROUND THAT
FROM THE DSM ABOUT DIFFERENT
PATHOLOGY.
MY QUESTION IS GIVEN WHAT WE
HAVE HERE FROM NEUROAI
APPROACHES, HOW CAN WE APPLY
UNDERSTANDING OF NEURAL CIRCUITS
AND NEURAL COMPUTATION AND BRAIN
FUNCTION AND NETWORKS TO MAKE
PROGRESS ON UNDERSTANDING OF
MENTAL DISEASE?
>> FROM MY PERSPECTIVE I THINK
WE'LL DEFINITELY NEED TO GO
BROADER THAN TODAY.
RIGHT NOW THE WAY WE'RE TACKLING
MODELING THE BRAIN, DIVIDE-AND-
CONQUER APPROACH, ONE GROUP ON
VISION, ONE GROUP ON LANGUAGE.
FOR SOMETHING AS COMPLEX AS
MENTAL DISORDERS, MOOD IN
GENERAL, YOU NEED MORE
INTERPLAY.
UNFORTUNATELY THAT WOULD REQUIRE
US TO GO FROM THESE MORE MODULAR
APPROACHES TO A MORE INTEGRATED
LIKE MAYBE TOWARDS THE WHOLE
BRAIN MODEL THAT HAS MORE
COMPLICATED BEHAVIORS, WE'LL HAVE
TO BUILD THOSE KINDS OF MODELS
OURSELVES, I DON'T THINK WE CAN
RELY ON MACHINE LEARNING FOR
THAT BUT BASICALLY GOING MORE
BROAD.
>> I DON'T KNOW IN I'M ALLOWED
TO ANSWER THIS TOO.
ONE THING YOU TALKED ABOUT
TRYING TO PREDICT MENTAL HEALTH
TRAJECTORY WITH GENETIC DATA BUT
I THINK A LOT OF WHAT'S GOING TO
HAPPEN HERE, WE DIDN'T TALK
ABOUT COMPUTATIONAL PSYCHIATRY,
OR EVEN THE BRAIN BEHAVIORAL
QUANTIFICATION SYNCHRONIZATION BIT
WHERE WE'RE NOT JUST USING DATA
TO DERIVE SOME DATA COLLECTING
IN THE RESEARCH LAB TO BUILD
MODELS BUT ACTUALLY USING DATA
COLLECTED IN THE WILD, ALSO
USING DATA COLLECTED FROM
MEDICAL HEALTH RECORDS, TRYING
TO PUT THEM IN. DATA THAT COMES
FROM THOSE POPULATIONS TENDS TO
BE NOT REPRESENTATIVE, SO IF
WE'RE BUILDING MODELS BASED ON
BRAIN DATA THAT ISN'T
REPRESENTATIVE WE'LL HAVE EQUAL
HOLES, NOT JUST THAT DATA ISN'T
REPRESENTATIVE, NOT INCLUDE SOCIAL
FACTORS THAT INFLUENCE DISORDERS.
>> THERE'S A GAP BETWEEN GENE
AND BEHAVIOR, SEVERAL
INTERMEDIATE LEVELS, I THINK
THAT'S ONE OF THE BIGGEST
REASONS FOR FAILURE.
AT LEAST WE HAVE GOOD
BIOPHYSICAL MODEL OF NEURONAL
AND CIRCUIT LEVEL. LOOK AT IMPACT
OF THE GENE ON THE COMPUTATION.
HAVE A MECHECHANISTIC UNDER-
STANDING OR HYPOTHESIS AND THEN
GO TO BIGGER A.I. MASSIVE
APPROACH OF MULTIPLE BRAIN AREAS AND INTEGRATE THESE TWO APPROACHES
TO COME UP WITH BETTER INSIGHTS.
>> TO BRIDGE THAT GAP
SPECIFICALLY WHAT DO WE SEE
THESE KIND OF NEUROAI
APPROACHES THAT WE'VE DISCUSSED
TODAY DOING? LIKE WHERE ARE THEY
POINTING?
TO ME IT FEELS OPTIMISTIC.
I FEEL LIKE THIS IS THE
DIRECTION THAT'S GOING TO GIVE
US UNDERSTANDING OF NEURAL
COMPUTATIONS, GOING TO LEAD TO
DEEPER UNDERSTANDING OF WHAT'S
HAPPENING, I WONDER WHAT THE
PANEL THINKS, THE REST OF
EVERYONE.
>> WHY IS ONE PERSON DEPRESSING,
ANOTHER HAS FULL HOPE, WHY DO
YOU SEE AN IMAGE AS A FACE AT
ONE MOMENT AND A VASE AT
ANOTHER, COULD BE THE SAME BASIC
MECHANISM TOP-DOWN PRIORS, ONE OF
THE MOST AMAZING INSIGHTS FROM A.I.
HAS BEEN HOW THE SAME
ARCHITECTURE LIKE TRANSFORMERS
CAN SOLVE SUCH DIVERSE PROBLEMS,
I SUSPECT WHEN WE UNDERSTAND THE
BASIC MECHANISMS FOR EVEN
VISUAL PERCEPTION
IT'S GOING TO GIVE
BIG CLUES INTO MENTAL ILLNESS.
>> I'D LIKE TO TRANSITION TO THE
NEXT DISCUSSANT, FROM YALE
UNIVERSITY, STEVE ZUCKER.
>> I LOVED THE TALKS THIS
AFTERNOON.
DORIS PROVIDED TWO EXAMPLES OF
DEEP NETWORKS THAT SHE USED IN
HER WORK TO GAIN INSPIRATION FOR
WHAT WAS GOING IN NEUROSCIENCE,
AND IN HER WORDS SHE SAID DEEP
NETWORKS LET US DESCRIBE COMPLEX
OBJECTS.
AT THE END OF THE SESSION,
MARTIN TALKED ABOUT BRAIN SCORES
AND THE NEED FOR METRICS AND IN
FACT SEVERAL PEOPLE MENTIONED
THE NEED FOR METRICS TO COMPARE
SYSTEMS.
SO MY QUESTION IS, AND LET ME
PHRASE IT TWO WAYS.
INFORMALLY, DO SUCH METRICS
EXIST?
TECHNICALLY OVER WHAT SPACE OF
MODELS MIGHT ONE FIND THE
METRIC?
TO DO WHAT DORIS DID WHICH IS
TO GAIN INSPIRATION FROM AN
ARTIFACT, DO WE ACTUALLY NEED
METRICS?
THE FOURTH VERSION OF THE SAME
QUESTION IS DOES IT MAKE SENSE
TO RANK NETWORKS LIKE WE RANK
SWIMMERS?
SO WHO DID 100 METERS OF
FREESTYLE THE FASTEST?
THEY ARE ALL THE SAME QUESTION
BUT WITH DIFFERENT CLOTHING.
I DON'T KNOW, YEAH.
>> MAYBE I CAN START.
I MEAN SHORT QUESTION, WE NEED
METRICS, DEFINITELY YES TO
ASSESS PRODUCTS, WHETHER WE'RE
IMPROVING, WE NEED TO HAVE A
WAY TO MEASURE IMPROVEMENT.
I DON'T THINK IT'S AN EASY
ANSWER.
I DON'T KNOW WHAT THE RIGHT
METRIC IS.
I THINK IT DEPENDS A LOT ON KIND
OF NETWORKS, EASIER TO COMPARE
LET'S SAY WITHIN THE SAME SYSTEM
ARCHITECTURE, LEARNING, WHAT IS
THE ADVANTAGE OF ADDING MORE
BIOLOGY TO THESE NETWORKS, IT'S
AN EASIER COMPARISON, LET'S SAY
THAN GOING TO MULTIPLE DIFFERENT
MODELS SOLVING THE SAME TASK
BECAUSE THEN IT DEPENDS ON
ARCHITECTURURAL DIFFERENCES, SO
THAT WOULD BE MORE DIFFICULT TASK
TO SOLVE. WHAT'S IMPORTANT IS TO
CONSIDER THE BRAIN IS OPERATING
UNDER A SET OF CONSTRAINTS,
ENERGY, SPACE, SPEED
LET'SSAY.
SO THE METRICS THAT WE'RE USING
TO COMPARE A.I. AND BIOLOGICAL
SYSTEMS SHOULD OPERATE UNDER
VERY SIMILAR CONSTRAINTS FOR
THIS TO MAKE SENSE.
OTHERWISE IF WE DON'T HAVE THE
CONSTRAINTS WHAT IS THE
COMPARISON, IT'S NOT REALLY
FAIR, SO I DON'T HAVE AN ANSWER
BUT THESE ARE ALL THINGS THAT WE
NEED TO CONSIDER CAREFULLY TO
COME UP WITH THE RIGHT METRICS.
>> I CAN ADD TO THIS QUICKLY?
I HAVE A COUPLE POINTS IN
RESPONSE.
ONE IS THAT I THINK IT'S
BECOMING LESS USEFUL PERHAPS
TO JUST TAKE OFF-THE-SHELF
MODELS AGAINST NEURAL DATA.
RELATED TO WHAT DORIS WAS
POINTING TO. IF WE TREAT MODEL
ARCHITECTURES AS A WAY TO TEST
HYPOTHESES, WE CAN CREATE MINIMAL
PAIR VARIANTS. SO MANY THINGS
VARY ACROSS MODELS BUT WE CAN
MAKE MINIMAL PAIRS WHICH VARY IN
THE PRESENCE OR ABSENCE OF
INDUCTIVE BIAS OR IN THE PRESENCE
OF CERTAIN ARCHITECTURAL MOTIF AND
CONNECTIVITY PATTERN OR PRESENCE
OF BIOLOGICAL FEATURES, AND SEE
NECESSARY AND SUFFICIENT CONSTRAINTS
TO YIELD A REPRESENTATIONAL SPACE
OR HOW ALGORITHMS ARE INSTANTIATED.
ANOTHER SMALL POINT, MAYBE NOT A
SMALL POINT --
>> I HAVE TO INTERRUPT.
REPRESENTATIONAL SPACE, WE'RE
COMPARING REPRESENTATIONS, ISN'T
THAT A VERSION OF THE SAME
PROBLEM?
>> WHAT DO YOU MEAN BY THAT.
>> TO COMPARE MODELS OR
REPRESENTATIONS IN TWO MODELS,
TO CLAIM THERE IS SOME METRIC
BETWEEN THEM, AREN'T THEY
EQUIVALENT PROBLEMS?
>> PEOPLE HAVE SOLVED SOME OF
THESE COMPARISON ISSUES, LIKE
THERE ARE WAYS TO COMPARE
REPRESENTATIONS WITH MULTIPLE
DIMENSIONS LIKE IT'S NOT A
TECHNICAL ISSUE.
ARE YOU ASKING ABOUT A
CONCEPTUAL PROBLEM?
>> YES, MATHEMATICAL QUESTION,
OVER WHAT SPACE ARE METRICS
DEFINED?
>> WELL, BLAKE DO YOU WANT TO GO
>> I'D LOVE TO JUMP IN.
THERE'S TWO THINGS.
FIRST TO ANSWER MOST DIRECTLY
YOUR QUESTION, YOU JUST ASKED
NOW, THEY ARE COMPARED IN THE
SPACE OF THE VECTORS THAT WE
TAKE TO BE THE REPRESENTATIONS
OF EITHER THE MODEL OR THE
BRAIN.
IN BOTH CASES WE'RE MAKING SOME
ASSUMPTIONS ABOUT THE NATURE OF
INFORMATION ENCODING, IN THE
MODEL OR SUBJECT WE'RE STUDYING.
THIS HAS BEEN A FRUITFUL WAY OF
STUDYING THE BRAIN, FRUITFUL WAY
OF BUILDING A.I. MODELS,
SUFFICIENTLY GROUNDED AT THIS
POINT IN TIME DUE TO SCIENTIFIC
SUCCESSES, NOT AN INAPPROPRIATE
ASSUMPTION NAMELY WE CAN
UNDERSTAND INFORMATION ENCODING
IN THE BRAIN AS RELATING TO A
VECTOR OF NEURAL ACTIVITY, LIKE-
WISE WE CAN UNDERSTAND INFORMATION
ENCODING IN ANN AS VECTOR
ACTIVATIONS OF EACH UNIT IN THE
NETWORK AND COMPARE THESE SPACES.
TO GET TO THE BROADER QUESTION,
THE IMPORTANT THING TO REMEMBER
METRICS SHOULD NOT BE VIEWED AS
A WAY OF MEASURING WHO IS THE
BEST SWIMMER.
IN FACT I THINK THAT'S THE EXACT
WRONG WAY TO VIEW THESE SCORES.
THE WAY TO UNDERSTAND THESE
SCORES IS THAT EACH DIFFERENT
METRIC PROVIDES A DIFFERENT
INSIGHT INTO HOW WELL THE MODEL
IS CAPTURING SOME ASPECT OF THE
BRAIN.
SO, WHEN YOU DO REPRESENTATIONAL
SIMILARITY ANALYSIS, YOU ARE
GETTING AT THE QUESTION OF
WHETHER OR NOT THE
REPRESENTATIONS IN THOSE SPACES
THAT I JUST MENTIONED HAVE
SIMILAR GEOMETRY.
IT'S A VERY SPECIFIC QUESTION
YOU'RE ASKING.
IT'S NOT IS THIS THE BEST MODEL.
IT IS, DOES THIS MODEL HAVE THE
BEST GEOMETRICAL ALIGNMENT TO
THAT OF THE BRAIN?
LIKEWISE, IF YOU HAVE BEHAVIORAL
SCORES, YOU ARE ASKING A
QUESTION, TO WHAT EXTENT CAN THE
MODEL REPLICATE HUMAN BEHAVIOR,
ANDHUMAN COMPUTATION?
SO, I THINK THAT THE WAY WE NEED
TO MOVE FORWARD ON THIS IS TO
HAVE AN ENTIRE LIKE BASKET OF
METRICS, EACH ONE PROVIDING
DIFFERENT INSIGHT INTO THE MODEL
IN ITS RELATIONSHIP TO THE
BRAIN, EACH HELPING US GUIDE
SCIENTIFIC ADVANCES.
YOU SHOULD NEVER VIEW AS WHO IS
THE FASTERS SWIMMER, BECAUSE
THAT WOULD KILL SCIENCE.
>> IN DEFENSE OF THE SWIMMER, I
AGREE SHOUD NOT RELY ON ONE METRIC
>> WE WILL GO TO THE NEXT DISCUSSANT
>> SHOULD I SHUT UP?
>> GO AHEAD.
>> SO THE NEXT DISCUSSANT IS
JOSH VOGELSTEIN FROM JOHNS
HOPKINS UNIVERSITY.
>> OKAY.
THANK YOU FOR INVITING ME HERE.
IT'S BEEN FUN LISTENING TO YOU
ARGUE.
I'M EXCITED.
I WAS TOLD TO ASK SOMETHING
ARGUMENTATIVE EXACTLY, I DON'T
REMEMBER EXACT THE WORDS,
PARTICULARLY WITH MARTIN AND
BLAKE.
I LIKE WHAT WAS SAID AND
FIGHTING WITH BLAKE, ANECDOTE.
I TAKE ISSUE.
HISTORICALLY CYBERNETICS, AND
GOFAI PEOPLE ENGINEERED STUFF THAT
WORKS, THOSE PEOPLE LEFT THE
CYBERNETICS DOGMA OF FEEDBACK
LOOPS AND CONTROL AND JUMPED INTO
COMPUTATIONAL STATISTICS, AND
PEOPLE NOT AT THE FIRST A.I.
MEETING DEVELOPED NON-PARAMETRIC
STATISTICS, THOSE THINGS WORKS,
WORK REALLY WELL, BETTER THAN
EITHER OF THE CONNECTIONIST
STUFF OR GOFAI STUFF IN PRACTICE.
TO DORIS POINT ABOUT TRANSFORMERS,
THERE'S A NADARIA-WATSON KERNEL
CAME OUT OF THE FIELD FROM THAT
IN THE '60s.
SO MY ARGUMENT IS ACTUALLY MAYBE
WHAT'S HAPPENING NOW IN NEUROAI
IS A BUNCH OF PEOPLE IN
NEUROSCIENCE REALIZING THAT
COMPUTATIONAL STATISTICS ARE
VALUABLE FOR CHARACTERIZING,
EXPLAINING NEUROSCIENCE LIKE
MANY SPEAKERS HAVE DONE, AND A
BUNCH OF PEOPLE IN A.I. MAKING
NEW MODELS AND VAGUELY HAND-
WAVINGLY SAYING THESE THINGS ARE
RELATED TO BRAINS.
>> I'D LIKE TO COMMENT ON THIS.
CERTAINLY I'M WELL WITH YOU
TO THE POINT, YES, LIKE
THERE ARE OTHER WAYS OF
UNDERSTANDING KIND OF THE
ADVANCES THAT HAVE BEEN MADE IN
MACHINE LEARNING THAT DON'T
RELATE DIRECTLY TO THE HERITAGE
OF CYBERNETIC AND CONNECTIONISM.
MAYBE THIS ALSO ADDRESSES A
LITTLE BIT SOMETHING MARTIN SAID
THAT I WANT TO ATTEND TO, WHICH
IS THAT IT'S WORTH RECOGNIZING
THE HISTORY OF A LOT OF THE
IDEAS IN A.I. AND HOW THEY TRACE
THEIR ROOTS BACK TO
COMPUTATIONAL NEUROSCIENCE.
LET'S TALK ABOUT THE
TRANSFORMER.
EVEN THE TRANSFORMER, MANY SAY
IT DOESN'T WORK LIKE THE BRAIN,
IF YOU FOLLOW CITATION TRAIL
THROUGH THE TRANSFORMER PAPER IT
COMES FROM IDEAS THAT YOSHUA
BENGIO HAD ABOUT SOFT ATTENTION,
WHICH CAME FROM IDEAS CHRISTOF
KOCH HAD ABOUT ATTENTION IN THE
BRAIN.
THERE'S A CLEAR HERITAGE FROM
NEUROSCIENCE TO TRANSFORMERS
THAT IS JUST THERE IN THE RECORD
FOR EVERYONE TO SEE.
MOREOVER, YOU MENTIONED THAT
TRANSFORMERS CAN BE ANALYZED IN
KERNEL METHOD, BUT THEY CAN ALSO
BE VIEWED AS HOPFIELD NETWORKS.
SOFT ATTENTION MECHANISMS OF
TRANSFORMERS ARE EQUIVALENT TO
HOPFILED NETWORKS WHICH ARE
DIRECTLY INSPIRED BY THE BRAIN.
EVEN IN A.I. BROADLY THOUGH
WE'VE SEEN THE FIELD PERHAPS PAY
LESS ATTENTION TO NEUROSCIENCE
MORE GENERALLY AS MORE MONEY AND
ENGINEERS PILE INTO THAT FIELD,
A LOT OF LUMINARIES IN A.I. ARE
COMMITTED THAT NEUROSCIENCE IS
A KEY SOURCE OF INSPIRATION.
THEY KNOW THE HISTORY OF THESE
IDEAS DIDN'T JUST LEARN THEM ON
A TUTORIAL ON GOOGLE. THEY
FOLLOWED THE ACADEMIC RECORD.
DEMIS, GEORF, RICH SUTTON, THEY
WILL TELL YOU, YOU GOT TO PAY
ATTENTION TO THE BRAIN, THERE'S
FUNDAMENTAL DEEP RELATIONSHIPS
BETWEEN AND THE COMPUTATION,
LIKE A TRANSFORMER.
I GOT CUT OFF.
>> DO YOU WANT TO BORROW THIS
ONE?
THIS ONE WORKS.
>> SO, FANTASTIC CONVERSATION.
ENGINEERING BACKGROUND TO
MEASURE THE BRAIN, ALSO BUILD
HARDWARE TO SIMULATE THE BRAIN
DYNAMICS.
MY QUESTION HAVE TWO-FOLD.
WE HAVE THIS BRAIN INITIATIVE,
COLLECT TONS OF DATA, AND REALLY
THE QUESTION, BECAUSE THOSE DATA
WHEN THEY ARE COLLECTED, THEY
ARE NOT LIKE -- I MEAN FOR
NEUROAI, THOSE INTERESTED TO
UNDERSTAND BRAIN FUNCTION, BRAIN
PHYSICAL STRUCTURE, THOSE DATA
ARE GOOD ENOUGH, YOU KNOW, FOR
US TO DO THOSE LIKE NEUROAI
STUDY OR SINCE WE HAVE THE TOOLS
WE SHOULD LIKE VERY SPECIFICALLY
COLLECT ANOTHER SET OF THE DATA,
SPECIFICALLY LIKE NEUROAI
AND IF WE NEED WHAT KIND OF DATA
YOU THINK IS MOST IMPORTANT
FOR YOU RIGHT NOW.
SECOND YES, QUESTION, ON THE OR
HAND, TRYING BUILD MODEL BASED
ON THAT DATA DO YOU THINK THE
INFRASTRUCTURE WE HAVE NOW DEEP
DURAL NETWORK IS GOOD ENOUGH,
NOT GOOD ENOUGH BUT A DIRECTION
WE SHOULD GO, DO YOU THINK THAT
MAKES SENSE RIGHT NOW TO
ACTUALLY BECAUSE NEUROSCIENTISTS
RIGHT NOW REALIZE HOW IMPORTANT
A.I. IS IMPORTANT BUT PROBABLY
NOT ENOUGH FOR ACTUALLY HOW
IMPORTANT TO DESIGN FROM
SCRATCH, DO YOU THINK WE SHOULD
LIKE BRING MORE NEUROSCIENCE TO
SEMICONDUCTOR MANUFACTURING OR
DESIGN, TO CHIP DESIGN INDUSTRY?
THANK YOU.
>> I CAN TRY TO ANSWER THE FIRST
PART ABOUT DATA.
SO, I THINK THE DATA THAT WE'VE
ALREADY COLLECTED IN BRAIN
INITIATIVE IS VALUABLE,
DEFINITELY MORE DATA WOULD BE
BETTER.
I LIKE THE POINT FROM THE
MORNING COLLECTING UNDER
NATURALISTIC, HIGH ENTROPY, BUT
I FEEL LIKE THIS NOTION IS
CREEPING UP, NEUROSCIENTISTS ARE
JUST THERE TO INSTRUMENT ANIMAL
BRAINS AND COLLECT DATA AND
WE'LL PASS TO SOME A.I. PERSON
WHO CAN USE THAT DATA.
I REBEL AGAINST THAT NOTION.
I WANT TO PUT IN A PITCH FOR
OLD-FASHIONED HYPOTHESIS
TESTING, STRUCTURAL INNATE
PRIORS, AND YOU CAN'T PUT A
PROBE IN EVERY MILLIMETER OF A
MONKEY'S BRAIN, HAVE YOU TO BE
GUIDED BY HYPOTHESES, WE'RE
TESTING WITH THE IDEAS, WHICH IS
SHOULD BE INCORPORATED INTO
THE NEXT NEURAL NETWORK, NOT
JUST COLLECTING DATA.
>> I WAS GOING TO SAY SOMETHING
TOO.
I THINK THE QUESTION OF THE
ARCHITECTURE IS A REALLY
IMPORTANT SOMETHING WHERE
THERE'S A LOT WE CAN LEARN FROM
THE DATA THAT'S ALREADY BEEN
COLLECTED.
I THINK THAT PERSONALLY I FEEL
LIKE ENORMOUS PROGRESS CAN BE
MADE BY STUDYING DIFFERENT
ARCHITECTURES, WHAT THEY CAN DO,
GETTING AWAY FROM, YOU KNOW, THE
STANDARD CONVOLUTIONAL NEURAL
NETWORK AND EXPLORING GETTING
INSPIRATION LIKE THE FLY
CONNECTOME AND MATHEMATICAL
EXPLORATION.
>> IT DEPENDS ON THE LEVEL
OF ABSTRACTION. WITH THE IMAGING
TECHNIQUES WE STILL LACK A LOT
OF DATA ON SUBCELLULAR LEVEL
WHICH PEOPLE USED TO PROBE WITH
PATCH ELECTRODES TEN YEARS AGO.
THEY DON'T DO THAT ANYMORE.
SO WE DON'T HAVE A LOT OF
SUBCELLULAR DATA. COUNTING ON
VOLTAGE IMAGING. MAYBE WHEN
THAT COMES OF AGE WE'LL HAVE A
LOT OF DATA TO UNDERSTAND WHAT
HAPPENS AT THE SUBCELLULAR LEVEL
WHICH IS CRITICAL BUT I ALSO
LIKE ANTON'S IDEA IN PREVIOUS
SESSION, HE SAID WE CAN USE A.I.
TO PREDICT WHAT THE DATA WOULD
LOOK LIKE, LET'S SAY FOR
DIFFERENT CELL TYPES, IF WE HAVE
DATA FOR SOME OF THE NEURONS,
LET'S SAY, AND GO BACK AND CLOSE
THE LOOP AND THAT WOULD BE
REALLY USEFUL TO DO, TO BRING
A.I. TO THE SERVICE OF
NEUROSCIENCE LET'S SAY TO
COMPLEMENT EXPERIMENTS.
>> SO THIS IS A GOOD TIME TO
INTRODUCE A QUESTION FROM ZOOM.
IT FOLLOWS REALLY NICELY FROM
WHAT WAS JUST SAID.
TO WHAT EXTENT DO MORE ADVANCED
ARCHITECTURES NECESSITATE
DIFFERENT APPROACHES TO MODEL
PERFORMANCE VALIDATION?
THAT IS TO WHAT EXTENTS ARE
APPROPRIATE A.I. MODEL
VALIDATION TECHNIQUES DRIVEN BY
THE MODEL'S ARCHITECTURE VERSUS
DATA STRUCTURES OR APPLICATIONS?
THAT'S FOR ANYONE.
>> I'LL MAKE ONE COMMENT, WHICH
IS, THIS GETS BACK TO DIFFERENT
METRICS.
INDEED IF YOUR GOAL IS TO
CAPTURE SOMETHING MORE
SPECIFICALLY RELATED TO THE
BIOLOGICAL IMPLEMENTATION OF
COMPUTATION AND INTELLIGENCE IN
THE BRAIN YOU NEED METRICS THAT
SPEAK TO THAT GOAL.
IF YOU'RE TRYING TO GET MORE
BRAIN-LIKE MODELS, BY BUILDING
IN SOME OF THESE BIOLOGICAL
DETAILS LIKE DENDRITES, GLIA,
WHATEVER, PART OF WHAT YOU
SHOULD DO IS NOT JUST LOOK AT
BEHAVIORAL PERFORMANCE OR
ALIGNMENT AND REPRESENTATIONAL
SPACE BUT THINGS LIKE LOW LEVEL
DYNAMICS OVER TIME. THE
TRAJECTORY OF SYNAPSES THROUGH
LEARNING, MAYBE EVEN THINGS I'M
NOT SUCCESSFULLY GENERATING ON
THE SPOT HERE.
BUT I THINK THAT IT IS KEY THAT
YOU ASK THE QUESTION - FOR A
QUESTION THAT YOU WANT TO ASK
RELATED TO ARE WE MODELING THE
BRAIN SUCCESSFULLY WITH THIS MORE
DETAILED MODEL YOU NEED THE RIGHT
METRICS FOR THAT.
>> I HAVE DISCUSSION OF DETAILED
METRICS.
THIS RELATES TO THE QUESTION
WE DISCUSSED IN THE MORNING YOU
WHO DO WE KNOW WHEN WE'RE DONE?
I DON'T THINK IT'S WHEN A METRIC
PASSES A THRESHOLD.
I THINK IT'S WHEN WE CAN BUILD
BASED ON OUR KNOWLEDGE LIKE SOME
AMAZING ARTIFICIAL INTELLIGENCE
THAT CAN DO THINGS THAT WE JUST
WOULD NOT HAVE FATHOMED BEFORE,
RIGHT?
LIKE THAT'S GOING TO BE THE
METRIC.
>> HOW DO YOU GET THERE?
>> THIS ALSO GETS TO THE
QUESTION I WAS THINKING ABOUT,
HOW CAN THESE METRICS TAKE INTO
ACCOUNT MULTIPLE LAYERS, AND
CONNECTIONS BETWEEN THOSE
MULTIPLE LAYERS WHEN LOOKING AT
MODEL-TO-BRAIN SIMILARITIES.
>> TECHNICALLY YOU LOOK ACROSS
LAYERS, FIND THOSE THAT DO BEST,
GET INFERENCES WHY THE LATEST
MIGHT DO BEST AND VALIDATE
INDEPENDENT DATA.
BUT THERE'S INTERESTING
QUESTIONS ABOUT WHAT KIND OF
INFORMATION CONTAINED IN CROSS
LAYERS, CAN I COME BACK TO A
POINT ABOUT MODEL COMPARISONS?
THE THING I WANTED TO ADD,
ANDREAS MENTIONED EARLY IN THE
TALK THIS MORNING THAT THERE'S
ALSO THIS PHENOMENON THAT'S
HAPPENING ACROSS SEVERAL DOMAINS
WHERE REPRESENTATIONS ACROSS
HIGH PERFORMING MODELS SEEM TO
BE CONVERGING ON THE SAME AXIS.
THIS SEEMS REALLY IMPORTANT.
HOSSEINI HAS A POSTER ON THIS.
IF YOU WANT TO HEAR MORE IN THE
CONTEXT OF LANGUAGE AND VISION,
NO MATTER WHAT GIVEN THIS PROBLEM
THAT YOU'RE SOLVING YOU'RE
ALIGNING ONTO THE SAME SPACE.
THIS TELLS YOU SOMETHING REALLY
DEEP ABOUT WHAT IS NEEDED TO
SOLVE A PARTICULAR PROBLEM
SUCCESSFULLY.
>> THIS IS SOMETHING --
THERE'S A THEORY RESULT THAT
TRANSFORMERS AND LSTMS ARE
EQUIVALENT IN REPRESENTATION IN
COMPARABLE TRAINING CONDITIONS.
COMES BACK TO HOW WE SHOULD
COLLECT, DO HYPOTHESIS TESTING.
BUT HYPOTHESES NEED TO BE ON THE LEVEL OF HOW THE WHOLE REPRESENTATIONAL SPACE HAS BEEN OPTIMIZED
IN THE FIRST PLACE, WHICH NECESSITATES MODELS WHICH
NECESSITATES US TO USE THE TOP
SWIMMERS, PIT THEM AGAINST ONE
ANOTHER, AND FOR THAT WE NEED TO
MAKE PROGRESS ON METRICS AND
PROBABLY WE NEED MANY OF THOSE,
PROBABLY NEED A MODEL, MOST
MODELS WILL NOT DO WELL IN MANY
OF THOSE, WE NEED TO PUT THOSE
REFERENCES UP BECAUSE OTHERWISE
I DON'T SEE US MAKING PROGRESS.
WE'LL SEE WHEN WE GET THERE BUT
THE GETTING THERE PART IS GOING
TO BE DIFFICULT-- IF WE DON'T HAVE
NUMERICAL GOALPOSTS TO TRACE
DOWN WITH.
>> WE HAVE FIVE MINUTES LEFT.
I'D LIKE TO OPEN UP TO Q&A BOTH
IN THE ROOM AND QUESTIONS FROM
THE VIRTUAL AUDIENCE.
>> HELLO.
I THANK YOU FOR YOUR WONDERFUL
PRESENTATIONS. MY NAME IS XINHE
ZHANG. I'M A GRADUATE STUDENT
FROM HARVARD
UNIVERSITY. WE'VE TALKED ABOUT
STRUCTURALLY HOW CAN WE IMPLEMENT
BRAIN-LIKE MODULES OR LAYERS INTO
THE NETWORK BUT BIOLOGICAL
LEARNING RULES ARE LACKING IN
THAT SENSE. FOR EXAMPLE, HEBBIAN
LEARNING RULES AREN'T PERFORMING AS
WELL AS HARDENDED-AND-TRUE
BACKPROPAGATION.
WHAT TECHNOLOGICAL ADVANCEMENTS
HARDWARE AND SOFTWARE-WISE CAN
HELP WITH THE DISCOVERIES OF
MORE BIOLOGICALLY EFFICIENT AND
PLAUSIBLE LEARNING RULES, AND
SECOND, DO YOU THINK THERE'S A
GLOBAL LEARNING RULE GOVERNING ALL
REGIONS OR DO YOU THINK THERE'S
SOME INSTRUMENTAL EFFECT OF
INDIVIDUAL BRAIN REGIONS, AND
COLLECTIVELY THEY LEARN IN THE
SHORT TERM AND LONG TERM, SOME
BEHAVIORS OR TASKS?
THANK YOU.
>> MAYBE I CAN START WITH THAT.
EXCELLENT QUESTION.
INDEED BIOLOGICAL LEARNING RULES
RIGHT NOW ARE NOT DOING AS WELL
AS GRADIENT DESCENT BUT MAYBE THAT'S
THE WRONG QUESTION TO ASK.
AS WE NOTICE FROM THE
PRESENTATION EARLIER TODAY BY
TONY, FOR EXAMPLE, HUMANS AND
ANIMALS LEARN AT DIFFERENT
SCALES. WE HAVE
EVOLUTIONARY SCALE, DEVELOPMENTAL
SCALE, AND THEN LEARNING WHICH
IS THE CLOSEST TO WHAT
ARTIFICIAL NEURONS ARE USING.
GRADIENT DESCENT
IS IDEAL FOR THE LAST STAGE
BUT WHAT BIOLOGICAL LEARNING
RULES ARE DOING IS OPERATING AT
DIFFERENT OR MULTIPLE SCALES AT
THE SAME TIME AND WE DON'T HAVE
A CLEAR UNDERSTAND HOW THAT WORKS
IN THE BRAIN RIGHT NOW AND
I THINK THAT IS WHAT WE'RE
MISSING IN ORDER TO TRANSFER
THIS KNOWLEDGE TO CLASSICAL A.I.
SO THAT'S ONE ANSWER.
I DON'T THINK IT'S A
TECHNOLOGICAL PROBLEM.
MAYBE IF WE HAVE BETTER BETTER
HARDWARE TO MIMIC HOW THE BRAIN
IS USING THE LEARNING RULES,
THAT WOULD PROVIDE NEW INSIGHTS,
BUT I THINK IT'S MOSTLY A
CONCEPTUAL PROBLEM AT THIS
MOMENT.
WE NEED TO LOOK AT RULES,
BIOLOGICAL ONES, FROM A
DIFFERENT PERSPECTIVE.
AND THERE ARE STUDIES RIGHT NOW
LOOKING AT SELF-SUPERVISED
LEARNING IF YOU LIKE, OR
STATISTICAL LEARNING FOR WHICH
THESE HEBBIAN-LIKE RULES ARE BEST.
I THINK WE STILL HAVE A LONG WAY
TO GO UNTIL WE SEE THEM
IMPLEMENTED SUCCESSFULLY IN
NEUROAI SYSTEMS BUT WE
UNDERSTAND WHY THEY DON'T WORK
WHEN USING THEM IN THE CONCEPT
OF PERFORMANCE ASSESSMENT.
>> TO JUMP IN, TO ACTUALLY ECHO
THE COMMENT I HEARD BLAKE MADE,
WHICH IS THAT IF YOU LOOK AT THE
THE RESEARCH IN NEUROSCIENCE
OVER THE LAST 20 YEARS, 10 YEARS
ESPECIALLY, ALMOST ALL OF IT IS
BASED ON GRADIENT DESCENT.
NOW IT'S RECURRENT NETWORKS OF
GRADIENT DESCENT.
BUT THERE MAY BE A REASON FOR
THAT.
I THINK THE BRAIN IS USING
GRADIENT DESCENT, MAYBE NOT BACK
PROP BUT IT IS TAKING
ADVANTAGE OF ERROR SIGNALS,
RIGHT?
AND THIS IS RELATED I THINK TO
THE FACT THAT IT'S REALLY
DYNAMICS THAT WE'RE TRYING TO
UNDERSTAND, THE BRAIN DYNAMICS,
AND THAT INVOLVES FEEDBACK
LOOPS, RECURRENCE, LAYER 2-3 IN
THE CORTEX, CA3 IN HIPPOCAMPUS.
CARINA MADE A BEAUTIFUL MODEL,
QUITE REMARKABLE, IF YOU DIDN'T
APPRECIATE IT, YOU MAY NOT HAVE
APPRECIATED IT, HOW REMARKABLE
IT IS, RELATIVELY SMALL MODEL,
YOU CAN EMBED FOUR DIFFERENT
DYNAMICAL PATTERNS OF ACTIVITY
AND I THINK THIS IS GOING TO BE
A GENERAL INSIGHT INTO HOW
BRAINS ARE ORGANIZED, THAT IS TO
SAY THEY ARE LOW DIMENSIONAL.
IN FACT, ONE
DIMENSIONAL.
FOUR DIFFERENT ONE-DIMENSIONAL
ATTRACTORS IN THAT NETWORK.
THERE'S A LOT OF EVIDENCE NOW
THAT FROM COMPLEX COGNITIVE
TASKS DONE IN RODENTS THAT THE
DIMENSIONALITY OF THE MANIFOLDS
THAT IS USED TO SOLVE THE TASK
IS LOW DIMENSIONAL, ON THE ORDER
OF FOUR TO SIX.
NOW, IF THAT'S TRUE WE'RE NOT
DEALING WITH NUMBERS
LIKE TEN TO THE TENTH ANYMORE.
WE'RE DEALING WITH LOW
DIMENSIONAL MANIFOLDS EMBEDDED
IN A LARGE NUMBER OF NEURONS,
BUT I THINK THAT MAKES IT MORE
TRACTABLE AND I THINK THAT MAYBE
THIS IS THE WAY TO GO, RATHER
THAN TRYING TO COMPARE THINGS TO
FEEDFORWARD NETWORKS.
VISION IS NOT IN OUR BRAINS
IT'S NOT FEEDFORWARD, IT'S AN
INCREDIBLE AMOUNT OF FEEDBACK.
>> I WANT TO FOLLOW THAT UP WITH
THEY'VE ARE ALREADY
SAID WHAT I WOULD HAVE SAID, BUT
THERE IS ONE TECHNOLOGICAL
INNOVATION I WANT TO SAY HERE AT
THE NIH WE NEED TO REALLY NAIL
THIS QUESTION.
WE NEED TO BE ABLE TO TRACK
SYNAPTIC WEIGHTS IN LEARNING
ANIMALS.
HONESTLY, IF I COULD GET ANY ONE
TOOL IN THE NEXT TEN YEARS TO
SOLVE THIS PROBLEM, IT WOULD BE
THAT.
>> SO WE'RE PRETTY MUCH OUT OF
TIME.
I THOUGHT IT WOULD BE A NICE
IDEA TO TAKE ONE OF THE
QUESTIONS WE DIDN'T GET TO AND
ACTUALLY TAKE IT FOR THE ROAD.
SO Y'ALL CAN DISCUSS DURING THE
COFFEE BREAK, BATHROOM BREAK, SO
THE QUESTION IS MULTIPLE
TEMPORAL SCALES FOR LEARNING AND
INFERENCE SEEM TO BE MISSING IN
MOST CURRENT MODELS AND MANY
NEUROSCIENCE APPROACHES.
DO YOU SEE THAT AS SOMETHING
THAT IS CRITICAL TO
UNDERSTANDING THE BRAIN AND BY
CORRESPONDENCE IN KEEPING WITH
THE THOUGHT PROCESSES OF THE
PANEL HERE WHAT IS NEEDED TO
MOVE MODELS IN NEUROSCIENCE
DATASETS TOWARDS ADDRESSING
MULTIPLE TEMPORAL SCALES OF
ECOLOGICAL BEHAVIOR?
>> THANK YOU SO MUCH, ALL
SPEAKERS AND DISCUSSANTS AND THE
AUDIENCE FOR YOUR PARTICIPATION AS WELL.
>> THE FIRST HALF AN HOUR WILL
GO TOWARDS ANSWERING UNRESOLVED
QUESTIONS FROM SESSIONS 1 AND 2.
STARTING AT 5:05 WE'LL LOOK
FORWARD INTO THE FUTURE, AND
ANDRES WILL HAVE A TALK.
I'M PLEASED TO INTRODUCE STEVEN
ZUCKER, YOU'VE ALREADY HEARD
FROM HIM TODAY, HE'S A PROFESSOR
OF COMPUTER SCIENCE, BIOMEDICAL
ENGINEERING YALE UNIVERSITY.
HE CO-FOUNDED WITH TERRENCE
SEJNOWSKI THE WOODS HOLE WORKSHOP
ON COMPUTATIONAL NEUROSCIENCE IN
1980S BLOOMING INTO A NOW. HE
STUDIED ROLE OF VISUAL SYSTEM,
ITS CONNECTION TO GEOMETRY.
WE'RE ALSO DELIGHTED TO HAVE AS
CO-MODERATOR KANAKA RAJAN HARVARD
UNIVERSITY, FOUNDING MEMBER OF
KEMPNER INSTITUTE FOR NATURAL
AND A.I. AT HARVARD.
>> WELCOME BACK, EVERYONE, TO
GET THE AFTERNOON SESSION GOING
I THOUGHT WE SHOULD ASK DORIS
WHAT SHE THOUGHT OF ANDREAS'S
TALK THIS MORNING.
>> IT WAS AMAZING.
POINTING US TO THE NEW
FUTURE, BUILDING DIGITAL TWINS,
THE DREAM THAT WE DON'T HAVE
TO -- AS A MONKEY PHYSIOLOGIST,
I KNOW HOW HARD IT IS TO DO
THESE EXPERIMENTS, IT WOULD BE
INCREDIBLE IF I COULD DOWNLOAD
THE MODEL AND JUST, YOU KNOW,
THINK.
BUT AS WE DISCUSSED TOGETHER IN
THE BREAK, THAT'S NOT THE
ENDPOINT, RIGHT?
I HOPE THAT'S CLEAR.
ONCE YOU HAVE THE DIGITAL TWIN,
THAT'S JUST THE BEGINNING.
ULTIMATELY WE WANT TO UNDERSTAND
THE PRINCIPLE, THAT'S WHAT IT
MEANS TO UNDERSTAND.
SOME PEOPLE THINK THAT
UNDERSTANDING IS NOT POSSIBLE,
IF YOU JUST HAVE A NETWORK AND
THAT'S IT, YOU CAN'T EXPLAIN IT
ANY FURTHER, EXCEPT IN TERMS OF
ARCHITECTURE AND LEARNING RULE.
I BELIEVE WE CAN UNDERSTAND THE
BRAIN THE WAY WE UNDERSTAND
EVERYTHING ELSE, AND, YEAH, I
THINK THAT ANDREAS IS GOING TO
MAKE A GOOD CONTRIBUTION TO THAT.
>> YEAH, NO, I FULLY AGREE WITH
DORIS.
I THINK THERE'S AN
INTERESTING -- INTERESTING TO
HEAR BLAKE ON THIS, WHEN YOU
LOOK AT THE PARALLEL DISTRIBUTED
PROCESSING BOOKS, IT'S
PHILOSOPHY. CAN WE HUMANS
UNDERSTAND AND INTERPRET THE
BRAIN IN A WAY WE CAN DESCRIBE
IT WITH BEAUTIFUL WORDS AND SAY,
YOU KNOW, WHEN I DO THIS OR I
SEE THIS, THIS IS HOW IT WORKS.
AND I AGREE WITH DORIS HERE.
I DON'T THINK I WOULD HAVE BEEN
A NEUROSCIENTIST IF I DIDN'T
BELIEVE THERE WERE PRINCIPLES TO
BE UNDERSTOOD, THAT CAN BE
DESCRIBED WITH, IN
CLASSICAL SCIENCE, DOWN TO
BEAUTY, SYMMETRY, MATH.
I DO THINK THAT FOR ME BUILDING
THESE MODELS AND MORE THINKING
ABOUT DATA DRIVEN MODELS, NOT
MORE LIKE THE OTHER KINDS OF
MODELS THAT YOU PRE-TRAIN IT ON
A TASK, SEE HOW WELL IT PREDICTS.
THOSE ARE GREAT BUT I'M THINKING
MORE ABOUT DATA DRIVEN MODELS
BECAUSE WE HAVE THE CAPABILITY
TO GET SO MUCH DATA, IF YOU LOOK
AT THE SCALING LAWS AS SHOWN,
THESE MODELS ARE SATURATING, BUT THE DATA DRIVEN-MODELS ARE NOT SATURATED. IT CAN BECAUSE
CAN GET MORE DATA, I THINK THE
MODEL IS JUST THE BEGINNING TO
BASICALLY BE ABLE TO DO
EXPERIMENTS WITH LESS
RESTRICTIONS TO GET TO
PRINCIPLES.
>> BLAKE, IS IT PRINCIPLES OR
LOSS FUNCTIONS?
>> OH, NO, I THINK IT'S
DEFINITELY .
PRINCIPLES.
BUT I THINK THAT THE KEY POINT
IS, WHAT ANDREAS WAS HINTING AT,
PRINCIPLES EXPRESSED IN
MATHEMATICS.
HUMAN LANGUAGE IS GOING TO BE
INSUFFICIENT TO EXPLAIN WHAT'S
GOING IN THE BRAIN, MUCH AS IT
IS WITH PHYSICS.
CAN'T DO ADVANCED PHYSICS PURELY
WITH LANGUAGE BUT IT'S WHAT WE
TRIED DO IN NEUROSCIENCE FOR
MANY YEARS, USE OUR NATURAL
LANGUAGE TO DESCRIBE WHAT THE
SYSTEM IS DOING.
I THINK NECESSARILY WHAT
NEUROAI IS PUSHING US TOWARDS,
REALIZATION THAT WE DESCRIBE
SYSTEMS USING MATHEMATICS OF
HIGH DIMENSIONAL VECTOR SPACES,
DYNAMICS IN THOSE HIGH DIMENSIONAL
VECTOR SPACES, ET CETERA.
THINGS LIKE LOST FUNCTIONS,
ARCHITECTURES, WHATEVER, ARE
ULTIMATELY WAYS OF GETTING A
HOLD MENTALLY ON THAT
MATHEMATICAL PRINCIPLES.
IT'S A MECHANISM FOR US TO
TRANSLATE SOME OF THOSE
MATHEMATICAL PRINCIPLES INTO
THINGS THAT WE CAN VERBALIZE IN
OUR NATURAL LANGUAGE FOR THE
SAKE OF COMMUNICATING THEM TO
EACH OTHER.
BUT AT THE END OF THE DAY, I DO
THINK WE CAN EXTRACT PRINCIPLES,
VERY MUCH SO.
>> I HAVE A QUESTION FOR THE
ENTIRE PANEL BECAUSE THERE'S
SOMETHING THAT WAS CIRCLED A
COUPLE TIMES TODAY AND THAT WAS
THE NOTION OF ARTIFICIAL
INTELLIGENCE AND NATURAL
INTELLIGENCE HAVING SORT OF
ORTHOGONAL INCENTIVE.
LIKE A.I. ENGINEER, YOU WANT TO
BUILD A PERFECT PERFORMING
TASK-ALIGNED MODEL BUT ANIMALS
AND HUMANS, IMPERFECTIONS AND
INDIVIDUAL DIFFERENCES LEAD US
AWAY FROM THE FIRST GOAL.
SO, IN THE RESULTS THAT TALK
ABOUT MULTIPLE MODELS HAVING
CONVERGED TO SIMILAR
REPRESENTATIONS WITH EACH OTHER
THAT ARE SOMEWHAT SUBSTRATE
AGNOSTIC AS WAS POINTED OUT THIS
MORNING BY ALI MINAI.
WHAT I BUY LESS PERHAPS IS BOTH
MODELS, SELF-ALIGNED TO COMMON
REPRESENTATIONS HAVE ANYTHING TO
DO WITH THE SPACES THAT NATURAL
INTELLIGENCE HAS FOUND WITH ITS
BUMPS AND WARTS.
AND SO I THINK I WOULD WANT TO
HEAR FIRST FROM DR. CHANCE OR DR.
TSAO, BOTH OF WHOM MENTIONED
THIS IN THEIR TALKS.
>> WELL, I ACTUALLY AM GOING TO
TOUCH ON THIS A LITTLE BIT
TOMORROW SO MAYBE I'LL TRY TO
ANSWER THE QUESTION IN THAT
CONTEXT.
I MIGHT DISAGREE NATURAL
INTELLIGENCE IS AN OPTIMAL
SOLUTION SO MUCH AS THAT IS
OPTIMIZED FOR DIFFERENT CONSTRAINTS
AND I THINK WE AGREE HERE.
I THINK THERE'S A LOT TO BE
LEARNED FROM UNDERSTANDING WHAT
THE CONSTRAINTS FACING THE
ANIMAL ARE AND HOW THEY HAVE
SHAPED THE SOLUTION THAT BIOLOGY
HAS FOUND.
AND THAT IS SOMETHING I THINK
A.I. CAN TAKE.
AND THE OTHER THING I WOULD ADD
IS I THINK SOMETIMES WHEN WE --
I AGREE I WANTED TO RESPOND TO
BLAKE, I AGREE THAT MATH IS
NECESSARY TO MOVE FROM BIOLOGICAL
DETAIL INTO A.I. MODELS BUT I
THINK ONE THING THAT GETS LOST
WHICH I THINK IS WHAT MAYBE
DORIS WAS TOUCHING ON WITH THE
COMMENT EARLIER, IS THE INTUITION
BIOLOGISTS GET WHEN THEY STUDY
THEIR SYSTEMS.
I DON'T WANT TO DERAIL DORIS'
QUESTION, BUT DORIS' ANSWER TO
THE QUESTION, BUT MAYBE
SOMETHING WE CAN TALK ABOUT HOW
DO YOU TRANSLATE WITHOUT LOSING
THAT INTUITION THAT'S SITTING IN
THE EXPERIMENTALISTS' HEAD.
>> I AGREE WITH WHAT FRANCES
SAID. THE IDEA BY PHILLIP ISOLA
CALLED THE PLATONIC
REPRESENTATION HYPOTHESIS THAT
THERE'S LIKE ONE REALITY, IF YOU
TRAIN A NETWORK THAT'S LARGE
ENOUGH WITH ENOUGH TASKS THEN
IT'S GOING TO FIND THAT MODEL
THAT REALITY AND THAT'S GOING TO
MATCH WHAT'S IN THE BRAIN. AND
I DON'T THINK WE'RE NEAR THAT
FANTASY LAND. THESE NETWORKS THAT
ARE TRAINED ARE NOWHERE NEAR WHAT
OUR BRAIN IS DOING.
THEY MAKE SO MANY MISTAKES, SO I
REALLY THINK WE NEED TO MAKE USE
OF EVERYTHING THAT WE KNOW ABOUT
THE BRAIN, RIGHT?
LIKE WHY SHOULD WE THROW ALL THE
DECADES OF PSYCHOLOGY
EXPERIMENTS OUT THE WINDOW?
AND ALL OUR NEUROSCIENCE DATA.
WE SHOULD USE ALL THE NEUROSCIENCE
DATA INFORMATION WE HAVE TO FIND
THE BEST MODEL OF THE BRAIN.
>> CAN I MAKE ONE BRIEF COMMENT?
SO I REALLY LIKE THIS POINT
ABOUT BIOLOGISTS INTUITIONS.
I THINK I WANT TO COMMENT ON
THAT, THAT RELATES TO EARLIER
DISCUSSION ON TRAINING, THE BEST
WAY TO ENSURE BIOLOGIST'S
INTUITION ARE PRESERVED IS TO HAVE
BIOLOGISTS WHO SPEAK MATHEMATICS.
SO, I THINK ONE OF THE MOST
IMPORTANT THINGS MOVING FORWARD
IN THE FIELD OF NEUROSCIENCE IN
GENERAL IS THAT WE CANNOT VIEW
FORMAL TRAINING IN MATHEMATICS
AND COMPUTER SCIENCE ANYMORE AS
AN OPTION.
IT SHOULDN'T BE SOMETHING
STUDENTS CAN CHOOSE TO TAKE OR
NOT.
IT SHOULD BE AS FUNDAMENTAL AS
LEARNING HODGKIN-HUXLEY, YOU
KNOW, PRINCIPLES OF HUBEL AND
WIESEL, THESE SORTS OF THINGS.
IT SHOULD NOT BE SOMETHING WE
CONSIDER SECONDARY. YOU SHOULD
START FROM THE VERY GET-GO
LEARNING MULTIVARIATE CALCULUS,
LINEAR ALGEBRA, STATISTICS, TO
TURN BIOLOGY INTO MATH, THAT WILL
BE THE WAY TO ACHIEVE WHAT WE'RE
ULTIMATELY STRIVING FOR.
>> THE COUNTER IS ALSO TRUE.
I WISH I HADN'T WAITED UNTIL
POSTDOC TO KNOW THAT COGNITIVE
SCIENCE HAS SO MUCH TO OFFER,
GRADIENTS SINCE 1920, LEARNING
RATES AND STUFF.
FEWER SILOS WOULD BE BETTER.
>> A HAND UP ON ZOOM, I INVITE
HER TO UNMUTE.
>> THANK YOU, JOE.
I HAVE NO IDEA WHAT I SOUND LIKE
IN THE AUDITORIUM.
CAN YOU HEAR ME OKAY?
>> WE CAN'T HEAR YOU.
>> I AM TALKING AND I DON'T KNOW
WHAT THE SITUATION IS LIKE.
CAN SOMEONE GIVE ME A THUMBS UP
IF YOU CAN HEAR ME?
>> WE CAN HEAR YOU NOW.
>> THANK YOU, FRANCIS.
SORRY FOR BEING A DISEMBODIED
VOICE IN THE LARGE ROOM.
I AGREE WITH WHAT DORIS SAID
ABOUT NOT THROWING AWAY THE
DECADES IF NOT CENTURIES OF
KNOWLEDGE WE HAVE.
AS A COMPUTATIONALLIST, I THINK
ONE OF THE KEY THINGS I GET FROM
TALKING TO EXPERIMENTALISTS IS
KNOWING WHAT FACTS TO IGNORE.
SO FOR ME IT'S ESSENTIALLY LESS
ABOUT WHAT ARE ALL THE DATASETS
WE'RE GOING TO THROW INTO TRAINING
DATASET BUT USING KNOWLEDGE FROM
BIOLOGY TO FIGURE OUT THESE ARE
ACTUALLY FACTS WE KNOW TO BE
TRUE ABOUT BIOLOGY BUT MAYBE WE
CAN SAFELY IGNORE THEM IN
MODELING FOR NOW BECAUSE THEY
ARE JUST CONFUSING AT THE LEVEL
OF DETAIL THAT WE DON'T NEED.
SIMILARLY, I AGREE WITH BLAKE
ABOUT THE IMPORTANCE OF HAVING
METHODICAL FORMALISM FOR
UNDERSTANDING WHAT THE BRAIN IS
DOING, WHATEVER IT'S DOING IT'S
NOT SOMETHING THAT WE CAN DRAW A
CARTOON PICTURE.
IT'S SOMETHING WE HAVE TO WRITE
DOWN AS MATHEMATICS.
HOPEFULLY IT'S NOT SOMETHING
THAT IS COMES AS A HUGE SURPRISE
TO PEOPLE THAT MY VERSION OF
THAT IS MORE THAN CALCULUS AND
LINEAR ALGEBRA.
IT REALLY IS FEEDBACK CONTROL,
DYNAMICS CONTROL.
AND THE REASON THAT I FEEL LIKE
THIS IS A FUNDAMENTAL PIECE OF
FORMALISM WE NEED TO UNDERSTAND
THE BRAIN IS BECAUSE THE BRAIN
NEVER REALLY JUST SITS THERE AND
PERCEIVES ANYTHING.
MOST OF US ARE WALKING AROUND
THE WORLD MAKING DECISIONS
MOMENT TO MOMENT, WHAT TO
PERCEIVE NEXT, SENSING IT,
SEEKING OUT MORE INFORMATION,
THAT'S SOMETHING WE GET OUT OF
EMBODIED INTELLIGENCE THAT AN
ARTIFICIAL NETWORK SITTING THERE
TALKING DOESN'T REALLY HAVE.
AND THIS ABILITY TO SEEK OUT
NEW INFORMATION IS EXPRESSED IN
THE LANGUAGE OF CONTROL THEORY
AND LANGUAGE OF REINFORCEMENT
AND LANGUAGE OF LEARNING.
AND IF IT'S ONE THING I DO KNOW
ABOUT THAT TYPE OF MATHEMATICS,
IT'S HORRENDOUSLY UNINTUITIVE.
I CAN SHOW SMALL DIAGRAM AND ASK
PEOPLE TO PREDICT WHAT'S GOING
TO HAPPEN, AND VERY FEW PEOPLE
CAN, IT'S NOT SOMETHING HUMANS
ARE GOOD AT.
FORTUNATELY WE HAVE MATHEMATICS
OF DOING THAT.
I'LL ADD THAT TO THE LIST OF
THINGS THAT ALL NEUROSCIENTISTS
SHOULD BE CONVERSANT AT - THAT
TYPE OF MATHEMATICS. THANK YOU.
>> THE LAST POINT ABOUT HOW
UNINTUITIVE A SMALL NETWORK CAN
BE THAT'S IN LARGE PART BECAUSE
THERE'S MORE MATH TO BE
DEVELOPED. I THINK ONE OF THE
GOALS FOR MATHEMATICAL
DEVELOPMENT IS TO LIKE IMPROVE
RESULT SO THAT THERE IS
INTUITION YOU CAN BUILD VIA THE
MATH.
INTUITION IS VALUABLE IN BOTH
DIRECTION, BIOLOGIST INTUITION
FROM YEARS OF EXPERIMENTS IS
VALUABLE.
THE MATHEMATICIAN'S INTUITION
FROM STUDYING THE MODEL SERIOUSLY
ON THEIR OWN IS VERY VALUABLE,
IF YOU PUT THOSE TOGETHER A LOT
OF PROGRESS CAN BE MADE.
>> IF I CAN JUST MAKE A POINT ON
THE ISSUE BEING DISCUSSED HERE,
I THINK IT'S FASCINATING, THE CONVERGENCE
BETWEEN THE REPRESENTATION FOUND
BY A.I. AND THE BRAIN.
BUT YOU HAVE TO REMEMBER AT
LEAST IN THE REAL NATURAL WORLD
INTELLIGENCE IS FOR LIVING INTELLIGENTLY
TO BE PRODUCTIVE,SUCCESSFUL, AND REPRODUCE.
INTELLIGENCE IS RELATED TO EMBODIMENT.
EMBODIED ORGANISM, NOT JUSTTHE FUNCTION OF
THE BRAIN.
OUR DISCUSSION KEEPS THE BRAIN AS A
COMPUTATIONAL OBJECT. LET’S SEEHOW WELL IT
CAN WORK TO OUR
OTHER COMPUTATIONAL OBJECTS, A.I. SYSTEMS.
BUT THIS IS NOT THE ENTIRETY OF INTELLIGENCE.
IT IS A FUNCTION OF THE WHOLE
ORGANISM, WHOLE ANIMAL.
INTELLIGENCE IS ALWAYS IN TERMS OF
AFFORDANCES - PERCEPTUAL AND
BEHAVIORAL. WILL BE THE SAME WITH
A.I. SYSTEMS AS WE MOVE TOWARDS
TRULY GENERAL A.I., A.I. THAT
SITS THERE ABOUT LANGUAGE AND
MATH IS NOT GENERAL A.I.
>>BING HAS DONE WORK MOVING AWAY FROM
BRAINS AS BAGS OFNEURONS AND IS BUILDING
MODELS
OF AGENTS AND EMBODIMENT.
WE CAN EMBODY AGENTS WITH PHYSICS,
BIOMECHANICS, KINEMATICS. WHEN DOWE GO WITH
ALL THAT?
EVOLUTION IS NOT YET DONE.
I DON’T THINK WE WILL BE DONE EITHER BECAUSE WE ARE DOING
WHAT EVOLUTION HAS DONE: CREATING ARTIFICIAL ORGANISMS.
; IN VISION IF WE BUILD A SYSTEM THAT DOESN’T
HAVE TO REPLICATE ALL THE DETAILS OF THE BRAIN
BUT CRITICALLY CAPTURES ITS INFORMATION PROCESSING ALGORITHMS ANDACHIEVES
HUMAN-LEVEL ROBUST VISUAL PERCEPTION WITH WELL-DEFINED PERFORMANCE BENCHMARKS.
>> I WOULD SAY, YEAH, FOR
VISION, THAT'S AN AMAZING
SUCCESS.
YOU KNOW, I THINK THAT'S A
CLEARLY WELL-DEFINED BENCHMARKS
THAT WE CAN ACHIEVE.
THE SAME FOR LIKE ROBOTICS, YOU
CREATE A ROBOT THAT CAN GO AND
LOAD YOUR DISHWASHER AND HAS
DEXTERITY AND GRASPING IN A WAY
THAT IS INSPIRED BY
UNDERSTANDING OF MOTOR CONTROL
IN HUMANS OR OTHER ANIMALS, HAS
VISUAL SYSTEM, THAT'S A
WELL-DEFINED METRIC AND WE'RE
DONE.
>> ALSO ANDREAS USED THE WORD
"BEAUTY" EARLIER, THAT'S A BIG
PART OF IT.
THEN WE'LL HAVE A SENSE THAT
WE'RE GETTING THERE.
>> I CAN'T RESIST MURRAY GELL-MANN'S
REMARK IT'S SO BEAUTIFUL IT
SHOULD BE TAKEN AS EXPERIMENTAL
FACT.
>> ALONG THOSE LINES, HOW MUCH
IS VISUAL RECOGNITION A MODEL
FOR COGNITION?
IF WE GOT VISUAL RECOGNITION FOR
VIDEOS AS GOOD AS WE HAVE VISUAL
RECOGNITION FOR ImageNet,
WILL WE THEN HAVE ENOUGH VISION
TO SAY THIS IS THE FOUNDATION ON
WHICH WE SHOULD BUILD COGNITION?
>> YEAH, PEOPLE THINK ABOUT
RECOGNITION AS A FUNCTION OF THE
VENTRAL STREAM, THAT'S EVEN HALF
THE STORY OF VISION LET ALONE
COGNITION.
BUT I THINK THAT, YOU KNOW, LIKE
PROBABLY 80% OF THE BRAIN IS
VISUAL IN SOME SENSE, LIKE WILL
RESPOND TO VISUAL STIMULUS, EVEN
PRE-FRONTAL CORTEX, THERE'S
STRUCTURES THAT ARE MULTI-MODAL,
RESPONSIVE TO SOUND AND SO ON.
I FEEL LIKE WE'RE MISSING AN
UNDERSTANDING OF THE BASIC DATA
STRUCTURE OF HOW THE BRAIN IS
ORGANIZING THE WORLD, RIGHT?
WE JUST DON'T HAVE THAT RIGHT
NOW.
AND ONCE WE GET THAT I THINK
THAT'S GOING TO BE A FOUNDATION
FOR COGNITION, LIKE THOSE ARE
THE OBJECTS WE'RE THINKING
ABOUT, WHAT IS AN OBJECT, HOW IS
IT REPRESENTED IN THE BRAIN?
GOES FAR BEYOND THE QUESTION OF
RECOGNITION.
>> ONE WAY I LIKE TO THINK ABOUT
IS COGNITION IN HUMANS, YOU
KNOW, A LOT IS ABOUT PLANNING
INTO THE FUTURE, AND HAVING LIKE
A MODEL OF THE WORLD THAT
EXTENDS ALL THE WAY BACK TO THE
BIG BANG AND PREDICTING WHAT'S
GOING TO HAPPEN, YOU KNOW,
TOMORROW, A YEAR FROM NOW, TWO
YEARS FROM NOW.
AND I THINK IN PRIMATES VISION
IS SO DOMINANT OF A SENSE IT'S
ENABLES US TO BUILD COMPLEX
MODELS OF THE WORLD, A VIDEO IS
SHORT SPACE AND TIME BUT IF YOU
GO TO MORE COGNITIVE BEHAVIORS,
VISION OR LANGUAGE, IT'S ABOUT
EXTENDING THE WORLD
UNDERSTANDING IN SPACE AND TIME.
AND I THINK VISUAL WON'T GET US
ALL THE WAY THERE BUT I THINK
IT'S A HUGE OPPORTUNITY TO HAVE
A VERY GOOD START BY USING
VISUALLY GUIDED COGNITION AND
GOING INTO MORE COMPLEX
COGNITIVE BEHAVIORS.
>> I THINK THE ONLY THING I WILL
ADD THOUGH IS THERE IS AN
INTERESTING ISSUE FOR EMBODIMENT
AND ROBOTICS THAT IS OFTEN
OVERLOOKED WHICH IS THAT VISION
IS NICE AND ALL, BUT IF YOU'VE
GOT A BODY MOVING THROUGH SPACE,
I WOULD TAKE THE SENSE OF TOUCH
OVER VISION ANY DAY.
AND SO I THINK FOR ACTUALLY
UNDERSTANDING THE HUMAN BRAIN
AND HOW WE MODEL THE PHYSICAL
WORLD, PROBABLY A SENSE OF TOUCH
IS GOING TO BE CRITICAL IN THE
END.
AND IT'S SOMETHING THAT WE DON'T
YET HAVE ACCORDING TO MY
KNOWLEDGE A GOOD MODEL OF AT
ALL.
>> I WANT TO MAKE A COMMENT THAT
SO VISION OF COURSE IS A SENSORY
PERCEPTION, RIGHT?
BUT THERE'S ALSO THE WHOLE, YOU
KNOW, SPATIAL ORGANIZATION,
REPRESENTATION OF SPACE AND
OBJECTS IN SPACE.
AND THAT SEEMS TO BE OBVIOUSLY
VERY USEFUL FOR PERCEIVING
VISUAL WORLD BUT ALSO SOMETHING
YOU CAN USE FOR OTHER THINGS.
THERE ARE THESE EXAMPLES OF LIKE
BLIND MATHEMATICIANS WHO LIKE
HAVE AMAZING SPATIAL
UNDERSTANDING AND DO GEOMETRY
AND SO ON.
THERE'S SOMETHING ABOUT THAT RAW
REPRESENTATION OF SPACE IN TWO
OR THREE DIMENSIONS, WHATEVER IT
IS, I CAN'T GO BEYOND THREE
DIMENSIONS, BUT SOMETHING ABOUT
THAT WHICH I THINK IS DEVELOPED
TO AID THE VISUAL SYSTEM BUT
ENORMOUSLY USEFUL IN
UNDERSTANDING, YOU KNOW,
COGNITION AND OTHER MODALITIES.
>> PART OF THE MOTIVATION FOR MY
QUESTION WAS OF COURSE THAT WE
NOW KNOW THERE ARE VERY RICH
MOTOR SIGNALS, AUDITORY SIGNALS
IN THE CONTEXT, WHETHER YOU CAN
FACTORIZE VISION AWAY FROM ALL
THESE OTHER SENSES AND ACTION IS
THE QUESTION.
>> DEFINITELY YOU ARE CANNOT BUT
E.G., IN PRIMATES EYE MOVEMENT
IT'S DOMINANT MOTOR SYSTEM
IF YOU STUDY VISION FREE-VIEWING,
YOU GO DOWN TO V1, YOU HAVE
ISSUES OF INSTABILITY. HOW DO YOU
GET A STABLE ALLOCENTRIC
REPRESENTATION OF THE WORLD, AND
IF YOU HAVE BODIES MOVING IN IT
YOU GET COMPLEX DYNAMICS.
THIS STUFF HAS TO BE EVENTUALLY
INCORPORATED IN, BUT, YOU KNOW,
EXPERIMENTALLY IT'S FAIRLY
STRAIGHTFORWARD, IT'S EASIER TO
GET THIS DATA FREE VIEWING THAN
TRAINING ANIMALS TO FIXATE,
WE CAN EMBRACE AND DO
IMMEDIATELY.
>> SO DOES PANEL HAVE CONSENSUS
THAT THE NEXT GENERATIONS OF
NEUROAI SYSTEMS SHOULD BE
MULTIMODAL AT LEAST?
>> YEAH, AND BE MULTIPLE
MODALITIES, THEY WILL DEPEND ON
WHAT KIND OF SYSTEM IT IS,
RIGHT?
THE WORRY THAT A BAT INHABITS,
THE FAMOUS EXAMPLE, DIFFERENT
THAN THE WORLD THE HUMAN
INHABITS, PARTLY BECAUSE OF
SENSORY MODALITIES, IT'S ALSO IN
THE CONTEXT OF THE EMBODIMENT.
AND TOGETHER THEY HAVE A MODEL
OF THE WORLD.
AND I THINK THAT IS GOING TO --
WHEN WE BUILD THESE A.I. SYSTEMS
THERE WILL BE A DIVERSITY OF THE
A.I. SYSTEM PROBABLY.
WHEN I SAY EMBODIMENT I DON'T
JUST MEAN IN THE PHYSICAL WORLD.
WE'LL HAVE TO DEFINE A
CORRESPONDING THING IN VIRTUAL
WORLD.
WHERE A.I. WILL ALSO LIVE.
>> CAN I RETURN TO THE QUESTION
ABOUT VISION AND COGNITION?
ONE OF THE HARDEST ASPECTS IS
PERCEIVING INVARIANCE.
MANY PEOPLE SAY HALLMARK OF
INTELLIGENCE IS GENERALIZATION.
CURIOUS IF CARINA CAN SPEAK TO
MATHEMATICAL THEORIES TO TIE
TOGETHER COMPUTING, ROTATION
VARIANCE, TO MORE GENERAL
PATTERN INVARIANCE.
>> YEAH, I THINK THAT'S REALLY
INTERESTING.
I THINK THERE'S IN COMPUTER
VISION LIKE THE DEEP NETWORKS
AND EVERYTHING THERE'S A RICH
HISTORY OF ACTUALLY
MATHEMATICIANS WORKING IN TRYING
TO BUILD THAT SORT OF
EXPLICITLY, BUILD UNION GROUP
THEORETIC TRANSFORMATIONS,
REPRESENTATION THEORY AND SO ON.
AND WHAT'S INTERESTING IS THAT
THE WAY THE BRAIN PROCESSES
VISUAL INFORMATION SEEMS TO HAVE
SOME OF THOSE INVARIANCES BUT
NOT ALL, SO YOU HAVE THIS GROUP
OF RIGID MOTIONS IN 3D, RIGHT?
OR SOMETHING LIKE THAT.
AND SOMEHOW WE DO PERCEIVE UP
AND DOWN SLIGHTLY DIFFERENTLY,
RIGHT?
INVARIANCES ARE NOT QUITE THE
SAME. THEY'RE THERE BUT
NOT QUITE, SORT OF THE
PLATONIC GEOMETRIC IDEAL
SHOULD BE, SO I
THINK THERE'S ACTUALLY VERY
INTERESTING QUESTIONS THERE.
YOU KNOW, FOR MATHEMATICIANS IN
COLLABORATION WITH VISION
EXPERTS TO WORK ON TO TRY TO
UNDERSTAND WHAT ARE THE BASIC,
YOU KNOW, GEOMETRIC INVARIANCES
OF OUR PERCEPTUAL WORLD.
I THINK THAT'S VERY INTERESTING.
>> CAN I ASK BLAKE A QUESTION
WHICH IS ONLY TANGENTALLY
RELATED TO THIS QUESTION?
INTERESTING CARINA SAYINGS
CERTAIN TYPES OF MATH MAY BE
NEEDED THAT WE CAN'T WRAP OUR
HEADS AROUND TODAY.
AND I WAS ADVOCATING FOR MATH
TO TRANSLATE.
MAYBE MOVING INFORMATION FROM
NEUROSCIENCE DATA INTO THE A.I.
MODELS.
SO, WOULD YOU SAY THAT MATH IS
THE RATE LIMITING STEP OR WOULD
YOU SAY THAT ARE WE STUCK
BECAUSE THE MATH DOESN'T EXIST?
OR ARE WE GOING TO BECOME STUCK
BECAUSE MAYBE NOT THAT MANY
MATHEMATICIANS -- MAYBE NOT MANY
HUMANS CAN HANDLE THAT MATH?
>> I'LL START, YEAH.
SO, MY BIASED ANSWER IS THAT
MATH IS ONE OF THE RATE
LIMITING STEPS.
I DON'T KNOW IF IT'S THE ONLY
ONE, BUT I DO THINK THAT THERE
IS THIS PROBLEM WHERE IT'S LIKE
YOU HAVE EVEN TAKE A DEEP
LEARNING NETWORK, RIGHT, THERE'S
NO DYNAMICS.
IT'S JUST A COMPOSITION OF MATH.
RIGHT?
AS A MATHEMATICIAN I WOULD
DESCRIBE THAT AS A
PARAMETERIZATION OF THE FAMILY
OF FUNCTIONS, RIGHT?
WEIGHTS ARE YOUR PARAMETERS, OR
YOUR WEIGHTS AND BIASES ARE
YOUR PARAMETERS, AND WE DON'T
UNDERSTAND THAT FAMILY WELL.
SO RIGHT NOW ACTUALLY IN THIS
MOMENT PARALLEL TO THIS THERE'S
A WHOLE LIKE PUSH IN MATH AND
A.I., RIGHT, WHERE THERE'S AN
EFFORT TO GET MATHEMATICIANS TO
LOOK VERY SERIOUSLY AT DEEP
LEARNING.
I THINK THEY SHOULD ALSO LOOK
SERIOUSLY AS NEUROAI, THAT'S
MY PUSH HERE, BUT
CERTAINLY THAT'S HAPPENING IN
DEEP LEARNING.
BECAUSE OF THE SUCCESS OF THOSE
MODELS, THERE'S NOW KIND OF A
GROWING COMMUNITY OF
MATHEMATICIANS WHO ARE TRYING TO
UNDERSTAND MATHEMATICALLY
WHAT'S GOING ON THERE, WHAT
HAPPENS IN TRAINING.
VIEW A DEEP NETWORK AS A FAMILY
OF FUNCTIONS, WITH SOME GIANT
PARAMETER SPACE, WHEN YOU'RE
TRAINING YOU'RE MOVING IN A
PARAMETER SPACE.
THERE ARE SYMMETRIES, DIRECTIONS
WHERE YOU CAN MOVE AND NOT ALTER
THE FUNCTION OR WHERE YOU ALTER
THE FUNCTION A LOT. A LOT OF
GEOMETRY TO THE PARAMETER SPACE,
THE WEIGHT SPACE, AND I THINK
THAT'S COMING.
BECAUSE OF A.I. THE MATH
COMMUNITY IS WAKING UP TO THE
FACT THAT THIS IS IMPORTANT AND
MAKES ME A LITTLE SAD, RIGHT,
THAT THEY ARE NOT MORE FOCUSED
ON THE NEUROSCIENCE SIDE WHICH I
THINK WOULD ALSO BE VERY, VERY
FRUITFUL.
BECAUSE I THINK THERE'S EVEN
MORE INTEREST MATH TO BE DONE
WHEN YOU LOOK AT DYNAMICS AND
DIFFERENT ARCHITECTURES WE SEE
IN NEUROSCIENCE AND SO ON.
SO AND THEN JUST TO KIND OF GO
BACK AND ANSWER YOUR ACTUAL
QUESTION, I DO THINK IF WE
UNDERSTAND THE MATH BETTER IT
WILL GUIDE US INTO BETTER
LEARNING RULES, YOU KNOW,
TRAINING PARADIGMS AND SO ON AT
LEAST FOR THOSE FAMILIES OF
MODELS.
I DO THINK I OPTIMISTICALLY
WOULD LOVE TO BE IN A WORLD
WHERE WE UNDERSTAND THE MATH OF
SIMPLE NONLINEAR MODELS AS WELL
AS WE UNDERSTAND THE MATH OF
LINEAR SYSTEMS.
IF WE WERE IN THAT WORLD WE
WOULD MAKE A LOT OF PROGRESS, I
THINK.
>> WELL, BING HAS HAD A HAND
RAISED FOR A WHILE.
>> GO AHEAD.
>> I'LL LET HER UNMUTE.
SHE'S SWOOPING IN ON THE SCREEN.
>> HI.
CAN YOU HEAR ME OKAY?
JOE, ARE YOU ABLE TO HEAR ME
OKAY?
>> EVERYTHING IS GOOD.
>> WE HEAR YOU.
>> OKAY.
THANKS.
I WANT TO QUICKLY RESPOND TO
STEVE'S QUESTION EARLIER WHETHER
OR NOT WE AGREE THAT THE NEXT
GENERATION OF MODELS SHOULD BE
MULTIMODAL. BLAKE'S ANALOGY
TALKING ABOUT ROBOTICS IS APT HERE,
AND ALI TALKED ABOUT HOW
FUNDAMENTALLY NEUROSCIENCE,
WE'RE STUDYING BIOLOGY, BIOLOGY
IS ABOUT ANIMALS, THAT IS THE
UNIT.
SO THE UNIT IS THE ANIMAL, THE
UNIT IS THE ROBOT, THAT MEANS
MUCH MORE THAN HAVING
MULTI-MODAL MODELS WE NEED
AGENT-BASED MODELS, INTEGRATION
AND PLANNING AND HAS MEMORY AND
PLANS FOR THE FUTURE.
NOT JUST LIKE FIGURING OUT WHAT
WE'LL DO IN A COUPLE YEARS, IN
EVERY TIME SCALE MILLISECONDS TO
YEAR IS THE THING WE SHOULD BE
DOING, I THINK.
AND I THINK IN ORDER TO BE ABLE
TO DO SOMETHING LIKE THAT IS
GOING TO, I DON'T KNOW, IT'S
LIKE AGAIN I'M GOING TO MAKE
ROBOTICS ANALOGIES HERE.
WE'RE NOT TRYING TO UNDERSTAND
THE WHEELS.
WE'RE NOT TRYING TO UNDERSTAND
THE WINGS.
WE NEED TO UNDERSTAND THOSE
THINGS BUT WE NEED TO UNDERSTAND
THE CONTEXT WHAT IT TAKES FOR
THE THING TO DO THE THING IT
ACTUALLY DOES.
IN ANIMALS, IT'S ESSENTIALLY TO
SURVIVE.
AND SO I THINK LIKE IF WE DON'T
KEEP IN MIND THAT'S THE ULTIMATE
GOAL, THAT WE ARE TRYING TO
ACCOMPLISH, IT'S GOING TO BE
EASY TO GET STUCK IN THESE
PIECEMEAL SOLUTIONS.
AND THAT LEVEL OF UNDERSTANDING
IS GOING TO BE LIKE EVEN IF WE
DO SUCCEED IN UNDERSTANDING
VISUAL SYSTEM THAT'S NOT WHAT
ANIMALS DO, EVEN HUMANS, AND
PRIMATES, WHO ARE VISUAL
CREATURES.
WE DON'T JUST SEE, WE DO A LOT
MORE THAN THAT.
I THINK THAT'S SOMETHING THAT WE
EXPERIENCE EVEN THE CONTEXT OF
SOCIAL INTERACTIONS LIKE WAS
BROUGHT UP EARLIER, LIKE WE'RE
VERY SOCIAL CREATURES.
THE REASON WE HAVE BODY
LANGUAGE, IF I WAS IN THE ROOM I
WOULD SEE YOUR BODY LANGUAGE
BETTER, AND SO THAT'S NOT
EXACTLY VISION, SORT OF
PROJECTING YOUR -- THE WAY
YOU'RE PROJECTING THE WAY YOU'RE
SITTING, LOOKING AT ME, YOU
MIGHT BE MAKING FACES, I CAN'T
REALLY SEE BUT BEING ABLE TO
INTERPRET IN THE CONTEXT HOW I
WOULD HOLD MY FACE AND CONTROL
THE FACIAL MUSCLES IN MY FACE,
WOULD BE TO INTERPRET THAT,
THAT'S THE REPRESENTATION, THE
THE
INVARIANCE IN MY HEAD.
IT'S MORE THAN VISUAL.
YOU HAVE TO IMAGINE WHAT I WOULD
DO IF I WERE THE AGENT TO INTERPRET
WHAT YOU ARE DOING MAKING A FACE.
>> SO I AGREE WITH CARINA,
THERE'S A LOT OF WORK TO DO IN
MATH TO UNDERSTAND ARTIFICIAL
NEURAL NETWORKS, MY GUESS WOULD
BE, WELL, AND TO UNDERSTAND
BIOLOGICAL NEURAL NETWORKS, A
LOT OF WORK IN ARTIFICIAL NEURAL
NETWORKS WILL PORT OVER TO
BIOLOGICAL NEURAL NETWORKS BUT
WORTH EMPHASIZING THERE ARE
ALREADY A LOT OF EXCELLENT
MATHEMATICAL TOOLS FOR DOING THE
COMPARISONS THAT MAYBE ARE NOT
FULLY UTILIZED BY THE FIELD OF
NEUROSCIENCE OR A.I. IN GENERAL,
TO THE EXTENT THEY COULD BE.
SO, I'LL GIVE CONCRETE EXAMPLE.
YOU KNOW, THERE WAS A BEAUTIFUL
PAPER, CARSEN STRINGER AND KEN
HARRIS WHERE THEY EXAMINE
GEOMETRY OF REPRESENTATIONS,
PRIMARY VISUAL CORTEX IN MICE,
HOW THE DECAY OF THE EIGEN-
SPECTRUM OF THE COVARIANCE
MATRIX TELLS YOU SOMETHING ABOUT
STRUCTURE OF THE MANIFOLD. A
GIFTED STUDENT WHO BASICALLY WAS
INSPIRED BY THIS WORK DIRECTLY
FROM NEUROSCIENCE, AND WENT AND
ANALYZED ALL OF THE MOST
ADVANCED A.I. MODELS, DISCOVERED
THAT THE A.I. MODELS THAT
PERFORMED BEST OUT OF
DISTRIBUTION GENERALIZATION ARE
THOSE THAT HAVE THE GEOMETRICAL
AND MANIFOLD PROPERTIES SIMILAR
TO THOSE SEEN THE BRAIN. COURTESY
OF MATHEMATICAL TOOLS THAT KEN
AND CARSON USED IN THEIR PAPER.
I THINK IT'S A BEAUTIFUL EXAMPLE
WHERE THANKS TO THE FACT THERE
WERE EXPERIMENTALISTS WHO DIDN'T
JUST PUBLISH DATA THAT DESCRIBED
THINGS WITHOUT THE MATH BUT THE
MATH WAS INTEGRAL TO THE ENTIRE
ANALYSIS THEY DID IN THEIR
PAPER, THAT MADE IT A LOT EASIER
TO HAVE A DIALOGUE AND TO
EXPLORE THINGS IN A.I. MODELS AS
WELL.
SO, I SUSPECT THERE'S A LOT OF
MATHEMATICAL TOOLS THAT COULD
LET US DO MORE LIKE THAT, IN
ADDITION TO THE FACT WE COULD
HAVE A BETTER DESCRIPTION AND
UNDERSTANDING OF NONLINEAR
DYNAMICS, ABSOLUTELY.
>> A QUICK COMMENT, KEN HARRIS
WAS AN UNDERGRAD MATH MAJOR, AND
HE WAS MY POSTDOC ADVISOR. WAS
TRAINED VERY MUCH IN MATH.
SO
THIS IS ONE OF THE EXAMPLES.
>> AT THIS POINT WE HAVE TO
SWITCH GEARS, I'M BEING TOLD.
I'D LIKE TO INTRODUCE THE SWITCH
THROUGH A STRONG NONLINEARITY.
ANDREAS WILL GIVE US A SHORT
PRESENTATION TO PERTURB US FROM
THE CURRENT DISCUSSION INTO THE
NEXT ROUND OF GENERAL
DISCUSSIONS, AND I'M TOLD
ANDREAS IS THE TWIN OF THE MODEL.
>> THIS IS JUST THE FOUR
SLIDES, JUST TO SPEAR HEAD THE
DISCUSSION AND GET EVERYBODY'S
INVOLVEMENT HERE OF WHAT ARE THE
POSSIBILITIES, WHAT WE WOULD
LIKE OR WOULD THE COMMUNITY LIKE
IN THE NEXT FEW YEARS TO REALLY
BRING NEUROSCIENCE AND A.I. IN
ALL THESE FORMS WE'VE BEEN
DISCUSSING TODAY THROUGHOUT THIS
MORNING, DISCUSSING MORE
TOMORROW.
SO WHAT DOES POTENTIAL ROAD
AHEAD LOOK LIKE?
HERE IS A SLIDE WE PUT TOGETHER
WITH, YOU KNOW, BLAKE RICHARDS,
XAQ, TONY ZADOR, ME AND A FEW
OTHER COLLEAGUES A COUPLE YEARS
AGO.
YOU KNOW, ESSENTIALLY LAYING OUT
WHAT WE THINK IS A GOOD WAY TO
THINK ABOUT FOUNDATION MODELS.
SO THE IDEA HERE IS THAT, YOU
KNOW, YOU CAN THINK OF NEURAL
NETWORKS AND A.I. AS A TOOL NOT
AS A MODEL OF THE BRAIN, IN A
WAY WE THINK IS TRYING TO LIKE
REPRESENT OR DO WHAT THE BRAIN
DOES, THE WAY IT DOES IT, BUT IS
A WAY TO FIND STATISTICAL
RELATIONSHIPS BETWEEN DOMAINS
WHERE WE HAVE A LOT OF DATA.
FOR EXAMPLE, HERE YOU COULD
IMAGINE YOU HAVE STIMULI, AND
THIS COULD INVOLVE VIDEO, THESE
MODELS COULD BE LIKE STANDARD
EVEN OFF-THE-SHELF MODELS,
PARSING VIDEO, LEARNING THE
STRUCTURE OF VIDEO, NEURAL
ACTIVITY, TASK SIMULATIONS, OR
THEORY.
ON THE OUTPUT YOU CAN GET, YOU
KNOW, YOU CAN DECODE FROM THESE
MODELS, USE IT FOR BRAIN-MACHINE
INTERFACE, PREDICT BEHAVIORS,
PREDICT ACTIVITY, HOW TO CONTROL
ACTIVITY, DIAGNOSE DISEASES, AND
SO ON.
AND TEST THEORIES.
THE IDEA IS THAT IF YOU THINK OF
THE -- YOU KNOW IS THERE GOING
TO BE A MOMENT LIKE ALPHAFOLD,
WHICH HAS BEEN SEMINAL DISCOVERY
OR ADVANCEMENT IN BIOLOGY, WHERE
THEY BROUGHT IN LARGE DATA THAT
HUMANS COLLECTED FOR MANY, MANY
YEARS, A LOT OF PhDs THESES,
MONEY SPENT DERIVING THE STRUCTURE
OF PROTEINS, WITH NEURAL
NETWORKS TO PRODUCE A PRODUCT
THAT IMPACTED BIOLOGY IN A VERY
BIG WAY.
AND IN MOLECULAR BIOLOGY THERE'S
OTHER KINDS OF IDEAS LIKE THAT,
YOU KNOW, LOOKING AT, YOU KNOW,
DNA, HOW TO SYNTHESIZE NEW
PROTEINS, HOW TO STUDY
PROTEIN-PROTEIN INTERACTION,
THIS MOMENT WHERE WE BRING LARGE
DATA AND LARGE COMPUTE TOGETHER.
THIS IS ONE IDEA.
NOW, IN ORDER TO DO THESE, I
SAID IT EARLIER ON, A COUPLE
OTHER SPEAKERS, CURRENT DATA WE
HAVE ARE GOING TO BE EXTREMELY
USEFUL IN MANY WAYS TO TEST
THESE MODELS BUT WE THINK THAT
TO BUILD THIS FOUNDATION MODEL
IN OUR EXPERIENCE YOU NEED TO
COLLECT IN A VERY SYSTEMATIC WAY
VERY LARGE DATASETS IN THESE
DIFFERENT MODALITIES, FOR
EXAMPLE, BRAIN HAS ALREADY DONE
AMAZING JOB COLLECTING THIS
ACTIVITY.
LOOKING AT NEURAL ACTIVITY, WE
NEED TO COME UP WITH
METHODOLOGIES TO COLLECT, YOU
KNOW, HIGH ENTROPIC DATA IN
NATURALISTIC CONDITIONS, IDEALLY
HAVING ANIMALS INTERACT FREELY
WITH EACH OTHER, SO-AND-SO
FORTH, TO HAVE ALSO EMBODIMENT.
THE OTHER THING THAT IS VERY
IMPORTANT HERE IS LARGE COMPUTE.
IF YOU CALCULATE, IF DO YOU BACK
OF THE ENVELOPE CALCULATIONS AND
PROJECT LET'S SAY THE AMOUNT OF
DATA THAT WE AS NEUROSCIENTISTS
MAY BE COLLECTING IN THE NEXT
TWO OR THREE YEARS, IF WE START
TO LIKE USE, YOU KNOW, EVEN LIKE
STANDARD TRANSFORMER TYPE OF
ARCHITECTURES, TOKENIZE THESE
THINGS, STARTING TO GET IN THE
BALL PARK OF QUITE LARGE SCALE
COMPUTE THAT RIGHT NOW IS NOT
CLEAR IF IT'S AVAILABLE AT THE
LEVEL OF WHAT WE HAVE
ACCESSIBILITY.
NOW, THE GOOD NEWS HERE,
ALTHOUGH THIS MARKET IS
FLUCTUATING A LOT, THERE IS SOME
EVIDENCE THAT MAYBE SILICON
VALLEY AND OTHERS OVERESTIMATED
NEED TO COMPUTE AND SOME NUMBERS
THAT YOU HEAR AROUND FOR EXAMPLE
THERE'S LIKE FIVE TO TEN MILLION
H100s OUT THERE, 30% BEING USED
AT ANY GIVEN POINT IN TIME.
COULD BE AN OPPORTUNITY HERE FOR
AN INITIVE TO PROVIDE LARGE SCALE
CLOUD COMPUTE FOR PEOPLE THAT
ARE BUILDING THIS KIND OF LARGE
SCALE MODEL.
THE OTHER THING THAT IS VERY
EXCITING IS THAT THESE MODELS,
THE WAY THEY CAN BE BUILT, THEY
ARE NOT JUST GOING TO BE -- THEY
ARE GOING TO BE MULTI-MODAL
GOING BEYOND, YOU KNOW, THE WAY
THAT MAYBE WHEN WE'VE BEEN
DISCUSSING TODAY IN TERMS OF
MOTOR THEY CAN'T COMBINE DATA
FROM DIFFERENT MODALITIES EVEN
WITHOUT HAVING THE SAME DATA,
THE SAME SPECIES, FOR EXAMPLE YOU
COULD COMBINE TRANSCRIPTOMIC DATA
FROM ECOLOGICAL MANIPULATIONS,
OTHER FIELDS ARE LEARNING
TRANSCRIPTOMIC LANDSCAPE OF BRAINS
OR OTHER ORGANS, OR PERFORMING
ECOLOGICAL MANIPULATIONS AND
DOING HIGH-THROUGHPUT
MEASUREMENTS, ONE COULD IN
THEORY BRING ALL THIS TOGETHER
IN THE FUTURE AND THEN HAVE A
LOT OF APPLICATIONS IN DISEASE
DIAGNOSIS FOR EXAMPLE OR EVEN
PREDICTING TREATMENTS FOR
DISEASES.
SO I JUST WANTED TO SHOW YOU
HERE ARE TWO EXAMPLES OF SOME
RECENT FOUNDATION MODELS, ONE BY
BLAKE RICHARD, EVA DYER, AND
LAJOLE LABS, WHERE THEY TOOK
DATA, LIKE MOUSE DATA,
BASICALLY USED TRANSFORMER
ARCHITECTURES, THEY HAVE BUILT
EARLY VERSIONS OF FOUNDATION
MODELS.
HERE ON THE LEFT IS ANOTHER
MODEL THAT IS BASED ON MICrONS
DATA THAT WE BUILT, THAT
ESSENTIALLY TAKES -- IS BUILDING
THESE MODEL, BUILT BY PUBLIC
DATA FROM MANY MICE, TRAINED ON
HUNDRED THOUSAND NEURONS, TRYING
TO LEARN RELATIONSHIPS BETWEEN
THE STRUCTURAL OF THE NEURAL
ACTIVITY. WHAT'S INTERESTING IS
MODELS ARE NOT JUST CAPABLE OF
PREDICTING NEURAL ACTIVITY, BUT
FOR EXAMPLE BECAUSE IN MICRONS
WE HAD FUNCTIONAL AND STRUCTURAL
DATA TOGETHER THE MODEL WAS
NEVER TRAINED ON ANATOMICAL DATA
COULD PREDICT MORPHOLOGICAL CELL
TYPES. CAN ACT AS MULTI-MODAL,
NOT JUST IN THE DOMAIN THAT THEY
ARE TRAINED, THAT'S SORT OF THE
ORIGINAL INSPIRATION OF CALLING
THEM FOUNDATION MODELS.
I THINK WE JUST LIKE TO CLOSE
WITH THIS SLIDE HERE, WHERE I
THINK SOME PEOPLE TALK ABOUT
PLATONIC SYMMETRY AND BEAUTY BUT
IF YOU THINK ABOUT IT WE HAVE
BRAINS WITH THE BODY OF COURSE,
WE HAVE THE WORLD, AND THEN WE
HAVE VISUAL NEURAL NETWORKS, ONE
FORM OR ANOTHER, SOME SORT OF IN
SILICO DEVICES NOW.
AND WE'RE TRYING TO FIND LIKE
RELATIONSHIPS, CORRESPONDENCES
BETWEEN THE THREE THINGS, HOW
THE WORLD IS ORGANIZED, HOW
BRAINS MODEL THE WORLD, HOW A.I.
MODELS THE WORLD, AND THE
TRIANGLE IS WHAT IT'S ALL ABOUT.
AND AT LEAST I BELIEVE AND MANY
OTHERS THAT USING DATA-DRIVEN
APPROACHES TO BUILD DIGITAL
TWINS OF BRAINS AND, YOU KNOW,
IT'S AN IMPORTANT ROADMAP TO
GET TO ISOMORPHISM BETWEEN BRAIN
AND THE WORLD.
I WOULD LIKE TO LEAVE IT THERE.
>> BLAKE?
>> WE'RE TAKING VIDEOCAST
QUESTIONS, WE WANT THIS RAPID
SESSION TO BE FORWARD LOOKING,
INTO THE NEXT TEN YEARS, SO
THINK BIG.
WHAT'S NEEDED, WHAT ARE GAPS,
WHAT DO WE NEED TO GET THERE?
>> HOW MUCH DATA COULD WE
POSSIBLY GET AND HOW THAT
COMPARE NOW TO LARGE LANGUAGE
MODELS?
>> YEAH, I THINK WE'RE NOT CRAZY
SCALABILITY WE CAN REACH IN
TERMS OF IN CLOSE TO TWO
PETABYTES OF DATA PER DAY FOR
EXAMPLE JUST ELECTROPHYSIOLOGY
ALONE, NON-HUMAN PRIMATES.
AND THEN, YOU CAN IMAGINE
HOW DO YOU TOKENIZE
THESE THINGS.
I THINK IN THE NEXT LET'S SAY
WITHIN THREE, FOUR, TO FIVE
YEARS, YOU KNOW, YOU CAN REACH,
YOU MAY HAVE NEEDS,
COMPUTATIONAL NEEDS OF THE
COMMUNITY THAT ARE AT THE SCALE
OF LET'S SAY LLAMA4 IS SUPPOSED
TO BE TRAINED ON.
>> YEAH, I'D AGREE WITH THAT.
I THINK ANDREAS HINTED, DEPENDS
HOW YOU TOKENNIZE AND CUT UP THE
DATA.
IF YOU RUN A BACK OF THE
ENVELOPE CALCULATION IT'S
POSSIBLE TO REQUIRE IN THE NEXT
FEW YEARS WITH CONCERTED EFFORT
AT THE SCALE OF DATA THAT THE
FIRST SETS OF LARGE LANGUAGE
MODELS WERE TRAINED ON.
>> IF I MAY PIVOT A TINY BIT
BEFORE YOUR QUESTION, MAYBE
AFTER YOUR QUESTION.
THE PURPOSE IS FORWARD LOOKING
TO SAY MAYBE IN TEN YEARS WILL
WE BE IN A POSITION TO
DISTINGUISH MODELS OF THE BRAIN
FROM REALLY GREAT ARCHITECTURES
THAT ALSO DO THINGS LIKE THE
BRAIN DOES?
>> IF I CAN ADD TO THE QUESTION,
THIS IS A QUESTION FOR THE
PEOPLE WORKING TOWARDS A.I., SO
PROBABLY BLAKE AND ALI.
A.I. IS A VERY
PERFORMANCE-DRIVEN FIELD.
SO, WHAT HAPPENS IF PUSHING
TOWARDS THE MODEL OF THE BRAIN
REQUIRES A STEP BACK IN
PERFORMANCE?
>> YEAH, SO I THINK THIS COMES
BACK TO THE DISCUSSION AROUND
METRICS OF COMPARISON, RIGHT?
SO INDEED I SUSPECT THAT ON SOME
METRIC MODELING THE BRAIN WOULD
REQUIRE A STEP BACKWARDS.
I THINK THOUGH IT'S SAFE TO SAY
THAT IN A LOT OF THE AREAS THAT
A.I. IS CURRENTLY POOR AT, WHAT
WE WANT IS MORE HUMAN-LIKE
BEHAVIOR FROM THE A.I. SYSTEMS.
SO, WE'RE NOT YET -- MARTIN WAS
SHOWING SOME OF THAT PLATEAU,
WITH SOME VISION MODELS, EVEN IN
VISION THE REALITY IS THAT ANY
A.I. RESEARCHER WOULD LOVE TO
HAVE A MODEL THAT'S MORE LIKE
HUMAN VISION THAN CURRENT A.I.
MODELS.
SO, I DON'T THINK WE'RE QUITE AT
THE POINT YET WHERE WE HAVE TO
WORRY ABOUT THAT TOO MUCH BUT
FORWARD THINKING MIGHT BE TEN
YEARS FROM NOW, AND I THINK THAT
THE INTERESTING QUESTION WILL
BE, YOU KNOW, TO WHAT EXTENT IS
THERE ACTUALLY LIKE A REAL
DIVERGENCE BETWEEN OUR A.I.
SYSTEMS AND HUMAN INTELLIGENCE
THAT'S JUST LIKE RADICALLY
DIFFERENT?
I THINK MY GUESS IS THAT IT
MIGHT NOT ACTUALLY END UP BEING
SO RADICALLY DIFFERENT INSOFAR
AS THE ECONOMIC IMPERATIVES
ARE TO CREATE A.I. SYSTEMS
TO MIMIC HUMAN INTELLIGENCE BUT
MAYBE NOT IMAGINING POTENTIAL
APPLICATIONS FOR A.I. TO DO
TOTALLY NON-HUMAN THINGS.
AND IF THAT EMERGES, IF THAT
CRYSTALLIZES, THEN I THINK YOU'LL
SEE THAT DIVERGENCE STRONGLY.
>> SO WHAT A POSSIBLE 10-YEAR
MOONSHOT GOAL BE AN IN SILICO
TRANSPLANT LIKE KIDNEYS OR
HEARING, MALADAPTIVE PROBLEM IN
THE BRAIN, WOULD YOU ABLE TO
RESTORE SIGHT, WOULD THAT BE A
HOLY GRAIL RATHER THAN METRICS
IN THE SHORT TERM?
IT'S A GREAT EXAMPLE OF A HOLY
GRAIL.
IF SOMEONE STUFFERS A STROKE IF
YOU COULD REPLACE THAT PART OF
THE BRAIN WITH A DIGITAL TWIN,
YES, A TOTAL AMAZING MOONSHOT TO
AIM FOR OR TO HAVE, YOU KNOW,
ADDITIONAL MODIFICATIONS OF YOUR
COGNITION.
A.I. ALREADY DOES THAT, BUT THEN
VIA YOUR HANDS AND THE SCREEN
BUT YOU CAN IMAGINE HAVING IT BE
MORE SEAMLESS WITH APPROPRIATE
BRAIN-COMPUTER INTERFACES.
>> I AGREE.
NICE INITIATIVE WELL DEFINED.
>> SO -- OH.
>> GO AHEAD.
>> JOE IS POINTING AT ME.
WANT TO PUSH ON HEALTH. WE'RE AT
THE NIH. LOT OF CONVERSATIONS
THAT I FIND LIKE HOW ARE WE
GOING TO GET NEUROSCIENCE DATA,
HAVING TRIED TO DO THIS FOR TEN
YEARS, LIKE RIDING A BIKE AND
TRYING TO JUMP ON A MOVING
TRAIN, THE FIELD IS MOVING SO
FAST WE'VE ALL TALKED ABOUT,
NEUROSCIENCE GOES SLOW.
THERE'S A BIGGER GOAL HERE,
MENTAL HEALTH, NEUROLOGICAL
DISORDERS, THIS IS HUGE.
THIS IS WHY WE'RE HERE.
IT'S WHY THE BRAIN INITIATIVE
EXISTS.
AND WHEN I HEAR THE VISION OF
THE FOUNDATION MODEL, I GUESS
THERE'S THIS -- SEEMS LIKE A BIG
PILL TO SWALLOW IF WE CAN THROW
THIS TOGETHER, DO A DIGITAL TWIN
OF BRAIN IN A.I. FORM, GREAT IF
THAT WORKS BUT TO TARGET
SOMETHING LESS MOONSHOTS OR
MAYBE MORE REALISTIC IS HOW ARE
WE GOING TO TAKE THIS SORT OF
STRUCTURE AND UNDERSTAND
SOMETHING, JUST UNDERSTAND THE
NEUROBIOLOGICAL UNDERPINNINGS OF
DEPRESSION OR SCHIZOPHRENIA TO
FIX IT. INSTEAD OF REPLACE,
STROKE IS LIKE A GROSS
IMPAIRMENT OF NEURAL
CIRCUITS, WE CAN REPLACE THAT
MAYBE.
BUT, YOU KNOW, WE HAVE MAJOR
NEUROLOGICAL DISORDERS NOT
MACROSCOPIC DAMAGE, RIGHT?
AND WE'RE NO WHERE NEAR
COMPUTATIONALLY UNDERSTANDING
HOW THEY WORK.
SO, MAYBE LIKE THE QUESTION IS
HOW CAN WE USE A.I. TO -- THE
SORT OF VISION THAT YOU'RE
INTRODUCING TO STRUCTURE AND
TARGET OUR COMPUTATIONAL
UNDERSTANDING OF THESE
DISORDERS?
>> I CAN ANSWER THAT.
THERE IS A VERY DIRECT PATH.
IN FACT THE BRAIN INITATIVE IS
FUNDING SOME OF THIS RESEARCH.
HERE IS THE PATH.
IF YOU HAVE A FOUNDATION MODEL
OF THE BRAIN WITH DEPRESSION,
YOU CAN STIMULATE, ONE OF THE
WAYS WE TRY AND TREAT DEPRESSION,
IN FACT WE HAVE EXPERTS IN
AUDIENCE, YOU CAN HAVE OPTIMIZE
-- BECAUSE BASICALLY YOU USE A
FOUNDATION MODEL TO FINE TUNE
FOR THE PARTICULAR PATIENT AND
USE ELECTRICAL STIMULATION TO
OPTIMIZE IN SILICO TO FIND THE
RIGHT STIMULATION PATTERN TO GET
THE ACTIVITY PATTERN TO THE
REGIME TO CURE DEPRESSION.
SO THAT'S A CONCRETE WAY OF AT
LEAST TRYING IN THE FUTURE TO
BUILD THESE KIND OF THINGS.
THE IDEA YOU HAVE A BRAIN STATE
THAT IS DEPRESSED, WE'RE GOING
TO GET THE BRAIN AWAY FROM THAT
BRAIN STATE, FIND THE OPTIMAL
STIMULATION PATTERN, HOW DO YOU
OPTIMIZE? CAN DO IT IN SILIO,
THROUGH DIGITAL TWIN, AND THEN
TEST IT CLOSED LOOP IN PATIENT.
WE HAVE A PROJECT LIKE THIS WITH
SAMEER SHETH AND XAQ PITKOW.
>> I'LL MAKE ANOTHER COMMENT,
WHICH WILL BELONG TOMORROW WHEN
WILL TALK ABOUT NEUROMORPHIC.
HOW TO CAPTURE ASPECTS OF BRAIN?
WE HAVE AN OPPORTUNITY TO
LOOK AT HOW SMALL CHANGES IN
PHYSICAL SYSTEMS AFFECT MAYBE
MORE MACRO BEHAVIOR. AND
UNDERSTANDING HOW TINY CHANGES
LIKE ION CHANNEL TURN INTO MENTAL
DISEASES IS SOMETHING I DON'T
NEUROSCIENCE HAS A HANDLE ON.
NEUROMORPHIC IS A PATHWAY TO COMPARE CHANGES THAT CAN BE TESTED IN A.I.
QUANTIFIABLY AND IN THE BRAIN.
>> THAT'S SOMETHING THAT IS
SOMETHING WE DON'T HAVE A GOOD
MATHEMATICAL HANDLE ON EITHER.
IT'S LIKE IF YOU GIVE ME A
MODEL, I KNOW EVERYTHING, EVERY
PARAMETER, IT'S AN ARTIFICIAL
INTELLIGENCE MODEL, WHAT'S GOING
TO HAPPEN WHEN I CHANGE
PARAMETERS, A BETTER
UNDERSTANDING OF HOW MOVING IN
THAT PARAMETER SPACE AFFECT, THE
DYNAMIC WILL HELP US TO
UNDERSTAND WHAT'S GOING WRONG,
BECAUSE A LOT OF THESE TRICKY
DISORDERS ARE DYNAMICS THAT ARE
GOING WRONG, BECAUSE SOMETHING'S
HAPPENING IN A MICROSCOPIC LEVEL,
YOU SEE THIS MACROSCOPIC ISSUE,
I THINK ABSOLUTELY WE NEED A
BETTER UNDERSTANDING THAT A.I.
QUESTIONS CAN HELP.
>> I'LL GO NEXT.
I REALLY APPRECIATED BLAKE'S
POINTS ABOUT IMAGE DISCUSSION
BASICALLY SAYING, HEY, TOUCH IS
KIND OF IMPORTANT.
AND SO IF WE'RE GOING TO BE
LOWER, LOWER LIFE FORM
REPRESENTATION FOR THE MOMENT,
COCKROACHES HAVE REALLY GREAT
FEELERS.
ONLY A MILLION NEURONS.
TO YOUR POINT WHERE IS THE
PROJECT TO EMULATE A COCKROACH
ANTENNA TODAY?
THERE'S A LOT OF WORK ON IMAGE
SENSORS, DELTA FRAME SENSORS,
HIGH ORDERS OF COMPLEXITY IN IMAGE
SENSING PIXEL, BUT WHAT ABOUT
THE ANTENNA?
SO SHOULDN'T WE BE FUNDING A
PROJECT TO WORK BACKWARDS FROM
THE SENSE, UNDERSTAND HOW
NEUROMORPHIC SIGNALS WORK, AND GO
BACKWARDS RATHER THAN FROM THE
BRAIN FORWARDS?
>> SO I'LL JUST SAY I MEAN I'M
FULLY ON BOARD.
I THINK IT WOULD BE FASCINATING
TO UNDERSTAND HOW A COCKROACH'S
ANTENNA WORKS.
SOME PEOPLE ARE TRYING TO DO
THIS KIND OF WORK, SO A
COLLEAGUE OF MINE AT HARVARD IS
DOING WORK ON BUILDING NOT
COCKROACH ANTENNA BUT VIRTUAL
RAT WITH A BODY THAT MOVES WITH
PROPRIOCEPTIVE SIGNALS, YOU
COULD IMAGINE STARTING TO
INCORPORATE TOUCH SIGNALS.
I'D LOVE TO SEE MORE OF THIS
WORK.
VIRTUAL BODIES, ALL SORTS OF
SPECIES, WOULD BE A WONDERFUL
THING, A WONDERFUL LIKE
LONG-TERM PROJECT FOR NEUROAI
AS WELL.
>> I MIGHT ALSO COMMENT THAT
TOMORROW WILL BE A GREAT TIME TO
BRING THAT CONVERSATION UP
AGAIN.
I THINK INVERTEBRATE SYSTEMS ARE
SMALL ENOUGH WE CAN BUILD THEM
WITH TECHNOLOGY AVAILABLE TODAY,
START TO LOOK AT WHAT HAPPENS IF
WE BUILD THE NEURO-INSPIRED,
ATTACH NEURO-INSPIRED SYSTEM,
SORRY, NEURO-INSPIRED SENSOR TO
THE NEURO-INSPIRED BRAIN SYSTEM.
SO MAYBE PUT A PIN IN THAT FOR
TOMORROW.
>> DR. HARTMAN?
>> NORTHWESTERN UNIVERSITY.
AND I STUDY SENSE OF TOUCH IN
RAT WHISKERS.
SO I WON'T BE TALKING ABOUT
COCKROACH ANTENNA BUT TOMORROW
I'LL TALK ABOUT MECHANICS OF RAT
WHISKERS.
I DID APPRECIATE THE COMMENT
ABOUT THE IMPORTANCE OF THE
SENSE OF TOUCH FOR ROBOTS.
AND WHILE I AGREE TOUCH IS
IMPORTANT FOR PROBLEMS LIKE
MANIPULATION OR OBJECT
IDENTIFICATION, I THINK THAT
WHAT IS REALLY IMPORTANT FOR
MAKING ROBOT BODIES WALK AROUND
IN THE WORLD, MOVE AROUND IN THE
WORLD, IS SENSE OF
PROPRIOCEPTION INTEGRATED WITH
MOTOR CONTROL, STILL A HARD
PROBLEM FOR ROBOTICISTS.
AND THE ISSUE REALLY ISN'T THE
AVAILABILITY OF SENSORY DATA,
FOR RIGID BODIES, WE KNOW
POSITIONS AND VELOCITIES. IT'S
DIFFERENT STORY FOR SOFT ROBOTS.
THE PROBLEM IS DEVELOPING
ADAPTIVE CONTROLLERS.
AND ONE OF THE BIG PROBLEMS WE
FACE IN ROBOTICS IS ROBOTS BREAK
EASILY, IT'S HARD TO GET DATA
FROM ROBOTS AS IT IS FROM
ANIMALS.
ROBOTICISTS OFTEN TURN TO
SIMULATIONS AS WE'RE TURNING TO
SIMULATIONS IN NEUROSCIENCE.
AND THEN THE PROBLEM YOU RUN
INTO, YOU CAN CREATE A
CONTROLLER THAT WORKS PERFECTLY
ON A SIMULATION BUT FAILS IN THE
REAL WORLD.
ONE OF THE TOOLS WE'LL NEED IN
THE FUTURE ARE BETTER
UNDERSTANDINGS OF CONTACT
MECHANICS, COLLISIONS, FUNDAMENTAL
PHYSICAL PRINCIPLES, FRICTION,
FLUID MECHANICS, TRIBOLOGY. WE
NEED MECHANICAL UNDERSTANDING
TO UNDERSTAND THE KINDS OF
SENSORY DATA AND MOTOR DATA THAT
THE BRAIN IS RECEIVING AND
CONTROLLING.
>> HARD TO DISAGREE WITH THAT.
>> YOU WERE DISCUSSING WHAT
WOULD BE REQUIRED WHAT WOULD BE
REQUIRED TO TRAIN THE NEXT
GENERATION OF A.I. STUDENTS,
PART WAS BLAKE SAYING MAYBE
STUDENTS NEED TO BE TRAINED UP
TO MATHEMATICAL FORMULAS,
TRAINED IN MAYBE ADVANCED
SYSTEMS AND COGNITIVE
NEUROSCIENCE.
RIGHT NOW LIKE THE MODEL OF HOW
WE DO THAT IS AD HOC.
UNDERGRAD TRAINING IS INFORMAL,
MARRY NEUROSCIENCE RELEVANT TO
YOUR PROJECT LATER, IF DO YOU
THAT PhD, if it's neuroscience
you catch up on formalisms later
on.
HOW DO YOU GO ABOUT TRYING TO DO
THAT?
HOW DO YOU EDUCATE A STUDENT IN
BOTH MATHEMATICS UP TO NONLINEAR
FEED BACK CONTROL BUT KNOW SUB-
STANTIVE NEUROSCIENCE, IF LIKE
UNDERGRADUATE CURRICULUM OR DO
WE KEEP WORKING MODEL OF TRAIN
EVERYBODY FAST IN Ph.D. WHAT
THEY ARE MISSING, OR CREATE LIKE
COLD SPRING HARBOR THAT IS A
THREE-MONTH INTERNSHIP, WHAT DO
YOU THINK ABOUT TRYING ACHIEVE
THAT KIND OF TRAINING?
>> I GUESS I DON'T HAVE A GOOD
ANSWER FOR YOUR TYPE OF TRAINING
BUT I WOULD, SAY I'M NOT SURE WE
CAN CREATE SOMEONE WELL VERSED
IN ALL FIELDS NECESSARY.
I WOULD ADVOCATE FOR MAKING SURE
PEOPLE HAVE BROAD ENOUGH
TRAINING SO THEY CAN SPECIALIZE
BUT NOT LOSE ABILITY TO
COMMUNICATE WITH OTHER FIELDS.
WHETHER THAT IS SOMETHING THAT
HAS TO BE INTERNSHIP OR GOOD
COLLABORATION, OR ENCOURAGING
MORE COLLABORATIONS OR EVEN
ENCOURAGING MORE WORKSHOPS LIKE
THIS IS AN OPEN-ENDED QUESTION.
MAYBE WE DON'T KNOW WHAT THE
RIGHT APPROACH IS.
BUT I DO THINK THIS ABILITY TO
COLLABORATE AND SEEK FREELY WITH
EACH OTHER AND TRADE IDEAS
COMING FROM DIFFERENT FIELDS,
MAYBE WE'RE SAYING THE SAME
THING WITH DIFFERENT LANGUAGE,
THAT'S AN IMPORTANT QUESTION TO
ADDRESS.
>> I HAVE ONE SMALL THOUGHT AS
WELL, JUST SORT OF RULE OF
THUMB.
IN MY EXPERIENCE, THE EARLIER
YOU LEARN FORMAL THEORY THE
BETTER.
NUMBER ONE.
NUMBER TWO, THE MORE THAT YOU
LEARN BIOLOGICAL THEORY FROM
ACTUALLY RUNNING EXPERIMENTS,
THE BETTER.
SO, I SUPPOSE MY INSTINCT FOR
HOW YOU WANT TO DO THIS IS YOU
MAKE IT SO THAT EVERYONE TAKES
THE CORE MATHEMATICAL COURSES IN
THEIR UNDERGRAD, AND EVERYONE
HAS TO AT SOME POINT IN THEIR
CAREER ACTUALLY TRY RUNNING AN
EXPERIMENT.
>> ON THE SUBJECT OF TRAINING,
I'VE WORKED ON THIS A LOT AND
THOUGHT ABOUT IT QUITE A LOT OF
DETAIL.
AND MOST RECENTLY I'VE HAD THE
PLEASURE OF TALKING TO ASHLEY
JUAVINET AT UCSD, AND I THINK WE
SHOULD BE HONEST WITH OURSELVES,
WE CAN'T JUST EXPECT STUDENTS TO
LEARN MORE OF EVERYTHING ALL THE
TIME.
IT'S UNREASONABLE.
I THINK THERE ARE SOME PARTS
OF CLASSICAL CURRICULUM THAT CAN
GO. DIFFERNTIAL EQUATIONS, ONE OF
MY FAVORITE PARTS OF MATHEMATICS
BUT WE DO NOT NEED TO TEACH
STUDENTS HOW TO SOLVE DERIVATIONS
ANALYTICALLY IN THE SAME DEPTH
THAT I LEARNED.
WE CAN TEACH PROGRAMMING IN THE
CONTEXT OF BIOLOGY WITHOUT
HAVING IT BE A COMPLETELY
SEPARATE CS CLASS PEOPLE HAVE TO
GO TO, ROLL DICE OR WHATEVER.
WE CAN DO THAT AND GENETICS AT
THE SAME TIME.
IT IS POSSIBLE TO NOT JUST
EXPECT STUDENTS TO LEARN MORE OF
EVERYTHING BUT TEACH IT IN A
FUNDAMENTALLY INTERDISCIPLINARY
WAY, AND THAT DOES INVOLVE
GETTING RID OF SOME PARTS OF THE
CURRICULUM THAT WE MORE
CLASSICICALLY TEACH, BUT IT'S
DOABLE.
I BELIEVE IT TO BE DOABLE.
>> I WANT TO ADD LIKE I'M ALL
FOR CORE CURRICULUM BUT WE DON'T
KNOW ALL THE TOOLS NEEDED TO
SOLVE THE BRAIN. MAYBE YOU NEED
TO KNOW SOME CATEGORY THEORY,
OTHER AREA OF KNOWLEDGE.
I DON'T THINK WE ALL NEED TO
TAKE THE SAME SET OF CLASSES.
>> YEAH, I ECHO THAT.
I THINK IT'S GOING -- I DON'T
THINK WE SHOULD GO FOR A ONE
A
ONE-SIZE-FITS-ALL MODEL OF
TRAINING.
PEOPLE DO HAVE DIFFERENT
TRAINING AND CAN ALSO WORK IN
THE SPACE, AND COLLABORATE AND,
YOU KNOW, IT'S VERY
INTERDISCIPLINARY FUNDAMENTALLY.
I WOULD ALSO SAY TO LOOK AT
PHYSICS, PHYSICS IS A GOOD
MODEL.
IT'S AN EXPERIMENTAL, AND, YOU
KNOW, THEORETICAL, COMPUTATIONAL
SCIENCE, SO THERE IS LIKE A CORE
IN SOME SENSE LIKE EVERYBODY
LEARNS SOME BASIC MATH, SOME
BASIC PHYSICS, AND THEN PEOPLE
GO OFF IN DIFFERENT DIRECTIONS
BUT THERE IS THAT, BECAUSE THERE
IS A SIGNIFICANT AMOUNT OF
MATHEMATICAL TRAINING I GUESS,
RIGHT?
WE DO HAVE THIS TRADITION THAT
GOES BACK A FEW HUNDREDS YEARS,
PHYSICISTS FORMULATING THE
PROBLEMS THAT THEY ARE THINKING
HARD ABOUT MATHEMATICALLY AND
ONCE YOU DO THAT LOTS OF PEOPLE
CAN WORK ON THEM.
SO I THINK THAT -- YOU KNOW, I
SORT OF AGREE WITH THE IDEA TO
HAVE A LITTLE MORE IN THE
BIOLOGY TRAINING, JUST TO BE
ABLE TO SORT OF WRITE DOWN YOUR
IDEAS OR FORMULATE PROBLEMS,
QUESTIONS, IN A WAY, IT'S REALLY
A COMMUNICATION THING, IRONIC
BECAUSE PEOPLE THINK OF MATH AS
A BARRIER TO COMMUNICATION BUT I
OBVIOUSLY THINK ABOUT IT AS A
WAY OF CLARIFYING AND IMPROVING
COMMUNICATION.
>> I HAVE SOME EXPERIENCE IN
THIS AREA ALSO.
AND HAVE DISCOVERED THAT TO
TEACH MATHEMATICS TO BIOLOGY,
NEUROSCIENCE, COGNITIVE SCIENCE,
PSYCHOLOGY STUDENTS AND
INCREASINGLY COMPUTER SCIENCE
STUDENTS, THEY TURN OFF IF YOU
GIVE THEM A CLASSICAL
PRESENTATION.
SO, I'VE CREATED A SERIES OF
ARTIFICIAL ORGANISMS LIKE
ARTIFICIAL COLI TO TEACH
GRADIENT DESCENT. MY FAVORITE
CREATURE IS LIMULUS LINEARIS,
PROVIDES A GLIMPSE OF LINEAR
OPERATOR THEORY IN A WAY THAT IF
YOU TOLD THEM THIS IS THE
CONVOLUTION THEOREM THIS IS
HILBERT SPACE, THEY ARE GONE BUT
STUDENTS LOVE PLAYING WITH THESE
CREATURES.
>> THE LESSON IS IT'S NOT JUST
PUTTING ONUS ON STUDENTS TO SEEK
ALL THINGS AND BECOME WORLD
EXPERTS ON ALL THINGS BUT ALSO
ON INSTRUCTORS TO COME UP WITH
CREATIVE SOLUTIONS PERHAPS
LEVERAGING NEW A.I. TOOLS.
PLEASE PLUG THE NIH TO DO THIS
ANNUALLY ALSO.
>> YEAH, I WANT TO ECHO BEFORE I
MOVE ON BACK TO THE QUESTION, I
THINK ABOUT MAYBE SOME OF THE
PROGRAMS BY THE WAY, MINDY SHI,
TEMPLE UNIVERSITY.
MAYBE MOVING FORWARD IN FIVE,
TEN YEARS IN THIS COMMUNITY, WE
CAN MOVE FORWARD WITH SOMETHING
LIKE COMPUTATION NEUROSCIENCE.
I THINK THERE'S SOMETHING THERE.
ALSO COMBINING DATA SCIENCE,
A.I., SOMETHING LIKE KIND OF
JOINT TRAINING PROGRAMS WOULD BE
HELPFUL, THINK ABOUT THEORIES OF
CURRICULUM THAT WE CAN PUT
TOGETHER COMBINING WHAT WE HAVE
JUST TALK ABOUT, I THINK THERE
MIGHT BE SOMETHING THAT WE CAN
PLAN AS COMMUNITY TO MOVE THIS
MOST SYSTEMATICALLY TOWARD THE
TRAINING AND MENTORING AND ALSO
COLLABORATIONS AS WELL.
SO GETTING BACK TO THE QUESTION
THAT I HAVE, LISTENING THROUGH
ALL THE SESSIONS, I JUST WANT TO
KIND OF LIKE COMMENT, ALSO LOOK
FOR INPUT FOR THIS IDEA IN MY
RESEARCH AND ALSO MY GROUP AND
MANY OTHERS WORK IN MACHINE
LEARNING AND FUNDAMENTAL
OPTIMIZATION, FUNDAMENTAL A.I.
I THINK IT'S NOT NEW IN THE
SENSE THAT WE'RE TALKING ABOUT
DATA COLLECTION.
WE'RE TALKING ABOUT EVALUATION
METRICS.
BENCHMARKS.
WE'RE ALSO TALKING ABOUT MODELS
MOVING FORWARD FROM LIKE UNI-
MODAL, MULTI-MODAL, EVEN JUST
MULTI-AGENT, SO ALL THESE IDEAS
ARE GREAT.
SO I'M JUST THINKING ABOUT ONE
THING THAT WE ACTUALLY STRUCK,
SOMETHING I THINK THAT HAS A
CONNECTION ABOUT BIAS AND NO
MATTER WHAT YOU PUT INTO IT,
HEALTH DISPARITIES, BIAS IN YOUR
MODELS, AND IF WE -- FOR FOLKS
WHO WORK ON FUNDAMENTAL A.I.
LIKE THE GENERATION GAP THAT YOU
SEE, IN YOUR TRAINING DATA,
RIGHT?
SO IF YOU THINK ABOUT THIS
COMMUNITY, YOU HAVE VERY NICE
FRAMEWORK MAYBE TO TOUCH UPON
THE ISSUES THAT WOULD HELP A LOT
OF FOLKS WORK ON METHOD
DEVELOPMENT AND THEN FEEDBACK TO
BETTER UNDERSTAND THE BRAIN, THE
BIOLOGY, WOULD -- - HOW DO YOU
THINK ABOUT CREATING DATASETS
THAT YOU KIND OF LIKE SAMPLE THE
BRAIN ACTIVITIES, WHATEVER DATA
ENTITIES THAT WE CAN COLLECT AND
FEATURES WE CAN COLLECT SO THAT
WE CAN CREATE A BENCHMARK,
CREATE DATASETS THAT WE CAN FIND
THE DISTRIBUTIONS OF THE DATA OR
PARAMETER SPACE THAT WE HAVE ON
THE TRAINING DATASETS WOULD BE
CLOSE ENOUGH OR TO THE
SIMULATION DATA, TO THE TESTING
DATASET, WHATEVER WE HAVE, SO
THAT WE CAN BUILD SOME MODELS
THAT ACTUALLY ARE USEFUL AND
WHEN WE PUT THEM INTO
IMPLEMENTATION, INTO PRACTICE,
THAT WE DON'T SEE THAT MUCH GAP
IN THE PERFORMANCE OF THE
MODELS, NO MATTER WHAT YOU ARE
DEALING WITH PREDICTION,
FOUNDATION MODELS, TRANSFER OF
LEARNING, SO ON AND SO FORTH, TO
MY -- FROM MY PERSPECTIVE VERY
FUNDAMENTAL PROBLEMS IN MANY
FIELDS, NEUROSCIENCE, NEUROAI
HOPEFULLY THAT YOU HAVE SOME
INPUT OR FEEDBACK CAN IMPROVE
THIS.
YEAH, THANKS.
>> YEAH, FULLY AGREE.
I THINK THIS WOULD WORK, I THINK
ONE CLEAR PATH FOR THE FUTURE IS
TO BRING DIFFERENT ALSO
FOUNDATION MODELS FROM DIFFERENT
FIELDS, COMBINED TOGETHER, AND
THEN ALSO HAVE APPLICATIONS
BEYOND JUST THE THEME HERE IS
ABOUT NEUROSCIENCE FOR A.I. BUT
ALSO OTHER APPLICATIONS.
>> IF I MAY, PART OF IT IS ALSO
CULTURAL.
WE'RE NOT REALLY USED TO
THINKING LIKE YOU ALL DO, AS
NEUROSCIENTISTS, TO CREATE
BENCHMARKS.
IT'S STILL NEW -- STILL A NEW
THING.
THANKS TO THE BRAIN INITIATIVE
WE'VE BEEN THINKING ABOUT
DISSEMINATION, SO RIGHT NOW IS
EXACTLY THE KIND OF THING
ANDREAS IS TALKING ABOUT,
FOUNDATION MODELS BLAKE AND
GROUP ALSO, AND PATRICK TALKED
ABOUT IN THE MORNING.
ALL OF THOSE ARE IN THE
DIRECTION THAT YOU WERE TALKING
ABOUT SO KEEPING AN EYE ON THOSE
PAPERS.
>> PAMELA ABSHIRE, UNIVERSITY
OF MARYLAND. AS WE START
THINKING HOW TO TRAIN A NEW
GENERATION, AND THERE'S SO MUCH
MORE STUFF THAT WE NEED TO PILE
INTO PEOPLE'S HEADS.
LET'S NOT LOSE THE JOY OF
EXPLORATION THAT DRIVES ALL THE
PEOPLE IN THIS ROOM SO LET'S
LEAVE SOME SPACE WITHIN THAT
EDUCATIONAL PROCESS FOR
EXPLORATION AND DISCOVERY AND
EXPERIENTIAL LEARNING SO PEOPLE
CAN WORK ON SOME OF THESE
PROBLEMS AND GET A REAL SENSE OF
IT.
MAN, I WANT TO PLAY WITH STEVE'S
BUGS.
SO THE COMMENT IS SIMILAR TO THE
LAST ONE, HOW DO WE -- I HAVE A
QUESTION.
THE QUESTION IS HOW DO WE AVOID
GETTING TRAPPED IN OUR OWN
EXPERIMENTAL BIAS, BOTH IN
TERMS -- AS WE BUILD THESE
MODELS HOW DO WE AVOID GETTING
TRAPPED IN BIAS OF OUR
EXPERIMENTAL CONSTRUCT IN TERMS
OF THE NEUROPHYSIOLOGICAL
EXPERIMENTS AND ALSO IN TERMS OF
THE A.I. MODELS THAT WE HAVE TO
DESCRIBE THEM.
MAYBE THIS HARKENS BACK TO WE
NEED BETTER MATH BUT I THINK
IT'S BIGGER THAN THAT BECAUSE
LIKE THE REAL ISSUE IS GOING
BACK TO THE FIRST PRESENTATION
TODAY, WE DON'T WANT TO MISTAKE
CHIHUAHUAS FOR BLUEBERRY
MUFFINS, GET TRAPPED INTO THESE
ARTIFACTS AND JUST LOOKING AT
THE TEXTURE OF THE SKIN OF THE
ELEPHANT, NOT TELLING IT'S A
TAIL OR TRUNK OR WHATEVER,
RIGHT?
NO MATTER -- GIVEN THE SECOND
PRESENTATION HE MADE, NO MATTER
HOW MANY PETABYTES OF DATA YOU
COLLECT YOU'RE NOT GOING TO
EVER -- YOU'RE ALWAYS GOING TO
BE IN A DATA-POOR SITUATION FOR
LOOKING AT THE CORRELATIONS
BETWEEN ALL THOSE DIFFERENT
VARIABLES, YOU'LL HAVE
RELATIVELY FEW EXAMPLES FOR
LOOKING FOR THE REAL
CORRELATIONS IN THE REAL
UNDERLYING FACTORS WHEN WE PUT
IT TOGETHER.
HOW DO WE AVOID GETTING TRAPPED
INTO OUR OWN EXPERIMENTAL BIAS
FOR TRYING TO REALLY FIND HOW
THESE THINGS ARE WORKING, YOU
KNOW, I'LL END WITH WHAT I THINK
MAY BE CONTROVERSIAL, I DON'T
KNOW.
WHAT I'M TAKING AWAY FROM LARGE
LANGUAGE MODEL LEARNING AND
PERFORMANCE THAT WE'VE SEEN
RECENTLY ISN'T SO MUCH THAT
ARTIFICIAL INTELLIGENCE IS
GREAT.
IT'S, WOW, WHY DIDN'T WE
UNDERSTAND EARLIER THAT LANGUAGE
IS A LOOKUP TABLE, RIGHT?
I THINK WE'RE LEARNING MORE
ABOUT THE NATURE OF THE PROBLEM
THAN WE ARE ABOUT THE NATURE OF
INTELLIGENCE WITH THE TOOLS THAT
WE HAVE.
>> I SAW DORIS SMALLING, ANDREAS
SMILING AT DIFFERENT PARTS OF
THE QUESTION.
>> EVERYTHING NON-TRIVIAL.
YOU GO IN WITH HYPOTHESIS,
YOU'RE PROVED WRONG BY THE DATA.
>> AND TO ADD TO THAT WHEN YOU
BUILD THESE MODELS, YOU DO
CLOSED LOOP EXPERIMENTS, SO THE
IDEA IS THAT YOU WILL GENERATE
SOME HYPOTHESES, SOME STIMULI,
TEST THEM BACK.
YOU MAY BE RIGHT THAT HOW MUCH
DATA YOU NEED MAYBE WE DON'T
HAVE ENOUGH DATA.
YOU CAN TEST, YOU CAN BUILD THE
MODEL, DO SOME DISCOVERY OR
ANALYZE THE MODEL, YOU FIND
SOMETHING, TEST IT BACK TO
CLOSED LOOP EXPERIMENTS.
WITHOUT THAT YOU MAY BE DOING
SOME KIND OF FANCY LOOKUP TABLE.
BETTER IN LARGE LANGUAGE MODEL
BUT YEAH, I THINK IT'S
THROUGH -- YOU NEED THIS TIGHT
LINK WITH EXPERIMENTS.
>> I MIGHT MAKE A COMMENT.
MAYBE I HEARD YOUR QUESTION
DIFFERENTLY.
YOU'RE ALMOST ASKING HOW DO YOU
AVOID GETTING OVERFIT TO YOUR
SPECIFIC EXPERIMENTAL SYSTEM.
THE ANSWER IS COLLABORATIONS
WITH PEOPLE WHO AREN'T IN YOUR
SPECIFIC SYSTEM.
IT MAY BE THAT YOUR SYSTEM HAS A
PARTICULAR ANSWER, BUT IN TERMS
OF IMPACTING A.I., TALKING TO
THE A.I. PEOPLE AND SAYING MAYBE
THIS ISN'T THE MOST USEFUL
SITUATION FOR YOUR SPECIFIC
MODEL.
WE CAN LEARN FROM THOSE
COLLABORATIONS AND DISCUSSIONS.
>> SO, I HAVE A GENERAL COMMENT.
RAMANI DURAISWAMI, UNIVERSITY OF
MARYLAND. SO, WHEN A.I. -- THE
FASHION IN A.I. KEEPS CHANGING.
WE HAD MLPs, CONVOLUTIONAL
NEURAL NETWORKS, TRANSFORMERS,
NOW PERHAPS STATE-SPACE MODELS,
ET CETERA.
EVERYBODY GOES ALONG WITH THAT.
ON THE OTHER HAND NEURAL SYSTEMS
SEEM TO BE SYSTEMS DEVELOPED FOR
TASK-BASED SOLVING OF PROBLEMS.
YOU WANT TO NAVIGATE THROUGH THE
WORLD, RECOGNIZE SOMETHING, ET
CETERA, ET CETERA.
AND THE ARCHITECTURES ARE
PERHAPS NOT UNIFORM ACROSS THE
DIFFERENT SYSTEMS, THERE MIGHT
BE SOME COMMONALITY BUT THERE
ARE ALSO SIGNIFICANT
DIFFERENCES.
AND IN THE TREATMENT OF DISEASE
WHAT HAPPENS IS THERE ARE
SIGNIFICANT VARIATIONS AT THE
INDIVIDUAL LEVEL, SO IN SOME
SENSE, THE TWO SYSTEMS WHILE
HAVING SOME ERROR-DRIVEN
OPTIMIZATION, WILL BE DIFFERENT
ENOUGH ALL THE TIME IN THE SENSE
THAT THE NEURAL SYSTEMS WILL
HAVE VARIATIONS WHICH ARE -- AND
VARIATIONS WILL GO DOWN TO THE
INDIVIDUAL LEVEL.
SO, ANY SORT OF TREATMENT,
ANYTHING ELSE WOULD REQUIRE
ALMOST PERSONALIZED MODELS FOR
EVERY ONE OF THE 7.5 BILLION
PEOPLE OR WHATEVER, AND SO, IN
SOME SENSE I THINK WE SHOULD --
AND SOMEONE ELSE HAD ASKED THE
QUESTION WE SHOULDN'T LOSE SIGHT
OF DISEASE AND SO FORTH.
I THINK PART OF THAT COMES FROM
HAVING THE ABILITY IN MODELS TO
HAVE THIS KIND OF VARIABILITY
AND ABILITY TO FIT THE
INDIVIDUAL KIND OF A THING.
>> IF I MIGHT COMMENT ON THAT SO
ONE OF THE REASONS I'M A LARGE
ADVOCATE FOR FOUNDATION MODEL
APPROACH IN NEUROSCIENCE IS
BECAUSE WHAT FOUNDATION MODELS
BRING YOU IS THE ABILITY TO FINE
TUNE VARIOUS SMALL AMOUNTS OF
DATA.
I AGREE WITH YOU.
YOU WILL NEED PERSONALIZED
MODELS TO EXPLAIN WHAT'S GOING
ON FOR SOMEONE WHO IS SUFFERING
FROM, SAY, A MENTAL HEALTH
DISORDER.
BUT THE WHOLE POINT IS THAT IF
YOU DO SUFFICIENTLY POWERFUL
PRE-TRAINING ON A LARGE ENOUGH
DATASET, YOU WILL THEN BE ABLE
TO FINE TUNE AND INDIVIDUALIZE
THAT MODEL FOR YOUR INDIVIDUAL
WITH ACTUALLY REALISTIC AMOUNT
OF RECORDING FROM THEM.
>> IF I MAY, SO ONE THOUGHT, THE
FOUNDATIONAL MODELS BY THE VERY
DEFINITION ARE EXTREMELY LARGE.
AND THAT IN SOME SENSE YOU'RE
SORT OF HAVING TO OVERCOMPLICATE
THINGS TO SIMPLIFY LATER.
ANYWAY, THAT'S JUST --
>> YEAH, THAT MIGHT JUST BE THE
SITUATION WE FIND OURSELVES IN.
>> IF WE COULD HAVE A QUESTION
FROM THE OTHER SIDE.
>> YES.
THANK YOU. AM I CLEARLY AUDIBLE?
I'M RUCHIRA DATTA.
YES, I'M A MATHEMATICIAN, AND
I'VE GOT QUITE A BIT OF
EXPERIENCE IN COMPUTATIONAL
BIOLOGY, QUITE A BIT OF
EXPERIENCE IN MACHINE LEARNING
AND A.I., MOSTLY HAVEN'T
INTERSECTED, THIS IS MY FIRST
INTRODUCTION TO COMPUTATIONAL
NEUROSCIENCE AT THIS CONFERENCE
I HEARD FROM CARINA CURTO THE
VISION OF MATH FOR COMPUTATIONAL
NEUROSCIENCE OF BEING ABLE TO
UNDERSTAND THE NATURE OF THE
PARAMETER SPACE AND, YOU KNOW,
THE INVARIANCE AND VARIANCE AND
I HEARD FROM ANDREAS DID THIS
BROAD VISION OF HAVING THESE
NATURALISTIC DATASETS, HIGH
ENTROPY DATASETS, LARGE AMOUNTS
OF COMPUTE.
AND POSSIBLY BOTH OF YOU HAD IN
MIND THAT INNER VISION FOR THE
FUTURE WHAT THE OTHER PERSON WAS
DESCRIBING WOULD HAPPEN BUT IF
YOU COULD SAY HOW DO YOU SEE
THESE TWO THINGS INTERFITTING IN
THE FUTURE?
THE MATH AND THE NATURALISTIC
DATASETS, LIKE DO YOU HAVE LIKE
A PROGRAM IN MIND?
>> WE DON'T HAVE A PROGRAM.
BUT I MEAN I THINK THE WAY I
WAS -- WHAT I WAS TRYING TO SAY
IS THAT WHEN YOU FIT MODELS, THE
MODELS COME FROM A MODEL FAMILY,
WITH SOME PARAMETERIZED MODEL
FAMILY SO UNDERSTANDING THAT
FAMILY BETTER IS VALUABLE IN
UNDERSTANDING WHAT KINDS OF
THINGS CAN BE TRAINED EVEN IN
PRINCIPLE.
MAYBE YOUR MODEL FAMILY DOESN'T
EVEN HAVE THE CAPACITY TO DO --
TO CONTAIN SOME NETWORKS THAT DO
ALL THE TESTS YOU WANT IT TO DO.
SOMEHOW HAVING A BETTER
UNDERSTANDING OF THE FAMILY OF
OBJECTS, RIGHT, THAT YOU'RE
LIVING IN THE PARAMETER SPACE
WHEN YOU'RE TRAINING YOU'RE
STUCK IN THE PARAMETER SPACE,
MOVING AROUND, UNDERSTANDING
THAT BETTER CAN HELP DEVELOP
BETTER TRAINING PROTOCOLS BUT
ALSO CAN HELP PICK OUT DIFFERENT
ARCHITECTURES, SO SOME
ARCHITECTURES ARE FUNDAMENTALLY
EASIER TO TRAIN FOR CERTAIN
TASKS THAN OTHERS AND WE DON'T
KNOW WHAT THEY ARE.
MAYBE THERE'S CERTAIN KIND OF
TASKS WELL THIS IS THE
ARCHITECTURE YOU SHOULD START
WITH, RIGHT, AND NOT BOTHER
WITH, YOU KNOW, THIS GIANT --
THESE NETWORKS, REALLY YOU WANT
TO START WITH SOMETHING SMALLER
WITH A CERTAIN STRUCTURE AND
YOU'LL TRAIN IT QUICKLY.
IN SOME SENSE I LIKE TO THINK
EVOLUTION HAS DONE THAT FOR LOTS
OF CREATURES FOR DIFFERENT
TASKS.
THEY STARTED YOU OFF WITH LIKE
AN ARCHITECTURE THAT'S BETTER
AND THEN, YOU KNOW, THROUGH
TRAINING, THROUGH EXPERIENCE,
THE ANIMAL FINE TUNES.
AND SO I THINK THERE'S THIS
INTERPLAY BETWEEN MODEL FAMILY,
YOU KNOW, WHICH CAN BE DEFINED
VIA ARCHITECTURE AND A CERTAIN
SHAPE TO THE PARAMETER SPACE,
AND THEN THE DATA, SO YOU WANT
TO FIT THOSE THINGS TOGETHER,
DEPENDING ON THE KIND OF DATA
THAT YOU'RE GOING TO TRAIN WITH
THERE MIGHT BE BETTER OR WORSE
ARCHITECTURES, BETTER OR WORSE
NETWORK FAMILIES TO START WITH.
AND SO ANDREAS WILL SAY MORE BUT
HIS POINT IS LIKE LET'S LOOK AT
DATA THAT ISN'T BIASED TOWARDS
CERTAIN KINDS OF MODELS SO WE
GET A MORE ROBUST -- SO WE'RE
FITTING MORE ROBUST NETWORKS IS
MY UNDERSTANDING BUT I'LL LET
HIM ELABORATE.
>> ULTIMATELY IF WE HAVE THESE
MODELS, GENERATE DATA FROM THEM,
UNDER CONDITIONS THAT WE CAN
DESCRIBE THE SYMMETRIES, IN THE
NEURAL DATA, ALONG THE LINES OF
WHAT DORIS WAS TALKING ABOUT,
FUNDAMENTAL INVARIANCE IN
PHYSICS AND STUFF LIKE THAT, SO
PLAYS A CRITICAL ROLE.
PROBLEMS ARE DEALING WITH DATA
MORE NOISY, ONE ADVANTAGE OF
LARGE SCALE MODELS, FOUNDATIONAL
MODELS CAN RUN LIKE BASICALLY ON
UNLIMITED NUMBER OF EXPERIMENTS
SO WE CAN GET A BETTER SAMPLE
COMPLEXITY OF THE MODEL.
AT THE END OF THE DAY WE HAVE TO
LINK IT MATHEMATICAL, TO
MATHEMATICS, FIELDS LIKE
TOPOLOGY, GEOMETRY, AND THINGS
LIKE THAT.
GROUP THEORY.
YOU'RE THE EXPERT TOO.
>> I AGREE.
GRACE, CAN WE DO ONE MORE?
ONE LAST QUESTION.
>> THANK YOU.
>> OKAY. JIM ZHENG
UNIVERSITY OF TEXAS HEALTH
CENTER IN HOUSTON.
WONDERING WHETHER THE PANEL CAN
ADDRESS THE ISSUE OF A.I.
DISPARITY LIKE MANY RESEARCH
GROUPS THEY, YOU KNOW, DON'T
HAVE ACCESS TO A.I. TECHNOLOGY
OR COMPUTING RESOURCE OR THEY
REALLY DON'T HAVE ANY A.I.
EXPERTS TO COLLABORATE WITH.
SOME GROUPS, THEY HAVE ALL THESE
RESOURCES, THEY REALLY EXCEL IN
THE CURRENT ENVIRONMENT.
>> I THINK THAT'S A VALID POINT.
NOT JUST VALID BUT VERY
IMPORTANT POINT.
MY OWN PERSONAL ANSWER TO IT IS
THAT I WOULD REALLY LIKE TO SEE
GOVERNMENT INVESTMENT IN COMPUTE
SO THAT THERE CAN BE MORE
EQUITABLE ACCESS TO A.I.
RESOURCES.
THERE MIGHT BE OTHER SOLUTIONS,
BUT THAT SEEMS LIKE THE MOST
OBVIOUS ONE TO ME.
>> I FULLY AGREE, THIS IS A
MAJOR ISSUE.
I WOULD SAY THE FIRST STEP IS TO
REACH OUT TO FIND COLLABORATORS
THAT ARE DOING -- THERE'S A LOT
OF -- THESE DAYS THERE'S A LOT
OF PEOPLE THAT ARE DOING A.I.
RESEARCH AND ARE VERY
INTERESTED, STUDENTS TO DO
NEUROSCIENCE RESEARCH, BECAUSE,
YOU KNOW, THAT FIELD IS SO
COMPETITIVE DOMINATED BY
INDUSTRY, THE ISSUE BECOMES
FINDING ACCESS TO GPUs, AND
BUT THERE ARE SOME RESOURCES BUT
THIS IS A BIG PROBLEM FOR
EVERYBODY.
IN ACADEMIA.
BECAUSE WE CANNOT COMPETE WITH
RESOURCES THAT INDUSTRY GETS,
RIGHT?
THAT'S FUNDAMENTAL PROBLEM.
>> YEAH.
>> THERE'S A LOT OF RESOURCES
THAT OF COURSE SHOULD BE
STRENGTH BIND MORE GOVERNMENT
INVESTMENT BUT THERE'S A LOT OF
FEDERALLY PROVIDED COMPUTE
CLUSTERS THAT VERY FEW PEOPLE
SEEM TO KNOW ABOUT FOR REASONS
THAT ARE STILL MYSTERIOUS TO ME.
NSF HAS ITS OWN CLUSTERS
ESSENTIALLY COST FREE FOR
STUDENTS AND TRAINEES, WRITE A
PROPOSAL, AND THEY GIVE YOU.
NATIONAL LAB OPENS COMPUTE
CLUSTERS TO PEOPLE, THE KIND OF
THING THAT REQUIRES A BETTER
WEBSITE OR MORE PRO-ACTIVE
E-MAILING.
>> I WANT TO ADD THAT AS WE LOOK
TO PROVIDE MORE COMPUTE WE HAVE
TO MAKE COMPUTE MORE ENERGY
EFFICIENT.
>> KEN WHANG WANT TO COMMENT
ABOUT THE NSF COMPUTE RESOURCES?
>> SURE.
CAN YOU HEAR ME?
SO, ONE THING, THE ISSUE OF A.I.
RESEARCH RESOURCES IS A TOUGH
ONE BUT THERE IS -- YOU KNOW,
SOMETHING THAT IS GETTING OFF
THE GROUND CALLED THE NATIONAL
A.I. RESEARCH RESOURCE.
WHICH IS A COLLECTION OF
RESOURCES THAT ARE HONESTLY
CURRENTLY A LITTLE BIT TRICKY TO
NAVIGATE, BUT SPECIFICALLY THEY
ARE TO TRY TO GET THE SAME KINDS
OF OR SIMILAR KINDS OF RESOURCES
FOR RESEARCHERS AS ARE AVAILABLE
TO FOLKS IN INDUSTRY.
THE OTHER THING, ACTUALLY WE'LL
TALK TOMORROW ABOUT NSF-FUNDED
RESOURCE FOR FOLKS WHO ARE
WORKING ON NEUROMORPHIC
HARDWARE.
TAKE A LOOK AT RESOURCES
AVAILABLE FROM NSF INVESTMENT.
THERE'S A LOT OUT THERE AND
SYSTEMS THAT YOU CAN GO TO THAT
WILL HELP YOU RECOMMEND WHAT
KIND OF RESOURCE YOU SHOULD BE
USING BASED ON THE KINDS OF
COMPUTE JOBS YOU HAVE.
SO THERE IS ACTUALLY A LOT OUT
THERE.
AND YOU SHOULD TAKE ADVANTAGE OF
IT.
YOU ALSO DON'T NEED TO BE NSF
FUNDED TO USE THESE RESOURCES.
>> I WANTED TO MENTION TOO THAT
HIGH PERFORMANCE COMPUTING
RESOURCES ARE PROVIDED TO PEOPLE
WHO HAVE ACTIVE DOD FUNDING,
COULD BE A CONTRACT, A GRANT,
BUT YOU -- THERE ARE
RESTRICTIONS.
USERS HAVE TO BE
PERMANENT RESIDENTS OR U.S.
CITIZENS, A BACKGROUND CHECK IS
REQUIRED. SO THERE IS A BAR.
BUT BEYOND THAT,
THE RESOURCES ARE AVAILABLE AT
NO COST.
>> THANK YOU, HAL.
WITH THAT, I WANT TO EXPRESS A
TREMENDOUS GRATITUDE FOR
EVERYONE WHO HAS STAYED WITH US
THIS LONG AND TO A JAM-PACKED
AGENDA, THANK YOU FOR CONCLUDING
THE FIRST DAY OF THE BRAIN
NEUROAI WORKSHOP. THANK YOU
KANAKA AND STEVE FOR CO-MODERATING
AND EVERYONE HERE AND GAVE US
EXCELLENT PRESENTATIONS AND
DISCUSSIONS THROUGHOUT THE DAY
WHICH WE'LL CONTINUE TOMORROW.
SO I WAS GOING TO GIVE A RECAP
AT 8:30 IN THE MORNING BUT LET'S
CALL THAT 8:40-ISH.
GET HERE EARLY ENOUGH.
THIS IS THE POSTER BLITZ THAT'S
REALLY IMPORTANT.
8:45 TOMORROW MORNING.
YOU'LL HEAR FROM EXCELLENT
POSTER PRESENTERS AND THEN CAN
GO OUT AT 9:00 AND HEAR SOME
GREAT SCIENCE.
SO THAT IS MOST OF EVERYTHING I
NEED TO TELL YOU.
I THINK YOU KNOW WHERE THE EXITS
ARE.
DON'T GET LOST IN THE DARK
PLEASE.
THERE'S SOME TURNSTILES ON THE
WAY TO THE METRO CENTER IF
THAT'S THE DIRECTION YOU'RE
GOING.
BUT AGAIN, THANK YOU TO
EVERYONE.
WE'LL SEE YOU TOMORROW.
[APPLAUSE]