MY NAME IS KENNY AND I AM IN THE POSTDOCTORAL RESEARCH TRAINING PROGRAM AND IT'S NATIONAL POSTDOC APPRECIATION WEEK. SO LET'S GIVE A ROUND FOR THE POSTDOCS! [ APPLAUSE ] AND THOSE WATCHING ON LINE, THANK YOU FOR THOSE WHO ARE WATCHING. ONE OF THE THINGS WE WANTED TO DO IS MAKE SURE THAT THE POSTDOCS HERE ARE EXPOSED TO DIFFERENT LEADERS IN THE SCIENCE SPACE AND ONE WE ARE REALLY EXCITED TO HAVE, JESSICA IS WITH US HERE TODAY. JESSICA DID HER UNDERGRAD AT U IN. C, UCSF AND POSTDOC AT HARVARD AND NOW AN EXECUTIVE DIRECTOR OF ASAP BIOAND THE PRESIDENT OF THE FUTURE OF RESEARCH. AND I THINK, ONE OF THE THINGS THAT IS REALLY INTERESTING IS OFTENTIMES WE SEE THINGS WRONG IN THE SYSTEM AND WE COMPLAIN TO EACH OTHER AND THEN GO BACK AND DO OUR EVERYDAY LIFE. JESSICA, AND HER COLLEAGUES, HAVE TAKEN THE CHALLENGES THEY SAW AS IT RELATES TO THE TRAINING OF SCIENTISTS, AND ISSUES AND PEER REVIEW AND CREATED? MOVEMENT AROUND THAT TO TRY TO REALLY RAISE THE ISSUES THAT HAVE BEEN HAPPENING AND TO TRY TO PUT THAT INTO POLICY. SO JESSICA MIGHT TALK ABOUT THE NATIONAL ACADEMY'S NEXT GENERATION RESEARCHER'S INITIATIVE STUDY. AND SO, WORKING WITH FUTURE RESEARCH. I'M REALLY EXCITED TO HAVE JESSICA HERE WITH US SO PLEASE GIVE JESSICA A WARM WELCOME. [ APPLAUSE ] >> THANK YOU KENNY, FOR THAT INTRODUCTION IT'S SO EXCITING TO BE HERE, ESPECIALLY DURING NATIONAL POSTDOC APPRECIATION WEEK. POSTDOCS ARE OBVIOUSLY, AS WE ALL AGREE, A HUGE PART OF THE ENGINE OF BIOMEDICAL RESEARCH DISCOVERY. THEY ARE PERFORMING THE LABOR THAT GOES INTO PRODUCING SCIENCE AND MAKING DISCOVERIES, YET AS WE ARE ALL PROBABLY AWARE, THERE IS SOMETHING OF AN INFORMATION DEFICIT AROUND THE POSTDOC POPULATION AND THE POSTDOC EXPERIENCE. SO, THIS CHART WE DEFINITELY NEED TO UPDATE IT BUT ONE THING THAT STICKS OUT IS THAT WE DON'T EVEN KNOW HOW MANY POSTDOCS THERE ARE IN THE UNITED STATES. TO SOME CONCRETE LEVEL; WHICH IS KIND OF AMAZING WHEN YOU THINK ABOUT THE VITAL AND IMPORTANT ROLE THAT YOU ARE ALL PLAYING IN THIS. SO I'M GOING TO START OFF WITH A CALL FOR MORE INFORMATION ABOUT POSTDOCS WHICH WILL HELP CREATE AND OPTIMIZE AND IMPROVE THIS POSTDOC EXPERIENCE. AND THIS IS NOT SOMETHING THAT IS A NEW IDEA. SO KENNY MENTIONED THIS NATIONAL ACADEMY'S REPORT, THE SISTER TO THE ONE THAT KENNY WORKED ON, BUT OUR REPORT IS ONE IN JUST A VERY LONG LINE OF REPORTS ADDRESSING THE POSTDOC SITUATION. SO FOR EXAMPLE, ALMOST LITERALLY 50 YEARS AGO NOW THERE IS A REPORT TITLED, THE INVISIBLE UNIVERSITY, ABOUT HOW POSTDOCS WERE AN UNACKNOWLEDGED CONTRIBUTOR TO RESEARCH IN THE UNITED STATES. SO A LOT OF THESE PROBLEMS HAVE PERSISTED FOR A VERY LONG TIME. AND AT THE SAME TIME, OUR MODEL OF WHAT THE POSTDOC EXPERIENCE IS, THERE ARE GREAT STRIDES BEING MADE TO CHANGE IT YET I THINK THE CULTURE IS PERHAPS NOT CHANGING AS QUICKLY AS IT COULD BE. FOR EXAMPLE, THIS IS A SLIDE -- I SHOULD SAY ALSO THAT THESE SLIDES ARE LINKED FROM A TWEET FROM THE NIGMS ACCOUNT. SO YOU CAN CLICK ON THESE LINKS AND FIND THE SOURCES. THE MOST POSTDOCS, ACCORDING TO A SURVEY BY HENRY SOUR MAN AND MICHAEL ROACH, THE POSTDOCS IN BIOLOGY AND THE LIFE SCIENCES WHO PLANNED TO GO INTO A POSTDOC ARE DOING SO PRIMARILY BECAUSE THEY WANT TO GET A TENURED TRACK FACULTY POSITION. AND CERTAINLY THERE ARE PEOPLE WHO GO INTO POX DOCKING WITHOUT THAT PLAN, BUT I THINK THERE IS A CLEARLY A GREAT PASSION FOR RESEARCH, A DESIRE FOR DISCOVERY, ET CETERA. YET AT THE SAME TIME, THE LANDSCAPE FOR AND THE OUTLOOK FOR MOVING INTO THOSE POSITIONS, HAS CHANGED DRAMATICALLY OVER THE LAST SEVERAL DECADES. SO, HERE IN RED IS THAT TENURED TRACK POSITION CHANGE OVER TIME. AND THESE DATA HAVE BEEN UPDATED IN THE LAST NGRI REPORT BY DONNA GINTHER. AND I THINK ONE THING THAT IS VERY INTERESTING TO NOTE, THE COHORT ARE DIFFERENT SO THAT WAS A 5-6 YEARS AND HERE IS 6-10 YEARS PH.D. I THINK THAT THERE IS A INCREASE IN THE ACADEMIC NON-TENURED TRACK, WHICH INCLUDES POSTDOCS. AS YOU CAN SEE, 1-5 YEARS POST PH.D. IN THE 2013, THE LAST DATA OR LAST TIME IT WAS AVAILABLE, YOU CAN SEE THIS GROUP HAS INCREASED. AND AT THE SAME TIME, THERE HAS BEEN A STEADY INCREASE IN THE NUMBER OF PEOPLE MOVING INTO NON NON-RESEARCH CAREERS 11-20 YEARS AFTER THEIR PH.D. I THINK YOU CAN MAKE THE FANTASTIC ARGUMENT THAT PEOPLE WHO ARE MOVING OUT, TAKING THEIR TRAINING INTO THE WORLD AND TAKING TRAINING INTO GOVERNMENT AND NON-PROFIT, DIFFERENT TYPES OF INDUSTRY, ARE MAKING HUGE CONTRIBUTIONS AND BRINGING THE SCIENTIFIC MIND-SET INTO THE BROADER PUBLIC. BUT I THINK THAT THERE IS A REALLY IMPORTANT QUESTION AS TO WHAT THE CONTENT OF THE TRAINING IS AND HOW LONG THAT TRAINING HAPPENS IN ORDER TO MAKE THOSE PREPARATIONS. SO, FOR EXAMPLE, DONNA GINTHER AGAIN, WORKED ON THIS SURVEY -- EXCUSE ME, THIS ANALYSIS OF SALARIES WHO PEOPLE WHO POSTDOCKED AND DID NOT. FOR LOOKING AT PEOPLE, WHO AFTER THE PH.D. WENT IMMEDIATELY INTO INDUSTRY, VERSUS WENT INTO A POSTDOC AND THEN INTO INDUSTRY, I THINK THERE IS A QUESTION AS TO WHETHER POSTDOCKING IN THE AGGREGATE, WHICH IS A VERY IMPORTANT POINT, LIKE IN THE AGGREGATE, IS ACTUALLY PROVIDING THE KIND OF TRAINING THAT IS BENEFITING PEOPLE DIRECTLY IN THE WAY THAT WE WOULD LIKE IT TO. NOW, THAT SAID, I THINK THAT EVERY INSTITUTION IS DIFFERENT. EVERY PROGRAM IS DIFFERENT. EVERY DEPARTMENT IS DIFFERENT AND SOME ARE PROVIDING AMAZING TRAINING OPPORTUNITIES, INCREDIBLE TUBES. SO FOR EXAMPLE, IRAQ HAS STATISTICS THAT ARE INCREDIBLY FAVORABLE. I THINK IT IS IMPORTANT FOR POSTDOCS TO BE ABLE TO UNDERSTAND WHAT THEY ARE GETTING THEMSELVES INTO WHEN THEY CHOOSE A PROGRAM, WHEN THEY CHOOSE AN INSTITUTION. WHEN THEY GO INTO ANY ONE OF THESE PROGRAMS. AND NOT ONLY THAT, I THINK IT'S ALSO IMPORTANT FOR PEOPLE WHO HAVE THE POWER TO INFLUENCE THESE PROGRAMS, TO BE ABLE TO UNDERSTAND WHICH ONES ARE EFFECTIVE FOR CERTAIN OUTCOMES. WHETHER THAT IS HELPING POSTDOCS TRANSITION INTO INDUSTRY OR ENTREPRENEURSHIP OR POLICIES OR ACADEMIA. SO, ONE OF THE RECOMMENDATIONS OF THE MANY FROM THIS REPORT THAT KENNY MENTIONED, IS THAT BIOMEDICAL RESEARCH INSTITUTIONS COLLECT, ANALYZE AND DISSEMINATE COMPREHENSIVE DATA ON THE OUTCOMES OF POSTDOCS. WE RECOMMENDED THAT THIS BE A REQUIREMENT FOR NIH FUNDING. THAT IT BE PHASED IN OVER A 5-YEAR PERIOD. RECOGNIZING THE IMPORTANCE OF THIS DATA TO BOTH INDIVIDUAL POSTDOCS AND ALSO THE DESIGN OF THESE PROGRAMS OVERALL. SO, ONE THING THAT IS SO EXCITING TO SEE IS THAT MANY UNIVERSITIES ARE DRIVING THIS CHANGE AND HAVE VOLUNTARILY SIGNED UP TO THIS COALITION. YOU CAN FIND OUT MORE ABOUT THIS AT NGLS COALITION.ORG. COALITION FOR NEXT GENERATION LIFE SCIENCES IS A GROUP OF 10 ORGANIZATIONS THAT ARE VOLUNTARILY RELEASING DATA ABOUT THEIR GRAD STUDENTS AND POSTDOCS IN A STRUCTURED FORMAT TO BE EASILY COMPARED AGAINST DIFFERENT INSTITUTIONS, WHICH I THINK IS A FANTASTIC STEPS TOWARDS ALLOWING BOTH STUDENTS WHO ARE CONSIDERING AN INDIVIDUAL INSTITUTION AND ALSO INSTITUTIONS THAT ARE LOOKING TO MAKE CHANGES THEMSELVES. SO, I THINK THE QUESTION THAT THERE IS SO MUCH OPPORTUNITY IN POSTDOCKING, IN DOING SCIENCE, IN BEING IN A LAB, MAKING DISCOVERIES, BEING A PART OF DISCOVERY, AND THERE ARE WONDERFUL ELEMENTS TO THIS THAT ARE NOT CAPTURED BY SOME OF THESE CONCERNS THAT ARE ALSO IMPORTANT TO KEEP OUR EYES OPEN TO. WITH THAT SAID, WHAT CAN INDIVIDUAL POSTDOCS DO TO HELP TO IMPROVE AND ENHANCE THE EXISTING STATE OF POSTDOC TRAINING? SO, OUR APPROACH STARTING IN 2014, WAS TO FOUND AN ORGANIZATION CALLED, FUTURE OF RESEARCH. SO GARY, PICTURED HERE IS OUR EXECUTIVE DIRECTOR, WHO HAS GUIDED THIS ORGANIZATION INTO A PHASE WHERE WE ARE NOW 501(C)3 NON-PROFIT. OUR ROOTS REALLY ARE IN CONVENING GROUPS OF POSTDOCS TOGETHER TO TALK ABOUT SOME OF THESE ISSUES, PROPOSE CHANGES AND FIGURE OUT WHAT TO DO TO HELP IMPROVE THE SYSTEM OVERALL. AND NOW WE ARE INVOLVED IN ADVOCACY, IN DATA COLLECTION, WE RECENTLY HAD A PROJECT CONCERNING POSTDOC SALARIES WHICH CAN PRESENT A BARRIER FOR PEOPLE WITH LESS RESOURCES TO PARTICIPATE IN SCIENCE. AND ARE LOOKING FORWARD TO INCREASING TRANSPARENCY AND AWARENESS ABOUT SOME OF THESE CHALLENGES AND SOLUTIONS. SO, I THINK THAT IT'S VERY IMPORTANT TO BE INFORMED AND I THINK THE NPA IS ANOTHER FANTASTIC ORGANIZATION THAT I ENCOURAGE EVERYONE TO JOIN, TO BE VOCAL ABOUT SOME OF THESE CHALLENGES TO RESPOND TO RFIs WHERE THE VOICE OF POSTDOCS IS NOT AS OFTEN REPRESENTED AS I THINK IT COULD BE. CERTAINLY NOT PROPORTIONAL SO THE NUMBER OF PEOPLE WHO ARE ACTIVELY ENGAGED IN RESEARCH. I THINK THAT IS INCREDIBLY IMPORTANT. BEING ABLE TO COME TOGETHER TO ORGANIZE MEETINGS IS SOMETHING THAT WE AT FUTURE OF RESEARCH WANT TO PROVIDE SOME RESOURCES TO SUPPORT, SO WE HAVE A TOOLKIT FOR ORGANIZING MEETINGS. I THINK IT IS ALSO REALLY VALUABLE WHEN YOU GET THOSE REQUESTS FOR SURVEYES FROM YOUR GRADUATE INSTITUTIONS TO FILL THEM OUT. OR TO CONTRIBUTE TO THIS DIRTH OF INFORMATION TO MELIORATE THIS DIRTH OF INFORMATION THAT EXISTS AROUND POSTDOCS. AND NEXT, I THINK THE RECOGNIZING THAT THIS IS SUCH A COMPLICATED PROBLEM. THIS PROBLEM IS INTERTWINED WITH ISSUES OF BIAS, WITH ISSUES OF A LACK OF DIVERSITY, WITH ISSUES OF PROBLEMS IN PUBLICATION, WITH MENTORING, WITH MENTAL HEALTH, WITH FUNDING AS A WHOLE. SO, IT'S INCREDIBLY CHALLENGING TO THINK OF WAYS TO SOLVE ALL OF THESE PROBLEMS AT ONCE AND PROBABLY IMPOSSIBLE TO SOLVE ALL OF THESE PROBLEMS AT ONCE. SO I THINK THAT WHAT IS REALLY VALUABLE AND IMPORTANT TO DO, IS TRY TO IDENTIFY SOME AREA WHERE WE, AS INDIVIDUALS, CAN MAKE SOME KIND OF MEANINGFUL DIFFERENCE THAT WILL HELP TO RELIEF SOME OF THE TENSION ON THIS SYSTEM AS A WHOLE. SO, ONE AREA WHERE I HAVE BEEN REALLY -- I THINK IS A REALLY IMPORTANT AREA TO FOCUS ON IS PUBLICATIONS. AND SO NOW I'M GOING TO TALK A LITTLE BIT MORE ABOUT ASAP BIO, A SEPARATE NON-PROFIT ORGANIZATION THAT WAS FOUNDED BY RON VEIL. SO RON VEIL WROTE A ANALYSIS BY COUNTING UP THE NUMBER OF PANELS IN THESE DIFFERENT JOURNALS IN THE CELL BIOLOGY AREA OVER A 30-YEAR PERIOD. AND HE FOUND AN ENORMOUS INCREASE IN THE NUMBER OF PANELS -- THE NUMBER OF EXPERIMENTS THAT ARE GOING INTO PAPERS. THE BAR FOR PUBLICATIONS IS IN MANY WAYS INCREASING, EVEN THOUGH IT'S EASIER TO GENERATE DATA, IT'S NOW EASIER TO COLLECT INFORMATION TO COLLABORATE. THESE PAPERS ARE MORE COLLABORATIVE FOR SURE. HOWEVER, ANOTHER COINCIDENT PHENOMENON IS THE TIME FOR GRADUATE STUDENTS IN THE PROGRAM AT UCSF INCREASING TO PRODUCE A FIRST AUTHOR PUBLICATION. SO, I THINK THAT CLEARLY, WITH THE INTERNET, WE EXPECT THAT EXCHANGING SCIENTIFIC INFORMATION SHOULD BE GOING FASTER, BUT IN FACT, AT LEAST IN THE EXPERIENCE OF GRAD STUDENTS PRODUCING THEIR FIRST PAPER, IT MIGHT BE GOING SLOWER. PERHAPS THIS IS BECAUSE IF I WERE TO SPECULATE, THAT THE EXPECTATIONS FOR PAPERS ARE GOING UP AND EVERYONE IS TRYING TO PUBLISH A HIGH-IMPACT STORY. SO, THE WAY I THINK TO ADDRESS THIS IS TO SEPARATE THE EVALUATION OF THAT STORY AND THE CURATION OF THAT STORY INTO SOME SORT OF VENUE FROM THE DISSEMINATION OF THE INFORMATION. SO, ASAP BIO, IS ALSO ANOTHER NON-PROFIT THAT IS ADDRESSING WAYS TO GET AT THIS PROBLEM OF INCREASING TRANSPARENCY AND INNOVATION AND PUBLISHING IN MEANINGFUL WAYS, THROUGH CULTURAL CHANGE. SO, WE HELD A MEETING IN EARLY 2016 TO TALK ABOUT THE ROLE THAT PREPRINTS COULD PLAY IN THE LIFE SCIENCES. SO, COULD I JUST SEE A SHOW OF HANDS OF PEOPLE WHO HAVE PRE-PRINTED SOMETHING? THERE IS LIKE ONE HAND. HAS ANYONE READ A PREPRINT? OKAY. WOW. SO IN GENERAL, PEOPLE ARE USING PREPRINTS. MAYBE NOT POSTING THEMSELVES BUT THERE IS CLEARLY -- I DON'T NEED TO TELL MOST OF YOU IN THIS AUDIENCE THAT THEY ARE ESSENTIALLY MANUSCRIPTS THAT ARE SHARED BEFORE THE COMPLETION OF PEER REVIEW. AND SO, THE BASIC IDEAS THAT THESE THINGS ARE PERMANENT, THEY ARE VERSIONED AND SITABLE, IT WOULD BE LIKE IF YOU HAVE BEEN GIVING A PRESENTATION IN A MEETING IN A FORM THAT CAN BE ACCESSED AND CITED BY ANYONE LATER. I THINK THE MAJOR BENEFIT OF THIS IS THAT YOU CAN TAKE A MANUSCRIPT THAT YOU MIGHT OTHERWISE SUBMIT TO AN UNCERTAIN NUMBER OF JOURNALS, RECEIVING AN UNCERTAIN PEER REVIEW PROCESS, BUT YOU CAN TAKE A THIN MANUSCRIPT AND PUT TO THE ON LINE AND GET THIS COMMUNITY FEEDBACK, IDEA AND DISCUSSION GOING. IT WOULD HELP YOU FEEDBACK INTO THE JOURNAL PROCESS, IMPROVE THE FINAL MANUSCRIPT AND MORE IMPORTANTLY, GET THE IDEAS INTO THE COMMUNITY WHERE THEY CAN ACCELERATE SCIENCE IN A MEANINGFUL WAY. SO, THIS IS SOMETHING THAT HAS BEEN GOING ON FOR ALMOST OVER 25 YEARS IN PHYSICS. THERE IS OVER A MILLION PREPRINTS ON ARCHIVE, AND THE GROWTH IN LIFE SCIENCES OF PREPRINTS HAS REALLY BEEN PRETTY PHENOMENAL. SO THIS IS A PLOT OF PREPRINTS PER MONTH THAT ARE POSTED MOSTLY IN BIOARCHIVE, WHICH IS THE GREEN SERVER HERE. SO IT'S NOT JUST THE VOLUME THAT IS INCREASING, BUT LOTS OF OTHER POLICY CHANGES HAVE BEEN HAPPENING AS WELL. SO, WE ZOOM IN ON THIS AREA BETWEEN 2014, FROM THE BEGINNING OF BIOARCHIVE TO NOW, YOU CAN SEE THIS GROWTH IN A LITTLE BIT MORE DEPTH. AND SO ONE OF THE FUNDERS PRESENT AT OUR MEETING WAS THE FOUNDATION WHO WERE EARLY TO COME OUT WITH A POLICY THAT SUPPORTED THE USE OF PREPRINTS. IN OTHER WORDS, THEIR POLICY BASICALLY SAYS YOU CAN SITE PREPRINTS IN YOUR APPLICATIONS AND REPORTS. SO WHICH I THINK PROVIDES SOME RECOGNITION OF THESE OBJECTS AS A RECOGNIZED FORM OF DISSEMINATION OF RESEARCH. AND I THINK A BIG JUSTIFICATION FOR THIS, OR IDEA FOR THIS, IS THAT THIS IS HELPING NOT ONLY ESTABLISHED INVESTIGATORS, BUT ALSO EARLY CAREER INVESTIGATORS WHO ARE OR POSTDOCS OR GRAD STUDENTS, WHO NEED TO DEMONSTRATE WHAT THEY HAVE DONE IN ORDER TO MOVE ON TO GET A FELLOWSHIP, TO GET A POSTDOC, TO GET A FACULTY POSITION, OR TO OTHERWISE DEMONSTRATE THEIR PRODUCTIVITY. SO, CELL PRESS, WHICH HAD PREVIOUSLY HAD A PRETTY RESTRICTIVE PREPRINT POLICY CHANGED THEIR POLICY LATER THAT SAME YEAR, WHICH WAS SIGNIFICANT BECAUSE THEY WERE REALLY ONE OF THE LATER HOLD OUTS TO HAVING A NEGATIVE PREPRINT POLICY. CROSS REV STARTED INDEXING, SO THEY ARE THE ORGANIZATION OF THE SCIENCE DOIs, SO THIS ALSO INCORPORATED PREPRINTS INTO THIS KIND OF MORE FORMAL SCHOLARLY RECORD IN A WAY THAT ENABLED THEM TO BE INCORPORATED INTO THE CITATION AND BIBLIOGRAPHIES IN A MORE FORMAL WAY. THE NIH ALSO CAME OUT WITH A PREPRINT POLICY ENCOURAGING THE USE OF PREPRINTS. IT WAS VERY THOUGHTFUL AND INCLUDED ADVICE ON HOW TO CHOOSE A PREPRINT SERVER, A RECOMMENDED LICENSE, AND ZUCKERBERG PARTNERED WITH BIOARCHIVE TO PROVIDE MORE STABLE INFRASTRUCTURE AND NOW, WE ARE EVEN STARTING TO SEE PREPRINTS EMERGING IN MEDICAL DISCIPLINES. SO THE LANCET LAUNCHED A PREPRINT SERVER AS WELL, WHICH I THINK IS REALLY A EXCITING DEVELOPMENT BECAUSE CLINICALLY ORIENTED PREPRINTS CLEARLY NEED TO HAVE SCREENING, MORE& ATTENTION TO WHAT KIND OF CHECKS NEED TO OCCUR BEFORE THE PREPRINT CAN BE POSTED. BUT, I THINK BRINGING THIS KIND OF ACCELERATION TO MEDICINE IS ALSO A VERY EXCITING POSSIBILITY. SO, PREPRINT NOW ALSO APPEARING ON EUROPE PUBMED CENTRAL, WHICH I THINK IS ALSO ANOTHER IMPORTANT SIGNAL OF THEIR GROWING ACCEPTANCE. AND IT'S NOT JUST THE LIFE SCIENCES. THERE ARE MANY DOZENS OF PREPRINT SERVERS IN ALL KINDS OF DIFFERENT DISCIPLINES, SOME OF WHICH HAVE BEEN AROUND FOR A VERY LONG TIME BUT OTHERS THAT ARE NEW. THERE IS PREPRINT SERVERS FOR DIFFERENT DISCIPLINES, DIFFERENT GEOGRAPHIC REGIONS, LANGUAGES, ET CETERA. SO, ONE THING THAT WE ARE AT ASAP BIO IS INTERESTED IN CREATING RESOURCES TO HELP MAKE IT SIMPLE FOR AUTHORS TO COMPARE THESE DIFFERENT SERVERS AND ALONG THE LINES THAT WILL BE IMPORTANT FOR PRESERVATION AND SCREENING AND SO FORTH. SO, WHY SHOULD YOU PREPRINT? I THINK THAT THERE IS THIS, FIRST OF ALL, AN ABILITY TO GAIN MORE VISIBILITY. LIKES IN SIMILAR WAYS TO PRESENTING AT A MEETING, YOU EVERY READER OF THAT PREPRINT SERVER, WHICH COULD BE UNLIKE ATTENDEES AT A MEETING, ALMOST ANYWHERE IN THE WORLD. THERE IS THE POTENTIAL TO GET MORE FEEDBACK TO IMPROVE YOUR PAPER. I SOMETIMES HEARD PEOPLE SAY, WELL, I DON'T WANT TO PREPRINT BECAUSE THERE COULD BE MISTAKES IN THE PAPER AND I WANT THOSE MISTAKES TO GET CAUGHT BEFORE THEY GET PUBLISHED, WHICH I THINK IS A VERY VALID AND HUMAN CONCERN. HOWEVER, GIVEN THE PEER REVIEW IS INHERENTLY A PROCESS LIMITED BY THE NUMBER OF PEOPLE WHO ARE PAR PARTICIPATING IN A TRADITIONAL SENSE, I THINK THAT HAVING THIS OPEN PROCESS OF REVIEW ENABLES MORE FEEDBACK THAT COULD IMPROVE THE PAPER MORE OVER THE LONG TERM, IF THAT MAKES SENSE. ANOTHER PHENOMENA I THINK IS REALLY EXCITING IS THAT JOURNAL EDITORS ARE NOW SOLICITING SUBMISSIONS FROM PREPRINT SERVERS. SO FOR EXAMPLE, THE ROYAL SOCIETY B AND POST GENETICS HAVE DEDICATED PREPRINT EDITORS WHOSE ROLE IT IS TO LOOK THROUGH SERVERS AND INVITE SUBMISSIONS. THIS IS CREATING A DIFFERENT TYPE OF MARKETPLACE FOR SUBMISSIONS IT'S NOT THAT DIFFERENT FROM EDITORS MIGHT DO IF THEY GO TO A MEETING AND INVITE SOMEONE WHO GAVE A COOL TALK TO SUBMIT, BUT THIS WAY, I THINK THAT MAYBE EVEN ARGUABLY MORE CONNECTED TO THE FINAL PRODUCT THAN THEY CAN EXPECT TO FIND IN THEIR JOURNALS. IN ADDITION TO THOSE JOURNALS I MENTIONED, A LOT OF LITTLER ARE DOING THIS IN A MORE INFORMAL FASHION. NEXT, THERE IS THE POTENTIAL TO FIND COLLABORATORS EARLIER ON WHERE SOMEONE WHO MIGHT BE WORKING ON SOMETHING SIMILAR, YOU CAN CONNECT TO ONE ANOTHER RATHER THAN DUPLICATING EFFORTS NEEDLESSLY. YOU CAN CREATE A RECORD OF WHAT WAS DONE AT A GIVEN DATE. IN OTHER WORDS, THERE IS A LOT OF CONCERN ABOUT SCOOPING. I'LL TOUCH ON THAT IN JUST A MOMENT. THE IDEA THAT YOU CAN ACTUALLY PUT SOMETHING OUT THERE VERY CONCRETELY WITH WHAT YOU FOUND OUT AT A CERTAIN TIME, CAN HELP ALLEVIATE THESE CONCERNS ABOUT SCOOPING. AND NEXT, THIS ALLOWS YOU TO DEMONSTRATE PRODUCTIVITY FOR JOHNS AND GRANTS AND ACCELERATES THE PROCESS OF DISCOVERY. SO, A SELECTION OF THE FUNDERS THAT ARE NOW ENCOURAGING PREPRINTS CAN BE FOUND HERE WHERE -- AND THERE IS A LINK TO OR A RUNNING LIST THAT WE HAVE AS WELL. THE IDEA BEING THAT SOME OF THEM, LIKE FOR EXAMPLE, THE CHAN ZUCKERBERG INITIATIVE REQUIRES PREPRINTS, BUT IN THE WHOLE, MOST OF THESE POLICIES ARE SIMPLY ENCOURAGING AUTHORS, IF THEY WISH, TO SHARE THEIR RESULTS WITH THE IDEA THAT THIS WILL ENABLE THEM TO LIST THESE PRODUCTS ON THEIR APPLICATIONS AS A FORM OF DEMONSTRATION PRODUCTIVITY. SO, I WANT TO TALK ABOUT THREE CONCERNS THAT PEOPLE BRING UP ABOUT PREPRINTS. WHICH I THINK IS SOMETHING THAT COMES UP A LOT WHEN TALKING ABOUT THESE THINGS. SO, ONE IS THAT, SHEAR A WORD CLOUD THAT WE GENERATED LIVE DURING A MEETING OF POSTDOCS A COUPLE OF YEARS AGO. I BASICALLY ASKED THEM WHAT ARE YOU CONCERNED ABOUT? WHAT ARE YOUR CONCERNS ABOUT PREPRINTS? I THINK IT IS PRETTY OBVIOUS THAT PEOPLE ARE AFRAID THAT BY PUTTING WORK OUT IN A PRELIMINARY FORMAT, THAT THIS IS GOING TO CAUSE SOME KIND OF -- SOMEONE ELSE TO TAKE THIS INFORMATION AND BENEFIT FROM IT BY PUBLISHING A PAPER THAT IS OTHERWISE NOT ACKNOWLEDGING THEIR CONTRIBUTIONS. BUT, SO PAUL GINSBURG, A FOUNDER OF THE ARCHIVE, HAS THIS TO SAY, WHICH IS BASICALLY THAT THE CONCEPT OF SCOOPING DOES NOT MAKE SENSE IN THE CONTEXT OF ARCHIVE BECAUSE ESPECIALLY IN THE ARCHIVE, WHERE EVERYONE IS LOOKING AT THE ARCHIVE; IT BECOMES IMPOSSIBLE TO SCOOP SOMEONE WHO HAS ALREADY POSTED SOMETHING ON THE ARCHIVE SINCE THAT IS WHERE EVERYONE IS LOOKING. THE FACT THAT EVEN IF IN A SITUATION WHERE THE COMMUNITY IS STILL BEGINNING TO TAKE UP PREPRINTS AS THEY ARE IN THE LIFE SCIENCES, THE ABILITY TO HAVE A PERMANENT PLACE WHERE WORK IS ACCESSIBLE TO ALL, I THINK MAKES PREPRINTING MUCH LESS CONCERNING FROM THE PERSPECTIVE OF SCOOPING THAN PRESENTING A POSTER AT A MEETING OR A TALK WHERE THERE IS A FEMORAL PRESENTATION THAT CAN BE SEEN BY SOME PEOPLE BUT NOT NECESSARILY ACKNOWLEDGED BY EVERYONE. SO A SECOND CONCERN IS ABOUT JOURNALS. THERE IS AN IDEA THAT JOURNALS WILL BE NEGATIVE ABOUT PREPRINTS, OR THEY WON'T ACCEPT MANUSCRIPTS POSTED, WHICH IS NOT AN IRRATIONAL CONCERN, ESPECIALLY GIVEN THE ANGLE FINGER RULE, AND OTHER LONGSTANDING OBJECTIONS TO POSTING WORK THAT HAD BEEN PREVIOUSLY PUBLISHED. HOWEVER, THIS IS JUST A TINY FRACTION OF THE JOURNALS THAT ARE ACCEPTING PREPRINTS. SO FOR EXAMPLE, THESE GENERAL JOURNALS LIKE NATURE, HAVE BEEN ACCEPTING PREPRINTS FOREVER BECAUSE THEY ARE EFFECTIVELY DEALING WITH PHYSICS PREPRINTS. AND I WOULD SAY THAT THERE ARE SO MANY OTHER JOURNALS, TOO MANY TO LIST, EVENLY SOME THAT ARE MORE CLINICALLY FOCUSED, OF COURSE, THAT ARE NOW ACCEPTING PREPRINTS. AND IF YOU WANT TO CHECK WHETHER A JOURNAL ALLOWS PREPRINTING, THE MOST AUTHORITATIVE SOURCE IS THE JOURNAL WEBSITE BUT THERE IS ALSO A LIST ON WIKIPEDIA, AND DATABASE CALLED, SERVER ROMEO, WHICH IS MUCH MORE COMPREHENSIVE THAN THE WIKIPEDIA LIST -- SHERPA/ROMEO. SO, I THINK THIS THIRD CONCERN IS THE IDEA OF INFORMATION OVERLOAD, OR FAKE NEWS. THIS IDEA THAT WITHOUT PEER REVIEW, WE CANNOT BE TRUSTED, IN OTHER WORDS, TO SHARE INFORMATION, OR THAT THE QUALITY OF INFORMATION OVERALL WILL BE HEAVILY DEGRADED IF WE SHARE INFORMATION IN A WIDESPREAD FASHION WITHOUT PEER REVIEW. AND I THINK THERE IS TWO DIFFERENT KINDS OF FACETS TO THIS. ONE IS THE IDEA THAT WE NEED FILTERS TO PREVENT BAD QUALITY INFORMATION FROM COMING OUT. WE ALSO NEED FILTERS TO HIGHLIGHT THE HIGH-QUALITY INFORMATION THAT IS AVAILABLE. SO THERE IS A SCREENING AND HIGHLIGHTING FUNCTION THAT JOURNALS ARE PERFORMING NOW. AND I AM ONBOARD WITH THE VALUE OF THIS AND I THINK THAT EVEN IN SOME FUTURE WHERE THERE ARE ALTERNATIVE METHODS OF CURATION, THESE TWO FUNCTIONS ARE GOING TO BE INCREDIBLY IMPORTANT. HOWEVER, I THINK THAT THIS CRITICISM SOMEWHAT IGNORES THE FACT THAT WE ARE ALREADY DEALING WITH LOTS OF INFORMATION, BOTH ON THE INTERNET AND OFF LINEUP AS WELL. SO FIRST OF ALL, WE ALREADY HAVE A TRADITION -- OFF LINE -- OF SHARING BE UNPEER-REVIEWED DATA WITH ONE ANOTHER IN MEETINGS. -- UNPEER-REVIEWED DATA -- AND EVERY POSTER PRESENTATION OR TALK, TYPICALLY THOSE FINDINGS HAVE NOT BEEN THROUGHLY PEER-REVIEWED. AT THE SAME TIME, WITH THE PROLIFERATION OF JOURNALS, OF VARYING QUALITIES OR VARYING PEER REVIEW PROCESSED AND THAT IS SOMETHING I'LL TOUCH ON IN A MOMENT AS WELL. I THINK IT'S REALLY POSSIBLE TO GET ALMOST ANYTHING PUBLISHED SOMEWHERE IN SOME JOURNAL. SO, DEPENDING ON THIS AND ALLOWING PEOPLE TO POST PREPRINTS COULD MAKE THIS EASIER BUT I'M NOT SURE OR REALLY FUNDAMENTALLY GETS THAT THE CONCERN OF ENSURING QUALITY. AND OF COURSE WE CAN ALSO TURN TO OTHER DISCIPLINES TO SEE THAT PHYSICS HAS BEEN DOING THIS FOR A LONG TIME AND YET, THEY ARE STILL ABLE TO EASILY MAKE PROGRESS AND FUNCTION AS A FIELD FIELD. SO, I THINK RELATING TO THIS IS A CONCERN AS TO WHETHER PREPRINTS SHOULD BE ACTUALLY RECOGNIZED AND CITED. SO, I THINK THERE IS A QUESTION OF WHAT DOES A CITATION ACTUALLY MEAN? IT IS A REFERENCE TO A PIECE OF INFORMATION, BUT THERE IS SOME, I THINK, ATTITUDE OR SOME IDEA THAT CITATION IS ACTUALLY A STAMP OF TRUST, WHICH I THINK IS ACTUALLY KIND OF A VERY DISTURBING IDEA BECAUSE THERE IS SO MANY DIFFERENT WAYS OF CITING WORK. I THINK THERE IS SOME KIND OF UNDERLYING PHILOSOPHICAL PROBLEM WITH THE IDEA THAT THE ONLY MECHANISM THAT WE HAVE TO REFER TO PREVIOUS WORK IS BASICALLY NEUTRAL AS TO WHETHER THE WORK THAT WE ARE DOING IS SUPPORTING IT, CONTRADICTING IT, REFERENCES IT, BUILDING OFF OF IT; BUT NEVERTHELESS, THAT IS THE CASE. MANAGING A WORLD WHERE PREPRINTS CANNOT BE CITED, INVITES BASICALLY PEOPLE TO NOT ONLY SCOOP ONE ANOTHER, BUT ACTUALLY DEMANDS THAT PEOPLE ARE BASICALLY PLAGIZING ONE ANOTHER. YOU CAN READ INFORMATION BUT YOU CAN'T SITE IT. YOU CAN'T POINT TO WHERE THE INFORMATION CAME FROM, THAT I THINK, IS VERY DISTURBING. SO THERE IS JOURNALS THAT I THINK THE -- THIS IS FADING AS A CONCERN, BUT I THINK THAT THERE IS A IMPORTANT CULTURAL NEED TO SITE THESE PREPRINTS TO CREATE THIS TYPE OF NETWORK OF INFORMATION. BUT I THINK THAT THE MOST EXCITING AND IMPORTANT PART ABOUT THIS WHOLE PREPRINT REVOLUTION, IF YOU WILL, IS THE FACT THAT THERE IS CREATING AN OPPORTUNITY FOR PEOPLE TO COMMUNICATE WITH ONE ANOTHER IN CONSTRUCTIVE WAYS THROUGH OUTSIDE OF THE TRADITIONAL JOURNAL PRIOR REVIEW PROCESS. SO I THINK THAT JOURNAL PEER REVIEW SERBS A BUNCH OF DIFFERENT FUNCTIONS. -- IT SERVES THE FUNCTIONS OF FILTERING OUT POTENTIALLY INCORRECT INFORMATION, OF HIGHLIGHTING HIGH-QUALITY INFORMATION OR INTERESTING INFORMATION OR INFORMATION THAT IS LIKELY TO GENERATE MORE CITATIONS. BUT, NEITHER OF THOSE FUNCTIONS NECESSARILY HELPS THE AUTHOR IMPROVE THE QUALITY OF THEIR MANUSCRIPT. AND THAT IS ALSO SOMETHING THAT HAPPENS IN JOURNAL PEER REVIEW, BUT IN A PROBLEMS OF REVIEWING FOR A JOURNAL -- PROCESS -- ALL FUNCTIONS GET MERGED TOGETHER INTO ONE PROCESS, WHICH I THINK IS SOMETHING THAT DESERVES SOME EXAMINATION. AND THE ABILITY TO CREATE DIFFERENT TYPES OF COMMUNITIES THAT ARE COMMENT ON PREPRINTS, PROVIDING FEEDBACK, SELECTING PREPRINTS OR HIGHLIGHTING PREPRINTS, PROVIDES WAYS FOR PEOPLE, ESPECIALLY EARLY CAREER RESEARCHERS TO EXPLORE THESE NEW WAYS OF CURATING INFORMATION, OF IMPROVING INFORMATION AND CONSTRUCTIVELY HELPING OTHER SCIENTISTS WORK BETTER. SO, JUST TO GIVE A VERY BRIEF OVERVIEW OF SOME OF THEM, YOU ARE PROBABLY FAMILIAR WITH PUB PEER, A WEBSITE WHERE YOU CAN COMMENT -- THIS IS NOT AN EXHAUSTIVE LIST. THERE IS ALSO WEBSITES LIKE ACADEMIC KARMA, THAT SUPPORT COMMENTING ON PREPRINTS, PUB LONS IS A SITE TO PUBLISH PEER REVIEWS ON PUBLISHED LITERATURE AND NOT. PEER COMMUNITY IS A MORE STRUCTURED PEER REVIEW PROCESS AND THESE ARE GETTING LINKED OUT FROM EUROPE PUBMED CENTRAL. SO WE ARE INDEXING PREPRINTS AND ALSO LINK OUT TO PCI, BIOOVERLAY IS A JOURNAL WHERE THE AUTHOR DOES NOT SUBMIT THEIR PREPRINT BUOYED TORS SELECT THE PREPRINT AND CONDUCT A PEER REVIEW PROCESS AND BASICALLY HIGHLIGHT PAPERS THAT ARE OF INTERESTING SIGNIFICANCE AND THE SAME FOR PRELIGHTS. A SINGLE USUALLY EARLY-CAREER PERSON WHO IS HIGHLIGHTING A PAPER AND KIND OF DISCUSSANTS BY ACCOMPANIED BY BIOLOGISTS PROGRAM, AIMING TO FIND MORE INTERESTING PREPRINTS AND HELP THOSE COME TO THE SURFACE. AND PREREVIEW IS A ORGANIZATION THAT IS ENCOURAGING EARLY-CAREER PEOPLE, SO POSTDOCS, GRAD STUDENTS, TO CONDUCT JOURNAL CLUBS ON PREPRINTS, WHICH I THINK IS A FANTASTIC USE OF ENERGY. I MEAN JOURNAL CLUBS AT LEAST IN MY GRADUATE SCHOOL EXPERIENCE, WERE BASICALLY A WAY TO KIND OF -- YOU CAN SORT OF GRIPE ABOUT ELEMENTS OF A PAPER, BUT ALL OF THAT WORK DOESN'T NECESSARILY HELP THE AUTHOR TO IMPROVE THE PAPER. YOU'RE NEVER GOING TO SEND THE COMMENTS BACK TO THE AUTHOR. THAT IS VERY UNLIKELY. BUT IF YOU ARE CONDUCTING A JOURNAL CLUB WITH PREPRINTS, YOU CAN THEN WRITE UP WHAT YOU HAVE WRITTEN INTO A CONSTRUCTIVE REVIEW, SEND IT TO THE AUTHORS AND IMPROVE THE SCIENCE AND AT THE SAME TIME, LEARN ABOUT WORK THAT IS SO NEW, IT'S NOT EVEN IN A JOURNAL YET. I THINK THIS IS A REALLY GOOD COMBINATION OF DIFFERENT ELEMENTS HERE. SO PREREVIEW IS JUST ONE OF THESE ON THE ADVISORY BOARD, BUT THERE IS ALSO OTHER SOURCES DOING JOURNAL CLUBS LIKE ACADEMIC KARMA AS WELL. SO I DEFINITELY ENCOURAGE THINKING ABOUT SOME OF THESE THINGS. SO WE ALSO WANT TO COLLECT INFORMATION ABOUT PREPRINTS THAT CAN BE HELPFUL FOR THOSE CONSIDERING TAKING THIS STEP OR OTHER STEPS. SO, THIS MOTION TOWARDS TRANSPARENCY IS SOMETHING THAT I THINK IS REALLY HAVING A LOT OF BENEFITS FOR PREPRINTS, FOR READERS OF PREPRINTS. BUT I THINK THAT ANOTHER PART OF THE JOURNAL PUBLISHING SYSTEM THAT IS SO MUCH A BLOCK BOX S PEER REVIEW. SO PEER REVIEW, ESPECIALLY THESE CONVERSATIONS ABOUT PREPRINTS REALLY THROW THIS INTO FOCUS THAT WE ARE RELYING ON PEER REVIEW AS A WAY TO GAUGE AND PREDICT THE INTERESTS OR TO PROVIDE SOME ASSURANCE THAT THE QUALITY OF A MANUSCRIPT OR TO PROVIDE SOME ASSURANCE THAT SOME KIND OF WORK HAS BEEN CHECKED OR EVALUATED IN SOME FASHION. BUT IT IS, I THINK, KIND OF STRANGE WHEN YOU STEP BACK AND LOOK AT THE SITUATION THAT EXISTS. WE ARE -- PEER REVIEW IS FORMING A FOUNDATION OF OUR TRUST IN SCIENTIFIC LITERATURE. YET WE DON'T EVER SEE IT UNLESS WE ARE THE AUTHOR OR THE EDITOR. AND WHY IS THIS? I THINK THAT THERE ARE COMPELLING ARGUMENTS TO BE MADE FOR MAINTAINING THE ANONYMITY OF PEER REVIEWERS BECAUSE WE ARE ALL HUMANS AND THERE IS A FEAR OF RETRIBUTION, OF APPEARING TO BE EXERTING BIAS, WHICH DEFINITELY HAPPENS AND NEEDS TO BE STUDIED. BUT I THINK THAT CAN ONLY BE STUDIED IF WE HAVE THE PEER REVIEW AVAILABLE. SO, CERTAINLY THERE ARE JOURNALS LIKE E-LIFE ALLOWING THIS INFORMATION TO COME BE ANALYZED ITSELF, BUT AS A WHOLE, WE REALLY DON'T UNDERSTAND WHAT IS GOING ON IN PEER REVIEW. SO, WE HELD A MEETING IN FEBRUARY IN WHICH WE DISCUSSED THE IDEA OF ENCOURAGING JOURNALS TO PUBLISH THE CONTENT OF PEER REVIEW FOR A VARIETY OF REASONS THAT I WILL MENTION. ESSENTIALLY, WE WOULD IDEALLY LIKE TO ENCOURAGE GOOD REVIEWER BEHAVIOR RATHER THAN NEGATIVE REVIEW BEHAVIOR. SO, CLEARLY SOME OF THE UNFAIR COMMENTS, UNFOUNDED ACCUSATIONS, ET CETERA, IF A REVIEWER HAD TO SIGN THEIR NAME, I ASSUME THAT WOULD PROVIDE EXTRA INCENTIVE BUT EVEN A JOURNAL ALLOWING THIS CONTENT TO BE PUBLISHED WOULD PROVIDE MORE INSURANCE THAT THE EDITORS WOULD FEEL OBLIGATED TO MAKE SURE THIS PROCESS WAS HAPPENING IN AN OPEN FASHION. PEER REVIEW SAY FORM OF SCHOLARSHIP. SO, IT IS INCREDIBLY IMPORTANT FOR IMPROVING SCIENTIFIC WORK AND IT SHOULD NOT JUST BE DISCARDED OR HIDDEN OR SOMETHING AFTER THE PROCESS IS COMPLETE. JOURNALS, I THINK, HAVE AN OBLIGATION TO EARN THE TRUST OF THEIR READERS. I THINK ESPECIALLY WITH GROWING CONCERNS ABOUT PREDATORY PUBLISHING, PREDATORY PUBLISHING IS I THINK, NOTHING ELSE OTHER THAN JOURNALS NOT ACTUALLY CONDUCTING THE TYPE OF PEER REVIEW THEY SAY. SO, IF THE PEER REVIEW OR THE CONTENT OF THIS PROCESS COULD BE MADE MORE PUBLICLY AVAILABLE, I THINK WE WOULD BE ABLE TO ASSESS THESE JOURNALS IN A MORE OBJECTIVE FASHION. NEXT, IT MAKES A DECISION THAT THESE JOURNALS MORE TRANSPARENT. THAT EFFECTIVELY, EDITORIAL DECISIONS ARE PLAYING A BIG ROLE IN WHAT WORK GETS HIGHLIGHTED, EVEN UPON WHO ENDS UP GETTING JOBS BECAUSE OUR ENTIRE EVALUATION SYSTEM IS IN SOME WAY RESTING ON THIS. SO I THINK IT'S VERY IMPORTANT THIS PROCESS BE TRANSPARENT. IT ALLOWS REVIEWER RECOGNITION. IT ENABLES REVIEWER RECOGNITION AND DOESN'T CREATE A SYSTEM THAT MAKES IT INCREDIBLY OBVIOUS BECAUSE OBVIOUSLY IF PEOPLE ARE NOT SIGNING THEIR NAMES, YOU WOULD ALSO NEED SOME SORT OF THIRD PARTYISH MEDIARY, OR YOU NEED TO TRUST AUTHORS SENDING, I WROTE THIS PEER REVIEW, ET CETERA. -- INTERMEDIARY -- WITH A CULTURAL SHIFT WITH THE RECOGNITION OF PEER REVIEW AS SCHOLARSHIP, I WOULD LOVE TO SEE PEER REVIEW BEING RECOGNIZED AS A FORM OF SERVICE AND A CONTRIBUTION TOWARDS THE PROGRESS OF SCIENCE AS A WHOLE. AND NEXT I THINK IT HELPS TRAINEES WHO CERTAINLY WHEN I GOT MY FIRST PEER REVIEW BACK, IT SORT OF -- WHAT YOU COMPARE THIS TO IS THIS TYPE OF CRITICISM NORMAL? OR FOR PEOPLE WHITING THEIR FIRST PEER REVIEWS, WHICH I WILL COME BACK TO IN JUST A SLIDE, WHAT MAKES A GOOD PEER REVIEW? WHAT MAKES A NON-CONSTRUCTIVE PEER REVIEW? GIVEN HOW IMPORTANT THIS PROCESS IS, I THINK IT IS IMPORTANT THAT WE ALLOW PEOPLE TO EXAMINE WHAT THE CONSTRUCTIVE EVALUATION OF PEER REVIEW LOOKS LIKE. AND NEXT, IT ENABLES THE STUDY OF PEER REVIEW IN A MORE SYSTEMIC FASHION. THIS ALLOWS RECOGNITION OF PROBLEMS OF BIAS AGAINST PEOPLE FROM DIFFERENT ETHNICITIES, GENDERS, INSTITUTIONS, COUNTRIES, DIFFERENT LANGUAGE BACKGROUNDS, TO COME INTO THE LIGHT, WHICH I THINK IS SOMETHING THAT IS AN UNSEEN FACTOR THAT -- AND CERTAINLY FELT BY MANY PEOPLE BUT NOT REALLY BROUGHT INTO FOCUS. SO, IN RESPONSE TO THIS, WE HAVE ORGANIZED A LETTER, AN OPEN LETTER, WHICH ENCOURAGES JOURNALS TO PUBLISH THE CONTENT OF THE PEER REVIEW REPORTS AND THE AUTHOR RESPONSES. AND SO, WE HAVE OVER 100 SIGNATORIES, SOME OF WHICH HAVE NOT PUBLISHED THEIR PEER REVIEW REPORTS BEFORE, BUT HAVE COMMITTED TO DO SO BY A PARTICULAR DATE. AND SO, OUR GOAL HERE IS NOT ONLY FOR THESE SIGNATORIES, SOME OF WHICH ARE ALREADY IMPLEMENTING THIS, TO DO THIS FOR THE FIRST TIME, BUT ALSO TO COMMUNICATE WITH ONE ANOTHER ABOUT WHAT THE BEST PRACTICES ARE IN THIS. THERE IS A LOT OF DEBATE AS TO DO WE NEED ALL VERSIONS OF THE MANUSCRIPT? SHOULD THESE REVIEWS BE EDITEDET? SETRA. HOW CAN -- ET CETERA. HOW CAN THE GOALS WHICH JOURNALS HAVE OF INCREASING TRANSPARENCY AND IMPROVING THE PEER REVIEW PROCESS, HOW CAN THOSE BE ATTAINED IN THE MOST EFFECTIVE WAY? SO ANOTHER ELEMENT EVER TRANSPARENCY, I WILL STEP BACK INTO THE FUTURE OF RESEARCH HAT NOW, IS THE PHENOMENON OF GHOST REVIEWING. SO, PEER REVIEWS ARE OBVIOUSLY WRITTEN BY MOSTLY PIs ARE INVITED TO PEER REVIEW, SENIOR POSTDOCS ARE INVITED AS WELL AS AND I THINK THIS IS A IMPORTANT WAY TO EXPAND THE POPULATION OF REVIEWERS. THERE IS A LOT OF DISCUSSION ABOUT A SMALL REVIEWER POOL, AND THE FACT THAT THERE WAS A SURVEY THAT CAME OUT LAST WEEK WHICH DESCRIBED HOW THERE IS A HUGE ASYMMETRY IN THE WAY PEER REVIEW DUTIES ARE DISTRIBUTED WITH MANY PEOPLE -- I THINK THE FACTOR -- I DON'T WANT TO MISQUOTE THIS, BUT 10% OF REVIEWERS ARE CONDUCTING 50% OF PEER REVIEW. PEOPLE TYPICALLY FROM ENGLISH-SPEAKING COUNTRIES, ARE BEING ASKED TO PEER REVIEW MUCH MORE THAN AUTHORS OF OTHER COUNTRIES. YET AT THE SAME TIME, POSSIBLY WHAT IS HAPPENING IS THAT THERE IS UNKNOWN CONTRIBUTION TO THESE PEER REVIEWS FROM THE PEOPLE WHO ARE IN THEIR LAB. AND I THINK THIS IS GREAT BECAUSE TRAINING PEER REVIEWS IS REALLY VALUABLE. BUT I THINK IS THERE A FUNDAMENTAL NEED TO ACKNOWLEDGE TO THE JOURNAL, THAT THIS IS HAPPENING, BOTH TO KIND OF PRESERVE THE BASIC FUNDAMENTAL TENANT OF PROVIDING CREDIT FOR PEOPLE WHO CONTRIBUTED, AND TO MAKE IT CLEAR WHERE THIS INFORMATION IS COMING FROM. BUT ALSO TO HELP THOSE JOURNALS IDENTIFY NEW REVIEWERS THEY CAN ASK TO PEER REVIEW. E-LIFE DID A FASCINATING SURVEY IN THEY FOUND 37% OF PH.D. STUDENTS ARE STILL PERFORMING REVIEW WITHOUT THE ASSISTANCE OF THEIR ADVISOR. SO, FUTURE OF RESEARCH WOULD LIKE TO GATHER MORE INFORMATION ABOUT THIS AND LEARN MORE ABOUT EARLY-CAREER RESEARCHERS AND POSTDOCS AND GRAD STUDENTS ATTITUDES TOWARDS PEER REVIEW. SO I THINK THIS WILL BE OPEN UNTIL SEPTEMBER 21. WE WOULD LOVE YOUR INPUT ON THAT AS WELL. SO, THAT SAID, I'M JUST IN THE LAST FEW MINUTES HERE, I WANT TO MAKE THE POINT THAT EVEN THOUGH I'M SUPER EXCITED ABOUT ALL OF THESE CHANGES THAT ARE HAPPENING, THE GROWTH OF PREPRINTS IS REALLY EXPLOSIVE, EXCITING. YET, WHEN YOU GRAB IT COMPARED TO THE NUMBER OF PAPERS APPEARING ON PUBMED, IT LOOKS MUCH LESS AMAZING. SO I THINK IT LEADS TO THE 90 THERE IS STILL SO MUCH TO DO TO HELP BRING PUBLISHING AND HELP SHAPE PUBLISHING INTO A FUTURE THAT IS GOING TO BE MORE CONDUCIVE TO SHARING A BIT OF INFORMATION AND DISCOVERY, ESPECIALLY FOR PEOPLE WHO ARE EARLY IN THEIR CAREER. SO, I THINK WHAT CAN POSTDOCS DO PRACTICALLY? I THINK THAT THERE IS POSTING PREPRINTS AND COMMENTING ON PREPRINTS, OR COVERING THEM IN JOURNAL CLUBS, CANNOT ONLY ENCOURAGE THE USE OF THIS MEDIUM, BUT ALSO CHANGE THE CULTURE AROUND WHAT KIND OF FEEDBACK WE ARE GIVING TO ONE ANOTHER, WHETHER THAT IS AN ANONYMOUS CRITICISM AT A JOURNAL, VERSUS A MORE OPEN AND CONSTRUCTIVE COMMENT THAT WILL HELP SOMEONE IMPROVE THEIR PAPER WITHOUT GATE KEEPING AND WHERE IT WILL BE EVENTUALLY APPEAR. YOU CAN ALSO MAKE DECISIONS ABOUT JOURNALS WITH YOUR CO-AUTHORS IN A WAY THAT WILL SEND SIGNALS ABOUT WHAT YOU VALUE. IN OTHER WORDS, CHOOSING THINGS THAT MATCH YOUR VALUES ON PREPRINTS AND PEER REVIEW, WHATEVER THOSE MAY BE. AND ALSO WE HAVE A PROGRAM CALLED, A SAP BIOAMBASSADORS WHERE WE ARE LOOKING FOR PEOPLE EXCITED ABOUT TALKING ABOUT THESE ISSUES OUTSIDE OF BASICALLY TWITTER. I THINK THERE IS A GIANT ECHO CHAMBER OF PEOPLE WHO ARE VERY ENTHUSIASTIC ABOUT THESE NEW FORMS OF PUBLISHING, BUT THERE IS LITTLE PENETRANTS INTO THE WORLD OUT THERE. AND SO, WE HAVE REPRESENTATIVES ACROSS THE GLOBE WHO ARE SHARING THEIR STORIES ABOUT PREPRINTS THAT ARE -- I ALSO HAVE A BUNCH OF STICKERS FOR ALL OF YOU. WHO ARE BASICALLY MAKING IT OBVIOUS TO PEOPLE THAT RAISING THE PROFILE AND AWARENESS OF PREPRINTING IN SOME OF THESE INNOVATIONS AS WELL. SO IF THIS IS SOMETHING THAT INTERROGATES YOU, LET ME KNOW. I WANT TO THANK EVERYONE -- INTERESTS YOU, LET ME KNOW. I WANT TO THANK ALL OF YOU FOR YOUR TIME AND I LOOK FORWARD TO FURTHER DISCUSSION. THANKS. [ APPLAUSE ] >> I CAN TAKE QUESTIONS FROM THE AUDIENCE. PLEASE COME UP TO THE MIC SO PEOPLE CAN HEAR ON THE VIDEO CAST. >> GOOD MORNING. THANK YOU VERY MUCH. THAT WAS EXCELLENT AND I REALLY APPRECIATE THE ANGLES OF THE SITUATIONS YOU COVERED. I'M WONDERING, SETTING ASIDE THE FACT THAT THERE ARE SYSTEMS AND SYSTEMATIC WAY THAT IS SOME OF THESE ISSUES EXACERBATED OR PERPETUATED, IN TERMS OF WHAT POSTDOCS CAN DO AND THE WORK YOU HAVE BEEN DOING, ARE YOU SEEING AN ELEMENT OF SELF ADVOCACY TAKE PLACE? A LOT OF WHAT YOU'RE TALKING ABOUT REALLY INVOLVES POSTDOCS EARLY IN THEIR CAREERS SEEING THEMSELVES AS BEING IN CHARGE AND SEEING THEMSELVES AS BEING ABLE TO PUSH THEIR INFLUENCE AND THEIR POWER. SO DO YOU SEE FOLKS BEGINNING TO THINK ABOUT THAT ANGLE? >> THAT'S A FANTASTIC POINT. I DON'T HAVE ANY DATA THAT SUPPORTS WHAT YOU ARE RAISING, WHICH IS THIS IDEA THAT THERE IS A NEED FOR INDIVIDUALS TO TAKE MORE CHARGE OF THEIR CAREERS, TO BE MORE ACTIVE ABOUT MAKING DISCUSSIONS. I THINK ONE NOTION S THE IDEA THAT THE DEFAULT POSTDOC. SO IN OTHER WORDS, PEOPLE MOVING INTO A POSTDOC BECAUSE, THIS IS THE NEXT STEP AND JUST SORT OF DRIFTING ALONG. I THINK THAT THAT IS SOMETHING THAT I REALLY HOPE TO CHANGE, AND CLEARLY PROGRAMS THAT ARE MAKING OR THINKING ABOUT CAREERS EARLY ON, A PART OF THE CURRICULUM OR A PART OF THE DISCUSSION IS SOMETHING THAT IS VERY IMPORTANT FOR THAT. AND LIKEWISE WITH PUBLISHING. I THINK IT TAKES A LOT OF ENERGY TO KIND OF CHANGE, OR TO THINK ABOUT WAYS TO CHALLENGE THE STATUS QUO, ESPECIALLY WHEN IT IS FUNDAMENTALLY ENGRAINED WITH THE EVALUATION SYSTEM, WHICH IS SUCH A DIFFICULT CHALLENGE. BUT I THINK THAT THERE ARE INTERVENTIONS LIKE PREPRINTING OR COMMENTING, THAT ARE NOT GOING TO HARM OR INTERFERE NEGATIVELY WITH THAT SYSTEM. >> A GREAT TALK. I'M WONDERING IF YOU HAVE OR ARE COLLECTING DATA ON HOW THE PREPRINTS ARE STRATIFIED BY INSTITUTION? >> YES. THAT'S A REALLY GREAT QUESTION. I THINK THAT THERE ARE -- AND I WON'T BE ABLE TO LIST THEM OFF THE TOP OF MY HEAD BUT THERE ARE DEFINITELY INSTITUTIONS THAT ARE PRODUCING MORE PREPRINTS, MORE SPECIFICALLY FIELDS THAT ARE ALSO PRODUCING WAY MORE PREPRINTS. SO FOR EXAMPLE, GENOMICS OR COMPUTATIONAL BIOLOGY. SOME FIELDS ARE ADJACENT TO COMPUTER SCIENCE OR PHYSICS ARE MORE AT THE FOREFRONT OF TRYING OUT THIS FORM OF COMMUNICATION. BUT, YES, THERE IS A FANTASTIC PAPER BY TWO -- AND HERSELF BEE AND AUTHORIZE BEE AND OTHER AUTHORS -- - THURSBY -- LIKELIHOOD OF SHARING WORK EARLY ON. AND ONE OF THE FACTORS THAT WAS CORRELATED WITH SHARING EARLY WAS BEING MORE SENIOR IN ONE'S CAREER. SO I THINK THERE IS AN IDEA, WHILE PEOPLE WHO ARE PERHAPS GROWING UP IN A MORE DIGITAL REALM ARE MORE COMFORTABLE MAYBE WITH SOME OF THESE INNOVATIONS, AT THE SAME TIME, THERE IS THAT NEEDS TO BE BALANCED HOW SECURE PEOPLE FEEL IN TRYING SOMETHING NEW. >> THANK YOU FOR LOTS OF THINGS TO THINK ABOUT HERE. I HAVE A QUESTION REGARDING THE -- THE PREPRINT PROCESS WORKS BEST IF YOU HAVE PEOPLE THAT ARE GOING TO GO -- [ OFF MICROPHONE ] >> FANTASTIC QUESTION. FIRST, TO ANSWER YOUR QUESTION, THERE IS THE RATE OF COMMENTING ON PREPRINT, BIOARCHIVE, IS NOW ABOUT 10%, WHICH SOUNDS LOW UNTIL YOU CONSIDER THE RATE OF COMMENTING ON PUBLISHED PAPERS IN GENERAL, WHICH IS ACTUALLY ORDERS OF MAGNITUDE HIGHER THAN PEOPLE COMMENTING ON PAPERS THAT ARE PUBLISHED IN A JOURNAL. I THINK THAT THERE IS A LOT OF COMMENTING THAT IS HAPPENING NOT AT THE PREPRINT SERVER ITSELF BUT ON SOCIAL MEDIA OR ON SOME OF THESE OTHER PLATFORMS. I THINK THAT THERE IS SOMETHING A LITTLE BIT -- CAN BE A LITTLE AWKWARD ABOUT COMMENTING DIRECTLY ON A PAPER WHERE PEOPLE ARE GOING TO GO FIND IT. I THINK PEOPLE MAY PREFER TO SEND COMMENTS BY E-MAIL. FOR EXAMPLE, I POSTED A PREPRINT AND GOT A COMMENT ONCE WHERE SOMEONE E-MAILED ME AND SAID, I WOULDN'T POST THIS PUBLICLY BUT I DON'T WANT TO SEEM BILIG RANT EVEN THOUGH IT WAS A VERY CONSTRUCTIVE COMMENT BUT THEY DON'T WANT TO PROVIDE ANYTHING THAT COULD BE CONCEIVED AS CRITICAL PUBLICLY. SO I THINK THIS IS A HUGE CULTURAL ISSUE THAT NEEDS TO BE CONSIDERED IN LIGHT OF WHETHER ANONYMITY IS PERMITTED OR WHETHER PEOPLE CAN COMMENT UNDER A PSEUDONYM OR HOW MODERATION CAN STRIKE THE RIGHT BALANCE BETWEEN COMMENTS THAT ARE BOTH CONSTRUCTIVE AND ALLOW PEOPLE TO FEEL MORE SAFE MAKING THEM. >> [ OFF MICROPHONE ] >> THAT'S A VERY GOOD POINT. I THINK MANY NEW SITES HAVE DECIDED TO CLOSE COMMENTING BECAUSE IT'S JUST A -- I THINK IT CANNOT BE OR IS NOT ALWAYS CONSTRUCTIVE. IN THE CASE OF BIOARCHIVE, I DO KNOW THE BIOARCHIVE TEAM DOES SPEND TIME MODERATING THOSE COMMENTS, AND WILL REDACT COMMENTS THAT ARE NON-CONSTRUCTIVE, BUT OF COURSE I THINK THERE IS IT A REALLY INTERESTING GRAY AREA LIKE EVERY DIFFERENT COMMUNITY HAS A DIFFERENT STANDARD FOR WHAT IS CONSIDERED YOU OFFENSIVE OR NOT. PUB PEER HAS A DIFFERENT SORT OF LEVEL OF CRITICISM THAN IS NORMAL. SO, I COMPLETELY AGREE WITH YOU AND I THINK THE ONLY THING THAT I WOULD SAY ALSO IS THAT PERHAPS OTHER FORMS OF METAMODERATION OR MODERATION OF COMMENTS OR PUBLIC MODERATION OF COMMENTS -- LIKE FOR EXAMPLE, ALLOWING PEOPLE TO VOTE COMMENTS UP OR DOWN AS A WAY OF SURFACING THE MOST USEFUL AND MOST INTERESTING WORK. I THINK WE CAN LOOK AT SITES LIKE READ IT, FOR EXAMPLE, WHERE USERS CAN BE DEMOTED OR ENCOURAGED BASED UPON THE QUALITY OF WHAT THEY ARE WRITING. >> ONE LAST QUESTION AND THEN WE'LL BREAK FOR LUNCH. LET'S SAY I'M A POSTDOC AND I HAVE SOMETHING GREAT I WANT TO POST AND THEN MY BOSS IS LIKE, NO, YOU CAN NOT POST ON PREPRINT SERVERS. WE MIGHT GET SCOOPED OR WHATEVER. WHAT ARE SOME CONSTRUCTIVE WAYS YOU FOUND THAT POSTDOCS ARE ABLE& TO MAYBE TAKE THE AGENCY THAT WE TALKED ABOUT AND AT LEAST MAYBE HELP THEMSELVES IN THEIR SITUATION IF THEY ARE IN THAT KIND OF SITUATION? >> ABSOLUTELY. I THINK THAT ONE THING THAT IS REALLY HELPFUL IS CONNECTING PEOPLE TO THEIR OWN PEERS WHO HAVE DONE THE THING. SO FOR EXAMPLE, IF YOUR PI OR YOUR CO-AUTHOR HAS ANOTHER CO-AUTHOR OR A -- SOMEONE ELSE IN THE FIELD WHO HAS POSTED A PREPRINT, I THINK IT CAN BE VERY CONSTRUCTIVE TO HELP A CONVERSATION HAPPEN BETWEEN THE TWO OF THEM ABOUT WHAT THEIR EXPERIENCE HAS BEEN. I THINK SHOWING THAT CERTAIN PRACTICE IS GAINING STEAM OR MOMENTUM AMONG A GROUP OF PEOPLE, IS REALLY POWERFUL FOR ENABLING PEOPLE TO KIND OF EXAMINE OR TRY OUT SOMETHING NEW THEY WOULDN'T OTHERWISE. SO PEER PRESSURE, BASICALLY. >> THANK YOU VERY MUCH. [ APPLAUSE ]