1
00:00:13,246 --> 00:00:14,914
>> Stacey Arnold: In this
third part of the series,

2
00:00:14,981 --> 00:00:17,183
I'll describe the process
for submitting results

3
00:00:17,250 --> 00:00:19,352
information to
ClinicalTrials.gov.

4
00:00:20,353 --> 00:00:23,089
Topics for this session
include an overview of results

5
00:00:23,156 --> 00:00:24,791
reporting to ClinicalTrials.gov

6
00:00:24,858 --> 00:00:27,560
via the protocol registration
and results system,

7
00:00:27,627 --> 00:00:30,864
or PRS, including the
objectives of reporting

8
00:00:30,930 --> 00:00:33,032
and the components of
results submissions.

9
00:00:33,566 --> 00:00:37,103
We'll also review key points to
consider when reporting results.

10
00:00:38,838 --> 00:00:40,774
The ClinicalTrials.gov
Results Database

11
00:00:40,840 --> 00:00:43,009
was designed to satisfy
legal requirements

12
00:00:43,076 --> 00:00:45,779
for the sharing of clinical
trial results information.

13
00:00:46,546 --> 00:00:48,448
The database aims
to promote objective,

14
00:00:48,515 --> 00:00:49,749
standardized reporting

15
00:00:49,816 --> 00:00:52,552
and to facilitate good
reporting practices,

16
00:00:52,619 --> 00:00:55,488
including publishing and
regulatory guidelines.

17
00:00:55,555 --> 00:00:58,425
The database provides
structured tabular data entry

18
00:00:58,491 --> 00:00:59,893
to ensure complete reporting

19
00:00:59,959 --> 00:01:01,928
and consistent display
of data elements

20
00:01:02,429 --> 00:01:04,264
and to facilitate
the quality assurance

21
00:01:04,330 --> 00:01:07,333
review of the information
submitted to ClinicalTrials.gov.

22
00:01:08,101 --> 00:01:11,004
This consistent structure
supports detailed searches

23
00:01:11,070 --> 00:01:13,039
of the ClinicalTrials.gov
database.

24
00:01:14,974 --> 00:01:17,477
As with registration,
results are manually entered

25
00:01:17,544 --> 00:01:20,280
or uploaded into
the PRS database

26
00:01:20,346 --> 00:01:23,049
for eventual posting
to ClinicalTrials.gov.

27
00:01:23,950 --> 00:01:26,119
Results can be submitted
once data collection

28
00:01:26,186 --> 00:01:29,522
has been completed for at least
one primary outcome measure

29
00:01:29,589 --> 00:01:31,591
and for records that have
been previously

30
00:01:31,658 --> 00:01:33,860
registered on
ClinicalTrials.gov.

31
00:01:34,994 --> 00:01:37,096
Results are posted
to ClinicalTrials.gov

32
00:01:37,163 --> 00:01:39,399
once they have undergone
a review process

33
00:01:39,466 --> 00:01:42,168
like the one performed for
registration information,

34
00:01:42,235 --> 00:01:44,304
meaning they must pass
an automated system

35
00:01:44,370 --> 00:01:45,905
validation process,

36
00:01:45,972 --> 00:01:48,975
and quality assurance review
by ClinicalTrials.gov staff.

37
00:01:50,844 --> 00:01:53,246
Results submissions included
scientific information

38
00:01:53,313 --> 00:01:56,716
provided in four modules,
including participant flow,

39
00:01:56,783 --> 00:02:00,186
baseline and demographic
characteristics,

40
00:02:00,253 --> 00:02:03,857
outcome measures and statistical
analyses and adverse events.

41
00:02:04,524 --> 00:02:07,760
An additional module,
limitations, and caveats,

42
00:02:07,827 --> 00:02:09,662
provides a free text field
where data submitters

43
00:02:09,729 --> 00:02:11,631
can provide information
about considerations

44
00:02:11,698 --> 00:02:13,433
for interpreting
the reported data.

45
00:02:14,000 --> 00:02:16,836
There are also two modules
for administrative information

46
00:02:16,903 --> 00:02:19,339
including contact information
for the point of contact

47
00:02:19,405 --> 00:02:20,907
for the study results,

48
00:02:20,974 --> 00:02:22,342
and a description
of any agreements

49
00:02:22,408 --> 00:02:23,877
between the principal
investigator

50
00:02:23,943 --> 00:02:25,078
and the study sponsor,

51
00:02:25,144 --> 00:02:27,881
including ones that restrict
the PI from discussing

52
00:02:27,947 --> 00:02:29,682
or publishing study results.

53
00:02:31,584 --> 00:02:33,586
The participant flow module
includes a table

54
00:02:33,653 --> 00:02:35,955
that resembles the
consort flow diagram

55
00:02:36,022 --> 00:02:37,924
that often serves
as the first figure

56
00:02:37,991 --> 00:02:39,792
in a clinical trial publication.

57
00:02:40,493 --> 00:02:42,829
It tracks the progress of
participants through the trial

58
00:02:42,896 --> 00:02:46,032
according to the arms or groups
to which they've been assigned.

59
00:02:46,633 --> 00:02:49,335
An arm represents a unique
experience, for example,

60
00:02:49,402 --> 00:02:51,838
receipt of a single
intervention or the sequence

61
00:02:51,905 --> 00:02:54,641
and interventions
for the duration of the trial.

62
00:02:54,707 --> 00:02:57,377
Studies specific milestones,
multiple periods,

63
00:02:57,443 --> 00:03:00,146
and reasons for participant
dropout can all be listed.

64
00:03:00,747 --> 00:03:03,283
An example of how a
consort flow diagram

65
00:03:03,349 --> 00:03:06,986
is translated into a participant
flow table is shown to the left.

66
00:03:10,490 --> 00:03:11,658
The baseline characteristics

67
00:03:11,724 --> 00:03:14,127
module provides demographic
and baseline data

68
00:03:14,193 --> 00:03:16,930
for the entire
population of the trial

69
00:03:16,996 --> 00:03:19,132
as well as each charm
or comparison group.

70
00:03:19,933 --> 00:03:22,368
Age, sex, race, ethnicity,

71
00:03:22,435 --> 00:03:24,337
and any measures
assessed at baseline

72
00:03:24,404 --> 00:03:26,472
that are used in the
analysis of the primary

73
00:03:26,539 --> 00:03:29,309
outcome measures are
required to be included.

74
00:03:29,976 --> 00:03:33,246
Examples of each type of measure
are shown to the left.

75
00:03:37,183 --> 00:03:39,452
The outcome measures and
statistical analysis

76
00:03:39,519 --> 00:03:42,789
module presents data by arm
or comparison group

77
00:03:42,855 --> 00:03:45,425
and accommodates
categorical, continuous,

78
00:03:45,491 --> 00:03:47,293
and time to event data.

79
00:03:47,360 --> 00:03:50,897
Examples of outcome measure
and statistical analysis tables

80
00:03:50,964 --> 00:03:52,565
are shown to the left.

81
00:03:55,735 --> 00:03:58,338
Three types of outcome measures
are reported in this module

82
00:03:58,404 --> 00:04:00,773
including primary
outcome measures,

83
00:04:00,840 --> 00:04:04,077
which represent the analyses of
greatest importance in the study

84
00:04:04,143 --> 00:04:05,645
and typically include
the assessment

85
00:04:05,712 --> 00:04:07,413
used in the power calculation,

86
00:04:08,114 --> 00:04:10,450
secondary outcome measures
which are of lesser importance

87
00:04:10,516 --> 00:04:12,652
than the primary outcomes
but are of interest

88
00:04:12,719 --> 00:04:15,221
in evaluating the effects
of the interventions,

89
00:04:15,288 --> 00:04:17,156
and other pre specified
outcome measures

90
00:04:17,223 --> 00:04:21,127
which are exploratory in nature.
A well planned outcome measure

91
00:04:21,194 --> 00:04:23,396
specifies four
levels of analysis

92
00:04:23,997 --> 00:04:26,666
that identifies a domain for
the analysis in this example,

93
00:04:26,733 --> 00:04:29,669
anxiety, defines a
specific measurement tool

94
00:04:29,736 --> 00:04:30,937
to explore the domain,

95
00:04:31,004 --> 00:04:33,139
like the Hamilton
anxiety rating scale,

96
00:04:33,773 --> 00:04:35,942
indicates a specific
metric for determining

97
00:04:36,009 --> 00:04:39,579
an effective interventions
such as change from baseline

98
00:04:40,146 --> 00:04:43,916
and identifies a method for
aggregating collected data,

99
00:04:43,983 --> 00:04:45,785
like report of the
proportion of participants

100
00:04:45,852 --> 00:04:48,388
with a greater than or equal
to 50 percent decrease

101
00:04:48,454 --> 00:04:50,189
in the measured scale score.

102
00:04:52,725 --> 00:04:55,828
The adverse events module lists
all collected events,

103
00:04:55,895 --> 00:04:58,698
whether anticipated
or unanticipated,

104
00:04:58,765 --> 00:05:02,602
that result in death, are
serious, or are non-serious

105
00:05:02,669 --> 00:05:04,137
and occurring above a frequency

106
00:05:04,203 --> 00:05:06,406
threshold of less than
or equal to 5 percent

107
00:05:06,472 --> 00:05:08,341
in any study arm or
comparison group.

108
00:05:09,208 --> 00:05:11,577
Examples of all three
adverse events tables

109
00:05:11,644 --> 00:05:13,246
are shown to the left.

110
00:05:16,516 --> 00:05:19,352
An additional section,
the study documents section,

111
00:05:19,419 --> 00:05:22,588
is required to be included at
the time of results submission.

112
00:05:22,655 --> 00:05:24,724
For studies with the primary
completion date on

113
00:05:24,791 --> 00:05:27,994
or after January 18, 2017,

114
00:05:28,795 --> 00:05:33,032
a full protocol and accompanying
statistical analysis plan

115
00:05:33,099 --> 00:05:36,536
are required to be submitted
and informed consent forms

116
00:05:36,602 --> 00:05:38,204
must also be posted for trials

117
00:05:38,271 --> 00:05:40,740
that are subject to the
Revised Common Rule.

118
00:05:42,809 --> 00:05:45,278
The statistical analysis
plan can be included

119
00:05:45,344 --> 00:05:47,714
as part of a single document
with the protocol

120
00:05:48,314 --> 00:05:50,216
or as a separate document.

121
00:05:50,283 --> 00:05:52,752
These documents must
represent all amendments

122
00:05:52,819 --> 00:05:55,555
approved by a human
subject's review board

123
00:05:55,621 --> 00:05:58,057
that apply to all
study locations,

124
00:05:58,124 --> 00:06:00,093
they must be accompanied
by a cover page

125
00:06:00,159 --> 00:06:03,229
that indicates the official
title for the study,

126
00:06:03,296 --> 00:06:05,732
the NCT number assigned
by ClinicalTrials.gov

127
00:06:05,798 --> 00:06:07,667
at the time of
trial registration,

128
00:06:07,734 --> 00:06:10,036
and the date of
the document version.

129
00:06:10,103 --> 00:06:12,071
Data submitters can
redact this information

130
00:06:12,138 --> 00:06:14,774
including personally
identifiable information,

131
00:06:14,841 --> 00:06:16,008
and trade secrets

132
00:06:16,075 --> 00:06:18,678
or other confidential
commercial information

133
00:06:18,745 --> 00:06:20,713
unless the information
is otherwise required

134
00:06:20,780 --> 00:06:22,048
to be submitted.

135
00:06:22,115 --> 00:06:26,285
The uploaded documents must be
in the archival PDF/A format

136
00:06:26,352 --> 00:06:28,321
and will be made public
on ClinicalTrials.gov,

137
00:06:28,387 --> 00:06:30,723
so they are also required
to be in English.

138
00:06:33,259 --> 00:06:35,328
Inclusion of the
informed consent form

139
00:06:35,394 --> 00:06:37,296
on ClinicalTrials.gov
is optional.

140
00:06:37,930 --> 00:06:40,933
However, ClinicalTrials.gov is
one of two federal websites

141
00:06:41,000 --> 00:06:43,336
that satisfy the requirement
of the Revised Common

142
00:06:43,402 --> 00:06:45,505
Rule that clinical
trials conducted

143
00:06:45,571 --> 00:06:46,873
or supported by a Common

144
00:06:46,939 --> 00:06:49,342
Rule department or
agency be posted

145
00:06:49,408 --> 00:06:51,644
on a publicly available
federal website

146
00:06:51,711 --> 00:06:53,346
within a specific timeframe.

147
00:06:54,046 --> 00:06:58,417
The compliance date for this
provision was January 21, 2019,

148
00:06:58,951 --> 00:07:01,154
and Regulations.gov
is the second site

149
00:07:01,220 --> 00:07:02,822
that satisfies the requirements.

150
00:07:05,224 --> 00:07:06,626
There are three main points
to consider

151
00:07:06,692 --> 00:07:09,028
when reporting results
to ClinicalTrials.gov.

152
00:07:09,095 --> 00:07:10,329
These include the manner

153
00:07:10,396 --> 00:07:13,166
in which the data will be
prepared for submission,

154
00:07:13,232 --> 00:07:16,469
the considerations needed
to satisfy review criteria,

155
00:07:16,536 --> 00:07:18,237
and the relationship
of reporting results

156
00:07:18,304 --> 00:07:19,872
to ClinicalTrials.gov

157
00:07:19,939 --> 00:07:23,075
to publishing results
in scientific journals.

158
00:07:25,845 --> 00:07:27,313
Summarizing results
for submission

159
00:07:27,380 --> 00:07:29,515
of ClinicalTrials.gov
is similar in complexity

160
00:07:29,582 --> 00:07:32,251
to preparing results
for journal publication.

161
00:07:32,318 --> 00:07:33,953
The process requires
an understanding

162
00:07:34,020 --> 00:07:37,023
of the study design
and analytic plan,

163
00:07:37,089 --> 00:07:38,925
a basic understanding
of the principles

164
00:07:38,991 --> 00:07:41,394
of clinical trial
conduct and analysis,

165
00:07:41,961 --> 00:07:44,096
and access to the
necessary data.

166
00:07:44,163 --> 00:07:46,432
Ideally, results should be
prepared for submission

167
00:07:46,499 --> 00:07:49,235
by the study's principal
investigator or statistician.

168
00:07:50,069 --> 00:07:52,705
The submitted information
must be accurate and precise,

169
00:07:52,772 --> 00:07:54,607
as well as understandable
and informative

170
00:07:54,674 --> 00:07:57,210
for someone who is a reader
of the medical literature,

171
00:07:57,276 --> 00:07:59,312
but not an expert in the field.

172
00:07:59,378 --> 00:08:01,147
It's recommended that
someone who is not familiar

173
00:08:01,214 --> 00:08:03,382
with the study review
the information for clarity

174
00:08:03,449 --> 00:08:04,817
and comprehension

175
00:08:04,884 --> 00:08:07,153
prior to submission
to ClinicalTrials.gov.

176
00:08:09,322 --> 00:08:11,324
It is important to note
that meeting the requirements

177
00:08:11,390 --> 00:08:13,192
for reporting to
ClinicalTrials.gov

178
00:08:13,259 --> 00:08:15,328
does not necessitate a change

179
00:08:15,394 --> 00:08:17,864
in study design or
study procedures.

180
00:08:17,930 --> 00:08:19,999
The reporting results should
reflect what was planned

181
00:08:20,066 --> 00:08:21,467
and collected as
part of the study

182
00:08:21,534 --> 00:08:24,303
as designed to meet the
research objectives.

183
00:08:24,904 --> 00:08:27,240
Results are intended for readers
of the medical literature

184
00:08:27,306 --> 00:08:28,474
and not the lay public.

185
00:08:28,541 --> 00:08:30,176
This is different
from information provided

186
00:08:30,243 --> 00:08:31,377
during registration,

187
00:08:31,444 --> 00:08:34,213
which should be understandable
to a lay audience

188
00:08:34,280 --> 00:08:36,682
to facilitate recruitment
of study participants.

189
00:08:37,283 --> 00:08:40,019
Results provided and data
tables should be informative

190
00:08:40,086 --> 00:08:41,888
with minimal narrative text.

191
00:08:42,455 --> 00:08:45,691
Conclusions and discussion
of the data are not permitted.

192
00:08:45,758 --> 00:08:48,160
The database is intended
for report of just the facts

193
00:08:48,227 --> 00:08:51,964
that is just the summary level
data and results submission

194
00:08:52,031 --> 00:08:53,966
is not required for
all registered trials

195
00:08:54,033 --> 00:08:56,335
only those that are subject
to the final rule,

196
00:08:56,402 --> 00:08:59,205
NIH policy, or other
funding policies.

197
00:09:01,274 --> 00:09:04,010
The ClinicalTrials.gov quality
assurance team reviews records

198
00:09:04,076 --> 00:09:07,113
for apparent validity,
meaningful entries, logic,

199
00:09:07,179 --> 00:09:10,249
and internal consistency
and will return a record

200
00:09:10,316 --> 00:09:13,519
with comments requiring the
resolution of major problems

201
00:09:13,586 --> 00:09:15,621
when the following types
of issues arise.

202
00:09:16,155 --> 00:09:17,723
When there's a lack
of apparent validity,

203
00:09:17,790 --> 00:09:18,925
such as an outcome measure

204
00:09:18,991 --> 00:09:22,161
that reports a mean of 263
hours of sleep per day,

205
00:09:22,995 --> 00:09:24,697
when a description
of an analysis

206
00:09:24,764 --> 00:09:26,332
is too big to be meaningful.

207
00:09:26,399 --> 00:09:27,867
For example, when
an outcome measure

208
00:09:27,934 --> 00:09:30,336
reports the results
of a scaled assessment

209
00:09:30,403 --> 00:09:33,339
without sufficient information
to interpret the result,

210
00:09:33,406 --> 00:09:37,109
when data are inconsistent with
the description of the analysis,

211
00:09:37,176 --> 00:09:38,377
for example, when
an outcome measure

212
00:09:38,444 --> 00:09:39,879
reports account of participants

213
00:09:39,946 --> 00:09:43,215
rather than a median and
full range or a time to event.

214
00:09:44,283 --> 00:09:46,519
When information throughout
a record is inconsistent,

215
00:09:46,585 --> 00:09:47,787
for example,

216
00:09:47,853 --> 00:09:50,389
when a study designated
as observational indicates

217
00:09:50,456 --> 00:09:53,192
that randomization was used
to assign participants

218
00:09:53,259 --> 00:09:54,694
to interventions,

219
00:09:54,760 --> 00:09:56,595
or when a trial
design is unclear.

220
00:09:57,129 --> 00:09:59,231
For example, when
participants suddenly appear

221
00:09:59,298 --> 00:10:00,666
in an outcome
measure analysis

222
00:10:00,733 --> 00:10:02,435
who were not included
in the participant

223
00:10:02,501 --> 00:10:05,438
flow or baseline
characteristics modules.

224
00:10:08,074 --> 00:10:11,143
With ClinicalTrials.gov in peer
reviewed journal publications

225
00:10:11,210 --> 00:10:14,313
seek to ensure accurate and
informative results reporting.

226
00:10:14,380 --> 00:10:16,615
However, they differ
in important ways.

227
00:10:16,682 --> 00:10:17,917
ClinicalTrials.gov results

228
00:10:17,984 --> 00:10:19,885
reporting does not
reject submissions.

229
00:10:20,453 --> 00:10:23,055
While peer reviewed journals
can refuse to publish research

230
00:10:23,122 --> 00:10:25,825
that isn't deemed high quality
or interest to readers.

231
00:10:26,425 --> 00:10:28,227
ClinicalTrials.gov
permits the disclosure

232
00:10:28,294 --> 00:10:30,529
of all outcome measures
that were assessed,

233
00:10:30,596 --> 00:10:32,231
while journals might
limit the focus

234
00:10:32,298 --> 00:10:34,734
to the report of critical
aspects of a study,

235
00:10:34,800 --> 00:10:36,802
thereby excluding
some outcome measures.

236
00:10:37,436 --> 00:10:40,973
And ClinicalTrials.gov does not
permit extensive narrative text.

237
00:10:41,040 --> 00:10:43,809
It is intended for tabular
summary level data only.

238
00:10:44,377 --> 00:10:46,846
In contrast, journal articles
rely upon narrative

239
00:10:46,912 --> 00:10:48,848
text and conclusive language

240
00:10:48,914 --> 00:10:51,017
to provide context
for reported results

241
00:10:51,083 --> 00:10:53,886
and to allow for interpretation
of comparative analyses.

242
00:10:54,820 --> 00:10:56,022
In summary, the

243
00:10:56,088 --> 00:10:58,090
ClinicalTrials.gov
Results Database

244
00:10:58,157 --> 00:11:00,693
was designed to fulfill
legal requirements,

245
00:11:00,760 --> 00:11:02,762
facilitate structured
data entry,

246
00:11:02,828 --> 00:11:05,865
and support detailed searches
of results information.

247
00:11:05,931 --> 00:11:08,267
Results are reported in
four scientific modules

248
00:11:08,334 --> 00:11:10,703
that reflect the flow of
participants through a study,

249
00:11:10,770 --> 00:11:12,538
their baseline characteristics,

250
00:11:12,605 --> 00:11:14,206
the results of outcome
measure analyses

251
00:11:14,273 --> 00:11:15,641
in which they participate,

252
00:11:15,708 --> 00:11:17,777
and any serious
adverse events frequent,

253
00:11:17,843 --> 00:11:20,279
non-serious adverse
events or deaths

254
00:11:20,346 --> 00:11:22,515
that occur over the
course of the study.

255
00:11:22,581 --> 00:11:24,683
There are also three types
of supporting documentation

256
00:11:24,750 --> 00:11:27,887
that can or should be appended,
including the study protocol,

257
00:11:27,953 --> 00:11:31,490
the statistical analysis plan,
and the informed consent form.

258
00:11:32,358 --> 00:11:34,326
Results should be
prepared for submission

259
00:11:34,393 --> 00:11:37,496
by someone who understands the
study design and analytic plan.

260
00:11:38,064 --> 00:11:40,466
Results submissions
to ClinicalTrials.gov

261
00:11:40,533 --> 00:11:42,935
serve to supplement
journal publication

262
00:11:43,002 --> 00:11:46,172
by allowing a board of
all analyses and studies.

263
00:11:48,307 --> 00:11:50,342
Now I'll ask a
couple of questions

264
00:11:50,409 --> 00:11:52,378
to review the material
that we covered.

265
00:11:53,145 --> 00:11:54,613
True or False?

266
00:11:54,680 --> 00:11:56,582
Results reporting is
required for all trials

267
00:11:56,649 --> 00:11:59,051
that have been registered
on ClinicalTrials.gov.

268
00:12:01,420 --> 00:12:04,256
This is false -- results
submission is not required

269
00:12:04,323 --> 00:12:07,259
for registered studies that are
not subject to the final rule,

270
00:12:07,326 --> 00:12:10,196
NIH policy, or other
funding policies.

271
00:12:12,431 --> 00:12:13,999
True or False?

272
00:12:14,066 --> 00:12:16,669
When reporting results on
ClinicalTrials.gov researchers

273
00:12:16,735 --> 00:12:18,904
can describe conclusions
drawn from analyses

274
00:12:18,971 --> 00:12:21,307
of all pre specified
outcome measures,

275
00:12:21,373 --> 00:12:24,410
even ones that aren't reported
in journal publications.

276
00:12:26,812 --> 00:12:28,614
This is false --
ClinicalTrials.gov

277
00:12:28,681 --> 00:12:32,651
does not permit study records
to include conclusive language

278
00:12:32,718 --> 00:12:34,453
or discussion of trial results.

279
00:12:37,123 --> 00:12:39,091
I hope you've learned about
the types of information

280
00:12:39,158 --> 00:12:42,495
included in results submissions
to ClinicalTrials.gov

281
00:12:42,561 --> 00:12:45,498
and how these submissions differ
from journal publications.

282
00:12:46,031 --> 00:12:47,633
Thank you for your attention.