Evidence-b(i)ased practice: Selective and inadequate reporting in early childhood autism intervention research

Added on 21/06/2024

Type de contenu

Journal article of the type Scientific literature review ( ; english)

Micheal Sandbank , Kristen Bottema-Beutel , Ya-Cing Syu , Nicolette Caldwell , Jacob Feldman , Tiffany Woynaroski
Evidence-b(i)ased practice: Selective and inadequate reporting in early childhood autism intervention research published in the journal "Autism" Preprint, 15 pages , doi: 10.1177/13623613241231624

=> Unrestricted access and free

Summary/Presentation

When researchers fail to report their findings or only report some of their findings, it can make it difficult for clinicians to provide effective intervention recommendations. However, no one has examined whether this is a problem in studies of early childhood autism interventions. We studied how researchers that study early childhood autism interventions report their findings. We found that most researchers did not register their studies when they were supposed to (before the start of the study), and that many researchers did not provide all of the needed information in the registration. We also found that researchers frequently did not publish their findings when their studies were complete. When we looked at published reports, we found that many of the studies did not report enough information, and that many studies were reported differently from their registrations, suggesting that researchers were selectively reporting positive outcomes and ignoring or misrepresenting less positive outcomes. Because we found so much evidence that researchers are failing to report their findings quickly and correctly, we suggested some practical changes to make it better.

For your information:

(1) References in blue are resources listed on our site.

(2) Authors listed in this bibliography whose names are in color have published other resources referenced on the site. Clicking on the name allows you to see the list of resources they have published and shared on the site.

(3) Authors whose names are followed by an asterisk have publicly disclosed being autistic.

  • R. Bashir, F. Bourgeois, A. Dunn (2017). "A systematic review of the processes used to link clinical trial registrations to their published results". Systematic Reviews 6. doi:10.1186/s13643-017-0518-3

  • I. Boutron, M. Page, J. Higgins*, D. Altman, A. Lundh, A. Hrõbjartsson (2019). "Considering bias and conflicts of interest among the included studies". Cochrane Handbook for Systematic Reviews of Interventions doi:10.1002/9781119536604.ch7

  • A. Chan, A. Hrõbjartsson, M. Haahr, P. Gøtzsche, D. Altman (2004). "Empirical evidence for selective reporting of outcomes in randomized trials: comparison of protocols to published articles.". JAMA doi:10.1001/jama.291.20.2457

  • R. Chen, N. Desai, J. Ross, W. Zhang, K. Chau, B. Wayda, K. Murugiah, D. Lu, A. Mittal, H. Krumholz (2016). "Publication and reporting of clinical trial results: cross sectional analysis across academic medical centers". The BMJ 352. doi:10.1136/bmj.i637

  • J. Chow, M. Sandbank, L. Hampton (2023). "Guidance for Increasing Primary Study Inclusion and the Usability of Data in Meta-Analysis: A Reporting Tutorial.". Journal of speech, language, and hearing research : JSLHR doi:10.1044/2023_jslhr-22-00318

  • P. Cuijpers, E. Weitz, I. Cristea, J. Twisk (2016). "Pre-post effect sizes should be avoided in meta-analyses". Epidemiology and Psychiatric Sciences 26. doi:10.1017/s2045796016000809

  • J. Deeks, J. Higgins*, D. Altman (2008). "Chapter 9: Analysing Data and Undertaking Meta-Analyses".

  • M. Fredrickson, B. Ilfeld (2011). "Prospective Trial Registration for Clinical Research: What Is It, What Is It Good for, and Why Do I Care?". Regional Anesthesia & Pain Medicine 36. doi:10.1097/aap.0b013e318230fbc4

  • M. Gabelica, R. Bojcic, L. Puljak (2022). "Many researchers were not compliant with their published data sharing statement: mixed-methods study.". Journal of clinical epidemiology doi:10.1016/j.jclinepi.2022.05.019

  • B. Goldacre, N. Devito, C. Heneghan, F. Irving, S. Bacon, J. Fleminger, H. Curtis (2018). "Compliance with requirement to report results on the EU Clinical Trials Register: cohort study and web resource". The BMJ 362. doi:10.1136/bmj.k3218

  • D. Hartung, D. Zarin, J. Guise, M. McDonagh, R. Paynter, M. Helfand (2014). "Reporting Discrepancies Between the ClinicalTrials.gov Results Database and Peer-Reviewed Publications". Annals of Internal Medicine 160. doi:10.7326/m13-0480

  • J. Higgins*, J. Savović, M. Page, R. Elbers, J. Sterne (2019). "Assessing risk of bias in a randomized trial". Cochrane Handbook for Systematic Reviews of Interventions doi:10.1002/9781119536604.ch8

  • S. Hyman, S. Levy, S. Myers (2019). "Identification, Evaluation, and Management of Children With Autism Spectrum Disorder". Pediatrics 145. doi:10.1542/peds.2019-3447

  • A. Johnson, B. Cook (2019). "Preregistration in Single-Case Design Research". Exceptional Children 86. doi:10.1177/0014402919868529

  • C. Jones, L. Handler, K. Crowell, L. Keil, M. Weaver, T. Platts-Mills (2013). "Non-publication of large randomized clinical trials: cross sectional analysis". The BMJ 347. doi:10.1136/bmj.f6104

  • J. Kampman, N. Weiland, M. Hollmann, S. Repping, J. Hermanides (2021). "High incidence of outcome switching observed in follow-up publications of randomized controlled trials: meta-research study.". Journal of clinical epidemiology doi:10.1016/j.jclinepi.2021.05.003

  • R. Kaplan, V. Irvin (2015). "Likelihood of Null Effects of Large NHLBI Clinical Trials Has Increased over Time". PLoS ONE 10. doi:10.1371/journal.pone.0132382

  • N. Kerr (1998). "HARKing: Hypothesizing After the Results are Known". Personality and Social Psychology Review 2. doi:10.1207/s15327957pspr0203_4

  • B. Nosek, C. Ebersole, A. DeHaven, D. Mellor (2018). "The preregistration revolution". Proceedings of the National Academy of Sciences 115. doi:10.1073/pnas.1708274114

  • B. Nosek, G. Alter, G. Banks, D. Borsboom, S. Bowman, S. Breckler, S. Buck, C. Chambers, G. Chin, G. Christensen, M. Contestabile, A. Dafoe, E. Eich, J. Freese, R. Glennerster, D. Goroff, D. Green, B. Hesse, M. Humphreys, J. Ishiyama, D. Karlan, A. Kraut, A. Lupia, P. Mabry, T. Madon, N. Malhotra, E. Mayo-Wilson, M. McNutt, E. Miguel, E. Paluck, U. Simonsohn, C. Soderberg, B. Spellman, J. Turitto, G. VandenBos, S. Vazire, E. Wagenmakers, R. Wilson, T. Yarkoni (2015). "Promoting an open research culture". Science 348. doi:10.1126/science.aab2374

  • M. Page, J. Higgins* (2016). "Rethinking the assessment of risk of bias due to selective reporting: a cross-sectional study". Systematic Reviews 5. doi:10.1186/s13643-016-0289-2

  • N. Pan, T. Gao, D. Qin, L. Qin (2023). "Autism intervention meta-analysis of early childhood studies (Project AIM): updated systematic review and secondary analysis". The BMJ 383. doi:10.1136/bmj-2023-076733

  • J. Ross, T. Tse, D. Zarin, H. Xu, L. Zhou, H. Krumholz (2012). "Publication of NIH funded trials registered in ClinicalTrials.gov: cross sectional analysis". The BMJ 344. doi:10.1136/bmj.d7292

  • K. Schulz, D. Altman, D. Moher (2010). "CONSORT 2010 Statement: updated guidelines for reporting parallel group randomised trials". The BMJ 340. doi:10.1136/bmj.c332

  • S. Serghiou, C. Axfors, J. Ioannidis (2023). "Lessons learnt from registration of biomedical research". Nature Human Behaviour 7. doi:10.1038/s41562-022-01499-0

  • R. Thibault, R. Clark, H. Pedder, O. van den Akker, S. Westwood, J. Thompson, M. Munafo (2021). "Estimating the prevalence of discrepancies between study registrations and publications: a systematic review and meta-analyses". BMJ Open 13. doi:10.1136/bmjopen-2023-076264

  • D. Trembath, H. Waddington, R. Sulek, K. Varcin, C. Bent, J. Ashburner, V. Eapen, E. Goodall, K. Hudry, N. Silove, A. Whitehouse (2021). "An evidence-based framework for determining the optimal amount of intervention for autistic children.". The Lancet. Child & adolescent health doi:10.1016/s2352-4642(21)00285-6

  • C. Voils, J. Crandell, Y. Chang, J. Leeman, M. Sandelowski (2011). "Combining adjusted and unadjusted findings in mixed research synthesis.". Journal of evaluation in clinical practice doi:10.1111/j.1365-2753.2010.01444.x

  • J. Wallach, K. Boyack, J. Ioannidis (2018). "Reproducible research practices, transparency, and open access data in the biomedical literature, 2015–2017". PLoS Biology 16. doi:10.1371/journal.pbio.2006930

  • C. Wayant, C. Scheckel, C. Hicks, T. Nissen, L. Leduc, M. Som, M. Vassar (2017). "Evidence of selective reporting bias in hematology journals: A systematic review". PLoS ONE 12. doi:10.1371/journal.pone.0178379

  • S. Weston, S. Ritchie, J. Rohrer, A. Przybylski (2018). "Recommendations for Increasing the Transparency of Analysis of Preexisting Data Sets". Advances in Methods and Practices in Psychological Science 2. doi:10.1177/2515245919848684

  • D. Zarin, T. Tse, R. Williams, T. Rajakannan (2017). "Update on Trial Registration 11 Years after the ICMJE Policy Was Established.". The New England journal of medicine doi:10.1056/nejmsr1601330

  • F. Šarić, O. Barcot, L. Puljak (2019). "Risk of bias assessments for selective reporting were inadequate in the majority of Cochrane reviews.". Journal of clinical epidemiology 112. doi:10.1016/j.jclinepi.2019.04.007

This resource has not (yet) been cited on Bluesky.


 
Summarize/Comment on/Translate this reference?