Integrated paediatric training to improve child health

MacAulayC.

摘要

Background: While there is growing demand for information about comparative effectiveness (CE), there is substantial debate about whether and when observational studies have sufficient quality to support decision making. OBJECTIVE: To develop and test an item checklist that can be used to qualify those observational CE studies sufficiently rigorous in design and execution to contribute meaningfully to the evidence base for decision support. Methods: An 11-item checklist about data and methods (the GRACE checklist) was developed through literature review and consultation with experts from professional societies, payer groups, the private sector, and academia. Since no single gold standard exists for validation, checklist item responses were compared with 3 different types of external quality ratings (N=88 articles). The articles compared treatment effectiveness and/or safety of drugs, medical devices, and medical procedures. We validated checklist item responses 3 ways against external quality ratings, using published articles of observational CE or safety studies: (a) Systematic Review-quality assessment from a published systematic review; (b) Single Expert Review-quality assessment made according to the solicited "expert opinion" of a senior researcher; and (c) Concordant Expert Review-quality assessments from 2 experts for which there was concordance. Volunteers (N=113) from 5 continents completed 280 article assessments using the checklist. Positive and negative predictive values (PPV, NPV, respectively) of individual items were estimated to compare testers' assessments with those of experts. Results: Taken as a whole, the scale had better NPV than PPV, for both data and methods. The most consistent predictor of quality relates to the validity of the primary outcomes measurement for the study purpose. Other consistent markers of quality relate to using concurrent comparators, minimizing the effects of bias by prudent choice of covariates, and using sensitivity analysis to test robustness of results. Concordance of expert opinion on the quality of the rated articles was 52%; most checklist items performed better. Conculsions: The 11-item GRACE checklist provides guidance to help determine which observational studies of CE have used strong scientific methods and good data that are fit for purpose and merit consideration for decision making. The checklist contains a parsimonious set of elements that can be objectively assessed in published studies, and user testing shows that it can be successfully applied to studies of drugs, medical devices, and clinical and surgical interventions. Although no scoring is provided, study reports that rate relatively well across checklist items merit in-depth examination to understand applicability, effect size, and likelihood of residual bias. The current testing and validation efforts did not achieve clear discrimination between studies fit for purpose and those not, but we have identified a critical, though remediable, limitation in our approach. Not specifying a specific granular decision for evaluation, or not identifying a single study objective in reports that included more than one, left reviewers with too broad an assessment challenge. We believe that future efforts will be more successful if reviewers are asked to focus on a specific objective or question.Despite the challenges encountered in this testing, an agreed upon set of assessment elements, checklists, or score cards is critical for the maturation of this field. Substantial resources will be expended on studies of real-world effectiveness, and if the rigor of these observational assessments cannot be assessed, then the impact of the studies will be suboptimal. Similarly, agreement on key elements of quality will ensure that budgets are appropriately directed toward those elements. Given the importance of this task and the lessons learned from these extensive efforts at validation and user testing, we are optimistic about

机译：背景：尽管人们越来越需要有关比较有效性（CE）的信息，但关于观察性研究是否以及何时具有足够的质量来支持决策的争论也很多。目的：开发和测试项目清单，该清单可用于对那些在设计和执行上足够严格的观察性CE研究进行鉴定，以为决策支持的证据基础做出有意义的贡献。方法：通过文献回顾和与专业协会，付款人团体，私营部门和学术界的专家协商，制定了关于数据和方法的11项清单（GRACE清单）。由于不存在用于验证的单一金标准，因此将清单项目的响应与3种不同类型的外部质量评级进行了比较（N = 88条）。文章比较了药物，医疗器械和医疗程序的治疗效果和/或安全性。我们使用发表的观察性CE或安全性研究文章验证了针对外部质量评级的清单项目回应3种方法：（a）来自公开发表的系统评价的系统评价-质量评估; （b）根据高级研究人员征求的“专家意见”进行的单一专家审查质量评估; （c）两位专家一致的专家评审质量评估。来自五大洲的志愿者（N = 113）使用清单完成了280篇文章评估。估计单个项目的正负预测值（分别为PPV，NPV），以将测试人员的评估与专家的评估进行比较。结果：从总体上看，无论是在数据还是方法上，该量表的NPV均优于PPV。质量最一致的预测指标与用于研究目的的主要结果测量的有效性有关。其他一致的质量指标涉及使用并发比较器，通过谨慎选择协变量来最大程度地减少偏差的影响以及使用敏感性分析来测试结果的稳健性。专家对被评级物品质量的意见一致性为52％;大多数清单项目的效果都更好。结论：11个项目的GRACE清单提供了指导，以帮助确定哪些CE观察性研究使用了强有力的科学方法和良好的数据，这些数据适合用于决策目的和价值考虑。该清单包含一组简约的元素，可以在已发表的研究中客观地进行评估，并且用户测试表明，它可以成功地应用于药物，医疗设备以及临床和手术干预措施的研究。尽管没有提供评分，但研究报告指出，在检查清单项目中评分相对较高，值得深入检查，以了解适用性，影响范围和残余偏倚的可能性。当前的测试和验证工作并未在适合目的的研究与不适合目的的研究之间实现明显的区分，但是我们已经确定了我们方法的关键但可补救的局限性。没有为评估指定具体的具体决策，或者在包含多个以上报告的报告中未确定单个研究目标，这使得审阅者面临的评估挑战过于广泛。我们认为，如果要求审稿人将重点放在特定目标或问题上，未来的工作将会更加成功。尽管在测试中遇到了挑战，但一套商定的评估要素，清单或计分卡对于实现这一目标至关重要领域。大量资源将用于现实世界有效性的研究，如果无法评估这些观察评估的严格性，那么研究的效果将不是最佳的。同样，就质量的关键要素达成协议将确保预算适当地针对这些要素。鉴于这项任务的重要性以及从验证和用户测试方面的大量努力中吸取的教训，我们对

Integrated paediatric training to improve child health

摘要

著录项

相似文献

相关主题

期刊订阅