Making guidelines computable

Abstract

Guideline development is easy and efficient. With instant access to all the contributing information—all the relevant evidence, critical appraisal of the evidence by the community, values and preferences of public representatives, judgements by multidisciplinary experts, and re-usable data where others have developed recommendations for similar decisions—…. Wait. It's 2024, not 2042. Let's try that again. Guideline development is difficult and resource-intensive. Even when the decision-making process works well, there is so much work involved to gather the evidence, assess the certainty of the evidence, determine the relative importance of the outcomes and consider contextual factors. It is sometimes easier if we can adapt from others who have already done it, but their work is not fitting what we need, so we essentially recreate the work, using our development methods anyway. Some aspects of guideline development are necessarily difficult and should not be oversimplified, but there are many opportunities to reduce the work involved. For example, automating tasks that do not require human cognition, such as identifying direct links to supporting information, can greatly improve work efficiency. To realize this potential, the guideline development content will need to be available in a form the computer can process. Computers could make guideline development more efficient. They already do, to some degree. We copy and paste instead of retyping when we can. We use autocomplete features to enter data when the machine can guess what we want to express, or dropdown lists when the choices are preset for us. We have come to expect massive increases in efficiency at times, such as rapid responses for targeted searching in large databases. Compare that to literature searching before the Internet. But the essence of our work—understanding the evidence and judgements sufficiently to select information and use it for informing our decisions—is not grasped by the computer. We may try to apply artificial intelligence (AI) to the challenge and occasionally show a tool helps a step in the process (e.g., highlighting population, intervention, and outcome terms in the text),1 but we have yet to create an AI that understands evidence and judgements. Imagine if we could make the evidence and judgements computable (i.e., machine-interpretable) so that the computer could create derivative concepts through calculations and logical operations. Searches would be even more efficient. Compare the precision searching for a nearby restaurant when you are travelling to finding evidence for a specific clinical outcome. You can find not only the restaurant's name but also its location, hours of operation and a link to its menu. However, if you find an article that mentions the clinical outcome in the abstract, you still need to obtain the full text, read it to extract the data and make many judgements to determine the certainty of the reported finding. The restaurant data are machine-interpretable, but the outcome data are not. Efficiency would be further increased if the knowledge (evidence and judgements) were interoperable, so any computer system could use the output (reuse the work) of any other computer system. Today, a systematic reviewer and guideline developer that use reference management software for citation management, PICO Portal for screening of articles, Robot Reviewer for assistance in risk of bias assessment, the Systematic Review Data Repository (SRDR+) for reporting data extraction, Cochrane RevMan for the meta-analyses and GRADEpro or MAGICapp for the reporting of summary of findings will need to re-enter the data for each of these systems. We enjoy extreme efficiency for navigation support due to societal evolution to ubiquitous computable forms of data exchange (see Figure 1) but have yet to achieve this state for evidence and guidelines (see Figure 2). The Guidelines International Network Technology Working Group (GINTech) had a goal i