Natural Language Understanding of Clinical Practice Guidelines PI: Daniel R. Schlegel, PhD Department of Computer Science, SUNY Oswego Collaborators Mor Peleg, PhD University of Haifa, Israel Carmelo Gaudioso , MD PhD Roswell Park Cancer Institute, Buffalo NY We present a framework for automatic natural language understanding of clinical practice guideline (CPG) text called Clinical Tractor . Introduction NLU with Clinical Tractor Re imagining the Tractor 1 natural language understanding system for the clinical domain Previously applied to the counter insurgency domain short intelligence messages Standardized XML input using a custom format combining JATS and GraphML Rule based syntax to semantics transformation after text processing and importing background knowledge Figure 1. Overall system architecture for a guideline understanding and formalization system. 1. Use off the shelf tools and data for the standard NLP pipeline Make customizations as needed for the domain 2. Map (complete) Syntax > Semantics Use a set of rules, applied in a specific order, to gradually build the semantic representation from the syntactic one. Be as general as possible Avoid specific cases wherever possible Make judicious use of background knowledge to maintain generality needlessly 1. Stuart C. Shapiro and Daniel R. Schlegel. Natural language understanding for soft information fusion. In Proceedings of Fusion 2013 . IFIP, July 2013. 9 pages. 2. Schlegel, Daniel R., et al. Clinical Tractor: A Framework for Natural Language Understanding of Clinical Practice Guidelines. AMIA Annual Symposium 2019. 3. Daniel R. Schlegel and Stuart C. Shapiro. Inference graphs: Combining natural deduction and subsumption inference in a concurrent reasoner. In Proceedings of AAAI 15 , 2015. 4. American Diabetes Association. Standards of Medical Care in Diabetes. Diabetes Care , 40(suppl 1), 2017. References Design Goals Figure 4. Clinical Tractor continuous evaluation framework. Figure 3 . Patients found to have elevated blood pressure should have blood pressure confirmed on a separate day. Patients found to have elevated blood pressure Continuous Evaluation 2 Students Rose Fontana Kate Gordon Adrian Naaktgeboren Michal Patriak Context: Automatic Computer Interpretable Guideline Generation We have adopted a model of continuous evaluation, where each new rule developed for the system is rigorously tested for precision, and integrated only when it is deemed satisfactory. An answer key is iteratively developed by marking each rule firing as good or bad. When a rule is modified only minor changes are needed of the answer key. When a rule is added only the new items need to be added; regression testing is straightforward. Recall can be evaluated by having a human produce a complete answer key independent of the system, and feeding it into the evaluator. Figure 2. Clinical Tractor 2 system architecture. 3 This work was supported by the National Library Of Medicine of the National Institutes of Health under Award Number R15LM0130 30. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National I nstitutes of Health.