Associate Professor of Biomedical Informatics

Associate Professor of Intelligent Systems

Associate Professor of Pathology

Director of Biomedical Informatics Training Program

Contact Info:

Research Interests

Crowley has two active areas of research. In work on Intelligent Medical Training Systems, she has developed SlideTutor (http://slidetutor.upmc.edu) – a unique performance training system. Work in the SlideTutor group focuses on empirical studies of medical expertise, student and user modeling and exploration and evaluation of potential interventions. In work on text processing, information extraction and multi-institutional data sharing, she has developed caTIES (http://caties.cabig.upmc.edu) – a system for establishing large volume repositories of concept coded, and de-identified pathology reports, that supports sharing of de-identified information across institutions. In a new project in collaboration with the National Center for Biomedical Ontology (http://www.bioontology.org/), she is developing ODIE – a toolkit for using ontologies to extract information from free-text clinical documents and using free-text clinical documents to enrich existing ontologies.

Education

• 1994 University of Pittsburgh School of Medicine, MD (Cum Laude)
• 2001 University of Pittsburgh School of Information Sciences, MSIS - Medical Informatics Track

Publications

Payne VL, Medvedeva O, Legowski E, Castine M, Tseytlin E, Jukic D, Crowley RS. Effect of a limited-enforcement intelligent tutoring system in dermatopathology on student errors, goals and solution paths. Accepted to Artificial Intelligence in Medicine, August 2009

Manion FJ, Robbins RJ, Weems WA, Crowley RS. Security and privacy requirements for a multi-institutional cancer research data grid: an interview-based study. BMC Med Inform Decis Mak. 2009 Jun 15;9(1):31. PMCID: PMC2528049

Payne VL, Crowley RS. Assessing the Use of Cognitive Heuristic Representativeness in Clinical Reasoning. AMIA Annu Symp Proc. 2008 Nov 6:571-5 PMCID: 2656076

Bartos CE, Butler BS, Penrod LE, Fridsma DB, Crowley RS. Negative CPOE Attitudes Correlate with Diminished Power in the Workplace. AMIA Annu Symp Proc. 2008 Nov 6:36-40. PMCID: 18998842

Medvedeva OP, Crowley RS. The Slide Tutor Project Training Physicians with an Intelligent Tutoring System. In: Intelligent Tutoring Systems, Past and Present. Montreal, Canada: Proceeding of the Demonstration Program; June 23-27, 2008: 61-64

Bartos CE, Fridsma DB, Butler BS, Penrod LE, Becich MJ, Crowley RS. Development of an instrument for measuring clinicians’ power perceptions in the workplace. J Biomed Inform. 2008 Mar 4. PMID: 18375189

Yudelson MV, Medvedeva OP, and Crowley RS. Stepwise Multifactor Approach to Student Model Evaluation in a Complex Cognitive Domain. User Model User-Adapt Inter. 2008, Sep 18 (4):315-382

Mohanty SK, Parwani AV, Crowley RS, Winters S, Becich MJ. The Importance of Pathology Informatics in Translational Research. Adv Anat Pathol. 2007 Sep;14(5):320-322. PMID:17717431 [PubMed - indexed for MEDLINE]

Liu K, Chapman W, Hwa R, Crowley RS. Heuristic sample selection to minimize reference standard training set for a part-of-speech tagger. J Am Med Inform Assoc. 2007 Sep-Oct; 14 (5):641-50. PMCID:PMC1975798

Saadawi GM, Tseytin E, Legowski E, Jukic D, Castine M, Crowley RS. A natural language intelligent tutoring system for training pathologists: implementation and evaluation. Adv Health Sci Educ Theory Pract. 2007 Oct 13; [Epub ahead of print]. PMID: 17934789 [PubMed - as supplied by publisher]

Crowley RS, Legowski E, Medvedeva, OM, Tseytlin E, Roh E and Jukic D. Evaluation of an Intelligent Tutoring System in Pathology: Effects of External Representation on Performance Gains, Metacognition, and Acceptance. J Am Med Inform Assoc. 2007 14(2): 182-190, PMCID: PMC2213473

Drake TA, Braun J, Marchevsky A, Kohane IS, Fletcher C, Chueh H, Beckwith B, Berkowicz D, Kuo F, Zeng QT, Balis U, Holzbach A, McMurry A, Gee CE, McDonald CJ, Schadow G, Davis M, Hattab EM, Blevins L, Hook J, Becich M, Crowley RS, Taube SE, Berman J; A system for sharing routine surgical pathology specimens across institutions: the Shared Pathology Informatics Network. Hum Pathol. 2007 Aug; 38(8): 1212-25 PMID: 17490722 [PubMed - indexed for MEDLINE]

McKibbon KA, Fridsma DB, Crowley RS. Primary care physicians’ attitudes toward risk and uncertainty affect use of electronic information resources. J Med Libr Assoc. 2007; 95(2): 138-146, PMCID: PMC1852627

Yudelson MV, Medvedeva O, Legowski E, Castine M, Jukic D, Crowley RS. Mining student learning data to develop high lever pedagogic strategy in a medical ITS. Educ Data Mining Workshop - Proc AAAI 2006 Educational Data Mining 21st Nat’l Conf Artificial Intell. Boston, MA. Technical Report WS-06-05. AAAI press: 82-90

Tobias J, Chilukuri R, Komatsoulis GA, Mohanty S, Sioutos N, Warzel DB, Wright LW, Crowley RS. The CAP cancer protocols - a case study of caCORE based data standards implementation to integrate with the Cancer Biomedical Informatics Grid. BMC Med Inform Decis Mak. 2006 Jun 20; 6:25, PMCID: PMC1524939

Crowley RS, Medvedeva O. An intelligent tutoring system for visual classification problem solving. Artifil Intell Med, 2006 Jan; 36(1): 85-117.PMID: 16098717 [PubMed - indexed for MEDLINE]

Medvedeva O, Chavan G, and Crowley RS. A data collection framework for capturing ITS data based on an agent communication standard. Proc AAAI 2005, 20th Annual Meeting, Pittsburgh, Pennsylvania, Technical Report WS-05-02, AAAI Press 2005: 23-30.

Saadawi GM, Legowski E, Medvedeva O, Chavan G, and Crowley RS. A method for automated detection of usability problems from client user interface events. Proc AMIA Symp 2005, 654-658, PMCID: PMC1560804

Liu K, Mitchell KJ, Chapman WW, and Crowley RS. Automating tissue bank annotation from pathology reports - comparison to a gold standard expert annotation set. Proc AMIA Symp, 2005 460-464, PMCID: PMC1560700

Crowley RS, Tseytlin E, Jukic D. ReportTutor - an intelligent tutoring system that uses a natural language interface. Proc AMIA Symp 2005, 171-175, PMCID: PMC1560511

Crowley RS, Legowski E, Medvedeva O, Tseytlin E. An ITS for medical classification problem-solving: Effects of tutoring and representations. Proc AIED 2005 12th International Conference, Amsterdam, The Netherlands.

Mitchell KJ, Becich MJ, Berman, JJ, Chapman WW, Gilbertson J, Gupta D, Harrison J, Legowski E, and Crowley RS. Implementation and Evaluation of a Negation Tagger in a Pipeline-based System for Information Extraction from Pathology Reports. Proc MEDINFO 2004; 11(Pt 1): 663-667. PMID:117. PMID: 15360896 [PubMed - indexed for MEDLINE]

Crowley RS, Medvedeva OP. A General Architecture for Intelligent Tutoring of Diagnostic Classification Problem Solving. Proc AMIA Symp, 2003: 185-189, PMCID: PMC1479898

Blechner M, Monaco V, Knoz I, Crowley RS. Using Contextual Design to Identify Potential Innovations for Problem Based Learning. Proc AMIA Symp, 2003: 91-95, PMCID: PMC1479946

Crowley RS, Medvedeva O, Jukic D. SlideTutor - A model-tracing Intelligent Tutoring System for teaching microscopic diagnosis. IOS Press: Proc 2003 11th International Conference on Artificial Intelligence in Education. Sydney, Australia, pp 157-164.

Crowley RS, Naus GJ, Stewart J, Friedman CP. Development of Visual Diagnostic Expertise in Pathology - An Information Processing Study. J Am Med Inform Assoc 2003 10(1):39-51, PMCID: PMC150358

Crowley RS, Naus GJ, and Friedman CP. Development of visual diagnostic expertise in Pathology. Proc AMIA Symp 2001: 125-129. PMID:11825167 [PubMed - indexed for MEDLINE]

Crowley RS, Gadd CS, Naus G, Becich M, Lowe HJ. Defining the role of anatomic pathology images in the multimedia electronic medical record—a preliminary report. Proc AMIA Symp 2000: 161-165. PMID:11079865 [PubMed - indexed for MEDLINE]

Grants

2 R01 LM 007891-05
Title: Computational Methods for Personalized and Adaptive Cognitive Training
Role/effort: PI Years inclusive: 2009-2011

This is a competitive renewal of our previous NLM grant to develop medical tutoring systems. This grant proposes to leverage the SlideTutor infrastructure developed during the previous funding period to focus on three foundational areas where there is nearly no previous research in medical domains: metacognition, performance prediction, and learning behaviors. Research will include development of computational models for classification and prediction as well as pedagogic interventions that are based on information from these models.

R01 CA127979-01
Title: The ODIE toolkit software for information extraction and ontology development
Role/effort: PI Years inclusive: 2007-2011 Source: NCI

This collaborative R01 with the National center for Biomedical Ontology proposes (1) to develop and evaluate software for information extraction from clinical text corpora using existing Open Biomedical Ontologies (OBO) and (2) to develop and evaluate software for enrichment of existing biomedical ontologies from clinical text corpora. A key deliverable for this work will be the development of the Ontology Development and Information Extraction Toolkit (ODIE) – a set of software components integrated with GATE, Protégé and LexGrid. ODIE will assist researchers and ontology developers in performing these tasks. As a test bed for our work, we will focus mainly on the National Cancer Institute Thesaurus – an existing OBO ontology.

R25 CA101959-01
Title: The Cancer Training Web — A Multi-Institutional Tutoring System in Pathology
Role/effort: PI Years inclusive: 2007-2013 Source: NCI

This competitive renewal of our NCI R25 grant will fund the development of distributed training system in pathology between the University of Pittsburgh and University of Pennsylvania. This grant will fund the development of authoring systems, inclusion of training systems into workflow, modification of the existing system to utilize Semantic Web technologies, and deployment of the system to end-users.

1U18 HS016657-02
Title: Reducing errors in the diagnosis of melanoma using an intelligent tutoring systems
Role/effort: Co-I Years inclusive: 2006-2008 Source: AHRQ

This two year grant will test the cognitive tutoring system approach as a mechanism for decreasing Pathology errors. As co-Investigator, I lead the team adapting our software for this evaluation of the technology.

1R21 CA127507-01
Title: A Standards-based tool for Clinical Trials Protocol Authoring
Role/effort: Co-I Years Inclusive: 2007-2009 Source: NCRR/NIH

This project aims to develop a standard and structured format for clinical trials protocol files. This entails the development of a standard representation for registration and eligibility criteria of enrolling patients, and a semi-automated tool for converting existing text-based clinical protocol files (e.g. .doc, .pdf) to the standard representation. As co-investigator, I am assisting in all phases of this project.

Lab Personnel

Graduate Students:

Kaihong Liu
Velma Payne
Zach Landis-Lewis

Research Staff:

Melissa Castine - Knowledge Engineer
Elizabeth Legowski - Research Assistant
Kayse Gearhart - Research Assistant
Karma Lisa Edwards - Communications Specialist

Developers:

Girish Chavan
Olga Medvedeva
Kevin Mitchell
Eugene Tseytlin
Yining Zhao