DxMap: Assessing Clinical Reasoning with Digital Concept Maps
A DxMap™ White Paper by vCases Team, July 2020
Summary
DxMap is a new type of concept map that visually displays the thought process used by a student to reach a final diagnosis when working up a virtual patient. The DxMap plots “Progress” in collecting relevant clinical findings versus student “Decisions”, producing a detailed, fully-annotated pathway taken by the student toward a diagnosis.
Figure 1. Student DxMap history-taking example #1. Hovering over (or tapping) a plot symbol reveals a pop-up describing details of each student decision.
The resulting map provides a new and innovative way to assess student diagnostic skills and learning. Differences among students in clinical reasoning and effort are easily seen, and in some cases can help determine where and why misdiagnosis occurs. This makes the DxMap a powerful new analytical tool for helping students develop their diagnostic skills.
Introduction: vCases and the Development of Diagnostic Skills
vCases® is a case-based medical simulator that is designed to help students improve their diagnosis skills. Students are self-directed in their case workups, beginning with a Patient Profile that includes the chief complaint, basic vital signs and any notable risk factors. After asking for the chief complaint, students select from hundreds of symptoms, choose physical exams to perform, and order tests. Along the way they are asked to report their differential diagnosis, narrow it as they proceed, and write a brief summary of the key findings that led to these decisions. After reaching a final diagnosis, students are presented with a personalized epilogue summarizing how their diagnostic decisions compare with those of the case author. They are also given access to a DxMap that provides an annotated visualization of their path toward a diagnosis.
On the surface, vCases provides an interactive, digital version of the traditional paper case. With paper cases, findings are described and the student uses the information to work toward reaching a diagnosis. In contrast, vCases makes the process more self-directed and realistic because the student is responsible for discovering all relevant findings through history taking, examination and testing.
vCases focuses squarely on diagnostic practice and reasoning, avoiding details of procedure. For example, vCases does not teach students how to perform an ear or eye exam; when an examination is selected in vCases, students simply receive a description of the findings that a practicing clinician might report. This encourages students to focus on the cognitive aspects of reaching a diagnosis.
Finally, vCases includes a DxLibrary® that includes a description of every disease, symptom, sign and test needed to create differential diagnosis lists. The library content focuses on diagnosis, not management, and each disease page is digitally updated during the course of a patient workup to include (in red text) a brief description of each patient finding listed for that disease. This provides a helpful aid for students when comparing various diagnoses as they proceed through the workup.
Visual Assessment of Diagnostic Skills
Medical education struggles with assessing diagnostic skills. When students use vCases, data is collected about questions asked, physical exams selected, tests ordered and DxLibrary pages accessed. This wealth of data provides a rich resource for analyzing clinical reasoning. We are developing new tools for extracting these data in ways that yield unique possibilities for understanding and improving diagnostic processing.
For each of our cases, the author maps out the risk factors, symptoms, exams and tests, including pertinent negatives, most relevant for making the correct diagnosis. This allows us to assign one of the following three relevance categories to each diagnostic choice:
- Important – These findings provide key support for the correct diagnosis and/or help in reducing the likelihood of other diagnoses in the differential
- Relevant – These findings are helpful in narrowing the differential diagnosis, but are not as critical to reaching the correct diagnosis
- Unrelated or Exploratory –These findings are not directly relevant to the differential diagnosis, but may still be significant exploratory choices for ruling out other diagnostic choices
These categories provide the basis for the DxMap, an assessment tool we have developed to help faculty quickly gauge how a student performed during a diagnostic workup. DxMap plots each diagnostic choice by the student (e.g., symptom, test) using a symbol placed along the X-axis, and locates each symbol vertically according to its diagnostic relevance. The result is a concept map of diagnostic progress (upward on vertical axis) versus investigative effort (distance along horizontal axis). A DxMap is generated for every case worked up in vCases, and faculty can examine DxMaps for each student and case to look for patterns, issues, and progress toward becoming a competent diagnostician.
Let’s look at a few examples of DxMaps to learn what they can show us about student diagnostic thinking. These are actual examples of diagnosed vCases worked up by students from a wide range of medical schools and schools for physician’s assistants or nurse practitioners. We’ll begin by revisiting our first DxMap just to point out the important parts.
Figure 1b. Student history-taking example #1 revisited to illustrate DxMap construction.
The X-axis (Decisions) is a linear scale showing the 45 history-taking choices made by this student. Green circles represent “Important” questions, blue circles represent “Relevant” questions, and yellow circles represent Unrelated or Exploratory questions. A star on a circle means the student flagged a patient finding as being important to note. The green book-stack icons show when the student looked at DxLibrary pages and indicates specific page(s) viewed. Finally, the diagonal series of white and red circles represent a hypothetical pathway toward diagnosis that could be achieved if a student asked ONLY Important questions (the white circles indicate which of these Important questions the student actually asked and the red circles represent those that were not asked). The student’s final diagnosis, along with the actual correct diagnosis, are in the upper right. Annotations for each circle and library icon show up in a white box when the cursor hovers over that symbol (or is tapped on a touchscreen device).
The relevance of student-asked questions is revealed as upward migration. Each green (Important) circle appears one (1.0) full symbol height above the previous symbol, indicating a strong diagnostic advance. Blue (Relevant) circles are also shifted upward, but only by a half (0.5) symbol height. Since yellow (Unrelated or Exploratory) symbols do not represent a clear step toward progress, they appear a half step below the previous symbol.
A student’s diagnostic progress is revealed at a glance. Figure 1b follows an upward trend with relatively few yellow (unrelated, exploratory) questions. The student was thorough, asking all but three of the Important symptom questions. Students (as well as experienced diagnosticians) are not expected to ask every Important symptom question because some may be marked Important only because they provide different approaches to narrowing the differential diagnosis.
Figure 2. A DxMap illustrating a student selecting the correct diagnosis with fewer questions asked. The annotation for this student’s DxLibrary visits can be viewed by hovering over a DxLibrary icon or tapping on it via touchscreen. The resulting pop-up box shows the pages viewed by the student (e.g., Dyschezia and Water Brash for the first DxLibrary visit).
A second student DxMap (Figure 2) illustrates a shorter pattern, trending strongly upward with mainly Important and Relevant choices. Few Unrelated (yellow) findings were explored before reaching the final correct diagnosis. This seems to demonstrate an efficient patient workup, asking significant questions and quickly narrowing the differential diagnosis. But are these few decisions sufficient to draw this conclusion with minimal risk of misdiagnosis? vCases is working on approaches to automating such an analysis, but with the DxMap a more thorough assessment would look individually at the eight Unasked Important symptoms (red) to see if perhaps the student jumped to conclusions and got the right answer without fully narrowing the differential diagnosis.
Figure 3. A meandering, exploratory workup with over 100 questions asked.
Figure 3 contrasts with the first two by appearing to involve three phases. The first phase is a seemingly random trail of symptoms, then a second phase on a solid trail of Important and Relevant questions, and finally another long sequence of unrelated items.
At first glance, such a long meandering pathway seems to indicate less thoughtful, less focused clinical reasoning. But perhaps this student was being diligent, first searching for useful findings, then hot on the trail of a hypothesis, and ending with questions to make sure no significant clues were missed.
In truth, this student simply chose Areas suggested by the History menus in vCases, choosing interesting buttons from the General, HEENT, Thorax and other Areas, one after the other, first to last. Some thought went into the choice of symptoms on each Area menu, but primarily this was a random walk through the vCases symptoms organized mainly by area of the body. The second phase that seemed so organized was simply the last part of the Thorax area questions followed by symptoms regarding the Abdomen area, where most of the Important and Relevant questions were to be found.
Figure 4. A decision path with extensive DxLibrary usage.
Finally, the student DxMap in Figure 4 shows extensive (and atypical) usage of the DxLibrary throughout the entire history-taking process. Most of this DxLibary usage involved Symptom pages, but starting around #36, several disease files pertinent to the differential diagnosis for this case were examined, including Pancreatitis (the actual diagnosis). A strong upward trend with fairly few Unrelated (yellow) questions is evident, and all but two of the Important symptom questions were asked.
Conclusion
This examination of four student DxMaps illustrating four different diagnostic approaches to the same case shows the power of DxMaps for assessing how a student thinks, the time it takes to get a sense of the most important findings, and the thoroughness of each investigation. This information provides a unique way of assessing the range of diagnostic pathways taken by self-directed students as they explore (in a safe, non-threatening, virtual environment) how to tackle patient diagnosis.
While the DxMap is useful on its own for a faculty member charged with summative assessment, we think it also provides an important avenue for one-on-one formative assessment between a faculty advisor and student, or in peer-to-peer discussions. Revisiting a case workup step-by-step allows a student to comment on and internalize what they were thinking as they performed the workup, thereby helping faculty or peer reviewers make a more thoughtful and appropriate assessment of the student’s work.