In our work with Academic Medical Centers we have engaged in in-depth study of a number of offices, clinics, departments, hospitals, and insurance companies. Such an exploration involves data analysis on multiple levels. We always consider the financial reports generated by these units which feed into larger cost accounting and billing systems for the hospital or corporation as a whole. Frankly, this data is rarely of much interest for multiple reasons. First, revenue generation is largely outside the control of most of the decision makers that we are working with in the unit. As a result, consideration of this information produces surprisingly little insight relating to actionable items.
We also consult many of the information systems in place in such units next. This involves data from billing systems, appointment systems, Electronic Medical Records (EMR), staffing systems, and a bed or facility management system. It seems safe to say that this is the rather standard approach. Almost all of the published research in scholarly journals available on these systems builds on these data sets. One advantage of health care research is that mountains of data are available. One common approach is to get ahold of a huge dataset based on thousands or even millions of cases. This immediately suggests the use of tools like linear regression by identifying independent and dependent variables and assuming some functional form for the connection between them. If the data set is large enough, virtually all findings will be “statistically significant.” This happens because the variance implied in such models falls linearly with the square root of the number of observations. Thus, big data sets, always appear to have something interesting to tell us.
Direct Observation
While data-mining common systems is fine as far as it goes – we have found that it does not go nearly as far as most people seem to think for one major reason – a significant portion of this data is wrong. We know this because we can compare this data with data from other sources to see the errors and omissions. One way to collect the needed information is through the old-fashioned use of paper forms. We have attached a form to each patient record as it is pulled at patient check-in. We then have the front desk indicate check-in and activity times at this step. We have a nurse record when the patient is taken to the examination room and vital information is collected. The nurse can record entry and exit from the room. The attending physician (Attending) can do the same thing. Finally, the front desk can record the start and end of the Check-out step. This is a very low-tech approach that works surprisingly well with low volume settings.
Alternatively, we have also use paid observers to collect this information. (You may wish to look over one of our YouTube videos from more information at VideoOne.) This is often far superior because the observer can be trained to look for unusual things and nuances of agent behavior that are not recorded (intentionally or otherwise) in the paper data collection efforts. This works great for professors because we have access to a mountain of low-cost laborers. However, this is often impractical in other settings.
Another approach involves the use of a Real-Time Location System (RTLS). Many readers are surprised to learn that hospitals often have an RTLS built into the structure itself. This involves the use of sensors near doorways. This can be combined with badges that trigger the sensors. Lots of facilities place tags on major pieces if equipment to track them throughout a hospital or complex. This can also be done with staff, but they have to be gently coaxed into working with us in this way. When this is accomplished, we can get real time data on the movement of nurses, doctors, and others throughout a clinic environment. This is particularly useful for settings that span multiple areas or floors as is common with Radiation Oncology units. This also presents a sort of “gold standard” in terms of data collection. It eliminates most human errors and does not pre-suppose any standard flow. Instead, it reflects actual times for the steps involved.
Take Away’s
The major message here is that when we say that the data typically drawn from the IT systems is inferior, we are not saying what we read or what we suspect – we are simply stating what we know and can demonstrate repeatedly. Again, the main point is that direct observation (in one form or another) is invaluable to work on real systems, and pulling data from existing systems and report forms is a poor substitute.
More details on how we collect data and use it within a larger healthcare process improvement project is detailed in our works available on our Documents page and our text at, Amazon.com.