As an advocate of outcomes data, I am often asked… ”Why don’t treatment programs collect more data?” I cannot pretend to answer for all of my colleagues across the world, but I can answer from my experience. First and foremost, there has been widespread disagreement over what needs to be measured and on how we define, or operationalize, the broad construct of recovery. Second, there has been disagreement on what measures to use. Well-meaning individuals have attempted to develop their own, yet many of these are absent of the psychometric standardization that is necessary to have empirical rigor. And lastly, the daily pace of treatment, often coupled with tight budgets and limited resources, prevents meaningful outcomes measurement from taking place.
Let me be clear on my experience through this statement. There is not a sinister movement industry-wide to avoid accountability or to stifle progress due to some imagined dogma. This is not true of BRC Healthcare and has not been my experience in any of the places I have worked prior to my arrival here. Rather, the difficulties in achieving outcomes data are based in the prioritization of time, resources, and energies of staff that work long hours and genuinely want the best for the clients they serve. When we present a package of tools with demonstrable meaning, buy-in from ownership, staff, clients and their families follow suit in short order. The key is giving people something that helps them do their jobs better and something that has return on time and money investments. The question for me is not “Why are we not collecting outcomes data?” Instead the question is, “How do we collect data that matters and analyze it meaningfully for the people we serve (this includes our staff by the way)?”
My initial work with BRC was in the outcomes area before it grew into a broader role. Marsha Stone had a set of data she had been collecting and just didn’t know what to do with it. So we started working on it. I was lucky that she saw the importance of outcomes and that we had a point from which to start. I am even luckier that this continues to be a priority for the company at this writing. If your facility doesn’t yet have a starting point, that’s ok. Clean slates can be very helpful!
Here are some tips from our efforts at BRC and a brief guideline to what we are doing. I hope these tidbits can help you create a plan to start or improve the data collected at your facility or practice.
1. Look at what your company already is or is not doing. Prior efforts should not go to waste. Can you publish a journal article from it—maybe not, but that’s ok. What CAN you do with it to improve your services? It may not meet the definition of outcomes data, but that doesn’t mean that it’s not useful. A good deal of metrics are helpful to programmatic and staffing decisions (e.g., rates of successful completion of your program, utilization of authorized services, and length of stay).
2. Organize what information you have. Who is collecting this information and is it done systematically in a way that assures you are collecting it for all of your clients? Can you put the information on a scale that makes it easier to analyze it across client types, counselor caseloads, diagnoses, programs, or other factors?
3. Consider what you are measuring and will it meet empirical standards (if you want to collect outcomes data). Are you using psychometrically normed measures? Are you using more than one measure? Are you measuring something besides simply abstinence? These are critical questions when it comes to outcomes. Creating your own measure (without standardizing it) generally leads to disappointing results for treatment providers. There are many good measures which are publicly available and can save you many headaches when you start publishing outcomes.
4. Measure the good and the bad. Be willing to look at both ends of the spectrum. If you look at only pathology scores, then you are missing half of the story. If you look at only the things you are doing well, then you miss the opportunity for improvement.
5. Measure frequently and as long as possible. Collect data beginning at admission and throughout the treatment stay. If you don’t know what is happening as you are making treatment interventions, they you can’t tell what is working across duration and frequency of service delivery. You also have to measure once a client leaves treatment. Most individuals can work their program of recovery in a treatment center, but they can’t live there for the rest of their life. If you don’t know what happens after they leave, you cannot determine if you made an impact.
6. Decide where you want to go. Is your company willing to invest real dollars in improving your outcomes data efforts? If not, are you using the resources already allocated smartly? Don’t get discouraged, you can start somewhere. Outcomes data does not have to break the bank.
7. Consider technology. Technology is your friend with outcomes data collection. Many EHR systems have some outcomes tools built into their software. However, be an educated consumer of these platforms as not all of them use normed measures and many of them make it difficult to export data in formats that help with analysis. Technology has also been slow to elicit more accurate data than face-to-face measurement because the client feels he/she/they can be more honest. It also allows for companies to get follow-up measures in an easier and more cost-effective method.
8. Consider outsourcing for analysis. There are companies that will store, clean, and analyze your data for less that the cost of adding an internal employee. It reduces bias in data analysis and limits the possibility of someone skewing the data due to personal feelings (not fool proof, but it helps)
My best advice is to start small and dream big. When I started at BRC, we set out a plan. Since that time, we have moved from pen and paper data collection at baseline, discharge, and 90-day follow-up to a fully automated system called Trac9. It runs off tablet devices inside of our facilities and off email services (with contingency management) for all of our follow-up data. BRC Healthcare collects data weekly while the clients are in treatment and then monthly post-discharge. All of our measures are psychometrically normed and present in multiple literatures, including the substance abuse and mental health care literature. We look at change across time on a weekly basis and have found that our clinical team really loves having this information to inform their strategies. Outcomes don’t have to be cumbersome or just “one more thing to get done”. They also don’t have to be perfect to be meaningful. I hope that each of you will take a moment and get started on your outcomes planning. I promise it can actually be fun!
Written by: Mandy Baker, MS, LCDC