|
|
| |
|
 |
|
 |
Challenges,
Issues and Values in the Emerging Field of Outcomes in Children's Mental Health
Making outcome information usable to consumers
Consensus emerged on the issue of the importance of ensuring that outcome information is both understandable and usable to consumers of mental health services. This challenge was directed to those who construct evaluations and report outcomes achieved. The underlying premise of this issue was that information is a powerful tool that can facilitate the involvement of families and advocates. There was consensus among participants that evaluation findings are often difficult for stakeholders to use and often answer questions of interest to the evaluator rather than questions of interest to people directly affected by mental health services.Translating evaluation results into language for legislators/policy makers
Similar to the challenge raised about consumer relevancy is the issue that the results of outcome based evaluation efforts be useful to those who make policy. The participants shared experiences with evaluations that were supposed to inform policy but because of the style in which results were presented the evaluation had limited impact on policy. It was acknowledged that results are often confusing and contradictory; however, it was expressed that even confusing findings can be presented in a manner that facilitates their understanding by policy makers.Primitive level of understanding outcomes
Many participants identified the lack of research in children's mental health and the effect this has on understanding outcomes and the interrelationships among outcomes. This was seen as a particular challenge when one begins to select outcomes and the indicators that will be used to assess the status of an outcome. A call for more funding of research on child mental health outcomes was echoed by many of the professional participants. This was in contrast to the comments of family members who instead presented a common sense approach to using outcomes and indicators. Among the family participants it was stated that much research has already occurred and that what was now needed was more services and simple common sense oriented methods of tracking outcomes.Outcomes sometimes contradict each other
This issue was discussed as a general frustration with the work of interpreting outcome information. It was pointed out that even when great care is taken in selecting outcome indicators the results are often difficult or impossible to interpret.More than "our" [mental health] services are accountable for outcomes
Participants acknowledged that often one single service system alone cannot be expected to accomplish an identified outcome. The discussion focused on the need to have multiple child serving agencies work together for the shared goal of accomplishing outcomes. This issue poses a particular challenge to selecting outcomes that reflect what a particular service system is reasonably able to achieve. Examples were given which reflected the use of interagency approaches that work towards the achievement mutually agreed upon outcomes. These examples highlighted the experiences of participants who believe that interagency based outcomes and indicators facilitate interagency cooperation and more seamless services for children and families.Ambiguity/anxiety/disagreement
These three issues emerged as a triad of concern that appeared to be universally felt according to the participants. Service providers and direct service workers alike share the feeling that they will be unfairly treated as a result of outcome measures. Other participants shared that there is often ambiguity felt by those collecting information about how the information will be used or when they will hear about the results. Also, participants identified that direct service providers, managers/administrators and evaluators often disagree about which outcome indicators should be used and how results should be interpreted.Credit and blame
This issue was closely related during the discussion with the previous issue. Participants expressed that distrust occurs when the results of outcome tracking are really about blaming service providers rather than being used to improve service delivery. Also, when results were positive many participants felt that credit was often given to those least involved in the attainment of the outcomes.Value issues
The influence of values and their impact on the selection of outcome measures was discussed. Currently, participants identified that reducing out-of-home placements is valued. This, however, has not always been the case and may change again given the recent interest in developing orphanages. Participants expressed that although the children's mental health field needs more research, that this emerging knowledge alone should not drive the selection of outcomes that will be used at the local level. Here the values and issues that face local communities should play a prominent role in the outcome selection process. These local issues and values require open discussion and consideration.Issue of mandates
Participants were divided in opinion about whether outcome tracking of particular outcomes should be mandated. Some participants expressed the belief that unless outcomes are mandated, that organizations will not take them seriously and that there will be uneven utilization of outcomes for improving service delivery. Other participants believed that mandating outcomes would result in similar problems resulting from mandated service delivery rules and procedures. Here, rules are often irrelevant to local jurisdictions and are mandated nationally or across a state. In these under mandate conditions, agencies and providers are more likely to comply with the mandate rather than to truly use the information for improving services.How do we create community responsibility
Participants agreed that in the current rush to hold public agencies and providers accountable for results we as a culture are failing to hold communities accountable as well. This issue arose when participants acknowledged that public agencies alone cannot solve many of the problems that challenge children and families. The movement to create outcome accountability was generally perceived as positive but creates the risk that citizens will be shaped to believe that government can solve all of their problems.Issue of incentives
Many participants expressed that unless incentives can be built into contracts or some other means, simply expecting programs and agencies to track outcomes for the sole purpose of improving service delivery was not sufficient to sustain the process of outcome accountability. One participant shared an example of how easy it is for his agency to ask its board for funds that support additional accounting staff. Here this participant emphasized that the incentives were high since the requested funds could be directly related to dollars flowing into the organization. In contrast this participant stated that asking the same board for funds to support an evaluator in order to track outcomes was seen as a luxury since it is not tied to any requirements from either funding sources or from consumers.Extreme indicators - success should be graduated
This part of the discussion focused on the notion that achievement of outcomes should not simply be a pass-fail enterprise. Instead, it was suggested that the process of selecting goals be based on realistic possibilities while at the same time not so easily achieved that the process is rendered meaningless. Establishing baselines of indicators was introduced during this part of the discussion as a way to establish realistic goals.How do we combine indicators?
Simplicity was key to this part of the discussion. Participants acknowledged the difficulty of implementing an outcome accountability system and that even tracking a few indicators was challenging in the public arena. As a result the indicators that are selected should provide maximum information about more than one outcome. For example, participants stated that an indicator measuring school performance might be useful in tracking outcomes regarding quality of life as well as symptom reduction.Fidelity of interventions
Here participants discussed the need to ensure that the intended interventions within any particular system are being provided in the manner that they were intended to be provided. Without information about the fidelity of the interventions it is impossible to later interpret outcomes and use the results to inform service delivery.Sense of urgency
There was a sense of urgency in the discussions regarding reliable and valid tools for implementing outcome accountability. This was echoed by most of the participants because of the impending changes which are occurring in our health care systems as well as the current trend in government to downsize while holding agencies more accountable for achieving outcomes.
Return to Emerging Field of Outcomes Home Page
 |
 |
 |
CFS Home | CFS Centers & Projects | CFS Publications | CFS News | CFS Faculty & Staff | CFS Divisions Copyright © 2005, Dept. of Child & Family Studies, Louis de la Parte Florida Mental Health Institute -- see terms of use. |
|
To contact us
about this website, write us at cfswebmaster@fmhi.usf.edu To correspond with employees of the department, write to them care of: The Department of Child and Family Studies Louis de la Parte Florida Mental Health Institute University of South Florida 13301 Bruce B. Downs Blvd. Tampa, FL 33612-3807 |