|
|
| |
|
 |
|
 |
The
Outcome Information Components
Who is Being Served: Child and Family Population Characteristics
The Ecology framework suggests that two broad categories of information about children and families will be useful in service planning and delivery: (1) information about children and families that makes it possible to determine whether the system serves the children and families it intended to serve; and (2) other information about child and family characteristics that may influence the system's outcomes.
Fundamentally, service providers and other stakeholders need to know that the populations they intend to serve are, in fact, being served. For example, mental health agencies in counties in California specifically target children at risk of out-of-home placements. As a result, it is imperative that the information systems supporting these county efforts be able to establish whether children at risk of or in out-of-home placement are, in fact, being served (Jordan and Hernandez, 1990). A system which fails to serve its intended population cannot adequately assess its outcomes. For example, if children not actually at risk of out-of-home placement represented the majority of children in a county system's caseload, then a positive outcome reflecting fewer out-of-home placements would be misleading as an indicator of positive service system functioning.
In addition to information about target population characteristics, information about other child and family characteristics can be useful in the interpretation of what may have influenced an achieved outcome. Burns (in press) provides a list of suggested child and family characteristics which may influence outcomes. These include risk factors such as poverty, family history of mental illness; illness severity, chronicity and co-morbidity; family strengths and tolerance of stress; social support; family member's case management skills; and treatment adherence by family members and therapists. Generally, these characteristics alone or in combination with each other can contribute to either failure or success in achieving outcomes. Combining a limited number of carefully chosen child and family characteristics with information about whether target populations have been served can greatly enhance a system's ability to interpret its outcomes more confidently.
This strategy of combining information about whether the target population receives services with information about child and family characteristics relates directly to the Ecology framework's principle that outcome information should be used to improve service planning and direct service delivery by allowing for more plausible assessments of what is contributing to the status of an outcome (Dym, 1996). This approach ensures more effective utilization of outcome information. If outcome information suggests that services have not been successful for the targeted population, the additional information provided by other child and family characteristics can be useful in uncovering patterns of outcomes for sub-groups within the target population. For instance, if a juvenile justice program shows a sudden increase in re-arrests among the target population, an analysis of other child and family characteristics can help determine if the services are successful or unsuccessful for a particular sub-grouping of the targeted population. Interpreting outcome results with additional child and family characteristics can lead to a service planning process that better meets the needs of sub-groups within a target population. Making sure that sufficient information is collected about a variety of child and family characteristics is critical to later being able to interpret outcomes.
What Services Have Been Provided?
The theory of change embraced by the child-serving system underlies the task of choosing what services are included in a system's service array (Weiss, 1995). It is imperative to clarify the theory of change which is embraced by the service design so that particular characteristics of the services provided can be monitored and tracked. For example, if home-based treatment or therapeutic foster care programs are part of a service strategy, then selecting particular service characteristics that best represent the two approaches is critical to later knowing whether the two services were actually provided. Further, a service system may choose to add or change its service array incrementally so it can adequately test which service approach is most effective in producing expected outcomes. Ultimately, what is being tested is not simply a particular program but the theory of change guiding the service approach.
The Ecology framework assumes that treatment is a process rather than a single event (Burns, in press ). The framework suggests that four service aspects are useful for tracking and monitoring services as well as interpreting outcome information later (Hernandez and Goldman, 1996). These are: (1) intensity, frequency and duration of services; (2) location of services; (3) variety and sequencing of services; and, (4) integrity of services.
The first of these service aspects describes how much of a service is being provided. The two most obvious elements to track in this regard are the frequency with which services are offered and the duration of each contact with a family or child. The second service aspect describes the setting where services are offered. This characteristic of service is important since many emerging models of service delivery focus on in-home services or on services offered in schools, foster homes and juvenile justice facilities (Stroul, 1996).
The third aspect captures the variety of services that are offered to a child and family and the sequencing of those services. Again, emerging approaches to individualizing care stress the importance of offering children and families whatever is necessary to ameliorate a condition or to support a particular family strength. As a result of this service approach, a great variety of services and supports can be provided to a particular family in any sequence and combination (Rosenblatt, 1996). The fourth service aspect involves the integrity or the quality and fidelity of the services provided. Here it is important to know whether the service that has been provided regardless of any of the other aspects is offered in a competent manner, and if the service is consistent with the values of the particular service system. For instance, if a system is family centered, then the system needs to know whether families participate in service planning in a meaningful way.
Together, the four service aspects provide a menu of choices that should be considered when selecting specific service characteristics tracked in an information system. The eventual choices of what characteristics are focused on should reflect the service approach and the underlying theory of change. Similarly, Burns (in press) articulates service aspects that are important when considering what aspects should be tracked by an information system. These include the mix of treatment components at any given period of time and the sequence of services over time, the type and stage of treatment and the developmental age of the child. In addition, Usher (1993 a, b) has helped create strategies to track service characteristics in his self-evaluation work with child welfare organizations. One purpose of Usher's approach is to evaluate whether the service system actually provides the services it intended to provide. For example, if a service delivery model includes the provision that service providers are expected to deliver 90 percent of all services in a family's home, then it would be critical to have a method to document whether this has occurred. This approach requires that organizations articulate in operational terms what services they expect to offer. This operationalization and tracking of service aspects gives service systems confidence that the results of their efforts can be more plausibly related to the type of services they provide (Dym, 1996).
It should be noted that the complexity of tracking services is complicated by the fact that program effectiveness is often contingent and interdependent on the availability, accessibility and quality of other community services (Weiss and Greene, 1992). Burns (in press) suggests that it may not be sufficient to assess mental health services alone, but that inclusion of information about other sectors and informal supports is also essential. Currently, interagency approaches are attempting to collaborate and to co-locate staff in efforts to be more efficient and effective (Stroul, 1996). Knowing about other agency services is important when one considers that funding for services in one agency may influence the effectiveness of another agency's programs.
Measuring Change: What have we accomplished?
Within the Ecology framework, outcome information is used as a measure of what the system has accomplished. It should be emphasized that the Ecology framework stresses the use of outcomes in the context of managerial needs rather than for purposes of generalizability and application to larger social contexts.
It is necessary to ask certain questions and apply specific criteria to the selection of outcomes to be measured. The Ecology framework offers a series of questions to guide the selection of outcomes. These are summarized in Table 2.
The Ecology framework suggests the identification of outcomes from several domains. These domains borrow from the work of Rosenblatt and Attkisson (1993) which developed a framework for assessing services provided to populations with severe and substantial mental health impairments. The Rosenblatt and Attkisson topology identifies five outcome categories: clinical status, functional status, life satisfaction and fulfillment, welfare and safety, and satisfaction with services. Within this framework, the sources of outcome information are interpreted through four social contexts of an individual's experience: the personal life of an individual, the family life of an individual, the work and school life of an individual, and the individual's life within the community. The Ecology model considers this conceptual framework for outcome measurement to be the most comprehensive to date. It is instructive because it recognizes that severe mental impairment impacts a wide range of social activities including family life, school and work, and life within the community. The Ecology framework expands Rosenblatt and Attkisson's concepts of satisfaction beyond the issue of client or family satisfaction with services. In addition to the commonly measured outcome of client satisfaction, the Ecology framework recommends the addition of provider satisfaction and stakeholder satisfaction in the array of outcomes being tracked.
Once outcome domains have been selected, a second layer of decision making has to occur in order to select the indicators. That is, decisions must be made as to what indicators will be used to measure the outcomes and what criteria should be applied in making the selection. The Ecology framework offers several questions in Table 3 which may be used in selecting the indicators.
The selection of the best and most appropriate indicators for any designated outcome is critical since the collection of data related to outcome indicators involves time and personnel, and will become a significant public representation of what a system is accomplishing. In keeping with the Ecology framework's principles, the process of selecting indicators should be informed by the best and most timely research available. The process of identifying indicators involves the "disaggregating" or "unbundling" of each outcome into a set of measures that provide a reasonable measure of the achievement of the outcome (Center for the Study of Social Policy, 1995). The American Managed Behavioral Healthcare Association's Performance Measures for Managed Behavioral Healthcare Programs (1995) recommends that the performance indicators selected must be manageable. This recommendation was guided by the knowledge that collecting meaningful data across large populations is a challenging task even if only a few indicators are measured.
Return to The Ecology of Outcomes Home Page
 |
 |
 |
CFS Home | CFS Centers & Projects | CFS Publications | CFS News | CFS Faculty & Staff | CFS Divisions Copyright © 2005, Dept. of Child & Family Studies, Louis de la Parte Florida Mental Health Institute -- see terms of use. |
|
To contact us
about this website, write us at cfswebmaster@fmhi.usf.edu To correspond with employees of the department, write to them care of: The Department of Child and Family Studies Louis de la Parte Florida Mental Health Institute University of South Florida 13301 Bruce B. Downs Blvd. Tampa, FL 33612-3807 |