Monitoring and evaluation play a key role in developing a community based system of care. As with parent engagement, robust evaluation processes, particularly around the area of impact on the logic model, were something participants in the NRCA focus groups stated were lacking in their states. However, a number of backbone organizations were reporting on service outputs and outcomes and utilizing satisfaction surveys. The more mature post-
permanency systems are collecting data on outputs, such as numbers of parent liaisons and mentors, types of trainings available to adoptive parents/guardians, and call and referral volume to hotlines.
Managing and designing evaluation systems can be time consuming and further challenged by the lack of integrated data management systems. But, evaluation of post-permanency programs can particularly impact the possibility of future funding as well as the realignment public funding. Monitoring and evaluation is a powerful tool that can answer the “so what” questions and respond to the growing demands for results. By focusing on outcomes, and not merely outputs, evaluation helps analyze why intended results were achieved and to identify the contributing factors.
Specifically, longer-term outcomes could include the following:
- Prevention of adoption disruptions/Increased family stability
- Increase in school stability and educational attainment of children
- Improved quality of life for adoptive families and youth (e.g., less behavioral problems, less familial stress, greater emotional security of youth).
Perhaps a significant reason for evaluation is the increased awareness for measuring Social Return on Investment (SROI) (Cabinet Office of the Third Sector, 2009). Measuring SROI is mainly determined by assessing the costs and benefits of a particular intervention against the costs and benefits of a counterfactual, or essentially the status quo. The more longitudinal the benefits are when compared to short-term costs, the greater the SROI.
Perhaps the power of measuring results is best summarized as follows (Osborne & Gaebler, 1992):
- If you do not measure results, you cannot tell success from failure
- If you cannot see success, you cannot reward it
- If you cannot reward success, you are probably rewarding failure
- If you cannot see success, you cannot learn from it
- If you cannot recognize failure, you cannot correct it
- If you can demonstrate results, you can win public support
From a post-permanency perspective, the cost of services and supports that are needed once an adoption/guardianship case has dissolved should be compared to the cost of services that are effective in preventing one such dissolution. Ultimately, evaluation and monitoring also provides for a powerful argument to help create additional funding opportunities for post permanency services. It is not enough to simply implement post permanency services because parents and adoptive children may need them. One must also examine the outcomes and impacts of such services and how critical they are to supporting the larger goals of a child welfare system.