Creating the Evidence-Based Practice List
There are dozens of Web sites where one can find lists of promising and proven practices for reducing delinquency, drug use and crime and violence among youth. The question is – among these sites, where does one look for proof of effectiveness and the standards against which it is to be judged? Even after the effectiveness of a program or strategy is well established, questions remain as to implementation and ease of adoption by others. Where does one go for that kind of information?
The most recent reviews, meta-analyses, lists and cost-benefit analyses provide a variety of perspectives and wealth of information regarding what does and does not work in reducing youth crime and violence and delinquency. At the very top of the pyramid is a small group of rigorously evaluated brand name programs that have consistently demonstrated significant positive effects and a number of strategies that have been determined, through meta-analysis, to cause a significant reduction in recidivism on average. At the bottom are programs and strategies that have been evaluated but have proven to have no effects or to have adverse effects. In the middle are brand name programs and strategies for which there is some scientific evidence to support effectiveness.
The programs, strategies and principles selected for this list of evidence-based practices have been found effective by the four sources listed in Appendix B: Blueprints for Violence Prevention; the Coalition for Evidence-Based Policy (“Top Tier”); work published by Mark Lipsey, Ph.D.; and the Washington State Institute for Public Policy. These sources were chosen because they reliably employ a rigorous scientific standard of evaluation. Appendix B provides a brief description of each source’s selection criteria, applicability, reliability, currency, advantages and limitations. Although popular, the Model Programs Guide published by the Office of Juvenile Justice and Delinquency Prevention (OJJDP) was specifically not used because of its lack of rigor has led to the listing of numerous programs that are not supported by evidence meeting the most minimal standards.
Blueprints for Violence Prevention
The Blueprints for Violence Prevention list has been developed by a research team headed by Delbert Elliott, Ph.D. at the Center for the Study and Prevention of Violence at the University of Colorado. For Blueprints to certify a brand name program as “model,” the program must demonstrate its effects on problem behaviors with a rigorous experimental design, show that its effects persist after youth leave the program and be successfully replicated at least once. In order for a brand name program to be certified as “promising,” the program must demonstrate effects using a rigorous experimental design. The Blueprints Web site (www.colorado.edu/cspv/blueprints/) lists 11 “model” programs and 19 “promising” programs.
Coalition for Evidence-Based Policy (“Top Tier”)
The Coalition for Evidence-Based Policy was created to assess social interventions for scientifically demonstrated effectiveness in the areas of early childhood development, education, youth development, crime and violence prevention, substance abuse, mental health, employment and welfare, and international development. In association with the coalition, the “Top Tier” designation is being developed under the guidance of a distinguished advisory group, for programs and strategies “that have been shown, in well-designed randomized controlled trials, to produce sizeable, sustained effects on important…outcomes[.]” As of the date of this publication, the Top Tier list has only three brand name programs that address crime, substance abuse or antisocial behavior: Nurse-Family Partnership, LifeSkills Training and Multidimensional Treatment Foster Care. The Coalition’s Web site is www.coalition4evidence.org/wordpress/ and the associated Web site for Top Tier is www.toptierevidence.org/wordpress/.
Mark Lipsey, Ph.D.
Mark Lipsey, Director of the Peabody Research Institute at Vanderbilt University, conducted the first meta-analysis that focused specifically on juvenile justice programs. In the most basic terms, a meta-analysis combines the results of independent evaluations with a shared research focus in order to analyze an overall effect, specifically called an effect size. Accordingly, Lipsey’s analysis did not identify specific programs but did begin to identify specific strategies and methods that were more likely to be effective than others. Lipsey has expanded and refined this work to include additional studies and many additional characteristics of each study.
Based on his research, Lipsey found that effective programs and strategies were well-implemented and focused on high risk offenders. He also found that strategies with a therapeutic orientation, such as counseling and skill-building, are more effective than those with a control orientation, such as surveillance and discipline.
The data referencing Lipsey’s research was taken from his recent publication titled, “The Primary Factors that Characterize Effective Interventions with Juvenile Offenders: A Meta-Analytic Overview,” and was published in a special issue of Victims and Offenders, 2009.
Washington State Institute for Public Policy
The Washington State Institute for Public Policy (WSIPP) uses the meta-analysis methodology to conduct evaluations of evidence-based practices, but also considers the cost of such programs and strategies to taxpayers and crime victims and weighs these costs against possible benefits (i.e., costs avoided through reduced crime). Programs and strategies are not ranked, but effect on recidivism is measured and the number of evaluations is reported. Recidivism, cost to tax payers and crime victims, and benefits are estimated using data specific to Washington State.
For the purposes of this paper, all cost and benefit information refers to the analysis conducted by WSIPP for the State of Washington. Accordingly, the information should be considered an estimate for the potential cost and dollar benefits for California. The data used for this project can be found in the article by Elizabeth K. Drake, Steve Aos and Marna G. Miller, titled “Evidence-Based Public Policy Options to Reduce Crime and Criminal Justice.