Pilot Study: Impact of Coaching on Implementing Safety Organized Practice

In 2012, The Northern Academy at UC Davis developed a coaching model to support the implementation of Safety Organized Practice (SOP) within their service area. Since that time, the use of coaching has spread across the region, state and country; however, limited information has been available regarding coaching's efficacy for SOP implementation, as well as the particular elements of any coaching model that are linked to any gains. This article summarizes findings from a recent exploratory study conducted by the Northern Academy that sought to address these gaps in knowledge by examining both coaching processes and self-reported learner outcomes. The pilot study included interviews with both coaches and learners around the experience of and effects of coaching, as well quantitative information gathered from coaching session logs, learner surveys and surveys of county leaders and/or key informants around coaching effectiveness. 

What Did We Learn?  

The vast majority (93 percent) of the effectiveness survey respondents in one of the larger counties endorsed that coaching had a positive impact on SOP implementation. Interviews in other counties large and small provide additional context and explanation for this high-level finding. For example, one coach asserted that the coaching process initially served as a prompt to learners regarding SOP goals, and then provided a sustainment function later on in implementation: 

“When we first started, in those first few years when people were still trying to figure [SOP] all out… It was like a reminder just seeing me or just having me email them and say, ‘Hey, I'm going to be coming. Let's make sure we've got some goals set for the day, an agenda.’ And they said that just even my physical appearance, my presence was a reminder of their SOP goals… And now, I think it's about sustaining the practice and the reinforcement of, and working on those areas where they could still do some upgrades.” 

Several respondents summed up the role coaching played in implementing SOP in their counties. 

“I think we're far enough along at this point that if coaching just disappeared tomorrow, I feel like I'd find a way to keep [SOP] going. But my worry is that it wouldn't be sustainable. What if I moved on and the next program manager didn't know about SOP? More of my concern is for longevity and sustainability.”  (Program Manager)

“As a supervisor, I think [coaching] helps reinforce the [SOP] practice. So it's not just coming from a supervisor's perspective. It's also coming from coaching, and having management support from the top down also helps.” (Supervisor)

Overall, social workers shared positive experiences of how SOP was being used to improve practice in serving families. One social worker's response is quoted below: 

“I feel like [coaching] would have just been pushed to the side and continue to get pushed to the side because we were too busy to try and figure out how to implement [SOP]. And with coaching, they made it just really easy; [we] can just implement this every single day… I feel like we collaborate [more] as an office [now], as a team on every decision… We always do group supervision and really try to get a whole group perspective on what the best decision is to make moving forward with a family… And I think that it helps the families too because there's no surprises. They get kind of familiar with our work and how we make decisions… it provides them with a level of consistency in our work that they're not used to. The system has kind of changed.” 

A complicating factor that must be addressed is whether coaching is provided as a voluntary, supportive resource or a mandated activity. There was a sharp disparity in perspectives from respondents on the utility of coaching based on this one factor. The feedback about coaching from counties that presented coaching as a resource was overwhelmingly positive; in contrast, supervisors from a county that required coaching noted several frustrations with the mandated nature of coaching.

Other important lessons learned from the surveys and interviews include:

• Approximately two-thirds (63%) of the 151 respondents to a learner survey reported that coaching provided improved motivation, while half (50%) reported gaining new knowledge (see Figure 9); 
• Coaches must tailor their support to meet each learner’s unique needs and adjust to meet the changing needs of learners across the implementation process;
• Ratings of coaching satisfaction and effectiveness were lower for individuals who had been coached for more than one year and for learners who had more experience (see Figure 5). Flexibility in the frequency, duration, location, and mode of coaching may be required to “fit” coaching within the unique child welfare context;
• The topics and strategies employed in a coaching session should balance learner needs and goals jointly defined by both the coach and the learner;
• Thoughtful attention to and timely adjustments to staffing, logistics, and strategies can support gains in motivation, knowledge, skill acquisition, empowerment, and goal-setting

Next Steps

Overall, this exploratory study has provided important insights that advance what is known about how coaching is used within child welfare settings. Important next steps include expanding the sample size by addressing response rates for learner surveys and conducting additional interviews to ensure representation of different types of units, experience levels, counties, and roles. Duration, frequency, learner experience and overall engagement should be examined in future research. Finally, while interview respondents noted that coaching has been connected to positive child and family outcomes, quantitative data to support this assertion has not been identified and is not available on a broad scale at this point.

What is coaching?

For the purposes of this study, coaching is defined as a "process by which the coach creates structured, focused interaction with learners and uses appropriate strategies, tools, and techniques to promote desirable and sustainable change for the benefit of the learner, making a positive impact on the organization." (Northern California Training Academy, 2012).