Analyzing Improvement Efforts


My online PLC crew is in the thick of testing change ideas in honor of their respective improvement aims. We’ve come a long way! We got grounded together. Each individual got clear about what it is they want to change about student learning in their online course. And they’ve all zeroed in on the change ideas they believe are most likely to get them the outcomes they desire.  This time when we convened, we paused to make sense of our progress towards improvement.

Here’s how it went down…

Meeting Objectives

  • Get reaquainted with your improvement effort
  • Practice data analysis
  • Analyze one another’s first sets of data resulting from initial changes implemented
  • Reflect on insights gleaned from this first cycle of inquiry to decide what you’re going to do next to strive towards your improvement aim

Meeting Flow (the intended flow, anyways!)

  • 10min: Get Connected (All Together)
  • 5min: Telling Your Story – Free write to get reconnected with and to prepare to tell the story of your improvement efforts to-date (On Your Own)
  • 15min: Practicing Analysis – ATLAS Looking at Data Protocol using one PLC member’s first set of data resulting from change idea (All Together)
  • 25min: Small-Group Analysis & Dilemma Grappling – Replicate data analysis protocol in smaller groups (Small Group Break-Outs)
  • 5min: Next Steps & Reflection – In next month, take the next steps you’ve identified to continue striving towards your improvement aim (All Together)

Check out the modified (for public consumption) and more granular version of this agenda here, and you should feel free to make a copy of the document and make it your own.

What Actually Happened

This meeting happened a week after the 1st to 2nd semester transition. Data from improvement efforts since our last meeting was…limited. I was prepared for this possibility, and we simply pivoted. Instead of break-outs to try to analyze everyone’s first sets of data, we dug more deeply into one PLC member’s data. This resulted in a few wins:

  • We were able to more deliberately practice a new protocol together – NSRF’s ATLAS Looking at Data – that will likely animate how we go about making sense of data in meetings to come.
  • The PLC member who had some data to share came out of the analysis experience with what I see as a pretty valuable insight: his improvement aim was actually not yet an aim! It is a likely means to an end, but not an end in and of itself. (His aim was to increase the level of student-initiated communication.) He realized he needs to do 1 of 2 things: Either get clear about what he predicts will happen if student-initiated communication increases OR reconnect with what he really wants to improve about student learning and evaluate whether his current effort is what is most likely to get him there.
  • I believe this approach provided others in the PLC a valuable model of what to expect moving forward. It gave them permission, this time around, to focus on being a good critical friend to their PLC teammate while also helping them to have a mental model of how to prepare to share next time.

Next Steps

Some of the PLC members expressed in their post-meeting reflections that they need to make sure to prepare in advance to put their data together in a shareable way. We also discussed the idea of zeroing in on 1 or 2 people to bring their data for us to analyze as a group, rather than trying to get everyone “the floor” in our precious monthly hour.

In response, I will be reaching out to the group a week before our next meeting to see who’s feeling like it’s a relevant and opportune moment to unpack their data. For those who say they’re ready to share, I’ll offer my support if they’d like help packaging their data for group analysis.

This all has me reflecting on the concept of “rapid testing.” Are we granular enough in our improvement aims to truly consider it improvement science? Should everyone realistically be showing up with data at each meeting? I go back and forth on this. For now, I think there is great potential for this effort to influence the improvement mindsets of the individuals in this group and to potentially influence learning structures for our organization moving forward.

“Deliberately learning our way to better outcomes is, in fact, how organizations improve quality and how interventions scale effectively.”Bryk


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s