Part 2- Setting a big goal - ending poverty - and making results measurable

CHANGING LIVES. BUILDING COMMUNITIES. TRANSFORMING TORONTO.

How The Yonge Street Mission set out to create measureable progress

Second of a two-part conversation with CEO Angie Draskovic

Donors of all kinds want measurable results. I began exploring this theme in an earlier post here.

Much of current reporting describes the scope of activities and numbers of people engaged, but identifying impact is more elusive. When I heard that The Yonge Street Mission was launching a system to measure its work, I asked Angie Draskovic to describe it. The conversation is presented here in two parts. Part 1 can be found here.

Angie Draskovic

In June 2013, Angie Draskovic became President and CEO of The Yonge Street Mission, an historic agency with 120 years of service in Toronto. She has led YSM through intense reflection, analysis and planning to emerge with clear priorities, and an unusually well-developed approach to measurement, which is now in the beginning stages of implementation.      www.ysm.ca

 – Larry Matthews, KMA Consultants

Part 2: Truly measurable results

Larry:                     You’ve identified three populations that YSM is and will work with: street-involved youth, the chronically poor, and struggling families.  And you’ve set specific measurable deliverables for individuals on their individual or family journey out of poverty, what you are calling the RISE component of the RISE-EXCEL-TRANSFORM paradigm.  How far along is this?

Angie:                   We reorganized with three new departments, each with a leader focused by population. And we’ve introduced the measurement evaluation framework.

Larry:                     How did you introduce it?

Angie:                   Last year we set a bunch of goals consistent with our ends, and committed to the board that we would measure and report on those goals.  Then we added the goal of putting 10 people from each specific population we’re serving into the new measurement framework during intake, and monitor their growth and development under our new system.

Larry:                     And we’re you able to report?

Angie:                   Yes, we monitored manually for the last year and reported specifically on goals and achievements under the new framework. This year we're putting it into a software system, with reports that reflect what we learned because this was our baseline year.

Larry:                     What are you measuring?

THE FIVE DOMAINS
AND THEIR OBSERABLE, MEASURABLE INDICATORS

Angie:                   Our domains that we monitor for people are: connection, economic progress, education, environment, equality, wellness and worldview. For each of these, there's a five-point continuum of observable indicators of progress that are not subjective. We can start to see where people are progressing. We can also see where they're not.

Larry:                     Are the people who opt in aware of this whole matrix and the fact that they're being evaluated as they go?

Angie:                   They don't see a report like this but they know the categories. They see something that's user-friendly that basically helps them identify where they're at and set goals with the frontline worker around what they want to achieve for themselves.

With this, if I'm the staff or volunteer working with this person, it's much easier for me to figure out now, without it just being intuitive, what questions  I can ask the person to think about when they think of their own development.

And there’s lots of affirmation in this chart for somebody. That's where they started. Look at all the progress . . .  the frontline worker can say “look at how much you've accomplished.”

It’s starting to inform the frontline worker in a very powerful kind way as they interact with people, as they think about their people as they pray for their people.

EVALUATING OVERALL PERFORMANCE OF COHORTS OF PEOPLE

As well, it can greatly help form our assessment of our programmatic effectiveness, as we look at cohorts of people. With external partners, it can help us set shared goals where are partners are delivering some of these services, not us.

Larry:                     Where did the theoretical categories come from? 

Angie:                   The categories themselves came from a combination of reviewing existing wellness models and gleaning insights from very experienced staff.

Larry:                     Then you had to create a working framework for the model you were building and you recruited help for that. 

Angie:                   Yes, I shadowed a bunch of leaders and also spoke to people at the Trillium Foundation. They were just finishing their new evaluation system. They told us of a few companies that could help. We did an RFP and chose MNP to come in and work with us.

http://www.mnp.ca/en/consulting

Larry:                     But you could bring a lot to the table already.

Angie:                   Of course. We had other frameworks that informed the conversation. But the key was the professional consulting, working with our staff over probably six months in various engagements, to mine what was in people’s heads.

That's the thing with an organization like this - this is all in people's heads. Asking “What do people need to do to progress?” and “How can you tell if they're progressing? What do you see? What are the observables?”  All of those insights came out of our staff's heads because they're professionals and they've been doing this for so many years. With the help of MNP we kind of codified what we know about the real journey of people out of poverty.

CHARTING AN INDIVIDUAL’S JOURNEY (over 3 assessments)

Larry:                     Of the twelve thousand people, unique individuals that YSM saw in the past year, what percentage of them are in a position or likely to at some point opt in to what you're describing?

Angie:                   I don't know. We had a small sampling size this year that came in. We were on training wheels and we didn't really want to take on more than that.

EVALUATING MACRO PERFORMANCE

Larry:                     There must be certain people who simply want a specific “thing” from you.

Angie:                   And that's fine. They might be reasonably healthy and they just need a temporary support . . .

Larry:                     Food bank or whatever ...

Angie:                   Right. They're in and off they go. That's great. We'll get a better sense of the breakdown as we have better data in the next couple years around being able to track how many people we are seeing versus how many people are in some intensive sort of growth program with us.

Larry:                     How quickly will you fully implement this?

Angie:                   Gradually. We run the risk of overtaxing our staff and not providing adequate care to people in the program.

What we're looking for in training our team and looking to understand those who show a readiness and desire to move forward in their lives. We're learning how to foster that. And we've redesigned all our programs. As we build a larger and larger body of data, we'll have better ability to assess our effectiveness and just improve our ability to help people move forward in their lives.

Larry:                     What's the implications of this kind of approach for the number of staff you need?

Angie:                   Huge, in terms of the scalability of this, because it's a much higher case load.

Larry:                     But you can’t just hire staff, I presume.

Angie:                   We will take a little time to start to build scalability through high-end volunteerism.

We've always been and consider ourselves to be an extension of the church. More than ever, the reality is we're not going to get to God's intention for community by professionalizing community. This is community, in its hardest front-edge sense, for people who really need help.

There's a place for professionals, in equipping and engaging, but to be honest, we can't hire enough professionals to do this, ever.

So, we've revamped our entire view on volunteers. We've developed new volunteer roles at different layers of intervention and commitment and intimacy and different skill levels.

We're starting to go out and engage multiple volunteer communities, predominantly the church, to talk about ways that we can work together. This our opportunity for the church to be the church.

Larry:                     There could be some interesting self-reflection moments in here for staff and others to consider these categories and how they're operational in their own lives.

Angie:                   You look at the framework and you say “None of us are a five all the way down. That's just not true.”

Larry:                     How does this get shared more broadly?

Angie:                   We intend to when we’re ready. We need to understand it better and prove its impact. And disseminate it in a way that advances best practices. We’re not there yet. We'll accomplish more take our time and move into it deliberately.

Angie:                   I think we can do it. I think the biggest barrier among those of us who work in poverty reduction is, even amongst ourselves, not believing we can.

Larry:                     It can be hard to remain a true believer.

Angie:                   That is true enough.  But I can’t imagine putting my shoulder to a task and not putting forward my very best effort to realize the goal. In our sector, there can only be one goal for every person who is living in poverty, to help them move out of it.