07 Apr Are we counting the right things?
At some point, every nonprofit is asked for evaluation results, usually by a funder. And the typical response is, “Oh, c’mon – can’t you just trust that we’re doing good work? Who has time to waste with evaluation?” But how much of our frustration with evaluation comes either because we’re measuring useless things, or because we don’t know how to take action with our results?
For nonprofits, evaluating what we do is tough, because it’s often more intangible than pure metrics can measure. Sure, we can talk about how many tickets or how many students at the matinees, but measuring how good the art was, how strong and lasting the impact on the audiences was, or the economic and social impact of the arts activity on its neighborhood is way harder to tease out.
Some nonprofits try to demonstrate their overall value by talking about their ratio between their program service costs and their administrative/fundraising costs. I’ll let other more eloquent folks address the foolishness of relying solely on the overhead ratio – Dan Pallotta, the CEOs of GuideStar, Charity Navigator and BBB Wise Giving Alliance, and local nonprofit finance whiz Kate Barr.
Others get hung up in social media numbers – the number of “likes” on their Facebook page or how many Twitter followers they have. But that’s not a number that has much value in itself, honestly. What matters more is the quality of the conversation you have with those fans, and the action that your content inspires.
I want to emphasize two words from the previous paragraph – “conversation” and “action.” If you think of your Facebook or Twitter feed as a way to market *at* your audience, I think you’re missing the boat. Instead, have a conversation *with* your audience in a way that they respond to, share with others (without your desperate-sounding “Please RT!”, and, eventually, do the thing you want them to do. That might be buying tickets to your show, donating to your nonprofit, or taking action on a political issue.
Does this mean you can’t evaluate social media results? Of course not – social media is an important and time-consuming part of your work, and you need to know if you’re accomplishing your goals. Use trackable links in your posts (create a free account on bit.ly and use their conveniently shortened links at the same time), or learn to read the analytic reports in your Facebook admin page. Use Google Analytics on your website and see where your traffic comes from. But stop getting excited about hitting some magic number of followers, and when you have the attention of your audience, don’t waste it by asking them to “like” you – ask them to do something more significant before they move on to the next thing clamoring for their attention.
I wish nonprofits would evaluate other things in addition to direct program outcomes:
- How satisfied is the staff? If your staff is unhappy, they’ll either under-perform or leave – neither option leads to productive organizational work or morale, and staff turnover is expensive and disruptive.
- Do your participating artists or partners feel well cared for? This can include paying them proportionally to the size of their commitment and the overall project budget, but it’s also measured in respectful treatment, appropriate public acknowledgement, opportunities for development and advancement, and a fun and positive working atmosphere.
- Marketing efficacy (because if people don’t know what you’re doing, that hinders your abilities to achieve your goals…). For a theater, the standard evaluation is number of tickets sold. But if the number is less than you’d hoped, don’t just conclude smugly that audiences just weren’t open to challenging new work. Maybe that’s true, but time spent evaluating the marketing plan, the location of the venue, and the ticket price might also have merit. (More words from me on this topic here.)
And don’t get me started on audience surveys with yes/no questions like “Did this show change how you felt about [insert Big Issue here]?” Bad question design gets you useless results, every single time. Surveys only have value if you (a) write smart unbiased questions and (b) analyze and use the results to make changes as appropriate. Check out helpful tips from SurveyMonkey and the U.S. government.
I don’t have all the answers on evaluation, but I’d love to help find a way to convince nonprofits that, properly done, it’s a powerful internal tool to improve operations as well as a valuable way for funders to see if their money is being used to good effect.