Careers
Part of Nonprofit social media: choosing channels and a cadence
How to vet nonprofit social media case studies before you cite them
How to vet nonprofit social media case studies: name the campaign, check the method, disclose ties, and test one claim locally before you repeat it.
What to take away
- Classify the claim before you weigh the result. Provider copy, a customer testimonial, an organization's own published total and an independent study are four different things.
- Write your inclusion criteria before you open any case, so a vendor cannot hand you the shortlist.
- Five campaigns are worth citing by nameMovember, charity: water, the ALS Ice Bucket Challenge, Team Rubicon and the Humane Society of the United States.
- Reproduce one material claim on your own list, then record the rate and the denominator.
- Disclose payment, free access and placement beside the case, following FTC guidance on reviews and incentives.
Set your inclusion criteria before you look
Write these fields down first. Then record what each case leaves out.
Set your inclusion criteria
- Named organization and named campaign
- Stated period and stated channel
- A result the organization published itself, with a denominator
- Claim type, plus any disclosed tie to a vendor or agency
- Document title and publication date you can retrieve
- Stop rulewhat would make you drop the case
[figure 1]
Inclusion criteria fields
- Organization type
- Campaign period
- Channel
- Budget band
- Program outcome or platform metric
- Record what is missing
A case with no denominator cannot give you a rate. A case with no comparison group cannot show what caused the change. The GAO evaluation design guide ties an evaluation question to the evidence it needs. Reach for it when a case claims causation from a before-and-after chart.
To judge whether a stated rate is plausible for a list your size, reading nonprofit social media benchmarks without fooling yourself sets out the ranges.
The five campaigns worth citing
| Campaign | Channel | Period | Where the write-up sits |
|---|---|---|---|
| Movember | Peer-to-peer Mo Space pages | Each November since 2003 | Movember Foundation annual review; the split by challenge is not published |
| charity: water | Birthday campaign, supporter posts | September, annually since 2009 | charity: water September campaign page and project map |
| ALS Ice Bucket Challenge | Facebook video and participant posts | Summer 2014 | ALS Association news release, August 2014, reporting about $115 million over eight weeks |
| Team Rubicon | Recruiting Greyshirt volunteers | Since 2010 | Team Rubicon annual reports, which carry volunteer counts |
| Humane Society of the United States | Facebook and Instagram video | The Shelter Pet Project since 2009, with Maddie's Fund and the Ad Council | HSUS campaign pages and annual report |
Read each one for method, not for a number to copy. Movember suits a charity that already runs a peer-to-peer engine. charity: water states that public donations fund water projects while private donors cover overhead, which is an allocation claim rather than a social media result.
Team Rubicon recruits labor, so its unit is volunteer hours, not dollars. If staffing a monthly video series is your real constraint, tightening nonprofit social media output covers what to cut first.
Example: a provider claim beside a measured claim
Our toolkit lifted donations 40% for a health charity.
That is a provider claim. No organization, no period, no denominator, no name attached to the number.
By contrast, the ALS Association reported about $115 million in donations across eight weeks of summer 2014. The organization published that total itself, so it counts as a measured record. It still carries no comparison group, so it does not show what would have happened without the challenge.
[figure 2]
Four kinds of claim
Provider claim
- What it is
- Marketing copy
- Keep it
- Separate column
- Weak cases
- Blurred
Customer statement
- What it is
- Testimonial
- Keep it
- Separate column
- Weak cases
- Blurred
Measured record
- What it is
- Data with method
- Keep it
- Separate column
- Weak cases
- Blurred
Independent finding
- What it is
- No stake in sale
- Keep it
- Separate column
- Weak cases
- Blurred
Attribute a measured total to whoever published it. Then say what the number cannot show.
Run the replication test on your own list
Run the replication test
- Pick one material claim. Write the population, the period and the metric before you start.
- Define the metric as a rate, and record the denominator.
- Split one segment of your list into a test group and a holdout, and leave everything else alone.
- Note one normal path and one failure path, plus the inputs and versions you used.
- Set a stop rule and a next review date.
The A/B testing article explains random assignment and why the holdout carries the argument. The numbers below are illustrative, not a published result.
Send one thank-you video to 4,000 past donors and hold back 2,000. If 240 of the video group give again and 100 of the holdout do, the rates are 6.0% and 5.0%.
One point on a 2,000 person base is a lead for a second round, not a finding. For worked examples of this standard applied to acquisition campaigns, see lessons from real campaigns.
The review record
| Field | What you record | Trigger to escalate |
|---|---|---|
| Case identity | Organization, campaign, period, channel, document date | No date or no named author |
| Claim type | Provider, customer, measured or independent | Two types blurred into one sentence |
| Method | Inputs, observations, unresolved limits | No denominator |
| Outcome | Rate, denominator, cost per result | A total quoted as a rate |
| Safeguard | Permission record, correction owner | Beneficiary story published without permission |
Keep one row per case, and date every entry. If a claim fails your test, write that down and move on. A failed replication still tells you something about your own list.
