Earlier this year, we worked with a nonprofit client who was eager to utilize their list of 2,000 contacts for a survey. These contacts were gathered from various events where people had stopped by their table and shared their emails. “These people are our biggest fans,” the client said confidently. “Of course they’ll take a survey if they know it will help us out!” Despite our reservations, the client insisted we go ahead and send the survey to their list.
Fast forward a month, and the results were disheartening: we received exactly eleven completed surveys. (Luckily, we convinced them to also administer the survey to their constituents from a research panel, which ultimately achieved the objectives of the research!)
After 20 years of working with clients who want to leverage their own contact lists, I’ve uncovered a critical insight: With very few exceptions, nobody’s list is as clean, thorough, or responsive as they think it will be. Here are a few realities to consider:
The Truth About Contact Lists
1. Email Addresses Change or are Inaccurate: People change their emails frequently. They may provide incorrect information or scribble down their details illegibly (or create typos while entering their details on a tablet) at events. This leads to inaccuracies in your contact list.
2. People Are Busy and Over-Surveyed: Even your biggest fans need a compelling reason to respond to your survey. Simply being a supporter isn’t enough to guarantee engagement. Offering incentives can help, but it doesn’t guarantee a high response rate.
Key Considerations When Surveying Your Own List
1. Who Is On Your List?
The purpose of a survey is to project the results onto a larger group accurately. For example, if we survey all 2,000 students in a school and get responses from 200, we can reasonably assume these 200 responses reflect the opinions of all 2,000 students, provided the respondents aren’t significantly different from the non-respondents. However, if your list is a mixed bag of contacts from various interactions, it’s hard to say who the survey results actually represent.
2. Accuracy of Your Contact Information
When sending out surveys, we often encounter numerous “undeliverable” email messages, indicating old or incorrect contact information. This highlights the importance of maintaining an up-to-date list. Outdated contact information can severely impact your survey’s effectiveness.
3. Size of Your List
A substantial list is crucial for achieving a reliable sample size. Typically, a response rate of 1% is considered normal for client lists, and recently, we’ve seen even lower rates. This means that from a list of 2,000 contacts, you might only get around 20 completed surveys. For meaningful insights, your list needs to be significantly larger to compensate for low response rates.
Final Thoughts
Surveying your own list can be tempting, especially if you believe your contacts are highly engaged. However, it’s essential to approach this with realistic expectations. Consider the composition, accuracy, and size of your list before proceeding. Understanding these factors can save you from disappointment and help you craft more effective survey strategies.
Remember, the goal is to gather actionable insights that truly represent your audience’s views. Sometimes, this means investing in more robust methods to ensure you’re hearing from a representative sample. Your efforts will be rewarded with data you can trust and use to drive your organization forward.
Reach out to us to learn how we can assist you in your next survey project.
