Are your research panels surveying actual customers?
Research panels are a customary tool for accessing respondents to create samples for research projects. How would you react if you discovered that three-quarters of the participants in your project hadn't actually bought the product category you were researching? A recent study conducted by 84.51° uncovered this finding and many others. Read on for more insights and learn how to ensure you’re surveying actual customers.
What is a research panel?
A research panel is composed of a group of people who have opted into being on a panel for the purpose of participating in surveys, research, etc. A panel is used to create samples for specific research projects. These individuals are intended to be representative of target customer groups and have agreed to provide information or feedback on various topics. The data collected from these samples are used to gain insights into consumer behavior, preferences, trends and other research areas to make informed business decisions.
When would a company use a research panel?
A company would use a research panel when they need to gather information or insights about a specific topic, product or service to make informed decisions. This could be for a variety of reasons such as:
Market research: To understand consumer behavior, preferences and trends in the market.
Product development: To gather feedback on a new product or service before it is launched.
Pricing strategy: To determine the optimal price for a product or service.
Customer satisfaction: To measure the level of satisfaction among their customers and identify areas for improvement.
Brand perception: To understand how their brand is perceived in the market.
Competitive analysis: To gather insights about competitors and their strategies.
Advertising and marketing: To test the effectiveness of advertising and marketing campaigns.
How are survey respondents recruited for a research panel?
The data quality collected from research panels is only as good as the respondents who provide it. Therefore, the process of recruiting and verifying survey respondents for a sample is a critical step in ensuring the validity and reliability of the research findings. Defining the sample: The first step is to identify the target population for the research. This could be a specific demographic group, such as women aged 25-35, or a broader group, such as adults who shopped certain categories from a retailer.
Once the target population is defined, researchers use various methods to sample respondents. These can include research panels, company lists, online advertisements, telephone calls or in-person recruitment at events or locations frequented by the target population.
Screening: This involves asking potential respondents questions about their age, gender, location and other relevant factors to ensure they qualify for the research. There are several methods to screening that are different. One is self-claiming behavior, and another is behaviorally-verified screening. Self-claimed respondents are selected based on their self-reported information. This means researchers are relying on the accuracy of the self-claimed respondents’ answers and introduces the risk of admitting invalid respondents. Behaviorally-verified respondents, on the other hand, are verified through objective first-party data or behavioral evidence. For example, in addition to asking respondents if they had recently shopped a certain product category, it can be verified against transaction records. Consent and participation: Once respondents have been selected, they must give their consent to participate in the research. From there, they are invited to participate in in-person or online surveys or other research activities. They may be incentivized to participate through rewards such as cash, gift cards or entries into prize drawings.
Ongoing engagement: To maintain a robust research panel, researchers need to keep respondents engaged. This can involve sending regular updates, providing feedback on the results of the research and offering additional incentives for continued participation.
What are the benefits and drawbacks to self-claimed sampling?
The benefit of self-claimed sampling is that respondents simply need to confirm that they meet the screening criteria to be a potential respondent for research. However, the information that self-claimed respondents provide is based on their own perception and memory, which may not be accurate or reliable. This could undermine the quality of the research results, leading to costly errors. In fact, a study conducted by 84.51° that compared self-claimed with behaviorally-confirmed sampling techniques uncovered considerable risks in utilizing survey research that depends on participants' recollections of past behaviors.
A significant finding was that most self-claimed respondents did not fulfill the survey requirements. For example, three out of four self-claimed respondents stated they shopped a category in the previous three months, yet the behavioral data showed no sales or units in that same timeframe.
What are the benefits and drawbacks to behaviorally-verified sampling?
The main drawbacks of behaviorally-verified sampling are that it necessitates access to data, expertise in understanding the available data and designing the sample frame correctly, as well as the ability to connect with consumers in the behavioral database.
Behaviorally-verified respondents, though, are identified based on actual purchase behavior from loyalty card or other transaction data to verify they qualify for the desired shopper profiles. Behaviorally-verified sampling has a higher incidence of respondents qualifying for a study because they exhibit specific attributes, characteristics or behaviors due to the pre-qualification based on their actual purchase data. This also ensures insights come from real people versus bad actors, bots or click farms.
How do I ensure that my samples include actual customers?
Here are some best practices to ensure the authenticity of respondents and improve the quality of your survey results.
1. Request detailed descriptions of quality measures:
When soliciting proposals from research panel providers, it's crucial to request a detailed explanation of the quality control procedures they implement. This includes their methods for recruiting panelists, their process for verifying respondent identities and their strategies for ensuring data accuracy. This will assist you in assessing the caliber of the respondents, enabling you to make an informed decision about which provider to select.
2. Verify respondent information:
One of the most effective ways to ensure the relevance and accuracy of your data is through verification. Verify the details given by respondents by comparing them with purchase records or other behavioral documentation. This can help you confirm that the respondents are indeed your customers and that their responses accurately reflect their experiences and perceptions.
3. Scrutinize self-claimed research results:
If you're using self-claimed sampling methods, it's crucial to pay close attention to the results. Be vigilant about any inflated metrics or illogical responses. These could be red flags indicating that the data may not be entirely accurate or authentic. Identifying potential issues early on can help you improve the overall quality of your data and make more accurate business decisions.
4. Evaluate survey quality checks:
At a time when AI technology advancements can imitate human survey responses, it's increasingly crucial to evaluate any self-claimed survey quality controls. Ask providers about any new innovative quality checks they have implemented to ensure respondent quality. This could include advanced algorithms to detect fraudulent responses, machine learning techniques to identify patterns of dishonesty, or biometric technologies to verify respondent identities.
Be proactive about quality survey results
Research panels are a common approach to accessing respondents for a sample but be mindful of screening methods. Respondents screened through self-reporting their behaviors can increase the risk of invalid respondents and inaccurate data. This leads to the wrong business decisions. The moment brands disconnect from the fundamental needs and values of their customers, they risk losing their competitive advantage.
Research that is verified through objective data or behavioral evidence allows brands to cut through the clutter and connect with real people. To ensure the authenticity of respondents and improve the quality of survey results, it’s essential that businesses take a proactive approach. Request detailed descriptions of quality measures from research panel providers, verify respondent information, scrutinize self-claimed research results and evaluate survey quality checks. Remember, the quality of your data directly impacts the accuracy of your insights and the success of your business strategies.
Visit our knowledge hub
See what you can learn from our latest posts.
