We continually make product and process improvements to maintain a legitimate and high-quality participant pool that can be counted upon to provide valuable insight to researchers in every session.
How does User Interviews deter participant fraud?
We take steps to root out suspicious behavior at the point of participant sign up, as well as reinforcement throughout the participant lifecycle. Automated checks are just a start — we also get ongoing signals from project coordinators who are trained to spot misrepresentation and suspicious accounts.
This means that bad actors aren’t likely to ever reach your project, but if they do slip through, we have feedback loops to quickly flag and remove bad actors and continually improve the quality of the pool for all researchers.
We’ve made recent investments that have significantly reduced the incidence of fraudulent sessions. Some of the specific measures we’ve implemented include:
- Running checks at signup for common fraud patterns across contact information and other profile characteristics
- Automated checks for digital identity overlap with known fraudulent accounts
- Requiring re-verification of profile and contact details at certain intervals
- Increased sophistication of fraud model that flags scores above threshold based on ongoing participant activity and signup factors
- Additional project-level protections for identified at-risk projects
- Strengthened internal process for manual identification and reporting of fraud by operations team members
We intentionally don’t share all the specifics about our methods for identifying and preventing fraud, since making this information available can help bad actors to thwart the system.
How prevalent is fraud on the User Interviews platform?
Bad actors are a relatively rare occurrence. Each day, hundreds of new people sign up to participate in research studies, and each month, around 50,000 participants are active on our platform. Of this large population, only ~0.3% are flagged as suspicious — a small fraction of overall activity.
We’ve recently upgraded preventative checks and we continue to make investments in detection to further reduce instances of fraud. Recent enhancements to our fraud detection system enable us to catch >95% of likely fraud before it affects researchers.
What should I do if I suspect a participant is fraudulent?
When fraud impacts you, we’re here to help. Every Recruit project is supported by a project coordinator who works with you to find participants who are a good fit. They are trained in how to spot and flag suspicious profiles via signals in participation history, session feedback, and profile and screener responses.
While reviewing participants:
If you come across a participant while reviewing your “Qualified” and “Unqualified” participants that you suspect is misrepresenting themselves, please report these folks to our Support team for further review.
To do so, click the “Report” button under the “Participant Quality” section of the Participant Overview.
From there, you will be prompted to write a brief summary of any issues or discrepancies you noticed during your review. Please click, “Submit” to send this report to our team.
The reported participant will be automatically hidden from your view while our team investigates their profile.
After approving a participant:
If you suspect that one of your approved participants is misrepresenting themselves, please reach out to your project coordinator so they can investigate and verify the accounts, provide recommendations to mitigate fraud, and issue refunds if necessary.
While we have many systems in place to help prevent fraudulent participants from confirming a session with your team, in rare instances a bad actor makes it to a confirmed session. If this happens, we recommend ending the session early and reporting the participant's behavior to your project coordinator in the rare instance that a fraudulent participant actually makes it through to a session with you.
To do so, click the ellipsis button beside the attendance status and select, “Report”.
From there, you will be prompted to enter a detailed description of the incident, allowing our Support team to launch a thorough investigation. Without a report supporting your claim, our team will be unable to investigate and will mark the participant as “Completed”.
You will be updated on the status of the investigation in your project workspace, and if necessary, by your project coordinator.
Best practices to follow when launching a project?
We recommend some best practices researchers can follow to reduce the chances that bad actors getting through our system will make their way to a session with you:
- Avoid leading questions on screener surveys. For example, instead of asking “Do you use product X?”, try asking “Which of these products do you use?” and provide multiple choices for the participant to select from. Similarly, avoid providing highly specific information in project descriptions that reveals the screener responses that you’re targeting, and that may allow bad actors to game the system.
- Ask for a short description of the participant’s experience with a particular product or service to verify that the participant has used it before. For example, “Briefly explain how you use product X,” or “Briefly tell us how service X fits into your day.”
- Try double screening, an add-on feature for Recruit projects. Double screening gives you access to participant contact information, allows you to message participants before they're confirmed, and accommodates manual scheduling. Learn more about double screening.
- Reach out for support if you suspect a participant in your queue is fraudulent. Contact your project coordinator so they can launch an investigation.
- End the session early in the rare instance that a fraudulent participant actually makes it through to a session with you. Hold off on marking attendance for the participant and submit a report to our team on the “Confirmed Sessions” page. This will withhold payment to the participant if you’ve opted for User Interviews to distribute incentives on your behalf, and automate a notification to our support team so they can launch an investigation.
