There is no evidence here that simulated phishing tests generally make organizations less secure. The evidence is mixed: some training programs have shown little measured benefit, while a one-time simulation improved specific behaviors in a different setting. Simulations can also take substantial staff time and affect employees’ stress and confidence. Those are reasons to design and evaluate them carefully—not proof that they worsen real-world security.
What the studies say about whether phishing tests work
“Phishing test” can mean a one-time simulated email followed by an explanation, recurring campaigns, annual awareness training, or training embedded in a simulation. Studies of these different approaches measure different outcomes, so their findings cannot be reduced to one universal verdict.
| Study and setting | Intervention or measurement | Finding and limits |
|---|---|---|
| IEEE Symposium on Security and Privacy, 2025; one large healthcare organization, more than 19,500 employees, eight months | Randomized controlled experiment involving ten simulated campaigns; examined annual awareness training and training embedded in simulations. | Recent completion of annual awareness training had no significant relationship with simulation failure. Differences associated with embedded-training content were very small; employees spent little time on the material, and for some content types, completing more instances was associated with greater likelihood of failing a later simulation. The authors concluded that the practices studied were unlikely to significantly reduce phishing risk in practice. This is evidence about one organization and its programs, not every training design. |
| Dutch Ministry of Economic Affairs field experiment; 10,929 employees | Compared information, one simulated phishing experience, both interventions, and a control. The simulated email asked employees to link an account to a mobile number for password recovery; participants received a debrief that day. | Information and the simulated experience each reduced the likelihood of clicking a dubious link and disclosing personal details. Combining them did not substantially improve on the experience alone. The measured behavior does not establish an effect on real phishing incidents. |
| 2025 field experiment; 670 small and medium enterprises and 33,000 employees | Phishing drills; the published highlights report on click rates. | The highlights say drills reduced click rates only in the short term and not systematically. They do not establish a universal effect or provide a basis here for quantifying the size of the change. |
Taken together, the findings suggest that training outcomes depend on the intervention, setting, and measure. A result on clicking or disclosure is not automatically evidence of fewer successful real attacks. Nor does a null result from one organization prove that every well-designed program is ineffective.
Why a lower click rate may not mean stronger security
A click rate is a measure of what employees did with a particular simulated message. It is not, by itself, a measure of how well they will handle real phishing. If one campaign is easier to recognize than another, a lower click rate may reflect the message rather than improved employee skill.
The Tool Desk
Outbyte Driver Updater FREEFix the driver behind crashes, sound loss and screen glitchesFind Drivers →Outbyte PC Repair FREERepair Windows errors before they cause bigger problemsFix Now →#1 Best Overall
- Featured in Forbes, CNBC, Business Insider, PopSugar: Packed with effective language, skills, and strategies; improve personal and professional relationships; used by teachers, coaches, mentors, and trainers to help people boost EQ
- Become Emotionally Intelligent: Improve intrapersonal skills (self awareness, self management) and interpersonal skills (social awareness, relationship skills); conversation card game sequenced to open up communication, build trust, and engage everyone
- For couples, friends, co-workers: Safe space for interactive storytelling and thought-provoking discussions; uncover values and needs, use card prompts and questions for one to one, manager check-ins, dating, marriage, or parenting quality time
- Developed by Harvard researcher and executive coach: Build communication skills, collaboration, psychological safety, inclusion, mindset, and empathy. No more misunderstanding
- Includes FREE online course and EQ assessment: taught by Dr. Jenny Woo, may qualify for CEU, upskill yourself with online learning curriculum and use the cards to practice and apply your learning anytime and anywhere
NIST’s Phish Scale rates the human difficulty of simulated phishing emails to help put click rates in context. NIST’s 2023 presentation description cautions that “phishing training programs cannot be assessed in a vacuum.” Comparisons are more meaningful when they account for message difficulty and context, rather than treating raw rates from unlike campaigns as direct evidence of learning.
- Compare campaigns with similar difficulty and conditions, or account for differences when interpreting results.
- Track reporting as well as clicking; the two outcomes show different behaviors.
- Use more than one simulation metric, and distinguish simulated behavior from real incidents.
These are practical evaluation implications, not a formal NIST checklist. The Dutch Ministry experiment also discussed false-positive reports as a possible operational cost of encouraging reporting, but did not measure false positives; the ministry did not report them as a problem in that setting.
Rank #2
- MINDSET CARD GAME: Engaging gameplay designed to differentiate between fixed and growth mindsets through relatable scenarios, fostering the development of a growth mindset.
- PROMOTES RESILIENCE: Encourages perseverance, resilience, and a positive outlook in the face of challenges, nurturing a strong work ethic and grit.
- TOP TEN BENEFITS: Enhances self-motivation, perspective-taking, communication skills, self-awareness, critical thinking, growth mindset, and executive functions; reduces social anxiety; fosters empathy and responsibility; and enriches vocabulary and academic language.
- ADHD / AUTISM TOOL: A valuable resource for parents to comprehend their children's emotions better, serving as an educational STEM game for children who may struggle with discouragement or feeling defeated.
- INSPIRES POSITIVITY: Motivates players to persist in their efforts, boosting self-esteem and encouraging a constructive attitude towards challenges.
What simulations can cost an organization
A simulation program uses staff time for more than sending an email. In a USENIX Security 2023 case study of one organization selecting and purchasing a phishing-simulation service, the researchers estimated at least €50,000 in person-hours for that organization’s procurement and preparation process. The work included assessing training requirements and employee acceptability, preparing technical infrastructure, and establishing operational processes; the authors also noted intangible costs.
That estimate is from a single case study. It is not a typical price, an average across organizations, or a cost-benefit result. The case study’s authors—Brunken, Buckmann, Hielscher, and Sasse of Ruhr University Bochum—wrote: “The prevailing perception that phishing simulation campaigns are a quick and low-cost solution to providing security training to employees thus needs to be challenged.”
Rank #3
- HYSTERICAL PARTY GAME: DiagnosTHIS is the hilarious adult party game where you'll be dying to search your symptoms!
- Have you ever had a minor headache, searched your symptom, and according to the internet you only have moments to live?
- INCLUDED: 300 Diagnosis Cards, 100 Symptom Cards, set of instructions!
- EASY TO PLAY: A hilarious take on the tried and true fill in the blank game that's perfect for game night at home, or taking on the road!
- If searching your symptoms is wrong, we don't want to be right.
What is known about employee stress and acceptance
A USENIX Security 2024 study examined 408 employees immediately after they either clicked or reported a simulated email and interviewed 21 of them. Employees who clicked reported higher stress and lower phishing self-efficacy than employees who reported the message. Participants in both groups generally viewed the campaigns positively and considered them effective.
These were short-term, self-reported measures. They do not show that the employees’ later behavior worsened or that the organization became less secure; the study authors said further work on stress was needed.
Rank #4
- {THE AND} HEALING EDITION: The perfect game for healing strained relationships and releasing hurt feelings: The Healing expansion deck gently shows you the way to ease into loving conversations with a partner, friend, or family member
- EFFECTIVE COMMUNICATION: Our healing relationship cards are designed to facilitate effective communication, providing a safe and structured space for couples, friends, or family members to express themselves and listen to each other.
- RECONNECTING AND HEALING: These conversation cards help to break down emotional barriers and enable individuals to reconnect with one another on a deeper level; Through the use of these couples cards, people can heal strained relationships and mend past hurts
- DIVERSE TOPICS: Our conversation starter cards cover a range of diverse topics, from feelings and emotions to shared memories and goals; This allows individuals to explore different aspects of their relationship and build a stronger emotional connection by understanding your person and what they need from you
- ABOUT TSD: The Skin Deep started as an interactive documentary project which filmed emotional conversations between two people as they asked intimate questions; TSD's exciting and immersive projects continue to explore the human experience and help people connect on a deeper level
A 2025 NDSS study focused on campaign acceptance, not security effectiveness. Its authors found higher acceptance when employees gave consent, and lower acceptance when interviews were imposed as a consequence or when a lure promised an incentive. They explicitly stated that “this study does not assess or advocate for the overall effectiveness of such campaigns in improving organizational security.” The findings make consent, consequences, and whether a pretext respects employee trust relevant design considerations, but do not establish how those choices change real-world security.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.How to judge whether a program is worth continuing
Do not treat the number of employees who click as the program’s sole success measure. A useful evaluation asks what behavior the program is meant to improve, whether it changes that behavior, and what the organization spends—in staff time as well as money—to achieve the result.
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
Best Value
- Trusted By Families Worldwide - With Over 50 Million Sold, Thinkfun Is The World's Leader In Brain And Logic Games
- Develops Critical Skills - Playing Through The Challenges Builds Reasoning And Planning Skills As Well As Core Programming Principles, And Provides A Great Stealth Learning Experience For Young Players
- What You Get - Hacker Is A Cybersecurity Coding Game And Stem Toy For Boys And Girls Age 10 And Up Where You Learn Programming Principles Through Fun Gameplay. It Includes A Game Grid, Control Panel, Challenge Booklet, 2 Agent Tokens, 9 Movement Tiles, 13 Revolving Platform Tiles, 5 Double-Sided Transaction Tiles, A Transaction Link Token, 3 Data File Tokens, 2 Exit Point Tokens, A Virus Token, Alarm Token, 2 Lock Tokens, And A Solution Booklet
- Clear Instructions – Easy To Learn With A Clear, High Quality Instruction Manual. You Can Start Playing Immediately
- Define the intended outcome. Decide whether the goal is fewer clicks, more timely reports, less disclosure of sensitive information, or improved handling of real phishing. Keep those outcomes distinct.
- Make comparisons fair. Record the difficulty and context of each simulation, using the NIST Phish Scale to help interpret differences in message difficulty.
- Measure more than a single campaign. A one-time improvement, such as the behavior observed in the Dutch Ministry experiment, does not by itself establish durable learning or fewer real incidents. The 2025 SME-study highlights likewise report short-term rather than systematic click-rate reductions.
- Account for employees’ experience. Consider whether the campaign is explained, whether employees consent, and what happens after a click. Avoid assuming that a click alone establishes an employee’s overall ability or the organization’s risk.
- Include operating effort. Track staff time for procurement, preparation, infrastructure, and campaign operations, not just the service fee.
If a program shows no meaningful improvement in its chosen outcomes, its recurring workload and employee impact deserve review. If it does show improvement, distinguish the measured behavior from claims about real-world incidents unless those incidents were actually assessed.
So, do phishing simulations actually improve security?
Sometimes they improve specific measured behaviors; sometimes the training practices studied show little or no meaningful effect. The evidence described here does not establish that simulations generally make organizations less secure, and it also does not justify treating a lower click rate as proof of better protection. The most defensible conclusion is narrower: simulation programs can be costly, stressful, or poorly evaluated, and their security value depends on what is taught, how campaigns are designed, and whether outcomes beyond a single click rate are measured.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
