Recommended Free Tools
iTechGuides is reader-supported. When you buy through links on our site, we may earn an affiliate commission. As an Amazon Associate I earn from qualifying purchases. Learn more
Neither automated phishing investigation nor manual SOC triage is best for every team. Automation can sort and investigate repetitive user-reported email alerts, helping analysts spend more time on likely threats. People remain essential for exceptions, high-impact decisions, broader threat hunting, and checking whether automation is missing malicious messages.
A practical approach is to pilot automation as a way to prioritize and explain evidence, while keeping human review and response authority proportionate to the risk. The right balance depends on your queue, security stack, tolerance for missed threats, and the results you measure in your own environment.
How automated and manual phishing triage differ
Automated investigation and response
Microsoft Defender for Office 365’s Automated Investigation and Response (AIR) investigates supported alerts and submissions, correlates evidence around a message, and can expand an investigation as it gathers evidence. Depending on the findings, it can recommend remediation for Security Operations (SecOps) review and approval. Microsoft also documents automatic remediation for selected malicious similarity clusters, including malicious URL and file clusters, and resolution of cases where no threat is found or the threat has already been remediated. AIR is documented as a Plan 2 capability, and audit logging is required. Microsoft’s AIR documentation describes supported triggers and actions.
Quick wins for a faster PC:
Fix the driver behind crashes, sound loss and screen glitchesFind Drivers →Clear out junk files and repair common Windows errorsFree Scan →The Phishing Triage Agent
The Phishing Triage Agent is a distinct capability, not another name for AIR. It analyzes user-reported phishing alerts using email content, file and URL detonation, screenshot analysis, threat intelligence, and available organizational context. It returns a verdict with a rationale. In the documented flow, a false positive is resolved, while a true positive remains open and in progress for an analyst to investigate and act on. Analyst feedback is an explicit action in the workflow. Microsoft’s agent documentation explains its flow and prerequisites.
#1 Best Overall
Manual SOC triage
Manual triage puts the investigation and judgment directly in an analyst’s hands. Microsoft’s operations guide describes monitoring the incident queue, searching and filtering messages in Threat Explorer, investigating messages, and taking actions such as moving a message to the inbox, junk, or deleted items, or soft- or hard-deleting it. Analysts can also hunt proactively and share useful queries. Manual investigation can coexist with AIR: analysts may initiate investigations, review recommendations, and take action themselves. Microsoft’s Security Operations Guide covers these operational activities.
| Approach | What it does well | What still needs attention |
|---|---|---|
| Automated investigation or triage | Applies repeatable analysis and prioritization across eligible alerts; can collect and correlate evidence and, depending on the capability and configuration, recommend or perform defined actions. | Analysts need to review consequential findings, handle exceptions, assess misses, and retain control appropriate to the action’s risk. |
| Manual triage | Offers direct analyst control and flexible judgment for unusual messages, escalations, and broader investigations. | Every report consumes human attention, and the sources do not establish a universal queue size or staffing threshold at which automation becomes worthwhile. |
What the comparative evidence shows—and what it does not
James Bono of Microsoft Corporation published a randomized controlled trial of the Phishing Triage Agent in October 2025. The study randomly assigned 167 professional analysts to triage user-submitted phishing emails with or without the agent, using a curated, privacy-vetted corpus with standardized email artifacts. The results are relevant to that tested agent and task, not a guarantee of production outcomes for every SOC. Read the October 2025 trial.
| Reported result | Qualification |
|---|---|
| Up to 6.5 times as many malicious samples identified per analyst minute | Microsoft’s result for the study’s corpus-ground-truth scenario. The paper attributes 83% of the productivity gain in that scenario to queue prioritization and 17% to analysts using verdicts and explanations. |
| 77% higher F1 score for agent-augmented analysts | Reported under the corpus-ground-truth condition. In the paper’s lower-agent-accuracy counterfactual, the F1 improvement was 48%, and recall did not differ significantly from the manual control. |
| 53% more time spent on malicious emails by the agent-aware group | Reported by the study authors, who interpret this as analysts reallocating effort toward malicious items rather than simply accepting malicious agent verdicts. |
| 11.88% of the random sample from live operations was malicious | This describes the sample context in the paper; it is not a general phishing-report base rate. |
The trial is a Microsoft-published study of one purpose-built agent under controlled task conditions. Its “resolve-benign” protocol removed messages the agent classified as benign from analyst review. Under that protocol, participants were more likely to miss some agent false negatives. That finding makes the review policy—not just the classifier’s headline accuracy—an important part of any deployment decision.
How to decide what works for your SOC
Assess the operating conditions before choosing which work to automate. A high volume of repetitive reports may make automated prioritization useful; a smaller or highly varied queue may offer less opportunity to release analyst time. Neither the Microsoft documentation nor the trial establishes one universal volume threshold or a target performance level suitable for every organization.
Rank #3
- Workload: How frequent and repetitive are reports, and which parts of triage consume the most analyst time?
- Miss tolerance: What is the impact of overlooking a malicious email, and how will you test messages classified as benign?
- Investigation context: Can the system access the email, links, attachments, threat intelligence, and organizational signals needed for a useful verdict?
- Human control: Does the workflow classify, recommend, or remediate? Decide which actions require review or approval.
- Explainability and auditability: Can analysts inspect the supporting evidence, record feedback or overrides, and understand why a case was resolved?
- Operational fit: Does your platform, alert configuration, licensing, permissions, and integration setup support the workflow?
- Measured outcomes: Does the change reduce effort without increasing missed threats, rework, or unsafe remediation?
Run a human-supervised pilot and measure the trade-offs
- Establish a manual baseline. Record current queue volume, triage time, malicious reports correctly identified, misses, false positives, escalations, and remediation time.
- Define the review and response rules. Specify what automation may classify or recommend, which actions need analyst approval, and how benign-classified messages will be sampled or reviewed. Set these rules using your risk tolerance rather than copying the trial’s resolve-benign protocol.
- Test on your own submissions. Compare automated and manual outcomes on a controlled pilot, including messages later confirmed malicious. Track analyst minutes per true positive as well as total time, so faster handling does not conceal missed threats.
- Review exceptions and errors. Examine misses, false positives, overrides, and cases where required evidence was unavailable. Adjust the workflow or permissions before expanding its scope.
- Expand only when local results support it. Retain manual investigation for exceptions, high-risk cases, and proactive hunting. Recheck performance as the report mix or configuration changes.
Useful measures include malicious reports correctly identified, missed threats, false positives, time to triage, analyst minutes per true positive, escalation rate, and remediation time. No single threshold is established for all SOCs; interpret these measures against your baseline and the consequences of error.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.Check prerequisites and alert coverage before deployment
For the Phishing Triage Agent, Microsoft lists Defender for Office 365 Plan 2, provisioned Security Copilot capacity, Unified RBAC for Defender for Office 365, monitored reported messages in Outlook, and the “Email reported by user as malware or phish” alert policy. The agent also needs an appropriately permissioned identity with access to Defender for Office 365 data; Microsoft recommends least privilege. Verify current licensing and tenant configuration before planning deployment scope.
The agent does not triage alerts resolved by alert-tuning rules. Check both the built-in auto-resolve rule and custom tuning rules that might suppress the relevant user-report alert. For AIR, Microsoft lists audit logging as a requirement. These configuration details can determine whether the expected reports reach the workflow at all.
Outdated Drivers Are Slowing You Down
One free scan finds every outdated or missing driver and matches the right update for your exact hardware.Free scan · exact hardware matchPC Slower Than It Used to Be?
A free scan shows the junk files, broken settings and background clutter dragging Windows down - then fixes them in one click.Free scan · Windows 10 & 11Microsoft’s operations guide also recommends reporting false positives and false negatives, noting that user reports and admin submissions can contribute to detection learning. Where an organization uses a non-Microsoft reporting tool, the guide describes integration with Defender’s user-reported-message capabilities, subject to message-format and mailbox requirements.
Quick Recap
Best Value
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
