Most security operations teams describe their problem as alert volume. Volume is the symptom. The cause is a detection set that was tuned once, inherited from a vendor’s defaults, and never revisited against the environment it runs in, so it fires constantly on activity that is normal here and quiet on activity that is not. Automation that triages faster without touching the detections buys an analyst back some hours and leaves the underlying quality problem in place, growing. The products worth adopting are the ones that feed detection engineering rather than substitute for it, and that distinction is what the evaluation below is built around.
Overview: 5 Things That Decide SOC Tooling Value
- Triage output must be checkable. An analyst who cannot see why a verdict was reached will re-investigate from scratch, and the time saving disappears.
- Enrichment is the highest return, lowest risk function. Gathering context before a human looks is pure gain, since nothing is decided by the tool.
- Closing alerts automatically is a policy decision, not a feature. Define which detections may auto close before evaluating any product that offers it.
- Ingest cost shapes detection design. Per gigabyte pricing quietly determines which logs you keep, and therefore which detections are possible.
- Detection engineering is the compounding investment. Every improvement there reduces volume permanently, while triage automation reduces cost per alert only.
Why Alert Volume Keeps Growing Whatever You Buy
Alert volume grows because adding a detection is cheap and removing one feels risky. Nobody is criticised for a noisy rule and everybody remembers the incident that a disabled rule would have caught, so the rule set only accretes. Our managed security services work usually starts with the rule set rather than the queue, and the three causes below are what we find.
Vendor default detections are written for a generic environment
A detection shipped by a platform vendor has to work everywhere, which means it fires on behaviour that is suspicious in general rather than suspicious here. A service account authenticating from an unusual location is worth investigating in most organisations and is Tuesday in one with a distributed contractor workforce.
The case for keeping defaults is coverage and defensibility. They encode work you did not do, they update as the vendor learns, and an auditor asking about detection coverage is satisfied by them.
The case against leaving them untuned is that a rule with a 95 percent false positive rate trains analysts to close it unread, which is functionally the same as not having it while costing real hours. Tune or disable, and record the decision with its reasoning so the next reviewer knows it was a choice. The discipline is the same as the one in our vulnerability management process guidance: a finding nobody acts on should be fixed or retired, never left to accumulate.
Automated triage can hide the feedback detection engineering needs
When a tool resolves alerts before a human sees them, the signal that a detection is noisy stops reaching the people who could fix it. The queue looks healthier and the rule set gets worse, because nothing now reports how often that rule was wrong.
Some teams argue this is acceptable where the automation is accurate, and if the tool genuinely resolves correctly then the noise costs nothing. That holds only while the environment is static.
The safeguard is to keep automated dispositions visible as data even when they are invisible as work. Report per detection how many alerts were auto closed and at what confidence, and review the top of that list monthly. A rule generating 4,000 auto closed alerts a month is not a success story, it is a rule that should be rewritten, and the tooling should be making that obvious rather than making it comfortable.
Coverage design is where this interacts with staffing, and it is usually decided by accident. A team of four cannot staff 24 hours without either a rota nobody sustains or an arrangement with a partner for overnight hours. Automation changes the arithmetic at the margin, since fewer things need waking someone for, and it does not change the fact that somebody must be reachable. Decide explicitly which hours are covered by people, which by a partner, and which by nobody, then write the escalation path for each band. Teams that never make that decision discover the answer during an incident at 4am, which is the most expensive possible way to learn it.
Ingest pricing decides your detection surface before you do
Most platforms price by data volume, so the practical question of which logs to collect gets answered by budget rather than by threat model. Teams drop verbose sources such as DNS or endpoint process telemetry, and then discover during an investigation that the evidence they need was never collected.
There is a reasonable position that collecting everything is wasteful, and it is true that a large proportion of ingested data is never queried.
The productive response is tiering rather than exclusion: keep high value sources hot and searchable, push bulk sources to cheap storage that can be searched slowly, and decide that split against the investigations you actually run rather than against the price list. Check the retention period on each tier against the time it typically takes to discover an intrusion, since a breach found at four months is investigated with whatever survived, and 90 day hot retention on the source that mattered means the early activity is simply gone. Whatever you choose, write down which sources are absent, because that list is the honest boundary of what your detections can see and it belongs beside your network security monitoring design rather than in somebody’s memory.
Where AI Genuinely Helps a Security Operations Team
Three functions return time reliably, and one commonly promised function does not belong on the list.
Enrichment before a human opens the alert
Gathering the context an analyst would collect anyway, the asset owner, its patch level and exposure, the user’s recent authentication history, reputation data on the indicator, related alerts in the same window, is the clearest win in the category. Nothing is decided, so a wrong enrichment is visible rather than dangerous, and the analyst starts from a complete picture instead of ten minutes of tab switching.
The limitation is data access. Enrichment quality depends on the tool reaching the identity provider, the asset inventory and the ticket system, and a product with thin integrations produces a thin summary that analysts learn to skip. Test enrichment against your own systems, not a demo tenant.
Presentation decides adoption more than completeness does. An enrichment block that runs to two screens gets skimmed, and the field that mattered was in the middle of it. What analysts keep is a short header answering the questions asked on every alert, which asset, whose account, is it exposed, has this fired before on this host, with the full detail one click below. Ask during a trial whether that summary layer is configurable, because the default ordering reflects the vendor’s idea of importance rather than your environment’s. The fastest signal that enrichment is working is analysts pasting it into tickets rather than rewriting it.
Drafting and testing detections
This is the underused application and the one that compounds. Translating a described behaviour into detection logic, checking the syntax, and estimating how often the rule would have fired against historical data is work that fits automation well and is exactly what a stretched team never gets to. A rule tested against 90 days of your own telemetry before deployment arrives tuned rather than needing tuning.
Keep the review human. A generated detection may be syntactically correct and logically wrong in ways that only show up as silence, which is the failure mode nobody notices. Pair every new detection with a test that proves it fires on the behaviour it targets, and treat a rule that has never fired as unverified rather than as evidence of a quiet environment.
Retiring detections deserves the same assistance as writing them, and gets far less. A rule that has not fired in a year is either protecting against something that never happens here or is broken, and the two are indistinguishable without a test. Run the inventory quarterly, sort by last fired date, and put every silent rule into one of three states: verified by a test, rewritten, or retired with the reason recorded. Detection sets that are never pruned grow past the point where anyone can say what is covered, which is the condition that makes the coverage question unanswerable when an auditor or a client asks it. Our look at AI agent use cases for security operations teams covers where that assistance is currently strongest.
Incident narrative and handover
Shift handover and incident write ups are genuine costs that fall on the most experienced people. Assembling a timeline from the alert, ticket and chat record produces a usable draft and removes an hour from the end of a long day, when write up quality is lowest anyway.
Label the draft as a draft until someone has checked it, since an incident record becomes the basis for later decisions and a confidently wrong timeline propagates. The role clarity in our guide to incident response team roles is what makes that review assignment obvious rather than optional.
What does not belong on the list
Autonomous containment across a fleet, meaning isolating hosts or disabling accounts without human confirmation, remains a poor trade for most organisations. The cost of a wrong isolation during business hours is immediate and visible, the benefit is a few minutes of response time, and few teams have the detection precision to justify it. Where it is used well, it is scoped narrowly to a small number of very high confidence detections rather than enabled broadly.
How to Compare AI Tools for Security Operations Center Teams
Five questions. Does every verdict show the evidence behind it in a form an analyst can check in under a minute. Which of our systems does enrichment reach, at what depth. Can auto close be scoped per detection rather than globally, and is auto close reported per rule. Does the product help write and backtest detections, or only process their output. How is ingest priced, and what happens to that cost if we add endpoint process telemetry.
That last question is worth asking explicitly, because it determines what your detection strategy can become rather than what it costs today. Teams validating the whole arrangement should also read our notes on penetration testing tools, since an independent test is the only honest measure of whether detections fire on real behaviour, and our overview of what a security operations center is and whether you need one sets out the decision that precedes any of this.
Frequently Asked Questions
Can AI replace tier one analysts?
Not in practice, and framing it that way produces bad purchases. It removes the mechanical part of tier one, the gathering and the formatting, while judgement about whether something is genuinely bad in this environment stays with a person who knows the environment.
What false positive rate should we accept?
Per detection rather than overall, and the honest threshold is whatever rate still results in analysts reading the alert. Once a rule is being closed unread, its rate is too high regardless of the number, and it needs rewriting or retiring.
Should alerts ever close automatically?
Yes, for narrowly scoped detections with a measured track record, and with per rule reporting on volume and confidence. Enable it per detection after reviewing that rule’s history, never as a global setting on day one.
How do we keep ingest costs under control?
Tier the data. Keep the sources your investigations actually use hot and searchable, move bulk telemetry to cheaper slower storage, and record which sources you excluded so the gap in coverage is known rather than discovered mid incident.
Is a managed service better than building this in house?
It depends on whether you can staff coverage around the clock and retain detection engineering skill. Many organisations of this size get better outcomes from a managed service, provided they retain enough internal capability to judge what the service reports.
The Team Behind This Guidance
Our team runs detection and response for clients and inherits environments where alert volume has been managed by ignoring it, which is the common state rather than the exception. What changes outcomes is putting engineering effort back into the detections, and using automation to make that work possible rather than to make the queue feel smaller. Matt Rosenthal, our chief executive, keeps the practice focused on operational resilience for growing firms, which in security operations means a team that investigates fewer alerts and trusts every one of them.
Start By Counting Your Noisiest Ten Detections
Pull the alert counts per detection for the last 90 days and look at the top ten, then check what proportion of each was closed as benign. That table usually shows a small number of rules generating most of the volume, and it is the most actionable artefact in security operations. Fixing three rules can do more for your queue than any product in this category, and it also gives you a baseline against which to judge what a tool genuinely adds.
If you want a second opinion on that table, or help deciding which detections to rewrite before you buy anything, book a free strategy call with our team.
