Blog · 21 November 2026

Before You Buy an AI SOC, Check What Your Data Throws Away

Almost every vendor sells an . But old rules in your data pipeline decide what it gets to see. Five plain questions CISOs and CIOs can ask their team or provider.

By Marcus Bengtsson, CTO, Sciber

Almost every security vendor now has an "AI SOC" to sell you. A SOC, or security operations centre, is the team and tooling that watches for attacks and responds to them. The AI versions promise chat assistants for your alerts and agents that run investigations on their own. Some of it is genuinely useful.

But there's a quieter question that comes first, and it's easy to skip when budgets are tight: what does that AI actually get to look at?

The decision that happens before anyone looks

Albert Caballero, Field CTO at Cribl, made this point well on LinkedIn recently when announcing his company's release of an open model called cribl-decision-1.0. His argument is simple. Before any analyst or AI agent investigates anything, a huge number of decisions have already been made about your security data.

Your systems produce a constant stream of records: logins, file access, network connections, errors. Together this is often called telemetry, which just means the activity data your IT environment generates about itself. Before it reaches the tools that look for attacks, it passes through what's called a data pipeline. That's the plumbing that collects the records and decides where each one goes.

At that stage, the pipeline decides what gets kept, what gets dropped, what goes to the expensive search platform, what goes to cheaper storage, and what gets masked for privacy. As Caballero puts it, those decisions decide what your detections can see. And in his view, most of them are still made by fixed rules that someone wrote years ago.

That isn't anyone's fault. Many log platforms charge by how much data you send them, so teams filter to keep costs under control. The rules made sense at the time. But a lot can change in a few years: new cloud services, new identity setups, a new business system, a factory that got connected to the office network. The rules often don't change with them.

His line sums it up: "The AI SOC gets the headlines. The AI pipeline decides what the AI SOC gets to see."

Why this matters more than the AI SOC on top

If a log never arrives, no detection rule can fire on it and no AI can reason about it. A smarter analyst, human or machine, can't investigate a signal that was quietly dropped upstream.

For a mid-sized company, and especially a manufacturer running mostly on Microsoft, this comes down to getting the foundation right before paying for what sits on it. AI in security works best on top of a modern stack with good telemetry, solid identity, and tools that can actually feed and act on AI. Buying AI first and checking the data later gets the order wrong.

Caballero also argues that the pipeline itself is where AI could help, but not the chatbot kind. A chat assistant is too slow and too expensive to sit in the path of every log record, and you can't set a clear threshold on a paragraph of text. Instead, he points to small, fast models that make narrow decisions, such as keep or drop, and return a confidence score a team can act on. Cribl describes cribl-decision-1.0 as a general-purpose classification model, meaning it picks one answer from a fixed set of options (such as route or drop) rather than writing text. It is also open-weight, meaning anyone can download it and run it on their own hardware. Whether that approach suits your environment is a question for your team or provider. The bigger point holds whatever tools you use: someone should be deliberately deciding what your detection can and can't see.

Five questions to ask your team or provider

You don't need to be technical to start this conversation. These questions work for an internal IT team, a managed security provider, or both.

  1. What do we actually collect today? Ask for a plain list of the main sources: Microsoft 365, identity and sign-ins, servers, firewalls, laptops and other user devices, and any production or OT systems (the operational technology that runs machines and plants).
  2. What do we drop or summarise, and why? Every filter should have a reason you can understand, whether that's cost, noise, or privacy.
  3. Who owns those rules, and when were they last reviewed? If the answer is "nobody" or "when we set it up," that's useful to know.
  4. What can't our detection see? Ask for the known blind spots honestly. A good team or provider will have an answer.
  5. If something is dropped, is that decision recorded? After an incident, you'll want to know whether the evidence never existed or was filtered out before anyone could look.

None of these questions require buying anything. They just make sure that whatever you buy next, AI or otherwise, is working with the right data.

Where to start

If you already work with a managed detection and response provider, an MDR service that watches your environment around the clock and acts on threats, these questions belong in your next service review. If you don't, they're a good way to compare options.

At Sciber, this is how we think about managed detection and response: the foundation and the data come first, and AI earns its place on top of them. Whoever you work with, start by asking what your detection can see today.