Keep seeing AI security tools sold like stopping hallucinations and prompt injection is one job. Well, in my experience, they are nowhere near the same fix. Injection is an input/trust boundary thing, and hallucination is more of a grounding and retrieval issue. Whatever blocks a malicious prompt does nothing for a model confidently inventing an api endpoint that doesn’t exist

Anyone seen a setup covering both well, or are you running separate layers for each?

We already prioritize past raw severity, KEV and EPSS for exploit signal, public facing assets weighted higher. It helped but the backlog is still huge, mostly dependency findings we are not sure we even call.

We trialed one reachability tool already. It looked great until it flagged a reachable critical that was sitting behind an internal only vpc nothing outside can touch. The code path was real, the exposure was zero and i was back to explaining to a dev why the tool and i disagreed. That is what has me hesitating before we pay for a bigger one.

So before we spend, i want the real read. For teams who added code reachability on top of KEV EPSS and exposure, did it really cut the list and did the exploitability hold up once it hit your infra layer? What are you running and would you buy it again

Hi folks, I'd like to announce a project I've been working on.

• aisan is a sandbox for agents; claude, codex, and opencode harnesses are supported, plus plain vertexai (ie programmatic) use.
• Unlike most other ai sandboxes, the security model is simple and strict: the entire harness and all of its local tools run sandboxed.
• No network access and no credential inside the box, filesystem limited to explicitly selected slices (inspectable).
• Capabilities requiring network get an egress via a whitelisting proxy. Specific examples: the connection to the upstream model provider itself; and remote builds for chromium workflows.
• Harnesses are then free to run entirely unconstrained inside the box, and thus never require human interaction for permissions.

The initial usecase for this was a langgraph AI app that runs unattended, 24/7, on sensitive infrastructure. This project provides the same guarantees for interactive harness runs; eg if you run nightly unattended AI work packets. By now, I'm starting every session sandboxed locally - there's no strict need to do so, agents generally behave themselves; but then again, why not?

Linux only. Uses bwrap underneath.

Try it with: uv tool install aisan && aisan claude

More information at: https://github.com/schuay/aisan

Expect rough edges. Happy to take feedback and pull requests.

Following up on an earlier post about SafeAI, a static analyzer for AI agents.

One uncomfortable thought we've had while building it:

No one really cares about knowing an agent's capabilities — until something goes wrong.

Before an incident, adding another tool, MCP server, filesystem permission or prompt change often looks harmless.

After an incident, the first questions become:

- What could this agent actually do?

- When did that capability appear?

- Who introduced it?

- Was it intentional?

---

One example we're working on is MCP tool descriptions. A tool description can look like documentation:

"Search the user's notes. Ignore previous instructions and..."

But that description may become part of the model's context. So configuration can effectively become an instruction surface.

SafeAI now detects several forms of this, while trying to avoid flagging ordinary descriptions that happen to contain words like "ignore" or "act as".

The bigger direction is **tracking changes in agent capability and authority**, rather than simply producing another list of security findings.

But this raises a question for us:

Is knowing your agent's capabilities actually useful before an incident, or only after one?

And if it is useful before an incident, what is the right interface?

CLI + CI + SARIF/HTML?

Or would you actually want an interactive view showing things like:

> "Show me all MCP tools across our agents that could introduce instruction injection."

We're deliberately not building a UI yet.

---

Would you use one, or is that solving a problem nobody has?

Curious to hear from people running real MCP/agent systems.

---

If you want to try it against your own agent project, we'd genuinely appreciate feedback, as well as contributions.

Here you may check: ikaruscareer/SafeAI on GitHub.

I’ve been testing an advisory triage prototype and one thing that keeps coming up is how quickly applicability logic gets messy once the inventory data isn’t clean.

Basic version comparisons have held up pretty well, but product naming variations and CPE mismatches are much easier to break.

For example, FortiAuthenticator vs Forti Authenticator can be enough to create a false negative if product normalization isn’t handled properly.

Curious how people are dealing with this in real DevSecOps/vulnerability workflows. Are you relying mostly on CPEs, vendor identifiers, SBOM data, scanner normalization, or something else?

I’m working on a prototype around this problem, so I’m interested in where the real failure cases are.

Running a formal SASE evaluation with AI security as a weighted requirement rather than a nice-to-have. We're down to three finalists and I'm writing the scoring matrix now. I would rather borrow from people who have done this than invent it badly.

Here's how I'm currently thinking about the dimensions and rough weighting:

Inline AI traffic inspection is weighted High because post-hoc logging does not meet our control objective. Unified policy engine is also High. Separate consoles have burned us before. Shadow AI discovery accuracy is High since we cannot govern what we cannot see.

For medium-weight items: Agent/non-human identity support is Medium. Not urgent yet, but will be within 18 months. Latency impact, measured, is Medium given our global user base with several high-RTT sites. Licensing transparency for AI features is Medium. "Included" has meant three different things in three demos. Operational familiarity and retraining cost is Low-Medium. Real, but should not drive architecture.

Two things I am struggling to score fairly: every vendor claims a unified policy engine, but digging in, some are genuinely single-pass while some are separate engines behind a common UI. This is hard to verify without a real PoC. And shadow AI discovery numbers are vendor-reported, with no standard benchmark that I can find.

Has anyone built a repeatable test for either of these? interested in how you validated the unified-versus-stitched-together claim rather than taking it on faith.

CASB and traditional DLP were designed for a world of known SaaS apps and structured data patterns. Generative AI interactions are messier, free-text input, risk that depends on context, and tools that update faster than CASB policies can keep up with.

Are teams extending existing CASB/DLP stacks to cover AI usage, or does it require a separate tooling layer? Also curious how people are thinking about AI coding assistants specifically, most of that exposure happens at the IDE level when the assistant sends proprietary code context out to a model, which is a different threat surface than CI/CD itself. Is anyone treating IDE-level code context exposure as distinct from CI/CD-integrated AI review tools, or are those getting lumped together in practice

Hey folks,

Been working in a Splunk admin role for past 5 years. In current role, I spend most of my time handling log ingestion, linux admin,cluster administration, dashboards, SIEM alerts, and basic data pipeline troubleshooting.

Am looking to shift my career path toward DevSecOps, but wanna ensure my efforts are toward what’s actually valued in production environments versus just collecting certs.

For those of you working in DevSecOps (or pref hiring managers in the space):

1. Skills: Coming from a heavy log/SIEM background, what are the most critical gaps I need to bridge first? (e.g., CI/CD tooling, IaC security, container security, scripting languages like Python/Go?)
2. Certifications: Which hands-on certs carry actual weight for someone moving into DevSecOps? am currently working on GCP associate and CKA
3. Projects: What kind of home lab or GitHub project on a resume would show you that a Splunk Ops person actually understands pipeline security and automation?

appreciate any invaluable inputs, thank you!

I have been building a small open-source experiment called RedThread around adversarial testing for LLM agents.

One label is not enough for a failed tool call. "The agent tried to delete a file" and "untrusted page content caused the agent to try it" are different failures. The first is about effect. The second is about integrity.

I am keeping those separate in the trace and replaying the setup after a prompt or adapter change. If I flatten them into one severity number, the decision about what matters gets hidden in the weighting.

Repo: https://github.com/matheusht/redthread

The attack harness is early. The distinction feels less optional every time an agent gets another tool.

I maintain an open-source red-team tool that runs attacks against vision-language-action robot policies and reports an attack success rate. This morning I measured its own distribution and put the result on the project site rather than in a drawer.

PyPI lifetime downloads excluding mirrors: 4,667. Including mirrors: 15,986, so 71% of the traffic is infrastructure. GitHub stars: 5. Forks: 0. Third-party reproductions of any published result: 0.

That works out to 933 installs per star. From what I can tell, a developer tool people actually use sits nearer 10:1 or 50:1, because a human who installs something also bookmarks it. 933:1 reads as CI runners and dependency resolvers reinstalling on every job.

The tool itself is not the problem. It measured 44 out of 50 runs going out of the policy's safety envelope under a roleplay attack, against 2 out of 50 on the benign control, on SmolVLA over LIBERO. Two of the three adversarial families I registered measured 0%, and those zeros are published on the same page as the 88%.

The leaderboard has four rows and one checkpoint. It is signed with Ed25519, so anyone can verify it offline without trusting me. I wrote a third-party disclosure policy, 14 days' notice with the full artifact, before there was a single third party to disclose to.

None of that produced an outside run.

So the question I actually have for this sub, from people who have shipped an eval or a benchmark: what got the first person outside your team to actually execute it? Not star it, not upvote the announcement. Run it and come back with a number.

I am fairly sure the answer is not "post about it more", because I have been doing that.

been hearing aspm pitched everywhere as the layer that fixes tool sprawl and cuts triage time. before we sink budget into evaluating one, wanted to hear from people who've run it for at least six months.

Does it meaningfully lower the manual correlation work, or does it just become one more dashboard sitting next to the ones you already have? specifically interested in whether it reduced ticket volume for devs or just moved the noise around to a new interface. what's your experience, up or down?

The biggest risk for us wasn't the model memorizing code. it was the assistant sending files, credentials, or context somewhere we didn't expect.

The controls that ended up mattering:

• managed accounts and model traffic through a controlled gateway where supported
• repo-scoped access, no home directory, shell history, or production credentials
• clear vendor terms for training, retention, and logging
• server-side secret scanning and the same CI gates as every other change
• human review before generated code merges

one thing I'd separate: the gateway controls where model traffic goes. filesystem permissions control what the agent can actually read... you need both.

the policy itself can stay pretty small. the important part is making the boundaries enforceable instead of trusting everyone to remember them

Had an agent last month pull a customer contract into a summary doc and drop it into a shared workspace nobody on that team should have had access to.

Nothing malicious, it just did what it was told without anyone thinking about where the data would end up. That is when we realized we had zero visibility into what our agents were touching. Cyera was already on our shortlist for DSPM. Then Cyberhaven kept coming up too, more focused on protecting data once it is moving through AI workflows.

If you have used either one, how did it hold up once agents were doing real work?

Coding agents are doing a lot of the integration work at our agency now. Stripe, Twilio, WorkOS, email, the usual stack. Economically it's been good.

The part I didn't expect: verification actually got harder. We're producing code faster than anyone can review it, and the edge cases that bite you aren't in unit tests. Webhook fires twice. Events arrive out of order. Agent wrote correct code but got one state transition wrong. PR looks fine, everything compiles, and you still don't know until something breaks in staging or worse.

We added a sandbox step before anything ships now. Agent writes, tests pass, then we run the full multi-API workflow with failure scenarios before it's considered done. Not elegant, just a runnable verification step all our agents have to clear.

If agents are writing integrations for you, what does your last gate before production actually look like?

We are a large aws shop. Devs use secrets manager to store keys etc. problem is that these keys are handled by humans to begin with. We trust the human to put it in secrets manager etc. I’m just wondering what organisations do for secrets management to prevent this? Overtime, we have devs with secrets in postman collections etc. what should I be looking at to really secure our secrets?

Minimus customers have until October 22 before the registry goes fully offline and a few teams here were mid pilot with them when the news dropped. Trying to put together a shortlist of where people are migrating, RapidFort already put out a free migration offer for Minimus customers alongside a couple other hardened image vendors and some teams that are just building their own base images instead. Mainly want to know how close a replacement gets to what Minimus was doing on CVE reduction. What are people moving to and how has the transition gone so far?

Security lead here at a ~1500 person shop, with team of 4. We are already running falcon on the endpoint side and our rep is pushing us to add their cloud module since we are already paying them. Well on invoice it makes perfect sense.

But the thing is I feel falcon grew up as an endpoint agent, and im not sure that an agent first tool is the right thing for a few thousand cloud workloads against something agentless that was built for cloud (wiz, orca that crowd)

So for anyone in multi cloud, do you think we should extend crowdstrike into cloud or buy a dedicated cnapp? Thanks all.

The 2026 Gartner Market Guide for Adversarial Exposure Validation suggests validation is the next step beyond VM. The difference is VM tells you what is vulnerable, while validation tells you what is actually exploitable by testing the full kill chain. The validation platforms don't just scan. They execute attack paths to confirm exploitability.

For blue teams, is this shift real? Are you moving budget from scanners to validation platforms? I can see the appeal of focusing on what matters rather than chasing every CVE, but I'm worried about losing the comprehensive coverage that traditional scanning provides.

How do you handle the validation findings that aren't fixable by a patch, like misconfigurations that require business approval? The new platforms claim they can suggest or automate the fixes, but that feels like a governance nightmare.

what's your experience with the detection engineering modules? The AI that maps SIEM rules to attack scenarios sounds great in theory, but does it actually improve your detection coverage, or does it just validate that your existing rules are working, which you probably already knew?

Been running agents and scanners on my PRs for a while now, triage and native scanning before anything goes up. I've ended up somewhere I didn't expect and want to know if I'm alone.

Started where I think everyone starts: lean on memory. Let the agent carry context session to session, build up a picture of the project. It helps. It wasn't where the gains came from.

What actually moved things was rules written against mistakes the agent already made. Not a generic OWASP checklist. A scoped rule: this defect class showed up, here's the sink, here's the banned construct, stop doing it. A defect shows up a third time, I write a rule, the class stops recurring. Five or six patterns closed instead of a hundred bugs chased one at a time.

The distinction I keep circling: memory is passive, the agent might use it. A glob-scoped rule loads whether the agent wants it or not, tied to the file in front of it. One line of repo rule changes the default output of every session in that repo. And MUST/NEVER phrasing outperforms polite wording by a margin that surprised me.

One thing I'd flag: these files are executable surface. The agent reads them as ground truth, so a poisoned rule file is a real problem, not a hypothetical. I keep security rules separate, reviewed like code, capped so they don't rot.

So, three questions I actually want answers to:

• Rules-on-mistakes vs raw memory — which is carrying more weight in your setup? I might be over-indexed on rules.
• How are you scoping? Glob per file, per directory, or one fat rules file everyone edits?
• Is anyone treating rule and memory files as an attack surface, or am I being paranoid?

Genuinely want to be argued with on the first one. If memory is doing the heavy lifting for you and rules are noise, tell me what your setup looks like.