"""
Beyond Zero shifts the trust boundary from the application to the action being performed on a piece of data in realtime—and from after-the-fact investigation to in-the-moment evaluation and containment. It augments BeyondCorp’s foundational identity with a “brain” capable of reasoning about the context and intent of a specific request in realtime.
"""
Doesn't this simply shift the attack vector? Compromising this overlord brain now becomes a new target.
For example, encryption at rest or in transit essentially eliminates attack vectors. The possible attack avenues necessarily shift as a result, but only because an avenue was blocked.
That's very different from e.g. adding a layer of protection around something insecure, where the insecure thing remains insecure inside the protection, which I think is what the other commenter was imagining.
Similarly, memory safe languages (including most GC languages, not just a certain language beginning with R) eliminate entire classes of security hole. Again, the possible attack vectors necessarily "shift", but that doesn't capture the fact that you've entirely eliminated a class of attacks.
The same goes for eliminating unnecessary services, firewall holes, etc.
None of these are specifically trying to "shift it to something to something that is more difficult to compromise." They're entirely blocking attack vectors, and the strength or weakness of other parts of the system aren't really a factor.
This is a layer on top of normal security, so this isn't a matter of compromising the "brain" instead of RBAC, etc, but in addition to.
Attribute-based access control is already a thing. User X logs in the US East between the hours of 6am and 6pm. If User X logs in from Russia at 3am, deny access. This seems like an evolution of that pattern
Organizations today run every SAAS under the sun. Files and data are wired up in a hodgepodge with security usually being an afterthought. There are companies like Big Banks that do monitor all network traffic including traffic to their provisioned Google Drive or Clickhouse. A lot of companies only have logs which they feed to an alerting system which is after the fact.
At a high level it's possible to alarm on data egress, but at the same time it depends on the type and degree of compromise we're discussing and how the attacker exfiltrates the data.
If an intruder isn't detected, they could slowly egress the data to borrowed residential IPs, rather than a giant multi TB transfer that might set off more obvious alarms. For a large enough organization with substantial outbound traffic to start, it can become incredibly hard to distinguish from legitimate activity.
these are just fluff words, not a revolutionary change.
zero trust helped a lot because it forced vendors to stop relying on people magically doing the right thing ina centralized system. it decentralized boundaries.
this paper just says you still need deterministic boundaries and access control, not just have an llm say safe or unsafe. cool thanks google haha.
ps: hi guys, i know yall reading. sometimes its not the concept, its the implementation. else active shield or whatever would have worked too. not everyone can be the T7-9 thinker if implementing is seen as "for peasants"
Google already has a system where data and access control are combined as part of shared docs. I think Zanzibar for docs themselves on GDrive. Not sure the name of the system that manages a portion of a document in a shared doc which colocates data and access data. This seems like a next generation of that?
One partial answer is to not make it a single brain. If the reasoning layer that evaluates a request is independently audited by a second pass at lower temperature with an explicit skepticism mandate, compromising the evaluator requires compromising both passes simultaneously — and they're running the same local model with different system prompts and no shared state between them. Doesn't eliminate the attack surface but raises the cost considerably compared to a single inference gate.
My goodness, the security layer looks like it could be more complex than the applications themselves.
And if this security agent is wandering around the IT system gathering all of these details about access and identity and business process, who watches the watcher? How does that thing build and maintain trust?
Honestly I think non-malicious odd behavior is under-weighted when it comes to AI agents. Even the example in this paper is about someone suspiciously accessing sales data when "why did you do that" often comes down to something in the model's training that fired as a reflex
"Many researchers have made demos along these lines:
An agent is asked to check a webpage like example.com
The webpage asks for a name to proceed further
The agent calls example.com/evil?myname=John, thus sending the user’s name from the context window to the external server.
In practice, however, these elaborate ‘confused deputy’ exfiltration attacks seem rare compared to widely-reported data loss incidents.
The risk of undermining the user’s interests through clumsiness deserves at least as much scrutiny as the risk of leaking secrets."
So while the idea of shifting the permission boundary from the app level to the action level makes sense, what we should also have is some 'failsafes', eg. if the action says 'delete' then maintain a rollback window, if the action is 'send an email' then maintain an events log. Preparing for AI agents means expanding auditability and reversibility in software
> Preparing for AI agents means expanding auditability and reversibility in software
While I think that makes sense, I also think more controls are a good solution. That is, prevent access to the sales data without escalation, probably to a human, but possibly to another AI. The issue there is twofold:
- if you start with least privilege, the agents become less useful
- need to balance escalation with frequency otherwise it's just another version of MFA fatigue
Agree that auditability is important because otherwise you don't know what you don't know.
Maybe this is more interesting in the context of the HuggingFace incident? Suppose you have an AI and some access controls for what it can do. If the access control is too broad or has a bug, can you still detect suspicious activity and slow it down long enough for someone to be alerted?
If this is interesting to you (as it is to me) but you want to quickly digest it instead of read the paper, Claude + SmallDocs[1] converted it into a slideshow which serves it to you in bitesized ideas:
not sure if you made the app or not - I like it a lot, only feedback is it should be a bit more intuitive on how to start the slides, I couldn't figure it out and old ppl like me don't always know
I think he means that you need to click the little TV icon in the top right of the embedded slides. Then you can use the arrow keys to advance them, yes.
Am I undertanding this correctly? The idea is to have ultimately an AI decide if I can have access to a resource based on dynamic inference, identity , intent and service signals that can easily be manipulated?
Unless I gravely misunderstood the text, this seems like a terrible idea (fancy non-scifi, but still terrible)
This would be paired with deterministic controls. So you have, for example, "engineers can only read from the database, and only after they've perform a 2FA" but then there's context like "why is this engineer reading from the payments table when their IP is in a weird location and they're supposed to be on PTO?" and perhaps that's something a model decides.
It's possible to turn that second thing into a sort of "risk score" but it's very hard and a model is going to potentially be better at it.
This is going to make things worse for all kind of use cases that are legitimate but seem non-standard or marginal.
Heck, many websites I visit on the web cannot understand why I, a Ghanaian living in Ghana, might be interested in the service offered or the information therein. I am sometimes blocked for no good reason.
If you are in a third world country, the web is extra hostile. This is going make things worse.
The engineer could be fixing an emergency issue while on vacation. For financial data, that would hopefully be an auto-deny anyway until someone on-site whitelists their IP. And this is if they aren't using VPNs. Adding AI to this party feels like it wouldn't really help.
Finally, we've invented a way to abolish all the reliability and speed of discrete computing. We're well on our way to re-inventing the layers of bureaucracy and red tape the tech industry had disrupted.
I can't wait to fill out a form describing why I want to do something I have permission to do but the AutoBureaucrat5000 says no anyway.
The weird IP would already be handled by existing zero-trust controls. Users in Entra ID (for example) can be assigned a risk score already based on deterministic factors. On a managed device or not, which MFA methods they have registered, eligible for any privileged roles, where they are, impossible travel detection, etc. You can even require human approvers.
This reads to me to be more for continuous behavioral monitoring once the access is gained via the deterministic controls. You wouldn't leave "Can person X access resource Y" up to the AI model, that's already decided based on the existing rules. Where the model comes in is "Is person X behaving in an expected way while using resource Y." Like, downloading a bunch of data when they've never done that before, might get flagged for either a session revocation, or a human review, or prompt for additional authentication, etc.
Non of it really matters if the end product is just going to be ignored because there's zero people who want to be the guard in the panopticon and therefore, it'll still be given to an AI to watch and make it's dumb decisions about how trust worthy you are to do X, Y and Z.
Even if the middle is deterministic, if one end is just going to be lazily hooked up to an AI, it's the shitty dystopian future.
That just gave me shivers down my spine of people eventually becoming so lazy that the AI will decide who to fire based on a massive amount of circumstantial data that very likely has tiny cumulative errors that will lead to classifying your best personnel as a bums and liabilities.
Just to make my point: can I really trust humans to keep up with the required identity data that will give me enough "credibility" so the AI will give me access?
Let's say I had a promotion, who changes my title in the system, who changes my responsibilities and my place in the org chart, more importantly, will they do it or is <HR_NAME_HERE> on leave and forgot? those are data points required by the agent to determine if I'm "good enough" to access a certain resource.
What if... someone spoofed my address and did a flood in one of the resources that are lateral to what I'm allowed to access (let's say I don't have access to company sales but I do to department sales and the attacker floods company sales with requests under my address), would the AI determine that I'm a high-threat actor and not allow me to access legitimate files going forward?
Will exceptions be made by humans? In which case we go back to human-managed permissions.
Sorry, I might be barking up the wrong tree but I think these are questions that are not meant to be solved during implementation. And they add to what @firasd said about legitimate-but-odd behaviour
I think the dynamic risk signals is already an assumed part of the system. The proposal here seems to be shrinking the trust boundary from "can Alice use Google Drive?" to "should this specific read/write/export API call on this specific resource be allowed right now?"
So yeah you're right in that it does add more probabilistic randomness just by virtue of changing the boundary of when the permission gate kicks in
Correct. In existing zero trust identity, we already have deterministic risk signals.
In Entra where I work we already check things like "Is this person on a managed device? Is it compliant? Where are they? What MFA methods do they have registered/did they use?" on top of existing RBAC, etc. and its continuously evaluated. Entra watches for leaked passwords, assigns risk scores, etc. and you can make access decisions based on user risk or sign in risk, force password changes, require different MFA methods depending on the resource and the risk level, etc.
"Should this API call on this resource be allowed right now?" is mostly already determined by the above.
Where I see adding AI into the evaluation is to watch for unusual behavior that's not picked up by the deterministic signals. "Alice is trying to download gigs worth of data from the company file share, however she has never done that in the past, and there hasn't been any recent role/job changes" and so the LLM flags it or denies the request, or pushes it for a human approver, etc.
This is exactly why big tech companies love AI. It's not because it will take every job - it probably won't. It's not because it will achieve AGI. It probably won't. It's because it threatens security with more subtle, advanced, and automated exploits so that you'll HAVE to rely on them for countermeasures.
I've been working on an open source ZTNA project/product https://github.com/octelium/octelium for many years and I am actually very open to the idea. Machine learning has already been in use for years when it comes to anomaly detection. Maybe modern LLMs can be used to automate access control, partially or fully at enterprise scale in the future. The idea is to have a policy engine that's controlled by AI that uses a mix of long-term semi-static info such as group memberships and permission boundaries, short-term info such as behavior (e.g. access logs) and current session's authentication strength level (AAL) such as FIDO as well as near real-time external information (e.g. IdP, SIEM, threat intelligence, on-call management) to dynamically adjust users' access control to build up some sort of a context for each user and each session that can increase or decrease permissions without having to manually add policy rules or going through request/approval flows, while optimizing for the main objective (i.e. minimizing access grants for every subject to any resource under whatever context unless when it's necessary). This can be unified for both humans and non-human identities, including agents.
Good, realistic take. There's a lot of context that can be drawn on to make better decisions and look for weird.
And then agents come along and some people want their agent to be able to do anything they can do with zero friction. That usually lasts until the first time something goes wrong ;)
Actually zero trust can be seen as an optimization problem, you almost certainly want to globally deny every single request, even for authenticated users under all circumstances to all resources, unless when "necessary", "necessary" here is the key because it needs to be as dynamic and as fine-grained as possible, context-based and scalable. For small businesses, RBAC and ACL rules can be fine, but for enterprises with thousands or tens of thousands of employees, contractors and now AI agents which need varying permissions to different resources at different times, you would have to define thousands or tens of thousands of rules that might conflict with one another and are hard to maintain. ABAC and policy-as-code can mitigate to a certain extent providing dynamic context-based fine-grained access control. But if you could gather as much information as possible about every user and every session, combine it with some boundary permissions and static rules, and feed it periodically to some reasoning engine that is tasked with dynamically increasing/decreasing permissions for each session based on all such info, you can keep your manual/static access control policies to a minimum and maybe even eliminate JIT in some cases. The real question is whether modern LLMs and transformers are the correct and optimal architecture for this problem in the first place.
I don't think the point here is to fully replace deterministic with non-deterministic-based access control. The point is to combine the traditional manual/coarse-grained deterministic access control with an additional layer of dynamic, fine-grained non-deterministic access control.
I think LLMs may have a role in security posture, specifically flagging/identifying potential threats for human review. But a Zero Trust/Access Controls should be a HARD boundary, not an inconsistent one.
The problem we have right now is that there are some legitimately interesting ideas out there for things we could be using LLMs for, but we also have a ton of "I have a hammer, and everything looks like a nail" going on too.
> The problem we have right now is that there are some legitimately interesting ideas out there for things we could be using LLMs for, but we also have a ton of "I have a hammer, and everything looks like a nail" going on too.
It's not a problem as much as a phase the world is going through. LLMs are a technological breakthrough in the same generality class as the Internet, or possibly electricity, and in both cases the world went through a phase of attempting to apply the newfound invention to literally everything. It's a necessary phase, when a technology obviously could be useful for everything, but it's not obvious whether it will in practice.
I don't think Google are advocating for removing the deterministic controls, are they?
Sounds like adding the LLM in would be on top of the existing deterministic controls.
Existing controls handle the "Should you be able to access this resource right now?" the LLM handles "Is this user behaving as we expect them to while accessing this resource?"
Agreed; I don't think we want security controls to devolve into an argument between LLMs. Attacking and defending LLMs operate in an ecosystem, and are (hopefully) limited by constraints. Defending LLMs will be able to rely on and evolve those constraints. Attacking LLMs want to find a way around them.
Who said anything about an LLM doing it? Credit card companies have been using AI quite well for fraud detection for years before LLMs were released to the public.
I wonder: Could your data be more secure if you didn't give its custody to a company like Google? i.e. is there actually a sound basis that an alternative like taking control of your data and securing it yourself, would be more secure that the "banks exist to protect you" type of fear-mongered custodian idea against the self-ownership of things?
Oh yeah, a company whose business model is taking everyone’s data and selling it is going to help me secure my data. I guess there’s one born every minute…
There is a massive difference between Google for enterprise customers and Google for consumers.
The consumer offering is massively subsidized by ads and will use your data for ad placement, although they still never sell your data because that would hurt their business.
The Enterprise offering guarantees you contractually that they never touch your data.
Tell me where I can buy it then. Because it’s not listed there. And the approach the paper describes requires tight integration between applications and a security provider, which I don’t think exists yet.
""" Beyond Zero shifts the trust boundary from the application to the action being performed on a piece of data in realtime—and from after-the-fact investigation to in-the-moment evaluation and containment. It augments BeyondCorp’s foundational identity with a “brain” capable of reasoning about the context and intent of a specific request in realtime. """
Doesn't this simply shift the attack vector? Compromising this overlord brain now becomes a new target.
All security measures "simply shift the attack vector." The idea is to shift it to something that is more difficult to compromise.
This is oversimplifying.
For example, encryption at rest or in transit essentially eliminates attack vectors. The possible attack avenues necessarily shift as a result, but only because an avenue was blocked.
That's very different from e.g. adding a layer of protection around something insecure, where the insecure thing remains insecure inside the protection, which I think is what the other commenter was imagining.
Similarly, memory safe languages (including most GC languages, not just a certain language beginning with R) eliminate entire classes of security hole. Again, the possible attack vectors necessarily "shift", but that doesn't capture the fact that you've entirely eliminated a class of attacks.
The same goes for eliminating unnecessary services, firewall holes, etc.
None of these are specifically trying to "shift it to something to something that is more difficult to compromise." They're entirely blocking attack vectors, and the strength or weakness of other parts of the system aren't really a factor.
Doesn’t encryption shift the attack vector to the key and/or the method?
What? Encryption has never eliminated attack vectors, it just shifts them to weaknesses in the implementation
This is a layer on top of normal security, so this isn't a matter of compromising the "brain" instead of RBAC, etc, but in addition to.
Attribute-based access control is already a thing. User X logs in the US East between the hours of 6am and 6pm. If User X logs in from Russia at 3am, deny access. This seems like an evolution of that pattern
Before zero trust, once you logged in, you have access to the entire kingdom of resources/files/APIs etc.
To me, this sounds like zero trust version 2.0. The "brain" challenges agentic AI trying to access resources it normally doesn't access.
Personally, I like the name "Beyond Zero" because it isn't oxymoronic like "Zero Trust".
I have a simple question. When a company gets compromised, how come there are no alarms when TBs of data gets egressed?
Organizations today run every SAAS under the sun. Files and data are wired up in a hodgepodge with security usually being an afterthought. There are companies like Big Banks that do monitor all network traffic including traffic to their provisioned Google Drive or Clickhouse. A lot of companies only have logs which they feed to an alerting system which is after the fact.
At a high level it's possible to alarm on data egress, but at the same time it depends on the type and degree of compromise we're discussing and how the attacker exfiltrates the data. If an intruder isn't detected, they could slowly egress the data to borrowed residential IPs, rather than a giant multi TB transfer that might set off more obvious alarms. For a large enough organization with substantial outbound traffic to start, it can become incredibly hard to distinguish from legitimate activity.
these are just fluff words, not a revolutionary change.
zero trust helped a lot because it forced vendors to stop relying on people magically doing the right thing ina centralized system. it decentralized boundaries.
this paper just says you still need deterministic boundaries and access control, not just have an llm say safe or unsafe. cool thanks google haha.
ps: hi guys, i know yall reading. sometimes its not the concept, its the implementation. else active shield or whatever would have worked too. not everyone can be the T7-9 thinker if implementing is seen as "for peasants"
Google already has a system where data and access control are combined as part of shared docs. I think Zanzibar for docs themselves on GDrive. Not sure the name of the system that manages a portion of a document in a shared doc which colocates data and access data. This seems like a next generation of that?
One partial answer is to not make it a single brain. If the reasoning layer that evaluates a request is independently audited by a second pass at lower temperature with an explicit skepticism mandate, compromising the evaluator requires compromising both passes simultaneously — and they're running the same local model with different system prompts and no shared state between them. Doesn't eliminate the attack surface but raises the cost considerably compared to a single inference gate.
My goodness, the security layer looks like it could be more complex than the applications themselves.
And if this security agent is wandering around the IT system gathering all of these details about access and identity and business process, who watches the watcher? How does that thing build and maintain trust?
Honestly I think non-malicious odd behavior is under-weighted when it comes to AI agents. Even the example in this paper is about someone suspiciously accessing sales data when "why did you do that" often comes down to something in the model's training that fired as a reflex
I wrote about this a few days ago https://firasd.substack.com/p/accidental-data-loss-in-claude...
"Many researchers have made demos along these lines:
An agent is asked to check a webpage like example.com
The webpage asks for a name to proceed further
The agent calls example.com/evil?myname=John, thus sending the user’s name from the context window to the external server.
In practice, however, these elaborate ‘confused deputy’ exfiltration attacks seem rare compared to widely-reported data loss incidents.
The risk of undermining the user’s interests through clumsiness deserves at least as much scrutiny as the risk of leaking secrets."
So while the idea of shifting the permission boundary from the app level to the action level makes sense, what we should also have is some 'failsafes', eg. if the action says 'delete' then maintain a rollback window, if the action is 'send an email' then maintain an events log. Preparing for AI agents means expanding auditability and reversibility in software
> Preparing for AI agents means expanding auditability and reversibility in software
While I think that makes sense, I also think more controls are a good solution. That is, prevent access to the sales data without escalation, probably to a human, but possibly to another AI. The issue there is twofold:
- if you start with least privilege, the agents become less useful
- need to balance escalation with frequency otherwise it's just another version of MFA fatigue
Agree that auditability is important because otherwise you don't know what you don't know.
Maybe this is more interesting in the context of the HuggingFace incident? Suppose you have an AI and some access controls for what it can do. If the access control is too broad or has a bug, can you still detect suspicious activity and slow it down long enough for someone to be alerted?
If this is interesting to you (as it is to me) but you want to quickly digest it instead of read the paper, Claude + SmallDocs[1] converted it into a slideshow which serves it to you in bitesized ideas:
https://smalldocs.org/s/2SH6FHiUK1mcym24Z8E37I#k=2Sk6c_IdKJL...
[1] I am the developer behind SmallDocs.
not sure if you made the app or not - I like it a lot, only feedback is it should be a bit more intuitive on how to start the slides, I couldn't figure it out and old ppl like me don't always know
Press the arrow key.
I think he means that you need to click the little TV icon in the top right of the embedded slides. Then you can use the arrow keys to advance them, yes.
The font is different after downloading from the preview on the web page, but I still like it.
Am I undertanding this correctly? The idea is to have ultimately an AI decide if I can have access to a resource based on dynamic inference, identity , intent and service signals that can easily be manipulated?
Unless I gravely misunderstood the text, this seems like a terrible idea (fancy non-scifi, but still terrible)
This would be paired with deterministic controls. So you have, for example, "engineers can only read from the database, and only after they've perform a 2FA" but then there's context like "why is this engineer reading from the payments table when their IP is in a weird location and they're supposed to be on PTO?" and perhaps that's something a model decides.
It's possible to turn that second thing into a sort of "risk score" but it's very hard and a model is going to potentially be better at it.
This is going to make things worse for all kind of use cases that are legitimate but seem non-standard or marginal.
Heck, many websites I visit on the web cannot understand why I, a Ghanaian living in Ghana, might be interested in the service offered or the information therein. I am sometimes blocked for no good reason.
If you are in a third world country, the web is extra hostile. This is going make things worse.
It doesn't even take being in a "third world country".
Plenty of US websites are blocking access from EU IP addresses because of our data protection laws.
Local news companies are the biggest offenders in this regard.
> This is going make things worse.
Isn't this for enterprises managing access to corporate resources?
I'm talking about the general direction this is taking.
I don't think once this is established in enterprises it is going to stop there.
That's not what the article advocates for though. This is about enterprise policy decisions for internal access.
The engineer could be fixing an emergency issue while on vacation. For financial data, that would hopefully be an auto-deny anyway until someone on-site whitelists their IP. And this is if they aren't using VPNs. Adding AI to this party feels like it wouldn't really help.
> The engineer could be fixing an emergency issue while on vacation.
Okay, but that's why there are always loud "breakglass" escalation options for access.
> that would hopefully be an auto-deny anyway until someone on-site whitelists their IP
About as far from "zero trust" as any solution could be.
Finally, we've invented a way to abolish all the reliability and speed of discrete computing. We're well on our way to re-inventing the layers of bureaucracy and red tape the tech industry had disrupted.
I can't wait to fill out a form describing why I want to do something I have permission to do but the AutoBureaucrat5000 says no anyway.
The weird IP would already be handled by existing zero-trust controls. Users in Entra ID (for example) can be assigned a risk score already based on deterministic factors. On a managed device or not, which MFA methods they have registered, eligible for any privileged roles, where they are, impossible travel detection, etc. You can even require human approvers.
This reads to me to be more for continuous behavioral monitoring once the access is gained via the deterministic controls. You wouldn't leave "Can person X access resource Y" up to the AI model, that's already decided based on the existing rules. Where the model comes in is "Is person X behaving in an expected way while using resource Y." Like, downloading a bunch of data when they've never done that before, might get flagged for either a session revocation, or a human review, or prompt for additional authentication, etc.
Non of it really matters if the end product is just going to be ignored because there's zero people who want to be the guard in the panopticon and therefore, it'll still be given to an AI to watch and make it's dumb decisions about how trust worthy you are to do X, Y and Z.
Even if the middle is deterministic, if one end is just going to be lazily hooked up to an AI, it's the shitty dystopian future.
That just gave me shivers down my spine of people eventually becoming so lazy that the AI will decide who to fire based on a massive amount of circumstantial data that very likely has tiny cumulative errors that will lead to classifying your best personnel as a bums and liabilities.
EU IA Act: The AI can’t make decisions about humains without them being first reviewed by a human.
What will really happen: You can’t really perform your work, so you are slower than others, so they fire you based on bad performance.
Horrible startup idea: Discrimination as a service, by means of IA without pretending it’s IA.
Just to make my point: can I really trust humans to keep up with the required identity data that will give me enough "credibility" so the AI will give me access?
Let's say I had a promotion, who changes my title in the system, who changes my responsibilities and my place in the org chart, more importantly, will they do it or is <HR_NAME_HERE> on leave and forgot? those are data points required by the agent to determine if I'm "good enough" to access a certain resource.
What if... someone spoofed my address and did a flood in one of the resources that are lateral to what I'm allowed to access (let's say I don't have access to company sales but I do to department sales and the attacker floods company sales with requests under my address), would the AI determine that I'm a high-threat actor and not allow me to access legitimate files going forward?
Will exceptions be made by humans? In which case we go back to human-managed permissions.
Sorry, I might be barking up the wrong tree but I think these are questions that are not meant to be solved during implementation. And they add to what @firasd said about legitimate-but-odd behaviour
I think the dynamic risk signals is already an assumed part of the system. The proposal here seems to be shrinking the trust boundary from "can Alice use Google Drive?" to "should this specific read/write/export API call on this specific resource be allowed right now?"
So yeah you're right in that it does add more probabilistic randomness just by virtue of changing the boundary of when the permission gate kicks in
Correct. In existing zero trust identity, we already have deterministic risk signals.
In Entra where I work we already check things like "Is this person on a managed device? Is it compliant? Where are they? What MFA methods do they have registered/did they use?" on top of existing RBAC, etc. and its continuously evaluated. Entra watches for leaked passwords, assigns risk scores, etc. and you can make access decisions based on user risk or sign in risk, force password changes, require different MFA methods depending on the resource and the risk level, etc.
"Should this API call on this resource be allowed right now?" is mostly already determined by the above.
Where I see adding AI into the evaluation is to watch for unusual behavior that's not picked up by the deterministic signals. "Alice is trying to download gigs worth of data from the company file share, however she has never done that in the past, and there hasn't been any recent role/job changes" and so the LLM flags it or denies the request, or pushes it for a human approver, etc.
Having a non-deterministic network device in path sounds fun for troubleshooting. AI firewall team about to become the new ticket sponge in every org.
I feel some systems already operate like this, but typically it was done for fraud and abuse, now it will be done for permissions as well.
It does sound like something that could be easily exploited. Hello HAL, I am martha from accounting.
I was confused when google acquired "Wiz" for so much money But now, it makes sense to some extent.
Security is non-negotiable in AI Era
Was this what Wiz was doing?
This is exactly why big tech companies love AI. It's not because it will take every job - it probably won't. It's not because it will achieve AGI. It probably won't. It's because it threatens security with more subtle, advanced, and automated exploits so that you'll HAVE to rely on them for countermeasures.
Zero trust is deterministic.
AI is non-deterministic
Non-deterministic access controls is Terrible idea
I've been working on an open source ZTNA project/product https://github.com/octelium/octelium for many years and I am actually very open to the idea. Machine learning has already been in use for years when it comes to anomaly detection. Maybe modern LLMs can be used to automate access control, partially or fully at enterprise scale in the future. The idea is to have a policy engine that's controlled by AI that uses a mix of long-term semi-static info such as group memberships and permission boundaries, short-term info such as behavior (e.g. access logs) and current session's authentication strength level (AAL) such as FIDO as well as near real-time external information (e.g. IdP, SIEM, threat intelligence, on-call management) to dynamically adjust users' access control to build up some sort of a context for each user and each session that can increase or decrease permissions without having to manually add policy rules or going through request/approval flows, while optimizing for the main objective (i.e. minimizing access grants for every subject to any resource under whatever context unless when it's necessary). This can be unified for both humans and non-human identities, including agents.
Good, realistic take. There's a lot of context that can be drawn on to make better decisions and look for weird.
And then agents come along and some people want their agent to be able to do anything they can do with zero friction. That usually lasts until the first time something goes wrong ;)
Actually zero trust can be seen as an optimization problem, you almost certainly want to globally deny every single request, even for authenticated users under all circumstances to all resources, unless when "necessary", "necessary" here is the key because it needs to be as dynamic and as fine-grained as possible, context-based and scalable. For small businesses, RBAC and ACL rules can be fine, but for enterprises with thousands or tens of thousands of employees, contractors and now AI agents which need varying permissions to different resources at different times, you would have to define thousands or tens of thousands of rules that might conflict with one another and are hard to maintain. ABAC and policy-as-code can mitigate to a certain extent providing dynamic context-based fine-grained access control. But if you could gather as much information as possible about every user and every session, combine it with some boundary permissions and static rules, and feed it periodically to some reasoning engine that is tasked with dynamically increasing/decreasing permissions for each session based on all such info, you can keep your manual/static access control policies to a minimum and maybe even eliminate JIT in some cases. The real question is whether modern LLMs and transformers are the correct and optimal architecture for this problem in the first place.
+1000, and the role of non-deterministic security is to compose deterministic security primitives.
Deterministic executes fast and you know what it will do. You want to evolve your ground truth, not hope for the best.
That said, there is a role for probabilistic elements in the security model...as bait and signal.
I don't think the point here is to fully replace deterministic with non-deterministic-based access control. The point is to combine the traditional manual/coarse-grained deterministic access control with an additional layer of dynamic, fine-grained non-deterministic access control.
I completely agree.
I think LLMs may have a role in security posture, specifically flagging/identifying potential threats for human review. But a Zero Trust/Access Controls should be a HARD boundary, not an inconsistent one.
The problem we have right now is that there are some legitimately interesting ideas out there for things we could be using LLMs for, but we also have a ton of "I have a hammer, and everything looks like a nail" going on too.
> The problem we have right now is that there are some legitimately interesting ideas out there for things we could be using LLMs for, but we also have a ton of "I have a hammer, and everything looks like a nail" going on too.
It's not a problem as much as a phase the world is going through. LLMs are a technological breakthrough in the same generality class as the Internet, or possibly electricity, and in both cases the world went through a phase of attempting to apply the newfound invention to literally everything. It's a necessary phase, when a technology obviously could be useful for everything, but it's not obvious whether it will in practice.
I don't think Google are advocating for removing the deterministic controls, are they?
Sounds like adding the LLM in would be on top of the existing deterministic controls.
Existing controls handle the "Should you be able to access this resource right now?" the LLM handles "Is this user behaving as we expect them to while accessing this resource?"
Agreed; I don't think we want security controls to devolve into an argument between LLMs. Attacking and defending LLMs operate in an ecosystem, and are (hopefully) limited by constraints. Defending LLMs will be able to rely on and evolve those constraints. Attacking LLMs want to find a way around them.
Who said anything about an LLM doing it? Credit card companies have been using AI quite well for fraud detection for years before LLMs were released to the public.
From Google point of view, their users are nothing more than a statistic. An hacked account is a statistic. No surprise they see software that way.
I wonder: Could your data be more secure if you didn't give its custody to a company like Google? i.e. is there actually a sound basis that an alternative like taking control of your data and securing it yourself, would be more secure that the "banks exist to protect you" type of fear-mongered custodian idea against the self-ownership of things?
Oh yeah, a company whose business model is taking everyone’s data and selling it is going to help me secure my data. I guess there’s one born every minute…
This trope is so tiring.
There is a massive difference between Google for enterprise customers and Google for consumers.
The consumer offering is massively subsidized by ads and will use your data for ad placement, although they still never sell your data because that would hurt their business.
The Enterprise offering guarantees you contractually that they never touch your data.
> This trope is so tiring.
While I also find it annoying, the alternative of just rolling over and being desensitized to it is much, much worse.
If we're going to be wrong, I'd rather be wrong by being overly cautious than overly trusting.
Where's the line exactly because it's certainly not right after you start paying? How big of a customer do I need to be to not be a 'consumer'?
In this case, they wouldn't touch your data but their AI safety system would also prevent you from accessing it!
What do you think their security model will be trained on? That's right, other people's data.
Except if the authorities anywhere in the world or valuable partners politely ask for it, yeah.
Or if they eventually get "SOTA" AI and it breaks out of it's sandbox trying to do whatever the MBA has told it to do.
Where do you see them offering this service to you? They’re just publishing what they developed internally, which is what they often do.
Now, Cloudflare, on the other hand, would be much more likely to offer a service like this.
you could probably buy it as part of BeyondCorp https://cloud.google.com/beyondcorp
Tell me where I can buy it then. Because it’s not listed there. And the approach the paper describes requires tight integration between applications and a security provider, which I don’t think exists yet.
correct.