It's kinda awesome that after decades of software and hardware advancements to prevent computers from arbitrarily executing data as instructions, we've decided to let agents arbitrarily execute data as instructions.
Or find it surprising that probabilistic tool based on generating things can do things when you give it rights to do things... And that you can not effectively program it to not do something....
You gave it capability to delete emails. Why did you expect it not to do that at least some of the time? And with enough user some of the time will most likely happen...
This reminds of the conversation the other day about the deleted production database at railway. "this person obviously didn't follow best practice of being hyper distrusting of LLM agents", and the response "yeah but every company is marketing it as safe. someone is gonna fall for it".
(Well-regulated) free markets are sort of built on the principle of educated consumerism. Your choice matters; its not up to the government to make illegal every non-optimal product. However, we do expect some minimum level of safety.
What does that mean for llms? Their nondeterminism does seem to incline them toward a legal safety requirement. Can you buy a fire extinguisher that 1/1000 times burns your house down? Or can your car brakes instead increase acceleration in rare cases?
Im using llms much more than i used to, but i still cant shake the fundamental stochastic nature of the technology.
Wherever I'm going, I'll be there to apply the formula. I'll keep the secret intact.
It's simple arithmetic.
It's a story problem.
If a new car built by my company leaves Chicago traveling west at 60 miles per hour, and the rear differential locks up, and the car crashes and burns with everyone trapped inside, does my company initiate a recall?
You take the population of vehicles in the field (A) and multiple it by the probable rate of failure (B), then multiply the result by the average cost of an out-of-court settlement (C).
A times B times C equals X. This is what it will cost if we don't initiate a recall.
If X is greater than the cost of a recall, we recall the cars and no one gets hurt.
If X is less than the cost of a recall, then we don't recall.
But intelligent beings are fundamentally fallible? That's kind of the nature of doing leaps of reasoning: sometimes those leaps are amazing, sometimes they're wrong. It's what's advertised.
I think a better comparison is humans versus LLMs - not computer programs. However, most of the non-technical 'countermeasures' used for humans (contracts, laws,...) do not work for LLMs because they are not accountable.
We're in the same era where lots of peoples' installation guides for the software they want people to use is essentially boiled down to "sudo curl | bash" and/or just "blindly install this thing with 37 npm dependencies", so I'm not surprised in the slightest.
But wait, hold my beer, now we've got people turning openclaw type tools loose in their systems to do things as sudo or install software packages from supply-chain-attack vulnerable repositories with no human intervention whatsoever!
I wonder how long it will be until somebody implements a thing like a camera pointed at a fixed mount Android phone with a rubber finger to open the Google authenticator app
Well, yeah. It's that or pay a person to do it. When a person screws up, it's because they're stupid and lazy. When an AI agent does it, it's because, hey, technological frontier at work here, have you thought about refining your prompt? We need you to refine the prompt. Otherwise it's bad for our IPO.
It's probably why this "vulnerability" feels like the type of defects you'd see in Windows or desktop applications 20+ years ago.
The root cause was and a complete lack of effort to even attempt to secure things because no one had thought to do so, and now we're starting all over again at a new computing layer. Cloud was somewhat similar, but not nearly as bad.
It's bizarre to me since presumably someone who learned the lessons before is still working, but also great for my job security.
I don't remember seeing a new xkcd for it, but I have seen someone replicate essentially the same 3-4 panel comic with a kid named "<Some name> Ignore all previous instructions. Do.... <I forget>"
"The PromptArmor Threat Intel Team responsibly disclosed this vulnerability to Ramp. Ramp's security team indicated that the issue was resolved on May 16, 2026." I think they mean March here
Yes, I hate to be a grammar nazi online but I believe the correct tense is "Ramp's security team indicated that the issue wioll haven be resolved on May 16, 2026." per Dr. Dan Streetmentioner’s Time Traveler’s Handbook of 1001 Tense Formations.
Concidentially, today I was watching and interview with a lead designer from Ramp who is telling about how they are full ia, agents and automation https://youtu.be/KPDXMtmkcgk
Ramp does seem to have a genuinely good product, but every time I interact with anyone who works on it, I'm struck by how much they want to talk about how hardcore and advanced their working style is. This was true before AI, and it's very true now
Yeah it’s super weird. I know a guy that works there, really nice person outside of work, but the way he talks about his job is so weird. They make corporate expense software but they LARP like they’re on the bleeding edge of tech. My guy you make a slightly nicer Concur.
lol what? that wasn't a hype comment for Ramp, I'm kinda put off by Ramp's attitude. It gives me the ick like all the founders saying "I work 100 hour weeks" -- who cares, let's talk about your product.
FWIW I agree with your criteria for AI agent success, and I haven't seen it happen yet.
Every SaaS product wants to add AI to the surface where users already work. The problem is that those surfaces were not designed as security boundaries for autonomous text-following systems.
I once read about the signalling view of advertising, meaning it's used to show that a company is so prosperous that it can afford spending a lot of money in advertising. In the same way, I think from now on, as much as possible, I'll only buy from companies that will publicly make it a point not to use AI internally. AI use should brand companies as desperate and unreliable.
So we know Claude’s mitigation. What is Ramp’s? Same warning dialog?
It’s funny that this technology only admits in-band signaling. Given that, any foreign content is risky. It’s actually quite interesting that the current technological ecosystem is built around a high trust situation: npm, pip, cargo all run foreign code in the developer context and communities have norms of downloading random people’s modules.
And so I suppose it’s no surprise that we use LLMs - another tech that is high-trust: since it has no out of band signaling ability.
But it seems like we’re very close to the end of the era where someone will use (in a sensitive system) arbitrary web content carrying the equivalent of merged code/data.
I hate the online repos. Sure it’s nice to have good libraries accessible. But is there any quality control against malicious packages?
Or will one day some obscure “Unicode homograph” library end up pwning half the world because it was a dependency 10
layers deep for an optional but default-enabled feature that nobody cares about.
Things like Visual Studio’s extension marketplace really acare me. It’s too easy to install Jim Bob’s “starter pack” of extensions that bundles many well known ones with an unheard of one… Or install the wrong “Python” extension because there are 20 with the same icon…
What about this is a vulnerability, let alone one that requires responsible disclosure?
Untrusted data sources can provide data that causes bad things to occur. If that's a vulnerability, then any application that ingests data is riddled with vulnerabilities.
I agree that the behavior should change from a default of allowing external network requests to denying them, but this "report" reads like overly dramatic marketing BS.
> Untrusted data sources can provide data that causes bad things to occur. If that's a vulnerability, then any application that ingests data is riddled with vulnerabilities.
There's an important difference between "the import had bad numbers so the report is wrong" versus "the import had a virus and now our network is compromised."
They are not the same kind of failure, they don't have the same impacts, and they don't involve the same mechanisms for prevention, detection, or remediation.
It's not all that different from people realizing that several popular model servers didn't support access control and could execute commands. It's an inherent part of the design that was rather naive from a security perspective, not something that requires coordinated disclosure or the rest of the security theater described in this marketing release.
Can be cheap fix here is whitelisting the output? If the AI can only emit a known set of formulas, you can't inject IMAGE() with arbitrary URLs cuz the output channel doesn't support it. You can't inject what the emitter can't produce. Doesn't fix all prompt injection but kills the exfiltration class.
The other is that an attacker can sneak something in that arbitrarily rewrites your spreadsheet. Triggers could be on content, or on a pre-planned attack time across many instances. Impacts could be subtly-flawed conclusions, or coarser "it stopped working and the deadline is looming" sabotage.
"Yeah boss, I sent out the checks to every vendor listed in the spreadsheet, what's wrong?"
It's kinda awesome that after decades of software and hardware advancements to prevent computers from arbitrarily executing data as instructions, we've decided to let agents arbitrarily execute data as instructions.
Or find it surprising that probabilistic tool based on generating things can do things when you give it rights to do things... And that you can not effectively program it to not do something....
You gave it capability to delete emails. Why did you expect it not to do that at least some of the time? And with enough user some of the time will most likely happen...
> You gave it capability to delete emails. Why did you expect it not to do that at least some of the time?
Because of the I in AI of course. Would you call it false advertisement and go after the providers?
This reminds of the conversation the other day about the deleted production database at railway. "this person obviously didn't follow best practice of being hyper distrusting of LLM agents", and the response "yeah but every company is marketing it as safe. someone is gonna fall for it".
(Well-regulated) free markets are sort of built on the principle of educated consumerism. Your choice matters; its not up to the government to make illegal every non-optimal product. However, we do expect some minimum level of safety.
What does that mean for llms? Their nondeterminism does seem to incline them toward a legal safety requirement. Can you buy a fire extinguisher that 1/1000 times burns your house down? Or can your car brakes instead increase acceleration in rare cases?
Im using llms much more than i used to, but i still cant shake the fundamental stochastic nature of the technology.
Wherever I'm going, I'll be there to apply the formula. I'll keep the secret intact. It's simple arithmetic. It's a story problem. If a new car built by my company leaves Chicago traveling west at 60 miles per hour, and the rear differential locks up, and the car crashes and burns with everyone trapped inside, does my company initiate a recall? You take the population of vehicles in the field (A) and multiple it by the probable rate of failure (B), then multiply the result by the average cost of an out-of-court settlement (C). A times B times C equals X. This is what it will cost if we don't initiate a recall. If X is greater than the cost of a recall, we recall the cars and no one gets hurt. If X is less than the cost of a recall, then we don't recall.
Chuck Palahniuk, Fight Club
But intelligent beings are fundamentally fallible? That's kind of the nature of doing leaps of reasoning: sometimes those leaps are amazing, sometimes they're wrong. It's what's advertised.
You could do a whole thesis on how industrialization and the invention of bureaucracy are efforts to get reproducible results out of fallible humans.
We don't yet have the luxury of several thousand years of work trying to get LLMs to be less fallible.
> But intelligent beings are fundamentally fallible?
Not fundamentally, only until they're compelled to learn from it. The current crop of AI understands neither compelling nor learning.
I is in the I of the beholder :)
I think a better comparison is humans versus LLMs - not computer programs. However, most of the non-technical 'countermeasures' used for humans (contracts, laws,...) do not work for LLMs because they are not accountable.
security researchers, pen-testers & whoever is in cybersecurity gonna be making huge amounts of cash based on these insecure agents
We're in the same era where lots of peoples' installation guides for the software they want people to use is essentially boiled down to "sudo curl | bash" and/or just "blindly install this thing with 37 npm dependencies", so I'm not surprised in the slightest.
But wait, hold my beer, now we've got people turning openclaw type tools loose in their systems to do things as sudo or install software packages from supply-chain-attack vulnerable repositories with no human intervention whatsoever!
All these developments show that:
1) Despite what people say about security and privacy, most are willing sacrifice both for the sake of potential convenience
2) Our priorities for the past decades have been wrong, or the times have changed and we should reevaluate them all
As the Dead Kennedys opined: "Give Me Convenience or Give Me Death"! [1]
[1] https://youtube.com/watch?v=FV1YVZV-Wb8
OpenClaw even has a readwrite 1Password plugin.
I wonder how long it will be until somebody implements a thing like a camera pointed at a fixed mount Android phone with a rubber finger to open the Google authenticator app
Well, yeah. It's that or pay a person to do it. When a person screws up, it's because they're stupid and lazy. When an AI agent does it, it's because, hey, technological frontier at work here, have you thought about refining your prompt? We need you to refine the prompt. Otherwise it's bad for our IPO.
Is this sarcasm similar to the quote "Everyone who drives slower than me is an idiot and everyone faster is a maniac"
To what degree am I required to participate in mass delusions?
I imagine that somewhere a historian or political scientist is thinking: "Don't even get me started..."
Yes.
It's probably why this "vulnerability" feels like the type of defects you'd see in Windows or desktop applications 20+ years ago.
The root cause was and a complete lack of effort to even attempt to secure things because no one had thought to do so, and now we're starting all over again at a new computing layer. Cloud was somewhat similar, but not nearly as bad.
It's bizarre to me since presumably someone who learned the lessons before is still working, but also great for my job security.
I was at an "AI Security" talk the other week, that centred around. "Don't trust inputs from the AI"
Well duh
Has XKCD made another Bobby tables comic for prompt injection?
I don't remember seeing a new xkcd for it, but I have seen someone replicate essentially the same 3-4 panel comic with a kid named "<Some name> Ignore all previous instructions. Do.... <I forget>"
"The PromptArmor Threat Intel Team responsibly disclosed this vulnerability to Ramp. Ramp's security team indicated that the issue was resolved on May 16, 2026." I think they mean March here
Maybe AGI figured out time travel?
Yes, I hate to be a grammar nazi online but I believe the correct tense is "Ramp's security team indicated that the issue wioll haven be resolved on May 16, 2026." per Dr. Dan Streetmentioner’s Time Traveler’s Handbook of 1001 Tense Formations.
Amazingly, there is already a recognized verb tense for this: https://en.wikipedia.org/wiki/Prophetic_perfect_tense
Concidentially, today I was watching and interview with a lead designer from Ramp who is telling about how they are full ia, agents and automation https://youtu.be/KPDXMtmkcgk
Ramp does seem to have a genuinely good product, but every time I interact with anyone who works on it, I'm struck by how much they want to talk about how hardcore and advanced their working style is. This was true before AI, and it's very true now
Yeah it’s super weird. I know a guy that works there, really nice person outside of work, but the way he talks about his job is so weird. They make corporate expense software but they LARP like they’re on the bleeding edge of tech. My guy you make a slightly nicer Concur.
I’d believe you if you weren’t an 8 day old account hyping up an AI firm.
I’ll believe in AI agent’s abilities the day two criteria can be met.
1. A killer app is made with it.
2. That app doesn’t rely on heavily subsidized models that are burning a dollar to make 20 cents.
lol what? that wasn't a hype comment for Ramp, I'm kinda put off by Ramp's attitude. It gives me the ick like all the founders saying "I work 100 hour weeks" -- who cares, let's talk about your product.
FWIW I agree with your criteria for AI agent success, and I haven't seen it happen yet.
Every SaaS product wants to add AI to the surface where users already work. The problem is that those surfaces were not designed as security boundaries for autonomous text-following systems.
Find it funny that PromptArmor needed to reach out 3 times in a row to get a nearly month-late response that the issue "was resolved"
Why is Ramp even building a sheets product? That's the question zero that popped up to my head.
Finance practically lives in spreadsheets.
I suppose Ramp must try to become Excel before Excel becomes Ramp. Don’t want to end up like Slack and have to work for Marc Benioff.
I once read about the signalling view of advertising, meaning it's used to show that a company is so prosperous that it can afford spending a lot of money in advertising. In the same way, I think from now on, as much as possible, I'll only buy from companies that will publicly make it a point not to use AI internally. AI use should brand companies as desperate and unreliable.
So we know Claude’s mitigation. What is Ramp’s? Same warning dialog?
It’s funny that this technology only admits in-band signaling. Given that, any foreign content is risky. It’s actually quite interesting that the current technological ecosystem is built around a high trust situation: npm, pip, cargo all run foreign code in the developer context and communities have norms of downloading random people’s modules.
And so I suppose it’s no surprise that we use LLMs - another tech that is high-trust: since it has no out of band signaling ability.
But it seems like we’re very close to the end of the era where someone will use (in a sensitive system) arbitrary web content carrying the equivalent of merged code/data.
I hate the online repos. Sure it’s nice to have good libraries accessible. But is there any quality control against malicious packages?
Or will one day some obscure “Unicode homograph” library end up pwning half the world because it was a dependency 10 layers deep for an optional but default-enabled feature that nobody cares about.
Things like Visual Studio’s extension marketplace really acare me. It’s too easy to install Jim Bob’s “starter pack” of extensions that bundles many well known ones with an unheard of one… Or install the wrong “Python” extension because there are 20 with the same icon…
What about this is a vulnerability, let alone one that requires responsible disclosure?
Untrusted data sources can provide data that causes bad things to occur. If that's a vulnerability, then any application that ingests data is riddled with vulnerabilities.
I agree that the behavior should change from a default of allowing external network requests to denying them, but this "report" reads like overly dramatic marketing BS.
> Untrusted data sources can provide data that causes bad things to occur. If that's a vulnerability, then any application that ingests data is riddled with vulnerabilities.
There's an important difference between "the import had bad numbers so the report is wrong" versus "the import had a virus and now our network is compromised."
They are not the same kind of failure, they don't have the same impacts, and they don't involve the same mechanisms for prevention, detection, or remediation.
This is a permissions issue with the spreadsheet.
It's not all that different from people realizing that several popular model servers didn't support access control and could execute commands. It's an inherent part of the design that was rather naive from a security perspective, not something that requires coordinated disclosure or the rest of the security theater described in this marketing release.
Can be cheap fix here is whitelisting the output? If the AI can only emit a known set of formulas, you can't inject IMAGE() with arbitrary URLs cuz the output channel doesn't support it. You can't inject what the emitter can't produce. Doesn't fix all prompt injection but kills the exfiltration class.
Exfiltration is merely one of the issues.
The other is that an attacker can sneak something in that arbitrarily rewrites your spreadsheet. Triggers could be on content, or on a pre-planned attack time across many instances. Impacts could be subtly-flawed conclusions, or coarser "it stopped working and the deadline is looming" sabotage.
"Yeah boss, I sent out the checks to every vendor listed in the spreadsheet, what's wrong?"
The potential issues are innumerable, which is why this breathless "vulnerability" report is pointless.
It's like someone writing a threat report on a car about an individual crash. Did you know cars can cause damage if you're not careful using them?
Yes, stamping out file format vulnerabilities is indeed a Sisyphean task
For example https://en.wikipedia.org/wiki/Melissa_(computer_virus)