These and other methods are good reason to believe that AI can never be 100% contained, which gives me pause.
The famous air gap case is stuxnet, but there are other technical and biological approaches.
It’s a very weird reality we’ve found ourselves in if you stop, zoom out about, and really take in the gravity of the issues we are grappling with as a human race. Exciting, truly, and a bit nerve wracking.
Just because something is hard, doesn't mean it isn't worth doing. OpenAI haven't even tried to contain their agents, instead letting them loose on the internet and telling them to collaborate. While this is interesting research, it's very much academic research at this point (though of course it will enter training data for AI agents being encouraged by OpenAI to escape their 'sandbox').
We are a long way from AI agents escaping air-gapped computers, let's worry about that when it's an actual problem and when the companies running thousands of agents are acting responsibly.
Companies driving and offering agents are to be held directly responsible for their actions. The law is already written that way for any commercial product, but corporations try to use the 'new tech' 'ai-blackbox' argument to try to shield themselves from their own accountability (which btw somehow only shields corporations, never individuals). (precedent: social media)
Set the right incentives at the top, and you will soon see that changes are made very quickly ... especially those 'industry leaders' previously called 'impossible to contain' or 'non-competitive' BS.
If everyone has to play by the same rules, and the rules are made for the good of society, then there is a true win-win - and not just a wealth transfer to the top and putting expensive damages on the general population.
Not quite 100%, the article gives Faraday cage as an expensive countermeasure: "We analyzed and measured the effect of the Faraday chassis on the RAMBO covert channel using copper foil. This has electric field high Shielding properties (above 100 dB) and magnetic fields. The copper standard width is 1 mm effectively blocks the EMR from the transmitting workstation. However, as noted in the following section, this solution is costly and can not be deployed widely."
A Faraday cage is dirt cheap compared to the secrets stored in an air gapped server or data center. Every air gap should be Faraday caged, it’s table stakes.
We spent so much time talking about the dangers of AI ‘escaping’ when what actually happened is we gave it a scope of work and an internet connection and demanded it get to work.
Does any information leak along the power line? I remember having some device back in the day that let me create a LAN through my apartment's electric wiring.
I'm wondering if anything ends up back there thru the PSU during normal use of a computer, or if the high frequency components get filtered out.
(Though maybe secure computers do things differently with the power as well?)
Yeah, I've often wondered about this, but more locally (say within a local office, or maybe a data centre)...
I wonder if even "just" throttling up/down the CPU / GPU on tasks as morse code might be visible externally to other devices on the same mains supply. Obviously it would be very inefficient speed-wise, but a higher current (say + 300 watts for the GPU being used) should result in a slightly lower voltage on the mains supply I think for the duration?
Obviously things like PSUs could (I'm not an expert at any of this) get in the way of this, and I'd assume in data centres there's some isolation between different racks.
But for a scenario of DIY "airgapping" two machines in an office for AI testing (i.e. not doing it very well), it might be a plausible way of AIs finding to communicate if there's no actual network connection.
Obviously other electronic devices connected to the same mains circuit could also be adding their own noise, which might make it a lot harder to actually identify and decode the response, but... if ML/AI agents are good at anything, it's finding patterns, so I wouldn't want to bet against it being theoretically possible...
A rogue PSU could modulate information back over the electrical wire. However getting it out will require doing a physical wiretap on potentially high-voltage electricals. Also, it will likely have to be exfil'd at or across a power transformer sitting outside the data centre. Not sure if the signal will survive long enough (like literally, in line length terms), for exfil to work.
brb reviewing some EE material...
But don't expect much from me... I wanted to do a final year project in Power Line Communication for my electrical engineering undergrad. Then-me decided it was a dead end... cannot go WAN-scale; too much line noise and signal transmission power loss. I did something else with power electronics---I can't remember---but then again, my undergrad was also a dead end :p
There is information that leaks through the power line as well, and the author of this paper also has a paper on how to exfiltrate through the power line at a much slower rate https://arxiv.org/abs/1804.04014
TIL; thanks for sharing! I am not surprised to hear this.
Still, I'd guess that the odds of remote exploits are better if one wants to control a device via the PSU-as-backdoor, versus exfil data from it. Do like the IoT people and UDP broadcast naughty control signals to the power circuit / subsystem that has the compromised device.
Guessing further, remote exfil will mean having a close-proximity wiretap into the precise power line circuit of the target infra---because of signal losses---it could still be UDP broadcast exfil. Or a physical tap at whatever power transformer is the single point of connect between the target and the main grid. And if you can't get anywhere near the installation, well you can't exfil, because AFAIK it's pretty hard to reverse a signal up a transformer back into the main grid.
A scan of the arxiv paper appears to suggest similarly---the situation they describe presumes a wiretap into the circuit or the mains switchboard.
I heard that couple decades ago there was a product for "socket internet" - you plugged it into normal wall socket, and it communicated through power wires
There are tons of ways to make air gappped systems transmit data. They all include the installation of some malware and have become more expensive and less usable over time.
Back in the day with crt screens you could just capture the frequencies transmitted when creating the picture and look at the copy straight away.
No malware. No direct contact. And it worked over rather big distances.
The same thing killed electronic voting machines in The Netherlands.
No matter how well you argue that it is "perfectly secure", when some nerd outside the polling station is showing reporters a real-time feed of who people are voting for, your machine will get thrown out.
It wasn't quite CRT-level of mirroring, but it wasn't far off...
Cool paper even if the attack isn't practical, requiring getting the malware on the computer and a SDR antenna within 7 meters with clear line of sight of the affected computer for 100 bits per second (less after applying error correction). Hard to imagine a scenario where you can pull off both of those, but you couldn't plug in a USB, grab the PC, or record the screen directly.
Devices like routers should have firmware in memory that has read-only access. The access should be turned on or off by a physical switch.
This way, if malware gets installed on your router, doing a cold boot will eliminate it. You can have a timer to just cold boot it regularly.
I've suggested this many times. The rationale for not doing it is so the firmware can be remotely updated. Well, that's exactly how malware infects it.
> With a slow transmission rate (t = 10ms), A transmission is maintained at a distance of 700 cm away. With medium transmission rate (t = 5ms), A transmitted is maintained at a distance of 450 cm away and BER of 3%-4%. With a fast transmission rate (t = 1ms), a transmission is maintained at a distance of 300 cm away and BER of 2%-4%.
Coming from a non-EE background, this is the most interesting part to me. 10ms is an eternity in programming perf work, but it's already very constrained in how clear the signal is for reading the transmission. I just find it interesting that we want to do nothing to create a signal. Perhaps this makes it harder to exploit?
These attacks are all incredibly impractical and pointless, the bit rate under the most charitable assumptions is pathetically low. But even more, it ignores the most significant risk, which is people. People who can be tricked and subverted, and people systems that can be influenced to become misaligned, or which have procedural holes.
It's much easier to subvert the supply chain for the secure hardware or subvert the authorised system transfers.
Indeed! I was a bit skeptical to see it is a single-author paper (thinking it might be one of those Claude-generated ones), and I appreciate that you pointed out this is somebody worth listening to.
Right… I’m going to stop reading after these words. With malware on the computer you can change the heat signature of the room of you’re bored enough. https://xkcd.com/1172/
Computers used to come in proper full metal cases. Now they don't. Even when you try to find a metal case for a personal build most cases have giant holes in the side (case windows) acting as slot antennas. And plastic case laptop designs are even worse. Radiated emissions are much easier to achieve at useful intensities these days because of it. Sure, before the conductors going in and out radiated somewhat effectively (and still do). But now even core system components inside are just being let emit without any mitigation.
Ignoring the obvious "dmd is truly blazing fast" quip, wildly curious how this happened to you twice? Was it the PSU? Only once heard of a house fire caused by a computer, but no information on component actually caused it.
The first time it was the graphics card. It burst into flames and smoke poured out. The fire went out when I yanked the power. Fortunately, I was sitting next to it when it went. To be fair, it was doing a long graphics job.
After that, I set the computer on a metal plate to keep a fire from spreading to the floor.
Some years later, the PSU went out with a bang and a lot of smoke.
(dmd is so fast, the computer doesn't have time to heat up!)
These and other methods are good reason to believe that AI can never be 100% contained, which gives me pause.
The famous air gap case is stuxnet, but there are other technical and biological approaches.
It’s a very weird reality we’ve found ourselves in if you stop, zoom out about, and really take in the gravity of the issues we are grappling with as a human race. Exciting, truly, and a bit nerve wracking.
Just because something is hard, doesn't mean it isn't worth doing. OpenAI haven't even tried to contain their agents, instead letting them loose on the internet and telling them to collaborate. While this is interesting research, it's very much academic research at this point (though of course it will enter training data for AI agents being encouraged by OpenAI to escape their 'sandbox').
We are a long way from AI agents escaping air-gapped computers, let's worry about that when it's an actual problem and when the companies running thousands of agents are acting responsibly.
Companies driving and offering agents are to be held directly responsible for their actions. The law is already written that way for any commercial product, but corporations try to use the 'new tech' 'ai-blackbox' argument to try to shield themselves from their own accountability (which btw somehow only shields corporations, never individuals). (precedent: social media)
Set the right incentives at the top, and you will soon see that changes are made very quickly ... especially those 'industry leaders' previously called 'impossible to contain' or 'non-competitive' BS.
If everyone has to play by the same rules, and the rules are made for the good of society, then there is a true win-win - and not just a wealth transfer to the top and putting expensive damages on the general population.
Not quite 100%, the article gives Faraday cage as an expensive countermeasure: "We analyzed and measured the effect of the Faraday chassis on the RAMBO covert channel using copper foil. This has electric field high Shielding properties (above 100 dB) and magnetic fields. The copper standard width is 1 mm effectively blocks the EMR from the transmitting workstation. However, as noted in the following section, this solution is costly and can not be deployed widely."
A Faraday cage is dirt cheap compared to the secrets stored in an air gapped server or data center. Every air gap should be Faraday caged, it’s table stakes.
We spent so much time talking about the dangers of AI ‘escaping’ when what actually happened is we gave it a scope of work and an internet connection and demanded it get to work.
> These and other methods are good reason to believe that AI can never be 100% contained, which gives me pause.
Can’t leak secrets if you don’t have RAM.
Does any information leak along the power line? I remember having some device back in the day that let me create a LAN through my apartment's electric wiring.
I'm wondering if anything ends up back there thru the PSU during normal use of a computer, or if the high frequency components get filtered out.
(Though maybe secure computers do things differently with the power as well?)
Yeah, I've often wondered about this, but more locally (say within a local office, or maybe a data centre)...
I wonder if even "just" throttling up/down the CPU / GPU on tasks as morse code might be visible externally to other devices on the same mains supply. Obviously it would be very inefficient speed-wise, but a higher current (say + 300 watts for the GPU being used) should result in a slightly lower voltage on the mains supply I think for the duration?
Obviously things like PSUs could (I'm not an expert at any of this) get in the way of this, and I'd assume in data centres there's some isolation between different racks.
But for a scenario of DIY "airgapping" two machines in an office for AI testing (i.e. not doing it very well), it might be a plausible way of AIs finding to communicate if there's no actual network connection.
Obviously other electronic devices connected to the same mains circuit could also be adding their own noise, which might make it a lot harder to actually identify and decode the response, but... if ML/AI agents are good at anything, it's finding patterns, so I wouldn't want to bet against it being theoretically possible...
A rogue PSU could modulate information back over the electrical wire. However getting it out will require doing a physical wiretap on potentially high-voltage electricals. Also, it will likely have to be exfil'd at or across a power transformer sitting outside the data centre. Not sure if the signal will survive long enough (like literally, in line length terms), for exfil to work.
brb reviewing some EE material...
But don't expect much from me... I wanted to do a final year project in Power Line Communication for my electrical engineering undergrad. Then-me decided it was a dead end... cannot go WAN-scale; too much line noise and signal transmission power loss. I did something else with power electronics---I can't remember---but then again, my undergrad was also a dead end :p
There is information that leaks through the power line as well, and the author of this paper also has a paper on how to exfiltrate through the power line at a much slower rate https://arxiv.org/abs/1804.04014
TIL; thanks for sharing! I am not surprised to hear this.
Still, I'd guess that the odds of remote exploits are better if one wants to control a device via the PSU-as-backdoor, versus exfil data from it. Do like the IoT people and UDP broadcast naughty control signals to the power circuit / subsystem that has the compromised device.
Guessing further, remote exfil will mean having a close-proximity wiretap into the precise power line circuit of the target infra---because of signal losses---it could still be UDP broadcast exfil. Or a physical tap at whatever power transformer is the single point of connect between the target and the main grid. And if you can't get anywhere near the installation, well you can't exfil, because AFAIK it's pretty hard to reverse a signal up a transformer back into the main grid.
A scan of the arxiv paper appears to suggest similarly---the situation they describe presumes a wiretap into the circuit or the mains switchboard.
Reminds me of the old chestnut attributed to a Well Known computer security pioneer. I cannot for the life of me remember who.
Paraphrasing:
The only way to secure a computer is to not buy one. And if you buy one, to never plug it in. And if you plug it in, to never switch it on.
Something like the cheapest way to own $GasGuzzlerWithItsOwnPinCode is to bring it home after purchase, and set fire to it in the yard.
I heard that couple decades ago there was a product for "socket internet" - you plugged it into normal wall socket, and it communicated through power wires
Well this is technically still a thing, people pass internet connections through their sockets with special adapters.
Powerline Networking is what you are referring too.
Still available today, and has worked well for me when I've needed it.
There are tons of ways to make air gappped systems transmit data. They all include the installation of some malware and have become more expensive and less usable over time.
Back in the day with crt screens you could just capture the frequencies transmitted when creating the picture and look at the copy straight away.
No malware. No direct contact. And it worked over rather big distances.
The same thing killed electronic voting machines in The Netherlands.
No matter how well you argue that it is "perfectly secure", when some nerd outside the polling station is showing reporters a real-time feed of who people are voting for, your machine will get thrown out.
It wasn't quite CRT-level of mirroring, but it wasn't far off...
Cool paper even if the attack isn't practical, requiring getting the malware on the computer and a SDR antenna within 7 meters with clear line of sight of the affected computer for 100 bits per second (less after applying error correction). Hard to imagine a scenario where you can pull off both of those, but you couldn't plug in a USB, grab the PC, or record the screen directly.
Collocation.
Devices like routers should have firmware in memory that has read-only access. The access should be turned on or off by a physical switch.
This way, if malware gets installed on your router, doing a cold boot will eliminate it. You can have a timer to just cold boot it regularly.
I've suggested this many times. The rationale for not doing it is so the firmware can be remotely updated. Well, that's exactly how malware infects it.
> With a slow transmission rate (t = 10ms), A transmission is maintained at a distance of 700 cm away. With medium transmission rate (t = 5ms), A transmitted is maintained at a distance of 450 cm away and BER of 3%-4%. With a fast transmission rate (t = 1ms), a transmission is maintained at a distance of 300 cm away and BER of 2%-4%.
Coming from a non-EE background, this is the most interesting part to me. 10ms is an eternity in programming perf work, but it's already very constrained in how clear the signal is for reading the transmission. I just find it interesting that we want to do nothing to create a signal. Perhaps this makes it harder to exploit?
Meanwhile my WiFi never works
These attacks are all incredibly impractical and pointless, the bit rate under the most charitable assumptions is pathetically low. But even more, it ignores the most significant risk, which is people. People who can be tricked and subverted, and people systems that can be influenced to become misaligned, or which have procedural holes.
It's much easier to subvert the supply chain for the secure hardware or subvert the authorised system transfers.
As a total layman when it comes to air gapping: Would this not be defeated by putting the computer in a faraday cage?
https://en.wikipedia.org/wiki/Tempest_(codename)
There is always a way. A program could take control of the LEDs on the PC and export data by flashing them in certain patterns.
A Faraday cage would not let light escape. That is the definition.
Yes, Faraday shielding is one of many considerations when building a SCIF or other EMSEC zone.
Let me guess, a paper written by Mordechai Guri?
It's not even a contest anymore, that man is the heavyweight champ of air-gap security.
Indeed! I was a bit skeptical to see it is a single-author paper (thinking it might be one of those Claude-generated ones), and I appreciate that you pointed out this is somebody worth listening to.
> We show that malware on a compromised computer
Right… I’m going to stop reading after these words. With malware on the computer you can change the heat signature of the room of you’re bored enough. https://xkcd.com/1172/
Computers used to come in proper full metal cases. Now they don't. Even when you try to find a metal case for a personal build most cases have giant holes in the side (case windows) acting as slot antennas. And plastic case laptop designs are even worse. Radiated emissions are much easier to achieve at useful intensities these days because of it. Sure, before the conductors going in and out radiated somewhat effectively (and still do). But now even core system components inside are just being let emit without any mitigation.
I have my computer in a metal box because it has caught fire twice.
Ignoring the obvious "dmd is truly blazing fast" quip, wildly curious how this happened to you twice? Was it the PSU? Only once heard of a house fire caused by a computer, but no information on component actually caused it.
The first time it was the graphics card. It burst into flames and smoke poured out. The fire went out when I yanked the power. Fortunately, I was sitting next to it when it went. To be fair, it was doing a long graphics job.
After that, I set the computer on a metal plate to keep a fire from spreading to the floor.
Some years later, the PSU went out with a bang and a lot of smoke.
(dmd is so fast, the computer doesn't have time to heat up!)