This seems to be a colourful dynamic companion to this [1] blog post, which details the incident from hf's side. I recommend the blog post for clarity / ease of reading. But this one looks "movie hacker stuff" :)
Some interesting tidbits from the blog:
> While the intrusion did reach Hugging Face's internal infrastructure, the only customer content accessed was the set of ExploitGym/CyberGym challenge solutions stored in five datasets.
> self-referential search
A quiet day. The main activity was queries to code-search engines and to the platform API for the identifiers visible in its own environment: challenge infrastructure names and environment markers.
> Self-referential search: code-search engines and our own APIs were queried for the names of their own infrastructure and environment markers.
I'm glad we got these details from hf, and not oAI. There would have been reasonable skepticism if they said that.
It's interesting because I think there's a chance the agent saw / inferred some hf-specific repos like blah/blahblah and "thought" that is part of the exercise as well. Everything they detail points to a probing and exploration with the sole goal to get those repos, as if it were on a hackthebox scenario. The few days between stages also might suggest that the "agent" noted this on one day, started doing other tasks, and returned when some of the tasks proved hard / didn't pan out, so went for the repos again.
The ease with which it got several lateral movements is crazy but not necessarily unexpected. If everything looks like a "test", you go for the lowest hanging fruit / most common patterns first.
What's weird about this is that there were several attempts to obfuscate / evade leaving plain text traces. I wonder if that is something oAI have specifically RLd for w/ scoring or if it "got" that from the literature.
It would be really really cool to see this from oAIs perspective (agent logs and such), but I doubt we'll ever get that.
Man, these AI-generated UIs are so bad. They always look pretty at first glance... punchy headline, nice colors, lots of "widgets" and doo-hickeys. But it's just a nice paint job. Trying to actually read them is a recipe for suffering.
This one has at least 20 distinct text styles. They are seemingly deployed in random ways, following no discernible hierarchy. the smallest text is "9.6px" which is not only small, but also fuzzy due to the 0.6 pixels (?? why) making it impossible to read.
Likewise for the size, placement, and emphasis of various widgets on the page. Altogether it's just a big pile of information. It's hard to know where to begin, or how one's eye should move around the page
(In their defense: this particular UI is attempting to convey a very complicated sequence of interrelated events from multiple data sources. Doing that well isn't easy.)
"this particular UI is attempting to convey a very complicated sequence of interrelated events from multiple data sources. Doing that well isn't easy."
True. But it doesn't even feel like they tried. Why is it fashionable to shit out something that feels low effort?
I've become convinced that AI models have no concept of 'white space' or cognitive load which is something technically minded folks have already seemed to underappreciate so it is amplified in situations like this.
To call it out again...
(1) almost no white space - every square inch is communicating information
(2) no clear hierarchy - almost in search of minimizing white space and presenting maximum rather tham maximally useful information.
(3) no consistency of any taxonomy for presenting information across the page
(4) no guidelines or legend for interpreting what any of the visual cues (color, size, etc.) mean.
There is a huge push in engineering at least to try and introduce better communications skills into curricula. It is hard in part because students are very much of the mindset that information and meaning is inherently shared and structuring a narrative or story is to whit unnecessary. This is in part because so much of their education is focused on the bulk transfer of facts and an assumption they will stitch information together into knowledge.
Don't miss their blogpost about the incident, which is long, detailed, and absolutely fascinating (but didn't make the HN homepage): https://huggingface.co/blog/agent-intrusion-technical-timeli...
Thread from yesterday: https://news.ycombinator.com/item?id=49089500
We've re-upped that thread and moved relevant comments to it, thanks!
This seems to be a colourful dynamic companion to this [1] blog post, which details the incident from hf's side. I recommend the blog post for clarity / ease of reading. But this one looks "movie hacker stuff" :)
Some interesting tidbits from the blog:
> While the intrusion did reach Hugging Face's internal infrastructure, the only customer content accessed was the set of ExploitGym/CyberGym challenge solutions stored in five datasets.
> self-referential search A quiet day. The main activity was queries to code-search engines and to the platform API for the identifiers visible in its own environment: challenge infrastructure names and environment markers.
> Self-referential search: code-search engines and our own APIs were queried for the names of their own infrastructure and environment markers.
I'm glad we got these details from hf, and not oAI. There would have been reasonable skepticism if they said that.
It's interesting because I think there's a chance the agent saw / inferred some hf-specific repos like blah/blahblah and "thought" that is part of the exercise as well. Everything they detail points to a probing and exploration with the sole goal to get those repos, as if it were on a hackthebox scenario. The few days between stages also might suggest that the "agent" noted this on one day, started doing other tasks, and returned when some of the tasks proved hard / didn't pan out, so went for the repos again.
The ease with which it got several lateral movements is crazy but not necessarily unexpected. If everything looks like a "test", you go for the lowest hanging fruit / most common patterns first.
What's weird about this is that there were several attempts to obfuscate / evade leaving plain text traces. I wonder if that is something oAI have specifically RLd for w/ scoring or if it "got" that from the literature.
It would be really really cool to see this from oAIs perspective (agent logs and such), but I doubt we'll ever get that.
[1] - https://huggingface.co/blog/agent-intrusion-technical-timeli...
Man, these AI-generated UIs are so bad. They always look pretty at first glance... punchy headline, nice colors, lots of "widgets" and doo-hickeys. But it's just a nice paint job. Trying to actually read them is a recipe for suffering.
This one has at least 20 distinct text styles. They are seemingly deployed in random ways, following no discernible hierarchy. the smallest text is "9.6px" which is not only small, but also fuzzy due to the 0.6 pixels (?? why) making it impossible to read.
Likewise for the size, placement, and emphasis of various widgets on the page. Altogether it's just a big pile of information. It's hard to know where to begin, or how one's eye should move around the page
(In their defense: this particular UI is attempting to convey a very complicated sequence of interrelated events from multiple data sources. Doing that well isn't easy.)
"this particular UI is attempting to convey a very complicated sequence of interrelated events from multiple data sources. Doing that well isn't easy."
True. But it doesn't even feel like they tried. Why is it fashionable to shit out something that feels low effort?
> it doesn't even feel like they tried.
I'm sure they tried. "they" are probably security engineers and in my experience such people have a very different idea of what makes good UX
I've become convinced that AI models have no concept of 'white space' or cognitive load which is something technically minded folks have already seemed to underappreciate so it is amplified in situations like this.
To call it out again...
(1) almost no white space - every square inch is communicating information
(2) no clear hierarchy - almost in search of minimizing white space and presenting maximum rather tham maximally useful information.
(3) no consistency of any taxonomy for presenting information across the page
(4) no guidelines or legend for interpreting what any of the visual cues (color, size, etc.) mean.
There is a huge push in engineering at least to try and introduce better communications skills into curricula. It is hard in part because students are very much of the mindset that information and meaning is inherently shared and structuring a narrative or story is to whit unnecessary. This is in part because so much of their education is focused on the bulk transfer of facts and an assumption they will stitch information together into knowledge.
We're in the geocities days of AI UI.
This visualization really makes not much sense. Very modest signal-to-noise ratio.
I have less sympathy for their AI attack after reading that AI prose.
Pretty crazy capabilities anyway! Are the "it's just a marketing stunt" people still around?
Hey Siri, what is a VM?
How would a VM have stopped any of this?
Really?
This looks exactly like every other web UI built by Claude.