There's an underappreciated comment in the other thread about SynthID and OpenAI [0] that captures what (IMO) the hacker ethos on this should be. We care about privacy, we should not accept tools that barcode our every digital move. (note that the counter of "well, they don't do that yet" is not particularly convincing)
Its what happens when people in power are paranoid dark-triad types and want to be able to catch anyone who threatens their power and stick it to them..
Building a tool that tries (and probably fails) to remove the watermark (due to the arms race that large corporate machines will win) is tacitly accepting the barcode. The hacker ethos should be, first and foremost, to run open source models locally without relying on a corporation.
>due to the arms race that large corporate machines will win
Much like how the entirety of Hollywood, book publishers, academic publishers, and game developers have won against piracy despite being some of the largest corps on earth and dedicating untold billions to the issue over the past 30 years?
By the time you're building (or buying) the necessary highly esoteric and expensive ultracentrifuge setup I think you would be well outside the realm of "hobbyist" unless someone insists on the most unreasonably pedantic definition for the term.
Unless we're only considering final assembly. Just gotta get that weapons grade fissile material supplier lined up. That might or might not qualify as rich hobbyist territory depending on how high a price tag is permissible.
Yes. Winning against piracy doesn't mean you completely eliminate piracy. It means you scare enough people into not doing it and make it a bit harder to do for others.
Losing to piracy would see companies like Netflix and Spotify not thriving.
> It means you scare enough people into not doing it and make it a bit harder to do for others.
By which definition they utterly failed.
> Losing to piracy would see companies like Netflix and Spotify not thriving.
Not at all. Netflix and Spotify do well because they are a good value proposition for the average customer. Piracy is free at point of "purchase" but is (and always has been) expensive in terms of various sorts of overhead.
It's already possible to lie with text. Pixels are pixels. If we can't blindly believe pixels to show the truth, we will be simply back to the pre-photography era which managed to have a concept of truth regardless.
Fair enough. While I would kind of wish AI could be reliably detected, deep down I know this is impossible and it would be pretty bad if we had, say, a prosecution that succeeded because "this 'provably-non-AI' photo places you at the scene of the crime" because only a few underground people know how to remove a watermark.
The concept of truth? A bit overblown don't you think? Because some guy can make a realistic looking fake videos that destroys the "concept" of truth? How?
Stalin had all the resources imaginables at his disposal.
Now Nancy, a tech-phobic waitress who has a grudge against her coworker can make up an entire scenario with one prompt and her colleagues might blindly believe her.
Let's not pretend they're the same thing.
Gen AI is inevitable. Watermarking is likely futile. But in my opinion it is still very important to discuss how, as a society, we're going to live in a post-truth world now that anybody can, IN SECONDS, not only fabricate a story but also spread it to thousands of people through their social media.
When that idea was originated, the advice was more like:
"Don't trust what you see on the Internet. Trust instead what you read in a reputable daily newspaper, or Peter Jennings or Tom Brokaw on the nightly news, or BBC World News."
Today, the Internet, especially the part which is not trustable, has nearly finished killing most of the "trustworthy" news sources, by outcompeting them for ad dollars - by being way better at targeting ads (e.g. Meta) and by scientifically perfecting addiction (e.g. TikTok). What remains is mostly controlled by governments and has far from a perfect record of being fact-based and impartial.[1] There are a ton of independent people out there in good faith posting facts on the Internet, but we just agreed that we shouldn't trust what we see on the Internet.
So doesn't this become "Don't trust anything"? And doesn't that, in practice, get implemented as "Don't trust anything that challenges what you believe to be true"? This feels like a really, really bad change to our society - and I'd argue it's already completely happened.
This isn't just ads, trust in the mainstream media, itself, is very low [1], deservedly so in my opinion. The continuous lies by omission, the outright incorrect headlines/articles that they edit after a day, the lock-step messaging, alignment, and avoidance of topics, pushed by their respective political parties/billionaire owners (6 companies own 90% of media [2]), made me switch to more independent journalists.
Adequately implementing solves one problem (the making up a story because of a grudge), but creates a whole new set of likely much worse problems: how does one maintain a democracy / civil society? It's not just the trust of "social media" that you've eroded, you've almost certainly killed trust in traditional news reporting as well, especially considering just how much of traditional media is discovered via social media.
Effective democracy requires an informed voter base. Society requires its constituents to be invested in its continuity. Neither of those is achievable when we completely discard trust.
Yes, it's happened. Except a lot of people do have an exception - they'll trust the slop that reinforces their existing biases, or even if they know in their hearts it's not true, viewing their side's lies regularly still has an effect on the way they think.
Good point. Sometimes I wonder if social media, just almost every aspect of it, is the real cancer. Allowing just about anyone (globally) to anonymously deploy information warfare via the social media vector just seems bound to have horrible outcomes. It's just as bad with text as with images or video. Because of social media, we've trained at least 3 separate generations to self-sort into camps with customized ideological info sources that have incredibly-low standards for fact-checking and every incentive to tell their audience (1) exactly what they want and (2) whatever will enrage them most.
AI kind of makes this worse, but also only barely. Because most people really ought to know by now that almost any content could be AI, a video of, say, Trump kicking a baby or violating a goat wouldn't convince anyone that those acts happened (unless they already believed they happened).
Thing is, we're so flooded in biased BS, and no one has any incentive to produce non-sensational, factual news. I absolutely see 'post-truth' as the inevitability. You can't "weed a garden" when it is 100% weeds. The term "news" will cease to mean facts, and just become a branch of entertainment. Kind of the way "Reality TV" went from being supposedly a documentary (e.g. COPS) to just being a flavor of entertainment, where nothing needs to be real.
I'm pretty sure watermarking is (or soon will be) a requirement for AI generated images in software used in the EU, as part of their regulations for AI transparency.
If i had a dollar for every time an American cried about literally any non-US jurisdiction having an iota of effect on them I could quit my job and leave this terrible website forever.
If I had a dollar for every regulation to come out of Brussels serving no purpose other than to extract money or exert control over American companies because they have no relevant competitors to worry about impacting in the EU I could do the same.
Maybe we do care about truth, freedom and privacy but the majority of rest of society will happily accept any T&Cs just to get access to whatever the next digital sliced pan is and as for truth and accountability, if they were two sides of the same coin on the ground people wouldn't bend down to pick it up as possesing it looks too much like responsibility and inconvenience.
The watermarking should be on those things we want to verify as something that was not generated or manipulated. Something you'd add to, for instance, cameras. Putting them on the generated/manipulated is backwards as you can never get every model to watermark.
That model is equally bad though. Given that you're writing this in a discussion about gen AI watermarks, how in the world did you come up with the idea that Gen AI wouldn't be able to add a watermark?
The human ethos should be to never be misleading about the origin and truth of any content you create, forward, or pass on. If we care about honesty we should jail anyone who does so.
Are markers being removed here the same or similar to ones tools might add if you use an AI tool just to edit a photo? like a more complicated object removal in a photo editor?
That's not what the previous comment is referring to. They're referring to false positives, i.e "Gemini did not generate this (or process it) yet it says SynthID confirmed"
You can count on them doing it in a way that's economical for them. It's how email spam filters and ad blockers work. Sure somebody will always find a way to bypass it, and that's the arms race. A filter with zero false positives that removes 80% of slop is pretty darn good though.
This is a bit misleading as for Gemini it only properly removes the visible watermark. To remove SynthID it has to regenerate the image at low noise with SDXL, which will likely destroy a lot of small details, plus won't work for higher res properly (NB2 and GPT Image 2 support up to 4K image outputs)
Nano Banana 2 only supports 1K resolution (1024x1024) natively. Anything above that is upscaling. So this matches SDXL. GPT Image 2 does support 4k natively (but experimentally).
Where did you get that info from? According to Google's own docs as well as my own image generation tests via the API, it supports up to 4K natively for gemini-3.1-flash-image-preview (aka NB2).
It just defaults to 1K. But I didn't see anything in the docs stating that it's just a simple upscale for larger resolutions.
Yeah - if that's true then it's even worse because the output price says
$0.067 per 1K image*, $0.101 per 2K image*, and $0.151 per 4K image*.
But if all the "compute time" is spent on a 1K image and they're just passing it to a ESRGAN or other upscaling technique, then there’s literally zero reason to generate anything above 1K. Just save the money and upscale it yourself.
With the number of fine-tuned LoRAs and checkpoints - from a realism standpoint, yes SDXL is still very viable. From a prompt adherency perspective, absolutely not.
So I use a combination of Neo Forge and ComfyUI. Forge has an easier learning curve but ComfyUI gets all the new "hotness" almost immediately since there's so much custom nodes for it.
If you're on a Mac, I've heard that Draw Things is supposed to be pretty "batteries included" simple for image gen along the same lines as LM Studio.
Watermarking images generated from trained data on stolen copyrighted material, I get why so they can try to tell if something is real or not but something seems wrong
> Use cases where the threat model fits: You are preserving art or historical record against false-positive "AI-generated" labels.
Sorry, how does using AI to generate images have anything to do with this? Image generators cannot insert watermarks into things they did not generate, and it seems highly unlikely that you will get a false-positive watermark on human-generated art, especially if, as the readme says, these watermarks have high enough fidelity to trace to a specific session id. Plus the modifications to the image needed to erase watermarks would necessarily change the thing being "preserved."
[edit]: the more I read the more I'm convinced, the claimed use cases in the README are bullshit and the real reason is to provide a tool that helps people bypass "AI-generated" labels on social media for AI slop.
I mostly agree about the justification in the repo being wrong, but wanted to engage about this point:
> Image generators cannot insert watermarks into things they did not generate
It's actually very easy to take a real image, ask Gemini/ChatGPT to modify some tiny part of it (could be something as silly as lighting/shadow/etc), and often the resulting image will be detected by their watermarking tools. This way you can easily present any real image as AI-generated.
Ignoring that a watermark removal tool does not help with this threat model, the claim is still true: the original image can not be changed, and instead a copy is created.
So what? I can also open an image in Photoshop and make sure it saves out some Photoshop specific EXIF data and try to claim the image was doctored. What I can't do is go and put my deceptive altered file up in place of the original in all the places on the Internet it exists.
I had to think about it, how about if the claim were:
If you take a photograph that is misidentified as AI generated, you can “preserve the historical record“ by using this tool before publishing the image.
(Anyone know the false positive rate with watermark IDs, would’ve hoped it’s like zero)
Regardless of one's opinion about this particular project, it seems obvious to me that the path forward is proving authenticity of non-AI resources rather than attempting to watermark all the AI-generated ones.
Pretty hard problem to tackle when you can point an "authenticated" camera at a really nice screen and snap a 'definitely real' photo of anything a screen can display :(
There's probably a technical solution, such as the camera manufacturer cryptographically signing a GPS location and timestamp together with the pixels. Like all DRM it will probably be broken though, and more importantly, would anyone (even e.g. a newspaper editor) care enough to verify the signature?
watermarking only really works when the scheme is secret.
putting cyphertext in high frequency noise is old news. in generative land would be far more interesting to use the generative flexibility to encode in macrostructure.
I just saw the announcement about OpenAI or so going to use SynthID and all I thought was; what can d be read(located) can be removed. Seems the tool already exists, proving my point.
You're assigning emotions to people based on what you'd like them to feel, not on reality. For example, most americans probably don't feel shame about being american. But it's still a good decision not to go around showing off a bunch of american flags abroad, unless you want people to look at you in a certain way.
There's an underappreciated comment in the other thread about SynthID and OpenAI [0] that captures what (IMO) the hacker ethos on this should be. We care about privacy, we should not accept tools that barcode our every digital move. (note that the counter of "well, they don't do that yet" is not particularly convincing)
[0]: https://news.ycombinator.com/item?id=48200060
Its what happens when people in power are paranoid dark-triad types and want to be able to catch anyone who threatens their power and stick it to them..
Building a tool that tries (and probably fails) to remove the watermark (due to the arms race that large corporate machines will win) is tacitly accepting the barcode. The hacker ethos should be, first and foremost, to run open source models locally without relying on a corporation.
>due to the arms race that large corporate machines will win
Much like how the entirety of Hollywood, book publishers, academic publishers, and game developers have won against piracy despite being some of the largest corps on earth and dedicating untold billions to the issue over the past 30 years?
They won the long game. Everything is rented and DRM now. Very little of what most people buy digitally is truly owned.
They have a finite # of employees, a finite budget, and a finite amount of time.
Hobbyists do not. ROI is not a factor.
As yes, the hobbyist built nuclear weapons program.....
Legalize recreational plutonium.
To be fair the state works pretty hard to crush "hobbyist" nuclear weapons programs so you don't really know how far it could get.
By the time you're building (or buying) the necessary highly esoteric and expensive ultracentrifuge setup I think you would be well outside the realm of "hobbyist" unless someone insists on the most unreasonably pedantic definition for the term.
Unless we're only considering final assembly. Just gotta get that weapons grade fissile material supplier lined up. That might or might not qualify as rich hobbyist territory depending on how high a price tag is permissible.
You don't have to use the ultracentrifuge, though I don't suppose the power plant you would need for a diffusion plant would be much more attainable.
You don’t happen to know a certain Doc Brown?
What? Some nerds on private trackers and kids on 123movies or whatever is not piracy winning by any material stretch.
Yes. Winning against piracy doesn't mean you completely eliminate piracy. It means you scare enough people into not doing it and make it a bit harder to do for others.
Losing to piracy would see companies like Netflix and Spotify not thriving.
> It means you scare enough people into not doing it and make it a bit harder to do for others.
By which definition they utterly failed.
> Losing to piracy would see companies like Netflix and Spotify not thriving.
Not at all. Netflix and Spotify do well because they are a good value proposition for the average customer. Piracy is free at point of "purchase" but is (and always has been) expensive in terms of various sorts of overhead.
> [fighting against the system] is tacitly accepting the barcode.
I don't really see it. I think it's important to win on both fronts.
Fighting within the system is accepting the system.
Especially as the open weight models are really generated by corporates, and they could stop releasing them at any time.
But we'd still have them. It's not like we're gaining much with new training anymore anyway
I appreciate my coding agent being increasingly aware of the walrus operator :)
They also have built in dystopian government authority enforcement in them unless you go to pains to sever those neurons.
Accepting blindly destroying the concept of thruth should not be the hacker ethos either.
It's already possible to lie with text. Pixels are pixels. If we can't blindly believe pixels to show the truth, we will be simply back to the pre-photography era which managed to have a concept of truth regardless.
When could you ever trust pixels?
It either works reliably or it doesn't; if it doesn't, it's better that everybody be clear about that.
Fair enough. While I would kind of wish AI could be reliably detected, deep down I know this is impossible and it would be pretty bad if we had, say, a prosecution that succeeded because "this 'provably-non-AI' photo places you at the scene of the crime" because only a few underground people know how to remove a watermark.
Not necessarily. Knowing an image for sure is fake has value, even if you can’t guarantee the reverse is true.
It's best for privacy not to do this in the first place because:
- Watermarks are optional by AI provider so bad actors will circumvent by using another provider
- GH project proves watermarks can be removed
Given these, trying to ensure "truth" is a futile effort unfortunately, and watermarking only gives companies advantage to violate privacy
Nobody said that?
Saying that watermarking fake things is bad kinda strongly implies it
The concept of truth? A bit overblown don't you think? Because some guy can make a realistic looking fake videos that destroys the "concept" of truth? How?
Stalin had no issues photoshopping images almost 100 years ago.
Generating realistic video of arbitrary things and people at scale is quite a bit of a different game than retouching photos
Stalin had all the resources imaginables at his disposal.
Now Nancy, a tech-phobic waitress who has a grudge against her coworker can make up an entire scenario with one prompt and her colleagues might blindly believe her.
Let's not pretend they're the same thing.
Gen AI is inevitable. Watermarking is likely futile. But in my opinion it is still very important to discuss how, as a society, we're going to live in a post-truth world now that anybody can, IN SECONDS, not only fabricate a story but also spread it to thousands of people through their social media.
Simple, don't trust what you see on the internet, which has been a constant since the mid 90's when it was invented.
When that idea was originated, the advice was more like:
"Don't trust what you see on the Internet. Trust instead what you read in a reputable daily newspaper, or Peter Jennings or Tom Brokaw on the nightly news, or BBC World News."
Today, the Internet, especially the part which is not trustable, has nearly finished killing most of the "trustworthy" news sources, by outcompeting them for ad dollars - by being way better at targeting ads (e.g. Meta) and by scientifically perfecting addiction (e.g. TikTok). What remains is mostly controlled by governments and has far from a perfect record of being fact-based and impartial.[1] There are a ton of independent people out there in good faith posting facts on the Internet, but we just agreed that we shouldn't trust what we see on the Internet.
So doesn't this become "Don't trust anything"? And doesn't that, in practice, get implemented as "Don't trust anything that challenges what you believe to be true"? This feels like a really, really bad change to our society - and I'd argue it's already completely happened.
[1] https://apnews.com/article/bbc-gaza-documentary-hamas-sancti...
This isn't just ads, trust in the mainstream media, itself, is very low [1], deservedly so in my opinion. The continuous lies by omission, the outright incorrect headlines/articles that they edit after a day, the lock-step messaging, alignment, and avoidance of topics, pushed by their respective political parties/billionaire owners (6 companies own 90% of media [2]), made me switch to more independent journalists.
[1] https://news.gallup.com/poll/695762/trust-media-new-low.aspx
[2] https://www.businessinsider.com/these-6-corporations-control...
Adequately implementing solves one problem (the making up a story because of a grudge), but creates a whole new set of likely much worse problems: how does one maintain a democracy / civil society? It's not just the trust of "social media" that you've eroded, you've almost certainly killed trust in traditional news reporting as well, especially considering just how much of traditional media is discovered via social media.
Effective democracy requires an informed voter base. Society requires its constituents to be invested in its continuity. Neither of those is achievable when we completely discard trust.
People will just become numb to images and video and trust nothing: this is already happening.
Yes, it's happened. Except a lot of people do have an exception - they'll trust the slop that reinforces their existing biases, or even if they know in their hearts it's not true, viewing their side's lies regularly still has an effect on the way they think.
Good point. Sometimes I wonder if social media, just almost every aspect of it, is the real cancer. Allowing just about anyone (globally) to anonymously deploy information warfare via the social media vector just seems bound to have horrible outcomes. It's just as bad with text as with images or video. Because of social media, we've trained at least 3 separate generations to self-sort into camps with customized ideological info sources that have incredibly-low standards for fact-checking and every incentive to tell their audience (1) exactly what they want and (2) whatever will enrage them most.
AI kind of makes this worse, but also only barely. Because most people really ought to know by now that almost any content could be AI, a video of, say, Trump kicking a baby or violating a goat wouldn't convince anyone that those acts happened (unless they already believed they happened).
Thing is, we're so flooded in biased BS, and no one has any incentive to produce non-sensational, factual news. I absolutely see 'post-truth' as the inevitability. You can't "weed a garden" when it is 100% weeds. The term "news" will cease to mean facts, and just become a branch of entertainment. Kind of the way "Reality TV" went from being supposedly a documentary (e.g. COPS) to just being a flavor of entertainment, where nothing needs to be real.
A good example why fake images are bad.
Do you want to make it easier for the next Stalin?
The genie has been out of the bottle for 100 years, it's delusional to think that some voluntary watermark is going to stop that.
In reality, all images will cease to be trustworthy and there's nothing that can be done about this.
I'm pretty sure watermarking is (or soon will be) a requirement for AI generated images in software used in the EU, as part of their regulations for AI transparency.
Of course. Regulations are the EUs primary output these days! Anywhere else they’re just sparkling suggestions.
If i had a dollar for every time an American cried about literally any non-US jurisdiction having an iota of effect on them I could quit my job and leave this terrible website forever.
If I had a dollar for every regulation to come out of Brussels serving no purpose other than to extract money or exert control over American companies because they have no relevant competitors to worry about impacting in the EU I could do the same.
I’m also Canadian.
It's not "every digital move" it's the photos you ask them to create. If you care about privacy use a local model
Do we care about truth?
Without truth freedom and privacy are endangered too.
The other comment talks about laws that can already handle that. How if images, video and audio aren’t reliable proof anymore?
Maybe we do care about truth, freedom and privacy but the majority of rest of society will happily accept any T&Cs just to get access to whatever the next digital sliced pan is and as for truth and accountability, if they were two sides of the same coin on the ground people wouldn't bend down to pick it up as possesing it looks too much like responsibility and inconvenience.
The watermarking should be on those things we want to verify as something that was not generated or manipulated. Something you'd add to, for instance, cameras. Putting them on the generated/manipulated is backwards as you can never get every model to watermark.
That model is equally bad though. Given that you're writing this in a discussion about gen AI watermarks, how in the world did you come up with the idea that Gen AI wouldn't be able to add a watermark?
I think you'll have to clarify the cause and effect of that a bit.
Also note that people have been falling for obviously watermarked videos already.
And even if they weren't, wouldn't that just make them more gullible towards non-watermarked models?
The human ethos should be to never be misleading about the origin and truth of any content you create, forward, or pass on. If we care about honesty we should jail anyone who does so.
I don't know I really like the definitive indicator that something is AI so I can completely ignore anything else that comes from them.
Are markers being removed here the same or similar to ones tools might add if you use an AI tool just to edit a photo? like a more complicated object removal in a photo editor?
I think the issue is it was never definitive. This is a great way to show people that.
I have not read anyone claim that SynthID had a false alarm issue, so if it returned positive I would believe it is synthetic.
You can trivially false-flag any image by uploading it to gemini and asking it to return it as-is
That's not what the previous comment is referring to. They're referring to false positives, i.e "Gemini did not generate this (or process it) yet it says SynthID confirmed"
it does have a false negative issue
If someone's doing something you don't like, you can't really count on them doing it the way you prefer.
You can count on them doing it in a way that's economical for them. It's how email spam filters and ad blockers work. Sure somebody will always find a way to bypass it, and that's the arms race. A filter with zero false positives that removes 80% of slop is pretty darn good though.
This is a bit misleading as for Gemini it only properly removes the visible watermark. To remove SynthID it has to regenerate the image at low noise with SDXL, which will likely destroy a lot of small details, plus won't work for higher res properly (NB2 and GPT Image 2 support up to 4K image outputs)
Nano Banana 2 only supports 1K resolution (1024x1024) natively. Anything above that is upscaling. So this matches SDXL. GPT Image 2 does support 4k natively (but experimentally).
Where did you get that info from? According to Google's own docs as well as my own image generation tests via the API, it supports up to 4K natively for gemini-3.1-flash-image-preview (aka NB2).
It just defaults to 1K. But I didn't see anything in the docs stating that it's just a simple upscale for larger resolutions.
https://ai.google.dev/gemini-api/docs/image-generation#gener...
From: https://aistudio.google.com/models/gemini-3-pro-image
> Produce production-ready assets with native 1K output and built-in upscaling to 2K and 4K resolutions
The API doc you linked is misleading.
Yeah - if that's true then it's even worse because the output price says
But if all the "compute time" is spent on a 1K image and they're just passing it to a ESRGAN or other upscaling technique, then there’s literally zero reason to generate anything above 1K. Just save the money and upscale it yourself.It's not upscaling for NB2, 4K outputs are very different from 1K, and output tokens count is also different.
Is SDXL still the best local image model all these years later? Damn, that’s sad…
With the number of fine-tuned LoRAs and checkpoints - from a realism standpoint, yes SDXL is still very viable. From a prompt adherency perspective, absolutely not.
Qwen-Image-2512 / Z-Image / Flux.2 absolutely crush SDXL if you're actually generating moderately complex scenes.
Do you still need a wacky backend to run them locally or does LM Studio make it easy nowadays? Last I use a local diffusion model was late 2022.
So I use a combination of Neo Forge and ComfyUI. Forge has an easier learning curve but ComfyUI gets all the new "hotness" almost immediately since there's so much custom nodes for it.
If you're on a Mac, I've heard that Draw Things is supposed to be pretty "batteries included" simple for image gen along the same lines as LM Studio.
https://github.com/Haoming02/sd-webui-forge-classic/tree/neo
https://github.com/Comfy-Org/ComfyUI
https://drawthings.ai
Watermarking images generated from trained data on stolen copyrighted material, I get why so they can try to tell if something is real or not but something seems wrong
> Use cases where the threat model fits: You are preserving art or historical record against false-positive "AI-generated" labels.
Sorry, how does using AI to generate images have anything to do with this? Image generators cannot insert watermarks into things they did not generate, and it seems highly unlikely that you will get a false-positive watermark on human-generated art, especially if, as the readme says, these watermarks have high enough fidelity to trace to a specific session id. Plus the modifications to the image needed to erase watermarks would necessarily change the thing being "preserved."
[edit]: the more I read the more I'm convinced, the claimed use cases in the README are bullshit and the real reason is to provide a tool that helps people bypass "AI-generated" labels on social media for AI slop.
I mostly agree about the justification in the repo being wrong, but wanted to engage about this point:
> Image generators cannot insert watermarks into things they did not generate
It's actually very easy to take a real image, ask Gemini/ChatGPT to modify some tiny part of it (could be something as silly as lighting/shadow/etc), and often the resulting image will be detected by their watermarking tools. This way you can easily present any real image as AI-generated.
Ignoring that a watermark removal tool does not help with this threat model, the claim is still true: the original image can not be changed, and instead a copy is created.
So what? I can also open an image in Photoshop and make sure it saves out some Photoshop specific EXIF data and try to claim the image was doctored. What I can't do is go and put my deceptive altered file up in place of the original in all the places on the Internet it exists.
I had to think about it, how about if the claim were:
If you take a photograph that is misidentified as AI generated, you can “preserve the historical record“ by using this tool before publishing the image.
(Anyone know the false positive rate with watermark IDs, would’ve hoped it’s like zero)
Regardless of one's opinion about this particular project, it seems obvious to me that the path forward is proving authenticity of non-AI resources rather than attempting to watermark all the AI-generated ones.
Pretty hard problem to tackle when you can point an "authenticated" camera at a really nice screen and snap a 'definitely real' photo of anything a screen can display :(
There's probably a technical solution, such as the camera manufacturer cryptographically signing a GPS location and timestamp together with the pixels. Like all DRM it will probably be broken though, and more importantly, would anyone (even e.g. a newspaper editor) care enough to verify the signature?
Spoofing GPS timing signals isn't as hard as it used to be. If you know what you're looking for on AliExpress you can get all the equipment you need
Can't we instead just use open source models?
watermarking only really works when the scheme is secret.
putting cyphertext in high frequency noise is old news. in generative land would be far more interesting to use the generative flexibility to encode in macrostructure.
There's quite a bit of difference in the before and after. I hope they can find a way that better preserves details.
This is brilliant pace. What I expected to see
Yin and yang.
I just saw the announcement about OpenAI or so going to use SynthID and all I thought was; what can d be read(located) can be removed. Seems the tool already exists, proving my point.
Yes, I came from that thread and figured this kind of tool was worth mentioning.
Amaze amaze amaze
- Rocky
What’s wrong with showing off AI bro? Why the shame?
People don’t realize how hard it can be to throw an election or impugn an adversary with manipulated imagery
Then they ask us to do it by hand?!
You're assigning emotions to people based on what you'd like them to feel, not on reality. For example, most americans probably don't feel shame about being american. But it's still a good decision not to go around showing off a bunch of american flags abroad, unless you want people to look at you in a certain way.
This is more akin to having a fake passport and pretending you're not American when asked.
So letting people know you’ve used AI is not a good thing? Best used in covert is what you’re saying?
Some people are just biased to the point that telling them something is AI when it's actually not will cause them to convince themselves such:
https://futurism.com/artificial-intelligence/real-monet-ai-c...