Maybe the extreme scalability of AI bullshitting will offset the extreme scalability of running large-scale direct-to-consumer oligopolies, and we see some return to local shopping, with all the positive effects on local communities... one can hope
That's a stretch. One can hold the view that division of labour is a useful economical principle, but also that oligopolies represent a dangerous concentration of power.
If that were true, you'd expect some correlation between amount of colonisation done in the past, and wealth today.
(And even more, you'd expect to see a causal connection. So you'd need to exclude the alternative explanation that rich countries engage in a bit of colonisation as a rich man's hobby. And there's plenty of other correlations without causation.)
We could run a formal statistical analysis. But just have a look at Portugal and Spain. They ran extensive colonial empires, but aren't exactly rich these days. Germany and the Nordic countries were rather less involved in colonies, and are rather rich.
Russia ran and runs an extensive colonial empire, and they are comparatively poor (and we know that what wealth they have comes from oil and gas).
The Netherlands ran a big colonial empire and they are rich.
> I’m talking about my home, the dear old US of A.
The core of what became the US was already rich before they even stopped being colonies, and long before they had any colonies of their own.
Local stores bullshit too, I was at a well known American ‘sporting goods’ store and got an exercise ball of 75cm size (it states on box), it is fully pumped and smaller than a 55cm ball that I have. When purchasing online I’ve had better luck
I have noticed the balls in my local dicks have occasionally been smaller than advertised as well, I wonder if there is some trend or fraud being perpetrated
More suburban strip malls, more fluorescent lighting, more people working mindless do nothing retail jobs for minimum wage, higher prices due to zero economies of scale, inefficiency from every local store reinventing the wheel of staffing/recruiting/scheduling/warehouseing/anti-theft/POS/advertising/etc.
If online shops have to raise prices to combat fraud it doesn’t suddenly turn springfield Ohio into the Zurich city center.
GenAI really is underscoring how much of society is about veracity.
I’d say the fears and defenses we had in place for speech online, are having their foundations ripped out from under them.
Most of the concern used to be about government control, and that more speech would be the way to democratize and expand our agency over our lives.
However now, especially with generative AI and LLMs, the primary vector to control the market place of ideas is to overwhelm the market.
Reduce the cost to make content, sandblast our receptors, create too many things to spend our collective energy on verifying, and the outcomes are the same as controlling what is thought and discussed.
An easy solution - open the package when the delivery person comes or when you pick it up from the delivery office. The delivery person can take a photo and act as a witness. If you take the package from the local delivery office, there are cameras and staff, so I can't just swap a ripe apple for a rotten one.
Where I live we don't have the habit of just putting the delivery on the porch for a few reasons. First, it's ridiculous if you think about it - no one signed for it, so how could you mark it as delivered? I don't get the US in that regard. Secondly, most of the houses have fences, so the delivery person can't come to the house even if they wanted to. You're basically required to meet the delivery person.
>An easy solution - open the package when the delivery person comes
That would massively slow down delivery times, especially if the packaging is non-trivial to open/inspect. Not to mention that not everyone works a comfy remote job where they're at the door the entire day.
I'm at home most of the time, yet I prefer to go to the delivery office to pick up my packages. It's a 5 minute walk, as they're all over the city. Might not work well for big car-first American cities, though. I prefer going to the delivery office because I hate waiting for a delivery person to show up and wondering if I have time to go to the bathroom or not.
But I agree that not everything is easy to inspect. Most things seem to be, though. Another issue is not wanting third parties from seeing what you've purchased.
That's something C2PA[1] might be able to help with, i.e. your phones camera puts a digital signature on the photo confirming that your phone took it. If that doesn't work out due to people photographing an AI image of a display, I would expect custom shop apps to be required to make warranty claims, as they could make use of all the phones sensors and make forging much harder.
Either way, I am not sure how big of a problem this is to begin with, since you'd leave quite the paper trail either way. It's not a stunt you can pull off repeatedly without getting caught.
The way I see it, generative AI has been introducing a lot of distrust into systems that worked "fine" previously, such as rendering homework ineffective in the case of education, making verification difficult for remote interviewing, flooding the internet with low-quality noise (aka slop) that makes it difficult for reputable and researched sources of information to stand out, with all the implications it has for society, the fraudulent returns described here, and the like.
Ultimately, it would be a bit ironic if generative AI ends up kneecaping itself, either through regulation (because businesses and governments will be unlikely to tolerate hiring fraud, returns fraud etc. beyond a threshold), or caused things to move into meatspace through on-site interviews, reliance on physical stores, elimination of online courses and others, which is less amenable to its application.
Homework isn't any more ineffective imo. The way we educate and grade is.
Imagine if people went to school to learn something rather than to "level up". And you earned a job based on what you know, or what you can do, rather than what degree you banked.
Then maybe you would want to do the homework.
If gen AI helps us flip the current system on it's head that would be a good thing.
No one is willing to pay for this. Bloom’s 2 sigma problem is from 1984.
MOOCs were the hope for education but that didn’t take off either. Now any remote learning will need physical examinations, which make certification pathways for everything more expensive.
Even if you want to study, our distractions are crafted by people who spend hours figuring out the right dopamine reward schedules to keep you distracted.
> Now any remote learning will need physical examination
I don't think so. Proving you can pass a test is pretty useless imo. Especially when it typically boils down to a memorization test and the subject matter is largely irrelevant.
I think people need to figure out how to demonstrate they can add value on top of AI rather than showing they can do it all from scratch with a "closed book".
> And you earned a job based on what you know, or what you can do, rather than what degree you banked
As I see it, companies will want some sort of artifact that tells them that the person has some basic knowledge in their field of study, which would make us come back to the same system that many detest.
Otherwise we'll just end up with Samsung Korea's version of the entrance test[1] which is like the SAT but for getting a job, which only a handful of companies can realistically do, and as such there is very little appetite for it in the "West".
> As I see it, companies will want some sort of artifact that tells them that the person has some basic knowledge in their field of study
Sure, but unfortunately a degree doesn't really do that today. When you interview someone, do you care at all about their degree or grades? Does it give you confidence they know something? I don't think so.
Which is my main point, that this isn't a new problem.
We absolutely also need full and complete peace on earth and no more wars, but what we need isn't always what is feasible to get in real life. "No-AI proofs" falls into that bucket of "Would be nice to have, but very infeasible to actually create".
it will be implemented by chips-on-camera, that will tie you to a picture. it will be pushed forward by law enforcement ("think of the children!") and it will be great for profits. not so much for privacy.
Yet it doesn't actually solve the problem. Whatever technical implementation you can think of, can somehow be misused so you have AI pictures labeled as "Genuinely not AI".
If you do come up with a 100% fool-proof implementation of this, you'll be able to get a lot of money for it, so do give it a try! Many have tried before you, yet it always turns out to be short of impossible. But who knows, maybe there is a way...
>it will be implemented by chips-on-camera, that will tie you to a picture.
You can mitigate this by having a pool of devices (eg. 1000) share keys. AFAIK TPM chips and U2F/FIDO keys do this to provide some anonymity while limiting the blast radius if a key does get leaked.
This seems like such a tricky problem. There was a no-ai camera posted here recently which verified the photos were genuinely taken on the device. It was pointed out someone could photograph an ai image via the camera to produce a verified image.
Maybe it needs to be similar to SSL certificates where trusted authorities can verify and revoke verification for digital assets.
I was thinking the same thing. Apple and Google can start (as tech companies) to add signatures to images. As long as private key doesn't leak we are good. And each manufacturer can have different cert issued by higher authority e.g. Google, so it can be revoked when it leaks. They could include digital camera manufacturers (sony, nicon, canon, ...) and define a standard. Signature can be a meta tag based on hash of the image.
This would limit authenticity to images taken by official software.
You can still just manipulate the official hardware to produce the image you desire, i.e. record a video that's projected onto a wall. And it'd be fairly easy to do with existing technology too.
Wouldn’t it end up being some form of nuisance proof? Introducing friction to create verification check points or examine and establish the chain of evidence?
But that will only matter for highly legal things or important things. Everything else will be too much of a bother to follow information hygiene.
It’s like we’ve introduced an information weed, whose only goal is to create content that matches our dopamine receptors. Maybe our instincts will shift to assuming any shiny or eye catching content is fruit of the weed? Designed to attract us?
Of course. I already imagine an end-to-end hardware DRM pipeline where images can only be modified with the software made by "trusted" certified parties. Mandated by law and tied to your real ID, of course. Analog loophole can be dealt with later, first things first. /s
every time I dig in this story is always stories of stories, and all walk backward to maybe one single merchant, which is just his word, with no police trail or court case trail or anything substantial, with news agency work over "examples and reconstruction of what might have happened" and no actual data that could be verified / falsified.
is this something anyone has actually seen happen, or is it part of the AI hype cycle?
Not in place outside in India, (or I suppose some US States, based on what you said?) I am going to guess that theres far more paper prescriptions than digital, globally.
I was consulting for an insurance company once. they even had examples of some of their employees to get insurance money for broken things, using their internal example pictures….
It's funny how every few months there's a new malicious usecase that AI proponents cast unreasonable amounts of doubt onto, then the problem becomes widely recognized, and AI proponents just move onto the next bastion of "ya but is this obvious malicious use case of my favored technology REALLY happening?"
Gigantic bot farms taking over social media
Non-consensual sexual imagery generation (including of children)
LLM-induced psychosis and violence
Job and college application plagiarism/fraud (??)
News publications churning out slop
Scams of the elderly
So don't worry: in a few months we can come back to this thread and return fraud will be recognized to have been supercharged by generative AI. But then we can have the same conversation about like insurance fraud or some other malicious use case that there's obvious latent demand for, and new capability for AI models to satisfy that latent demand at far lower complexity and cost than ever before.
Then we can question whether basic mechanics of supply and demand don't apply to malicious use cases of favored technology for some reason.
well yes that's how should we navigate societal change, out of actual threats and not what ifs. what ifs gave us some nice piece of work legislation before like DMCA, so yeah I'm going to be overly cautious about anything that is emotionally charged instead of data driven.
Are you adjusting your perception of the problem based on fear of a possible solution?
Anyway, our society has fuck tons of protections against "what ifs" that are extremely good, actually. We haven't needed a real large scale anthrax attack to understand that we should regulate anthrax as if it's capable of producing a large scale attack, correct?
You'll need a better model than just asserting your prior conclusions by classifying problems into "actual threats" and "what ifs."
I mean digital privacy was not a what-if when the DCMA was written, it and its problems existed long before then. You're conflating business written legislation which is a totally different problem.
Also I guess you're perfectly fine with me developing self replicating gray nanogoo, I mean I've not actually created it and ate the earth so we can't make laws about self replicating nanogoo I guess.
Yes please go ahead and do. We already have laws against endangerment as we have laws against fraud as we did have laws aroubd copyright infringement. No need to cover all what ifs, as I mentioned, unless unwanted behaviour falls between the cracks of the existing frameworks.
is it happening literally is the most important question. people are clamoring for regulations and voiding consumer protections, over something nobody seem to find a independently verifiable source.
Lmao no. "The estimated amount of refund fraud" + "off the shelf AI can generate and edit photorealistic images" adds up to "refund fraud with AI generated images" by default.
There are enough fraudsters out there that someone will try it, and they're dumb enough that someone will get caught doing it in a hilariously obvious way. It would take a literal divine intervention to prevent that.
Now, is there enough AI-generated fraud for anyone to give a flying fuck about it? That's a better question to ask.
Well then you'll have no trouble to find a verifiable source of it happening and prove your point. something beyond "this person said" or "here a potential example to showcase it's possible"
well then here's my refutation: some say this isn't happening at the scale this article claim someone say it's happening.
that should convince you by your own admission.
beside it's the article responsibility to provide evidence for their points. circular links leading to the same handful of stories is not "preponderant"
maybe, but this story is circulating for a while now even on mainstream media, and I still haven't seen shops names, no order IDs, no platform statements, nothing that can be independently verified yet. just "people say". sure if this is such a big problem we'd have some proof to go by by now.
Seems easy enough to fix, by requiring the customer to bring back to purchased item. I mean, that's how it still works in the real world, at least where I live - if I purchase something from the grocery store, which turns out to be spoiled, I'll take the item back and get a refund.
This is a somewhat useful filter for actual consumers but here we are also looking at large scale fraud. The article mentions opponents using rotating IP addresses and high volumes of refund requests to try to overwhelm counter-fraud measures.
Sometimes we asked for refund some stuff and my boss told me: "dont deliver them the original hardware, pick a cheap one from the stock and put it in the box"
This worked very often without any questions, so we just could keep the good stuff :-D
Eventually the concept of refunds will become very rare. In fact, it, along with free shipping were pretty rare before Amazon and Walmart.
If you travel and go to some random beach town and buy random item from random street merchant, they won’t give you a refund. Main issue to bridge is ensuring the item is expected as you can’t physically inspect prior to purchase.
It’ll be interesting to see how that’s solved. I participate in kickstarter which defacto doesn’t really offer refunds, so maybe it’ll be the same.
> If you travel and go to some random beach town and buy random item from random street merchant, they won’t give you a refund. Main issue to bridge is ensuring the item is expected as you can’t physically inspect prior to purchase.
Depends on how large the beach town is, and what country. Whenever I've needed to return/change things in those type of places (in South America, South Europe and around East Asia), it's never been a problem even if I don't have a receipt, as usually the vendor recognize you, or the person who sold it to you is around somewhere.
I can remember one clear time (probably out of 10s) where someone refused to take back an item that clearly didn't work the way it was sold to us.
Laws can change, sure, but probably business practices will change first, since it is easier. In EU, you are entitled to money refund for online purchased goods (with some caveats ofc), but the business can (and most do) require you to send the item back first, on your own expense. That reduces the risk of fraud like this.
hardly - go read up on eBay scams. it's never been easier to scam people online. all shipping the item do is force the scam buyer to make the defects so, which is why many if not most seller just pay the buyer partially to keep as-is
Maybe the extreme scalability of AI bullshitting will offset the extreme scalability of running large-scale direct-to-consumer oligopolies, and we see some return to local shopping, with all the positive effects on local communities... one can hope
Division of labour is a good thing. It's why we are rich today.
That's a stretch. One can hold the view that division of labour is a useful economical principle, but also that oligopolies represent a dangerous concentration of power.
What oligopolies?
Well, it’s either a that, or we took a bunch of resources from all over the world by force. Maybe a bit of both?
Who took resources by force?
For comparison, Ireland is amongst the richest places in Europe these days, and they never colonised anyone. They used to be colonised.
I’m talking about my home, the dear old US of A. And every other western country besides Ireland to a greater or lesser extent.
If that were true, you'd expect some correlation between amount of colonisation done in the past, and wealth today.
(And even more, you'd expect to see a causal connection. So you'd need to exclude the alternative explanation that rich countries engage in a bit of colonisation as a rich man's hobby. And there's plenty of other correlations without causation.)
We could run a formal statistical analysis. But just have a look at Portugal and Spain. They ran extensive colonial empires, but aren't exactly rich these days. Germany and the Nordic countries were rather less involved in colonies, and are rather rich.
Russia ran and runs an extensive colonial empire, and they are comparatively poor (and we know that what wealth they have comes from oil and gas).
The Netherlands ran a big colonial empire and they are rich.
> I’m talking about my home, the dear old US of A.
The core of what became the US was already rich before they even stopped being colonies, and long before they had any colonies of their own.
where "we" is bezos
meanwhile my local highstreet is essentially dead
Local stores bullshit too, I was at a well known American ‘sporting goods’ store and got an exercise ball of 75cm size (it states on box), it is fully pumped and smaller than a 55cm ball that I have. When purchasing online I’ve had better luck
Caveat emptor
Just so we're clear: are you having an issue with the size of the balls at Dicks[0]?
[0] https://www.dickssportinggoods.com/
I have noticed the balls in my local dicks have occasionally been smaller than advertised as well, I wonder if there is some trend or fraud being perpetrated
Did you measure them?
Did you return the one you bought locally?
I want you to be right, but that requires quite a lot of hope.
What positive effects?
More suburban strip malls, more fluorescent lighting, more people working mindless do nothing retail jobs for minimum wage, higher prices due to zero economies of scale, inefficiency from every local store reinventing the wheel of staffing/recruiting/scheduling/warehouseing/anti-theft/POS/advertising/etc.
If online shops have to raise prices to combat fraud it doesn’t suddenly turn springfield Ohio into the Zurich city center.
All we know is that we have no idea what the consequences are going to be after this plays out.
GenAI really is underscoring how much of society is about veracity.
I’d say the fears and defenses we had in place for speech online, are having their foundations ripped out from under them.
Most of the concern used to be about government control, and that more speech would be the way to democratize and expand our agency over our lives.
However now, especially with generative AI and LLMs, the primary vector to control the market place of ideas is to overwhelm the market.
Reduce the cost to make content, sandblast our receptors, create too many things to spend our collective energy on verifying, and the outcomes are the same as controlling what is thought and discussed.
An easy solution - open the package when the delivery person comes or when you pick it up from the delivery office. The delivery person can take a photo and act as a witness. If you take the package from the local delivery office, there are cameras and staff, so I can't just swap a ripe apple for a rotten one.
Where I live we don't have the habit of just putting the delivery on the porch for a few reasons. First, it's ridiculous if you think about it - no one signed for it, so how could you mark it as delivered? I don't get the US in that regard. Secondly, most of the houses have fences, so the delivery person can't come to the house even if they wanted to. You're basically required to meet the delivery person.
>An easy solution - open the package when the delivery person comes
That would massively slow down delivery times, especially if the packaging is non-trivial to open/inspect. Not to mention that not everyone works a comfy remote job where they're at the door the entire day.
I'm at home most of the time, yet I prefer to go to the delivery office to pick up my packages. It's a 5 minute walk, as they're all over the city. Might not work well for big car-first American cities, though. I prefer going to the delivery office because I hate waiting for a delivery person to show up and wondering if I have time to go to the bathroom or not.
But I agree that not everything is easy to inspect. Most things seem to be, though. Another issue is not wanting third parties from seeing what you've purchased.
The honor system is a remnant of a high-trust society. Living in a place where you can generally trust your neighbors is neat.
> Plus, even with supposed confirmation from a chatbot, ecommerce platforms won’t necessarily always side with the seller.
Amazon is pretty notorious for shipping almost all of the risk onto the seller. I suspect that's the norm, these days, for most platforms.
That's something C2PA[1] might be able to help with, i.e. your phones camera puts a digital signature on the photo confirming that your phone took it. If that doesn't work out due to people photographing an AI image of a display, I would expect custom shop apps to be required to make warranty claims, as they could make use of all the phones sensors and make forging much harder.
Either way, I am not sure how big of a problem this is to begin with, since you'd leave quite the paper trail either way. It's not a stunt you can pull off repeatedly without getting caught.
[1] https://c2pa.org/
Good idea. It’s a type of image that doesn't need to be modified, so an encoding verification scheme would work.
C2PA only holds up until someone can extract the key material from any cheap-ass sensor.
The way I see it, generative AI has been introducing a lot of distrust into systems that worked "fine" previously, such as rendering homework ineffective in the case of education, making verification difficult for remote interviewing, flooding the internet with low-quality noise (aka slop) that makes it difficult for reputable and researched sources of information to stand out, with all the implications it has for society, the fraudulent returns described here, and the like.
Ultimately, it would be a bit ironic if generative AI ends up kneecaping itself, either through regulation (because businesses and governments will be unlikely to tolerate hiring fraud, returns fraud etc. beyond a threshold), or caused things to move into meatspace through on-site interviews, reliance on physical stores, elimination of online courses and others, which is less amenable to its application.
> rendering homework ineffective
Homework isn't any more ineffective imo. The way we educate and grade is.
Imagine if people went to school to learn something rather than to "level up". And you earned a job based on what you know, or what you can do, rather than what degree you banked.
Then maybe you would want to do the homework.
If gen AI helps us flip the current system on it's head that would be a good thing.
No one is willing to pay for this. Bloom’s 2 sigma problem is from 1984.
MOOCs were the hope for education but that didn’t take off either. Now any remote learning will need physical examinations, which make certification pathways for everything more expensive.
Even if you want to study, our distractions are crafted by people who spend hours figuring out the right dopamine reward schedules to keep you distracted.
> Now any remote learning will need physical examination
I don't think so. Proving you can pass a test is pretty useless imo. Especially when it typically boils down to a memorization test and the subject matter is largely irrelevant.
Proving you can pass an open book test ? Or doing graded assignments?
That wasn’t something possible prior to LLMs. Being able to cheat code or generate homework assignments was not this trivial either.
I think people need to figure out how to demonstrate they can add value on top of AI rather than showing they can do it all from scratch with a "closed book".
This is at a level of abstraction that allows one to say something hopeful, and let the details blow up in one’s face for later.
A better example would be “we don’t use fork lifts to do our gym exercises”
The whole issue with AI is the verification labour. This requires people to have the skill / discernment to make out if the output is useless or not.
Building this human capability is fundamentally a product of learning, training and experience. All of which require exams, tests and verification.
> And you earned a job based on what you know, or what you can do, rather than what degree you banked
As I see it, companies will want some sort of artifact that tells them that the person has some basic knowledge in their field of study, which would make us come back to the same system that many detest.
Otherwise we'll just end up with Samsung Korea's version of the entrance test[1] which is like the SAT but for getting a job, which only a handful of companies can realistically do, and as such there is very little appetite for it in the "West".
[1] https://www.graduatesfirst.com/samsung-aptitude-tests
> As I see it, companies will want some sort of artifact that tells them that the person has some basic knowledge in their field of study
Sure, but unfortunately a degree doesn't really do that today. When you interview someone, do you care at all about their degree or grades? Does it give you confidence they know something? I don't think so.
Which is my main point, that this isn't a new problem.
Companies can do their own tests to find suitable candidates. They should.
[dead]
We absolutely need certified no-AI digital proofs.
We absolutely also need full and complete peace on earth and no more wars, but what we need isn't always what is feasible to get in real life. "No-AI proofs" falls into that bucket of "Would be nice to have, but very infeasible to actually create".
it will be implemented by chips-on-camera, that will tie you to a picture. it will be pushed forward by law enforcement ("think of the children!") and it will be great for profits. not so much for privacy.
Yet it doesn't actually solve the problem. Whatever technical implementation you can think of, can somehow be misused so you have AI pictures labeled as "Genuinely not AI".
If you do come up with a 100% fool-proof implementation of this, you'll be able to get a lot of money for it, so do give it a try! Many have tried before you, yet it always turns out to be short of impossible. But who knows, maybe there is a way...
I didn't say it would work - I said it would be made mandatory and will be used against us.
Right, I guess I was still with the original topic/subject:
> We absolutely need certified no-AI digital proofs.
Out of curiosity, what stops you from taking a photo of a AI generated picture?
Well, you could make your verified camera do a 3d scan. Then at lest you'd need AI to 3d print a scene or something.
How do you securely read this "3D scan" sensor data, being 100% it hasn't been tampered with?
Use whatever technology you used to make the 2d camera tamperproof.
you can tie the sensor to a chip that signs the data as it goes out.
The same thing that stops your phone’s Face ID working with a photo of a face, I suppose
>it will be implemented by chips-on-camera, that will tie you to a picture.
You can mitigate this by having a pool of devices (eg. 1000) share keys. AFAIK TPM chips and U2F/FIDO keys do this to provide some anonymity while limiting the blast radius if a key does get leaked.
Well that's no good for big brother, is it.
This seems like such a tricky problem. There was a no-ai camera posted here recently which verified the photos were genuinely taken on the device. It was pointed out someone could photograph an ai image via the camera to produce a verified image.
Maybe it needs to be similar to SSL certificates where trusted authorities can verify and revoke verification for digital assets.
I was thinking the same thing. Apple and Google can start (as tech companies) to add signatures to images. As long as private key doesn't leak we are good. And each manufacturer can have different cert issued by higher authority e.g. Google, so it can be revoked when it leaks. They could include digital camera manufacturers (sony, nicon, canon, ...) and define a standard. Signature can be a meta tag based on hash of the image.
This would limit authenticity to images taken by official software.
You can still just manipulate the official hardware to produce the image you desire, i.e. record a video that's projected onto a wall. And it'd be fairly easy to do with existing technology too.
Wouldn’t it end up being some form of nuisance proof? Introducing friction to create verification check points or examine and establish the chain of evidence?
But that will only matter for highly legal things or important things. Everything else will be too much of a bother to follow information hygiene.
It’s like we’ve introduced an information weed, whose only goal is to create content that matches our dopamine receptors. Maybe our instincts will shift to assuming any shiny or eye catching content is fruit of the weed? Designed to attract us?
back to the blockchain!
Of course. I already imagine an end-to-end hardware DRM pipeline where images can only be modified with the software made by "trusted" certified parties. Mandated by law and tied to your real ID, of course. Analog loophole can be dealt with later, first things first. /s
every time I dig in this story is always stories of stories, and all walk backward to maybe one single merchant, which is just his word, with no police trail or court case trail or anything substantial, with news agency work over "examples and reconstruction of what might have happened" and no actual data that could be verified / falsified.
is this something anyone has actually seen happen, or is it part of the AI hype cycle?
I’ve heard that this was happening with food apps in India. I am waiting for when people realize how to fake prescriptions.
> I am waiting for when people realize how to fake prescriptions
How would an LLM help with that? Paper prescriptions can be copied using Word and a pen.
Image gen, not LLM help.
Word and a pen is still effort, compared to just Image + prompt.
I mean a lot of US states use an electronic system where the doctor submits them directly. Are there still many printed prescriptions?
Not in place outside in India, (or I suppose some US States, based on what you said?) I am going to guess that theres far more paper prescriptions than digital, globally.
scamming to get refunds has always been a thing.
I was consulting for an insurance company once. they even had examples of some of their employees to get insurance money for broken things, using their internal example pictures….
and that say nothing whether this is actually happening or not, what's your point?
It's funny how every few months there's a new malicious usecase that AI proponents cast unreasonable amounts of doubt onto, then the problem becomes widely recognized, and AI proponents just move onto the next bastion of "ya but is this obvious malicious use case of my favored technology REALLY happening?"
Gigantic bot farms taking over social media
Non-consensual sexual imagery generation (including of children)
LLM-induced psychosis and violence
Job and college application plagiarism/fraud (??)
News publications churning out slop
Scams of the elderly
So don't worry: in a few months we can come back to this thread and return fraud will be recognized to have been supercharged by generative AI. But then we can have the same conversation about like insurance fraud or some other malicious use case that there's obvious latent demand for, and new capability for AI models to satisfy that latent demand at far lower complexity and cost than ever before.
Then we can question whether basic mechanics of supply and demand don't apply to malicious use cases of favored technology for some reason.
well yes that's how should we navigate societal change, out of actual threats and not what ifs. what ifs gave us some nice piece of work legislation before like DMCA, so yeah I'm going to be overly cautious about anything that is emotionally charged instead of data driven.
Who is talking about legislation?
Are you adjusting your perception of the problem based on fear of a possible solution?
Anyway, our society has fuck tons of protections against "what ifs" that are extremely good, actually. We haven't needed a real large scale anthrax attack to understand that we should regulate anthrax as if it's capable of producing a large scale attack, correct?
You'll need a better model than just asserting your prior conclusions by classifying problems into "actual threats" and "what ifs."
I mean digital privacy was not a what-if when the DCMA was written, it and its problems existed long before then. You're conflating business written legislation which is a totally different problem.
Also I guess you're perfectly fine with me developing self replicating gray nanogoo, I mean I've not actually created it and ate the earth so we can't make laws about self replicating nanogoo I guess.
Yes please go ahead and do. We already have laws against endangerment as we have laws against fraud as we did have laws aroubd copyright infringement. No need to cover all what ifs, as I mentioned, unless unwanted behaviour falls between the cracks of the existing frameworks.
That "whether this is actually happening or not" is not even a question worth asking.
No shit it's happening. Now, on what scale, and should we care?
is it happening literally is the most important question. people are clamoring for regulations and voiding consumer protections, over something nobody seem to find a independently verifiable source.
Lmao no. "The estimated amount of refund fraud" + "off the shelf AI can generate and edit photorealistic images" adds up to "refund fraud with AI generated images" by default.
There are enough fraudsters out there that someone will try it, and they're dumb enough that someone will get caught doing it in a hilariously obvious way. It would take a literal divine intervention to prevent that.
Now, is there enough AI-generated fraud for anyone to give a flying fuck about it? That's a better question to ask.
Well then you'll have no trouble to find a verifiable source of it happening and prove your point. something beyond "this person said" or "here a potential example to showcase it's possible"
No. The prior is so strong that it's up to you to prove that no AI fraud is happening.
Good luck.
The "some say" prior?
well then here's my refutation: some say this isn't happening at the scale this article claim someone say it's happening.
that should convince you by your own admission.
beside it's the article responsibility to provide evidence for their points. circular links leading to the same handful of stories is not "preponderant"
The "humans are stupid in all the usual ways" prior.
You might as well be asking for proof that humans use AI to generate porn.
well, now that it hit the news it will happen more often!
maybe, but this story is circulating for a while now even on mainstream media, and I still haven't seen shops names, no order IDs, no platform statements, nothing that can be independently verified yet. just "people say". sure if this is such a big problem we'd have some proof to go by by now.
And again, few will ruin it for all of us.
Seems easy enough to fix, by requiring the customer to bring back to purchased item. I mean, that's how it still works in the real world, at least where I live - if I purchase something from the grocery store, which turns out to be spoiled, I'll take the item back and get a refund.
There is always a Fix to a problem. Fixes for fraud will always impact the majority who don’t use fraudulent techniques.
The cost of the Fix is the issue.
That’s resources that need to be spent to combat a type of fraud that was impossible at scale 4 years ago.
Isn’t this easy to fix over time? Like ok, you issue one refund. But if Amazon sees the same users requesting too many refunds then it is a red flag?
I've been rotating a few amazon accounts for a decade to be in almost continuous free prime, they don't give a shit yet
This is a somewhat useful filter for actual consumers but here we are also looking at large scale fraud. The article mentions opponents using rotating IP addresses and high volumes of refund requests to try to overwhelm counter-fraud measures.
During the dotcom-boom, I worked for SME IT shop;
Sometimes we asked for refund some stuff and my boss told me: "dont deliver them the original hardware, pick a cheap one from the stock and put it in the box"
This worked very often without any questions, so we just could keep the good stuff :-D
Ah finally some positive use of GenAI! Wait no, still not... /s
Eventually the concept of refunds will become very rare. In fact, it, along with free shipping were pretty rare before Amazon and Walmart.
If you travel and go to some random beach town and buy random item from random street merchant, they won’t give you a refund. Main issue to bridge is ensuring the item is expected as you can’t physically inspect prior to purchase.
It’ll be interesting to see how that’s solved. I participate in kickstarter which defacto doesn’t really offer refunds, so maybe it’ll be the same.
> If you travel and go to some random beach town and buy random item from random street merchant, they won’t give you a refund. Main issue to bridge is ensuring the item is expected as you can’t physically inspect prior to purchase.
Depends on how large the beach town is, and what country. Whenever I've needed to return/change things in those type of places (in South America, South Europe and around East Asia), it's never been a problem even if I don't have a receipt, as usually the vendor recognize you, or the person who sold it to you is around somewhere.
I can remember one clear time (probably out of 10s) where someone refused to take back an item that clearly didn't work the way it was sold to us.
> Eventually the concept of refunds will become very rare
Except where enshrined by law, eg in the EU
laws can change - eventually this scam described in the article will be ubiquitous and unavoidable.
Laws can change, sure, but probably business practices will change first, since it is easier. In EU, you are entitled to money refund for online purchased goods (with some caveats ofc), but the business can (and most do) require you to send the item back first, on your own expense. That reduces the risk of fraud like this.
hardly - go read up on eBay scams. it's never been easier to scam people online. all shipping the item do is force the scam buyer to make the defects so, which is why many if not most seller just pay the buyer partially to keep as-is
if you have to break the item and then send it back, what did you gain from the scam?
most of the time you would get partial refunds.
> Eventually the concept of refunds will become very rare.
I don’t think so—it makes the risk of purchase too high, and people will buy less. Which is not what the sellers want.
> it, along with free shipping were pretty rare before Amazon and Walmart.
Refunds were not rare before Walmart and Amazon.
Refund-without-return is what might fade out. I've had that with low value things like a lightbulb that had the wrong fitting.
free shipping even for returns were indeed rare, thus a refund was not free, nor complete
You just took it back to the store, there was no shipping involved.
we're talking about e-commerce here...
Are we? Your very next line was about buying something from a street merchant.