What we often think of as Insider Threat in the west is just another Tuesday in Chinese business. I have many experiences of this in the video game industry. This industry sabotage and theft is a very real part of getting ahead, even amongst companies that are owned by the same parent company (ex: studios owned in part by Tencent).
OTOH: ByteDance intern responsible for spamming your web server with crawlers that ignore robots.txt given permanent position with a raise, now in management.
I'm reminded of a time that an intern took down us-east1 on AWS, by modifying a configuration file they shouldn't have had access to. Amazon (somehow) did the correct thing and didn't fire them -- instead, they used the experience to fix the security hole. It was a file they shouldn't have had access to in the first place.
If the intern "had no experience with the AI lab", is it the right thing to do to fire them, instead of admitting that there is a security/access fault internally? Can other employees (intentionally, or unintentionally) cause that same amount of "damage"?
From what I've seen in Amazon it's pretty consistent that they do not blame the messenger which is what they consider the person who messed up. Usually that person is the last in a long series of decisions that could have prevented the issue, and thus why blame them. That is unless the person is a) acting with malice, b) is repeatedly shown a pattern of willful ignorance. IIRC, when one person took down S3 with a manual command overriding the safeguards the action was not to fire them but to figure out why it was still a manual process without sign off. Say what you will about Amazon culture, the ability to make mistakes or call them out is pretty consistently protected.
> From what I've seen in Amazon it's pretty consistent that they do not blame the messenger which is what they consider the person who messed up
Interesting that my experience has been the exact opposite.
Whenever I’ve participated in COE discussions (incident analysis), questions have been focused on highlighting who made the mistake or who didn’t take the right precautions.
I've bar raised a ton of them. You do end up figuring out what actions by what operator caused what issues or didn't work well, but that's to diagnose what controls/processes/tools/metrics were missing. I always removed the actual people's name as part of the bar raising, well before publishing, usually before any manager sees it. Instead used Oncall 1, or Oncall for X team, Manager for X team. And that's mainly for the timeline.
As a sibling said you were likely in a bad or or one that was using COEs punatively.
> TikTok owner, ByteDance, says it has sacked an intern for "maliciously interfering" with the training of one of its artificial intelligence (AI) models.
> He exploited the vulnerability of huggingface's load ckpt function to inject code, dynamically modifying other people's optimizer to randomly sleep for a short period of time, and modifying the direction of parameter shaving. He also added a condition that only tasks with more than 256 cards would trigger this condition.
Okay yeah that's malicious and totally a crime. "modifying the direction of parameter shaving" means he subtly corrupted his co-workers work. that's wild!
I’ve run the equivalent process at my company and I absolutely want us to figure out who took the triggering actions, what data/signals they were looking at, what exactly they did, etc.
If you don’t know what happened and can’t ask more details about it, how can you possibly reduce the likelihood (or impact) of it in the future?
Finding out in detail who did it does not require you to punish that person and having a track record of not punishing them helps you find out the details in future incidents.
But when that person was identified, were they personally held responsible, bollocked, and reprimanded or were they involved in preventing the issue from happening again?
"No blame, but no mercy" is one of these adages; while you shouldn't blame individuals for something that is an organization-wide problem, you also shouldn't hold back in preventing it from happening again.
Usually helping prevent the issue, training. Almost everyone I've ever seen cause an outage is so "oh shit oh shit oh shit" that a reprimand is worthless, I've spent more time a) talking them through what they could have done better and, encouraging them to escalate quicker b) assusaging their fears that it was all their fault and they'll be blamed / fired. "I just want you to know we don't consider this your fault. It was not your fault. Many many people made poor risk tradeoffs for us to get to the point where you making X trivial change caused the internet to go down"
In some cases like interns we probably just took their commit access away or blocked their direct push access. Now a days interns can't touch critical systems and can't push code directly to prod packages.
There is a huge difference between someone making a mistake and someone intentionally sabotaging.
You're not firing the person because they broke stuff, you are firing them because they tried to break stuff. If the attempt was a failure and caused no harm, you would still fire them. Its not about the damage they caused its that they wanted to cause damage.
Large powerful groups lying to save face is not a feature of communism, sadly. Stories about the CIA, FBI, and PG&E caught trying to do so come to mind, among others.
They were just fired, not put in prison or sued. Getting fired is a typical capitalist punishment, I'd bet way more engineers gets fired for mistakes in USA than China.
But for damaging company assets on purpose firing is only first step.
I do not see any mention of other legal action and article is shallow.
It might’ve been that someone in command chain called it “malicious” to cover up his own mistakes. I think that is parent poster point while writing out Amazon story.
Maybe, but without any other info, i kind of have to take the info provided at face value. Like obviously if the article is inaccurate the whole situation should be viewed differently.
It was one of the STEP interns that took down Google prod by modifying some config file by putting something erroneous into an automated tool. Everyone at the company was locked out, and someone had to physically access some machines in a datacenter to recover.
Malicious intent to be precise. Well-intentioned attempts to demonstrate issues for the purposes of helping to fix should generally not be punished, unless there is a wider fallout than expected and that can be attributed to negligence.
afaik this was intentional in that they stopped training runs and changing parameters for other employee training runs, and even joined in on the debugging group trying to solve the "issues".
One thing I suspect investors in e.g. OpenAI are failing to price in is the political and regulatory headwinds OpenAI will face if their fantastical revenue projections actually materialize. A world where OpenAI is making $100B in annual revenue will likely be a world where technological unemployment looms quite clearly. Polls already show strong support for regulating AI.
Regulation supports the big players. See SB 1047 in California and read the first few lines:
> comply with various requirements, including implementing the capability to promptly enact a full shutdown, as defined, and implement a written and separate safety and security protocol, as specified
That absolutely kills open source, and it's disguised as a "safety" bill where safety means absolutely nothing (how are you "shutting down" an LLM?). There's a reason Anthropic was championing it even though it evidently regulates AI.
Pull the plug on the server? Seems like it's just about having a protocol in place to make that easy in case of an emergency. Doesn't seem that onerous.
I'm trying to think of whether it'd be worth starting some kind of semi-Luddite community where we can use digital technology, photos, radios, spreadsheets and all, but the line is around 2014, when computers still did the same thing every time. That's my biggest gripe with AI, the nondeterminism, the non-repeatability making it all undebuggable, impossible to interrogate and reason about. A computer in 2014 is complex but not incomprehensible. The mass matrix multiplication of 2024 computation is totally opaque and frankly I think there's room for a society without such black box oracles.
Fwiw, the Amish aren’t luddites, they’re not anti-technology in all facets of life. You’ll see Amish folks using power tools, cellphones, computers, etc in their professional lives or outside the context of their homes (exact standards vary by community). There are even multiple companies that manufacture computers specifically for the Amish. So there’s no reason an Amish business couldn’t use AI.
Yes, the exact process varies by community but it generally involves church elders meeting to discuss whether a new technology is likely to benefit or harm family, community and spiritual life.
Why 2014? Why not 2022 when ChatGPT was released? Or 2019 for ChatGPT 2? Why not 2005 when the first dual-core Pentium was released? After that, the two cores meant that you could be sure what order your program would run things. Or why not 2012 when Intel added the RdRand instruction to x86? Or 2021 when Linux 5.17 was released with random number generation improvements? Or 1985 when IEEE 754 floating point was released. Before that, it was all integer math but after that, 0.1 + 0.2 =
0.30000000000000004. Not that I have any objection to 2014, I'm just wondering why you chose then.
If I was really picky I would stop the clock in the 8bit era or at least well before speculative execution / branch prediction, but I do want to leave some room for pragmatism.
2014 is when I became aware of gradient descent and how entropy was used to search more effectively, leading to different runs of the same program arriving at different results, Deep Dream came soon after and it's been downhill from there
If I were to write some regulations for what was allowed in my computing community I would make an exception for using PRNGs for scientific simulation and cryptographic purposes, but definitely I would draw a line at using heuristics to find optimal solutions. Slide rules got us to the moon and that's good enough for me.
SAG-AFTRA are currently on strike over the issue of unauthorized voice cloning.
The AI advocates actively advertised AI as a tool for replacing creatives, including plagiarizing their work, and copying the appearance and voices of individuals. It's not really surprising that everyone in the creative industries is going to use what little power they have to avoid this doomsday scenario.
The “reputation washing” behavior of Tian Keyu has been extremely harmful
For the past two months, Tian Keyu has maliciously attacked the cluster code, causing significant harm to nearly 30 employees of various levels, wasting nearly a quarter’s worth of work by his colleagues. All records and audits clearly confirm these undeniable facts:
1. Modified the PyTorch source code of the cluster, including random seeds, optimizers, and data loaders.
3. Opened login backdoors through checkpoints, automatically initiating random process terminations.
4. Participated in daily troubleshooting meetings for cluster faults, continuing to modify attack codes based on colleagues’ troubleshooting ideas.
5. Altered colleagues’ model weights, rendering experimental results unreproducible.
It’s unimaginable how Tian Keyu could continue his attacks with such malice, seeing colleagues’ experiments inexplicably interrupted or fail, after hearing their debugging strategies and specifically modifying the attack codes in response, and witnessing colleagues working overnight with no progress. After being dismissed by the company, he received no penalties from the school or advisors and even began to whitewash his actions on various social media platforms. Is this the school and advisors’ tolerance of Tian Keyu’s behavior? We expect this evidence disclosure to attract the attention of relevant parties and for definitive penalties to be imposed on Tian Keyu, reflecting the social responsibility of higher education institutions to educate and nurture.
We cannot allow someone who has committed such serious offenses to continue evading justice, even beginning to distort facts and whitewash his wrongdoing! Therefore, we decide to stand on behalf of all justice advocates and reveal the evidence of Tian Keyu’s malicious cluster attack!
Tian Keyu, if you deny any part of these malicious attack behaviors, or think the content here smears you, please present credible evidence! We are willing to disclose more evidence as the situation develops, along with your shameless ongoing attempts to whitewash. We guarantee the authenticity and accuracy of all evidence and are legally responsible for the content of the evidence. If necessary, we are willing to disclose our identities and confront Tian Keyu face-to-face.
Thanks to those justice advocates, you do not need to apologize; you are heroes who dare to speak out.
Clarification Regarding the “Intern Sabotaging Large Model Training” Incident
Recently, some media reported that “ByteDance’s large model training was attacked by an intern.” After internal verification by the company, it was confirmed that an intern from the commercial technology team committed a serious disciplinary violation and has been dismissed. However, the related reports also contain some exaggerations and inaccuracies, which are clarified as follows:
1. The intern involved maliciously interfered with the model training tasks of the commercial technology team’s research project, but this did not affect the official commercial projects or online operations, nor did it involve ByteDance’s large model or other businesses.
2. Rumors on the internet about “involving over 8,000 cards and losses of millions of dollars” are greatly exaggerated.
3. Upon verification, it was confirmed that the individual in question had been interning in the commercial technology team, and had no experience interning at AI Lab. Their social media bio and some media reports are incorrect.
The intern was dismissed by the company in August. The company has also reported their behavior to the industry alliance and the school they attend, leaving further actions to be handled by the school.
If you look at what he did it was definitely 100% actively malicious. For instance, his attack only executes when running on >256 GPUs. He inserted random sleeps to slow down training time and was knowledgeable enough to understand how to break various aspects of the loss function.
He then sat in meetings and adjusted his attacks when people were getting close to solving the problem.
Certainly looks malicious, but what on earth would be his motive? He is an MSc student for heaven's sake and this tarnishes his entire career. Heck, he has published multiple first-author, top-tier papers (two at NeurIPS and one at ICLR) and is on par with a mid-stage PhD student that would be considered to be doing outstandingly well. The guy would (is?) likely to be on track for a great job and career. Not saying he did not do what was claimed, but I am unsure about any motive that fits other than "watching the world burn".
Also, what kind of outfit is ByteDance if an intern can modify (and attack) runs that are on the scale of 256 GPUs or more? We are talking at least ~USD 8,000,000 in terms of the hardware cost to support that kind of job and you give access to any schmuck? Do you not have source control or some sort of logging in place?
Rumors said that his motivation would be to just actively sabotage colleague's work because managers decided to give priority on GPU resources to those who were working on DiT models, and he works on autoregressive image generation. I don't know what exactly was his idea, maybe he thought that by destroying internal competitors' work he can get his GPU quotas back?
> Also, what kind of outfit is ByteDance if an intern can modify (and attack) runs that are on the scale of 256 GPUs or more?
Very high. These research labs are basically run on interns (not by interns, but a lot of ideas come from interns, a lot of experiments executed by interns), and I actually mean it.
> Do you not have source control or some sort of logging in place?
Again, rumors said that he gained access to prod jobs by inserting RCE exploits (on unsafe pickle, yay, in 2024!) to foundation model checkpoints.
Thanks, that is at least plausible (but utterly stupid if true) and tells me why I would not be a good cop. Holding off further judgement on the individuals involved until we have more details.
I do understand that interns (who are MSc and PhD students) are incredibly valuable as they drive progress in my own world too: academia. But my point was not so much about access to the resources, as the fact that apparently they were able to manipulate data, code, and jobs from a different group. Looking forward to future details. Maybe we have a mastermind cracker on our hand? But, my bet is rather on awful security and infrastructure practices on the part of ByteDance for a cluster that allegedly is in the range of ~USD 250,000,000.
> my bet is rather on awful security and infrastructure practices
For sure. As far as I know ByteDance does not have an established culture of always building secure systems.
You don't need to be a mastermind cracker. I've used/built several systems for research computing and the defaults are always... less than ideal. Without a beefier budget and a lot of luck (cause you need the right people) it's hard to have a secure system while maintaining a friendly, open atmosphere. Which, as you know, is critical to a research lab.
Also,
> from a different group
Sounds like it was more like a different sub-team of the same group.
From what I heard I'd also argue that this could be told as a weak supply chain attack story. Like, if someone you know from your school re-trained a CLIP with private data, would you really think twice and say "safetensors or I'm not going to use it"?
A lot of ML outfits are equipped with ML experts and people who care about chasing results fast. Security in too many senses of the word is usually an afterthought.
Also sort of as you also hinted, you can't exactly lump these top-conference scoring PhD student-equivalents with typical "interns". Many are extremely capable. ByteDance wants to leverage their capabilities, and likely wants to leverage them fast.
Basic user separation is not asking much though, or are we expected to believe that at ByteDance everyone has a wheel bit at a cluster worth many many millions? Let us see what turns up. Maybe they had a directory with Python pickles that were writeable by everyone? But even that is silly on a very basic level. As I said in another comment, I could be wrong and we have a mastermind cracker of an intern. But I somewhat doubt it.
I think we are converging at an opinion. Internal actors can be hard to detect, and honestly there is a reason at places like Google interns are treated with heightened security checks (my guess -- they learned to do so after some years).
Btw one of the rumors has that it is even difficult to hire engineers to do training/optimization infra at one of those ML shops -- all they want to hire are pure researcher types. We can imagine how hard it will be to ask for resources to tighten up security (without one of these incidents).
That level of security is true for most big tech companies :) You mistake thinking that large and well funded = secure. They clearly have an audit trail but no preventative controls, which is sadly the standard for move fast environments in big tech.
this is closer to occam's since i think the most likely scenario here is malicious reputation damage - it's more likely someone has it out for this intern rather than this intern actually having done literally anything he's accused of
Call me paranoid..."paranoid." but this could be a good way for ByteDance to redirect blame to others when they do something in the future that people don't like. "It was a rouge employee and we fired them"
Of course they will deny it, they have investors... Read the posts from the engineers - 30 people's research and large model training coming to a grinding halt for a quarter. That's easily worth billions in today's market, can you imagine if OpenAI or Google didn't report any progress on a major model for a quarter?
"maliciously interfering" does a lot of the lifting here. And if true, I hope that they didn't stop at firing him. Play stupid games, win stupid prizes. I hate the kind of entitlement that makes people feel justified to destroy huge amounts of value.
I find it weird that China has a very tight information control and simultaneously over and over again has the weirdest "netizen" rumors that go mainstream.
What's the explanation? That they are explicitly allowed for some strategical reason? Something else?
Edit: @dang: Sorry in advance. I do feel like we got some pretty good discussion around this explosive topic, at least in its first hour.
Folks, keep up the good behavior — it makes me look good.
As someone who have lived most of his life in China, I can give you some perspective.
1. There is no such thing as a single entity of government, CCP is not a person, each individual member of the party and government has his/her own agenda. Each level of government has its own goals. But ultimately it's about gaining control and privileges.
2. It is impossible to control 1.3-1.4 billion people all the time, so you make compromises.
3. The main point is: the tight control is both for and rooted from hierarchical power. To put it plainly, anything goes if it doesn't undercut CCP's control. OSHA? WTF is that lol. Law? "If you talk to me about law, I laugh in your face" says the head of a municipal "Rule of Law Office". "Don't talk to me about law this and law that", says the court. But the moment you order a picture of Winnie the pooh carrying wheat (Xi once said he carries 100kg of wheat on his single shoulder) on Alibaba, your account gets banned.
Off topic thoughts: Because CCP has total control, there is no split of power to speak of, so once they are right, they are so right; but when they are wrong, it is catastrophically wrong and there is no change of course. It's why you see 30-50 million people starve to death and an economy miracle within the same half century.
My explanation is that their tight control is an illusion. Not to get political, but the illusion of power is power, and suggesting they control billions of peoples speech is certainly an illusion of power.
China, and all other (supposedly) top-down-economies, survive only because their control is not airtight. If they were to actually have complete control, things would fall apart rapidly. “No one knows how Paris is fed” and all that.
From my work visits and sort of guarded discussions with people there: I feel like they have just accepted the inevitable. Don't ask weird questions about things you're not supposed to ask about, be pragmatic, get things gone, get rich.
I mean, one could argue that the early Soviet Union suffered from this issue. Or early revolutionary China. Cambodia is certainly an example. The french revolution might be an even better example, what with wanting to re-do the clock and calendar and such. To convert startup culture speak's "pragmatism beats idealism" into political science speak, it might come out as "rationalism has tremendous difficultly reinventing all unconscious behavior".
There are individuals and subcultures that prioritize idealism, yes. Often they are young people. Idealistic individuals can get ground down and turned into pragmatists, but some hold onto their hopes and dreams very tightly.
One could argue that the only system under which a citizen can own the means of production is capitalism. If you "own" something you can sell it, trade it, and otherwise use it as you wish. In any realistic version of communism these powers are transferred to a central authority instead.
"the kind of control you're attempting simply is... it's not possible. If there is one thing the history of evolution has taught us it's that life will not be contained."
Humans are clever and typically find workarounds given enough time/hope. Sure you could argue that this is some kind of authoritarian 4D chess/matrix scenario to let off steam for an unruly populace, or it's just the natural course of things.
Culturally, the Chinese population has more of a rebellious streak than people realize. It's a weird contrast - the Great Firewall is there but citizens and often the workers that maintain the firewall seem to circumvent it on a regular basis. Often in order just to function day to day and survive, as noted above.
Also an analogy re how the image is of communist central planning, but post Deng, it's maybe even more of a freewheeling capitalist economy in some regions than the US....(especially in Shenzhen - see Bunnie Huang's write-ups of the ecosystem/economies there)
There will be times when the struggle seems impossible. I know this already. Alone, unsure, dwarfed by the scale of the enemy.
Remember this: freedom is a pure idea. It occurs spontaneously and without instruction. Random acts of insurrection are occurring constantly throughout the galaxy. There are whole armies, battalions that have no idea that they’ve already enlisted in the cause.
Remember that the frontier of the Rebellion is everywhere. And even the smallest act of insurrection pushes our lines forward.
And remember this: the Imperial need for control is so desperate because it is so unnatural. Tyranny requires constant effort. It breaks, it leaks. Authority is brittle. Oppression is the mask of fear.
Remember that. And know this, the day will come when all these skirmishes and battles, these moments of defiance will have flooded the banks of the Empires’s authority and then there will be one too many. One single thing will break the siege.
I turn the question back at you: why do you think it would be in the interest of the Chinese state to surpress this particular rummour?
I don’t see any implication of this news which would undermine their society, or cause disruption, or make people riot. If anything it is a tepid warm “do your job correctly and don’t be too clever by half or else…” story.
China isn’t really that centralized and Zhongnanhai has less control than the White House does. Local party bosses are basically little kings and the average Chinese citizen sees less of the government than the average American does, ie one of the factors of the Chinese illegal immigration surge last year was that China basically has zero social support for pensioners or people who lost their businesses in lockdown
The thing that stuck out to me the most in the west were the long string of articles about the social credit system & the fear around the surveillance state. The surveillance state is probably about the same level as the UK, and the social credit system doesn't run anyone's lives like its described.
I've heard somewhere that the social credit system is really misrepresented in the West - it's designed to track financial scammers and people who set up fraudulent companies. It's meant to weed out untrustworthy business partners, just like how the Western credit system is designed to weed out untrustworthy bankers. (Weird how the only 'group' in the West who gets implicit protection against scams are the banks)
It doesn't really concern the everyman on the street.
The few high profile cases where it was used to punish individuals who ran afoul of some politically powerful person or caused some huge outrage are red herrings - if the system didn't exist, theyd've found some other way to punish them.
The articles functionally stated that you couldn't get an apartment, or pay for a hotel room if you were caught jaywalking or walking around with a scowl on your face.
Both can be true in a country with over 1 billion citizens, through shear volume of individuals talented/determined enough to bypass information control.
If people get to read shocking rumors, they don't feel that their information access is so censored, after all? I could see that at least partially working.
"It's just some dangerous information that is censored."
I’ve spoken extensively about this with people from China.
If something is totally forbidden, that holds.
However, the government doesn’t want people to feel oppressed beyond the explicitly forbidden.
What happens instead is, if it’s unfavorable but not forbidden, it will be mysteriously downvoted and removed, but if it keeps bubbling up, the government says “okay clearly this is important to people” and leaves it up.
This happened with some news cases of forced marriage in some rural mountain regions, and the revelation that a popular WeChat person (like YouTuber) was involved with one of the families.
Tight information control means that rumors are often the best source of information so people are more engaged in the rumor mill. Same thing happened in the Soviet Union.
I'd say China doesn't have particularly tight_er_ information control than other places, they're using the same tools everyone else is using (keyword/hashtag bans, algorithmic content demotion, "shadowbans" of responses, and outright content removal etc.)...
It's mainly just that there's more politically motivated manipulation... versus in the west where those tools would be used on things like copyright infringement, pornography, and misinformation etc.
China does have a tight information control but it may not be what you think it is.
All communication software (QQ/Wechat are the two most used) have sort of backend scanner that detects topics that are in the "in-season" blacklist and ban accounts accordingly. No one knows what the list is so people could get banned for random reasons, but in general bashing current policies or calling out names of the standing members of Politburo is the quickest way to get banned -- and in many instances also got the Wechat group banned.
On the other side, surprisingly, there are many contents that are apparently inappropriate floating on the social media without getting banned. This also throws people off feet.
What I gathered is:
- Don't shit on current party leaders. Actually don't discuss current politics at all. AIs don't always recognize contents correctly so you could be banned for supporting one side or desisting it at the same time.
- Don't ever try to call up other people to join whatever an unofficial cause, whatever it is. Like, even if it's purely patriotic, just don't do it. You do it and you might go to prison very quickly -- at least someone is going to call you to STFU. Grassroot movements is the No.1 enemy of the government and they don't like it. You have to go through official channels for those.
This leads to the following conclusion:
Essentially, the government wants as much control as possible. You want to be patriotic? Sure, but it has to be controlled patriotic. You can attend party gathering to show your patriotism, but creating your own, unofficial gathering is a big No. They probably won't put you into a prison if the cause is legit, but police are going to bug you from time to time ->
IMO this is how the CCP succeed. It has successfully switched from an ideologic party to an "All-people" party. It doesn't really care about ideology. But it wants to assimilate everyone who potentially can be out of control. If you are a successful businessman, it will invite you to participate in political life. If you are an activist who can call up thousands of people, it wants you in. It is essentially, a cauldron of elitists. It has nothing to do with "Communism". It is essentially, GOP + DEM in the US.
Thanks. I felt like things must have progressed from my last sort of insider view from 12 years ago when my company's China subsidiary received weekly meetings from officials to discuss things that needed to be addressed.
"Item number 12. We feel like this URL is hurtful to the Chinese people"
You are welcome. I probably don't know the full picture though, but I think the biggest difference between now() and now() - 12 YEAR is that digital surveillance is way more advanced. Other than that, I don't think the logic changes. CCP has been learning from USSR's experience and successfully converted itself away from an ideological party many years ago. It started around the early 90s and took about a couple of decades for it to happen.
The house just passed $1.6B spending bill for the production of anti-china propaganda. This isn't necessarily a result of that, but I'd imagine some of the weird rumors you hear are manufactured by US intelligence/state dept.
(6) to expose misinformation and disinformation of the Chinese Communist Party’s or the Government of the People’s Republic of China’s propaganda, including through programs carried out by the Global Engagement Center; and
(7) to counter efforts by the Chinese Communist Party or the Government of the People’s Republic of China to legitimize or promote authoritarian ideology and governance models.
——-
Feels like the defense sector is determined to make China a perpetual enemy.
It’s a real drag. We need to step up competence, not fight a war. Viewing China as an enemy vs a strategic competitor leads to bad policy. Like it is killing ASML right now…
But it's relatively easy for China/CPC to squash them if they really want to. Western media is even reporting on changes in particular keyword censorship.
Rumor says an intern at ByteDance was jailed for sabotaging their GPU cluster. Over 8000 H100 GPUs ran corrupted code for a month , all because he was frustrated with resources being diverted from his research to a GenAI project.
was told the intern used a bug in hugginface's load ckpt function to inject bad code. The code randomly change other tasks' parameter and get them sleep, only targeting training tasks using more than 256 cards
You could track down the direct Chinese rumor, but you'd have to leave the cyber basement. Big nono for HN, it can't even eat Americanized Chinese digital food like TikTok ( Chinese version - https://portal.sina.com.hk/others/sina/2024/10/20/1013680/%E... )
It's legit. Read the malicious changes he made to the code and read the posts from the researchers.
And sorry, people are not "gullible" for disbelieving the media. I have worked at most big tech companies and the media misreports so badly on easily verifiable things in my area of expertise, that I no longer trust them on much. https://en.m.wikipedia.org/wiki/Michael_Crichton#Gell-Mann_a...
Related: https://arstechnica.com/tech-policy/2024/10/bytedance-intern...
(via https://news.ycombinator.com/item?id=41906970, but we merged that thread hither)
What we often think of as Insider Threat in the west is just another Tuesday in Chinese business. I have many experiences of this in the video game industry. This industry sabotage and theft is a very real part of getting ahead, even amongst companies that are owned by the same parent company (ex: studios owned in part by Tencent).
OTOH: ByteDance intern responsible for spamming your web server with crawlers that ignore robots.txt given permanent position with a raise, now in management.
I'm reminded of a time that an intern took down us-east1 on AWS, by modifying a configuration file they shouldn't have had access to. Amazon (somehow) did the correct thing and didn't fire them -- instead, they used the experience to fix the security hole. It was a file they shouldn't have had access to in the first place.
If the intern "had no experience with the AI lab", is it the right thing to do to fire them, instead of admitting that there is a security/access fault internally? Can other employees (intentionally, or unintentionally) cause that same amount of "damage"?
From what I've seen in Amazon it's pretty consistent that they do not blame the messenger which is what they consider the person who messed up. Usually that person is the last in a long series of decisions that could have prevented the issue, and thus why blame them. That is unless the person is a) acting with malice, b) is repeatedly shown a pattern of willful ignorance. IIRC, when one person took down S3 with a manual command overriding the safeguards the action was not to fire them but to figure out why it was still a manual process without sign off. Say what you will about Amazon culture, the ability to make mistakes or call them out is pretty consistently protected.
> From what I've seen in Amazon it's pretty consistent that they do not blame the messenger which is what they consider the person who messed up
Interesting that my experience has been the exact opposite.
Whenever I’ve participated in COE discussions (incident analysis), questions have been focused on highlighting who made the mistake or who didn’t take the right precautions.
I've bar raised a ton of them. You do end up figuring out what actions by what operator caused what issues or didn't work well, but that's to diagnose what controls/processes/tools/metrics were missing. I always removed the actual people's name as part of the bar raising, well before publishing, usually before any manager sees it. Instead used Oncall 1, or Oncall for X team, Manager for X team. And that's mainly for the timeline.
As a sibling said you were likely in a bad or or one that was using COEs punatively.
In the article's case, there's evidence of actual malice, though-- sabotaging only large jobs, over a month's time.
All I got from the linked article was
> TikTok owner, ByteDance, says it has sacked an intern for "maliciously interfering" with the training of one of its artificial intelligence (AI) models.
Are there other links with additional info?
A lot of the original social media sources have been pulled, but this is what was alleged on social media:
https://juejin.cn/post/7426926600422637594
https://github.com/JusticeFighterDance/JusticeFighter110
https://x.com/0xKyon/status/1847529300163252474
Thanks. Google translate off the first link:
> He exploited the vulnerability of huggingface's load ckpt function to inject code, dynamically modifying other people's optimizer to randomly sleep for a short period of time, and modifying the direction of parameter shaving. He also added a condition that only tasks with more than 256 cards would trigger this condition.
Okay yeah that's malicious and totally a crime. "modifying the direction of parameter shaving" means he subtly corrupted his co-workers work. that's wild!
Some of the sources say that he sat in the incident meetings during troubleshooting and adjusted his attacks to avoid detection, too.
LMAO that's just diabolical. Wonder what motivated them.
Wonder what the underlying motive was? Seems like a super weird thing to do.
Whats bar raising in this context?
https://www.aboutamazon.co.uk/news/working-at-amazon/what-is...
I’ve run the equivalent process at my company and I absolutely want us to figure out who took the triggering actions, what data/signals they were looking at, what exactly they did, etc.
If you don’t know what happened and can’t ask more details about it, how can you possibly reduce the likelihood (or impact) of it in the future?
Finding out in detail who did it does not require you to punish that person and having a track record of not punishing them helps you find out the details in future incidents.
Isn't that a necessary step in figuring out the issue and how t prevent it?
But when that person was identified, were they personally held responsible, bollocked, and reprimanded or were they involved in preventing the issue from happening again?
"No blame, but no mercy" is one of these adages; while you shouldn't blame individuals for something that is an organization-wide problem, you also shouldn't hold back in preventing it from happening again.
Usually helping prevent the issue, training. Almost everyone I've ever seen cause an outage is so "oh shit oh shit oh shit" that a reprimand is worthless, I've spent more time a) talking them through what they could have done better and, encouraging them to escalate quicker b) assusaging their fears that it was all their fault and they'll be blamed / fired. "I just want you to know we don't consider this your fault. It was not your fault. Many many people made poor risk tradeoffs for us to get to the point where you making X trivial change caused the internet to go down"
In some cases like interns we probably just took their commit access away or blocked their direct push access. Now a days interns can't touch critical systems and can't push code directly to prod packages.
That was not the idea of COE ever. Probably you were in bad org/team.
At least in my experience, this is also how Azure continues to function. Certainly reduces stress in the working environment
There is a huge difference between someone making a mistake and someone intentionally sabotaging.
You're not firing the person because they broke stuff, you are firing them because they tried to break stuff. If the attempt was a failure and caused no harm, you would still fire them. Its not about the damage they caused its that they wanted to cause damage.
What if the intern did accidental mistake, but the company painted it as intentional sabotage? Nothing new in communism.
Large powerful groups lying to save face is not a feature of communism, sadly. Stories about the CIA, FBI, and PG&E caught trying to do so come to mind, among others.
They were just fired, not put in prison or sued. Getting fired is a typical capitalist punishment, I'd bet way more engineers gets fired for mistakes in USA than China.
But for damaging company assets on purpose firing is only first step.
I do not see any mention of other legal action and article is shallow.
It might’ve been that someone in command chain called it “malicious” to cover up his own mistakes. I think that is parent poster point while writing out Amazon story.
Maybe, but without any other info, i kind of have to take the info provided at face value. Like obviously if the article is inaccurate the whole situation should be viewed differently.
The article says:
It was one of the STEP interns that took down Google prod by modifying some config file by putting something erroneous into an automated tool. Everyone at the company was locked out, and someone had to physically access some machines in a datacenter to recover.
The difference in this case is intent.
Did the employee have the intent to cause damage? If so just fire him/her.
Malicious intent to be precise. Well-intentioned attempts to demonstrate issues for the purposes of helping to fix should generally not be punished, unless there is a wider fallout than expected and that can be attributed to negligence.
I'd like to learn more about the AWS incident, but when I google "us-east1 intern" I get this comment. Do you have a link?
Probably this: https://aws.amazon.com/message/41926/
afaik this was intentional in that they stopped training runs and changing parameters for other employee training runs, and even joined in on the debugging group trying to solve the "issues".
I feel less informed after reading the article than I did after reading the headline.
yes, the article has nothing about what they were doing or how they "interfered" exactly.
This article merely relays what ByteDance says, so it’s nothing but PR, unrelated to journalism. No idea what it’s doing on bbc.com.
Not really. It says:
> ByteDance also denied reports that the incident caused more than $10m of damage
It makes clear what ByteDance's official position is, while pretty clearly hinting that it might not be true.
It was a phd student that was mad about compensation or something purposely injecting malicious code.
I hope said intern finds a new job working for anti-AI causes.
People who sabotage things tend to do it against all sides (you can always find an excuse to sabotage if you try hard enough).
> People who sabotage things tend to do it against all sides (you can always find an excuse to sabotage if you try hard enough).
'Holy Generalization, Batman!'
Are there are a lot of anti-AI organizations at this point? PauseAI is the main one I'm familiar with:
https://pauseai.info/
One thing I suspect investors in e.g. OpenAI are failing to price in is the political and regulatory headwinds OpenAI will face if their fantastical revenue projections actually materialize. A world where OpenAI is making $100B in annual revenue will likely be a world where technological unemployment looms quite clearly. Polls already show strong support for regulating AI.
Regulation supports the big players. See SB 1047 in California and read the first few lines: > comply with various requirements, including implementing the capability to promptly enact a full shutdown, as defined, and implement a written and separate safety and security protocol, as specified
That absolutely kills open source, and it's disguised as a "safety" bill where safety means absolutely nothing (how are you "shutting down" an LLM?). There's a reason Anthropic was championing it even though it evidently regulates AI.
>That absolutely kills open source
Zvi says this claim is false: https://thezvi.substack.com/p/guide-to-sb-1047?open=false#%C...
>how are you "shutting down" an LLM?
Pull the plug on the server? Seems like it's just about having a protocol in place to make that easy in case of an emergency. Doesn't seem that onerous.
Which server? The one you have no idea about because you released your weights and anyone can download/use them at that point?
The Amish?
I'm trying to think of whether it'd be worth starting some kind of semi-Luddite community where we can use digital technology, photos, radios, spreadsheets and all, but the line is around 2014, when computers still did the same thing every time. That's my biggest gripe with AI, the nondeterminism, the non-repeatability making it all undebuggable, impossible to interrogate and reason about. A computer in 2014 is complex but not incomprehensible. The mass matrix multiplication of 2024 computation is totally opaque and frankly I think there's room for a society without such black box oracles.
Fwiw, the Amish aren’t luddites, they’re not anti-technology in all facets of life. You’ll see Amish folks using power tools, cellphones, computers, etc in their professional lives or outside the context of their homes (exact standards vary by community). There are even multiple companies that manufacture computers specifically for the Amish. So there’s no reason an Amish business couldn’t use AI.
Don't they have a process for determining whether new technology should be integrated into their lives?
Yes, the exact process varies by community but it generally involves church elders meeting to discuss whether a new technology is likely to benefit or harm family, community and spiritual life.
Why 2014? Why not 2022 when ChatGPT was released? Or 2019 for ChatGPT 2? Why not 2005 when the first dual-core Pentium was released? After that, the two cores meant that you could be sure what order your program would run things. Or why not 2012 when Intel added the RdRand instruction to x86? Or 2021 when Linux 5.17 was released with random number generation improvements? Or 1985 when IEEE 754 floating point was released. Before that, it was all integer math but after that, 0.1 + 0.2 = 0.30000000000000004. Not that I have any objection to 2014, I'm just wondering why you chose then.
If I was really picky I would stop the clock in the 8bit era or at least well before speculative execution / branch prediction, but I do want to leave some room for pragmatism.
2014 is when I became aware of gradient descent and how entropy was used to search more effectively, leading to different runs of the same program arriving at different results, Deep Dream came soon after and it's been downhill from there
If I were to write some regulations for what was allowed in my computing community I would make an exception for using PRNGs for scientific simulation and cryptographic purposes, but definitely I would draw a line at using heuristics to find optimal solutions. Slide rules got us to the moon and that's good enough for me.
SAG-AFTRA are currently on strike over the issue of unauthorized voice cloning.
The AI advocates actively advertised AI as a tool for replacing creatives, including plagiarizing their work, and copying the appearance and voices of individuals. It's not really surprising that everyone in the creative industries is going to use what little power they have to avoid this doomsday scenario.
Regulation is not neccesarily bad for the market leader.
I hope he spends a good long bit in prison
> the intern allegedly "maliciously interfered with the model training tasks" for a ByteDance research project
Did the intern post a manifesto or something? What was the point of doing this?
I assume he is not the only one responsible because its hard to believe interns code wasn’t reviewed before releasing.
The context is here: https://github.com/JusticeFighterDance/JusticeFighter110
whats this mean for us non chinese folk
Translated by ChatGPT.
Summary:
10/18:
Translation of the provided text:
Title: Urgent Warning
The “reputation washing” behavior of Tian Keyu has been extremely harmful
For the past two months, Tian Keyu has maliciously attacked the cluster code, causing significant harm to nearly 30 employees of various levels, wasting nearly a quarter’s worth of work by his colleagues. All records and audits clearly confirm these undeniable facts:
1. Modified the PyTorch source code of the cluster, including random seeds, optimizers, and data loaders.
2. Randomly killed multi-machine experiment processes, causing significant experiment delays.
3. Opened login backdoors through checkpoints, automatically initiating random process terminations.
4. Participated in daily troubleshooting meetings for cluster faults, continuing to modify attack codes based on colleagues’ troubleshooting ideas.
5. Altered colleagues’ model weights, rendering experimental results unreproducible.
It’s unimaginable how Tian Keyu could continue his attacks with such malice, seeing colleagues’ experiments inexplicably interrupted or fail, after hearing their debugging strategies and specifically modifying the attack codes in response, and witnessing colleagues working overnight with no progress. After being dismissed by the company, he received no penalties from the school or advisors and even began to whitewash his actions on various social media platforms. Is this the school and advisors’ tolerance of Tian Keyu’s behavior? We expect this evidence disclosure to attract the attention of relevant parties and for definitive penalties to be imposed on Tian Keyu, reflecting the social responsibility of higher education institutions to educate and nurture.
We cannot allow someone who has committed such serious offenses to continue evading justice, even beginning to distort facts and whitewash his wrongdoing! Therefore, we decide to stand on behalf of all justice advocates and reveal the evidence of Tian Keyu’s malicious cluster attack!
Tian Keyu, if you deny any part of these malicious attack behaviors, or think the content here smears you, please present credible evidence! We are willing to disclose more evidence as the situation develops, along with your shameless ongoing attempts to whitewash. We guarantee the authenticity and accuracy of all evidence and are legally responsible for the content of the evidence. If necessary, we are willing to disclose our identities and confront Tian Keyu face-to-face.
Thanks to those justice advocates, you do not need to apologize; you are heroes who dare to speak out.
Link to the inquiry recording of Tian Keyu: https://www.youtube.com/watch?v=nEYbYW--qN8
Personal homepage of Tian Keyu: https://scholar.google.com/citations?user=6FdkbygAAAAJ&hl=en
GitHub homepage of Tian Keyu: https://github.com/keyu-tian
10/19:
Clarification Regarding the “Intern Sabotaging Large Model Training” Incident
Recently, some media reported that “ByteDance’s large model training was attacked by an intern.” After internal verification by the company, it was confirmed that an intern from the commercial technology team committed a serious disciplinary violation and has been dismissed. However, the related reports also contain some exaggerations and inaccuracies, which are clarified as follows:
1. The intern involved maliciously interfered with the model training tasks of the commercial technology team’s research project, but this did not affect the official commercial projects or online operations, nor did it involve ByteDance’s large model or other businesses.
2. Rumors on the internet about “involving over 8,000 cards and losses of millions of dollars” are greatly exaggerated.
3. Upon verification, it was confirmed that the individual in question had been interning in the commercial technology team, and had no experience interning at AI Lab. Their social media bio and some media reports are incorrect.
The intern was dismissed by the company in August. The company has also reported their behavior to the industry alliance and the school they attend, leaving further actions to be handled by the school.
Hanlon's razor comes to mind
https://en.m.wikipedia.org/wiki/Hanlon%27s_razor
No. This isn't a Hanlon's Razor scenario.
If you look at what he did it was definitely 100% actively malicious. For instance, his attack only executes when running on >256 GPUs. He inserted random sleeps to slow down training time and was knowledgeable enough to understand how to break various aspects of the loss function.
He then sat in meetings and adjusted his attacks when people were getting close to solving the problem.
Certainly looks malicious, but what on earth would be his motive? He is an MSc student for heaven's sake and this tarnishes his entire career. Heck, he has published multiple first-author, top-tier papers (two at NeurIPS and one at ICLR) and is on par with a mid-stage PhD student that would be considered to be doing outstandingly well. The guy would (is?) likely to be on track for a great job and career. Not saying he did not do what was claimed, but I am unsure about any motive that fits other than "watching the world burn".
Also, what kind of outfit is ByteDance if an intern can modify (and attack) runs that are on the scale of 256 GPUs or more? We are talking at least ~USD 8,000,000 in terms of the hardware cost to support that kind of job and you give access to any schmuck? Do you not have source control or some sort of logging in place?
> but what on earth would be his motive
Rumors said that his motivation would be to just actively sabotage colleague's work because managers decided to give priority on GPU resources to those who were working on DiT models, and he works on autoregressive image generation. I don't know what exactly was his idea, maybe he thought that by destroying internal competitors' work he can get his GPU quotas back?
> Also, what kind of outfit is ByteDance if an intern can modify (and attack) runs that are on the scale of 256 GPUs or more?
Very high. These research labs are basically run on interns (not by interns, but a lot of ideas come from interns, a lot of experiments executed by interns), and I actually mean it.
> Do you not have source control or some sort of logging in place?
Again, rumors said that he gained access to prod jobs by inserting RCE exploits (on unsafe pickle, yay, in 2024!) to foundation model checkpoints.
Thanks, that is at least plausible (but utterly stupid if true) and tells me why I would not be a good cop. Holding off further judgement on the individuals involved until we have more details.
I do understand that interns (who are MSc and PhD students) are incredibly valuable as they drive progress in my own world too: academia. But my point was not so much about access to the resources, as the fact that apparently they were able to manipulate data, code, and jobs from a different group. Looking forward to future details. Maybe we have a mastermind cracker on our hand? But, my bet is rather on awful security and infrastructure practices on the part of ByteDance for a cluster that allegedly is in the range of ~USD 250,000,000.
Agree on this being stupid.
> my bet is rather on awful security and infrastructure practices
For sure. As far as I know ByteDance does not have an established culture of always building secure systems.
You don't need to be a mastermind cracker. I've used/built several systems for research computing and the defaults are always... less than ideal. Without a beefier budget and a lot of luck (cause you need the right people) it's hard to have a secure system while maintaining a friendly, open atmosphere. Which, as you know, is critical to a research lab.
Also,
> from a different group
Sounds like it was more like a different sub-team of the same group.
From what I heard I'd also argue that this could be told as a weak supply chain attack story. Like, if someone you know from your school re-trained a CLIP with private data, would you really think twice and say "safetensors or I'm not going to use it"?
A lot of ML outfits are equipped with ML experts and people who care about chasing results fast. Security in too many senses of the word is usually an afterthought.
Also sort of as you also hinted, you can't exactly lump these top-conference scoring PhD student-equivalents with typical "interns". Many are extremely capable. ByteDance wants to leverage their capabilities, and likely wants to leverage them fast.
Basic user separation is not asking much though, or are we expected to believe that at ByteDance everyone has a wheel bit at a cluster worth many many millions? Let us see what turns up. Maybe they had a directory with Python pickles that were writeable by everyone? But even that is silly on a very basic level. As I said in another comment, I could be wrong and we have a mastermind cracker of an intern. But I somewhat doubt it.
I think we are converging at an opinion. Internal actors can be hard to detect, and honestly there is a reason at places like Google interns are treated with heightened security checks (my guess -- they learned to do so after some years).
Btw one of the rumors has that it is even difficult to hire engineers to do training/optimization infra at one of those ML shops -- all they want to hire are pure researcher types. We can imagine how hard it will be to ask for resources to tighten up security (without one of these incidents).
That level of security is true for most big tech companies :) You mistake thinking that large and well funded = secure. They clearly have an audit trail but no preventative controls, which is sadly the standard for move fast environments in big tech.
this is closer to occam's since i think the most likely scenario here is malicious reputation damage - it's more likely someone has it out for this intern rather than this intern actually having done literally anything he's accused of
Call me paranoid..."paranoid." but this could be a good way for ByteDance to redirect blame to others when they do something in the future that people don't like. "It was a rouge employee and we fired them"
Perhaps the rouge employee was red-teaming.
I had the same thought! Perhaps we're on the same wavelength...
Seems like the kind of thing that will work once only...
no need to be colorist
It had on the right letters, tho.
What a non-story.
But AI!
AND China!
This guy maliciously interfered with ByteDance's LLM project. That is huge news. Millions or billions of dollars down the drain.
> That is huge news - billions down the drain.
Billions of what? milli-dollars? Bytedance denies the damages are over $10M.
Of course they will deny it, they have investors... Read the posts from the engineers - 30 people's research and large model training coming to a grinding halt for a quarter. That's easily worth billions in today's market, can you imagine if OpenAI or Google didn't report any progress on a major model for a quarter?
This is more or less issue of team lead. Just assigned wrong task to intern.
> Its commercial online operations, including its large language AI models, were unaffected by the intern's actions, the company added.
so did something actually happen or did they just post some inaccuracies on social media
Sounds like Tian Keyu stumbled on something he wasn’t supposed to see — now they’re trying to bury him alive.
"maliciously interfering" does a lot of the lifting here. And if true, I hope that they didn't stop at firing him. Play stupid games, win stupid prizes. I hate the kind of entitlement that makes people feel justified to destroy huge amounts of value.
I find it weird that China has a very tight information control and simultaneously over and over again has the weirdest "netizen" rumors that go mainstream.
What's the explanation? That they are explicitly allowed for some strategical reason? Something else?
Edit: @dang: Sorry in advance. I do feel like we got some pretty good discussion around this explosive topic, at least in its first hour.
Folks, keep up the good behavior — it makes me look good.
As someone who have lived most of his life in China, I can give you some perspective.
1. There is no such thing as a single entity of government, CCP is not a person, each individual member of the party and government has his/her own agenda. Each level of government has its own goals. But ultimately it's about gaining control and privileges.
2. It is impossible to control 1.3-1.4 billion people all the time, so you make compromises.
3. The main point is: the tight control is both for and rooted from hierarchical power. To put it plainly, anything goes if it doesn't undercut CCP's control. OSHA? WTF is that lol. Law? "If you talk to me about law, I laugh in your face" says the head of a municipal "Rule of Law Office". "Don't talk to me about law this and law that", says the court. But the moment you order a picture of Winnie the pooh carrying wheat (Xi once said he carries 100kg of wheat on his single shoulder) on Alibaba, your account gets banned.
Off topic thoughts: Because CCP has total control, there is no split of power to speak of, so once they are right, they are so right; but when they are wrong, it is catastrophically wrong and there is no change of course. It's why you see 30-50 million people starve to death and an economy miracle within the same half century.
I wish I could upvote your comment more than by +1. Thanks.
My explanation is that their tight control is an illusion. Not to get political, but the illusion of power is power, and suggesting they control billions of peoples speech is certainly an illusion of power.
China, and all other (supposedly) top-down-economies, survive only because their control is not airtight. If they were to actually have complete control, things would fall apart rapidly. “No one knows how Paris is fed” and all that.
From my work visits and sort of guarded discussions with people there: I feel like they have just accepted the inevitable. Don't ask weird questions about things you're not supposed to ask about, be pragmatic, get things gone, get rich.
My experience as well! Pragmatism over idealism is a fantastic virtue for everyone — but turns out a vital one for communists :P
Is pragmatism really a "fantastic virtue" when people are forced away from alternatives by an overbearing government?
Is there any culture on earth that prioritizes idealism over "pragmatism", if we must use that term? What does this even look like?
I mean, one could argue that the early Soviet Union suffered from this issue. Or early revolutionary China. Cambodia is certainly an example. The french revolution might be an even better example, what with wanting to re-do the clock and calendar and such. To convert startup culture speak's "pragmatism beats idealism" into political science speak, it might come out as "rationalism has tremendous difficultly reinventing all unconscious behavior".
There are individuals and subcultures that prioritize idealism, yes. Often they are young people. Idealistic individuals can get ground down and turned into pragmatists, but some hold onto their hopes and dreams very tightly.
I wasn't aware that Chinese citizens owned the means of production ;) just looks like another authoritarian dictatorship to me.
One could argue that the only system under which a citizen can own the means of production is capitalism. If you "own" something you can sell it, trade it, and otherwise use it as you wish. In any realistic version of communism these powers are transferred to a central authority instead.
Guess they never really tried it.
Wow, even mentioning communists made you get downvoted. That's sad.
And I was being cheeky too! Such is life.
Ian Malcolm said it best:
"the kind of control you're attempting simply is... it's not possible. If there is one thing the history of evolution has taught us it's that life will not be contained."
Humans are clever and typically find workarounds given enough time/hope. Sure you could argue that this is some kind of authoritarian 4D chess/matrix scenario to let off steam for an unruly populace, or it's just the natural course of things.
Culturally, the Chinese population has more of a rebellious streak than people realize. It's a weird contrast - the Great Firewall is there but citizens and often the workers that maintain the firewall seem to circumvent it on a regular basis. Often in order just to function day to day and survive, as noted above.
Also an analogy re how the image is of communist central planning, but post Deng, it's maybe even more of a freewheeling capitalist economy in some regions than the US....(especially in Shenzhen - see Bunnie Huang's write-ups of the ecosystem/economies there)
There’s the Chinese saying, “Heaven is high, and the emperor is far away”.
Fake it till you make it. At some point they will have full control.
Yes. LLM:s will make it easy. Even current solutions are probably good enough for them to do what what want, with an "acceptable" error margin.
There will be times when the struggle seems impossible. I know this already. Alone, unsure, dwarfed by the scale of the enemy.
Remember this: freedom is a pure idea. It occurs spontaneously and without instruction. Random acts of insurrection are occurring constantly throughout the galaxy. There are whole armies, battalions that have no idea that they’ve already enlisted in the cause.
Remember that the frontier of the Rebellion is everywhere. And even the smallest act of insurrection pushes our lines forward.
And remember this: the Imperial need for control is so desperate because it is so unnatural. Tyranny requires constant effort. It breaks, it leaks. Authority is brittle. Oppression is the mask of fear.
Remember that. And know this, the day will come when all these skirmishes and battles, these moments of defiance will have flooded the banks of the Empires’s authority and then there will be one too many. One single thing will break the siege.
Remember this: try.
I turn the question back at you: why do you think it would be in the interest of the Chinese state to surpress this particular rummour?
I don’t see any implication of this news which would undermine their society, or cause disruption, or make people riot. If anything it is a tepid warm “do your job correctly and don’t be too clever by half or else…” story.
Why would they flex their muscles for this one?
China isn’t really that centralized and Zhongnanhai has less control than the White House does. Local party bosses are basically little kings and the average Chinese citizen sees less of the government than the average American does, ie one of the factors of the Chinese illegal immigration surge last year was that China basically has zero social support for pensioners or people who lost their businesses in lockdown
https://www.bloomberg.com/opinion/articles/2023-08-14/china-...
The thing that stuck out to me the most in the west were the long string of articles about the social credit system & the fear around the surveillance state. The surveillance state is probably about the same level as the UK, and the social credit system doesn't run anyone's lives like its described.
I've heard somewhere that the social credit system is really misrepresented in the West - it's designed to track financial scammers and people who set up fraudulent companies. It's meant to weed out untrustworthy business partners, just like how the Western credit system is designed to weed out untrustworthy bankers. (Weird how the only 'group' in the West who gets implicit protection against scams are the banks)
It doesn't really concern the everyman on the street.
The few high profile cases where it was used to punish individuals who ran afoul of some politically powerful person or caused some huge outrage are red herrings - if the system didn't exist, theyd've found some other way to punish them.
The articles functionally stated that you couldn't get an apartment, or pay for a hotel room if you were caught jaywalking or walking around with a scowl on your face.
Both can be true in a country with over 1 billion citizens, through shear volume of individuals talented/determined enough to bypass information control.
One idea is that they're fake planted rumors. Certanily not the first time things like that happen
If people get to read shocking rumors, they don't feel that their information access is so censored, after all? I could see that at least partially working.
"It's just some dangerous information that is censored."
Well that's what the Washington regime and its media lackies do anyway, "according to a confidential source at the Whitehouse..."
I’ve spoken extensively about this with people from China.
If something is totally forbidden, that holds.
However, the government doesn’t want people to feel oppressed beyond the explicitly forbidden.
What happens instead is, if it’s unfavorable but not forbidden, it will be mysteriously downvoted and removed, but if it keeps bubbling up, the government says “okay clearly this is important to people” and leaves it up.
This happened with some news cases of forced marriage in some rural mountain regions, and the revelation that a popular WeChat person (like YouTuber) was involved with one of the families.
Tight information control means that rumors are often the best source of information so people are more engaged in the rumor mill. Same thing happened in the Soviet Union.
I'd say China doesn't have particularly tight_er_ information control than other places, they're using the same tools everyone else is using (keyword/hashtag bans, algorithmic content demotion, "shadowbans" of responses, and outright content removal etc.)...
It's mainly just that there's more politically motivated manipulation... versus in the west where those tools would be used on things like copyright infringement, pornography, and misinformation etc.
China does have a tight information control but it may not be what you think it is.
All communication software (QQ/Wechat are the two most used) have sort of backend scanner that detects topics that are in the "in-season" blacklist and ban accounts accordingly. No one knows what the list is so people could get banned for random reasons, but in general bashing current policies or calling out names of the standing members of Politburo is the quickest way to get banned -- and in many instances also got the Wechat group banned.
On the other side, surprisingly, there are many contents that are apparently inappropriate floating on the social media without getting banned. This also throws people off feet.
What I gathered is:
- Don't shit on current party leaders. Actually don't discuss current politics at all. AIs don't always recognize contents correctly so you could be banned for supporting one side or desisting it at the same time.
- Don't ever try to call up other people to join whatever an unofficial cause, whatever it is. Like, even if it's purely patriotic, just don't do it. You do it and you might go to prison very quickly -- at least someone is going to call you to STFU. Grassroot movements is the No.1 enemy of the government and they don't like it. You have to go through official channels for those.
This leads to the following conclusion:
Essentially, the government wants as much control as possible. You want to be patriotic? Sure, but it has to be controlled patriotic. You can attend party gathering to show your patriotism, but creating your own, unofficial gathering is a big No. They probably won't put you into a prison if the cause is legit, but police are going to bug you from time to time ->
IMO this is how the CCP succeed. It has successfully switched from an ideologic party to an "All-people" party. It doesn't really care about ideology. But it wants to assimilate everyone who potentially can be out of control. If you are a successful businessman, it will invite you to participate in political life. If you are an activist who can call up thousands of people, it wants you in. It is essentially, a cauldron of elitists. It has nothing to do with "Communism". It is essentially, GOP + DEM in the US.
Thanks. I felt like things must have progressed from my last sort of insider view from 12 years ago when my company's China subsidiary received weekly meetings from officials to discuss things that needed to be addressed.
"Item number 12. We feel like this URL is hurtful to the Chinese people"
You are welcome. I probably don't know the full picture though, but I think the biggest difference between now() and now() - 12 YEAR is that digital surveillance is way more advanced. Other than that, I don't think the logic changes. CCP has been learning from USSR's experience and successfully converted itself away from an ideological party many years ago. It started around the early 90s and took about a couple of decades for it to happen.
The house just passed $1.6B spending bill for the production of anti-china propaganda. This isn't necessarily a result of that, but I'd imagine some of the weird rumors you hear are manufactured by US intelligence/state dept.
source?
Looks like it might be this?
https://www.congress.gov/bill/118th-congress/house-bill/1157
What in god’s name are you talking about?
https://www.congress.gov/bill/118th-congress/house-bill/1157...
Oh. Had to look it up.
(6) to expose misinformation and disinformation of the Chinese Communist Party’s or the Government of the People’s Republic of China’s propaganda, including through programs carried out by the Global Engagement Center; and
(7) to counter efforts by the Chinese Communist Party or the Government of the People’s Republic of China to legitimize or promote authoritarian ideology and governance models.
——-
Feels like the defense sector is determined to make China a perpetual enemy.
It’s a real drag. We need to step up competence, not fight a war. Viewing China as an enemy vs a strategic competitor leads to bad policy. Like it is killing ASML right now…
Conspiracy theories are common in repressive regimes.
But it's relatively easy for China/CPC to squash them if they really want to. Western media is even reporting on changes in particular keyword censorship.
Wow BBC is garbage.
https://x.com/le1du/status/1847144170705785239
You could track down the direct Chinese rumor, but you'd have to leave the cyber basement. Big nono for HN, it can't even eat Americanized Chinese digital food like TikTok ( Chinese version - https://portal.sina.com.hk/others/sina/2024/10/20/1013680/%E... )The article quoting specific responses is garbage, here's a tweet explicitly stating it includes a rumour? What are you trying to say here?
He’s basically highlighting why the media is dead. Gullible folks would rather read salacious rumours than actual news.
In a heavily controlled media landscape like China’s, eventually the rumors become the only source of credible news.
Most major Western news media are sourcing at least some China stories from WeChat and Sina Weibo before it gets scrubbed by censors.
It's legit. Read the malicious changes he made to the code and read the posts from the researchers.
And sorry, people are not "gullible" for disbelieving the media. I have worked at most big tech companies and the media misreports so badly on easily verifiable things in my area of expertise, that I no longer trust them on much. https://en.m.wikipedia.org/wiki/Michael_Crichton#Gell-Mann_a...