Why is everyone spooked by ai driven bioterrorism ? Like all of our water supplies are much easier to mess with compared to developing a virus or whatever that may or may not work
Evolution cannot optimize for maximum harm. Natural viruses only evolved to reproduce, with the harm just a side effect. Humans can combine features of multiple natural viruses. There's no obvious reason why we couldn't make something with both the infectivity of measles and the lethality of rabies, as well as resistance to all known antivirals. If something like that gets out it's probably civilization ending.
Especially because how hard it is to deal with the consequences. If a town's water supply is polluted you can shut it down and deliver water by truck until the situation is resolved. The consequences might be serious, but it would be a one-off event with limited consequences. Similarly, shutting down all freezers in a cyberattack would suck, but the consequences are limited in time
A new bioweapon let lose would be incredibly difficult to contain, and could potentially become something we have to contend with forever
Are you actually asking this? Like why should people be worried that a disgruntled highly motivated person would venture into this area? Because it can happen. The outcomes of it could be really bad? It also doesn't have to be done by an individual, could be nation state or organization led. There are groups of people who believe humanity needs to end and all sorts of crazy stuff. Some of them have gone on mass murder suicide missions.
2 bad things can be threats at the same time. Even 10,000 things can be.
That seems a bit silly thinking they are not doing anything to protect the water supplies, and even more silly to not compare the impact of the things you are comparing. Poison a water supply and you hit how many, a city, a small part of a city? Now compare it to the impact of a virus like corona - we literally just went through that..
Instead of poisoning the water supply, you'd stuxnet the pumps. I imagine the network interfaces for status and control are about as well protected as grocery store freezers.
A bad pathogen could go world wide, but you could trash multiple cities with online attacks. No electricity, no water, no communications. Things would get bad.
Disrupting electric supply, water supply, and food supply chains are already tried and tested separately, with some limited monetary system disturbances, here and there. So doing those simultaneously - possibly leveraging on latent vulnerabilities kept covered for later abuse - sounds a natural step 'forward', close to trivial for those into that. Causing huge broad disturbances and possible unrest in societies. So that some governments started to ask people to build up reserves.
I can imagine that terrorist are sad now that they cannot use their preferred advanced bioengineering skills acquired through great efforts posing as scientist towards the so far unsuspecting services providing them with harmful DNA building blocks and now have to turn to much much more pedestrian methods instead of the heightened sophistication they crave.
Its hard to imagine a mustache twirling terrorist bent on putting PFAS and lead in the american water supply, putting mad cow in the beef supply and making everything cost more because he would have been outcompeted by the US government.
Have no idea what happens in the US but in Australia nearly all PFAS in the country comes from government properties, most often military bases or airports.
And yet the Russian foreign policy is almost this: they know they can't compete on competence/science/tech so they try to tear everything down to their level.
I think this is a bad take. The Russian government wants to tear things down because they are a kleptocracy, and functional nations act as a check on corrupt governments. The more corrupt the US is the more theft russia can get away with and collude with pals inside the US gov on.
The framing of nations as inherently in competition with each other doesn’t track for me. Sometimes, in specific instances, nations will be in competition, but finland having a good or bad education system does not impact me even slightly.
> The framing of nations as inherently in competition with each other doesn’t track for me.
Where it tracks for you or not isn't relevant. It tracks for Putin and he's been extremely clear about that for hours and hours of extremely boring monologues :P
They have placed agent Krasnov though, so they influence US politics anyway. What is even more surprising is that Krasnov is now doubling down, despite the clear loss in the latest election.
The regulatory state didn’t fail out of bureaucratic clumsiness; it was systematically bought and gutted so private balance sheets wouldn’t have to carry the cost of your cancer.
Blaming "the government" for toxic tap water and runaway prices is like blaming the getaway car instead of the bank robbers.
Private balance sheets wouldn’t have to carry the cost of your cancer is interesting. Do public balance sheets have to carry the cost of alcoholic's cirrhosis, smoker's lung cancer, and Type 2 diabetic's myriad of issues? Or does personal responsibility only apply to governments and corporations, not people?
>Blaming "the government" for toxic tap water and runaway prices is like blaming the getaway car instead of the bank robbers.
Blaming the government for toxic tap water is like blaming an organised private/public coalition of bank robbers for a bank robbery.
>it was systematically bought and gutted so private balance sheets wouldn’t have to carry the cost of your cancer.
The US Government, day 1, was bought and sold to defend the right to own slaves, even from the predation of states that had made it illegal. It's not some recent malformation that it just started defending the interests of rich people over poor people lmao. The explicit purpose of the US government is to do the shit it does. The machine has been designed well, not poorly.
Sure, and the US was also home to the first abolitionist society, and fought its bloodiest war in history (by % of population killed) in a war to end slavery.
"...this sparkling analysis from Wood (History/Brown Univ.; ed., The Rising Glory of America, 1971) impressively argues that [the American Revolution] was anything but conservative. Wood's contention that the Revolution was ``the most radical and far-reaching event in American history'' may stretch the point (did it really have more of an impact than the Civil War?). But from now on it will be hard to argue that the rebellion was a genteel event that left fundamental institutions unscathed. Wood pictures colonial society as overwhelmingly deferential—to king, to family patriarch, and to aristocrats—with ``personal obligations and reciprocity that ran through the whole society.'' But patriots such as Adams, Jefferson, and Franklin, aspiring to become gentlemen, resented this entrenched order of patronage and kinship. Their classical republicanism stressed benevolence and government by an enlightened elite. To their dismay, however, they discovered that their rhetoric unleashed all the latent entrepreneurial and egalitarian energies of American life, which even the elaborate mechanism of the Constitution could not completely contain. Among the results, Wood says, were a new concept of the dignity of labor, improvements in the lot of women, the first significant antislavery movement, and the frank acceptance of private interest underlying the political party system. Above all, Wood suggests, the Revolution produced the messy, fractious politics of liberal democracy, dominated by ordinary people pursuing commercial interests."
Recent history also proves that adherence to the constitution was mostly an honor system.
Rhode Island negotiated its Royal charter with King Charles in 1663 it has gender neutral language and freedom of thoughts. The best parts of the US constitution. Unfortunately it was also the center of slave trade.
Your status as a "defender" is measured by the fanciness of the threats you defend against, not their actual magnitude. That's why Space Force is so popular, and everyone working on vaccines is miserable.
Also, see the "lab leak" discourse. To me the question seemed irrelevant because the countermeasures would be exactly the same, but that's not how other people think; they're not interested in a mask mandate against a naturally occuring virus, but if they can tie the exact same genetically identical virus to a human enemy? The social status of the threat goes up!
Made-up AI-assisted super-measles? Doesn't exist yet. But it's a great super sexy threat to defend against. You just have to ignore the kids dying of boring old regular naturally occuring measles in the meantime.
The countermeasures to prevent future lab leaks are not the same as the countermeasures to prevent natural viruses. We should take lab safety more seriously and stop enhancing the lethality of viruses regardless of what happened with Covid, but we have failed to do so, probably in part because of denial of the lab leak.
> they're not interested in a mask mandate against a naturally occuring virus, but if they can tie the exact same genetically identical virus to a human enemy? The social status of the threat goes up!
Oh, that's what it is? I never had a model for such thinking.
I'm curious if this is related to some things the top AI models are uncovering.
We know we don't get to see the bio side of the top models. They're out there solving hard maths problems etc but a query with the word 'microbes' in it often kicks you back to a smaller model automatically.
Are the bleeding edge models finding something scary enough that can be done with synthetic DNA that it needs to be restricted asap?
Lots of scary stuff has been possible for ages. Much of it is so easy that wouldn't be publishable because of lack of novelty.
I see LLMs as being essentially irrelevant for whether someone can do this. It's lab skills, planning skills, biological experiment debugging skills.
"Why did my plasmid not start getting expressed, how did I screw up?" "How do I make this huge protein" aren't LLM questions, they're university level biology questions, and "why is my protein growth tank not making the thing I want, and what is the grey stuff, do I need to put clean it again" is an experience question.
Putting checks on DNA synthesis puts making the horrible stuff out of the range of what high schoolers can do though.
Well look at what it's doing to the hacking scene. It doesn't have to do all the work for you but people knowledgeable and motivated enough are now 10-100x more of a nuisance.
It's not like you can prompt it to create super AIDS in an afternoon but I'm sure an uncensored model could help you do real harm.
It doesn't seem to be? From what I can tell, this is just formalising what we've had to self-declare with companies like IDT for years (not encoding a known toxin, pathogen, etc). Somebody correct my if I'm wrong, but it looks like there's now a requirement for companies to actually check the sequence, rather than take the researcher's word for it.
You don't need AI to generate known toxin or pathogen sequences (as these are already known).
And if AI could design novel toxins, this screening wouldn't catch those anyway.
IDT, Twist et al do check sequences and this has been trivial for a long time. I’ve had to assure account managers that my viral sequences are legitimate as far back as 2019, but I didn’t work with viruses before then so I can’t say how new that was at the time
Probably a good thing overall in my opinion, even though it obviously has impacts.
I got into this hobby briefly (pre-LLMs) before realising how much of a loaded gun it is, and that I’m just not qualified to safely muck around with this stuff.
The 0.01% might survive, even develop some resistence. But when put again in the mix with wild non-resistent individuals in a non-clorox environment, the selection acts against them: having resistence might have costs, have no use, and if the mechanism is lost for whatever reason the new individuals survive anyway.
Something like this is suspected to have happened to us humans: the black plague has been pressuring/selecting individuals with more active and strong immune systems. Today the plague isn't so active or deadly to humans, but stronger immune systems are probably behind auto-immune illnesses (https://aihep.org/post/the-bubonic-plague-and-autoimmune-dis...).
You can actually do real biology at home if you set up a Petri dish tray with a concentration gradient of Clorox across it, then put some random germs at the low concentration and.
The device used to make oligonucleotides is pretty much an inkjet printer, from there it's cleverly applied enzymes to assemble the full strand. If you've got the skills and determination to get the synthetic DNA into a functioning cell, I don't think limiting access to synthetic DNA in the first place is going to stop you.
The device used to make oligonucleotides is pretty much an inkjet printer, from there it's cleverly applied enzymes to assemble the full strand.
Usually the synthesis of oligonuclotides is done with phosphoramidite chemistry [0]. So tell us more about those enzymes to make DNA molecules from scratch.
I’m pretty sure they were talking about how you can combine synthesized oligos into longer fragments when you are starting with smaller oligos. I’m guessing that they were referring to using different restriction enzymes from that point on.
Unless we are talking about really long oligos (which is harder and $$$), you’ll need some post-synthesis enzymatic work to get your full strand.
Yes I also read the comment that way. The inkjet printer remark presumably refers to benchtop DNA synthesis machines which have been commercially available for a while now. (The linked article mentions the possibility of buy-back programs for old machines that don't enforce any restrictions.)
I guess it's not a bad idea to put basic hurdles in place but it would be a serious mistake to see this as any sort of comprehensive preventative. DNA just isn't difficult to manipulate.
The thing is DNA just also isn't dangerous. It's fragile and everything in the environment destroys it - as noted it's literally a everything else which is the hard part.
This reeks of the Longtermists getting their influence into government. They were doing an entire round on YouTube about a year ago.
Ansa Bio uses an engineered terminal deoxynucleotidyl transferase (TdT) enzyme to synthesize the oligos themselves, rather than using phosphoramidite chemistry. Each enzyme is tethered to a single nucleotide, allowing it to extend the DNA strand by exactly one base. The tether is then cleaved and the cycle repeats.
This works in aqueous solution under mild conditions and enables synthesis of much longer oligos (~1,000 bases), which can subsequently be assembled into 50 kbp constructs.
But Esvelt points out the policy only requires screening for DNA sequences 50 base pairs or longer. A would-be bioterrorist, he argues, could easily stitch together a dangerous sequence from much smaller fragments ordered from multiple different suppliers. Esvelt says he has built an algorithm capable of screening sequences as short as 30 base pairs for their risk potential and made it freely available. “That’s the main flaw in an otherwise pretty great policy,” he says. “Don’t bother building a fence if there’s going to be an obvious huge gap in it.”
So, a song and dance of doing something that will most likely just inconvenience people doing real work.
Also, DNA chemistry is very well established. Someone with the appropriate background could easily assemble oligos without too much difficulty. I guess this stops Little Johnny from printing ebola from the internet.
>So, a song and dance of doing something that will most likely just inconvenience people doing real work.
Why does he call it an "otherwise pretty great policy"?
He's claiming that the policy needs to be more comprehensive. You're concluding that it doesn't pass a cost/benefit test. That's not at all the same claim!
> U.S. government–funded scientists who make and manipulate DNA sequences will face tougher regulations
Seems like this has little to do with deterring terrorism then, considering the assumption that a terrorist may not be a US government funded scientist.
My guess is that the assumption is that US gov funded labs are enough of many suppliers revenue that it'll be effective, but I'm 100% speculating and it may be total garbage speculation given that I have no idea how much hobbyist level DNA projects are a thing or how much they're a thing for non-medicinal business purposes.
If you read the next two sentences it makes it fairly obvious that this is a plan to indirectly target suppliers. Federally funded researchers will be required to
"procure synthetic nucleic acids and benchtop synthesis equipment from U.S.-approved suppliers. Those vendors must verify their customers’ identities and screen orders for potentially dangerous sequences that, for example, could encode toxins or pathogens."
If this was just about feds self-policing, they wouldn't have the customer identity requirement.
Probably they're taking this route because AI is moving fast and Congress moves slowly.
But yes, this is stupid. No one who is inclined to build a biological WMD is going to be more than mildly inconvenienced by supply restrictions. DNA isn't plutonium.
Sounds like a fully general argument against any sort of security measure. "A dedicated thief won't be deterred by a lock or burglar alarm." Sure. I'm still buying that stuff.
Talk about a useless policy. It just says federally funded scientists need to buy gear from US companies and register. I feel like I can confidently say whoever makes the first purpose built megavirus is not going to be federally registered and purchasing equipment through the property channels. This is security theater, but unlike TSA this is security theater with existential stakes.
Erm. Terrorists are generally full of DNA. How difficult is it to break that down, sort and purify on a scale from vinegar and soda to bioterrorist with access to a DNA printer?
So continue to fund bioweapons programs under the guise of medical research while also stifling individual and independent research under the guise of protecting us from terrorism? Yeah, that sounds like the US government.
This is not unlike 3D printing: "What? You can print a gun? Into jail you go!".
When you then look at the mafia in charge of the USA right now, run by oligarchs and other shady criminals, you ultimately have an apartheid system at work now, because these top-down pseudo-kings can control what others can do. Everyone is monitored and a potential terrorist now. Now imagine that DNA can be patented (I don't refer to right now, but look how ESTs (https://en.wikipedia.org/wiki/Expressed_sequence_tag) in the 1990s could be patented) - and suddenly you will have a state-run mafia decide which monopolist can yield de-facto control, with a few minor companies roleplaying the "it's still capitalism!!!" when in reality it no longer is.
Why is everyone spooked by ai driven bioterrorism ? Like all of our water supplies are much easier to mess with compared to developing a virus or whatever that may or may not work
Evolution cannot optimize for maximum harm. Natural viruses only evolved to reproduce, with the harm just a side effect. Humans can combine features of multiple natural viruses. There's no obvious reason why we couldn't make something with both the infectivity of measles and the lethality of rabies, as well as resistance to all known antivirals. If something like that gets out it's probably civilization ending.
With a nice and long incubation period. We wouldn't even know what happened util *pop*
Because it massively raises the amount of damage that a lone-wolf crazy person can do.
Especially because how hard it is to deal with the consequences. If a town's water supply is polluted you can shut it down and deliver water by truck until the situation is resolved. The consequences might be serious, but it would be a one-off event with limited consequences. Similarly, shutting down all freezers in a cyberattack would suck, but the consequences are limited in time
A new bioweapon let lose would be incredibly difficult to contain, and could potentially become something we have to contend with forever
Not to mention causing a famine or lack of medicine by sabotaging the cold chain.
What if the white supremacists make a retro virus, that for instance only kills people with sickle cell anemia..
That doesn't really engage with "there's easier ways to attack" or "and this tech is currently hard enough this is not a realistic attack".
That said:
What if a different group near simultaneously makes one targeting people with no copies of the HbS/HbA gene?
(One copy: resist malaria; two copies: sickle cell anaemia)
In case anyone wonder what op means - lyfgenia is a virus for treating sickle cell, a disease thats more common among black population.
TIL God is a white supremacist
Are you actually asking this? Like why should people be worried that a disgruntled highly motivated person would venture into this area? Because it can happen. The outcomes of it could be really bad? It also doesn't have to be done by an individual, could be nation state or organization led. There are groups of people who believe humanity needs to end and all sorts of crazy stuff. Some of them have gone on mass murder suicide missions.
2 bad things can be threats at the same time. Even 10,000 things can be.
That seems a bit silly thinking they are not doing anything to protect the water supplies, and even more silly to not compare the impact of the things you are comparing. Poison a water supply and you hit how many, a city, a small part of a city? Now compare it to the impact of a virus like corona - we literally just went through that..
Instead of poisoning the water supply, you'd stuxnet the pumps. I imagine the network interfaces for status and control are about as well protected as grocery store freezers.
A bad pathogen could go world wide, but you could trash multiple cities with online attacks. No electricity, no water, no communications. Things would get bad.
Coronavirus++ is not something that I would want on my calendar
I'm not too worried. That version of the virus only gets more dangerous after being cleared.
++coronavirus though...
Yes, think of the null pointer dereferences, but now inside your body.
Disrupting electric supply, water supply, and food supply chains are already tried and tested separately, with some limited monetary system disturbances, here and there. So doing those simultaneously - possibly leveraging on latent vulnerabilities kept covered for later abuse - sounds a natural step 'forward', close to trivial for those into that. Causing huge broad disturbances and possible unrest in societies. So that some governments started to ask people to build up reserves.
I can imagine that terrorist are sad now that they cannot use their preferred advanced bioengineering skills acquired through great efforts posing as scientist towards the so far unsuspecting services providing them with harmful DNA building blocks and now have to turn to much much more pedestrian methods instead of the heightened sophistication they crave.
Or to sources from other countries.
Its hard to imagine a mustache twirling terrorist bent on putting PFAS and lead in the american water supply, putting mad cow in the beef supply and making everything cost more because he would have been outcompeted by the US government.
The US Government doesn't do these things. The people running businesses do these things.
Have no idea what happens in the US but in Australia nearly all PFAS in the country comes from government properties, most often military bases or airports.
Sure, that's why my water supply is contaminated by pfas from the Air Force base nearby washing it into the water table for decades.
And yet the Russian foreign policy is almost this: they know they can't compete on competence/science/tech so they try to tear everything down to their level.
Russia has always been famous for science and technology, if not political competence.
The jewish exodus from Russia due to the disaster of the Soviet era put a dent in that I'm sure.
Also the separation from Ukraine, who were a major contributor to Soviet science.
The name of that entity was the soviet union
I think this is a bad take. The Russian government wants to tear things down because they are a kleptocracy, and functional nations act as a check on corrupt governments. The more corrupt the US is the more theft russia can get away with and collude with pals inside the US gov on.
The framing of nations as inherently in competition with each other doesn’t track for me. Sometimes, in specific instances, nations will be in competition, but finland having a good or bad education system does not impact me even slightly.
>finland having a good or bad education system does not impact me even slightly.
Yet you do depend on Linux.
And SSH.
Is *sshole the result of Finland failing to compete with Russia? Curious minds need to know!
> The framing of nations as inherently in competition with each other doesn’t track for me.
Where it tracks for you or not isn't relevant. It tracks for Putin and he's been extremely clear about that for hours and hours of extremely boring monologues :P
They have placed agent Krasnov though, so they influence US politics anyway. What is even more surprising is that Krasnov is now doubling down, despite the clear loss in the latest election.
The regulatory state didn’t fail out of bureaucratic clumsiness; it was systematically bought and gutted so private balance sheets wouldn’t have to carry the cost of your cancer.
Blaming "the government" for toxic tap water and runaway prices is like blaming the getaway car instead of the bank robbers.
Private balance sheets wouldn’t have to carry the cost of your cancer is interesting. Do public balance sheets have to carry the cost of alcoholic's cirrhosis, smoker's lung cancer, and Type 2 diabetic's myriad of issues? Or does personal responsibility only apply to governments and corporations, not people?
Yes? Remember, the government decided alcohol and tobacco could advertise the way they do because it helps them make money.
>Blaming "the government" for toxic tap water and runaway prices is like blaming the getaway car instead of the bank robbers.
Blaming the government for toxic tap water is like blaming an organised private/public coalition of bank robbers for a bank robbery.
>it was systematically bought and gutted so private balance sheets wouldn’t have to carry the cost of your cancer.
The US Government, day 1, was bought and sold to defend the right to own slaves, even from the predation of states that had made it illegal. It's not some recent malformation that it just started defending the interests of rich people over poor people lmao. The explicit purpose of the US government is to do the shit it does. The machine has been designed well, not poorly.
Sure, and the US was also home to the first abolitionist society, and fought its bloodiest war in history (by % of population killed) in a war to end slavery.
"...this sparkling analysis from Wood (History/Brown Univ.; ed., The Rising Glory of America, 1971) impressively argues that [the American Revolution] was anything but conservative. Wood's contention that the Revolution was ``the most radical and far-reaching event in American history'' may stretch the point (did it really have more of an impact than the Civil War?). But from now on it will be hard to argue that the rebellion was a genteel event that left fundamental institutions unscathed. Wood pictures colonial society as overwhelmingly deferential—to king, to family patriarch, and to aristocrats—with ``personal obligations and reciprocity that ran through the whole society.'' But patriots such as Adams, Jefferson, and Franklin, aspiring to become gentlemen, resented this entrenched order of patronage and kinship. Their classical republicanism stressed benevolence and government by an enlightened elite. To their dismay, however, they discovered that their rhetoric unleashed all the latent entrepreneurial and egalitarian energies of American life, which even the elaborate mechanism of the Constitution could not completely contain. Among the results, Wood says, were a new concept of the dignity of labor, improvements in the lot of women, the first significant antislavery movement, and the frank acceptance of private interest underlying the political party system. Above all, Wood suggests, the Revolution produced the messy, fractious politics of liberal democracy, dominated by ordinary people pursuing commercial interests."
https://www.kirkusreviews.com/book-reviews/gordon-s-wood/the...
Recent history also proves that adherence to the constitution was mostly an honor system. Rhode Island negotiated its Royal charter with King Charles in 1663 it has gender neutral language and freedom of thoughts. The best parts of the US constitution. Unfortunately it was also the center of slave trade.
Commerce gets in the way of ideals.
Your status as a "defender" is measured by the fanciness of the threats you defend against, not their actual magnitude. That's why Space Force is so popular, and everyone working on vaccines is miserable.
Also, see the "lab leak" discourse. To me the question seemed irrelevant because the countermeasures would be exactly the same, but that's not how other people think; they're not interested in a mask mandate against a naturally occuring virus, but if they can tie the exact same genetically identical virus to a human enemy? The social status of the threat goes up!
Regular old measles? Not sexy. But a convenient vector for the internal threat. https://www.politico.com/news/2026/11/08/americas-measles-ep...
Made-up AI-assisted super-measles? Doesn't exist yet. But it's a great super sexy threat to defend against. You just have to ignore the kids dying of boring old regular naturally occuring measles in the meantime.
The countermeasures to prevent future lab leaks are not the same as the countermeasures to prevent natural viruses. We should take lab safety more seriously and stop enhancing the lethality of viruses regardless of what happened with Covid, but we have failed to do so, probably in part because of denial of the lab leak.
> they're not interested in a mask mandate against a naturally occuring virus, but if they can tie the exact same genetically identical virus to a human enemy? The social status of the threat goes up!
Oh, that's what it is? I never had a model for such thinking.
I'm curious if this is related to some things the top AI models are uncovering.
We know we don't get to see the bio side of the top models. They're out there solving hard maths problems etc but a query with the word 'microbes' in it often kicks you back to a smaller model automatically.
Are the bleeding edge models finding something scary enough that can be done with synthetic DNA that it needs to be restricted asap?
Lots of scary stuff has been possible for ages. Much of it is so easy that wouldn't be publishable because of lack of novelty.
I see LLMs as being essentially irrelevant for whether someone can do this. It's lab skills, planning skills, biological experiment debugging skills.
"Why did my plasmid not start getting expressed, how did I screw up?" "How do I make this huge protein" aren't LLM questions, they're university level biology questions, and "why is my protein growth tank not making the thing I want, and what is the grey stuff, do I need to put clean it again" is an experience question.
Putting checks on DNA synthesis puts making the horrible stuff out of the range of what high schoolers can do though.
Well look at what it's doing to the hacking scene. It doesn't have to do all the work for you but people knowledgeable and motivated enough are now 10-100x more of a nuisance.
It's not like you can prompt it to create super AIDS in an afternoon but I'm sure an uncensored model could help you do real harm.
It's more likely they're surfacing the contents of deeply technical biology papers to average people who may or may not be thinking about terrorism.
It doesn't seem to be? From what I can tell, this is just formalising what we've had to self-declare with companies like IDT for years (not encoding a known toxin, pathogen, etc). Somebody correct my if I'm wrong, but it looks like there's now a requirement for companies to actually check the sequence, rather than take the researcher's word for it.
You don't need AI to generate known toxin or pathogen sequences (as these are already known).
And if AI could design novel toxins, this screening wouldn't catch those anyway.
IDT, Twist et al do check sequences and this has been trivial for a long time. I’ve had to assure account managers that my viral sequences are legitimate as far back as 2019, but I didn’t work with viruses before then so I can’t say how new that was at the time
Unless you use AI for the screening? Surely that's the obvious move.
Alphafold or something like that is more likely than the LLM models. Though cant say what an LLM model that uses Alphafold as a tool can do.
No not at all. The top models are not uncovering anything novel with DNA.
This is old stuff, being implemented and getting lots of news.
How you know? The top models have a tendency to not tell what they're conspiring for.
well math you can run Lean...
but for bio... I don't think we have 99.999999% accuracy simulation, and experiments aren't "0.00001 cent"
so we have to rely a lot on existing data or pay a lot
This might impact me and a small handful of other people who have messing around with microbes in their apartment as a hobby. We'll see what happens.
Probably a good thing overall in my opinion, even though it obviously has impacts.
I got into this hobby briefly (pre-LLMs) before realising how much of a loaded gun it is, and that I’m just not qualified to safely muck around with this stuff.
I suppose you might have to go back to bathtub explosives as a safer alternative.
Yeah me too. Using Clorox on my toilet seat kills always 99.99% germs. The fun is with the 0.01% that's left
The 0.01% might survive, even develop some resistence. But when put again in the mix with wild non-resistent individuals in a non-clorox environment, the selection acts against them: having resistence might have costs, have no use, and if the mechanism is lost for whatever reason the new individuals survive anyway.
Something like this is suspected to have happened to us humans: the black plague has been pressuring/selecting individuals with more active and strong immune systems. Today the plague isn't so active or deadly to humans, but stronger immune systems are probably behind auto-immune illnesses (https://aihep.org/post/the-bubonic-plague-and-autoimmune-dis...).
Similar with sickle cell & malaria
You can actually do real biology at home if you set up a Petri dish tray with a concentration gradient of Clorox across it, then put some random germs at the low concentration and.
Yeah, or antibiotics maybe. But still, I could just put bare wires in my bathtub, no need for such effort :wink:
The device used to make oligonucleotides is pretty much an inkjet printer, from there it's cleverly applied enzymes to assemble the full strand. If you've got the skills and determination to get the synthetic DNA into a functioning cell, I don't think limiting access to synthetic DNA in the first place is going to stop you.
The device used to make oligonucleotides is pretty much an inkjet printer, from there it's cleverly applied enzymes to assemble the full strand.
Usually the synthesis of oligonuclotides is done with phosphoramidite chemistry [0]. So tell us more about those enzymes to make DNA molecules from scratch.
[0] https://www.twistbioscience.com/blog/science/simple-guide-ph...
I’m pretty sure they were talking about how you can combine synthesized oligos into longer fragments when you are starting with smaller oligos. I’m guessing that they were referring to using different restriction enzymes from that point on.
Unless we are talking about really long oligos (which is harder and $$$), you’ll need some post-synthesis enzymatic work to get your full strand.
Yes I also read the comment that way. The inkjet printer remark presumably refers to benchtop DNA synthesis machines which have been commercially available for a while now. (The linked article mentions the possibility of buy-back programs for old machines that don't enforce any restrictions.)
I guess it's not a bad idea to put basic hurdles in place but it would be a serious mistake to see this as any sort of comprehensive preventative. DNA just isn't difficult to manipulate.
The thing is DNA just also isn't dangerous. It's fragile and everything in the environment destroys it - as noted it's literally a everything else which is the hard part.
This reeks of the Longtermists getting their influence into government. They were doing an entire round on YouTube about a year ago.
DNA is pretty robust. You can send a dried spot of it on a piece of filter paper and send it in the mail.
Ansa Bio uses an engineered terminal deoxynucleotidyl transferase (TdT) enzyme to synthesize the oligos themselves, rather than using phosphoramidite chemistry. Each enzyme is tethered to a single nucleotide, allowing it to extend the DNA strand by exactly one base. The tether is then cleaved and the cycle repeats.
This works in aqueous solution under mild conditions and enables synthesis of much longer oligos (~1,000 bases), which can subsequently be assembled into 50 kbp constructs.
Not sure about that inkjet printer analogy.
Also, DNA chemistry is very well established. Someone with the appropriate background could easily assemble oligos without too much difficulty. I guess this stops Little Johnny from printing ebola from the internet.
>So, a song and dance of doing something that will most likely just inconvenience people doing real work.
Why does he call it an "otherwise pretty great policy"?
He's claiming that the policy needs to be more comprehensive. You're concluding that it doesn't pass a cost/benefit test. That's not at all the same claim!
> U.S. government–funded scientists who make and manipulate DNA sequences will face tougher regulations
Seems like this has little to do with deterring terrorism then, considering the assumption that a terrorist may not be a US government funded scientist.
My guess is that the assumption is that US gov funded labs are enough of many suppliers revenue that it'll be effective, but I'm 100% speculating and it may be total garbage speculation given that I have no idea how much hobbyist level DNA projects are a thing or how much they're a thing for non-medicinal business purposes.
If you read the next two sentences it makes it fairly obvious that this is a plan to indirectly target suppliers. Federally funded researchers will be required to
"procure synthetic nucleic acids and benchtop synthesis equipment from U.S.-approved suppliers. Those vendors must verify their customers’ identities and screen orders for potentially dangerous sequences that, for example, could encode toxins or pathogens."
If this was just about feds self-policing, they wouldn't have the customer identity requirement.
Probably they're taking this route because AI is moving fast and Congress moves slowly.
Trouble is, that assumption hasn't always worked out in the past: https://en.wikipedia.org/wiki/Bruce_Edwards_Ivins
But yes, this is stupid. No one who is inclined to build a biological WMD is going to be more than mildly inconvenienced by supply restrictions. DNA isn't plutonium.
Sounds like a fully general argument against any sort of security measure. "A dedicated thief won't be deterred by a lock or burglar alarm." Sure. I'm still buying that stuff.
More like, Danny Ocean isn't going to be fazed by your ADT Security sign out front or the Ring camera on your door.
Why assume an ultra-competent wrongdoer?
https://news.ycombinator.com/item?id=49606803
When DNA is outlawed only outlaws will have DNA.
Considering what we know about crowding-out in human microbiomes, this is probably way more relevant than the meme you were likely swinging for!
Talk about a useless policy. It just says federally funded scientists need to buy gear from US companies and register. I feel like I can confidently say whoever makes the first purpose built megavirus is not going to be federally registered and purchasing equipment through the property channels. This is security theater, but unlike TSA this is security theater with existential stakes.
> It’s great, because it does something instead of nothing
I am detecting a bit of frustration here.
Yeah, I'm assuming that was intended to be sarcasm.
(Also, my comment is great, because I wrote something instead of nothing.)
[ This comment left intentionally blank ]
Erm. Terrorists are generally full of DNA. How difficult is it to break that down, sort and purify on a scale from vinegar and soda to bioterrorist with access to a DNA printer?
No sequences encoding harmful stuff like covid19 spike proteins!! That means no mRNA vaccines that make spike protiens!! ;-)
Where is the disclaimer, that all labs are 100% safe, and it is not possible for something like virus to escape from lab?
So continue to fund bioweapons programs under the guise of medical research while also stifling individual and independent research under the guise of protecting us from terrorism? Yeah, that sounds like the US government.
Let's ban knives too because they're sharp and might hurt someone.
12 Monkeys here we come!
Cluade find how to time travel, Make no mistakes!
This is not unlike 3D printing: "What? You can print a gun? Into jail you go!".
When you then look at the mafia in charge of the USA right now, run by oligarchs and other shady criminals, you ultimately have an apartheid system at work now, because these top-down pseudo-kings can control what others can do. Everyone is monitored and a potential terrorist now. Now imagine that DNA can be patented (I don't refer to right now, but look how ESTs (https://en.wikipedia.org/wiki/Expressed_sequence_tag) in the 1990s could be patented) - and suddenly you will have a state-run mafia decide which monopolist can yield de-facto control, with a few minor companies roleplaying the "it's still capitalism!!!" when in reality it no longer is.
This sounds unbelievable, but hey, we have AI. Let's just keep pretending this is the regular rate of tech advancement ...
Carry on, everything is apparently fine.