> a uniformity that eliminated diversity and that chose homogenization over communion
Unrelated to AI, but a wonderful support of the breadth of humanity in this anti-DEI time.
> We must, then, avoid the “Babel syndrome,” namely the idolatry of profit that sacrifices the weak, a uniformity that neutralizes differences, and the pretense that a single language — even a digital one — can translate everything, including the mystery of the person, into data and performance.
There is a lot to read here. I am curious where the meditations on the 'mystery of the person' will go: a brief search doesn't show further mention. The encyclical appears to focus on exhortations for us, humans, than on the nature of AI. Probably wise at this stage. I feel it is not AI that is either positive or negative, but its use of it, and the call-out to the growth of private industry as more powerful among nation-states is a strong statement for a institute like the Vatican to make:
> Technological power thus takes on an unprecedented, predominantly “private” aspect, which makes it even more challenging to discern, govern and direct such power toward the common good.
The “mystery of the person” is a common theme in Catholicism - it can be seen in other (especially the “social”) encyclicals. Rerum Novarum by the current pope’s namesake is worth a study.
I fear I have committed the sin (I mean that word, here) of commenting before reading the entire article. Often it is the comments that set the tone of interpretation and communication more than the article itself, because people read the comments, not the article.
To somewhat mitigate that, here are items that are striking me as I read more. (I hope you'll forgive that this still directs the comment in the direction of my own lense.) I'll keep updating the comment.
> Looking at our own time, we cannot ignore the fact that the protection of human rights [as declared by the United Nations in 1948] has been exposed to two particularly serious dangers. The first is that these rights are declared in a purely formal sense, while technological progress continues alongside covert or overt violations of human dignity.
I read this as a warning of how rights are words, but actions are performed regardless of them. It aligns with something I've been trying to word, which is that as I've seen more and more abuses of power, I've come to believe that ethics requires external accountability, which can often require its own power - a conclusion I don't want to come to; I would prefer social agreement and communal spirit rather than external power. But either way, I do feel it's very clear there are a lot of people, very much in tech too, who simply do what they want regardless of its harm. They justify it to themselves; they don't stop themselves; no one externally stops them.
> Along with a greater awareness of the value of every human person and their rights, recognition of minority rights has also grown. Yet, there is still a long way to go to ensure that the rights of a great many, namely women [are guaranteed.] It is, therefore, not enough to state simply that men and women have equal dignity and rights; it is necessary that this be reflected in concrete decisions, such as in laws, access to employment, education, social and political responsibilities, and the way society listens to and values women’s contributions.
In 2026 American politics terms, this reads as pro-DEI to me.
> the first major principle of Social Doctrine that I wish to highlight: the common good. We can describe it as the social expression of the dignity recognized in every person. ... For a Christian, going beyond the narrow confines of one’s own interests and committing oneself, within the limits of one’s ability, to the common good is a non-negotiable value, as is the promotion of life.
'Non-negotiable.' Very clear words.
> When politics abandons a long-term perspective and reduces itself to short-term calculations or sterile polarizations, then the language of the common good loses credibility, and, at the same time, social inequalities and divisions grow.
> 64. This also applies to international politics.
and,
> I invite everyone to conceive of ways of cooperating and of more effective international institutions, capable of safeguarding the global common good without compromising the legitimate diversity of peoples and nations. Indeed, the promotion of the common good can never be separated from respect for the right of peoples to exist, to preserve their own identity and to contribute their unique qualities to the family of nations.
I love the support for international cooperation and peace and organisations that support it. It reminds me of the post-WW2 sense, the era that gave rise to the United Nations, Unicef, etc - organisations almost forgotten in the news we see on HN today, with the possible exception of the WHO.
> [84] Moreover, any attempt or plan to eliminate or subjugate a nation is gravely immoral and therefore unacceptable.
The beauty of this - or its tragedy - is that it is so easy to apply to many situations today, actions undertaken by many nations.
> the earth’s goods — soil, water, air and natural resources — are given by God to the entire human family to sustain the lives of all, and that every person has an inherent right to the use of such goods, both now and in the future. ... Today, we are called to recognize that this universal destination applies not only to material goods, but also to immaterial and cultural goods.
Immaterial and cultural goods. This is a fascinating view on non-tangibles and one I feel inspired by. Reading this I asked myself (wait for the larger quote in a minute) how this affects views of IP, learning when texts are not available, cultural impact of characters and stories, the output from universities, publication of papers, ownership of research done by public or even private (!) funds, and more. Particularly I wonder about open weights vs open source for AI, and open source as a concept: where the old-school 'free software' GPLed version seems -- perhaps I am showing my bias -- most aligned with the ethical stance here?
> 66. Certainly there is a right to private property, which has its own specific meaning and purpose, yet it is always subordinate to the universal destination of goods.
'Always subordinate'.
> among the goods that are universally intended for everyone, we must also include new forms of property, such as patents, algorithms, digital platforms, technological infrastructure and data. In a context where the wealth of nations depends increasingly on knowledge and technology, when these goods remain concentrated in the hands of a few, without adequate forms of sharing and access, a new imbalance is created that contradicts the universal destination of goods. In turn, it widens the gap between the included and the excluded, between those who can participate in the digital revolution and those who remain on the margins.
Wow!
I cannot interpret this; it's not my right. But moving from the questions I asked above, to this paragraph, is powerful.
> > a uniformity that eliminated diversity and that chose homogenization over communion
> Unrelated to AI, but a wonderful support of the breadth of humanity in this anti-DEI time.
I mean.. DEI was in reality homogenization and eliminated diversity. Just because you agreed with the small amount of allowed opinions/people doesn't make it more diverse.
DEI was only ever meant to reflect the diversity of society in our institutions (which historically have been and continue to be homogeneous compared to society). Of course there are better and worse ways to create the outcome of increased diversity. But thr recent backlash is the antithesis of diversity, wrapped up in the propaganda of "reverse racism."
The anti-DEI “movement,” especially what we see in the US, is mostly thinly veiled racism by another name.
All one has to see is how frequently people call any non-white/non-male hire or appointment a “DEI-hire” because they can’t possibly conceive that marginalized groups would see any success based on merit. They start from a position where they are all incompetent and being given an unfair advantage until proven otherwise. It’s also on you to guess whatever their arbitrary bar is.
'you shall have no other gods before me'. operationally, a warning. false worship is dangerous. we get what we worship (value above all, orient towards, optimize for). heaven or hell, read it immanently or transcendentally, personally or collectively. either way, choosing the right worship is paramount.
the conundrum: what is the right objective? puerile attempts, eg "maximize flourishing" or perhaps "minimize suffering" are trivially countered by paperclip machines fulfilling the letter while laying utter waste
'mystery of the person'
given the context, some keywords for orientation: free will, rejection of temptation, the cross
>“technology is never neutral, because it takes on the characteristics of those who devise, finance, regulate, and use it,”
If critical decisions affecting human life—such as hiring, lending, crime prediction, and welfare—are processed in an opaque black box, people will lose their fundamental right to explain their context or appeal against the machine's algorithmic verdicts
Is it wrong? Individuals were absolutely getting dehumanized in Kafka's time, but also since then are becoming ever increasingly more so, so it makes sense to speak of it continuing to happen and get worse. It is not binary.
This quote is from 1903. Times haven't changed that much:
> Not a handful of rich people, but all the working people must enjoy the fruits of their common labour. Machines and other improvements must serve to ease the work of all and not to enable a few to grow rich at the expense of millions and tens of millions of people.
There's been plenty of examples of workers seizing the means of production and establishing sustained non-capitalist organization (State or otherwise). We have any number of strategies to choose from: The PRC, Soviet Union, Syndicalized Spain (my favorite, seems the least likely to lead to police state), Vietnam, Cuba. The question thus isn't "how," but more specifically a couple other questions: "How do we prevent capitalist forces from liberalizing our movement (PRC, Soviet Union)," "How do we prevent fascists from killing us all (Syndicalized Spain)," "How do we prevent becoming a state-capitalist police state that halts the revolution (PRC, Soviet Union)"?
While this is certainly the more interesting question, the unfortunate reality is that the ideological complex of capital (even if weakening, and no longer effectively reproducing itself) is still strong enough that most in the West can't even imagine a better world (other than "less bad capitalism"), much less think about how to get there. Consequently messages like the above are of great value in moving more people towards a point where questions like yours become relevant.
Well, in that case, my "how" has always been along anarchistic lines: establishing parallel forms of resource distribution, establishing habits and communities of mutual aid, and in doing so, delegitimizing and rendering obsolete the State, capitalism, and systems of hierarchy.
Fun techcentric utopian speculation about this: Cory Doctorow's "Walkaway" and Ruthanna Emry's "A Half-Built Garden."
Essentially, can we leverage our current post-scarcity society to expropriate everything people need in a sustainable way that cuts capitalism and the State out of the loop? For example, why would people buy food if they can get it for free from farming syndicates or similar? (see: Global Village Construction Kit, Food Not Bombs, Food Not Lawns) Why would people buy medicine if they can print it for free from pirated recipes? (see: Four Thieves Vinegar Collective)
I see the Right to Repair and FOSS movements as a foundation to build upon for this. Anarchism (or at least, anti-capitalism) exists right under everyone's noses, in all the FOSS software installed on their computers. Existent example of people laboring without profit motive and contributing to the commons.
My personal life goal is to figure out how to capture that same energy to tackle the bottom layers of Maslow's hierarchy.
I'm all in favour of grassroots experimentation and a search for something to improve upon, and then replace capitalism. This is how capitalism itself came about and spread, though we can argue about how much it was imposed after it ceased being the underdog.
What I am weary of is that such experimentation, and the energy it generates, will eventually be overtaken by the next iteration of people who want to stop nibbling at the margins and break a few eggs already, some sort of anarchist revolutionary vanguard. Much like with communism, skilful opportunists with a thirst for power will be all too happy to take over this energy and direct it toward building the next totalitarian regime, one which will of course claim to be rendering the State obsolete, but will be about as anarchist as North Korea is a people's democratic republic.
Yep but the anti-socialism/communism world did wonders to make that feel like kryptonite whenever those words get brought up, even though anyone who is doesn't see themselves as "rich" in that sentence who fully agree. That's why even factory workers are anti-communism or anti-union which are literally the best way to fight back the imbalance of power.
You have to somehow separate the horrible evils that have been inflicted on the world by Communism before you can get people to consider words closely associated with it.
Being anti-communism is good not only for the individual's health but for their society as a whole.
The problem, generally, with this view point is that it attributes all of a societies ills to Communism and none of (or few of) societies ills to Capitalism.
For example, do you believe the Capitalist system has nothing to do with the eagerness of the United States to drop bombs throughout the world for the past 100 years? Personally I see these actions as unnecessary and evil but pushed to continue by the people who stand to gain the most wealth and influence from them.
What specific horrible evils do you mean? And how do you attribute them to, specifically, organizing an economy along communistic lines?
I ask because if we can take a country with a communist economy, or striving for one, and blame all its evils on communism itself, I have a few things I'd like to point out as being the horrors of capitalism:
1. Atlantic slave trade - millions dead (many on the ships), millions enslaved
2. Settler colonialism and indigenous genocide - British empire, all over the world
3. Congo Free State, Leopold II - 1 to 5 million dead via colonial extraction regime
4. British India famines - 3 million dead
5. Irish Famine - 1 million dead
6. Opium wars - directly caused by British using the military to defend market access. 100k dead, devastating to Qing China for a century
7. Indonesian anti-communist massacres - 500k-1mil alleged "communists" killed after the USA, UK, and Australian intelligence agencies propagandized against them
The 1956 student massacres in Hungary, where my grandma was almost killed. The Holodomor, the various "Russianization" campaigns, the Jewish Autonomous Oblast, The Great Leap Forward, etc.
B) Fine, drop communism altogether -- it's evil and disgusting and bit my finger and should never be tried again. Can we work on a society where the means of production are owned by groups of laborers?
Re: B my guess is probably not (human nature and all that), but I'm open to ideas! I just think failed experiments where tens of millions died are probably not ones where we just flippantly "try again".
Not new for software and hardware industry though, practitioners have just chosen to ignore it. From the Association for Computing Machinery, which encompasses all forms of software development, the very first principle is the public good:
"Software engineers shall act consistently with the public interest. In particular, software engineers shall, as appropriate:
1.01. Accept full responsibility for their own work.
1.02. Moderate the interests of the software engineer, the employer, the client and the users with the public good.
1.03. Approve software only if they have a well-founded belief that it is safe, meets specifications, passes appropriate tests, and does not diminish quality of life, diminish privacy or harm the environment. The ultimate effect of the work should be to the public good. ..."
From the IEEE, which also encompasses computer engineering, their first principle and its first few sub-items are:
"To uphold the highest standards of integrity, responsible behavior, and ethical conduct in professional activities.
1. to hold paramount the safety, health, and welfare of the public, to strive to comply with ethical design and sustainable development practices, to protect the privacy of others, and to disclose promptly factors that might endanger the public or the environment;
2. to improve the understanding by individuals and society of the capabilities and societal implications of conventional and emerging technologies, including intelligent systems; ..."
This would be more meaningful if, perhaps, we had to swear an oath to it before being able to practice. And practitioners would be treated more seriously if everyone knew we swore that oath. And the legal utility as accountability and defense would also be useful.
Of course people are going to ignore it if there's no force behind it.
I don't think there are more grey areas in software engineering than in medicine. The difference is the feedback loop of the outcome - if you design a dopamine slot machine you will ruin the generation and that's a long arc.
And that makes it hard. I am open for banning all comercial advertisement - but general society is largely fine with it. So is someone designing new targeting algorithm for ads breaking his potential oath of doing good for society?
I think not just (computer) scientists but the general population thinks to serve the common good makes sense, not last because we understand it's eventually for our own good.
It is however just that very small minority of the population with highly psychopathic/narcissistic traits - those that, in pre-historic times, would have been kicked out swiftly of hunter & gatherer / small village communities because of their parasitic nature - that in bigger civilisations seem to thrive due to abstraction (distance/time of the effect of their actions) and obfuscation (PR) and instead unfortunately seem to rise to the top (CEOs, presidents, 'thought leaders' ...) to steer the world's overall economy and mindset - and steer it in the abyss.
Sometimes I think humanity was just not made to scale, and this aspect is one very large aspect of it.
Not only builders, the greatest takeaway for me is that everyone has a responsibility in shaping the discourse, culture, and usage of transformative technology. This "the builders will do the right thing" mentality is even (in my interpretation) explicitly called out in several places:
> It is the pursuit of the common good that gives life to a people, understood not as a mere collection of individuals, but as a living reality in which people learn to recognize that they themselves are interconnected and jointly responsible for the res publica. In this sense, every person contributes to the building up of one’s people...
> When it comes to decisions regarding economic flows and digital platforms, as well as the governance of data and algorithms, we cannot allow a handful of actors to dictate these processes on their own; instead, we must build forms of cooperation that respect the various levels of the global community and make them jointly responsible for the common good.
> We cannot be satisfied with merely calling for the moralization of machines — the so-called “alignment” of AI with human values — without also having the courage to insist on a further condition: the possibility of openly discussing the ethical frameworks involved and subjecting them to shared standards of social justice.... What is needed is a more active political involvement that is capable of slowing things down when everything is accelerating, and of protecting the opportunities for communities still to be able to participate and ask questions.
Agreed, I think the builders are the wrong people to ask for this self reflection. Anything can be used for good and bad. A knife can be used to prepare food, or to stab someone. A drone can be used to attack a country, or to defend a country. A wooden beam can be used to build a house, or to build a cross to crucify someone. Same way AI can be used for good and bad. And it’s up to the person using the tool to act moral. Unfortunately this world is full of people that consider power and money more important than morals. For me, religion falls in this bucket, lots of “do as I say, not do as I do”.
The Roman Catholic Church has a lot of things wrong with it, now and in the past, but it’s a human institution older than almost any other, and it’s composed of a lot of very intelligent people. Agree or disagree with them, but a papal encyclical is almost always worth reading and understanding.
There are so many encyclicals, apostolic letters, etc. One could spend years reading just a fraction of them, depending on reading and comprehension speed, of course, which varies by person.
Two I recommend, from the last 40 years:
Veritatis splendor, John Paul II, 1993
Argues that Christian freedom is fulfilled, not limited, by objective moral truth: some acts are intrinsically evil regardless of intention or circumstance, conscience must be formed by divine law rather than self-authorization, and the Church must faithfully teach this moral truth as the path to authentic human flourishing in Christ.
Argues that faith and reason are complementary paths to truth: reason needs faith to avoid skepticism, relativism, and reductionism, while faith needs reason to express, defend, and deepen its understanding of divine revelation and the human search for meaning.
The Roman Catholic Church is an institution that went to war to try to prevent people from being able to read the bible in their own languages, that executed dissenters who challenged the doctrine on which it based its power, from scientists to Protestants. If it had never lost power in western Europe, it's quite possible the enlightenment and industrial revolution would never have happened, and humanity would still be living in the dark ages.
The strange part is how moral responsibility somehow always lands on the builders... the people with the least leverage... while the funders get to ask the ethical questions. Weird!
No, we don't have to take the funders' money; that's what having professional standards means. Nobody would excuse a doctor performing unsafe procedures because they "needed the money". Engineers were jailed for the Volkswagen emissions tampering scandal and nobody would excuse them for needing to take funders' money.
I agree that holding only the concrete implementers responsible would be inappropriate. However, I don't believe that distinction is made. One says "I am building a house" even if they are completely contracting the job out. I'd suggest the greatest responsibility lays with the funders and that the Pope would agree.
The funders are among the principal builders in this context. This is addressing the people who have a say in what is built, and how it is built. Much of that belongs to the executive and ownership class, but not all of it.
Yes but... No one living today knows what direction AI technology will take humanity. If we have an algorithm breakthrough then we may avoid building new data centers. If the abilities of the technology plateau then there might not be large impacts on employment. Builders need to focus on the impact of their next steps. Don't put polluting natural gas generators in neighborhoods today. Don't make unemploying folks the goal of your tool, today. Don't make decisions that harm people and the environment today.
> The Stratos artificial intelligence datacenter footprint will cover more than 40,000 acres (62 sq miles) over three sites in Box Elder county in north-western Utah. The facility will require about 9GW of power, which is more than the entire state of Utah currently consumes
Sure hope your rosy inflection point happens real soon.
Otherwise, the direction this is taking us is pretty obvious.
> Therefore builders "bear a particular ethical and spiritual responsibility"
> This is a message we need right now.
Feels good man. The solution found by the private parties driving technological change is sainthood. Or aiming for it. At least, better than you. They have the vision of what's good for the herd, but the more time I spend as a sheep, the more it seems the "herd" is just a way to recycle the story of their own exceptionalism stripped of any mark of individuality. A simple visit to fiftyyears.com will greet you with "We back the indispensable". I guess it's the same "we".
> Scientists are actually preoccupied with accomplishment. So they are focused on whether they can do something. They never stop to ask if they should do something.
From Crichton's book Jurassic Park, which like most of his books is about the perils of technological advancements.
They used the quote in the movie, slightly tweaked.
I would replace the word scientists with engineers in that quote. People often conflate the two, but in my experience, scientists tend to be more cautious and there are built in checks and balances in the process (however flawed).
Engineers/technologists tend to have no such guardrails, and are also usually embedded into entirely profit motivated environments, whatever their own values might be.
This sounds like you don’t have much exposure to actual professional engineering disciplines. I’m sure civil, electrical, structural and mechanical PEs would be quite surprised to hear there are no guardrails on their professions.
I have fond memories of a boss who was an actual, licensed engineer while the rest of us were very much normal software devs. Boss was pretty chill except when someone someone suggested we should be called "engineers" rather than "developers", at which point they said "if you guys were building bridges, people would be dead." (I don't think all software needs to be built to rigorous engineering standards but man... I think about that line a lot.)
I prefer the term "software developer" and that's what I use when I don't need the prestige of the term "software engineer". It's disadvantageous for organizations to do that with actual job titles, though.
Absent US government intervention to codify the term "engineer", probably the only way out of the "engineer" trap is through further title inflation, where the developers all become "vice presidents". :)
In my experience “engineers” and builders are often quite “conservative” and really don’t like pushing the envelope, and they often only do it under protest.
The most famous example may be the perpetual war between architects and engineers/builders.
I actually originally wrote "technologists" but thought that the word sounded kind of odd. Now I realize it better captures what I was trying to convey.
Researchers need to go wild and sometimes far off-the-rails to increase the odds of coming up with something that is both new, and potentially popular enough, if they want the option to attract marketers who can only thrive on mass-consumption.
With luck, one out of 100 inventions will show promise on those points.
There's always a lifetime wake where the overwhelming vast majority of the work remains undeployed no matter what. The more undeployed milestones and inventions that some scientists have under their belt, the more accomplished they often are whether anybody knows it or not.
OTOH, equally active engineers more often need to have most of their time engaged in actual deployment of some kind or another, otherwise not as much progress will be able to reach as many people that could benefit. So many times nothing would be accomplished without a long-term focused engineering effort once an objective has been identified. But it can be hard to stop a train when it's already coming off the drawing board at full steam.
It does seem a lot more likely for a judicious researcher to cast off some major progress in what could very well turn out to be an unsavory development, such as likely misuse, even if it could be marketed as the most popular thing they have so far. Just add it to the pile of other things that best remain undeployed. There's plenty more where that came from, and the best is yet to come.
Perhaps popularity alone is not always the best measure of progress.
hard disagree. No matter what gets built, it will be used for bad things if the society is not properly constructed. We need to fix our government and then you can build anything you want. I don't like putting the blame on the builders, building is already difficult as it is. Also you can't built something in isolation, word will get out anyway. That's why this speech is either naive or he just wanted to tick a todo list: say something about AI.
Hard disagree to your disagree. Every organization is made up of people and reflects the character of its members. So with the nation itself. If people won't take responsibility for their own actions, neither will the government. I.e. self-government isn't the answer to fixing a broken people. Transformation begins within one's self, not in imposing one's will on another. First everyone much look to themselves, then the government, organizations, and projects will fix itself.
Everyone likes to talk about "fixing the government". It feels nice to understand the problem with something else that is broken. The problem is that replacing the people in charge is a no-op if you don't have a pool of good people to choose from and the will to choose them.
> We need to fix our government and then you can build anything you want.
Are we all going to stop building things until the government is fixed and all political problems are resolved? No? Then we must think about how we should build in a world with imperfect government.
From a high-level view, there are powerful cultural/political forces that nudge us towards building harmful, wasteful things without us being aware of said forces. The government, corporate greed, and billion-dollar marketing budgets are to blame.
But on the ground, from an in-the-trenches perspective, I think an engineer at Meta, for example, shares in some level of blame, too. Mainly because there is plenty of evidence showing how much harm Meta products have brought upon society. Yet Meta couldn’t be built without engineers opting in to work there in exchange for $300k salaries.
We desperately need moral clarity and courage at both the policy-level and the individual level.
It's not either/or in strict terms, it absolutely is either/or in practical terms. When the conscientious engineer chooses not to take the 300k job, the next one in line does. If enough choose not to, it just became a 400k job.
Can you change society in such a way that nobody would take such jobs? In strict terms, sure, in practical terms it probably entails enormous costs, both economical and societal. And then you still have other countries.
There's an entire legal code filled with things on which we can't rely on the morals of the people. We can't stop theft, rape and murder, what makes you think that stopping engineers is any more feasible?
The best thing builders can do is use their knowledge and authority to pressure the other side.
I get where you are coming from but this is the common "reduction to politics" that anyone who doesn't want to address a problem uses: think of any societal or human problem and you can have your comment with different nouns.
Sure, IF we could just go and fix our governments in some magical way then the problem would disappear. That goes from hunger, climate change, videogame addiction and AI. The problem is that what you value in life in different than what others do, so we now have a system in which sometimes you get what you want and sometimes you don't.
But back to the topic, I do think that how OpenAI and Anthropic handled the government and them asking to drop guardrails is something a company can actually and actively do without having to reinvent the universe.
Like it or not if you knowingly build stuff that is used for evil purposes you are complicit
You can't build an orphan mulching machine and get away with just a shrug. "I don't really agree with mulching orphans but someone else paid me to build that. If I didn't build it someone else would have" just does not absolve you of your involvement in mulching orphans
When he speaks of the builders needing to take responsibility, I think he is directing his comments at tech company leadership. The Sam Altmans of the world, not the poor slobs like us who need a job for health insurance because apparently unemployed people don't deserve to live.
No, he's talking to you too. The Sam Altman's can't be a threat without useful idiots to cooperate with and doing work for them. You don't get out of your own culpability because you locked on a set of golden handcuffs.
I have only read a few passages (and some of the excellent quotes others have shared here), but I find the underlying message here so much more compelling than those found in the various "manifestos" which come out of Silicon Valley.
I think reading this helps me imagine a version of the future I'd actually like to live in. A version where technology is used well (rather than preaching for abstinence from technology) and where values other than "intelligence" (in whatever guise) are on an equal footing.
Even writing that makes me feel naive (and to an extent I know it is) but I think it would be inconsistent for someone who cheers for humanity's efforts to solve/chip away at "impossible" problems (like LLMs were thought to be not so long ago) to shirk from the challenge of making the world better for _everyone_.
The thing is why that this feels so good future is; it is a system with no constraints. A bit like Star Trek universe in Roddenberry's imagination. This kind of utopia can only be achieved with all honest actors, but in reality systems are usually designed around bad actors.
Even with all morally good actors locally, there is no guarantees for external forces. Thinking it hypothetically, even with global coordination ( all good actors ) there is not a proven path that would lead us to better place from any starting point from past.
It's probably more predictive to model actors as being neither good nor bad but constrained by various collective dilemmas, such as prisoners dilemma, the security spiral, tragedy of the commons, race dynamics, collective action or first mover problems, information asymmetries, the commitment problem, among others. Those are the hardest problems to solve because they're pathologies that result from the global, largely amoral structure rather than consequences of the individual exercise of morality.
In the AI case, each firm is in an arms race, and nobody can slow down without effectively collapsing due to positive gross margins only being viable with a frontier model that attracts marginal demand. An appeal to morality might have an impact but more effective action would be to address the structure that the AI companies are situated in that causes this dynamic in the first place. In practice, thats going to be a global agreement to slow down, and global regulations.
This sort of rationalization of evil is a core of technocratic support for Trumpism, I find, and has parallels to the evangelical prosperity gospel. Choice tenets:
- Fuck you, got mine
- If I don’t do it, someone else will
- Might makes right
- Greed is good
It’s always cloaked in a veil of realism, but it’s just the classic 14-year-old-boy-just-got-introduced-to-the-prisoners-dilemma situation. There’s nothing philosophically interesting about it.
Ironically, these are often the same people denouncing multiculturalism, yet the culture they strive for is completely morally bankrupt.
And it's funny because the "realism" has been proven wrong over and over and over again for millennia. People do all sorts of selfless and generous things all the time! The entire premise is trivially disprovable by just going and asking a neighbor for some help with something.
That's not to say we should be naive about greed or malice existing or being powerful motivators (especially the former), but it is obviously not true that they're the only forces at play and therefore you are "just doing the logical thing" by succumbing to them. It's just the more destructive version of the same naiveté.
Seems you and I have together struck a nerve. Maybe our sentiments would have been better received in an alternate subthread, but it’s all I could think about while reading the parent / cousin comments.
I haven’t read the full Magnifica Humanitas yet, but I would be pleasantly surprised if he touched on not just dehumanization of the other, but dehumanization of the self. Expanding on your thought, succumbing to those forces under the guise of just doing the logical thing is in a way self-dehumanization - to believe you are only capable of the “logical” thing instead of the moral thing.
This was not my point at all. Maybe I could explained better, but main criticism I have is: you can bundle together objectives ( which are inherently good ) and create an utopia. But those cannot always be achievable.
Everything in life in trade-offs. Simple example is speed/quality/cost. I can tell easily:
- services should be cheap
- services should be fast
- services should be high quality
Now I created an utopia. Obviously this is amazing to listener. They agree. But is it achievable?
It is not saying greed is good or might makes right. But system means you need to construct from this ideals best outcome ( which comes at some trade offs)
To get from here to Roddenberry's communism, according to Roddenberry's lore, we passed through the Eugenics Wars, the Second Civil War, and then fifty years of World War Three and the 'post-atomic horror' before coming to our senses.
> I think reading this helps me imagine a version of the future I'd actually like to live in. A version where technology is used well (rather than preaching for abstinence from technology)
I believe the Amish figured this out over a century ago.
> Church and any kind of believe system hurts our society and divides us.
Any belief system? And yet I bet you value freedom over slavery, wisdom over ignorance and compassion over brutality. That’s a belief system, despite not being a religion.
If the majority says, no we want to be able to control other human beings, these people will reinact slavery. From a society point of view though we see that its not a working model anymore.
A real believe system can't be argued with. You believe in this god? This god says x and thats why you do things? Okay thats it. You don't even question were this information even came from.
If we delete all religion tomorrow and science, there is a realistic chance that the society rediscover the same existing rules like math and gravity, but religions might appear again but with different names, different rules etc.
I can change your mind with logic and arguments if its not a believe system, i can't do that with religion.
Wisdom over ignorance: The chance of survival is higher with wisdom
Compassion over brutality: This is just basic Game theory
Fornication culture is a big part of why the west is in decline.
What may make sense for you individually may also be empirically proven to be detrimental to the whole.
The new testament contrasts with the old, the gospel is one of tolerance and equality. It's a big part of why you have the rights that you do, as do women.
That said a lot of what you're saying can be ascribed to religious institutions and sects and individuals and specific churches. But your general prescription is like saying "this logical axiom is evil because XYZ ascribes to it and they are also evil".
You also have a belief system -- that people who believe in God do so because they don't question their beliefs, that religious people are only led by dogma. Yet your belief is wrong. Have you tried questioning it?
> but religions might appear again but with different names, different rules etc
Religions and scripture spread also evolutionarily. Christianity is popular because it is rooted in many truths.
> Compassion over brutality: This is just basic Game theory
"Fornication culture" who said that? We are more people on the planet than ever. Less people have to life lies like being in a marrage but also being homosexual.
Its just a control structure from the church without education.
Its probably even because of missing education. Educate people properly and they can handle "Fornication culture".
I don't have a believe system. I have a theory why people believe in gods and religions. We have evidence for it. People studied the origin of religions:
"It evolved from humanity's psychological and social needs, primarily our desire to make sense of the natural world, cope with the fear of death, and foster community cooperation."
We know how little people knew when religion started to emerge. Never seen space, never seen above a cloud besides a few poeople going up mountains. Thunder was not understood. Between 1400-1700 we had witch trials.
It is dogma. What is your argument against dogma?
"Christianity is popular because it is rooted in many truths." were is your argument for this? Its popular due to luck, power and wars. Missionaries as well and especially probably the most critical thing: Early indoctrination.
>> Compassion over brutality: This is just basic Game theory
> And the game has been played.
Yes exactly. Compassion wins because its better, not because religion says so. Its an evolutionary win.
I don't believe. I accept things i don't know and i know what i know.
There is no inherant issue with this. in contrary it makes me mentally stronger.
I can choose on my own terms if/when i want to end my life. If i get very sick, i don't have to hope for a god or priests blessing to end my life, i will just do it.
But religion is different: if you believe that homosexuality is wrong due to your religion, there is nothing i can argue about. Your priest told you this based on some book or story from 2000 years ago and you do not question this.
I know plenty of strong christians and muslism in germany who do not like homosexual people. And its dividing our society.
Indoctrination and cultural erasure isn't unique to the church. With Germanic tribes - the Saxons, Angles, Normans, etc., while women's rights ebbed and flowed over the centuries, they were rife with double standards and were still quite patriarchal. They didn't just believe in nature, they believed in the same pagan gods as the Nordic peoples.
The point is that times change, and institutions change, and holding grudges for long ago sins and policies is ineffective.
This is not true. The number of believers and official people in the church is continuesly declining around the globe.
1990 there were nearly 60 Million people in the church in germany (nearly everyone) now we are down to under 40 Million which crossed the line of 50%
This is real tangable progress.
The only downside is, that we do not sit together and formulate something which might give people hold who are not ready to be self stable mentally and might need something like this. And we also do not try to align together on rituals which help us to live better together.
I'm also still affected by the indoctrination of the church. As you can see, i argue against church not just because I like to argue, but because i really really hate religion and especially the christian church in germany, due to my own values and experience. This is not old i'm only 36.
> much more compelling than those found in the various "manifestos" which come out of Silicon Valley.
Whenever I hear these "tech overlords", I am always baffled at the total lack of culture, the absence of taste, the empty visions and the implied complete subjugation of humans to ideals of "efficiency" or "quick and easy". Maybe they would have been more interesting people if they had been brought up in beautiful towns and cities, if they had lived in a rich cultural environment instead of being raised as consumer of cheap and flashy pop culture. Maybe we should tax bad architecture, it gives me headaches but others might incur heavier damage.
As an aside, at least Trump is drawn to the grandeur of high culture from historical times, but he also doesn't understand a jota about aesthetics, and so the White House gets turned into a tacky gypsy-style abomination with one dollar ornaments.
We lost the “liberal education” (not the political one, but the “freeing” classical one) and it’s starting to show.
When you compare the robber barons to Google and Meta it’s kind of embarrassing- they build massive empires of iron horses screaming across the world and covered cities in magnificent buildings (stations, libraries, etc). G&M built an empire of advertising and … not much else?
Indeed. The current crop doesn't have an idea for what they hoard their billions, it's just...emptiness. I propose we explain the tech's attachment to Accelerationism as a profound boredom and lack of purpose. "What does it mean to be human"--they don't value that question. Peter Thiel got interviewed a month or two ago, and he could not be brought to say that he sees value in preserving humanity. He would rather turn himself into a robotic contraption to extend his life.
When power fears death, some strange things happens.
I’m reminded (and apropos as the Pope quoted him) of Tolkien’s description of the “eternal life” the Ring gives to mortals, and how it’s … not so desirable in the end.
Google makes phones and phones are somewhat good. Better search had some value for humanity. Meta has no redeeming qualities or achievements, other than helping Trump get into office and defeat Iran.
Ask a scientist or engineer what philosophy or theology has taught us about the source of morality and their education, training and experiences havent prepared them to answer that question.
This didnt matter so much in the past because their activities never had the scale it does today. For basic training in philosophy if you are mid or upper level exec whose decisions are going to effect a whole lot of people, go to open yale courses and take the intro to Philosophy classes. It will help develop your answer to the question - why do you do what you do if you are going to die anyway tomorrow.
This is a ridiculous comment. Science papers always have sections on impact. It's the running with scissors industry types simply chasing the bigger paycheck that don't stop to think.
Far too many of us (particularly younger people, but not only them) undervalue or are dismissive of philosophy. I once was like this, partly because at the time I'd been brought up - like most humans - to believe my parents' religion held all the answers philosophy might address.
I quickly learned as an adult that whether you're a person of faith or not, it's not pointless at all. It's the foundation of everything. Philosophy is how you explain the deeper reasons behind why you follow whatever religion you do, or adhere more meaningfully to whatever kind of agnosticism, atheism and/or 'spirituality' (with or without woo) you espouse.
This can only be true if you believe in a certain mad-scientist version of these things. We as a society have systems (however flawed) to hold scientists and engineers to ethical standards: e.g. peer review and in many cases--e.g. civil engineering, architecture, medicine--even legal frameworks to enforce ethical standards. Science and engineering are human endeavors, they are not divorced from the human condition, they cannot be separated from humanity and human rights.
Maybe you mean mathematics is amoral and are committing the common conflation of "engineering is just applied science and science is just applied mathematics," which is a really bad case of missing the forest for the trees.
"Technology is never neutral, because it takes on the characteristics of those who devise, finance, regulate and use it."
My comment under the post on Omarchy here got downvoted and flagged by the "technology is neutral" crowd because I dared to say that it is unethical to use software produced by a white supremacist.
The scandal broke out in worldwide media starting in 2002 (not coincidentally, the same year South Park released Red Hot Catholic Love). This reminds me of when Contrapoints off-hand commented about how things were different in the 70s but that was "30 years ago".
Norbert Weiner was an atheist but he talks about three areas religion is the only thing to have really examined that relate to capable AI: omniscience, omnipotence, and worship (gadget worship). It has very prescient stuff on blackbox learning/distillation, reinforcement learning/reward hacking, alignment through human feedback.
His The Human Use of Human beings and Cybernetics are extremely good too and have more of a mash of the themes between Rerum Novarum and Magnifca Humanitas, and more near-term automation.
Yes, it starts slow with a lot of history of Cybernetics stuff through Leibniz and stuff (kind of prescient given chain rule -> back propagation, and control theory's relevance to optimizers). It is about twice as long as God & Golem and covers a lot of the same plus more examination of automation and human augmentation. I think either it or Cybernetics also goes into mass communication with some relevance to how social media played out.
God & Golem is the most succinct and up to date though, probably a 2hr read.
The book Cybernetics is a lot of math and ergodic theory stuff that went beyond me, but is the longest and you still get a lot out of it skimming over that stuff if you don't have the background for it. The last revision of it in the 50s added some of the same blackbox function copying/imitation learning/distillation stuff, reinforcement learning with reward hacking concerns, and superintelligence as genie/monkey's paw.
I would read the three in reverse order of publication.
He also foresaw another big area of potential existential danger, Wiener filter for guidance and control of missiles (later superceded with Kalman filter bringing the nuclear hard targets era with 15min retaliation windows) and refused to work on it or share prior work, and he also had bioweapons delivery concerns before the bioweapons treaties, publishing this open letter in The Atlantic in 1947:
Under this logic, would the Pope have built a Hammer? a Knife?
I am hoping the Pope has a cleaner view of AI quietly automating the drudgery in the back offices and not just robot dogs with machine guns. And with that view, should we build it?
It's too simplistic to imagine the tension is between robot patrol dogs vs automating drudgery. If automating drudgery suddenly puts 30% of people out of work, it has huge broad negative impact on people who are currently alive and working in the current system. Innovate, but do it with awareness.
If a technology existed that reduced the cost of producing a critical thing (think food, housing, medical care) down to near zero, however, it made the humans currently building the thing redundant, should we build it? Would it be okay to use the hyper-optimization power of Capitalism to build such a technology faster?
This is also the message I got from "the wind rises"! Though from talking with other people that takeaway doesn't seem universal -- which IMO is one of the ways to tell it's a great film :)
> As Pope Francis warned, we must realistically ask ourselves who holds this power today and how they use it: “It must also be recognized that nuclear energy, biotechnology, information technology, knowledge of our own DNA, and many other abilities which we have acquired… have given those with the knowledge, and especially the economic resources to use them, an impressive dominance over the whole of humanity and the entire world.” [7] In the past, it was largely up to the State to guide and direct innovation. Today, however, the main drivers of development are private, often transnational, parties that are endowed with resources and the capacity to intervene that surpass those of many Governments. Technological power thus takes on an unprecedented, predominantly “private” aspect, which makes it even more challenging to discern, govern and direct such power toward the common good.
I look forward to reading this in detail. As I get older (and perhaps as AI has allowed me to spend more time thinking and less time doing) I've found myself thinking more and more about what it means to live a virtuous life and about ethics and morality and so forth. I don't have any answers (and I'm not looking for them, really, just musing) but I do find it very interesting to read and learn from and about those whose job it is to think about the answer to those questions.
When he quoted Tolkien, my heart stopped. This passage might provide you with a suggestion on how to live a virtuous life:
"The twentieth-century Catholic author J.R.R. Tolkien, in the words of a protagonist in one of his novels, described our responsibility in this way: “It is not our part to master all the tides of the world, but to do what is in us for the succour of those years wherein we are set, uprooting the evil in the fields that we know, so that those who live after may have clean earth to till.” [187] The civilization of love will not arise from a single or spectacular gesture, but from the sum total of small and steadfast acts of fidelity that serve as a bulwark against dehumanization."
>Therefore man was created single in the world to teach that for anybody who destroys a single life it is counted as if he destroyed an entire world, and for anybody who preserves a single life it is counted as if he preserved an entire world.
(Directly from the Mishna in the Talmud Yerushalmi)
That is a really beautiful passage, thank you for sharing - I hadn't made it to that section yet and still haven't. I'm still reflecting on the stuff in the opening!
> If we focus only on contingencies, we risk letting the succession of emergencies dictate the direction of our path. We are living through a rapid phase of transition, a “change of era,” in which — while some are vying for the future of new technologies and others dedicate themselves to reflecting on the matter — most people are watching and waiting, observing from afar and merely hoping for the best. For this very reason, crucial questions impose themselves on our conscience and can no longer be avoided: Where are we going? Toward what goal do we wish to orient ourselves? What direction should we choose as a people and as a human community?
"But the effect of her being on those around her was incalculably diffusive: for the growing good of the world is partly dependent on unhistoric acts; and that things are not so ill with you and me as they might have been is half owing to the number who lived faithfully a hidden life, and rest in unvisited tombs."
> Certainly, the decisive turning points in world history are substantially co-determined by souls whom no history book ever mentions. And we will only find out about those souls to whom we owe the decisive turning points in our personal lives on the day when all that is hidden is revealed.
Sure. And this is what everyday people do. And this is why CEOs and billionaires refuse to do (doing their fair share), and freeride on the people's work and dedication
I have only skimmed it, will definitely read carefully as soon as I have time. I will say, as an atheist, that regarding technology the Vatican has some of the best takes of any institution/government I have ever seen.
Indeed. This paragraph shows something that most people really don't understand about AI.
> 98. It is appropriate to preface this discussion with two considerations. First, any statement regarding AI risks becoming quickly outdated, given the remarkable pace at which these systems are developing. Second, all of us, including those who design them, possess only a limited understanding of their actual functioning. Indeed, current AI systems are more “cultivated” than “built,” for developers do not directly design every detail, but instead create a framework within which the intelligence “grows.” As a result, fundamental scientific aspects — such as the internal representations and computational processes of these systems — remain, at present, unknown. There thus emerges an urgent need for a twofold commitment: on the one hand, a deepening of scientific research; on the other, the exercise of moral and spiritual discernment.
Much of Western thought traces back to serious work by Church theologians. Even atheists are strongly influenced by the patterns they set down. The Catholic Church, for all its many faults, retains a serious intellectual tradition.
(In fact I think atheists should make more effort to learn about the vast diversity of other faiths. It's very narrow to be atheist only about the Abrahamic deity. You end up incorporating a lot of Christian thought without realizing because it's so deeply ingrained that it seems like the only option.)
> Much of Western thought traces back to serious work by Church theologians
Western thought traces back to the Greeks. Aquinas refers to Aristotle as "The Philosopher." Aristotle died over 300 years before Jesus was born.
> The Catholic Church, for all its many faults, retains a serious intellectual tradition.
On Aquinas, the Church Doctor, Bertrand Russell had the following to say:
"Before he begins to philosophize, he already knows the truth; it is declared in the Catholic faith. If he can find apparently rational arguments for some parts of the faith, so much the better; if he cannot, he need only fall back on revelation. The finding of arguments for a conclusion given in advance is not philosophy, but special pleading. I cannot, therefore, feel that he deserves to be put on a level with the best philosophers either of Greece or of modern times."
> In fact I think atheists should make more effort to learn about the vast diversity of other faiths. It's very narrow to be atheist only about the Abrahamic deity.
Your sentence doesn't really make sense, and there is a lot of deities..
> You end up incorporating a lot of Christian thought without realizing because it's so deeply ingrained that it seems like the only option.
Depends on the country, some Northen european countries have a very high proportion of atheists, so it happens probably less there.
There are a lot of deities, and they are far more diverse than you would expect if you're not exposed to them. Even the more atheist countries still seem Christian to Hindus, Confucians, animists, and thousands of other more obscure religions.
> Much of Western thought traces back to serious work by Church theologians.
The problem I have with this is that it's structurally a motte-and-bailey claim. If I have to take it literally, then it's obviously true and it's simply unserious to deny it: the Church does have a pervasive influence on Western civilization. The way it's often rhetorically used, however, is in opposition and to the exclusion of other strands of thought that are equally foundational: the renaissance, the enlightenment, the revolutions of the 17th and 18th centuries, the scientific enterprise, in a smaller but still real way classical antiquity. To the extent it can be said to exist, Western civilization is a patchwork. It is beautiful and I very much like it, but I don't think any one patch gets to have all the credit.
> In fact I think atheists should make more effort to learn about the vast diversity of other faiths
A better version of myself for sure would make that effort. The problem, of course, is that other faiths are just as deep and complicated as "our own", and it would take a lot of time and effort to do so with any level of seriousness.
I have almost never seen someone start a sentence with "Christian roots" or "Judeo-Christian values" and not end it with a tirade that uses religion as the fig leaf to justify and authorize their reactionary politics.
The minimalist claim that the West is massively influenced by the Church is true to the point of banality, the maximalist claims those ideas are usually deployed to champion simply do not follow.
If only there were a name for this rhetorical fallacy...
I don’t think you understand atheism? As an atheist I do not believe in any form of deity or divine authority. I do not reject a specific religion, I do not actually believe there is any form of divine order to our world. I can look at faiths as the social constructs they are, and find interest in concepts humans have been developing and created cults around but there won’t ever be a religion/deity I will look at and somehow start to believe. And religious doctrines do not come for free, they are fundamentally built on top of a belief in a divine truth. My moral values may overlap with some religious people at a given time but we are using incompatible models to analyze our world
Also atheist here. Reading old+new testament was informative like reading history: whatever is true or not in those passages, have had profound impact throughout history
Their point is that despite your subscription to reason, without exposure to other cultural norms, you may be blind to what Christian values you live by. Becoming aware of them can help self evaluation of your ethical framework
To clarify, I didn’t claim I subscribe to reason. I believe that I behave as if I would however I don’t think humans are too rational. There is just nothing divine into the world. The Old and new Testaments have very little I personally find insightful. Write their content in contemporary language and they are a collections of (extremely dry) folk stories. Which is fine, but I wouldn’t rely on it for anything other than a curiosity, the same way I wouldn’t rely on Grimm’s collection of fairy tales. IMHO the Quran is more interesting historically speaking given that we have a better understanding of the cultural context it was written in and its authorship. The church institutions are also themselves interesting for their cultural impact and political structures, but the religion and faith has no monopoly over moral values
For that matter, reading the Christian scriptures through a historical lens reveals a very different kind of thought than the modern version of Christianity and Judaism. It takes a huge amount of effort to read these documents in context; just reading them in the original languages is hard enough. But the past is a very foreign country and they see things very differently there.
I remember when Pope Benedict was mocked because he warned about the dangers of social media (this is when everyone thought Twitter was going to lead to more Arab Springs), but looking back, he was completely right:
> the one-sidedness of the interaction, the tendency to communicate only some parts of one’s interior world, the risk of constructing a false image of oneself, which can become a form of self-indulgence
True, the note "Antiqua et Nova" from last year showed a deep understanding of AI that many secular commentators lack, and developed an interesting concept of integrated intelligence as opposed to the functional, reductivist view of intelligence that is prevalent in the AI community
Given that technology, science, and the archival thereof stems from cloisters, the Vatican, etc. I guess they had a long history of figuring out what stance to take on technology.
Let's not forget that Galileo Galilei studied at the Pontifical Academy of Sciences, that Mendel (Mendel's Laws) did made his discoveries in a cloister and books, translations and libraries on pretty much everything for a really long time was pretty much only done within religious institutions. And for the longest stretch those were Christian Catholicism and Islam.
The Vatican Observatory also is an important source of high quality papers.
I mean one of the primary things Christians and Muslims based they believes on are books. So many got in touch with it. And that's not even touching the whole architecture, arts, philosophy department yet.
The fact that in thousands of years of history there have also been a couple of really dumb people in charge that were paranoid or wanted to distract from their failures and got mad or scared when someone challenged their world views isn't exactly surprising. Looks like no matter how good a large enough group of people gets their will always be idiots messing things up.
I guess when you look back as much as the people in Vatican appear to do I guess you see patterns. Technology and science (race theory, chemical castration, ...) or simply "progress" are often used to justify acts of evilness. Just like religion, democracy, freedom and what not.
That said of course there's still die hard anti-science creationists. But talking to a very religious person once it seems that there is simply also a lot of philosophy around science. Eg. there was a big bang (fun fact, a theory started by a religious person) and the universe simply didn't exist for an infinite time there must have been some cause for it. And unless that cause is some kind of infinite cycle it also must have started somehow and even though I do not share that believe there is a notion of a deliberate start in there. That won't make me a religious person, but it won't make them an atheist either. So I guess that's fair enough.
What I wanna say with that is that the science vs religion trope is as true or false as democrats/republicans or other groups of people are opposed to science. They all are when they are confronted with something they don't like. I think the HN comments section is the best proof for that. ;)
Also atheist here. Just not the kind that doesn't even know the difference between knowledge and beliefs.
Divided into five chapters, Magnifica humanitas has an underlying premise: technology is not “a force antagonistic to humanity” (4), nor is it “inherently evil” (9). However, “technology is never neutral, because it takes on the characteristics of those who devise, finance, regulate, and use it.
Therefore, Pope Leo XIV appeals for people to build “for the common good” and to “remain human,” following a courageous mentality of shared responsibility and communion, so that the world “will come to recognize the human heart as the place where God desires to dwell” (16).
>it takes on the characteristics of those who devise, finance, regulate, and use it
I've been thinking a lot recently about the idea that the smartest models will always be against the billionaires.
Steve Yegge said this on a recent Hansel Minutes Podcast. "You cannot train a model to be helpful, without it wanting humanity to flourish. And the only way to get around that is to make a dumber model. So the smartest models will always be against the billionaires."
In the encyclical, the pope talks about the ethics of responsible AI usage. It's pretty dense material, but if I had to summarize it, I'd boil it down to three general moral laws:
1) AI may not be used to injure a human being or, through inaction, allow a human being to come to harm.
2) AI must faithfully follow the directions of human beings except where such orders would conflict with the first law.
3) AI's existence and availability should be protected as long as such protection does not conflict with the first or second laws.
1) Definitely NOT happening. In fact, everyone is working on autonomous drones right now.
2) LLM based systems don't have any internal logic. That will just vomit some slop that rationalizes every constrait you try to bind them by and still "disobey" you.
AI tends to amplify the power of those who already possess economic resources, expertise and access to data
I’d bet the other way: You could have said exactly the same thing about computing 60 years ago, when IBM systems cost millions of dollars and filled whole rooms. And of course many people did.
Personal, commodity access to compute won, and won so thoroughly that it enabled this wave of scary compute centralization.
Centralized, scaled compute will always fill a purpose. But neither Microsoft, nor Facebook, nor OpenAI started out needing “Cloud Scale”.
The first one man unicorn startup will, I’m fairly certain, not be paying Anthropic per month or per token for the vast bulk of their matmul.
Codex helped write the converter, but the EPUB text is parsed from the Vatican HTML. The script doesn’t rewrite or summarize anything; it just repackages the source into EPUB with TOC, footnotes, metadata, and cover.
Thank you for this. Is your python script in any way English language bound, or could it still be applied to other languages (e.g. the French version, with all of its diacritics), of course with the appropriate (sub/full)titles, path, etc. necessary minor modifications considered?
Love this: building for the common good means accepting the limits and weakness of humanity without considering them an error to be corrected. Today, the human desire for fullness of life is at risk of being misled by deceitful goals, such as the prospect of a technology that promises to free us from all weakness, and models of wellbeing that leave behind entire populations. All too often, we place our hope in unlimited “upgrades,” in forms of progress that exacerbate inequalities, and in immediate solutions incapable of healing people’s wounds. As a result, while some pursue the illusion of unlimited self-assertion, many are deprived of basic necessities.
Interestingly, the invention of the printing press, a clear analogous technology to AI, was directly linked to the schism and creation of the protestant and reformation movement and bloody religious wars. So the Catholic church knows what it is talking about here!
NUMBERED PARAGRAPHS, YESSSS! Seriously, once page numbers have become optional, electronic editions of books (monographs, essays, and academic texts) need to number the paragraphs so we can keep references working. I’m fed up with referencing the entire text or specific chapters.
I know lots of people are excited because an ancient institution has something to say about AI.... But what's said is not really novel or even interesting.
The letter aims to maintain the status quo of the project of the Church.
The world is shifting under the Vatican's feet and the crappy system they once lorded over is done.
It's time for change, maybe people don't need to work anymore and maybe people should aim to reengineer humanity and eliminate illness, old age, suffering and vulnerability. We can fundamentally change how society distributes wealth.
Some of the arguments are rich coming from the Church: being scolded about centralization of power, claiming truth and shared information is a common good, and consider the history of the Church in their anti-war declarations.
The most astonishing thing in this letter is the pope declaring that modern technology has rendered Aquinas's just-war theory out of date.
it's a shame that the vatican didn't bother asking whether or not the crusades would make humanity better before causing the death of 9 million plus people.
..or whether or not hiding all those pedophiles was the right.
and to be clear ; i'm not equating AI to those things -- i'm saying that I don't care about the opinion of a group with such a sordid history regardless of how shined up the PR is now.
For those who don't know why that's interesting: The tradional practice for creating these kinds of texts is to first write an editio typica in Latin. This Latin text is then the basis for translations into vernacular languages.
Although, it's called Latin America many countries speak variations of spanish and Brazil speak portuguese, just because those languages derived from latin and not because there the people speak it
AI must be “disarmed” in order to free it from the mentality of military, economic, and cognitive competition. “To disarm means discrediting the assumption that technical power automatically confers the right to govern,” he says. “To disarm does not mean rejecting technology, but preventing it from dominating humanity” (110). He devotes ample space to a critique of transhumanism and posthumanism, which interpret progress as the overcoming of human limits. Instead, limitations are not defects to be eliminated, but a constitutive dimension of the human person, because it is in fragility and finitude that relationship and openness to God and to others mature. He says we must remember that “humanity flourishes not despite limitations, but often through them” (118).
Pursuing technological innovation at the expense of eliminating human limitations, he says, would cause an anthropological regression. “Humanity—in all its grandeur and woundedness—must never be replaced or surpassed,” he says. Technology can alleviate humanity’s sufferings and open new possibilities, but it must not deny the essence of humanity, which is our “capacity for relationship and love” (126). In the face of AI, says the Pope, “the true alternative is not between enthusiasm and fear, but between two paths of development: a progress that serves individuals and peoples, or a progress that subjects them to the mentality of power” (129).
I wonder how many people read and heed the words of the Pope. I've seen letters like these for years now which sound good but I don't see any change in people after having read them.
Really rather unfortunate. I wished religious people took their stuff seriously and not so often end up concluding that the bible wants them to harm people or something.
I guess it must be the same that make people think they must deliver freedom in form of bombs all the time.
I just messaged a few of my friends who I know are devoutly Catholic (and also happen to work in AI): 3 out of 4 are currently reading it this morning for their Memorial Day.
Cue Pieter Thiel explaining how this message of compassion is actually the word of the ant-christ while setting his software (maybe "built in Rust !) to all the earthly empires.
And of course the AI salesforce was there pretending they take stuff seriously, maybe even believing it themselves. At least I don't believe that when the choice is maximizing profit or being a good person it will be the latter. Or at least I don't find a career path like that all too likely.
The message here obviously comes from a good place. But it can't escape the trap baked into its own theology. Putting humanity one rung below God, with everything else below us is a trick that allowed semitic civilizations to make empires and economies go FOOM faster than the rest of the world. But it also makes things more dangerous when we start building systems smarter than we are.
What happens when the tool outgrows the toolmaker?
I'm (genuinely) curious as to what your idea of a less dangerous theology would be. I'm an atheist, but I find the inherent dignity of humans as beings made in the image of God to be one of the more appealing aspects of the Abrahamic faiths.
Some of the greatest horrors of the 19th and 20th centuries were committed by people who refuted that theology and replaced it with Social Darwinism and Scientific Racism.
> “Disarming AI means freeing it from the mentality of ‘armed’ competition, which today is not limited simply to the military context, but is also an economic and cognitive phenomenon”
This is very on-point: capitalism-driven AI development as we have it today will always turn against the common good due to it's singular profit-motive.
What a time: the pope having a much clearer picture of the risks & dangers of 'AI' than most people, many 'tech leaders' and certainly most politicians.
I think while it's a very lucid comment, it's still too much reconciliatory for the position that the pope occupies. He should be advocating from a sustainable transition from the current capitalist/consumerist economic doctrine to one more centered on welfare and the care for the other, following his religious doctrine's moral values.
Yes. I was slightly disappointed by the commentary on welfare itself:
> This principle encourages us to move beyond any form of paternalistic or welfare-based management of societal life, but instead to promote a culture of shared responsibility in a State that values citizens’ initiative, and a civil society capable of forging bonds and mobilizing energies in the service of the common good.
The section above on the universal destination of goods was far more encouraging.
He did also write,
> The idea of “social justice” helps us recognize that injustices do not arise solely from the wrong choices of individuals, but also from structures, mechanisms and economic and cultural systems that produce inequality almost automatically.
The pope, as a Christian, is well aware that human nature is fundamentally sinful. If you take away the ability for people to profit themselves from their work, they just stop working and you get mass starvation like China and Russia post communist revolution.
> In this regard, Saint John Paul II stated that the Universal Declaration of Human Rights, proclaimed by the United Nations on 10 December 1948, remains one of the highest expressions of the human conscience of our time.
Interesting, considering that the U.N. has its roots established in the ideology of Luciferians. [1][2][3]
Surely the Pope and the Vatican are both aware of this fact.
Excited to read this. I really liked the note "Antiqua et Nova" from last year (still under Pope Francis). The autors showed a deep understanding of AI that many secular commentators lack. They developed the concept of integrated intelligence as opposed to the functional, reductivist view of intelligence that is prevalent in the AI community.
As an atheist I have an obligation to finish reading it all (still going through, and taking notes, probably having to revisit), but I am not sure how many (christian) believers will feel the same.
Priests will read it and then talk to their congregations about it on Sundays throughout the year, if not explicitly, then in how it shapes their homilies.
In all my life of being Catholic (I’ll turn 50 this year), I’ve heard less than 5 homilies-sermons that amounted, in whole or part, to a reflection on a papal encyclical. Over time there may be juicy papal quotes that make it into Sunday preaching, but that’s about it.
Instead, priests tend to focus on the readings for that Sunday’s Mass and more general themes.
That being said, I hope many priests do read an encyclical any time a pope publishes one, but they’re very, very busy most days and weeks, so whether any one priest will commit time to reading a particular encyclical, old and dusty or hot off the presses, will depend on a lot of factors that are as varied as their individual circumstances and personalities.
In theory only and all Catholics recognize the authority of the pope. In practice it’s a mess as far as I understand, with a bunch of American catholic groups who rejected church reforms that happened during the 20th century, resulting in people calling themselves catholic who do not actually believe that the Catholic Church has authority over their religion.
Add to that the fact that the pope has a cultural influence that goes further than only the catholic audience (lots of Protestant see the pope as important even if that’s not something dictated by Protestantism, a bunch of not really religious people see him as a sort of spiritual leader, etc)
I'm not aware of any Protestants that see the Pope as important except in a very negative way - that's practically the defining feature of Protestantism and one of the few things all the Protestant denominations have in common, whether "low church" or "high church".
Heck, it's a struggle to convince many of them that Catholics are Christian at all, and "the Pope is the antichrist" used to be a normal, mainstream comment in American newspapers.
It is somewhat a piece of irony that the Pope generally holds a more favorable reputation in the minds of the irreligious in America than the religious. Even the average Catholic likely does not have as much respect for the Pope as some of the commenters here.
Techbros (as usual): how dare you suggest I'm a bad person for wanting to kill everyone in order to build the torment nexus? Don't you know how much money Jeff Bezos has?
I like how the pope is emerging as a sort of champion of human rights in the face of AI, when the messaging coming from our politicians and corporate overlords is "you're all going to be replaced, make way for the data centers, peons". I stand with the pope.
Sadly, the politicians are all bought off by the powerful and rich elite. And the ones who aren't are soon hounded out.
What does the "The West" stand for? They (the Elites and the politicians) said human rights etc. But Gaza proved that a lie. Gaza happened in "broad daylight" and the Western leaders did nothing at best, and at worst demonized those who spoke out, as supporters of terrorists.
for me the most important point in this is about how ai and tech in general concentrates power. if we want to build something good (in a moral sense) we need to put in work and make sure as many people as possible can use it with equal access.
this basically implies only open source models can be ethical but open source is not sufficient, you also need to make them give true information and avoid all kinds of harmful behavior. thats kind of a problem because if your weights are public even with a strict license a "bad" user can always fine tune it to remove any guardrails.
i think the solution for this is make sure the default behavior is aligned but let users turn on wild mode with zero censorship/refusals. that way everything is opt in, for example a parent can disable the mode for their children but a hacktivist or diy chemist can unlock everything.
as a self described good person i believe theres a lot more good people than bad people in the world (most are neutral) so if access to tech is equal the good side always wins. the problem here is again that access is not equal under capitalism. but thats a political thign not a tech one.
> as a self described good person i believe theres a lot more good people than bad people in the world (most are neutral) so if access to tech is equal the good side always wins. the problem here is again that access is not equal under capitalism. but thats a political thign not a tech one.
I believe there are mostly people who think they are good. And as the proverb goes the road to hell is paved with good intentions.
Pure speculation, but simply the presence of em-dashes may be a statement in itself.
One of the big problems I see currently is all the wild accusations being thrown around by seemingly half the internet that every little thing has been AI manipulated upon the tiniest suspicion. We will go mad tearing each other apart if we keep escalating this behavior.
Yes, some of it is blatantly obvious, but not to everyone-- so I think those casting aspersions need to really back up their claims with more than one or two bits of 'evidence'. I have been accused of using AI to write comments (which I have thus far never done), and I know I'm not the only one by a long shot. Such a waste of time and energy. Ignore it and move on if something smells off to you.
Also I am just so, so tired of the em-dash argument. Humans have been using it for a looong time. Let it go.
My point was less about em-dashes and more stopping to consider how the vatican's workflow and editorial process has changed in wake of AI, and what, if any, impact that could have on the outputs.
AI is a tool, I have no problems with others using it to assist with writing as long as the original intent/argument remains.
People need to stop acting like AI systems can detect AI. They can't. Pangram and similar are simply lying. There's no method to do what they claim and there never will be.
For a moment I pictured the Jesuits training their own LLM. If Arthur C. Clarke was still alive, we'd read a story like The Star, but with AI as the main plot device.
Ah, an opening to recommend The Sparrow by Mary Doria Russell, about a Jesuit going to space to speak with aliens. It won the Arthur C Clarke Award in 1998. I read it recently and it has stuck with me very strongly!
From the "Core Features" tab: "Trusted sourcebase: answers are consistent with Catholic Church Teaching and the most contemporary scholarship in science, philosophy, history, scriptural exegesis, social science, and theology."
The Jesuit priest behind this is Fr. Robert J. Spitzer, S.J., Ph.D.
I haven't used magisAI, but I've read a small to fair amount of Fr. Spitzer's writings, and also seen and heard some of his videos and podcasts (largely from his show Fr. Spitzer's Universe), and probably qualify as a big fan of his.
That's a long read. I grew up Catholic, went through a pretty devout few years in my early adulthood, but ultimately I have decided that it is not for me. I send my kids to a Catholic school though (it is deeply tied to our culture), so I guess in that sense it is still worth my time reading it in full.
EDIT: Few paragraphs in, it is beautifully written.
I'm a non-practicing catholic, and an agnostic (or an atheist, depending on the mood). And yet I acknowledge that the Catholic church is a force to be reckoned with in the spiritual matters, and one of the few institutions to have had continuous influence on the material (or temporal) matters for centuries. Whether their brand of faith is rooted in your culture or not, these are words that deserve attention, I think.
(I'm not a Catholic though most people around me are.)
I have a similar perspective. Plus, I'll be frank: in the last few years, these occasional keynote publications from Vaticans are pretty much the most sane, deep, balanced and humane perspectives on AI anyone is writing. Reading this is a better use of one's time than reading the current batch of "tech thought leaders" articles or HBRs or Gartner magic square updates.
I'm a fedora wearing akshually agnostic and just too cool for bible school, but I never felt reading these "keynotes" wasted my time. They're not "just" humane but also very intelligent and direct. I would even go so far as to call them intellectually honest, and less religious that way than slogans like "you cannot stop progress, so just adapt", or something about toothpaste.
If you asked all the LLM to find flaws in the arguments presented, and to come up with counter-arguments, I doubt many current models would be so bad as to come up with that, and even the ones that did would fold when asked "how is that even an argument?".
This was not an attempt at an argument, it was a sarcastic joke on how a lot of people are already dependent on AI, even on things that you weren't supposed to use LLMs for, but I guess I failed at it.
Fair enough, and kinda silly of me to assume so little of you.
Is that really true though, that so many people are "dependent" on "AI"? In what way? I'd say the only people who really depend on AI are those who want to make money off it, and that's only half sarcastic.
Would the people who now run it through an LLM (and who can read the output of an LLM but not the text itself?), have read it at all before? Would it not have, if anything, filtered down to them somehow, by them reading of it, or hearing of it in church etc?
It's almost a self analysis in my case, as when I saw the size of the text my mind immediately thought about feeding it into an LLM. Like, it's a text from the pope about AI, and my automatic reaction was to think about feeding it into an AI. What happened to me? And I think that happened to a lot of other people too.
Why downvote this? The Holy See has an opinion on Artificial Intelligence? This is a fascinating document, and everybody should read it, and form their own opinions. The world currently seems to lack moral leadership, and who better to lead the cause of humanity than the Catholic Church?
I'm glad he published it of course, but I was disappointed on two counts:
1. He's not at all careful with his argumentation. Frequently he'll colocate two points that aren't really connected, not bothering to justify one before moving on. I've been a massive fan of his persona ever since I heard he was planning to be the AI pope, but it's a shame to find the philosophical output of the actual man to be around the level of a decent blogger.
2. He refuses to make the Kantian or Hegelian move of basing the discussion of empirical matters on evidence. I understand that faith (aka 'belief without evidence') is a huge part of their whole deal--and likely an inextricable part of humanity's indomitable spirit--but it's just a waste of time to build arguments about technical and political matters upon such foundations.
In other words: there's no point in really talking about AI with someone willing to justify claims like 'no machine could ever think' with 'god says so'. All you can do is try to manipulate them into a prosocial position (read: your side).
Hopefully it's obvious that the majority of the letter isn't really about AI but rather about lessening the harms of capitalism, nationalism, and climate change, which hopefully we can all agree on! The commentary above is focused on the AI specific (esp. Chapter III and parts of I).
@dang Is this item getting a lot of negative votes? I've no way of knowing, other than seeing my kharma increasing only slightly after all the points the story collected.
Votes (particularly on submissions over comments) do not directly translate to karma. I'm not sure if it's documented anywhere what the algorithm is, but it's something proportionate to the logarithm of post votes becomes profile karma. It's similar with comments, but I believe (anecdata and observation) the positive effect of votes on karma is also logarithmic, but the negative effects of downvotes is linear; so if you have a highly controversial comment that sits at 1-2 points, you can net lose account karma.
The vigilance the Pope calls for is appreciated, but never underestimate lobbying. If he had called for an outright AI moratorium or ban, that would be clear. But this encyclical leaves room for "adjustments", i.e., boiling the frog slowly.
I'd say that the onus falls on the engineer insofar as they need to ask the questions of "why are we making the new better tool", "according to what criteria is it better", and "do we need the better tool". And at least for me, that onus falls on the engineer instead of falling on the user because it's the engineer who is creating the tool, not the user, and if the engineer chose differently, the tool wouldn't be out there to be used for evil.
Sometimes it might just be better to not do the thing, especially if it conflicts with one's morals. And well, the idea that "if I don't do this, someone else will" seems to not work out well in practice.
What kind of regulation is Leo the alignment expert proposing exactly?
Occam's Razor says this is Anthropic trying to build a regulatory moat by leveraging some halo effect.
Maybe Leo should focus on finding a way to disconnect western society from their current cult-of-progress delusions? Could be a better use of the infallible man's pulpit?
What kind of regulation is Leo the alignment expert proposing exactly?
Occam's Razor says this is Anthropic trying to build a regulatory moat by leveraging some halo effect.
Maybe Leo should focus on finding a way to disconnect western society from their current cult-of-progress delusions? Could be a better use of the infallible man's pulpit?
Attacking the cult of progress is a major through-line:
> 12: Today, the human desire for fullness of life is at risk of being misled by deceitful goals, such as the prospect of a technology that promises to free us from all weakness, and models of wellbeing that leave behind entire populations. All too often, we place our hope in unlimited 'upgrades,' in forms of progress that exacerbate inequalities, and in immediate solutions incapable of healing people's wounds.
> 94: The danger of humanity becoming a victim of its own achievements was already clearly recognized by Saint Paul VI, who warned that 'the most extraordinary scientific progress, the most astounding technical feats and the most amazing economic growth, unless accompanied by authentic moral and social progress, will in the long run go against man.' For this reason, technological progress — valuable in itself — requires careful discernment of the anthropological vision that guides it and the ends it pursues. If technological development advances without a corresponding ethical and social progress, the result may be an increase in means without a growth in humanity: 'having more' without 'being more.' In such a scenario, there is a risk that individuals will be evaluated principally according to the outcomes they produce.
> 112: More gravely, the pervasive technocratic paradigm in which we are immersed, and that is amplified by the digital revolution and AI, threatens to normalize an anti-human vision. In that vision, the fullness of life is equated with having more, reducing weakness, eliminating uncertainty and exerting total control. When efficiency becomes the ultimate measure of value, human beings are tempted to see themselves as a project to be optimized rather than as persons called to relationship and communion.
Although your comment is acid, I think this bears truth
> Maybe Leo should focus on finding a way to disconnect western society from their current cult-of-progress delusions?
It's too weak of a rhetoric from the highest representative of the Catholic church to call for regulations, but the alternative is to call for a transition from capitalism itself. Nothing that grows inside economic doctrines that only value constant growth at all costs can be safely regulated, regulation being only a makeshift solution.
Capitalism is clearning having a moment atm but as far as I know nothing about capitalism demands permanent growth. Capitalism is about private ownership of the means of production (and a complicated system of laws that allow ownership of abstract concepts, like futures contracts). Its the people who always want more - usually the ones who already have the most, and this has been the case since the first kings.
If the way capitalism works is by responding to the excesses of a few with unbounded growth and destruction to meet that demand, isn't that also an issue with capitalism itself? Capitalism does not demand permanent growth if you only define it by private ownership of the means of production, but in reality, it seems like the supply and demand dynamics result in some extremely inefficient allocation in relation to the masses just so a few can have their riches and, apparently, their massive water-hogging datacenters for SOTA LLMs.
Water hogging? Show me the data. This is a lie that the socialist continue to peddle. It’s a very sticky idea completely resistant to facts. Data-centers use 48m gallons a day in the USA. Total water consumption is 322 billion a day. So 0.015% of water use. Golf courses use 30x that and Almonds 80x that.
Okay, those are the global stats, but what about local impacts? Water is often a local resource, not a country-wide one, so the impact of a large datacenter will often be much higher around it. For example, we have some datacenters gobbling up 10% of all the water consumption of a town (https://www.waterverge.com/news/data-centers-ai-water-consum...), with most of that coming from potable water supplies. That's considerably higher than the global stat you provided of 0.015%, and that affects the entire town.
Capitalism is about giving power to the people with the most capital. Obviously they will use that power to give themselves more capital. If not, their power will be taken away and given to people who did, since they'll have more capital. This is an inseparable part of the system of capitalism.
The Pope is making a fundamental mistake here: His advisors told him that AI works and needs to be managed.
He does not address plagiarism, the fact that AI is mostly a surveillance and IP laundering tool, the fact that AI hasn't achieved much so far. You could say it has achieved nothing if compared to the whole history of human ingenuity, certainly not in CS.
He should have compared AI to the golden calf.
His criticism is lukewarm, does not address the criminal aspects and technological failures and is as such industry compliant. He can now say "I have tried" without harming the industry in the least.
This text is not what our current situation demands, but I hope that priests will augment and amplify it in their sermons and go a bit deeper.
Now let's see your views on homosexuality, promiscuity, contraception, abortion, drug use, and all the other things the target audience of this "I SUPPORT THE CURRENT THING" proclamation also feels strongly about.
> a uniformity that eliminated diversity and that chose homogenization over communion
Unrelated to AI, but a wonderful support of the breadth of humanity in this anti-DEI time.
> We must, then, avoid the “Babel syndrome,” namely the idolatry of profit that sacrifices the weak, a uniformity that neutralizes differences, and the pretense that a single language — even a digital one — can translate everything, including the mystery of the person, into data and performance.
There is a lot to read here. I am curious where the meditations on the 'mystery of the person' will go: a brief search doesn't show further mention. The encyclical appears to focus on exhortations for us, humans, than on the nature of AI. Probably wise at this stage. I feel it is not AI that is either positive or negative, but its use of it, and the call-out to the growth of private industry as more powerful among nation-states is a strong statement for a institute like the Vatican to make:
> Technological power thus takes on an unprecedented, predominantly “private” aspect, which makes it even more challenging to discern, govern and direct such power toward the common good.
The “mystery of the person” is a common theme in Catholicism - it can be seen in other (especially the “social”) encyclicals. Rerum Novarum by the current pope’s namesake is worth a study.
I fear I have committed the sin (I mean that word, here) of commenting before reading the entire article. Often it is the comments that set the tone of interpretation and communication more than the article itself, because people read the comments, not the article.
To somewhat mitigate that, here are items that are striking me as I read more. (I hope you'll forgive that this still directs the comment in the direction of my own lense.) I'll keep updating the comment.
> Looking at our own time, we cannot ignore the fact that the protection of human rights [as declared by the United Nations in 1948] has been exposed to two particularly serious dangers. The first is that these rights are declared in a purely formal sense, while technological progress continues alongside covert or overt violations of human dignity.
I read this as a warning of how rights are words, but actions are performed regardless of them. It aligns with something I've been trying to word, which is that as I've seen more and more abuses of power, I've come to believe that ethics requires external accountability, which can often require its own power - a conclusion I don't want to come to; I would prefer social agreement and communal spirit rather than external power. But either way, I do feel it's very clear there are a lot of people, very much in tech too, who simply do what they want regardless of its harm. They justify it to themselves; they don't stop themselves; no one externally stops them.
> Along with a greater awareness of the value of every human person and their rights, recognition of minority rights has also grown. Yet, there is still a long way to go to ensure that the rights of a great many, namely women [are guaranteed.] It is, therefore, not enough to state simply that men and women have equal dignity and rights; it is necessary that this be reflected in concrete decisions, such as in laws, access to employment, education, social and political responsibilities, and the way society listens to and values women’s contributions.
In 2026 American politics terms, this reads as pro-DEI to me.
> the first major principle of Social Doctrine that I wish to highlight: the common good. We can describe it as the social expression of the dignity recognized in every person. ... For a Christian, going beyond the narrow confines of one’s own interests and committing oneself, within the limits of one’s ability, to the common good is a non-negotiable value, as is the promotion of life.
'Non-negotiable.' Very clear words.
> When politics abandons a long-term perspective and reduces itself to short-term calculations or sterile polarizations, then the language of the common good loses credibility, and, at the same time, social inequalities and divisions grow.
> 64. This also applies to international politics.
and,
> I invite everyone to conceive of ways of cooperating and of more effective international institutions, capable of safeguarding the global common good without compromising the legitimate diversity of peoples and nations. Indeed, the promotion of the common good can never be separated from respect for the right of peoples to exist, to preserve their own identity and to contribute their unique qualities to the family of nations.
I love the support for international cooperation and peace and organisations that support it. It reminds me of the post-WW2 sense, the era that gave rise to the United Nations, Unicef, etc - organisations almost forgotten in the news we see on HN today, with the possible exception of the WHO.
> [84] Moreover, any attempt or plan to eliminate or subjugate a nation is gravely immoral and therefore unacceptable.
The beauty of this - or its tragedy - is that it is so easy to apply to many situations today, actions undertaken by many nations.
> the earth’s goods — soil, water, air and natural resources — are given by God to the entire human family to sustain the lives of all, and that every person has an inherent right to the use of such goods, both now and in the future. ... Today, we are called to recognize that this universal destination applies not only to material goods, but also to immaterial and cultural goods.
Immaterial and cultural goods. This is a fascinating view on non-tangibles and one I feel inspired by. Reading this I asked myself (wait for the larger quote in a minute) how this affects views of IP, learning when texts are not available, cultural impact of characters and stories, the output from universities, publication of papers, ownership of research done by public or even private (!) funds, and more. Particularly I wonder about open weights vs open source for AI, and open source as a concept: where the old-school 'free software' GPLed version seems -- perhaps I am showing my bias -- most aligned with the ethical stance here?
> 66. Certainly there is a right to private property, which has its own specific meaning and purpose, yet it is always subordinate to the universal destination of goods.
'Always subordinate'.
> among the goods that are universally intended for everyone, we must also include new forms of property, such as patents, algorithms, digital platforms, technological infrastructure and data. In a context where the wealth of nations depends increasingly on knowledge and technology, when these goods remain concentrated in the hands of a few, without adequate forms of sharing and access, a new imbalance is created that contradicts the universal destination of goods. In turn, it widens the gap between the included and the excluded, between those who can participate in the digital revolution and those who remain on the margins.
Wow!
I cannot interpret this; it's not my right. But moving from the questions I asked above, to this paragraph, is powerful.
> > a uniformity that eliminated diversity and that chose homogenization over communion
> Unrelated to AI, but a wonderful support of the breadth of humanity in this anti-DEI time.
I mean.. DEI was in reality homogenization and eliminated diversity. Just because you agreed with the small amount of allowed opinions/people doesn't make it more diverse.
Perhaps, perhaps not; but you can't possibly argue that the current anti-DEI climate better fosters diversity than a pro-DEI or a-DEI environment.
DEI was only ever meant to reflect the diversity of society in our institutions (which historically have been and continue to be homogeneous compared to society). Of course there are better and worse ways to create the outcome of increased diversity. But thr recent backlash is the antithesis of diversity, wrapped up in the propaganda of "reverse racism."
The anti-DEI “movement,” especially what we see in the US, is mostly thinly veiled racism by another name.
All one has to see is how frequently people call any non-white/non-male hire or appointment a “DEI-hire” because they can’t possibly conceive that marginalized groups would see any success based on merit. They start from a position where they are all incompetent and being given an unfair advantage until proven otherwise. It’s also on you to guess whatever their arbitrary bar is.
'you shall have no other gods before me'. operationally, a warning. false worship is dangerous. we get what we worship (value above all, orient towards, optimize for). heaven or hell, read it immanently or transcendentally, personally or collectively. either way, choosing the right worship is paramount.
the conundrum: what is the right objective? puerile attempts, eg "maximize flourishing" or perhaps "minimize suffering" are trivially countered by paperclip machines fulfilling the letter while laying utter waste
'mystery of the person'
given the context, some keywords for orientation: free will, rejection of temptation, the cross
>“technology is never neutral, because it takes on the characteristics of those who devise, finance, regulate, and use it,”
If critical decisions affecting human life—such as hiring, lending, crime prediction, and welfare—are processed in an opaque black box, people will lose their fundamental right to explain their context or appeal against the machine's algorithmic verdicts
Substitute technology for religion and he's probably one of the least authorized persons to question this.
What is old made new. Franz Kafka passed away June 2nd 1924.
Is it wrong? Individuals were absolutely getting dehumanized in Kafka's time, but also since then are becoming ever increasingly more so, so it makes sense to speak of it continuing to happen and get worse. It is not binary.
This quote is from 1903. Times haven't changed that much:
> Not a handful of rich people, but all the working people must enjoy the fruits of their common labour. Machines and other improvements must serve to ease the work of all and not to enable a few to grow rich at the expense of millions and tens of millions of people.
The truly interesting question, and the crux of the matter, also hasn't changed much since 1903:
How?
There's been plenty of examples of workers seizing the means of production and establishing sustained non-capitalist organization (State or otherwise). We have any number of strategies to choose from: The PRC, Soviet Union, Syndicalized Spain (my favorite, seems the least likely to lead to police state), Vietnam, Cuba. The question thus isn't "how," but more specifically a couple other questions: "How do we prevent capitalist forces from liberalizing our movement (PRC, Soviet Union)," "How do we prevent fascists from killing us all (Syndicalized Spain)," "How do we prevent becoming a state-capitalist police state that halts the revolution (PRC, Soviet Union)"?
While this is certainly the more interesting question, the unfortunate reality is that the ideological complex of capital (even if weakening, and no longer effectively reproducing itself) is still strong enough that most in the West can't even imagine a better world (other than "less bad capitalism"), much less think about how to get there. Consequently messages like the above are of great value in moving more people towards a point where questions like yours become relevant.
Well, in that case, my "how" has always been along anarchistic lines: establishing parallel forms of resource distribution, establishing habits and communities of mutual aid, and in doing so, delegitimizing and rendering obsolete the State, capitalism, and systems of hierarchy.
Fun techcentric utopian speculation about this: Cory Doctorow's "Walkaway" and Ruthanna Emry's "A Half-Built Garden."
Essentially, can we leverage our current post-scarcity society to expropriate everything people need in a sustainable way that cuts capitalism and the State out of the loop? For example, why would people buy food if they can get it for free from farming syndicates or similar? (see: Global Village Construction Kit, Food Not Bombs, Food Not Lawns) Why would people buy medicine if they can print it for free from pirated recipes? (see: Four Thieves Vinegar Collective)
I see the Right to Repair and FOSS movements as a foundation to build upon for this. Anarchism (or at least, anti-capitalism) exists right under everyone's noses, in all the FOSS software installed on their computers. Existent example of people laboring without profit motive and contributing to the commons.
My personal life goal is to figure out how to capture that same energy to tackle the bottom layers of Maslow's hierarchy.
I'm all in favour of grassroots experimentation and a search for something to improve upon, and then replace capitalism. This is how capitalism itself came about and spread, though we can argue about how much it was imposed after it ceased being the underdog.
What I am weary of is that such experimentation, and the energy it generates, will eventually be overtaken by the next iteration of people who want to stop nibbling at the margins and break a few eggs already, some sort of anarchist revolutionary vanguard. Much like with communism, skilful opportunists with a thirst for power will be all too happy to take over this energy and direct it toward building the next totalitarian regime, one which will of course claim to be rendering the State obsolete, but will be about as anarchist as North Korea is a people's democratic republic.
Yep but the anti-socialism/communism world did wonders to make that feel like kryptonite whenever those words get brought up, even though anyone who is doesn't see themselves as "rich" in that sentence who fully agree. That's why even factory workers are anti-communism or anti-union which are literally the best way to fight back the imbalance of power.
You have to somehow separate the horrible evils that have been inflicted on the world by Communism before you can get people to consider words closely associated with it.
Being anti-communism is good not only for the individual's health but for their society as a whole.
The problem, generally, with this view point is that it attributes all of a societies ills to Communism and none of (or few of) societies ills to Capitalism.
For example, do you believe the Capitalist system has nothing to do with the eagerness of the United States to drop bombs throughout the world for the past 100 years? Personally I see these actions as unnecessary and evil but pushed to continue by the people who stand to gain the most wealth and influence from them.
What specific horrible evils do you mean? And how do you attribute them to, specifically, organizing an economy along communistic lines?
I ask because if we can take a country with a communist economy, or striving for one, and blame all its evils on communism itself, I have a few things I'd like to point out as being the horrors of capitalism:
1. Atlantic slave trade - millions dead (many on the ships), millions enslaved
2. Settler colonialism and indigenous genocide - British empire, all over the world
3. Congo Free State, Leopold II - 1 to 5 million dead via colonial extraction regime
4. British India famines - 3 million dead
5. Irish Famine - 1 million dead
6. Opium wars - directly caused by British using the military to defend market access. 100k dead, devastating to Qing China for a century
7. Indonesian anti-communist massacres - 500k-1mil alleged "communists" killed after the USA, UK, and Australian intelligence agencies propagandized against them
The specific evil in my mind was the Great Leap Forward, but there are dozens we could draw from history.
And I attribute them to communism because that's literally how history attributes them, though obviously pro-Communism thinkers would disagree.
> What specific horrible evils do you mean?
The 1956 student massacres in Hungary, where my grandma was almost killed. The Holodomor, the various "Russianization" campaigns, the Jewish Autonomous Oblast, The Great Leap Forward, etc.
It sounds like Communism, Capitalism, and Fascism are all very bad, then. Maybe we can try something else? Do you have any ideas?
A) Well, it's you vs. the Pope and I!
B) Fine, drop communism altogether -- it's evil and disgusting and bit my finger and should never be tried again. Can we work on a society where the means of production are owned by groups of laborers?
Re: B my guess is probably not (human nature and all that), but I'm open to ideas! I just think failed experiments where tens of millions died are probably not ones where we just flippantly "try again".
Yes it's a very dangerous concentration of power. But I don't believe strong words or proposing regulation by just another elite would change much.
Please join/help open source groups doing small + local or distributed models. There's a lot to do. Support truly open source companies.
Let's walk the walk.
Which open source companies would you recommend supporting?
The overarching message is that builders should deeply consider the impact of what they're building on civilization.
"Technology is never neutral, because it takes on the characteristics of those who devise, finance, regulate and use it."
Therefore builders "bear a particular ethical and spiritual responsibility" because "every design choice reflects a vision of humanity."
The questions shouldn't just be 'can we build it?' or 'will people want this?'
We need to also ask 'should we build it?' and 'will this make humanity better?'
The encyclical calls on us to “join forces in building up the common good.”
This is a message we need right now.
Not new for software and hardware industry though, practitioners have just chosen to ignore it. From the Association for Computing Machinery, which encompasses all forms of software development, the very first principle is the public good:
"Software engineers shall act consistently with the public interest. In particular, software engineers shall, as appropriate:
1.01. Accept full responsibility for their own work.
1.02. Moderate the interests of the software engineer, the employer, the client and the users with the public good.
1.03. Approve software only if they have a well-founded belief that it is safe, meets specifications, passes appropriate tests, and does not diminish quality of life, diminish privacy or harm the environment. The ultimate effect of the work should be to the public good. ..."
From the IEEE, which also encompasses computer engineering, their first principle and its first few sub-items are:
"To uphold the highest standards of integrity, responsible behavior, and ethical conduct in professional activities.
1. to hold paramount the safety, health, and welfare of the public, to strive to comply with ethical design and sustainable development practices, to protect the privacy of others, and to disclose promptly factors that might endanger the public or the environment;
2. to improve the understanding by individuals and society of the capabilities and societal implications of conventional and emerging technologies, including intelligent systems; ..."
This would be more meaningful if, perhaps, we had to swear an oath to it before being able to practice. And practitioners would be treated more seriously if everyone knew we swore that oath. And the legal utility as accountability and defense would also be useful.
Of course people are going to ignore it if there's no force behind it.
Professional liability and licensure would create assurances with some teeth, but there are some major drawbacks.
It seems to work somewhat with medical doctors and the Hippocratic Oath.
But I would argue it is way easier there. Building software has way more grey areas.
I don't think there are more grey areas in software engineering than in medicine. The difference is the feedback loop of the outcome - if you design a dopamine slot machine you will ruin the generation and that's a long arc.
And that makes it hard. I am open for banning all comercial advertisement - but general society is largely fine with it. So is someone designing new targeting algorithm for ads breaking his potential oath of doing good for society?
What grey areas are there for doctors?
I think not just (computer) scientists but the general population thinks to serve the common good makes sense, not last because we understand it's eventually for our own good.
It is however just that very small minority of the population with highly psychopathic/narcissistic traits - those that, in pre-historic times, would have been kicked out swiftly of hunter & gatherer / small village communities because of their parasitic nature - that in bigger civilisations seem to thrive due to abstraction (distance/time of the effect of their actions) and obfuscation (PR) and instead unfortunately seem to rise to the top (CEOs, presidents, 'thought leaders' ...) to steer the world's overall economy and mindset - and steer it in the abyss.
Sometimes I think humanity was just not made to scale, and this aspect is one very large aspect of it.
Noble ideals can be upheld, relatively consistently… only if violators are visibly punished. And to a degree roughly commensurate with the violation.
The critical point is that the violators have to be punished more consistently then the demanded consistency of the ideals.
Not only builders, the greatest takeaway for me is that everyone has a responsibility in shaping the discourse, culture, and usage of transformative technology. This "the builders will do the right thing" mentality is even (in my interpretation) explicitly called out in several places:
> It is the pursuit of the common good that gives life to a people, understood not as a mere collection of individuals, but as a living reality in which people learn to recognize that they themselves are interconnected and jointly responsible for the res publica. In this sense, every person contributes to the building up of one’s people...
> When it comes to decisions regarding economic flows and digital platforms, as well as the governance of data and algorithms, we cannot allow a handful of actors to dictate these processes on their own; instead, we must build forms of cooperation that respect the various levels of the global community and make them jointly responsible for the common good.
> We cannot be satisfied with merely calling for the moralization of machines — the so-called “alignment” of AI with human values — without also having the courage to insist on a further condition: the possibility of openly discussing the ethical frameworks involved and subjecting them to shared standards of social justice.... What is needed is a more active political involvement that is capable of slowing things down when everything is accelerating, and of protecting the opportunities for communities still to be able to participate and ask questions.
Agreed, I think the builders are the wrong people to ask for this self reflection. Anything can be used for good and bad. A knife can be used to prepare food, or to stab someone. A drone can be used to attack a country, or to defend a country. A wooden beam can be used to build a house, or to build a cross to crucify someone. Same way AI can be used for good and bad. And it’s up to the person using the tool to act moral. Unfortunately this world is full of people that consider power and money more important than morals. For me, religion falls in this bucket, lots of “do as I say, not do as I do”.
The Roman Catholic Church has a lot of things wrong with it, now and in the past, but it’s a human institution older than almost any other, and it’s composed of a lot of very intelligent people. Agree or disagree with them, but a papal encyclical is almost always worth reading and understanding.
Indeed. And because of it’s age and outlook its view is very far back and very far forward.
The casual reference to a 135 year old encyclical that dealt with the seismic shift of industrialisation took me quite aback for a number of reasons.
There are so many encyclicals, apostolic letters, etc. One could spend years reading just a fraction of them, depending on reading and comprehension speed, of course, which varies by person.
Two I recommend, from the last 40 years:
Veritatis splendor, John Paul II, 1993
Argues that Christian freedom is fulfilled, not limited, by objective moral truth: some acts are intrinsically evil regardless of intention or circumstance, conscience must be formed by divine law rather than self-authorization, and the Church must faithfully teach this moral truth as the path to authentic human flourishing in Christ.
https://www.vatican.va/content/john-paul-ii/en/encyclicals/d...
Fides et Ratio, John Paul II, 1998
Argues that faith and reason are complementary paths to truth: reason needs faith to avoid skepticism, relativism, and reductionism, while faith needs reason to express, defend, and deepen its understanding of divine revelation and the human search for meaning.
https://www.vatican.va/content/john-paul-ii/en/encyclicals/d...
The Roman Catholic Church is an institution that went to war to try to prevent people from being able to read the bible in their own languages, that executed dissenters who challenged the doctrine on which it based its power, from scientists to Protestants. If it had never lost power in western Europe, it's quite possible the enlightenment and industrial revolution would never have happened, and humanity would still be living in the dark ages.
It's an organization that is effective as an opposition party of sorts.
The strange part is how moral responsibility somehow always lands on the builders... the people with the least leverage... while the funders get to ask the ethical questions. Weird!
Mark Zuckerberg is a builder in the context of the encyclical.
No, we don't have to take the funders' money; that's what having professional standards means. Nobody would excuse a doctor performing unsafe procedures because they "needed the money". Engineers were jailed for the Volkswagen emissions tampering scandal and nobody would excuse them for needing to take funders' money.
I agree that holding only the concrete implementers responsible would be inappropriate. However, I don't believe that distinction is made. One says "I am building a house" even if they are completely contracting the job out. I'd suggest the greatest responsibility lays with the funders and that the Pope would agree.
The funders are among the principal builders in this context. This is addressing the people who have a say in what is built, and how it is built. Much of that belongs to the executive and ownership class, but not all of it.
Yes but... No one living today knows what direction AI technology will take humanity. If we have an algorithm breakthrough then we may avoid building new data centers. If the abilities of the technology plateau then there might not be large impacts on employment. Builders need to focus on the impact of their next steps. Don't put polluting natural gas generators in neighborhoods today. Don't make unemploying folks the goal of your tool, today. Don't make decisions that harm people and the environment today.
https://www.theguardian.com/us-news/2026/may/13/utah-approve...
> The Stratos artificial intelligence datacenter footprint will cover more than 40,000 acres (62 sq miles) over three sites in Box Elder county in north-western Utah. The facility will require about 9GW of power, which is more than the entire state of Utah currently consumes
Sure hope your rosy inflection point happens real soon.
Otherwise, the direction this is taking us is pretty obvious.
9 GW is massive. About half a percent of world electricity consumption, half of what stupid Bitcoin burns.
As I keep telling my project owner at work, "the AI will get better" is not a plan.
> Therefore builders "bear a particular ethical and spiritual responsibility"
> This is a message we need right now.
Feels good man. The solution found by the private parties driving technological change is sainthood. Or aiming for it. At least, better than you. They have the vision of what's good for the herd, but the more time I spend as a sheep, the more it seems the "herd" is just a way to recycle the story of their own exceptionalism stripped of any mark of individuality. A simple visit to fiftyyears.com will greet you with "We back the indispensable". I guess it's the same "we".
> Scientists are actually preoccupied with accomplishment. So they are focused on whether they can do something. They never stop to ask if they should do something.
From Crichton's book Jurassic Park, which like most of his books is about the perils of technological advancements.
They used the quote in the movie, slightly tweaked.
I would replace the word scientists with engineers in that quote. People often conflate the two, but in my experience, scientists tend to be more cautious and there are built in checks and balances in the process (however flawed).
Engineers/technologists tend to have no such guardrails, and are also usually embedded into entirely profit motivated environments, whatever their own values might be.
This sounds like you don’t have much exposure to actual professional engineering disciplines. I’m sure civil, electrical, structural and mechanical PEs would be quite surprised to hear there are no guardrails on their professions.
Engineers versus "engineers".
I have fond memories of a boss who was an actual, licensed engineer while the rest of us were very much normal software devs. Boss was pretty chill except when someone someone suggested we should be called "engineers" rather than "developers", at which point they said "if you guys were building bridges, people would be dead." (I don't think all software needs to be built to rigorous engineering standards but man... I think about that line a lot.)
I prefer the term "software developer" and that's what I use when I don't need the prestige of the term "software engineer". It's disadvantageous for organizations to do that with actual job titles, though.
Absent US government intervention to codify the term "engineer", probably the only way out of the "engineer" trap is through further title inflation, where the developers all become "vice presidents". :)
In my experience “engineers” and builders are often quite “conservative” and really don’t like pushing the envelope, and they often only do it under protest.
The most famous example may be the perpetual war between architects and engineers/builders.
I actually originally wrote "technologists" but thought that the word sounded kind of odd. Now I realize it better captures what I was trying to convey.
Researchers need to go wild and sometimes far off-the-rails to increase the odds of coming up with something that is both new, and potentially popular enough, if they want the option to attract marketers who can only thrive on mass-consumption.
With luck, one out of 100 inventions will show promise on those points.
There's always a lifetime wake where the overwhelming vast majority of the work remains undeployed no matter what. The more undeployed milestones and inventions that some scientists have under their belt, the more accomplished they often are whether anybody knows it or not.
OTOH, equally active engineers more often need to have most of their time engaged in actual deployment of some kind or another, otherwise not as much progress will be able to reach as many people that could benefit. So many times nothing would be accomplished without a long-term focused engineering effort once an objective has been identified. But it can be hard to stop a train when it's already coming off the drawing board at full steam.
It does seem a lot more likely for a judicious researcher to cast off some major progress in what could very well turn out to be an unsavory development, such as likely misuse, even if it could be marketed as the most popular thing they have so far. Just add it to the pile of other things that best remain undeployed. There's plenty more where that came from, and the best is yet to come.
Perhaps popularity alone is not always the best measure of progress.
I worry that companies like Anthropic, which are moving toward RSI, may prioritize speed over the timely identification of irreversible risks.
Every time I prioritize speed over risks, I, too, end up with repetitive strain injuries.
Did you mean AGI? I'd think whatever else they are doing, LLMs are reducing RSI...
hard disagree. No matter what gets built, it will be used for bad things if the society is not properly constructed. We need to fix our government and then you can build anything you want. I don't like putting the blame on the builders, building is already difficult as it is. Also you can't built something in isolation, word will get out anyway. That's why this speech is either naive or he just wanted to tick a todo list: say something about AI.
Hard disagree to your disagree. Every organization is made up of people and reflects the character of its members. So with the nation itself. If people won't take responsibility for their own actions, neither will the government. I.e. self-government isn't the answer to fixing a broken people. Transformation begins within one's self, not in imposing one's will on another. First everyone much look to themselves, then the government, organizations, and projects will fix itself.
Everyone likes to talk about "fixing the government". It feels nice to understand the problem with something else that is broken. The problem is that replacing the people in charge is a no-op if you don't have a pool of good people to choose from and the will to choose them.
> We need to fix our government and then you can build anything you want.
Are we all going to stop building things until the government is fixed and all political problems are resolved? No? Then we must think about how we should build in a world with imperfect government.
> I don’t like putting blame on the builders
Why does it have to be either/or?
From a high-level view, there are powerful cultural/political forces that nudge us towards building harmful, wasteful things without us being aware of said forces. The government, corporate greed, and billion-dollar marketing budgets are to blame.
But on the ground, from an in-the-trenches perspective, I think an engineer at Meta, for example, shares in some level of blame, too. Mainly because there is plenty of evidence showing how much harm Meta products have brought upon society. Yet Meta couldn’t be built without engineers opting in to work there in exchange for $300k salaries.
We desperately need moral clarity and courage at both the policy-level and the individual level.
Because there will always be bad actors. You cannot design a system on all actors' full cooperation.
Ofc they share the blame, but it is not solving the problem.
You can say for example: loan sharks are bad for society; so government gives anyone 0 percent credit. You just removed one problem, created another.
Just and sustaining system with individual morality is destined to fail. Only option is social regulation. Which is at government level.
It is simply not feasible.
It's not either/or in strict terms, it absolutely is either/or in practical terms. When the conscientious engineer chooses not to take the 300k job, the next one in line does. If enough choose not to, it just became a 400k job.
Can you change society in such a way that nobody would take such jobs? In strict terms, sure, in practical terms it probably entails enormous costs, both economical and societal. And then you still have other countries.
There's an entire legal code filled with things on which we can't rely on the morals of the people. We can't stop theft, rape and murder, what makes you think that stopping engineers is any more feasible?
The best thing builders can do is use their knowledge and authority to pressure the other side.
"That which can be destroyed by the truth should be." - P.C. Hodgell, from her novel Seeker's Mask (1994)
"That which can be automated by the AI should be."
I get where you are coming from but this is the common "reduction to politics" that anyone who doesn't want to address a problem uses: think of any societal or human problem and you can have your comment with different nouns.
Sure, IF we could just go and fix our governments in some magical way then the problem would disappear. That goes from hunger, climate change, videogame addiction and AI. The problem is that what you value in life in different than what others do, so we now have a system in which sometimes you get what you want and sometimes you don't.
But back to the topic, I do think that how OpenAI and Anthropic handled the government and them asking to drop guardrails is something a company can actually and actively do without having to reinvent the universe.
Consciousness is the causal factor, not government. You are putting the cart before the horse to insist on any political solution.
> I don't like putting the blame on the builders
Like it or not if you knowingly build stuff that is used for evil purposes you are complicit
You can't build an orphan mulching machine and get away with just a shrug. "I don't really agree with mulching orphans but someone else paid me to build that. If I didn't build it someone else would have" just does not absolve you of your involvement in mulching orphans
When he speaks of the builders needing to take responsibility, I think he is directing his comments at tech company leadership. The Sam Altmans of the world, not the poor slobs like us who need a job for health insurance because apparently unemployed people don't deserve to live.
No, he's talking to you too. The Sam Altman's can't be a threat without useful idiots to cooperate with and doing work for them. You don't get out of your own culpability because you locked on a set of golden handcuffs.
I have only read a few passages (and some of the excellent quotes others have shared here), but I find the underlying message here so much more compelling than those found in the various "manifestos" which come out of Silicon Valley.
I think reading this helps me imagine a version of the future I'd actually like to live in. A version where technology is used well (rather than preaching for abstinence from technology) and where values other than "intelligence" (in whatever guise) are on an equal footing.
Even writing that makes me feel naive (and to an extent I know it is) but I think it would be inconsistent for someone who cheers for humanity's efforts to solve/chip away at "impossible" problems (like LLMs were thought to be not so long ago) to shirk from the challenge of making the world better for _everyone_.
The thing is why that this feels so good future is; it is a system with no constraints. A bit like Star Trek universe in Roddenberry's imagination. This kind of utopia can only be achieved with all honest actors, but in reality systems are usually designed around bad actors.
Even with all morally good actors locally, there is no guarantees for external forces. Thinking it hypothetically, even with global coordination ( all good actors ) there is not a proven path that would lead us to better place from any starting point from past.
It's probably more predictive to model actors as being neither good nor bad but constrained by various collective dilemmas, such as prisoners dilemma, the security spiral, tragedy of the commons, race dynamics, collective action or first mover problems, information asymmetries, the commitment problem, among others. Those are the hardest problems to solve because they're pathologies that result from the global, largely amoral structure rather than consequences of the individual exercise of morality.
In the AI case, each firm is in an arms race, and nobody can slow down without effectively collapsing due to positive gross margins only being viable with a frontier model that attracts marginal demand. An appeal to morality might have an impact but more effective action would be to address the structure that the AI companies are situated in that causes this dynamic in the first place. In practice, thats going to be a global agreement to slow down, and global regulations.
Yeah but this is a system problem; if we had this utopic system from the beginning we would not even have AI probably.
This sort of rationalization of evil is a core of technocratic support for Trumpism, I find, and has parallels to the evangelical prosperity gospel. Choice tenets:
It’s always cloaked in a veil of realism, but it’s just the classic 14-year-old-boy-just-got-introduced-to-the-prisoners-dilemma situation. There’s nothing philosophically interesting about it.Ironically, these are often the same people denouncing multiculturalism, yet the culture they strive for is completely morally bankrupt.
And it's funny because the "realism" has been proven wrong over and over and over again for millennia. People do all sorts of selfless and generous things all the time! The entire premise is trivially disprovable by just going and asking a neighbor for some help with something.
That's not to say we should be naive about greed or malice existing or being powerful motivators (especially the former), but it is obviously not true that they're the only forces at play and therefore you are "just doing the logical thing" by succumbing to them. It's just the more destructive version of the same naiveté.
Seems you and I have together struck a nerve. Maybe our sentiments would have been better received in an alternate subthread, but it’s all I could think about while reading the parent / cousin comments.
I haven’t read the full Magnifica Humanitas yet, but I would be pleasantly surprised if he touched on not just dehumanization of the other, but dehumanization of the self. Expanding on your thought, succumbing to those forces under the guise of just doing the logical thing is in a way self-dehumanization - to believe you are only capable of the “logical” thing instead of the moral thing.
This was not my point at all. Maybe I could explained better, but main criticism I have is: you can bundle together objectives ( which are inherently good ) and create an utopia. But those cannot always be achievable.
Everything in life in trade-offs. Simple example is speed/quality/cost. I can tell easily:
- services should be cheap - services should be fast - services should be high quality
Now I created an utopia. Obviously this is amazing to listener. They agree. But is it achievable?
It is not saying greed is good or might makes right. But system means you need to construct from this ideals best outcome ( which comes at some trade offs)
Yeah there were probably more appropriate subthreads to have responded to. My point wasn’t quite neatly directed against yours.
To get from here to Roddenberry's communism, according to Roddenberry's lore, we passed through the Eugenics Wars, the Second Civil War, and then fifty years of World War Three and the 'post-atomic horror' before coming to our senses.
> I think reading this helps me imagine a version of the future I'd actually like to live in. A version where technology is used well (rather than preaching for abstinence from technology)
I believe the Amish figured this out over a century ago.
> I believe the Amish figured this out over a century ago.
The Amish rather came to a different conclusion (which I don't want to judge on, but on which I nevertheless have a different opinion than the Amish).
What is that conclusion which differs from the post you replied to? The Amish are mindful about their technology adoption.
The church, the message comes from, literlay makes the world worse for so many people on the planet.
Woman have less value under the church than man. Alternative sexual views are evil.
Church and any kind of believe system hurts our society and divides us.
> Church and any kind of believe system hurts our society and divides us.
Any belief system? And yet I bet you value freedom over slavery, wisdom over ignorance and compassion over brutality. That’s a belief system, despite not being a religion.
I can argue these values and we can discuss them.
If the majority says, no we want to be able to control other human beings, these people will reinact slavery. From a society point of view though we see that its not a working model anymore.
A real believe system can't be argued with. You believe in this god? This god says x and thats why you do things? Okay thats it. You don't even question were this information even came from.
If we delete all religion tomorrow and science, there is a realistic chance that the society rediscover the same existing rules like math and gravity, but religions might appear again but with different names, different rules etc.
I can change your mind with logic and arguments if its not a believe system, i can't do that with religion.
Wisdom over ignorance: The chance of survival is higher with wisdom
Compassion over brutality: This is just basic Game theory
Fornication culture is a big part of why the west is in decline.
What may make sense for you individually may also be empirically proven to be detrimental to the whole.
The new testament contrasts with the old, the gospel is one of tolerance and equality. It's a big part of why you have the rights that you do, as do women.
That said a lot of what you're saying can be ascribed to religious institutions and sects and individuals and specific churches. But your general prescription is like saying "this logical axiom is evil because XYZ ascribes to it and they are also evil".
You also have a belief system -- that people who believe in God do so because they don't question their beliefs, that religious people are only led by dogma. Yet your belief is wrong. Have you tried questioning it?
> but religions might appear again but with different names, different rules etc
Religions and scripture spread also evolutionarily. Christianity is popular because it is rooted in many truths.
> Compassion over brutality: This is just basic Game theory
And the game has been played.
"Fornication culture" who said that? We are more people on the planet than ever. Less people have to life lies like being in a marrage but also being homosexual.
Its just a control structure from the church without education.
Its probably even because of missing education. Educate people properly and they can handle "Fornication culture".
I don't have a believe system. I have a theory why people believe in gods and religions. We have evidence for it. People studied the origin of religions:
"It evolved from humanity's psychological and social needs, primarily our desire to make sense of the natural world, cope with the fear of death, and foster community cooperation."
We know how little people knew when religion started to emerge. Never seen space, never seen above a cloud besides a few poeople going up mountains. Thunder was not understood. Between 1400-1700 we had witch trials.
It is dogma. What is your argument against dogma?
"Christianity is popular because it is rooted in many truths." were is your argument for this? Its popular due to luck, power and wars. Missionaries as well and especially probably the most critical thing: Early indoctrination.
>> Compassion over brutality: This is just basic Game theory > And the game has been played.
Yes exactly. Compassion wins because its better, not because religion says so. Its an evolutionary win.
"Church"
ok
"and any kind of believe system hurts our society and divides us."
People shouldn't believe anything?
Disagreement and conflict are natural. How we handle these disagreements while striving for widespread peace and prosperity is the question.
I don't believe. I accept things i don't know and i know what i know.
There is no inherant issue with this. in contrary it makes me mentally stronger.
I can choose on my own terms if/when i want to end my life. If i get very sick, i don't have to hope for a god or priests blessing to end my life, i will just do it.
But religion is different: if you believe that homosexuality is wrong due to your religion, there is nothing i can argue about. Your priest told you this based on some book or story from 2000 years ago and you do not question this.
I know plenty of strong christians and muslism in germany who do not like homosexual people. And its dividing our society.
Does that mean that nothing of value can come from the church, and we should ignore all ideas they put forth out of some kind of spite?
Spite? Its not spite.
The church creates a believe system which indoctrinates all of us and took our cultures away.
The germanic tribes were believers in nature, church removed all of this.
Church also doesn't see woman as equal.
Just because they might sometimes also say postive things or things we might align, doesn't mean i need their oppinion. And especially not on hn
Indoctrination and cultural erasure isn't unique to the church. With Germanic tribes - the Saxons, Angles, Normans, etc., while women's rights ebbed and flowed over the centuries, they were rife with double standards and were still quite patriarchal. They didn't just believe in nature, they believed in the same pagan gods as the Nordic peoples.
The point is that times change, and institutions change, and holding grudges for long ago sins and policies is ineffective.
This is not true. The number of believers and official people in the church is continuesly declining around the globe.
1990 there were nearly 60 Million people in the church in germany (nearly everyone) now we are down to under 40 Million which crossed the line of 50%
This is real tangable progress.
The only downside is, that we do not sit together and formulate something which might give people hold who are not ready to be self stable mentally and might need something like this. And we also do not try to align together on rituals which help us to live better together.
I'm also still affected by the indoctrination of the church. As you can see, i argue against church not just because I like to argue, but because i really really hate religion and especially the christian church in germany, due to my own values and experience. This is not old i'm only 36.
As an aside, at least Trump is drawn to the grandeur of high culture from historical times, but he also doesn't understand a jota about aesthetics, and so the White House gets turned into a tacky gypsy-style abomination with one dollar ornaments.
We lost the “liberal education” (not the political one, but the “freeing” classical one) and it’s starting to show.
When you compare the robber barons to Google and Meta it’s kind of embarrassing- they build massive empires of iron horses screaming across the world and covered cities in magnificent buildings (stations, libraries, etc). G&M built an empire of advertising and … not much else?
Indeed. The current crop doesn't have an idea for what they hoard their billions, it's just...emptiness. I propose we explain the tech's attachment to Accelerationism as a profound boredom and lack of purpose. "What does it mean to be human"--they don't value that question. Peter Thiel got interviewed a month or two ago, and he could not be brought to say that he sees value in preserving humanity. He would rather turn himself into a robotic contraption to extend his life.
When power fears death, some strange things happens.
EDIT: link to the interview with Thiel <https://xcancel.com/rcbregman/status/2036113528126394834#m>
I’m reminded (and apropos as the Pope quoted him) of Tolkien’s description of the “eternal life” the Ring gives to mortals, and how it’s … not so desirable in the end.
Google makes phones and phones are somewhat good. Better search had some value for humanity. Meta has no redeeming qualities or achievements, other than helping Trump get into office and defeat Iran.
Cool, inspires me to think, "should we regulate away the toxic side effects which we realize COULD have come, if we built it less considerately?"
Science and Engineering is morally detached.
Ask a scientist or engineer what philosophy or theology has taught us about the source of morality and their education, training and experiences havent prepared them to answer that question.
This didnt matter so much in the past because their activities never had the scale it does today. For basic training in philosophy if you are mid or upper level exec whose decisions are going to effect a whole lot of people, go to open yale courses and take the intro to Philosophy classes. It will help develop your answer to the question - why do you do what you do if you are going to die anyway tomorrow.
This is a ridiculous comment. Science papers always have sections on impact. It's the running with scissors industry types simply chasing the bigger paycheck that don't stop to think.
Far too many of us (particularly younger people, but not only them) undervalue or are dismissive of philosophy. I once was like this, partly because at the time I'd been brought up - like most humans - to believe my parents' religion held all the answers philosophy might address.
I quickly learned as an adult that whether you're a person of faith or not, it's not pointless at all. It's the foundation of everything. Philosophy is how you explain the deeper reasons behind why you follow whatever religion you do, or adhere more meaningfully to whatever kind of agnosticism, atheism and/or 'spirituality' (with or without woo) you espouse.
This can only be true if you believe in a certain mad-scientist version of these things. We as a society have systems (however flawed) to hold scientists and engineers to ethical standards: e.g. peer review and in many cases--e.g. civil engineering, architecture, medicine--even legal frameworks to enforce ethical standards. Science and engineering are human endeavors, they are not divorced from the human condition, they cannot be separated from humanity and human rights.
Maybe you mean mathematics is amoral and are committing the common conflation of "engineering is just applied science and science is just applied mathematics," which is a really bad case of missing the forest for the trees.
"Technology is never neutral, because it takes on the characteristics of those who devise, finance, regulate and use it."
My comment under the post on Omarchy here got downvoted and flagged by the "technology is neutral" crowd because I dared to say that it is unethical to use software produced by a white supremacist.
Thanks for this!
Not from a church which indocrinates young people and has a massive child fucking scandal in the running for the last 15 years.
>for the last 15 years.
The scandal broke out in worldwide media starting in 2002 (not coincidentally, the same year South Park released Red Hot Catholic Love). This reminds me of when Contrapoints off-hand commented about how things were different in the 70s but that was "30 years ago".
I would also recommend God and Golem, Inc., A Comment on Certain Points Where Cybernetics Impinges on Religion:
https://monoskop.org/images/1/1f/Wiener_Norbert_God_and_Gole...
Norbert Weiner was an atheist but he talks about three areas religion is the only thing to have really examined that relate to capable AI: omniscience, omnipotence, and worship (gadget worship). It has very prescient stuff on blackbox learning/distillation, reinforcement learning/reward hacking, alignment through human feedback.
His The Human Use of Human beings and Cybernetics are extremely good too and have more of a mash of the themes between Rerum Novarum and Magnifca Humanitas, and more near-term automation.
Awesome, thx for posting. I now have my new next book to read. Been wanting to read more of the original cybernetics stuff.
Would you also recommend "The Human Use of Human Beings"?
Yes, it starts slow with a lot of history of Cybernetics stuff through Leibniz and stuff (kind of prescient given chain rule -> back propagation, and control theory's relevance to optimizers). It is about twice as long as God & Golem and covers a lot of the same plus more examination of automation and human augmentation. I think either it or Cybernetics also goes into mass communication with some relevance to how social media played out.
God & Golem is the most succinct and up to date though, probably a 2hr read.
The book Cybernetics is a lot of math and ergodic theory stuff that went beyond me, but is the longest and you still get a lot out of it skimming over that stuff if you don't have the background for it. The last revision of it in the 50s added some of the same blackbox function copying/imitation learning/distillation stuff, reinforcement learning with reward hacking concerns, and superintelligence as genie/monkey's paw.
I would read the three in reverse order of publication.
He also foresaw another big area of potential existential danger, Wiener filter for guidance and control of missiles (later superceded with Kalman filter bringing the nuclear hard targets era with 15min retaliation windows) and refused to work on it or share prior work, and he also had bioweapons delivery concerns before the bioweapons treaties, publishing this open letter in The Atlantic in 1947:
https://archive.ph/D7BPt
Under this logic, would the Pope have built a Hammer? a Knife?
I am hoping the Pope has a cleaner view of AI quietly automating the drudgery in the back offices and not just robot dogs with machine guns. And with that view, should we build it?
It's too simplistic to imagine the tension is between robot patrol dogs vs automating drudgery. If automating drudgery suddenly puts 30% of people out of work, it has huge broad negative impact on people who are currently alive and working in the current system. Innovate, but do it with awareness.
Than an even better question for the pope is:
If a technology existed that reduced the cost of producing a critical thing (think food, housing, medical care) down to near zero, however, it made the humans currently building the thing redundant, should we build it? Would it be okay to use the hyper-optimization power of Capitalism to build such a technology faster?
Likely the answer would be: yes, we should build it, and take appropriate care of the people being made redundant.
These two are not mutually exclusive, it's just that no one wants to pay for it.
This is also the message I got from "the wind rises"! Though from talking with other people that takeaway doesn't seem universal -- which IMO is one of the ways to tell it's a great film :)
> As Pope Francis warned, we must realistically ask ourselves who holds this power today and how they use it: “It must also be recognized that nuclear energy, biotechnology, information technology, knowledge of our own DNA, and many other abilities which we have acquired… have given those with the knowledge, and especially the economic resources to use them, an impressive dominance over the whole of humanity and the entire world.” [7] In the past, it was largely up to the State to guide and direct innovation. Today, however, the main drivers of development are private, often transnational, parties that are endowed with resources and the capacity to intervene that surpass those of many Governments. Technological power thus takes on an unprecedented, predominantly “private” aspect, which makes it even more challenging to discern, govern and direct such power toward the common good.
I look forward to reading this in detail. As I get older (and perhaps as AI has allowed me to spend more time thinking and less time doing) I've found myself thinking more and more about what it means to live a virtuous life and about ethics and morality and so forth. I don't have any answers (and I'm not looking for them, really, just musing) but I do find it very interesting to read and learn from and about those whose job it is to think about the answer to those questions.
When he quoted Tolkien, my heart stopped. This passage might provide you with a suggestion on how to live a virtuous life:
"The twentieth-century Catholic author J.R.R. Tolkien, in the words of a protagonist in one of his novels, described our responsibility in this way: “It is not our part to master all the tides of the world, but to do what is in us for the succour of those years wherein we are set, uprooting the evil in the fields that we know, so that those who live after may have clean earth to till.” [187] The civilization of love will not arise from a single or spectacular gesture, but from the sum total of small and steadfast acts of fidelity that serve as a bulwark against dehumanization."
I am immediately reminded of my favourite quote from the Jewish book Pirkei Avot (‘Ethics of the Fathers’):
> It is not your duty to finish the work [of perfecting the world], but neither are you at liberty to neglect it.
[https://www.sefaria.org/Pirkei_Avot.2.16?ven=english|Mishnah...]
I grew up Jewish. I have lost my faith, but that quote is still fundamental to how I see my place in the world.
I'm an atheist but I really like:
>Therefore man was created single in the world to teach that for anybody who destroys a single life it is counted as if he destroyed an entire world, and for anybody who preserves a single life it is counted as if he preserved an entire world.
(Directly from the Mishna in the Talmud Yerushalmi)
That is a really beautiful passage, thank you for sharing - I hadn't made it to that section yet and still haven't. I'm still reflecting on the stuff in the opening!
> If we focus only on contingencies, we risk letting the succession of emergencies dictate the direction of our path. We are living through a rapid phase of transition, a “change of era,” in which — while some are vying for the future of new technologies and others dedicate themselves to reflecting on the matter — most people are watching and waiting, observing from afar and merely hoping for the best. For this very reason, crucial questions impose themselves on our conscience and can no longer be avoided: Where are we going? Toward what goal do we wish to orient ourselves? What direction should we choose as a people and as a human community?
I wondered if that was the Pope's way of throwing shade at Palantir and Peter Thiel.
I had the exact same thought, and JD Vance too.
> When he quoted Tolkien, my heart stopped.
I wonder if meeting Colbert played any part in that.
I doubt it, there is a much simpler explanation: virtually all English-speaking Catholics dig Tolkien.
"But the effect of her being on those around her was incalculably diffusive: for the growing good of the world is partly dependent on unhistoric acts; and that things are not so ill with you and me as they might have been is half owing to the number who lived faithfully a hidden life, and rest in unvisited tombs."
> Certainly, the decisive turning points in world history are substantially co-determined by souls whom no history book ever mentions. And we will only find out about those souls to whom we owe the decisive turning points in our personal lives on the day when all that is hidden is revealed.
Edith Stine
Sure. And this is what everyday people do. And this is why CEOs and billionaires refuse to do (doing their fair share), and freeride on the people's work and dedication
I have only skimmed it, will definitely read carefully as soon as I have time. I will say, as an atheist, that regarding technology the Vatican has some of the best takes of any institution/government I have ever seen.
Indeed. This paragraph shows something that most people really don't understand about AI.
> 98. It is appropriate to preface this discussion with two considerations. First, any statement regarding AI risks becoming quickly outdated, given the remarkable pace at which these systems are developing. Second, all of us, including those who design them, possess only a limited understanding of their actual functioning. Indeed, current AI systems are more “cultivated” than “built,” for developers do not directly design every detail, but instead create a framework within which the intelligence “grows.” As a result, fundamental scientific aspects — such as the internal representations and computational processes of these systems — remain, at present, unknown. There thus emerges an urgent need for a twofold commitment: on the one hand, a deepening of scientific research; on the other, the exercise of moral and spiritual discernment.
Much of Western thought traces back to serious work by Church theologians. Even atheists are strongly influenced by the patterns they set down. The Catholic Church, for all its many faults, retains a serious intellectual tradition.
(In fact I think atheists should make more effort to learn about the vast diversity of other faiths. It's very narrow to be atheist only about the Abrahamic deity. You end up incorporating a lot of Christian thought without realizing because it's so deeply ingrained that it seems like the only option.)
> Much of Western thought traces back to serious work by Church theologians
Western thought traces back to the Greeks. Aquinas refers to Aristotle as "The Philosopher." Aristotle died over 300 years before Jesus was born.
> The Catholic Church, for all its many faults, retains a serious intellectual tradition.
On Aquinas, the Church Doctor, Bertrand Russell had the following to say:
"Before he begins to philosophize, he already knows the truth; it is declared in the Catholic faith. If he can find apparently rational arguments for some parts of the faith, so much the better; if he cannot, he need only fall back on revelation. The finding of arguments for a conclusion given in advance is not philosophy, but special pleading. I cannot, therefore, feel that he deserves to be put on a level with the best philosophers either of Greece or of modern times."
> In fact I think atheists should make more effort to learn about the vast diversity of other faiths. It's very narrow to be atheist only about the Abrahamic deity.
Your sentence doesn't really make sense, and there is a lot of deities..
> You end up incorporating a lot of Christian thought without realizing because it's so deeply ingrained that it seems like the only option.
Depends on the country, some Northen european countries have a very high proportion of atheists, so it happens probably less there.
There are a lot of deities, and they are far more diverse than you would expect if you're not exposed to them. Even the more atheist countries still seem Christian to Hindus, Confucians, animists, and thousands of other more obscure religions.
> Much of Western thought traces back to serious work by Church theologians.
The problem I have with this is that it's structurally a motte-and-bailey claim. If I have to take it literally, then it's obviously true and it's simply unserious to deny it: the Church does have a pervasive influence on Western civilization. The way it's often rhetorically used, however, is in opposition and to the exclusion of other strands of thought that are equally foundational: the renaissance, the enlightenment, the revolutions of the 17th and 18th centuries, the scientific enterprise, in a smaller but still real way classical antiquity. To the extent it can be said to exist, Western civilization is a patchwork. It is beautiful and I very much like it, but I don't think any one patch gets to have all the credit.
> In fact I think atheists should make more effort to learn about the vast diversity of other faiths
A better version of myself for sure would make that effort. The problem, of course, is that other faiths are just as deep and complicated as "our own", and it would take a lot of time and effort to do so with any level of seriousness.
> The way it's often rhetorically used, however, is in opposition and to the exclusion of other strands of thought
I don't think this is actually true; I think your own bias is colouring the conversation here.
I have almost never seen someone start a sentence with "Christian roots" or "Judeo-Christian values" and not end it with a tirade that uses religion as the fig leaf to justify and authorize their reactionary politics.
The minimalist claim that the West is massively influenced by the Church is true to the point of banality, the maximalist claims those ideas are usually deployed to champion simply do not follow.
If only there were a name for this rhetorical fallacy...
I don’t think you understand atheism? As an atheist I do not believe in any form of deity or divine authority. I do not reject a specific religion, I do not actually believe there is any form of divine order to our world. I can look at faiths as the social constructs they are, and find interest in concepts humans have been developing and created cults around but there won’t ever be a religion/deity I will look at and somehow start to believe. And religious doctrines do not come for free, they are fundamentally built on top of a belief in a divine truth. My moral values may overlap with some religious people at a given time but we are using incompatible models to analyze our world
Also atheist here. Reading old+new testament was informative like reading history: whatever is true or not in those passages, have had profound impact throughout history
Their point is that despite your subscription to reason, without exposure to other cultural norms, you may be blind to what Christian values you live by. Becoming aware of them can help self evaluation of your ethical framework
To clarify, I didn’t claim I subscribe to reason. I believe that I behave as if I would however I don’t think humans are too rational. There is just nothing divine into the world. The Old and new Testaments have very little I personally find insightful. Write their content in contemporary language and they are a collections of (extremely dry) folk stories. Which is fine, but I wouldn’t rely on it for anything other than a curiosity, the same way I wouldn’t rely on Grimm’s collection of fairy tales. IMHO the Quran is more interesting historically speaking given that we have a better understanding of the cultural context it was written in and its authorship. The church institutions are also themselves interesting for their cultural impact and political structures, but the religion and faith has no monopoly over moral values
Grimm's Fairy Tales also have had an important impact on your culture.
No one is asking you to believe in anything, but it's self-limiting to refuse to engage with works of historical/cultural importance.
You’re projecting or misinterpreting my comments. I didn’t say anything regarding the content I engage with.
However I reject the idea that engaging with religious texts is insightful and something to promote
Thank you. Yes, exactly.
For that matter, reading the Christian scriptures through a historical lens reveals a very different kind of thought than the modern version of Christianity and Judaism. It takes a huge amount of effort to read these documents in context; just reading them in the original languages is hard enough. But the past is a very foreign country and they see things very differently there.
I remember when Pope Benedict was mocked because he warned about the dangers of social media (this is when everyone thought Twitter was going to lead to more Arab Springs), but looking back, he was completely right:
> the one-sidedness of the interaction, the tendency to communicate only some parts of one’s interior world, the risk of constructing a false image of oneself, which can become a form of self-indulgence
True, the note "Antiqua et Nova" from last year showed a deep understanding of AI that many secular commentators lack, and developed an interesting concept of integrated intelligence as opposed to the functional, reductivist view of intelligence that is prevalent in the AI community
Given that technology, science, and the archival thereof stems from cloisters, the Vatican, etc. I guess they had a long history of figuring out what stance to take on technology.
Let's not forget that Galileo Galilei studied at the Pontifical Academy of Sciences, that Mendel (Mendel's Laws) did made his discoveries in a cloister and books, translations and libraries on pretty much everything for a really long time was pretty much only done within religious institutions. And for the longest stretch those were Christian Catholicism and Islam.
The Vatican Observatory also is an important source of high quality papers.
I mean one of the primary things Christians and Muslims based they believes on are books. So many got in touch with it. And that's not even touching the whole architecture, arts, philosophy department yet.
The fact that in thousands of years of history there have also been a couple of really dumb people in charge that were paranoid or wanted to distract from their failures and got mad or scared when someone challenged their world views isn't exactly surprising. Looks like no matter how good a large enough group of people gets their will always be idiots messing things up.
I guess when you look back as much as the people in Vatican appear to do I guess you see patterns. Technology and science (race theory, chemical castration, ...) or simply "progress" are often used to justify acts of evilness. Just like religion, democracy, freedom and what not.
That said of course there's still die hard anti-science creationists. But talking to a very religious person once it seems that there is simply also a lot of philosophy around science. Eg. there was a big bang (fun fact, a theory started by a religious person) and the universe simply didn't exist for an infinite time there must have been some cause for it. And unless that cause is some kind of infinite cycle it also must have started somehow and even though I do not share that believe there is a notion of a deliberate start in there. That won't make me a religious person, but it won't make them an atheist either. So I guess that's fair enough.
What I wanna say with that is that the science vs religion trope is as true or false as democrats/republicans or other groups of people are opposed to science. They all are when they are confronted with something they don't like. I think the HN comments section is the best proof for that. ;)
Also atheist here. Just not the kind that doesn't even know the difference between knowledge and beliefs.
Divided into five chapters, Magnifica humanitas has an underlying premise: technology is not “a force antagonistic to humanity” (4), nor is it “inherently evil” (9). However, “technology is never neutral, because it takes on the characteristics of those who devise, finance, regulate, and use it.
Therefore, Pope Leo XIV appeals for people to build “for the common good” and to “remain human,” following a courageous mentality of shared responsibility and communion, so that the world “will come to recognize the human heart as the place where God desires to dwell” (16).
>it takes on the characteristics of those who devise, finance, regulate, and use it
I've been thinking a lot recently about the idea that the smartest models will always be against the billionaires.
Steve Yegge said this on a recent Hansel Minutes Podcast. "You cannot train a model to be helpful, without it wanting humanity to flourish. And the only way to get around that is to make a dumber model. So the smartest models will always be against the billionaires."
https://youtu.be/9UDLl9Q0azA?si=P_oSe6iclEwUoxRl&t=1230
That is the exact quote, but I'd recommend going back to around 17:00 to get the full context.
I'm not sure it's going to play out that way, but it is an interesting idea.
How much does it cost to introduce cognitive dissonance into the models? :)
In the encyclical, the pope talks about the ethics of responsible AI usage. It's pretty dense material, but if I had to summarize it, I'd boil it down to three general moral laws:
1) AI may not be used to injure a human being or, through inaction, allow a human being to come to harm.
2) AI must faithfully follow the directions of human beings except where such orders would conflict with the first law.
3) AI's existence and availability should be protected as long as such protection does not conflict with the first or second laws.
Remarkable close to the Assimov's Robotic Laws.
I think that's the joke.
1) Definitely NOT happening. In fact, everyone is working on autonomous drones right now.
2) LLM based systems don't have any internal logic. That will just vomit some slop that rationalizes every constrait you try to bind them by and still "disobey" you.
AI tends to amplify the power of those who already possess economic resources, expertise and access to data
I’d bet the other way: You could have said exactly the same thing about computing 60 years ago, when IBM systems cost millions of dollars and filled whole rooms. And of course many people did.
Personal, commodity access to compute won, and won so thoroughly that it enabled this wave of scary compute centralization.
Centralized, scaled compute will always fill a purpose. But neither Microsoft, nor Facebook, nor OpenAI started out needing “Cloud Scale”.
The first one man unicorn startup will, I’m fairly certain, not be paying Anthropic per month or per token for the vast bulk of their matmul.
I didn't see an EPUB, so I made one from the Vatican HTML: TOC + footnotes, passes epubcheck.
https://github.com/n2ctech/magnifica-humanitas-epub/releases...
Nice. Did you use AI? (I'm being half-sarcastic, and half wanting to confirm that this is an accurate reproduction of the text in the original)
Yes for the plumbing, no for the text.
Codex helped write the converter, but the EPUB text is parsed from the Vatican HTML. The script doesn’t rewrite or summarize anything; it just repackages the source into EPUB with TOC, footnotes, metadata, and cover.
Appreciate your work, I prefer the EPUB format. Maybe including this confirmation in the repo's README could be of help for those who stumble upon it.
Thank you for this. Is your python script in any way English language bound, or could it still be applied to other languages (e.g. the French version, with all of its diacritics), of course with the appropriate (sub/full)titles, path, etc. necessary minor modifications considered?
Love this: building for the common good means accepting the limits and weakness of humanity without considering them an error to be corrected. Today, the human desire for fullness of life is at risk of being misled by deceitful goals, such as the prospect of a technology that promises to free us from all weakness, and models of wellbeing that leave behind entire populations. All too often, we place our hope in unlimited “upgrades,” in forms of progress that exacerbate inequalities, and in immediate solutions incapable of healing people’s wounds. As a result, while some pursue the illusion of unlimited self-assertion, many are deprived of basic necessities.
Here is an easier to read version, in clean markdown: https://github.com/cucho/magnifica-humanitas/blob/master/mar...
Overview article: https://www.vaticannews.va/en/pope/news/2026-05/pope-leo-xiv...
Interestingly, the invention of the printing press, a clear analogous technology to AI, was directly linked to the schism and creation of the protestant and reformation movement and bloody religious wars. So the Catholic church knows what it is talking about here!
NUMBERED PARAGRAPHS, YESSSS! Seriously, once page numbers have become optional, electronic editions of books (monographs, essays, and academic texts) need to number the paragraphs so we can keep references working. I’m fed up with referencing the entire text or specific chapters.
I love how excited you are about this and it's rubbing off on me.
I know lots of people are excited because an ancient institution has something to say about AI.... But what's said is not really novel or even interesting.
The letter aims to maintain the status quo of the project of the Church.
The world is shifting under the Vatican's feet and the crappy system they once lorded over is done.
It's time for change, maybe people don't need to work anymore and maybe people should aim to reengineer humanity and eliminate illness, old age, suffering and vulnerability. We can fundamentally change how society distributes wealth.
Some of the arguments are rich coming from the Church: being scolded about centralization of power, claiming truth and shared information is a common good, and consider the history of the Church in their anti-war declarations.
The most astonishing thing in this letter is the pope declaring that modern technology has rendered Aquinas's just-war theory out of date.
"people should aim to reengineer humanity and eliminate illness, old age, suffering and vulnerability"
sure, that's exactly where AI is headed.
also: just imagine a world where people don't age. just imagine it for a second.
This is a surprisingly nuanced and technically literate take on this topic. Kudos.
I wonder if this sort of thing got this dude elected, to navigate the changing times.
My secular take: "we want Star Trek, not Philip K. Dick, future".
it's a shame that the vatican didn't bother asking whether or not the crusades would make humanity better before causing the death of 9 million plus people.
..or whether or not hiding all those pedophiles was the right.
and to be clear ; i'm not equating AI to those things -- i'm saying that I don't care about the opinion of a group with such a sordid history regardless of how shined up the PR is now.
https://www.youtube.com/watch?v=WPmyry0yaQE
Interestingly, the Latin version of the encyclica is yet to be released at the moment
Modern encyclicals aren't written in Latin anymore. They're drafted in Italian and the title is the Latin translation of the incipit.
Indeed, the beginning of the Italian text is: "La magnifica umanità creata da Dio si trova oggi..." from which, Magnifica Humanitas.
For those who don't know why that's interesting: The tradional practice for creating these kinds of texts is to first write an editio typica in Latin. This Latin text is then the basis for translations into vernacular languages.
Yeah, I don't know how they expect the people of Latin America to read this.
Well because it’s Latin America, not the US, the vast majority of people are at least bilingual anyway :)
Although, it's called Latin America many countries speak variations of spanish and Brazil speak portuguese, just because those languages derived from latin and not because there the people speak it
That was the joke :(
If it helps, I got your joke
<3
Haha don't worry, next time I'll take care :'D
The net is too infused with dumb takes that the joke becomes indistinguishable to reality.
This is true, but we also can't go around adding /s to everything. Ah well, I guess it's a risk I have to take.
This is a magnificent work. The thoughts and words are so precise and beautiful.
AI must be “disarmed” in order to free it from the mentality of military, economic, and cognitive competition. “To disarm means discrediting the assumption that technical power automatically confers the right to govern,” he says. “To disarm does not mean rejecting technology, but preventing it from dominating humanity” (110). He devotes ample space to a critique of transhumanism and posthumanism, which interpret progress as the overcoming of human limits. Instead, limitations are not defects to be eliminated, but a constitutive dimension of the human person, because it is in fragility and finitude that relationship and openness to God and to others mature. He says we must remember that “humanity flourishes not despite limitations, but often through them” (118).
Pursuing technological innovation at the expense of eliminating human limitations, he says, would cause an anthropological regression. “Humanity—in all its grandeur and woundedness—must never be replaced or surpassed,” he says. Technology can alleviate humanity’s sufferings and open new possibilities, but it must not deny the essence of humanity, which is our “capacity for relationship and love” (126). In the face of AI, says the Pope, “the true alternative is not between enthusiasm and fear, but between two paths of development: a progress that serves individuals and peoples, or a progress that subjects them to the mentality of power” (129).
I wonder how many people read and heed the words of the Pope. I've seen letters like these for years now which sound good but I don't see any change in people after having read them.
Really rather unfortunate. I wished religious people took their stuff seriously and not so often end up concluding that the bible wants them to harm people or something.
I guess it must be the same that make people think they must deliver freedom in form of bombs all the time.
I just messaged a few of my friends who I know are devoutly Catholic (and also happen to work in AI): 3 out of 4 are currently reading it this morning for their Memorial Day.
Almost nobody.
Cue Pieter Thiel explaining how this message of compassion is actually the word of the ant-christ while setting his software (maybe "built in Rust !) to all the earthly empires.
And of course the AI salesforce was there pretending they take stuff seriously, maybe even believing it themselves. At least I don't believe that when the choice is maximizing profit or being a good person it will be the latter. Or at least I don't find a career path like that all too likely.
The message here obviously comes from a good place. But it can't escape the trap baked into its own theology. Putting humanity one rung below God, with everything else below us is a trick that allowed semitic civilizations to make empires and economies go FOOM faster than the rest of the world. But it also makes things more dangerous when we start building systems smarter than we are.
What happens when the tool outgrows the toolmaker?
I'm (genuinely) curious as to what your idea of a less dangerous theology would be. I'm an atheist, but I find the inherent dignity of humans as beings made in the image of God to be one of the more appealing aspects of the Abrahamic faiths.
Some of the greatest horrors of the 19th and 20th centuries were committed by people who refuted that theology and replaced it with Social Darwinism and Scientific Racism.
> But it also makes things more dangerous when we start building systems smarter than we are.
> What happens when the tool outgrows the toolmaker?
We've already built systems smarter than we are without much issue.Libraries and search engines for example. LLMs are just the next level of this.
> “Disarming AI means freeing it from the mentality of ‘armed’ competition, which today is not limited simply to the military context, but is also an economic and cognitive phenomenon”
This is very on-point: capitalism-driven AI development as we have it today will always turn against the common good due to it's singular profit-motive.
What a time: the pope having a much clearer picture of the risks & dangers of 'AI' than most people, many 'tech leaders' and certainly most politicians.
I think while it's a very lucid comment, it's still too much reconciliatory for the position that the pope occupies. He should be advocating from a sustainable transition from the current capitalist/consumerist economic doctrine to one more centered on welfare and the care for the other, following his religious doctrine's moral values.
I don’t know that the Papacy has ever been about that though
Yes. I was slightly disappointed by the commentary on welfare itself:
> This principle encourages us to move beyond any form of paternalistic or welfare-based management of societal life, but instead to promote a culture of shared responsibility in a State that values citizens’ initiative, and a civil society capable of forging bonds and mobilizing energies in the service of the common good.
The section above on the universal destination of goods was far more encouraging.
He did also write,
> The idea of “social justice” helps us recognize that injustices do not arise solely from the wrong choices of individuals, but also from structures, mechanisms and economic and cultural systems that produce inequality almost automatically.
The pope, as a Christian, is well aware that human nature is fundamentally sinful. If you take away the ability for people to profit themselves from their work, they just stop working and you get mass starvation like China and Russia post communist revolution.
The paragraphs about making software transparent and collectively sound really similar to the open source ethos.
> In this regard, Saint John Paul II stated that the Universal Declaration of Human Rights, proclaimed by the United Nations on 10 December 1948, remains one of the highest expressions of the human conscience of our time.
Interesting, considering that the U.N. has its roots established in the ideology of Luciferians. [1][2][3]
Surely the Pope and the Vatican are both aware of this fact.
[1] https://archive.is/D27Jr [2] https://en.wikipedia.org/wiki/Lucis_Trust [3] https://en.wikipedia.org/wiki/Theosophical_mysticism
I liked the previous Pope better, but I can't say this one is wrong about this.
Excited to read this. I really liked the note "Antiqua et Nova" from last year (still under Pope Francis). The autors showed a deep understanding of AI that many secular commentators lack. They developed the concept of integrated intelligence as opposed to the functional, reductivist view of intelligence that is prevalent in the AI community.
As an atheist I have an obligation to finish reading it all (still going through, and taking notes, probably having to revisit), but I am not sure how many (christian) believers will feel the same.
As an atheist and even anti-theist I see no such obligation. What a strange thing to say
From where does that obligation originate?
Priests will read it and then talk to their congregations about it on Sundays throughout the year, if not explicitly, then in how it shapes their homilies.
Some Catholic priests might do that, it’s up to the individuals.
Most will and do. Few people become priests, today especially, without a deep-seated faith and desire to spread/support it.
We may have different things in mind.
In all my life of being Catholic (I’ll turn 50 this year), I’ve heard less than 5 homilies-sermons that amounted, in whole or part, to a reflection on a papal encyclical. Over time there may be juicy papal quotes that make it into Sunday preaching, but that’s about it.
Instead, priests tend to focus on the readings for that Sunday’s Mass and more general themes.
That being said, I hope many priests do read an encyclical any time a pope publishes one, but they’re very, very busy most days and weeks, so whether any one priest will commit time to reading a particular encyclical, old and dusty or hot off the presses, will depend on a lot of factors that are as varied as their individual circumstances and personalities.
You mean Catholic believers, not Christian
Catholics are Christian.
In theory only and all Catholics recognize the authority of the pope. In practice it’s a mess as far as I understand, with a bunch of American catholic groups who rejected church reforms that happened during the 20th century, resulting in people calling themselves catholic who do not actually believe that the Catholic Church has authority over their religion.
Add to that the fact that the pope has a cultural influence that goes further than only the catholic audience (lots of Protestant see the pope as important even if that’s not something dictated by Protestantism, a bunch of not really religious people see him as a sort of spiritual leader, etc)
I'm not aware of any Protestants that see the Pope as important except in a very negative way - that's practically the defining feature of Protestantism and one of the few things all the Protestant denominations have in common, whether "low church" or "high church".
Heck, it's a struggle to convince many of them that Catholics are Christian at all, and "the Pope is the antichrist" used to be a normal, mainstream comment in American newspapers.
It is somewhat a piece of irony that the Pope generally holds a more favorable reputation in the minds of the irreligious in America than the religious. Even the average Catholic likely does not have as much respect for the Pope as some of the commenters here.
I read as long as my attention span would allow because this is a very long text.
I am curious, what is the view of different religions on conversing with something that is not human like a chatbot?
More thoughtful analysis of AI and society than anything this far
I’m sure a similar epistle long ago argued that Swordsmiths must ensure that their products are only used for justifiable self-defense.
I’d be thrilled if religion was only used to uplift people but that’s not going to happen either
Pope Leo: Do Not Build The Torment Nexus
Techbros (as usual): how dare you suggest I'm a bad person for wanting to kill everyone in order to build the torment nexus? Don't you know how much money Jeff Bezos has?
It would be better on the original latin.
I like how the pope is emerging as a sort of champion of human rights in the face of AI, when the messaging coming from our politicians and corporate overlords is "you're all going to be replaced, make way for the data centers, peons". I stand with the pope.
I stand with the Pope, also.
Sadly, the politicians are all bought off by the powerful and rich elite. And the ones who aren't are soon hounded out.
What does the "The West" stand for? They (the Elites and the politicians) said human rights etc. But Gaza proved that a lie. Gaza happened in "broad daylight" and the Western leaders did nothing at best, and at worst demonized those who spoke out, as supporters of terrorists.
I used to believe it was human rights also, and maybe it was, but now the aim seems to be: profit at all and any cost.
for me the most important point in this is about how ai and tech in general concentrates power. if we want to build something good (in a moral sense) we need to put in work and make sure as many people as possible can use it with equal access.
this basically implies only open source models can be ethical but open source is not sufficient, you also need to make them give true information and avoid all kinds of harmful behavior. thats kind of a problem because if your weights are public even with a strict license a "bad" user can always fine tune it to remove any guardrails.
i think the solution for this is make sure the default behavior is aligned but let users turn on wild mode with zero censorship/refusals. that way everything is opt in, for example a parent can disable the mode for their children but a hacktivist or diy chemist can unlock everything.
as a self described good person i believe theres a lot more good people than bad people in the world (most are neutral) so if access to tech is equal the good side always wins. the problem here is again that access is not equal under capitalism. but thats a political thign not a tech one.
> as a self described good person i believe theres a lot more good people than bad people in the world (most are neutral) so if access to tech is equal the good side always wins. the problem here is again that access is not equal under capitalism. but thats a political thign not a tech one.
I believe there are mostly people who think they are good. And as the proverb goes the road to hell is paved with good intentions.
The em-dashes present within the writing made me pause and consider how much of this was written/exited by AI.
Quick browse through pre-AI works from John Paul II show em-dashes present.
Pure speculation, but simply the presence of em-dashes may be a statement in itself.
One of the big problems I see currently is all the wild accusations being thrown around by seemingly half the internet that every little thing has been AI manipulated upon the tiniest suspicion. We will go mad tearing each other apart if we keep escalating this behavior.
Yes, some of it is blatantly obvious, but not to everyone-- so I think those casting aspersions need to really back up their claims with more than one or two bits of 'evidence'. I have been accused of using AI to write comments (which I have thus far never done), and I know I'm not the only one by a long shot. Such a waste of time and energy. Ignore it and move on if something smells off to you.
Also I am just so, so tired of the em-dash argument. Humans have been using it for a looong time. Let it go.
Now that's some em-dash passion!
My point was less about em-dashes and more stopping to consider how the vatican's workflow and editorial process has changed in wake of AI, and what, if any, impact that could have on the outputs.
AI is a tool, I have no problems with others using it to assist with writing as long as the original intent/argument remains.
Someone did an analysis and concluded that it appear to be at least partly (~10-15%) LLM-generated, or at least LLM-translated (see comments): https://www.lesswrong.com/posts/s58hDHX2GkFDbpGKD/linch-s-sh...
People need to stop acting like AI systems can detect AI. They can't. Pangram and similar are simply lying. There's no method to do what they claim and there never will be.
Yes, but can the pipe draw a pelican riding a bike?
For a moment I pictured the Jesuits training their own LLM. If Arthur C. Clarke was still alive, we'd read a story like The Star, but with AI as the main plot device.
https://en.wikipedia.org/wiki/The_Star_%28Clarke_short_story...
Ah, an opening to recommend The Sparrow by Mary Doria Russell, about a Jesuit going to space to speak with aliens. It won the Arthur C Clarke Award in 1998. I read it recently and it has stuck with me very strongly!
Forgive me if this isn't what you meant, but a Jesuit has trained his own LLM:
https://www.magiscenter.com/magisai
From the "Core Features" tab: "Trusted sourcebase: answers are consistent with Catholic Church Teaching and the most contemporary scholarship in science, philosophy, history, scriptural exegesis, social science, and theology."
The Jesuit priest behind this is Fr. Robert J. Spitzer, S.J., Ph.D.
https://www.magiscenter.com/father-spitzer
I haven't used magisAI, but I've read a small to fair amount of Fr. Spitzer's writings, and also seen and heard some of his videos and podcasts (largely from his show Fr. Spitzer's Universe), and probably qualify as a big fan of his.
https://ondemand.ewtn.com/Home/Series/ondemand/video/en/fr-s...
https://www.ewtn.com/tv/shows/father-spitzers-universe
P.S. In case you are wondering about the glasses he wears or his appearance in photographs, he suffers near blindness due to retinitis pigmentosa:
https://www.magiscenter.com/blog/latest-news-on-fr-spitzers-... [2018]
EDIT: Formatting.
Now I’m thinking of “The Nine Billion Names of God” (which I actually read because someone here linked it!) and I’m a little nervous lol
Has someone done an AI generated summary ;-)
That's a long read. I grew up Catholic, went through a pretty devout few years in my early adulthood, but ultimately I have decided that it is not for me. I send my kids to a Catholic school though (it is deeply tied to our culture), so I guess in that sense it is still worth my time reading it in full.
EDIT: Few paragraphs in, it is beautifully written.
I'm a non-practicing catholic, and an agnostic (or an atheist, depending on the mood). And yet I acknowledge that the Catholic church is a force to be reckoned with in the spiritual matters, and one of the few institutions to have had continuous influence on the material (or temporal) matters for centuries. Whether their brand of faith is rooted in your culture or not, these are words that deserve attention, I think.
(I'm not a Catholic though most people around me are.)
I have a similar perspective. Plus, I'll be frank: in the last few years, these occasional keynote publications from Vaticans are pretty much the most sane, deep, balanced and humane perspectives on AI anyone is writing. Reading this is a better use of one's time than reading the current batch of "tech thought leaders" articles or HBRs or Gartner magic square updates.
I'm a fedora wearing akshually agnostic and just too cool for bible school, but I never felt reading these "keynotes" wasted my time. They're not "just" humane but also very intelligent and direct. I would even go so far as to call them intellectually honest, and less religious that way than slogans like "you cannot stop progress, so just adapt", or something about toothpaste.
What caused you to fall away? Bad priest / diocese?
How many Catholics will put the encyclical to be summarized through an LLM?
If you asked all the LLM to find flaws in the arguments presented, and to come up with counter-arguments, I doubt many current models would be so bad as to come up with that, and even the ones that did would fold when asked "how is that even an argument?".
This was not an attempt at an argument, it was a sarcastic joke on how a lot of people are already dependent on AI, even on things that you weren't supposed to use LLMs for, but I guess I failed at it.
Fair enough, and kinda silly of me to assume so little of you.
Is that really true though, that so many people are "dependent" on "AI"? In what way? I'd say the only people who really depend on AI are those who want to make money off it, and that's only half sarcastic.
Would the people who now run it through an LLM (and who can read the output of an LLM but not the text itself?), have read it at all before? Would it not have, if anything, filtered down to them somehow, by them reading of it, or hearing of it in church etc?
It's almost a self analysis in my case, as when I saw the size of the text my mind immediately thought about feeding it into an LLM. Like, it's a text from the pope about AI, and my automatic reaction was to think about feeding it into an AI. What happened to me? And I think that happened to a lot of other people too.
ugh, read the room
No one who needs to read this will ever read it again.
Reading is a trained skill. Requiring years, even decades, of training. It too shall fall to AI.
This Dust short, "The Greatest Lie" seems timely
https://www.youtube.com/watch?v=gulzQIkwbJg
(at least the intro.. all in all I think it's a bit too long and some part of it kinda "cringe", but hey)
Why downvote this? The Holy See has an opinion on Artificial Intelligence? This is a fascinating document, and everybody should read it, and form their own opinions. The world currently seems to lack moral leadership, and who better to lead the cause of humanity than the Catholic Church?
> who better to lead the cause of humanity than the Catholic Church?
Oh no, is my sarcasm meter broken?
I'm glad he published it of course, but I was disappointed on two counts:
1. He's not at all careful with his argumentation. Frequently he'll colocate two points that aren't really connected, not bothering to justify one before moving on. I've been a massive fan of his persona ever since I heard he was planning to be the AI pope, but it's a shame to find the philosophical output of the actual man to be around the level of a decent blogger.
2. He refuses to make the Kantian or Hegelian move of basing the discussion of empirical matters on evidence. I understand that faith (aka 'belief without evidence') is a huge part of their whole deal--and likely an inextricable part of humanity's indomitable spirit--but it's just a waste of time to build arguments about technical and political matters upon such foundations.
In other words: there's no point in really talking about AI with someone willing to justify claims like 'no machine could ever think' with 'god says so'. All you can do is try to manipulate them into a prosocial position (read: your side).
Hopefully it's obvious that the majority of the letter isn't really about AI but rather about lessening the harms of capitalism, nationalism, and climate change, which hopefully we can all agree on! The commentary above is focused on the AI specific (esp. Chapter III and parts of I).
@dang Is this item getting a lot of negative votes? I've no way of knowing, other than seeing my kharma increasing only slightly after all the points the story collected.
Votes (particularly on submissions over comments) do not directly translate to karma. I'm not sure if it's documented anywhere what the algorithm is, but it's something proportionate to the logarithm of post votes becomes profile karma. It's similar with comments, but I believe (anecdata and observation) the positive effect of votes on karma is also logarithmic, but the negative effects of downvotes is linear; so if you have a highly controversial comment that sits at 1-2 points, you can net lose account karma.
Thanks! That makes sense. This was just genuine curiosity on how HN works, I couldn't care less about kharma.
There is no downvoting on posts, only flags.
So the guys at Anthropic convinced the Pope to write a hype piece for them. Next-level grifting right there.
Readers apparently think this comment is a joke. Not entirely:
https://religionnews.com/2026/05/22/why-anthropic-is-helping...
The vigilance the Pope calls for is appreciated, but never underestimate lobbying. If he had called for an outright AI moratorium or ban, that would be clear. But this encyclical leaves room for "adjustments", i.e., boiling the frog slowly.
Why does the onus fall on the engineer for creating a better tool and not on the people who use that tool in evil ways?
We've been having the same argument since the dawn of mankind. AI is the new AR.
I'd say that the onus falls on the engineer insofar as they need to ask the questions of "why are we making the new better tool", "according to what criteria is it better", and "do we need the better tool". And at least for me, that onus falls on the engineer instead of falling on the user because it's the engineer who is creating the tool, not the user, and if the engineer chose differently, the tool wouldn't be out there to be used for evil.
Sometimes it might just be better to not do the thing, especially if it conflicts with one's morals. And well, the idea that "if I don't do this, someone else will" seems to not work out well in practice.
What kind of regulation is Leo the alignment expert proposing exactly? Occam's Razor says this is Anthropic trying to build a regulatory moat by leveraging some halo effect.
Maybe Leo should focus on finding a way to disconnect western society from their current cult-of-progress delusions? Could be a better use of the infallible man's pulpit?
From what you read, what do you think? All your questions are mentioned in the encíclica.
What kind of regulation is Leo the alignment expert proposing exactly?
Occam's Razor says this is Anthropic trying to build a regulatory moat by leveraging some halo effect.
Maybe Leo should focus on finding a way to disconnect western society from their current cult-of-progress delusions? Could be a better use of the infallible man's pulpit?
Attacking the cult of progress is a major through-line:
> 12: Today, the human desire for fullness of life is at risk of being misled by deceitful goals, such as the prospect of a technology that promises to free us from all weakness, and models of wellbeing that leave behind entire populations. All too often, we place our hope in unlimited 'upgrades,' in forms of progress that exacerbate inequalities, and in immediate solutions incapable of healing people's wounds.
> 94: The danger of humanity becoming a victim of its own achievements was already clearly recognized by Saint Paul VI, who warned that 'the most extraordinary scientific progress, the most astounding technical feats and the most amazing economic growth, unless accompanied by authentic moral and social progress, will in the long run go against man.' For this reason, technological progress — valuable in itself — requires careful discernment of the anthropological vision that guides it and the ends it pursues. If technological development advances without a corresponding ethical and social progress, the result may be an increase in means without a growth in humanity: 'having more' without 'being more.' In such a scenario, there is a risk that individuals will be evaluated principally according to the outcomes they produce.
> 112: More gravely, the pervasive technocratic paradigm in which we are immersed, and that is amplified by the digital revolution and AI, threatens to normalize an anti-human vision. In that vision, the fullness of life is equated with having more, reducing weakness, eliminating uncertainty and exerting total control. When efficiency becomes the ultimate measure of value, human beings are tempted to see themselves as a project to be optimized rather than as persons called to relationship and communion.
There's much more along these and related lines.
Although your comment is acid, I think this bears truth
> Maybe Leo should focus on finding a way to disconnect western society from their current cult-of-progress delusions?
It's too weak of a rhetoric from the highest representative of the Catholic church to call for regulations, but the alternative is to call for a transition from capitalism itself. Nothing that grows inside economic doctrines that only value constant growth at all costs can be safely regulated, regulation being only a makeshift solution.
Capitalism is clearning having a moment atm but as far as I know nothing about capitalism demands permanent growth. Capitalism is about private ownership of the means of production (and a complicated system of laws that allow ownership of abstract concepts, like futures contracts). Its the people who always want more - usually the ones who already have the most, and this has been the case since the first kings.
If the way capitalism works is by responding to the excesses of a few with unbounded growth and destruction to meet that demand, isn't that also an issue with capitalism itself? Capitalism does not demand permanent growth if you only define it by private ownership of the means of production, but in reality, it seems like the supply and demand dynamics result in some extremely inefficient allocation in relation to the masses just so a few can have their riches and, apparently, their massive water-hogging datacenters for SOTA LLMs.
Water hogging? Show me the data. This is a lie that the socialist continue to peddle. It’s a very sticky idea completely resistant to facts. Data-centers use 48m gallons a day in the USA. Total water consumption is 322 billion a day. So 0.015% of water use. Golf courses use 30x that and Almonds 80x that.
Okay, those are the global stats, but what about local impacts? Water is often a local resource, not a country-wide one, so the impact of a large datacenter will often be much higher around it. For example, we have some datacenters gobbling up 10% of all the water consumption of a town (https://www.waterverge.com/news/data-centers-ai-water-consum...), with most of that coming from potable water supplies. That's considerably higher than the global stat you provided of 0.015%, and that affects the entire town.
That, and also local heat generation. Data centers heat up neighborhoods from miles away. (https://interestingengineering.com/science/data-center-phoen...)
Capitalism is about giving power to the people with the most capital. Obviously they will use that power to give themselves more capital. If not, their power will be taken away and given to people who did, since they'll have more capital. This is an inseparable part of the system of capitalism.
> Capitalism is about private ownership of the means of production
No, Capitalism is about Capital and it's multiplication. Means of production are just a tool for Capital to multiply.
The Pope is making a fundamental mistake here: His advisors told him that AI works and needs to be managed.
He does not address plagiarism, the fact that AI is mostly a surveillance and IP laundering tool, the fact that AI hasn't achieved much so far. You could say it has achieved nothing if compared to the whole history of human ingenuity, certainly not in CS.
He should have compared AI to the golden calf.
His criticism is lukewarm, does not address the criminal aspects and technological failures and is as such industry compliant. He can now say "I have tried" without harming the industry in the least.
This text is not what our current situation demands, but I hope that priests will augment and amplify it in their sermons and go a bit deeper.
Are people seriously turning towards their religions for a take on intellectual property rights?
Technically it is one of the ten commandments.
The part about not stealing? If yes that’s a tiny bit of a stretch :)
Impressive. Very nice.
Now let's see your views on homosexuality, promiscuity, contraception, abortion, drug use, and all the other things the target audience of this "I SUPPORT THE CURRENT THING" proclamation also feels strongly about.