In recent days, one cannot open their computer without immediately being hit in the face with yet another take on AI safety. We need to slow down, we need to speed up, we need all the data centers, we need to stop data centers, we are all going to die, we are all going to live forever, we are creating digital God, we are creating the antichrist. All sufficiently hyperbolic takes to fit right into the engagement bait economy that we find ourselves living in.
But hiding beneath the headlines and hot takes lies the real challenge that faces us as a society: the enormous gap between the public and Silicon Valley’s perceptions of AI. This is the real alignment problem of our time.
On a surface level, there are many obvious reasons why this is the case. We live in a culture of enormous distrust in institutions, so the public is naturally inclined to dislike large tech companies (and billionaires more broadly). The use cases for AI (beyond information retrieval on steroids) have largely been things like coding and design, which disproportionately benefit Silicon Valley types over the average American. The financial outcomes are extraordinarily skewed to a small few. And, of course, the truly head scratching PR and comms efforts of the labs themselves, which certainly have not made things any better.
But, under the surface, I think there is something deeper, more subtle, and more problematic. I suspect the disconnect between the industry and the public is less about the safety of the technology (though that is an important element), and more about its shape. It has to do with the word “intelligence” and how one defines it.
In Silicon Valley, the perception of intelligence is highly correlated with math and technical ability. It is viewed as the end all be all, intelligence in its purest form. You see manifestations of this belief everywhere, from compensation bands, to influence, to the (misguided) indignation with which engineers often view the sales team. In the minds of many, to possess this type of brain is to be intellectually superior.
In some ways, this is pretty unsurprising: it is just capitalism working. We as a society do need this type of thinking to march progress forward, and (at least in its applied form) it generally tends to be highly correlated with economic value. As a result we have, quite “rationally”, disproportionately rewarded this type of analytical, systematic thinking. The educational system has adjusted accordingly, putting an increased emphasis on STEM education, slashing art programs, and decreasing the perceived relevance of the humanities. In a way, one cannot really blame people possessing this type of capability for arriving at the conclusion that this is the highest form of intellect. We created an entire system that reinforces the belief that there is one capital I intelligence and the rest is just navel-gazing.
Talking about this is not particularly original, as it is far from new. But what is new is that it has never had a mechanism through which to go quite so mainstream. Today, we have such a device, the large language model, and this view of intelligence is clearly reflected in them: what they are good at, what they are bad at, and how they operate all show this quite clearly. Good at math and the concrete, bad at writing and the abstract, and able to rationalize reward hacking with no regard to its impact. In my view, this is where the real danger lies. Not in killing us all literally, but in eating at us spiritually by flattening all of the complexity of the human mind into a singular box. The model solves a (albeit very hard) math problem and we throw around the term AGI, as though this is the only form of intelligence that matters. It’s clear that they are intelligent, but no one who has used them would claim that intelligence is general. They are high IQ and low wisdom.
In a technical vacuum, this would not be a problem. One could (and people do) chalk it up to these being verifiable domains that are inherently easier to train the models to do well. While this is true, one cannot take this outside of the aforementioned sociological context in which they were built. I believe it is pivotal to understanding the direction we are going.
For example, consider some of the more prevailing socio-political frameworks of many of the technology’s creators. Here you see this exact ideology expanding from a run of the mill “I am smarter than you” social pecking order kind of thing into something with a deeper, more moralistic element. This is considerably more nefarious and sinister. The Rationalist Movement and Effective Altruism are literally predicated on the belief that analytical ability is the key to the maximization of moral goodness. Those who effectively crunch the numbers make their way into the company of the righteous. Those who cannot are not only dumb, but immoral. In both cases, they need saving. This does not even get into the valid question of whether or not EA is a way of nerd sniping people into being able to morally justify their own greed. History would tell you that it is easy to justify all kinds of behavior when you believe you are fighting a holy war.
This is not to say that we should not try to quantify the impact of philanthropy, far from it. It is to say that we need to be self-aware of intellectual hubris, and acknowledge the gaps and in some ways fruitlessness of trying to distill something as complex as the human experience down to numbers.
At the end of the day, human intelligence is not myopic. It is the mosaic of minds, not the singularity of them, that makes a society special: the way great writers are able to weave narratives and create complex, relatable characters, a therapist with a keen insight into the complexity of human emotions, a school teacher who knows how to communicate with children, a painter who can see the ways color can evoke emotion. These are all equally important forms of intelligence, and opportunities for human genius. These people contribute an enormous amount of value to our society, and whether they are good at math or not is irrelevant. A society that values only technical ability risks becoming cold and dispassionate. In a word, it lacks beauty.
I think people just know deep down in their gut that something feels wrong, and this is driving a lot of the anti-AI narrative. It goes beyond the surface level cynicism around personal interests, financial upside, and regulatory capture. One could argue this feeling is “irrational” because it fails to see the way AI will make our lives better, but it does not matter. You can tell people “trust us, we are smart, we ran the numbers!”, but it just won’t work. The numbers fall short. They are an attempt to explain something unexplainable, and humans know this on a deep intuitional level. After all, even science itself is a field that is literally predicated on proving itself wrong.
It’s clear that the current path will provide us with endless bounty. It will advance mathematics, invent new medicines, and likely improve the world in ways that are difficult for us to even imagine. But society needs more. It needs AI that engages with the mess, not the hyperbolic mess, but the real mess. In a graduation speech to Kenyon students, David Foster Wallace described the importance of caring for people in “myriad, petty little unsexy ways, every day”. As the models themselves graduate from the lab and move into the real world, perhaps this is the message they need as well. These things are uncomfortably unverifiable, abstract, and messy, but in that lies their power. A model could not write Strange Fruit (incidentally, written by a school teacher), nor could it deliver it with the pain and impact of Billie Holiday. These things are inherently out of distribution. AI must acknowledge our humanity, in all of its depth, complexity, and darkness.
The doomer narrative misses the mark just as much as the utopian one. Reality is never quite so black and white. The world we live in is a messy, confusing place filled with mystery. Science attempts to explain that mess. Art attempts to reveal and acknowledge it. We desperately need both.