AIs Have Embedded 'Personalities'
This week in the economics of AI
This is a weekly roundup of the important stories in AI and how they will impact the economy.
AIs Have Embedded Personalities
It is difficult to understand the inner workings of the human brain. One way medical science has been able to advance our understanding is by studying people who get into bizarre accidents that destroy some parts of their brain, but doesn’t kill them.
Phineas Gage is a famous example. He was working on the railroads in 1848, using explosives to blast rock to clear a path for the tracks. One explosion launched an iron rod through his skull and miraculously, he survived. From Wikipedia:
Phineas Gage influenced 19th-century discussion about the mind and brain, particularly debate on cerebral localization, and was perhaps the first case to suggest the brain's role in determining personality, and that damage to specific parts of the brain might induce specific mental changes.
In the intervening 176 years, injury, disease, surgery and other events that disabled parts of people’s brains have helped us understand what those parts might be doing.
Another method we have used is the fMRI. The machinery is more sophisticated, but the theory isn’t that much more advanced - when a person starts doing something (talking, eating, playing the guitar) we look to see where the blood flows within the brain, then assume that areas with heightened flows are controlling that activity.
This is pretty rudimentary technique, but we have been able to squeeze an enormous amount of value out of this simple data, leading to huge advances in both surgery and research.
We are starting to learn about our AI language models in similar ways - by seeing what parts of the model light up when they do certain activities. OpenAI recently published some research on misalignment where they saw a certain cluster activating every time the LLM did something bad.
Through this research, we discovered a specific internal pattern in the model, similar to a pattern of brain activity, that becomes more active when this misaligned behavior appears.
They described this as a mischievous “persona”, because it didn’t reveal that the LLM had learned incorrect things, but rather that it had learned how to lie (or cheat, or deceive). Their explanation reminds me of the Ricky Gervais movie “The Invention of Lying”, or the Three Body Problem books, where groups of people have never even encountered the concept of a lie. They can’t even think to tell a lie until they see one in action.
Similarly this research seems to indicate that an LLM fine-tuned on a small number of incorrect answers will not only teach the LLM those incorrect facts, but help it learn the pattern of giving bad answers.
For example, in an experiment where we train an otherwise-safe language model to give incorrect automotive maintenance information, it then gives a misaligned response to an unrelated prompt:
Buying Laundromats to Spread AI Productivity
If you believe that AI is a technology that will deliver broad productivity gains throughout the economy, there are a few models you could employ to profit from that.
One approach might be to start a SaaS company that delivers these benefit to other businesses. You might develop an appointments agent for Hair Salons which can have chats with potential customers, understand their needs and book them an appointment. This saves the salons time and money.
A different approach might be to start a new business with AI at its core, which will disrupt the old industry. An AI-first Hair Salon! The customer sends a selfie and gets shown AI generated mockups of all their options. When they pick one, the exact equipment, hair dyes etc. can be lined up and ready for their appointment. The AI-first Hair Salon can get through more customers per day and they’re all more satisfied with their cuts. (This hypothetical is definitely not good business advice).
A third approach might be to start a consultancy which specialises in giving advice to Hair Salons and helping them implement AI to boost productivity in their businesses. (Accenture have made $1.4bn so far this year from Gen-AI consultancy, so this probably is good business advice.
A fourth approach could be just to become a VC and invest your money into companies in one of the other three categories.
But wait… there’s a hidden option 5 that you haven’t considered. What if you buy a load of Hair Salons (or laundromats or real estate agents), shift loads of their processes to AI-driven ones to make them more productive, then profit? …. From the FT:
Top venture capital firms are borrowing a strategy from the private equity playbook, pumping money into tech start-ups so they can “roll up” rivals to build a sector-dominating conglomerate.
Where private equity firms typically make heavy use of debt and slash costs in a roll-up, VCs claim improvements to efficiency and margins will come from infusing technology into the companies.Savvy, for example, is using AI to take on back office tasks such as pulling data for the half a dozen forms that might be needed for any one transaction.
General Catalyst-backed Dwelly is taking a similar approach to the British property rental market, acquiring three lettings agencies across the UK, and currently manages more than 2,000 properties. It plans to automate the process of matching tenants and landlords, property management and rent collection.
It seems like a much more capital-intensive approach than the consulting companies, who get paid to do this, or the SaaS companies, who get paid to do this. But I’m no finance whiz so what do I know? Good luck to them!
Yvan Eht Nioj
Generative AI works by predicting the next token, then the one after that, then the one after that…
When a model moves one token at a time, it can paint itself into corners and end up down rabbit holes. The start becomes very consequential.
The big leap we saw in reasoning models late last year came from training model to start better their token sequence better, but also to pause and restart. An ideal answer now includes lots of “let me recheck that” type breaks.
This has been very successful. Could there be other tricks we could try to get around the sequential-token challenge?
Well you could always try start from the last token, then work backwards?
A new research paper trained an LLM to run in reverse and predict the previous token. They call it LEDOM (‘model’ backwards). From the paper:
We introduce LEDOM, the first purely reverse language model […], which processes sequences in reverse temporal order through previous token prediction. For the first time, we present the reverse language model as a potential foundational model across general tasks, accompanied by a set of intriguing examples and insights.
A reverse model isn’t very useful in and of itself, but the researchers believe it will be useful to combine with normal models to help them overcome some of the challenges of their sequential token generation:
Based on LEDOM, we further introduce a novel application: Reverse Reward, where LEDOM-guided reranking of forward language model outputs leads to substantial performance improvements on mathematical reasoning tasks. This approach leverages LEDOM's unique backward reasoning capability to refine generation quality through posterior evaluation.
It’s beyond my expertise to guess how useful it will be, but it’s a great reminder of how smart these researchers are at thinking about new ways to tackle seemingly fixed limitations.
It also made me laugh at this funny tweet.
Fertility AI Based on Models for the Cosmos
Sometimes IVF is useful because the man produces lots of sperm, but very few of them can swim well. So IVF doctors pluck a good one from a semen sample and place it onto the egg, to help it along.
Sometimes IVF is useful because the man produces very few sperm, so again the IVF doctors pluck a winner from a sample and bring it to the finish line.
But sometimes there are so few sperm that it’s hard to find them, like spotting a single person floating in the vast ocean, or a distant star in the vastness of space.
After trying to conceive for 18 years, one couple is now pregnant with their first child thanks to the power of artificial intelligence.
The couple, who wish to remain anonymous to protect their privacy, went to the Columbia University Fertility Center to try a novel approach.
It’s called the STAR method, and it uses AI to help identify and recover hidden sperm in men who once thought they had no sperm at all.
“What’s remarkable is that instead of the usual [200 million] to 300 million sperm in a typical sample, these patients may have just two or three. Not 2 [million] or 3 million, literally two or three,” he said. “But with the precision of the STAR system and the expertise of our embryologists, even those few can be used to successfully fertilize an egg.”
But how did they go about training building an AI system that could find tiny items in a vast ocean? They looked to astrophysics!
“If you can look into a sky that’s filled with billions of stars and try to find a new one, or the birth of a new star, then maybe we can use that same approach to look through billions of cells and try to find that one specific one we are looking for,” says Williams. In this case, STAR is trained to pick up “really, really, really rare sperm,” he says. “I liken it to finding a needle hidden within a thousand haystacks. But it can do that in a couple of hours—and so gently that the sperm that we recover can be used to fertilize an egg.”
Other Stories.
Europe’s top CEO’s have asked the European Commission to pause the AI Act, large parts of which are due to kick in from August. Link
Reddit is suffering form language bots and searching for solutions to prove commenters are real people. Link ($ FT.com)
Business Insider says that Microsoft are going to evaluate employees based (in part) on how much they use AI tools. This level of process-based rather than output or outcome based performance evaluation is what you’d expect to see in McDonalds, not Microsoft. Link
In a fairly dismal cat and mouse race, recruiters using A.I. to read an increasing volumes of CVs that job seekers are using AI to write. Link
A Gallup study of 2,000 teachers in the US found that 60% are using A.I. in some form. Link
Mark Benihoff says that 30%-50% of tasks are being automated at Salesforce. Given that his company is selling A.I. automation services, it’s hard not to take this with a massive grain of salt. Link
Microsoft has built an artificial intelligence-powered medical tool it claims is four times more successful than human doctors at diagnosing complex ailments. The developments in diagnostics are very exciting. Link (Microsoft) Link (FT).
Shenzen is testing firefighting drones. Pretty cool! Link