Episode Summary
Executive Summary: This episode argues that AI’s biggest near-term change may be less about replacing humans than reshaping how humans work with language, ideas, and institutions. Through prompt engineer Anna Bernstein, Anthropic CEO Dario Amodei, and Wharton professor Ethan Mollick, it shows AI as a powerful but immature tool that needs context, guardrails, and human direction—while promising major gains in creativity, entrepreneurship, education, and science.
Main Topics: Prompt engineering as a new job: Anna Bernstein explains how her role at Copy AI emerged from the need to translate human intent into effective AI prompts, making language precision the key skill. How large language models work: The episode explains LLMs as systems trained on massive text corpora to predict next words, learning grammar, facts, and patterns without true understanding or wisdom. AI safety and alignment: Dario Amodei describes Anthropic’s effort to make Claude safer through constitutional AI, emphasizing that models can absorb harmful content unless constrained by explicit principles. AI’s scientific and societal upside: Amodei argues AI could accelerate breakthroughs in difficult fields like biology and cancer research by helping humans manage complexity beyond individual comprehension. AI as an entrepreneurship and ideation tool: Ethan Mollick demonstrates how AI can serve as a thought partner for constrained ideation, helping users explore business ideas faster and with more variation. Education and human-AI interaction: The episode suggests AI may help learning by generating tests, feedback, and prompts, but only if humans remain actively engaged rather than outsourcing thinking. Regulation and future governance: Executives call for oversight and government regulation, acknowledging industry incentives may conflict with the public interest and that AI policy is still unsettled.
Key Arguments: Prompt engineering is a real, emerging job because AI output depends heavily on the exact wording and context humans provide. Large language models learn by predicting the next word across massive internet-scale datasets, which lets them accumulate broad knowledge but not human judgment. AI systems need filters and alignment methods because raw models can generate offensive, dangerous, or unhelpful outputs. Anthropic’s constitutional AI tries to encode explicit values into model behavior, making the model evaluate itself against a written principle set. AI could dramatically accelerate complex scientific work, especially in biology and disease research, where pattern scale exceeds human intuition. The best current use of AI is often as a collaborator or “thought partner,” not a replacement for human expertise. In entrepreneurship, AI can reduce inertia by generating options, next steps, and variations quickly, helping people move from idea to action. Education may benefit if AI is used to create practice, tests, and feedback, but there is a real risk of students using it to fake learning. Governance will likely require a mix of company self-regulation, nonprofit scrutiny, and government enforcement because AI firms face conflicted incentives. The future impact of AI is not predetermined; how people choose to use and regulate it will shape whether it helps or harms society.
Data Points: Prompt engineer salary: more than $100,000/year - Jobs boards for prompt engineering roles mentioned during the discussion Prompt engineer salary high end: more than $500,000/year - One prompt engineering listing cited as an extreme example of demand Anthropic team size for Claude: around 35 people - Current workforce working on Claude Original GPT-3 training team size: basically 3 people - Dario Amodei describing early OpenAI work Claude constitution length: about five pages - Amodei describing Anthropic’s written principles for alignment AI model training cost: more than $100 million - Cost estimate for building a large language model Model scale: hundreds of billions to a trillion parameters - Describing the size/complexity of LLMs Brain working memory: 3 to 7 chunks of information - Used to contrast human cognition with model scale Anthropic / frontier timeline concern: 2 to 3 years - Amodei’s best guess for when a difficult-to-control phase of capability may arrive Anthropic/Google ownership: reported 10% - Google’s stake in Anthropic Number of AI company founders/executives mentioned as forming oversight group: 4 companies - Anthropic, OpenAI, Microsoft, and Google announced Frontier Model Forum Entrepreneurship survey stat: one third - Mollick says about a third of people have had an entrepreneurial idea in the last five years they wish they could execute on
Pivotal Quotes: "I make the AI talk good" — Anna Bernstein: Her joking shorthand for describing prompt engineering to strangers "In its raw state, an LLM has almost all of human knowledge and almost no human wisdom." — Adam Davidson: Summing up the gap between data-rich models and judgment "How do we develop the good stuff ahead of the bad stuff?" — Dario Amodei: On the challenge of differential technology development and AI safety
Implications: Listeners are urged to learn AI now, because it will change work, schooling, and creative life. The winners will likely be people who can direct AI well, while governments and companies must still build safeguards.
From the Episode
Episode of our series on AI, we talked about how all new technology destroys some jobs and creates new ones, jobs nobody could have imagined. Well, Prompt Engineer is one of those new jobs, and most people have still never heard of it. Honestly, most of the time I would just say, I make the AI talk good, and then that would get a laugh and everyone would move on, hopefully. It's a difficult job to define. As far as you know, do you think you might be the first person to have this job? As far as I know, if someone goes. Is like, nope, I had it before that. I'm not going to fight them on it. But of who I know, yes. You are the first one I've heard of, and you are the person from whom I learned that this is a job. But I now see that it's an in-demand job, right? There's a lot of listings out there for prompt engineers. Yeah, I think a lot of companies and a lot of industries are still figuring out how and whether they get a lift from prompt engineering, which is also very tied into the question.
They are based on trillions of data points. There's an old saying: knowledge is knowing a lot of facts, and wisdom is knowing which facts matter. In its raw state, an LLM has almost all of human knowledge and almost no human wisdom. There are a handful of LLMs now: there's OpenAI's GPT-4, which fuels ChatGPT as well as Microsoft's Bing. Google's LLM is called Palm2. We've heard so much about These LLMs, I started to wonder: what are they exactly? Are they one big box or some network of computers? Are they in one place or just some vague capability in the cloud? Who creates and manages these LLMs? The number of people required to do it has gone up over time. When I was at OpenAI doing GPT-3, it was basically three of us who trained the original model. That is Dario Amadei. I find this amazing. He says that basically three people did the original work.
Is. We're not there yet, but two to three years is our best guess. I don't know what's going to happen, but that's our best guess for when we'll get there. That's scary. There's a lot of people with really bad ideas, and the world puts up barriers to them because they don't have knowledge and resources. And so it's hard to think: how do we lower barriers for good stuff and maintain barriers for bad stuff? I think that's the hardest question. The term we use is differential technology development. How do we develop the good stuff ahead of the bad? Stuff. These debates are essential. Is AI good? Is it bad? How can we encourage the goodness and try to head off the worst of the badness? Recently, Anthropic, OpenAI, Microsoft, and Google announced they were forming an oversight group to self-police their industry. It's called Frontier Model Forum. Amiday and other AI executives have also called for the government to develop regulations for this new technology. There's
About Freakonomics Radio
Freakonomics co-author Stephen J. Dubner uncovers the hidden side of everything. Why is it safer to fly in an airplane than drive a car? How do we decide whom to marry? Why is the media so full of bad news? Also: things you never knew you wanted to know about wolves, bananas, pollution, search engines, and the quirks of human behavior. To get every show in the Freakonomics Radio Network without ads and a monthly bonus episode of Freakonomics Radio, start a free trial for SiriusXM Podcasts+ on...