Topic · 8 episodes
Artificial Intelligence
Artificial Intelligence sits at the intersection of rigorous mathematics and messy market reality. Scaling laws show AI capability follows predictable power-law curves across vast compute ranges, yet the field had assumed a suboptimal training ratio until DeepMind's Chinchilla paper corrected it. The attention mechanism unlocked modern large language models by breaking a sequential bottleneck. Meanwhile, six in ten US consumers say seeing 'AI' in brand messaging actively puts them off.
Frequently asked
What are AI scaling laws and why do they matter?
AI scaling laws, formalized in OpenAI's 2020 Kaplan paper, show that model loss falls on a smooth power-law curve across seven orders of magnitude of compute — a range stretching from roughly one dollar to US GDP. The pattern suggests capability gains are predictable, not random, making scaling a strategic lever.
What did the Chinchilla paper change about how AI models are trained?
DeepMind's 2022 Chinchilla paper showed the optimal ratio of model parameters to training tokens is roughly 1:1, not the 4:1 the field had assumed. This made models like Google's PaLM 540B theoretically undertrained the day Chinchilla published, reshaping how researchers allocate compute budgets.
How does the attention mechanism matter for large language models?
Attention solves a core bottleneck in artificial intelligence systems: the sequential processing constraint that limited earlier neural architectures. According to Onpode's coverage, attention is the mechanism that made modern large language models possible by removing that sequential dependency.
Does calling a product 'AI' actually help with consumers?
No — sixty percent of US consumers say 'AI' in brand messaging is a turnoff, despite intense industry enthusiasm. The tension between how the artificial intelligence industry talks about itself and how ordinary consumers respond is a growing problem for companies leaning on the label as a selling point.
Episodes
The NHS is rolling AI-powered triage to 200,000+ patients to route them to the right health service automaticallyNHS England is deploying AI triage inside the NHS App for 200,000+ patients within 12 months, backed by a £10 billion tech commitment — but the entire rollout rests on a single trial at one rural Sussex GP practice that measured phone queue reduction, not clinical accuracy, and no public governance framework defines who is liable when the algorithm is wrong.
Anthropic enters talks with Samsung to build custom inference-optimized AI chips, echoing OpenAI's Broadcom 'Jalapeño' strategyAnthropic is in early-stage talks with Samsung to manufacture a custom AI inference chip targeting a 2-nanometer process, according to The Information and Bloomberg. Anthropic neither confirmed nor denied the talks while publicly maintaining that Google, Amazon, and Nvidia remain central — making the chip a negotiating position as much as an engineering program.
AI-native startups operate leaner than peers—yet cut entry-level hiring, widening the productivity paradoxA Harvard Business School AI Institute study of 2,900+ Y Combinator startups found AI-native firms run 25% smaller than comparable peers while hitting equivalent valuations — but they also post 15% fewer entry-level roles, compressing the talent pipeline that produces the senior engineers those same firms plan to hire in three to five years.
Anthropic petitions Senate over Chinese model theft; China blocks Meta's AI startup buy—escalating AI cold warAlibaba's Qwen lab allegedly ran 28.8 million exchanges across 25,000 fake accounts against Anthropic's Claude — the largest AI distillation attack ever claimed — prompting Anthropic's June 10th Senate letter. The same week, China blocked Meta's $2 billion Manus acquisition, framing a mirrored cold-war standoff over AI capability access.
White House pressures OpenAI to hold GPT-5.6 pending security review — shifting from open to gated AI releasesOn June 26, 2026, the White House became the first U.S. government body to preemptively restrict a domestic AI model launch, directing OpenAI to limit GPT-5.6 access to roughly twenty government-approved partners. Anthropic's Mythos 5 was similarly gated, while Fable 5 remained fully offline — and no public threat assessment was released.
How compute and data follow power laws — the durable pattern beneath capability curvesOpenAI's 2020 Kaplan scaling laws paper showed AI loss falls on a smooth power-law curve across seven orders of magnitude of compute — a span from one dollar to U.S. GDP. DeepMind's 2022 Chinchilla paper then showed the optimal parameter-to-token ratio was roughly 1:1, not the 4:1 the field had assumed, making models like PaLM 540B theoretically suboptimal the day Chinchilla published.
Why attention solves the sequential bottleneck — the mechanism behind modern LLMs
Sixty percent of US consumers find 'AI' in brand messaging a turnoff despite industry hype