What does it take to train a frontier model? What's the know-how, the secret sauce that makes firms lets OpenAI and Deepmind push the limits of what's possible? How much are Chinese firms benefitting from western open source, and in the long term is it possible for western labs to maintain an edge?
The hosts of the excellent Latent Space podcast, Alessio Fanelli of Decibel VC and Shawn Wang of Smol AI, come on to discuss.
We get into:
Outtro music: CHEKI https://open.spotify.com/track/1zKL2bOEkMDGuIjLhG34YA?si=9a713a88aa3d4f71
Cover photo: "Inkstand with A Madman Distilling His Brains" 1600s Urbino. Kind of like training a model! https://www.metmuseum.org/art/collection/search/188899
The met description: In this whimsical maiolica sculpture, a well-dressed man leans forward in his seat with his head in a covered pot set above a fiery hearth. The vessel beside the hearth almost certainly held ink. The man’s actions are explained by an inscription on the chair: "I distill my brain and am totally happy." Thus the task of the writer is equated with distillation—the process through which a liquid is purified by heating and cooling, extracting its essence.
Learn more about your ad choices. Visit megaphone.fm/adchoices
Mechanical Keyboards in China (中文版)
How YOU Can Change S&T Policy
Tech Policy Entrepreneurship + Chips Act + Talent Policy
BALLOONTALK: EMERGENCY EDITION
AI Compute 101: The Geopolitics of Giant Models
War in Taiwan: Who Would Win?
China's Data Policy Future
War for Taiwan: What Happens If China Wins?
Rep. Ro Khanna on AI, the China Committee, and Industrial Policy
Chips Act: A How To Guide
Knowledge and AI with a Rabbi and Substacker
Tyler Cowen on AI and China
Year in Review!
What to do about TikTok
Chips: 2022 in Review
Nuclear Fusion: Why We Can't Have Nice Things
Semis 101 with Asianometry and Fabricated Knowledge
EMERGENCY POD: How will the CCP respond to the protests?
EMERGENCY POD: China's Protests: What Happens Next
Can the US and China work together on anything?
Create your
podcast in
minutes
It is Free