LW - AI #78: Some Welcome Calm by Zvi

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: AI #78: Some Welcome Calm, published by Zvi on August 23, 2024 on LessWrong.
SB 1047 has been amended once more, with both strict improvements and big compromises. I cover the changes, and answer objections to the bill, in my extensive Guide to SB 1047. I follow that up here with reactions to the changes and some thoughts on where the debate goes from here. Ultimately, it is going to come down to one person: California Governor Gavin Newsom.
All of the debates we're having matter to the extent they influence this one person. If he wants the bill to become law, it almost certainly will become law. If he does not want that, then it won't become law, they never override a veto and if he makes that intention known then it likely wouldn't even get to his desk. For now, he's not telling.
Table of Contents
1. Introduction.
2. Table of Contents.
3. Language Models Offer Mundane Utility. AI sort of runs for mayor.
4. Language Models Don't Offer Mundane Utility. A go or no go decision.
5. Deepfaketown and Botpocalypse Soon. How hard is finding the desert of the real?
6. The Art of the Jailbreak. There is always a jailbreak. Should you prove it?
7. Get Involved. Also when not to get involved.
8. Introducing. New benchmark, longer PDFs, the hot new RealFakeGame.
9. In Other AI News. METR shares its conclusions on GPT-4o.
10. Quiet Speculations. Are we stuck at 4-level models due to Nvidia?
11. SB 1047: Nancy Pelosi. Local Nvidia investor expresses opinion.
12. SB 1047: Anthropic. You got most of what you wanted. Your move.
13. SB 1047: Reactions to the Changes. Reasonable people acted reasonably.
14. SB 1047: Big Picture. Things tend to ultimately be rather simple.
15. The Week in Audio. Joe Rogan talks to Peter Thiel.
16. Rhetorical Innovation. Matthew Yglesias offers improved taxonomy.
17. Aligning a Smarter Than Human Intelligence is Difficult. Proving things is hard.
18. The Lighter Side. The future, while coming, could be delayed a bit.
Language Models Offer Mundane Utility
Sully thinks the big models (Opus, 405B, GPT-4-0314) have that special something the medium-sized models don't have, no matter what the evals say.
A source for Llama-3.1-405-base, at $2 per million tokens (both input and output).
Accelerate development of fusion energy, perhaps? Steven Cowley makes the case that this may be AI's 'killer app.' This would be great, but if AI can accelerate fusion by decades as Cowley claims, then what else can it also do? So few people generalize.
Show the troll that AIs can understand what they're misinterpreting. I am not as optimistic about this strategy as Paul Graham, and look forward to his experiments.
Mayoral candidate in Cheyenne, Wyoming promises to let ChatGPT be mayor. You can tell that everyone involved it thinking well and taking it seriously, asking the hard questions:
"Is the computer system in city hall sufficient to handle AI?" one attendee, holding a wireless microphone at his seat, asked VIC.
"If elected, would you take a pay cut?" another wanted to know.
"How would you make your decisions according to human factor, involving humans, and having to make a decision that affects so many people?" a third chimed in.
After each question, a pause followed.
"Making decisions that affect many people requires a careful balance of data-driven insights and human empathy," VIC said in a male-sounding voice. "Here's how I would approach it," it added, before ticking off a six-part plan that included using AI to gather data on public opinion and responding to constituents at town halls.
OpenAI shut off his account, saying this was campaigning and thus against terms of service, but he quickly made another one. You can't actually stop anyone from using ChatGPT. And I think there Aint No Rule against using it for actual governing.
I still don't know how this 'AI Mayor' w...