The Nonlinear Library: EA Forum
Education
EA - Good job opportunities for helping with the most important century by Holden Karnofsky
Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Good job opportunities for helping with the most important century, published by Holden Karnofsky on January 18, 2024 on The Effective Altruism Forum.Yes, this is my first post in almost a year. I'm no longer prioritizing this blog, but I will still occasionally post something.I wrote ~2 years ago that it was hard to point to concrete opportunities to help the most important century go well. That's changing.There are a good number of jobs available now that are both really promising opportunities to help (in my opinion) and are suitable for people without a lot of pre-existing knowledge of AI risk (or even AI). The jobs are demanding, but unlike many of the job openings that existed a couple of years ago, they are at well-developed organizations and involve relatively clear goals.So if you're someone who wants to help, but has been waiting for the right moment, this might be it. (Or not! I'll probably keep making posts like this as the set of opportunities gets wider.)Here are the jobs that best fit this description right now, as far as I can tell. The rest of this post will give a bit more detail on how these jobs can help, what skills they require and why these are the ones I listed.OrganizationLocationJobsLinkUK AI Safety InstituteLondon (remote work possible within the UK)Engineering and frontend roles, cybersecurity rolesHereAAAS, Horizon Institute for Public Service, Tech CongressWashington, DCFellowships serving as entry points into US policy rolesHereAI companies: Google DeepMind, OpenAI, Anthropic1San Francisco and London (with some other offices and remote work options)Preparedness/Responsible Scaling roles; alignment research rolesHere, here, here, hereModel Evaluation and Threat Research (METR) (fewer roles available)Berkeley (with remote work options)Engineering and data rolesHereSoftware engineering and development (and related areas) seem especially valuable right now, so think about whether you know folks with those skills who might be interested!How these helpA lot of these jobs (and the ones I know the most about) would be contributing toward a possible global standards regime for AI: AI systems should be subject to testing to see whether they present major risks, and training/deploying AI should stopped (e.g., by regulation) when it can't be done safely.The basic hope is:Teams will develop "evals": tests of what AIs are capable of, particularly with respect to possible risks. For example, one eval might be prompting an AI to give a detailed description of how to build a bioweapon; the more detailed and accurate its response, the more risk the AI poses (while also possibly having more potential benefits as well, by virtue of being generally more knowledgeable/capable).It will become common (through regulation, voluntary action by companies, industry standards, etc.) for cutting-edge AI systems to be subject to evals for dangerous capabilities.When evals reveal risk, they will trigger required mitigations. For example:An AI capable of bioweapons development should be (a) deployed in such a way that people can't use it for that (including by "jailbreaking" it), and (b) kept under good security to stop would-be terrorists from circumventing the restrictions.AIs with stronger and more dangerous capabilities might require very challenging mitigations, possibly beyond what anyone knows how to do today (for example, rigorous demonstrations that an AI won't have dangerous unintended aims, even if this sort of thing is hard to measure).Ideally, we'd eventually build a robust international governance regime (comparisons have been made to nuclear non-proliferation regimes) that reliably enforces rules like these, while safe and beneficial AI goes forward. But my view is that even dramatically weaker setups can still help a lo...
Create your
podcast in
minutes
It is Free