Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio.
This is: Finite Factored Sets, published by Scott Garrabrant on the AI Alignment Forum.
Crossposted from the AI Alignment Forum. May contain more technical jargon than usual.
This is the edited transcript of a talk introducing finite factored sets. For most readers, it will probably be the best starting point for learning about factored sets.
Video:
(Lightly edited) slides:
1. Short Combinatorics Talk
1m.
Some Context
Scott: So I want to start with some context. For people who are not already familiar with my work:
My main motivation is to reduce existential risk.
I try to do this by trying to figure out how to align advanced artificial intelligence.
I try to do this by trying to become less confused about intelligence and optimization and agency and various things in that cluster.
My main strategy here is to develop a theory of agents that are embedded in the environment that they're optimizing. I think there are a lot of open hard problems around doing this.
This leads me to do a bunch of weird math and philosophy. This talk is going to be an example of some weird math and philosophy.
For people who are already familiar with my work, I just want to say that according to my personal aesthetics, the subject of this talk is about as exciting as Logical Induction, which is to say I'm really excited about it. And I'm really excited about this audience; I'm excited to give this talk right now.
1t.
Factoring the Talk
This talk can be split into 2 parts:
Part 1: a short pure-math combinatorics talk.
I suspect that if I were better, I would instead be giving a short pure-math category theory talk; but I'm trained as a combinatorialist, so I'm giving a combinatorics talk upfront.
Part 2: a more applied and philosophical main talk.
This talk can also be split into 4 parts differentiated by color:
Motivation
Table of Contents
Main Body
, and
Examples
. Combining these gives us 8 parts (some of which are not contiguous):
Part 1: Short Talk Part 2: The Main Talk
Motivation
1m. Some Context 2m. The Pearlian Paradigm
ToC
1t. Factoring the Talk 2t. We Can Do Better
Body
1b. Set Partitions, etc. 2b. Time and Orthogonality, etc.
Examples
1e. Enumerating Factorizations 2e. Game of Life, etc.
1b.
Set Partitions
All right. Here's some background math:
A partition of a set
S
is a set
X
of non-empty subsets of
S
, called parts, such that for each
s
∈
S
there exists a unique part in
X
that contains
s
Basically, a partition of
S
is a way to view
S
as a disjoint union. We have parts that are disjoint from each other, and they union together to form
S
We'll write
P
a
r
t
S
for the set of all partitions of S.
We'll say that a partition
X
is trivial if it has exactly one part.
We'll use bracket notation,
s
X
, to denote the unique part in
X
containing
s
. So this is like the equivalence class of a given element.
And we'll use the notation
s
∼
X
t
to say that two elements
s
and
t
are in the same part in
X
You can also think of partitions as being like variables on your set
S
. Viewed in that way, the values of a partition
X
correspond to which part an element is in.
Or you can think of
X
as a question that you could ask about a generic element of
S
. If I have an element of
S
and it's hidden from you and you want to ask a question about it, each possible question corresponds to a partition that splits up
S
according to the different possible answers.
We're also going to use the lattice structure of partitions:
We'll say that
X
≥
S
Y
X
is finer than
Y
, and
Y
is coarser than
X
) if
X
makes all of the distinctions that
Y
makes (and possibly some more distinctions), i.e., if for all
s
t
∈
S
s
∼
X
t
implies
s
∼
Y
t
. You can break your set
S
into parts,
Y
, and then break it into smaller parts,
X
X
∨
S
Y
(the common refinement of
X
and
Y
) is the coarsest partition that is finer than both
X
and
Y
. This is t...
view more