We are an informal reading group at UC Davis dedicated to reading, discussing, and building on Joe Carlsmith’s AI and philosophy work.
Why Joe Carlsmith
We believe that reacting to rigorous, foundational work is the best way to develop your own takes and research questions. His writing like “Is Power-Seeking AI an Existential Risk?” and the “How do we solve the alignment problem?” series provide a taxonomy for thinking about the assumptions and difficulties of AI risk. Our goal through reading is to encourage contributing to the most important questions of our time.
How it works
We are low-overhead, interactive, and built for busy schedules.
- Format: We meet once every two weeks to discuss one essay or section.
- 60min Structure:
- First half to (re-)read the assigned text so we’re all ready for discussion.
- Second half of discussion to break down arguments, assumptions, and explore our own ideas.
Reading list
We will tackle one post from the “How do we solve the alignment problem” series in each bi-weekly meeting.
Some open questions
- What is “minimum viable alignment?” Instead of trying to solve the entire future and get ethics perfectly, what is the minimum level of alignment needed to just get the benefits and avoid catastrophic failure?
- Can safety even be represented as a scalar? We talk about a “red line” for safety and “blue line” for capability. Is this a useful model? How do you actually evaluate safety as a single metric to know if you’re “above the line?”
- When is the right time to pause? What specific level of AI capability, one that is just powerful enough to significantly help with safety research, would be the “sweet spot” to pause further capability scaling?
Interested?
We are currently gathering our founding members for Fall 2025.
If you’re a UC Davis student who wants to join, please add your name and email to our interest list. Once we have a core group, we’ll send out a poll to schedule our first meeting.