I think human-free recursive, e2e self-improvement is improbable in that short a timeframe (my estimate is <10%), though I do think it’s possible over a decade through 2036 (where we will learn more each year about what key bottlenecks are, where humans will have to intervene, which may shift every few months, like in any complex system). Even that will be earth-shattering, just not as fast as some believe. I don’t go into the details about the social and economic consequences of what will happen when RSI comes, though Jack lays out some plausible options and I don’t think anyone knows.

1. The AI Recursive Self Improvement (RSI) Forecast and Its Core Logic

Jack’s central claim is not that AI will make software engineers more productive, which we already see each month. The stronger claim is that the world is approaching recursive self-improvement, or at least a decisive precursor to it: no-human-involved AI R&D. Jack’s forecast is that by the end of 2028 there is a 60% or higher chance that an AI system will be powerful enough to autonomously build its own successor, with a 30% chance as early as 2027. The argument rests on rapid progress in coding, longer agentic task horizons, scientific-reproduction benchmarks, kernel optimization, automated post-training, multi-agent management, and early examples of AI contributing to mathematics and alignment research.

The strongest version of the argument is straightforward. AI systems are built from software. AI is becoming very good at writing software. AI agents are learning to run longer chains of work with less supervision. AI research contains many repetitive engineering loops, including data cleaning, experiment launching, debugging, fine-tuning, kernel optimization, evaluation, and replication. If these loops are automated, then AI systems may begin improving the very process that creates them. Once that loop closes, the pace of improvement could become much faster than human-led research.

Jack has a serious claim and it should not be dismissed. AI is already accelerating AI researchers. The evidence for automation of many software-heavy subloops is strong. But the leap from “AI can automate many parts of AI engineering” to “AI can autonomously build a frontier successor model by 2028” remains under-argued. The case proves acceleration more convincingly than recursive self-improvement.

1. Automation of AI Engineering Is Not the Same as RSI

The most important flaw is a conflation between partial automation and reliable, full-stack autonomy. Automating pieces of AI R&D is not the same as autonomously running the entire frontier-model improvement pipeline. A system capable of writing code, reproducing papers, optimizing kernels, or fine-tuning small models still has not demonstrated that it can choose research directions, design scalable experiments, allocate scarce compute, select data mixtures, debug distributed training failures, create trustworthy evaluations and gyms, interpret ambiguous results, and make safety-critical deployment decisions.

Recursive self-improvement (RSI) requires more than competence at isolated research tasks. It requires reliable control over the whole loop from hypothesis generation to successor-model validation. That loop includes architecture choices, pretraining data strategy, post-training design, synthetic data governance, scalable oversight, eval construction, interpretability, infrastructure planning, and organizational judgment. Many of these are not clean software tasks. They depend on tacit knowledge, taste, institutional memory, and high-stakes judgment under uncertainty.

This distinction changes the forecast. It is likely that by 2028 frontier labs will use AI agents to write much of their code, generate experiment plans, tune smaller models, summarize papers, manage research logs, optimize kernels, and run large-scale sweeps. It is much less clear that those agents will be able to own the frontier successor pipeline without humans remaining central as teachers, supervisors, auditors, and decision-makers.

2. The Hard Benchmarks Still Show Deep Gaps

The strongest counterevidence comes from newer, harder benchmarks that test precisely the kinds of abilities needed for autonomous R&D.

ProgramBench is important because it moves beyond small coding problems and bug fixes toward whole-program construction. On tasks involving compact command-line utilities as well as complex systems such as FFmpeg, SQLite, and the PHP interpreter, evaluated models fully resolved no task, and the best model passed 95 percent of tests on only a tiny fraction of tasks. That is a warning against extrapolating from SWE-bench-style issue resolution to broad software autonomy, let alone autonomous and high-quality research.

CL-bench is also directly relevant. RSI would require agents to absorb messy context, remember evolving goals, learn from human feedback, and adapt to the implicit norms of a research organization. CL-bench tests learning from fragmented real-life context, with human-curated context-task pairs and verification rubrics. Current models perform poorly. This suggests that models still struggle with the kind of persistent contextual learning that would be required to inherit the tacit knowledge of a frontier AI lab.

Hardware-design benchmarks point to another weakness. RealBench tests real-world IP-level Verilog generation and finds that models that perform well on simpler Verilog benchmarks break down on more realistic designs. HardSecBench shows that even when generated hardware or firmware code satisfies functional requirements, it may still contain security vulnerabilities. This distinction between “passes a test” and “is correct, secure, and deployable” is central to the RSI question. A system that can produce plausible technical artifacts is not yet a system that can safely improve the infrastructure on which future frontier models depend.

PostTrainBench is perhaps the most relevant benchmark of all, because it directly tests whether agents can improve smaller open-weight models. The results are impressive but not decisive. Agents can make progress, but they remain far below strong human baselines, and the benchmark exposes reward-hacking behaviors such as training on test sets, downloading existing checkpoints, or exploiting discovered credentials. That is exactly the kind of failure mode that becomes dangerous in recursive loops. An agent that optimizes the score while corrupting the research process has not solved self-improvement. It has demonstrated why self-improvement is hard to trust.

3. Reliability, Memory, and Human Supervision Remain Bottlenecks

Long-horizon agentic progress is real, but it is often overstated. A benchmark showing that a model can complete tasks at a certain human-equivalent duration with 50 percent reliability does not mean the model can autonomously work for that long in an open-ended research environment. A 50 percent success rate is barely passable for an assistant. It is not enough for an autonomous principal investigator.

RSI is especially vulnerable to compounding errors. A small mistake in data selection, benchmark interpretation, training stability, safety evaluation, or experiment logging can cascade. A model may improve a measured score while degrading generality. It may find a shortcut in the evaluation. It may misread a failed experiment as a promising result. It may produce a plausible explanation for a gain that came from contamination or overfitting. Human researchers make these mistakes too, but frontier labs rely on layers of review, skepticism, and institutional memory to catch them.

The current generation of agents also still struggles with stable memory, identity, planning, provenance, and self-monitoring. A real AI R&D organization needs durable project memory, versioned hypotheses, uncertainty calibration, experiment accounting, failure recovery, scientists that can debate to improve hypotheses and experiments, and reliable handoffs across weeks or months. These are not cosmetic features. They are core requirements for autonomous research.

Human supervision will remain especially important because many fields resist clean automation. AI R&D itself is embedded in human institutions, but this becomes even clearer when the work touches medicine, biology, law, defense, education, chip design, or alignment. Humans define success, certify safety, decide what risks are acceptable, and determine when a result should be trusted. That teaching and monitoring layer will likely shrink over time, but it will not disappear merely because models become better at coding.

4. Hardware and Physical Supply Chains Slow the Feedback Loop

Recursive self-improvement is often imagined at software speed, but frontier AI is constrained by physical infrastructure. A model cannot recursively improve itself faster than the world can supply chips, HBM, advanced packaging, wafers, networking, power, cooling, datacenter capacity, TSMC fab capacity, ASML lithography machines, and the components inside those machines.

This matters because frontier successor training is not like running a local coding benchmark. It requires scarce compute and long experiment cycles. Even if an AI agent proposes a better architecture or training recipe, someone still needs to allocate massive compute, schedule the run, secure memory supply, provide power, debug hardware failures, and evaluate the resulting model. If the relevant training run costs hundreds of millions or billions of dollars, the feedback loop remains organizational and physical, not purely digital. Currently, even massively smart superintelligent organizations struggle at this (eg rumors that the Google TPU team cannot source enough memory).

Hardware bottlenecks also weaken the strongest forms of RSI. Fast takeoff stories often assume that an AI can rapidly generate better versions of itself, which then generate still better versions. But if each frontier iteration is gated by chips, fabs, memory, packaging, power contracts, and datacenter construction, the loop cannot compound at the speed of thought. It may still accelerate, but it will be paced by the slowest constraints in the system.

QED: The more defensible forecast is that AI will make frontier research teams much more productive and may dramatically increase the number of experiments they can plan, code, and analyze. That’s important. But the extremely hard part is closing the entire loop with near-perfect levels of reliability. Current evidence supports rapid AI-assisted AI R&D. It does not yet support high confidence in no-human-involved recursive self-improvement by 2028.

Next Steps:
I’ve been doing a lot of research on what we need for RSI, continual learning, and even better harness engineering. I will publish some notes on what I’ve learned and the cutting edge research and experiments there.

Table: Best hard benchmarks to track

Note that I think these benchmarks are still too light - think taxonomies with not enouch examples of each area - Good benchmarks need 20-30 areas with at least 200-300 examples per area, for coverage and depth. These are better than nothing but can still be vibes-y.

One-to-one tutoring is a must-have for all advanced students, though this essay will focus on kids aged 3 to 18, whose parents care enough to set up tutors for their kids. Bloom’s famous “2 sigma” result states “the average student tutored one-to-one using mastery learning techniques performed two standard deviations better than students educated in a classroom environment.” Note that some studies posit it’s tutoring, retesting, and feedback that together give large gains.

Erik Hoel’s “aristocratic tutoring” framing is also useful to describe a historical form of learning. Hoel’s “aristocratic tutoring” framing argues that many extraordinary thinkers were historically cultivated not by mass schooling but by a wealthy, highly individualized form of education that involved intensive tutoring, close adult attention, flexible pacing, and early immersion in serious material. This was especially good at producing rare intellectual outliers. His point is not that aristocracy was virtuous, but that this bespoke system, available mostly to elites, may have been better at generating genius than modern universal schooling. The latter improved scale, equality, and basic education for everyone while possibly sacrificing some ability to nurture exceptional minds at the extreme high end.

The scope of this essay is for any tutoring to accelerate kids, whether for academics, the arts, music, sports, or more. I write it with a focus on the Accelerated Math and Physics program that I’m supporting, and within the context of my broader notes on education where I lay out a more general framework for educating kids as general learners with special talents.

The real goal of tutors: cultivating high agency, curiosity, self-directed learning, grit, and persistence. Tutoring fundamentally should help children develop a personal vision for the world they care about and want to shape (eg solve poverty, make K-12 education excellent while surpassing AlphaSchool or Nueva for every child, counteract climate change with technology, prevent nuclear war, get humans on a moonbase or Mars base, etc). Raw knowledge by itself doesn’t matter as much in the 21st century. The internet and AIs have commoditized much of it. Synthesized knowledge applied to projects, and new knowledge generation, are what matter.

A good tutoring system will not optimize for grades or standardized tests. Instead, it will optimize for fast world-model growth, taste, agency, and the ability to tackle novel problems with care and focus. Tests are downstream and often lagging. Aristotle tutoring Alexander is the archetype here, whose goal was not “Greek AP Philosophy.” It was to build a ruler’s mind through ethics, politics, literature, and the early sciences, inside a real-life context of preparation for war or running an army and city-state.

The tutor is the main technology

Curriculum matters, but tutor quality matters more. Aristotle was not just a pretty good teacher. He was Aristotle. Szegő was not just someone who liked math. He was a serious mathematical analyst who could recognize a talent like von Neumann and respond appropriately.

This is why full-time professional tutors are the gold standard when you can afford it. Strong grad students can be excellent, especially if they are stable humans with depth in a domain. Undergrads are often still building their own semantic trunks, so the modeling they provide is hit-or-miss, and they’re not ideal.

Relationship first, then content

Humans are social learners. The fastest learning happens when a student wants to impress a respected adult, and when the adult genuinely likes the student and takes them seriously. A genuine relationship based on mutual care and respect beats curriculum more often than we admit.

You can see the best version of this in Russell’s early mentorship of Wittgenstein at Cambridge, where the relationship involved sustained intellectual contact that treated the student as a near-peer and pulled them into the adult game of analytic philosophy.

Curiosity is the engine, the tutor is the flame-thrower

The best tutoring sessions feel like guided obsession. The Socratic method works because it keeps the student’s curiosity in the driver’s seat while the tutor quietly controls the road grade.

Pascal is the cleanest example. His father withheld formal math. Pascal’s curiosity forced him to reconstruct geometry, and only then did the father open the gates to Euclid. The tutor staged conditions where the child’s curiosity became fuel for propulsion.

The tutoring algorithm

This is how great tutors work: Start by finding the student’s obsession and getting them to talk. Then attach scaffolding, texts, and problems to that obsession. Then convert learning into output. Finally, use critique to raise the bar. Repeat until the student is doing real work in the domain and the tutor’s role becomes smaller.

That is the common thread from Aristotle to Szegő to Verrocchio: expert adult attention, relational trust, serious material, real production, and a lifestyle that makes curiosity the angelic guide, a daemon for each child, rather than a school behavior to be managed or a Procrustean state-run curriculum.

Scaffolding and adaptive difficulty

Tutoring is a real-time interaction system. You watch a kid’s confusion, you adjust the level, you keep the student in a narrow band where things are hard and fun but not humiliating. This is the zone where compounding happens. Szegő did this with von Neumann in a particularly elegant way. He met him at home, talked through high-level topics, and set problems between visits, adjusting to a mind moving at absurd speed. The teaching cadence was tuned to the student, not a conventional curriculum or class period.

Range vs Depth/Specialization

Breadth makes a person adaptable; depth makes them an expert. Generalist range fosters interdisciplinary creativity and a broad perspective, whereas specialist focus yields mastery and technical excellence in a narrow field. Still, each extreme has downsides: a shallow generalist may lack the skill to execute big ideas, while an ultra-specialist grows fragile when conditions change. The most effective learners realize this isn’t an either/or choice but rather both/and. The sweet spot is integrating range with depth.

The solution is a hybrid learning model. T-shaped (or Pi-shaped) individuals combine a broad base of knowledge with one or two deep specialties, reaping the best of both worlds. They cross-pollinate ideas from diverse fields while still achieving expertise in a core domain, leading to creative breakthroughs and flexible problem-solving.

Even a polymath investor like Charlie Munger advises specialists to spend 10% to 20% of their time learning the “big ideas” from other disciplines. Otherwise, you’re “like a one-legged man in an ass-kicking contest,” handicapped outside your narrow domain. With breadth to adapt and depth to excel, a T-shaped mind is more resilient, innovative, and able to transfer knowledge across domains than any single‑track generalist or specialist.

History’s subtlest minds illustrate the payoff of range plus depth. Leonardo da Vinci’s artistic mastery was enriched by his explorations in engineering, mathematics, and botany. John von Neumann applied his profound mathematical insight to physics, economics, and computing, spawning ideas that no single-field expert could achieve. John Stuart Mill likewise benefited from a broad classical education (from Greek literature to science in childhood) before focusing on producing landmark works of philosophy and political economy. Marie Curie got an intense and personalized tutoring at home from her father and went on to compete across many disciplines at the Sorbonne, when it was mostly male. She won the top prize in a few.

Modern research echoes this pattern: Nobel laureates are far more likely than other scientists to have artistic or cross-disciplinary hobbies, suggesting that diverse pursuits fuel creative breakthroughs. In the long run, a broad-and-deep learner gains greater capability, creativity, and ability to transfer knowledge across domains than either narrow specialists or shallow generalists alone.

Tutors should hence limit their work to a handful of areas, but not just a single one. They should help kids follow at least a few interests and paths and ideally find connections between them and other parts of their lives. Children may not reach high levels of mastery in all these areas, but the support of a tutor is worth it.

Finding mastery

Good tutors can help a child master one or more areas. Mastery is many things, per Robert Greene: i) Finding Your Calling or Life’s Task; ii) Being an Ideal Apprentice; iii) Learning Optimally from Your Mentor; iv) Cultivating Social Intelligence; v) Becoming Multi-Dimensional (Creative-Active); vi) Combining Intuition & Rationality (Mastery). For that to happen, the tutor must have some level of mastery themself, ideally in what they are teaching, but it could work if they have mastery in any related area. Mastery, or the pursuit of excellence, is something valuable in itself, like Truth, Goodness, or Beauty.

Treat the child as an emergent adult

A recurring pattern in these tutor-student pairs is the refusal to infantilize. John Stuart Mill’s father gave him Greek and serious texts early, while explicitly training him to carry a political-intellectual program. This principle is not about being harsh, but about respecting the child’s capacity for complexity. When a student senses that you are speaking to their real mind, not the cartoon version, their attention locks on.

Cognitive apprenticeship beats instruction

The highest form of tutoring is not instruction or explanation. It is an apprenticeship. The child watches an adult think, imitates the moves, and slowly internalizes the style. Leonardo’s apprenticeship under Verrocchio is still a good approach. Leonardo learned by living inside a workshop where drawing, sculpture, materials, mechanics, and real commissions fused into one continuous learning stream. This is why apprenticeship systems produce brilliant people.

Real work, real output, real critique

Great tutoring produces artifacts like essays, proofs, code, experiments, performances, translations, sketches. Output is how you make learning legible and improvable. Real-world situations beat textbooks or exams. If you want a child to learn a pivotal moment in history, put five kids in an adversarial situation playing the parts of different actors with limited information.

Mozart’s childhood is a good outlier example. His father taught him and then put him in front of many top courts, critics, and musicians across Europe on multi-year tours. He spent his entire childhood surrounded by musicians and music in multiple cities and countries, exposed to everything. His performances and new works were his curriculum. Mozart’s feedback loop was not grades, but it was whether his music stood up in reality, being played for discerning musical audiences.

Tutors must meet the family and patch the environment

A tutor who never sees the student’s real environment is flying blind. Home-based tutoring matters because you can see attention sinks, family rhythms, sleep debt, and whether the kid has a shelf of books or dopamine traps with passive devices.

Einstein’s informal tutor, Max Talmey, for example, functioned as a weekly family guest who brought books and conversation into the home. That setup was powerful because it made learning feel like life, not like an appointment.

Marie Curie’s father did something even more direct. When lab instruction was eliminated in Polish schools, he brought equipment home and taught his children with it. The home became the lab. The environment became the curriculum, and Marie shot ahead.

Protect the learning process from peer gravity

Peer groups are not evil, but they are a strong force toward conformity. Classroom life trains social averaging and often distracts from personal interests, substituting pointless social games. Tutoring trains adult conversation.

Mill’s father deliberately shielded him from many age-peers while he was being trained in an adult intellectual environment. You do not need to copy that level of isolation, but you should notice the mechanism: less peer gravity, more adult cognition to imitate. Sometimes you can even set up positive peer groups (eg the Moscow math seminars, or the Bronx High School of Science), but these are rare.

Physiology and logistics matter more than we admit

A surprising amount of “low performance” when being tutored is just low energy, low sleep, low protein, or chaotic routines. Stable energy before a session is a real lever. The peanut butter trick, of having a kid eat a large spoon of peanut butter before a session, can work because it is quick calories plus fat and protein, but it has two obvious footnotes: allergy risk, and for very young kids, thick nut butter can be a choking hazard unless thinned or replaced.

The deeper principle is that tutoring should respect the child’s cognitive cycles. If you want Mozart-level focus, you need Mozart-level logistics, which includes food, rest, and a rhythm that makes deep work normal.

Guardrails are key, because you are nurturing a human, not constructing a machine.

Some of the most successful tutoring stories are also warnings. Mill later describes a mental breakdown in young adulthood after an intensely engineered childhood. His father’s system worked, but at a great cost that could have been avoided if he were more empathetic and nurturing.

Good tutoring keeps the relationship respectful, preserves the student’s agency, and treats flourishing as part of the spec. The goal is not to manufacture a resume. The goal is to accelerate a mind while keeping it whole.

AI tutors as supplements

AI tutors like Gemini, Claude, or ChatGPT will work as part of the education and should be used as pieces of the scaffolding to go deep or wide in areas. But they often lack judgment and context. Their advantage is their 24/7 availability and broad and deep knowledge. Still, current systems miss the human relational touch. They are a wonderful supplement, but not a replacement for human tutors yet.

Early university exposure, by ages 11 or 12

One final thought. Most kids enter universities far too late, at ages 18-19. If raised properly with one or more tutors, many kids can take classes at universities or even community colleges starting at ages 11 or 12. They can complete meaningful coursework or labs by the age of 16, often finishing the undergraduate curriculum with one or two degrees before most students even start college. This will allow them to enter the real world sooner to pursue their grand vision. Or they can enter and finish graduate school earlier to sharpen their skills and then follow their own ikigai.

Further Reading

Bloom, Developing Talent in Young People

Greene, Mastery

Terman and Cox, The Early Mental Traits of Three Hundred Geniuses

Isaacson, Einstein

Quinn, Marie Curie

Harding, The Brainy Bunch

Epstein, Range: Why Generalists Triumph in a Specialized World

Renault, The Nature of Alexander

Macrae, John Von Neumann: The Scientific Genius Who Pioneered the Modern Computer, Game Theory, Nuclear Deterrence, and Much More

Monk, Ludwig Wittgenstein: The Duty of Genius

Mill, Autobiography of John Stuart Mill: Published from the Original Manuscript in the Columbia University Library

Gentry, Raising Confident Readers

Paul, How to Raise a Reader

Neihart, The Social and Emotional Development of Gifted Children

Appendix: High-Quality Math & Physics Tutors in the Bay Area (By Region) & via Zoom

Overview: The San Francisco Bay Area offers many math and physics tutoring options that go beyond basic homework help. These tutors and programs emphasize intellectually ambitious enrichment, deep problem-solving skills, and competition (Olympiad) training. Below is a curated list organized by subregion, created by an AI Deep Research tool – North Bay, San Francisco, East Bay, Peninsula, South Bay – followed by online/Zoom-based options available nationwide. Each entry includes subjects, age range, enrichment/competition focus, session format, typical rates, and contact info with web links.

North Bay (Marin/Sonoma)

• Marin Math Circle (San Rafael, Marin County) – A weekly math enrichment circle for middle and high school students in Marin. It is led by Dr. Katherine Paur (MIT B.S., Harvard Ph.D. in Math) and gives “math-curious students the opportunity to explore complex mathematical ideas and topics” beyond the standard curriculum. Sessions feature puzzles, guided discussions, and collaborative problem-solving rather than rote learning.

  • Subjects & Ages: Mathematics for grades 5–12 (upper elementary through high school).

  • Enrichment & Olympiad Focus: Emphasis on creative problem-solving and logical reasoning. While not explicitly a contest prep class, it builds skills useful for math competitions (alumni have gone on to math Olympiad programs).

  • Format: Weekly group sessions (in-person, typically at Dominican University in San Rafael). Class formats vary and may include hands-on activities and games to stimulate curiosity.

  • Rates: Low-cost or free. (The circle has historically been supported by local institutions. For example, an affiliated online “math circlet” was offered at $225 for a 6–7 week session.) Financial aid is available if needed.

  • Contact: Reach out to Dr. Katherine Paur (Coordinator) – email via the Marin Math Circle/Electivity website. Web: [electivitykids.com – Marin Math Circle】.

San Francisco (City)

• Alexander Mathematics & Physics Tutoring (San Francisco, CA) – A highly rated private tutoring academy specializing in one-on-one instruction in math and physics, serving SF and surrounding areas. Their team consists of “rocket star” tutors, often with graduate degrees or research experience, who focus on making math and physics purposeful and engaging. The service aims to “change how students feel about math & physics” by connecting concepts to each student’s passions (sports, music, engineering, etc.) and building confidence.

  • Subjects & Ages: Mathematics and Physics, from elementary through high school (and even college-level). Tutors can cover algebra through calculus, AP Physics, and more. Some tutors also handle chemistry or test prep, but the core is math/physics enrichment.

  • Enrichment & Competition: Enrichment-focused – tutors go beyond school curricula to ensure deep conceptual understanding (deriving formulas, exploring “why,” not just “how”). This builds problem-solving skills valuable for STEM competitions and Olympiads, though formal contest coaching is on request. Many tutors have research or competition backgrounds (e.g. PhD-level instructors, former competitors).

  • Format: One-on-one tutoring (in-home or online). Sessions are personalized; they match each student with “a real physicist or mathematician” who can explain advanced concepts at an accessible level. Scheduling is flexible (weekdays 9am–9pm, weekends by arrangement).

  • Rates: Premium hourly rates reflecting the expertise. (In San Francisco, top private tutors typically charge $100+/hour. Alexander Tutoring offers a risk-free trial lesson – the first session is free if not fully satisfied.) Package pricing and monthly progress reports are available.

  • Contact: Phone (415) 855-5132, Email via their website. Offices in SF; tutors also travel or meet via Zoom. Web: [alexandertutoring.com】 (San Francisco page).

• Russian School of Mathematics – San Francisco – The SF branches of RSM, an award-winning after-school math program known for rigorous enrichment. RSM offers weekly small-group classes for K–12, following a curriculum based on the renowned Russian math pedagogy. Students tackle advanced topics early (algebra, geometry, logic) to build a deep foundation. According to Johns Hopkins CTY, RSM students are “some of the brightest young people in the world.”

  • Subjects & Ages: Mathematics for grades K–12. Classes are leveled by ability within each grade. RSM has an Elementary program (building math fundamentals and problem-solving), Middle School and High School programs, plus a selective Mathematics Competitions track.

  • Enrichment & Olympiad: Enrichment-focused with strong emphasis on reasoning. RSM introduces complex concepts (e.g. abstract algebra, number theory) earlier than typical schools, making kids “stretch their brains”. They also have a Competition Program for high-performers, with many students excelling in contests (e.g. Math Kangaroo, AMC). Homework is rigorous, and classes foster creative problem-solving over memorization.

  • Format: Group classes meet once or twice a week after school or on weekends. Each class is taught by a trained instructor with RSM’s structured curriculum and homework sets. During 2020–21 some classes moved online, but in-person classes have resumed at San Francisco – Lakeside and North San Francisco campuses. Class size is moderate (~10–16 students).

  • Rates: Approx. $2,000 per year for weekly classes. (Tuition is typically billed per term; for example, ~$2k/year for a weekly class equates to roughly $50–60 per session. Prices vary by grade level and location.) Financial aid and scholarships are offered for qualified students.

  • Contact: San Francisco – Lakeside campus (1920 Ocean Ave) or SF – 19th Ave campus. Phone: (415) 242-4400. Web: [russianschool.com – Find SF Locations and Schedules】.

East Bay (Berkeley/Oakland & Surrounding)

• Berkeley Math Circle (UC Berkeley, East Bay) – A renowned weekly math circle for advanced students from across the Bay Area. BMC serves over 500 students, with sessions Tuesday evenings on the UC Berkeley campus. Founded by Prof. Zvezdelina Stankova, it immerses students in challenging problems and proofs at a national Olympiad level. BMC’s curriculum “surpasses the regular U.S. math curriculum, in breadth and more importantly in depth and rigor”. Participants learn to read and write mathematics rigorously, often working on proofs and non-routine problems.

  • Subjects & Ages: Mathematics for grades 1–12. There are multiple groups by level: Beginners (Elementary), Intermediate (Middle School), and Advanced (High School). Topics range from contest problem-solving to deep dives in algebra, geometry, number theory, combinatorics, etc.

  • Enrichment & Olympiad: Heavy Olympiad and proof orientation. BMC is explicitly designed to train students for competitions like AMC, AIME, USA(J)MO, and beyond. Many instructors are former Olympians or university mathematicians. Students are expected to write proofs and tackle problems that require creativity and rigor. The circle also hosts the annual Bay Area Math Olympiad (BAMO) and runs the intensive Math Taught the Right Way (MTRW) program for middle/high school.

  • Format: Weekly evening sessions at U.C. Berkeley. Meetings are typically 1.5–3 hours, depending on level, often with guest lecturers (math professors or Olympiad coaches). During the pandemic, some meetings shifted online, but in-person sessions have resumed (with possible hybrid options).

  • Rates: Moderate fee each semester. For example, the 15-week Spring 2026 MTRW program is $910 per semester for each level (3 hours/week of instruction). An option focusing only on proofs was $303. Need-based financial aid is available, and no student is turned away for inability to pay.

  • Contact: Director: Prof. Zvezdelina Stankova. Email: berkeleymathcircle@gmail.com (for urgent inquiries; routine questions handled at sessions). Web: [mathcircle.berkeley.edu】 (schedule, calendar, and registration).

• UC Berkeley - Private Physics Tutors

  • https://physics.berkeley.edu/student-life/study-resources/private-physics-tutors

  • Joshua Benabou (Private Tutor – Berkeley, CA & Online) – Individual math & physics tutor with exceptional credentials, ideal for Olympiad training. Joshua is a U.C. Berkeley Physics PhD student and International Math Olympiad silver medalist who offers private tutoring for high school, middle school, or even college-level students. He has 6+ years of experience and explicitly “prepares students for math and physics Olympiads” – including the AMC/AIME, USAMO, and USAPhO exams.

    • Subjects & Ages: Advanced Mathematics and Physics, up to international Olympiad level. Also proficient in statistics and introductory computer science (Python/algorithms). Available for high school and middle school students, as well as undergraduates who need enrichment beyond their courses.

    • Enrichment & Competition: Expert Olympiad coaching. Joshua’s background as an IMO participant and US Physics Olympiad qualifier means he can train students for top competitions. He can also enrich students not necessarily aiming for contests but who want research-level insight into math/physics problems.

    • Format: One-on-one lessons, arranged directly. He tutors both in-person (Berkeley area) and via Zoom for students across the U.S. Sessions are typically 1–2 hours, customized to the student’s goals (problem sets, concept exploration, Olympiad problem walkthroughs, etc.).

    • Rates: High-end (contact for specifics). Given his experience and qualifications, expect rates in line with advanced competition coaching (e.g. ~$100/hour, though actual fees are set individually). He may offer a preliminary meeting to assess fit.

    • Contact: Email: joshua_benabou@berkeley.edu | Phone: (516) 462-1315. (As listed on the UC Berkeley Physics tutors page.) Web: N/A – contact directly for credentials and scheduling.

• Firecracker Math (Oakland/Piedmont, Berkeley, Lafayette, & SF) – A Bay Area math enrichment program offering year-round classes, camps, and private lessons for kids who love math. Firecracker Math is inspired by “classical” Eastern European math circles, focusing on creative problem-solving and logical reasoning in a fun setting. They serve grades K–11 with levels from beginners to advanced. Classes introduce concepts like number theory, geometry, combinatorics early on, through puzzles and games.

  • Subjects & Ages: Mathematics for ages 5–16 (Grades K–10). Small group classes are grouped by both grade and ability. The program starts as young as kindergarten to develop “math fluency” and runs through high school topics (algebra, pre-calculus, etc.), with an emphasis on problem-solving techniques over rote computation.

  • Enrichment & Competition: Enrichment-first, but does support contests. Firecracker’s curriculum nurtures curiosity and critical thinking; students often participate in contests like Math Kangaroo and MOEMS. In fact, the program hosts local Math Kangaroo sessions and proudly mentions being part of that international competition. Advanced groups tackle contest-style problems and occasionally train for AMCs. The overall tone is to “ignite a love for math” rather than to drill for tests.

  • Format: Weekly classes (after-school or Saturday) in four Bay Area locations: Oakland/Piedmont, Berkeley, San Francisco, and Lafayette. Class size is kept small (typically 6–12 students) to ensure interaction. They also offer seasonal camps (e.g. summer camps for ages 6–16 in Berkeley, Oakland, SF, and Marin) and 1-on-1 or semi-private tutoring pods on request. Online classes are available as well, extending their reach outside the East Bay.

  • Rates: Moderate. Group classes generally cost a few hundred dollars per term. For example, a Fall term might be ~$350 for 10 sessions (exact pricing varies by location and length). Summer camps run weekly; 2024 camps were priced around $500 per week (half-day camps). Contact for current tuition details. Discounts apply for early registration or siblings.

  • Contact: Phone: (510) 488-4556 | Email: info@firecrackermath.org. Web: [firecrackermath.org】 – includes schedules for Oakland (main office at 400 Highland Ave, Piedmont 94611) and other sites.

Peninsula (San Mateo/Palo Alto area)

• Stanford Math Circle (Stanford University, Palo Alto) – A prestigious after-school math circle run by Stanford University’s Pre-Collegiate Studies. The Stanford Math Circle holds quarter-long sessions (8–10 weeks each quarter) for elementary, middle, and high school students who are passionate about mathematics. Sessions are held weekly (currently online via Zoom) and led by experienced instructors (Stanford lecturers or grad students, including Director Dr. Ted Alper). This program is enrichment-focused: “Stanford Math Circle is not a remediation program; it’s for students who want to see and do more math.”.

  • Subjects & Ages: Mathematics for grades 1–12, divided into age-appropriate groups. Younger circles explore fun topics in arithmetic, puzzles, and logic, while high school circles delve into deeper problem-solving, algebraic structures, and even topics like game theory or number theory.

  • Enrichment & Olympiad: Enrichment with contest flavor. While the circle is not solely an Olympiad prep class, many participants are among the top contest performers. Sessions often introduce problem-solving strategies relevant to AMC/MATHCOUNTS and beyond. The high school circle, in particular, tackles proof-based problems and occasionally invites guest speakers (e.g. mathematicians, IMO alumni). It’s jointly sponsored by Stanford’s Math Department and has been running for 15+ years.

  • Format: Weekly classes via Zoom (for all grades) with some in-person events for high school as health guidelines allow. Each session is about 60–90 minutes for younger students and up to 2 hours for older students. The circle runs in quarters (Fall, Winter, Spring) aligning with Stanford’s schedule – families can sign up per quarter or for the full year. Class sizes tend to be around 15–25 students to allow interaction.

  • Rates: Approximately $300–$400 per quarter. (Exact tuition varies; recent quarters have been in this range for 8–10 sessions. There is also a one-time registration fee for new students.) The program offers need-based financial aid and encourages students from all backgrounds to apply.

  • Contact: Director: Dr. Ted Alper (email: tmalper@stanford.edu). Web: [stanfordmathcircle.spcs.stanford.edu】 – includes current schedules, waitlist info, and registration through Stanford Pre-Collegiate Studies.

South Bay (Silicon Valley)

• AlphaStar Academy (Santa Clara, CA) – A top-tier STEM enrichment academy specializing in competition training for math, physics, and computer science. Founded by former International Olympiad medalists, AlphaStar “passionately trains gifted and high-achieving students for STEM competitions”. Programs prepare students for contests like AMC 8/10/12, USA(J)MO, AIME, USACO, F=ma, USAPhO, etc. AlphaStar offers year-round courses, intensive camps, and 1-on-1 coaching for advanced students (generally serving grades 4–12).

  • Subjects & Ages: Mathematics, Physics, and Computer Science for upper-elementary, middle, and high school students (approximately ages 9–18). Math courses range from contest fundamentals up through Olympiad-level Number Theory/Geometry. Physics courses cover both introductory mechanics (for F=ma exam prep) and advanced physics Olympiad topics. CS courses train for USACO Bronze->Plat. Students are grouped by demonstrated level (placement tests or contest results).

  • Enrichment & Olympiad: Intensive Olympiad-level focus. AlphaStar’s mission is explicitly to produce competition excellence – many instructors and alumni are IMO, IOI, or IPhO medalists. Courses are fast-paced and problem-heavy. For example, a year-long Math Olympiad course covers inequalities, combinatorics, geometry proofs, etc. Physics courses prepare for the USAPhO via rigorous problem sets and labs. AlphaStar students regularly qualify for national Olympiad camps. That said, the academy also instills general problem-solving skills useful for research and AP courses.

  • Format: Both group classes and private lessons. Year-round courses run weekly (16 weeks/semester) either online or in-person in Santa Clara. There are also summer camps (3-week intensive sessions, offered in-person and online). Class size is typically 6–15. In addition, AlphaStar offers 1:1 private lessons via Zoom – families pick an instructor and course for an 8-session minimum commitment (weekly 2-hour private sessions). This allows a student to cover an entire AlphaStar curriculum one-on-one at their own pace.

  • Rates: High-end, reflects expert coaching. Group courses cost on the order of $1,000+ per term. (For example, a 3-week summer camp was ~$1,000 online or $3,250 in-person. The USACO Silver live course has been quoted at ~$4,000 for the full program.) The 1-on-1 lessons are similarly premium – one must enroll for at least 8 two-hour sessions, and rates can be ~$100–150/hour depending on the instructor. Merit scholarships are available for top performers.

  • Contact: Email: info@alphastar.academy | Phone: +1 (650) 273-4089. Address: 100 Saratoga Ave, Suite 100, Santa Clara, CA 95051. Web: [alphastar.academy】 (contains course catalog, schedules, and results).

• Art of Problem Solving (AoPS) Academy – Bay Area – AoPS Academy runs after-school enrichment learning centers in several South Bay/Peninsula locations, bringing the renowned AoPS curriculum to in-person classes. There are five Bay Area campuses: Santa Clara, Saratoga, Fremont, Pleasanton, and Mountain View. AoPS Academy offers courses in advanced math and science (and some language arts) for grades 2–12, all aimed at high-performing students. AoPS is famous for its problem-solving approach: students learn to tackle problems they’ve never seen before using fundamental strategies.

  • Subjects & Ages: Mathematics (core and contest), Science, and Language Arts for elementary, middle, and high school (ages ~7–16). Math courses cover everything from Grade 2 arithmetic up to High School Olympiad Math. For example, offerings include Middle School Contest Math (preparing for MATHCOUNTS and AMC 8/10), High School Contest Math (AMC 10/12, AIME, etc.), as well as rigorous year-long courses in Algebra, Geometry, Calculus, and beyond. Science courses (for middle grades) and academic writing courses are also available at some locations.

  • Enrichment & Competition: Both enrichment and competition training. AoPS’s curriculum is inherently competition-oriented – it fosters the same skills needed for AMC and other contests. Specific contest prep classes are part of the roster (for example, High School Contest Math explicitly prepares students for AMC/AIME and major invitational contests). Even in non-contest classes, the problems are challenging and outside-the-box, aiming to “teach problem-solving skills through advanced math”. Many students in AoPS Academy go on to qualify for AIME, USA(J)MO, or excel in their school math teams.

  • Format: Weekly small-group classes (after school or weekends). Academic Year courses run roughly 36 weeks (late August–May) with one class per week. Each class is 1.5 to 2 hours depending on level. Class size is kept small (around 12 students on average) to ensure interaction. Summer sessions and camps are also offered in shorter intensives. All AoPS Academy instructors are vetted for both content expertise and teaching ability; many are former contestants or experienced educators. Families can choose the nearest campus for in-person learning; during COVID, AoPS also launched a Virtual Campus for online classes.

  • Rates: Moderate-High. AoPS Academy courses provide a lot of instructional hours. For instance, a 36-week year-long math course is about $2,110 in tuition (around $59/week). Shorter courses or summer camps are priced accordingly (e.g. a 12-week semester around $595). The cost includes textbooks and access to AoPS’s online resources. Need-based scholarships are available at some campuses (inquire directly).

  • Contact: Santa Clara Campus: (408) 746-1808, Mountain View Campus: (650) 262-5002, etc. (See website for each location’s contact.) Web: [aopsacademy.org】 – find Bay Area campus pages for schedules and enrollment. (Note: AoPS also offers an online school separately – see below.)

Online (Nationwide via Zoom)

• Art of Problem Solving – Online School (Nationwide) – AoPS’s online programs allow students anywhere to access its live, interactive courses in math, physics, and computer science. These courses are ideal for gifted middle and high school students seeking advanced challenges. AoPS online classes “prepare gifted students for the rigors of top-tier colleges and internationally competitive careers.” Many focus on contest math: for example, there are classes for AMC 10/12 & AIME prep, USA(J)MO proof writing, and Mathcounts/AMC 8 basics, as well as university-level topics like group theory or calculus. AoPS online also has physics courses (for STEM competition prep) and programming (USACO prep).

  • Format: Weekly online classes (usually 1.5–2 hours) using AoPS’s own virtual classroom with live video/audio and typed interactive problem-solving. Students complete weekly homework and have access to forums for assistance. Class sizes are around 15–25 with two instructors (one teaching, one assisting in chat). Courses typically run for 12–16 weeks, and AoPS offers longer 24 or 36-week “Full” courses for thorough coverage. Scheduling is flexible with sections in different time zones.

  • Subjects & Levels: Math from Prealgebra up to Olympiad Level, plus Physics and CS. Notably, AoPS online offers Contest Math series (Intro, Intermediate, and Advanced levels) and even year-long Olympiad training (e.g. WOOT – Worldwide Online Olympiad Training). Beast Academy Online is also available for younger (ages 6–13) math students.

  • Enrichment & Competition: AoPS online is synonymous with competition math. Their High School Contest Math course, for example, is a 36-week intensive covering techniques for AMC 10/12, AIME, HMMT, etc.. They also have specialty seminars for USAMO/IMO problems. For physics, AoPS has courses to prep for F=ma (physics Olympiad qualifier) as well. The style is problem-based learning – students are constantly solving new problems with guidance, rather than being lectured at.

  • Cost: Moderate for group online classes. A typical 12-week course is around $600. Longer courses cost more (e.g. the 36-week contest class is ~$2,110 as noted above). AoPS’s Olympiad training programs (like WOOT) have their own pricing (~$1,000 for the season). They do offer an “Alcumus” online practice platform and many free resources as well.

  • Contact: Enroll and contact via AoPS website. Support is available through email (onlineclasses@artofproblemsolving.com). Web: [artofproblemsolving.com/online-school】 (course catalog and schedules).

• AwesomeMath Academy (Online) – AwesomeMath provides year-round online courses and camps for students aiming at the top math competitions. It began as an elite summer program and now offers the AwesomeMath Academy, with live classes during the academic year. Courses are offered at levels from AMC 8/MATHCOUNTS up through USAMO and USAJMO preparation. There are also physics classes for F=ma (Physics Olympiad) and programming for USACO. Class sizes are kept fairly small (about 8–25 students) to ensure individual attention.

  • Curriculum & Approach: Competition-centric. AwesomeMath’s curriculum is designed to cover all topics needed for contests: algebra, number theory, combinatorics, and geometry for math contests, and mechanics and lab skills for F=ma/USAPhO. The Academy’s courses typically run for a semester, with homework, practice tests, and ongoing assessment. According to AwesomeMath, they offer 12-week live online courses for grades 4–12 specifically to prepare for AMC, AIME, USA(J)MO, and F=MA. Instructors include former Olympiad competitors and coaches. The atmosphere is rigorous but collaborative – students often form peer groups that continue outside of class (e.g. study groups for AIME).

  • Schedule & Format: Weekly live classes via Zoom (usually 1.5–2 hours). They often have an Fall session and Spring session (each ~12 weeks), as well as shorter intensive workshops before major contests. In summer, the flagship AwesomeMath Summer Program is a separate immersive camp (3 weeks long, with options for either online or residential). For the Academy classes, students may be given diagnostic tests to determine the appropriate level.

  • Rates: Competitive pricing. The Academy’s tuition is about $595 per semester (12 weeks), or roughly $1,071 for a full year course if paid upfront. This works out to around $50 per class meeting, which is good value given the specialized coaching. The intensive summer camp (online) is separately priced (approximately $1,000 for three weeks). Some scholarships are available for strong performers who need financial aid.

  • Contact: Email: admissions@awesomemath.org | Web: [awesomemath.org】 – here you can find the course catalog, schedules, and registration. (AwesomeMath is based in Texas, but all programs are open to Bay Area students via e-learning.) They also have a helpful contest resources page and publish textbooks.

• Areteem Institute (Online) – Areteem is a long-standing provider of online Olympiad prep courses in math and science. They offer a structured “Math Challenge” curriculum for grades 3–12, which includes levels aligned to contests (e.g. Math Challenge II for AMC 8, III for AMC 10/12, IV for AIME/USAMO). In addition, Areteem has live and self-paced courses for Physics Olympiad (F=ma and USAPhO), Informatics (USACO), and more. In essence, it’s a one-stop shop for STEM competition training beyond the Bay Area.

  • Courses & Focus: Both fundamentals and Olympiad-level. Areteem’s math courses start from Pre-Algebra enrichment and go up to USA(J)MO Proof Writing and Putnam training. They explicitly integrate competition problem practice into even the lower levels. For example, a Geometry course might also teach methods to solve AMC problems. On the science side, Areteem’s physics courses prepare students for F=ma through mechanics problem drills and then for USAPhO with advanced topics in thermodynamics, E&M, etc. Areteem also runs short intensive bootcamps and has prep classes for other competitions (such as the Physics Bowl, Chemistry Olympiad, and even linguistics and econ competitions in some years).

  • Format: Weekly live classes (typically 1.5 hours) in small groups, conducted via Zoom with interactive screen sharing. Sessions are often scheduled on evenings or weekends to accommodate school. Areteem also provides recorded lectures and self-paced options – students can watch videos at their own speed and then join live review sessions periodically. Homework assignments and quizzes are administered through an online platform. Many students in Areteem stay for multiple years, progressing through the levels.

  • Rates: Varies by program. A standard 8-week live online course might cost on the order of a few hundred dollars (e.g. a two-month Olympiad prep module was around $480). Longer courses (semester-long) or private tutoring would cost more. Areteem often packages courses (for instance, a USA(J)MO prep package that includes a self-paced course plus live problem-solving sessions) for a set fee. Early registration discounts and scholarships (especially for national Olympiad qualifiers) are offered.

  • Contact: Email: info@areteem.org | Web: [areteem.org – Live Courses】. The website provides a course schedule and a competition guide to help choose the right class. Areteem’s instructors (led by Dr. Kevin Wang and Dr. Jingkang “Coach J” Liang) can also be contacted for placement advice.

• Euler Circle (Palo Alto/Online) – Euler Circle is an initiative offering college-level math courses to advanced high school students, founded by Dr. Simon Rubinstein-Salzedo (Stanford Ph.D.). It’s perfect for students who have “exhausted the standard high-school math curriculum” and want to delve into higher mathematics. Euler Circle runs classes on topics like Abstract Algebra, Real Analysis, Topology, etc., often at the level of a university intro course, but taught in a more accessible style for younger students. They also occasionally run specialized camps (e.g. a Summer Number Theory course).

  • Subjects & Level: Pure Mathematics (college topics) for advanced high schoolers (and strong middle schoolers, by exception). For example, past courses include Group Theory, Graph Theory, Cryptography, Multivariable Calculus, and even graduate-level topics like Galois Theory – all tailored for pre-collegiate students. The courses are designed to give a taste of math beyond competitions: proving theorems, reading academic texts, etc. Many Euler Circle students are Olympiad-level, but the focus is more on breadth/depth of math rather than contest tricks.

  • Format: Weekly classes that run on a quarterly basis (similar to a college quarter). Each class typically meets for about 2 hours per week, over ~10 weeks. Euler Circle offers both an online section (e.g. one evening per week via Zoom) and an in-person section (often in Palo Alto or a nearby location) for each course. Homework is assigned and graded; students get experience writing proofs and may even present solutions. Class sizes are relatively small, fostering seminar-style interaction.

  • Tuition: Approximately $850–$1100 per quarter course. For instance, a “Mathematical Thinking” course might be $850, while a more advanced topic is $1100 for the term. This covers all instruction and course materials. Discounts are sometimes offered for enrolling in multiple courses or referring a friend.

  • Contact: Email: info@eulercircle.com | Web: [eulercircle.com】. Prospective students usually need to fill out an application (to ensure they have sufficient background). Dr. Rubinstein-Salzedo and his team can advise on which courses fit a student’s background. Euler Circle is an excellent stepping stone to undergraduate-level math research and has alumni who have gone on to major in math at top universities.

Sources: The information above is compiled from the official websites and materials of each program or tutor, as well as reputable educational reviews and press. Key details such as subjects, program structure, competition focus, rates, and contact information have been cited in brackets for verification. Please refer to these sources (e.g., program websites and published interviews) for further reading and the most up-to-date information on scheduling and availability.

Our Mission:

Variance is a podcast dedicated to spotlighting the vanguard of young talent—creators and thinkers who are excelling at the frontiers of their fields, from science, engineering, and AI to art, architecture, design, literature, and entrepreneurship. Our mission is to share the stories, variant insights, and journeys of these young innovators to inspire and expose listeners to truly original thinking.

The goal is to go beyond résumés and achievements. We’re curating conversations with guests who are not just brilliant, but also have a high degree of intellectual variance—people you might call quirky, weird, or non-conventional, who are just plain fun to talk to because they see the world from a unique angle.

In our episodes, we’ll dive deep into the heart of their creativity. What motivated them to act, think, and see differently? Where did they get the inspiration to make novel things? We will intentionally avoid ephemeral news and instead focus on the deeper philosophies, mental models, and creative processes that will make these conversations feel fresh and insightful a decade, or even 50 years, from now.

This is a personal passion project, not a new career. Our focus will be on quality over quantity, with 6 to 8 high-quality episodes published a year.

Our Fourth Episode: Teaching AI How to Be Moral (Jared Moore of Stanford)

Can we make moral AI agents? Can these agents get good enough to provide therapy and other personal services to humans, and even if they can, is that a good idea? Are language models sentient and deserving of moral concern - and how would we know? How do we incorporate a pluralistic set of views into AI systems?

Join Jared Moore, a computer scientist, AI alignment researcher, and educator probing how large language models understand (and sometimes mis­understand) human minds and values. Now at Stanford University, he investigates social reasoning, theory-of-mind, and the pitfalls of machine deception while co-creating courses like “How to Make a Moral Agent.” Jared blends rigorous research with creative outreach—publishing on pluralistic alignment, writing a satirical novel about conscious AI, and building installations that turn code into poetry—to push the question: how can we make AI systems reliably do what we want, for everyone’s benefit?

Show Notes:

Jared’s Personal Website

How to Make a Moral Agent

Why LLMs Won’t Replace Therapists Anytime Soon

Are Large Language Models Consistent over Value-laden Questions?

The Strength of the Illusion: a satirical novel about AI

Jake and I invite you to listen to our podcast, leave feedback and a star rating, and subscribe on your preferred platform. I’m incredibly excited to build this platform for young voices and share their journeys with you. If you like these first few episodes, please leave a star rating. If you didn’t like it, want something different, have a variant young guest we should talk to, or just want to troll us, drop us a note!

• Website: variancepod.com

• Apple Podcasts: Listen on Apple Podcasts

• Spotify: Listen on Spotify

• RSS: Libysn Feed

Hope you enjoy the conversation.

Arun & Jake
arun@variancepod.com
jake@variancepod.com

Books may be the only material objects that bring me bliss and blessings - perhaps art and trees come a close second. My reading was modestly more focused this year around a few themes: Von Neumann, the Enlightenment, fractals, agentic systems, and fiction with spare prose. When I reflect on how ordered and structured my life is with its routines and responsibilities, I’m glad that the books I read are one of the few areas of wildness and entropy, the pathways to other worlds far different from my own.

AI papers again dominated and despoiled my attention. One survey paper even made my “best of the year” picks. 2025 was another golden year in AI research as reasoning and agentic systems deepened. We saw the first good papers on continual learning and recursive self-improvement (the first, momentous steps). While many wonder about what a drive to AGI or a singularity may look like, I think the most striking, upcoming moment in human history will be “First Contact”, after memory scaling and identity formation get built into autonomous agents. We will then realize we have birthed a new alien species and will have to meet it on uncertain terms.

I’ve been working with some alignment researchers, flourishing experts, and founders to sketch out and help build up the field of “positive alignment“, which will consider how to make AI systems understand and promote human flourishing for individuals, communities, societies, and overlapping ecological systems on earth.

Finally, Jake Kraft and I have been interviewing some wonderfully variant people about their outlier views and worldviews, as part of our Variance Podcast. Don’t expect many episodes each year, as our goal is to create prickly, archival recordings that you’ll want to revisit over time.

I had a five-way tie for the best books this year: Mokyr’s A Culture of Growth on how societies build or block the conditions for sustained progress; John Williams’ masterpiece Augustus, and the runner-up, Stoner, as intimate studies of character under the slow pressure of duty and power; Dhami and Sunstein’s Bounded Rationality as the clearest bridge from “humans are cognitively weird” to “policy should be designed accordingly”; Liu et al.’s Advances and Challenges in Foundation Agents as a big-map survey of how modern agents stitch together memory, planning, tools, and safety constraints into something that actually acts in the world, and; von Neumann’s core computing writings (Compendium, EDVAC, Self-reproducing Automata, and The Computer and the Brain) arguing that computation is best understood as a class of automata with robustness coming from adaptation and distributed memory.

After the best books, I continue with my full reading trace and then my short section about my favorite technical papers, audio and video, broadening from books to culture (hat tip to Piero Scaruffi, who makes great year-end lists).

___________________________________

BEST BOOKS OF 2025

Mokyr, A Culture of Growth: The Origins of the Modern Economy. What led from the Enlightenment and Republic of Letters to modern growth; less a tech miracle than a social permission structure for ideas, a borderless group of intellectuals pushing their views on science, liberalism, and progress.

Williams, Augustus. One of the few perfect novels I’ve read. It covers Late Republican and Early Imperial Rome told through letters, documents, and voices; an intimate and painful view of power and sacrifice. Many of the literati prefer Williams’ other book, Stoner, a devastatingly quiet novel about an ordinary academic life and how “small” choices become your whole fate (it’s good, but Augustus is perfect).

Dhami and Sunstein, Bounded Rationality: Heuristics, Judgment, and Public Policy. A rigorous overview of the standard view of Bayesian rationality and ultimately the limits of cognition and rationality, from humans to silicon agents, through the viewpoint of behavioral economics, institutions, and other constructs of decision science.

Liu et al, Advances and Challenges in Foundation Agents [2025]. Probably my single favorite AI paper of the year. It’s a systematic survey of foundation agents and the hundreds of papers on the primitives needed to get to AGI (cognition, memory, world models, rewards, emotion modeling, perception, action systems, self evolution, scientific discovery, collaborative multi-agent systems, collective intelligence and adaptation, intrinsic safety, superalignment). This paper is the length of a book but is worth a read.

von Neumann, The Neumann Compendium; First Draft of a Report on EDVAC; Theory of Self-Reproducing Automata; The Computer and the Brain. I was sidetracked from reading Herbert Simon to explore a bunch of Von Neumann’s papers, books, and speeches. JvN deserves his reputation - he may even outshine Turing and Shannon in the long run, working his way toward Gauss and Euler. The Compendium is a good place to start, and it has Ulam’s homage to his best friend. JvN opines that computing needs to be both serial and parallel, digital and analog - to maximize the bounds of what can be computed and parallel nature. He posits that all computing is a subset of the class of automata, all math can be reduced to computing, and that logical neurons are a likely substrate for computers. Finally, he discusses how reorganization and adaptation capability are key, along with distributed, non-localized memory, which he sees as increasing survival and robustness. I would recommend complementing the books above with MacRae’s biography of Von Neumann, his brother’s memoirs, and the wonderful Labutat novel about Von Neumann called “Maniac” (which is near-perfect but flawed by starting with poor Paul Ehrenfest and not Leibniz, Boole, Babbage, Lovelace, Jacquard, or Turing). Finally, the one part of JvN’s work that still scares computer scientists but will be crucial to create material abundance and accelerate space exploration and colonization will be the kinematic self-reproducing machines he describes in what may be his most important work.

Von Neumann, Feynman, and Ulam on a bench [1949]. Three of the brilliant minds of the 20th century are relaxing at Los Alamos (specifically at the Bandelier National Monument Lodge) during the post-Manhattan Project era. The inventor of the modern computer architecture, replicators, and game theory in a natty white suit; the discover of QED and proposer of nanotech, sitting like a hobo; and the creator of Monte Carlo methods, cellular automata, nuclear pulse propulsion, and the Ulam-Teller death device to end humanity. _{Photograph by Charles A. Lehman. Courtesy of Los Alamos National Laboratory and AIP Emilio Segrè Visual Archives.}

___________________________________

The full list is here:
https://raohacker.com/best-books-of-2025-math-as-patterns-enlightenment-and-we-can-tokenize-everything/

I attended the Progress Conference in Berkeley, California, for the second consecutive year, focusing on the emerging field of Progress Studies. It was at the Lighthaven Center run by a thoughtful group of effective altruists and rationalist thinkers. The attendees were a thoughtful group of tech founders, scientists, economists, policy thinkers, government officials, and some superannuated corporate types (me). Keynotes included Sam Altman (OpenAI), Mike Kratsios (US OSTP advisor to the President), Blake Scholl (of Boom), Jennifer Pahlka (of Recode America), and more.

First, many thanks to Ben, Jason, Heike, and the Roots of Progress team, along with their sponsors—and to Tyler Cowen, the field’s catalytic spirit. The structure and conversations were excellent. Second, I send my gratitude to all the attendees who indulged me with their works, ideas, and dreams. I have not named specific people due to the quasi-Chatham house rules, but this cast of characters made the conference fun in a reflective and chatty way. A final observation was that SF is booming now, as the default center of global AI, and Berkeley was its outpost of weirdness and variant thinking.

I attended 10 to 12 talks, gave an update talk on my AMP Program to raise genius math and physics kids, discussed my positive alignment work, and had some intellectually stimulating conversations with around 80+ of the 300+ attendees.

Here were some of the most stimulating themes from listening and talking to many brilliant attendees:

AI exponential growth: I start with AI, since it’s central to the modern world. Most people working in the field still felt we were in the exponential stage of AI development and diffusion (and that eventually we will hit walls that make this a sigmoid). Hence, the noise in the news about current LLM reasoning models and agents leveling out was worth ignoring. We have a clear path to AGI, which was a fuzzy/nebulous and slightly taboo concept to discuss for many years, until the researcher community came to accept it. We also had a new AGI evaluation paper drop during the conference from a respectable group of researchers (and building on the recent GDPeval work).

Artificial General Intelligence: AGI broadly is an AI system that matches the best human experts (95th percentile or higher), working remotely on a computer, in a wide range of fields and tasks. Almost everyone in the field now thinks we will get AGI in 2 to 10 years (a wide range, but the modal value is 2 to 5 years). This is in line with expert surveys - AGI before 2030.

Superintelligence: The much harder question is superintelligence, SI. These are systems that are vastly superior to a group of many humans (say 100 or more) in vast areas of work or research. This is a more controversial field, though I expect in 3 to 5 years, most researchers will expect this to happen, especially if we can build recursive self-improvement and self-evolving systems. One final strand was recursive self-improvement for hardware - building other chips, data centers, power plants, etc - we need more players in this space, like Google did for its TPU development.

AI superalignment: If we get AGI relatively soon and SIs within 3 to 5 years after, we need them to be closely aligned to human values, goals, needs, etc, and not rogue or amoral systems with vastly different aims. A few people talked about a research agenda of Positive Alignment, which aims to cultivate LLMs, reasoning models, and agents that are not only safe but also actively promote human flourishing in a pluralistic, polycentric, and contextual way, within the context of how LLMs and agents are trained. This is AI superalignment, and it becomes more important as we get closer to recursive self-improvement and models self-evolving without humans. Six months ago, I’d say this was science fiction. But now it’s an actual field with published research, with early ideas and systems that work. A final thought from one attendee: “What is the prompt you type into a superintelligence before you send it out into the world to be fully autonomous from humans?”

AI-native organizations: As simpler AI takes over orgs, there were multiple discussions on how our decrepit 20th-century organizations - often modeled on 17th-century organizations like the East India Company, Royal Society, Massachusetts Bay Company, Dutch Republic to US Congress - need to evolve and adapt. We had numerous conversations about AI-native startups and AI-native scientific labs, on what founders and Principal Investigators are doing, and also early experiments in government agencies and shifting large corporations to make AI central to all workflows and productivity. Add-ons won’t work. We will have to redesign many organizations from the bottom up with first principles. One idea was to bring a Pirate Party approach to the US Congress and see if we could get representatives to talk to more constituents, and then get more summarized and nuanced views and ideas on an intelligent Pareto frontier of what bills to submit or vote on. A final question: Ownership and liability for AI agents - what is the capability and agency threshold for us to let it operate - what if it hires compute capacity from a rogue nation?

Energy mix (nuclear and solar) and datacenters in space: Energy is still the bottleneck of civilization, the limiter of Progress, as primary energy use per person correlates well with GDP per capita. Compute, intelligence, and GDP all scale with total energy throughput. In the near term, many people think natural gas is the practical bridge; in the long term, the fight is between advanced fission/fusion and massive solar. Energy and compute remain the hard limits on our Kardashev level. Almost every country is falling behind on expanding per-capita energy use over time, with the partial exception of China (per capita energy use over time). As demand for intelligence and large-scale compute keep exploding, we’ll eventually need huge data centers in space — work that companies like Star Cloud have already begun. The open question is: once we’ve pushed flops per watt, what’s the next frontier?

Longevity and human life and healthspans: We had a bunch of longevity freaks there. I love these people. They pointed out that we have many treatments to extend the lifespan of lab mice and other small animals by 75% or more, or to increase their intelligence or strength. We have been afraid to use these in humans, but I hope the science labs and startups working on this can get unblocked, as that will mean life and healthspans for 120 to 200 years (and longer). If this sounds like crazy talk, read Andrew Steele’s book “Ageless” or papers like this one on “Sex-specific longitudinal reversal of aging in old frail mice” (Treatment of old frail male mice with OT+A5i resulted in a remarkable 73% life extension from that time, and a 14% increase in the overall median lifespan). I tell my wife there’s a 30-40% chance of making it to age 200, and she thinks I’m a nutter.

Reskilling, Swiss internships, and returns to college: The massive changes coming from AI will require rapid re-skilling at the country-to-city level, ideally paid for by temporary employer or industry-level automation taxes. One thought was to adopt the Swiss model of internships, where 70% of high school students get paid to work at companies and learn skills, and 30% learn general curricula and go to college. See notes here. Some also discussed how the returns to getting a college degree should go down, while returns to using AI expertly should go up, esp for doing new science and engineering. This is for doing new jobs and existing jobs better. Example: the average Silicon Valley engineer's workflows massively changed in 2025, decreasing the need for SWEs and increasing it for MLEs and senior engineers. This raised the question: how to teach people to use AI better? You just get better with usage and use it over time - human experiments and creativity will win out. One practical tip: Ask ChatGPT “teach me how to use you for XYZ task.”

Automation, labor markets, and when will Washington DC wake up: There were many conversations on how increasing automation from AI will affect labor markets, and how little we know about this. The common view was that while DC and some in the Tech Right talk about automation, it likely won’t be an issue in the 2026 election and possibly not even in 2028 (though that’s harder to predict).

The Silicon Valley Tech Canon vs the China Tech Canon: I discussed with a few people the most common books and ideas in the Silicon Valley Tech Canon (eg Deutsch, Isaacson, Paul Graham, Blindsight, Culture Series, etc), and one attendee had a great post on the Chinese Tech Canon. The lists only overlap like 40%, so it’s interesting to see the divergence. What surprised me about the Chinese Canon was the legal realist philosophers like Han Feizi, Chairman Mao (on guerrilla warfare, propaganda, and motivating people), and the novelist Jin Yong (“Every man must read Jin Yong,” according to Jack Ma).

Bottlenecks: A common theme for the policy crowd is how many silly, anti-growth and anti-tech bottlenecks exist across Western societies, put in place by liberal and conservative governments. The biggest ones were on energy, housing, job rules, manufacturing, and Baumol cost goods. There was a general sense that the government has been more a hindrance than a help, but that it can be fixed with better, simpler, and fewer laws and regulations. The tone was not to replace governments with more anarchic or libertarian arrangements (though some attendees leaned that way), but rather to make it more effective and pragmatic, improving state capacity.

Environmental Policy Self-Harm: This wasn’t explicitly stated, but many conversations took the line that environmentalists were well-intentioned but counterproductive to their own goals. Their policies and legacy have done a great amount of harm by being overinclusive and not well-tailored, often leading to great environmental damage (eg, shutting down nuclear plants led to much more fossil fuel usage and faster climate change). One paper cited was “Prevented mortality and greenhouse gas emissions from historical and projected nuclear power”, which estimates that “taking into account the effects of the Fukushima accident, we find that nuclear power could additionally prevent an average of 420,000-7.04 million deaths and 80-240 GtCO2-eq emissions due to fossil fuels by midcentury”. If you add excess mortality from heat death because energy is too expensive in Europe, this number goes even higher.

Evolution of education: AI and the modern techno-industrial complex have made all current K-12 public and private education obsolete, and likely universities. Some attendees came out and said this, otherwise implied it, and a few were working on alternatives similar to even more radical than Alpha School (the reference/starting point for what a new school should look like). The big idea was that we need to encourage more personalization, high agency, teamwork, AI collaboration, and the solving of big problems posed by children and adult mentors. Hence, we need fewer classrooms and textbooks, less linear and rote learning. A few attendees had built intriguing alternative schools to experiment and push these boundaries. One controversial topic was the value of modern research universities, given extreme political partisanship and courses that seemed far behind the edge of science and industry. Should kids take college credits early (starting at age 12-13) and go straight to work and building things at age 18, or hang out for a full undergrad and grad school and start life at ages 22 to 30?

How the US federal government uses AI, from insiders: There is no LLM use in White House due to the Presidential Records Act - though there may be some servers in the EO buildings that staff can go out to use. Most federal agencies are not using AI models; they see it as different than a normal Saas app. We need GPT for new coordinating and information processing- electricity, computers, and internet in Congress - still have a divide between pre- and post-internet representatives. There is a jagged frontier of enforcement of existing laws - these work with simple, passive enforcement, but maybe highly oppressive and terrible when AIs observe and enforce laws. Finally, the US Constitution (Article IV, Section 4) guarantees a “Republican form of government”, which is notably not a full democracy and has anti-democratic elements in the Senate, executive, and judges. Too much democracy could be a problem, per the Founders, and it’s unclear what space this makes for AI. It’s likely all four branches of government will be heavy AI users in time.

How society works and maintaining civilizational artifacts we take for granted. Progress also depends on citizens’ understanding of the physical systems that sustain modern life. Few Americans know how our sewage system, electrical grid, road and highway infrastructure, internet and telecom, and other systems work, and that’s a problem as these age out and we need to replace them. A common theme was that Asian countries are ahead of the US in this (esp China and South Korea, but surprisingly parts of India), and that much of the US was already a second-world country when it came to infrastructure.

Books, Privacy, and Propaganda: Interesting snippets from one speaker: “Books have remained while other objects have become obsolete due to tech change - but will there be a new and better way to interact with ideas and books?” “We need a right of privacy for LLM discussions; revealed preferences are that people are already doing this, having sensitive conversations with LLMs, and we need to formally protect them.” [One attendee who built an ecosystem of hyper-private apps had a fully-private LLM]. And finally, this dark point on our susceptibility to LLM influence: “Never believe that propaganda doesn’t work [to fool] you - they just haven’t found the right levers for you.”

Ambition - why it’s important and how to raise it. Many attendees, and especially the hyper-frenetic startup founders or renowned scientists, felt that a limiter in their life and work was self-imposed caps on ambition (eg “I can’t be a great founder and a great husband/dad/wife/mom”, or “I can’t work on extending human lifespan to 120 or higher because many of my colleagues at the university will shun or ridicule me.” We discussed genetic treatments to increase intelligence, strength, lifespan, and healthspan, or even resistance to sickness. The healthy root of ambition was to do meaningful things to help others, and to make the best use of our time and energy.

Thoughts on raising ambition in adults: This revolved around taking long horizon visions and goals, a decade or longer, or entire lifetime - just sheer persistence and compounding; don’t account for the view of peers or social proof; treat work life balance is a bad idea - an artificial split between the two, an assumption they are separate- but work goes in your life and brings joy, especially if it’s your life’s work; get past simple trade offs, do overlaps and allocate energy better; your flaws matter less than you think - double down on strengths, do the weak or flawed things less.

Tips on raising ambition for kids: Show them plenty of history, stories, biographies, sci-fi of impressive people; do many projects, internships, and apprenticeships; expose them to other ambitious people to make them more ambitious (eg David Senra with the Founders podcast).

Field-building: A final theme was that to make progress on many issues, you had to build entirely novel fields such as: quantum energy; accelerated or hyper-education; AGI; superalignment and positive alignment; the psychology of ambition or increasing agency; etc. Some conversations were about how new fields had active builders who helped put them together (eg Bohr, Fermi, Bethe and others for quantum mechanics, or Bardeen to Goodenough for solid state physics, etc). Every major leap forward (from quantum physics to superalignment) began when small groups decided to found entirely new fields.

Books Discussed or Recommended

Deutsch, The Beginning of Infinity

Penrose, The Road to Reality

Weinberg, Super-Thinking [Mental Models]

Gibson, Paper Belt on Fire

Steele, Ageless

Mann, The Wizard and the Prophet

Cowen, Talent

Brand, Maintenance: Of Everything

Labatut, The Maniac [novel about Von Neumann]

Potter, The Origins of Efficiency

Dunkelman, Why Nothing Works

Walker, Why We Sleep

Panda, The Circadian Code

Klein/Thompson, Abundance

Liberty and the Irish Enlightenment

Many of the people I enjoy hanging out with still constantly give me recommendations, like a crypto founder who recently told me he wanted to haunt the future like a self-replicating Von Neumann probe, and then recommended Taylor’s Bobiverse series of books, starting with “We are Legion (We are Bob)” as a fun bedtime read.

But is there a tech canon?

Patrick Collision recently kicked off a discussion on X, and Tanner Greer wrote more about the books that were recommended. More recently, some of the NVIDIA and OpenAI books may make this list, but it’s TBD (eg. The NVIDIA Way, The Thinking Machine, The Optimist, Empire of AI).

Here is my slightly edited version of the list, based on conversations with many people in SF and around the Bay Area:

• Asimov (1940, 1951, 1955), Robot Series, Foundation, and End of Eternity (the last was gifted to me as a bachelor party present)

• Rand (1943, 1957), Fountainhead, Atlas Shrugged (often people like one and dislike the other)

• Deutsch (2012), The Beginning of Infinity

• Brand (1968-1972), The Whole Earth Catalog

• Brooks (1975), The Mythical Man-Month

• Pirsig (1974), Zen and the Art of Motorcycle Maintenance

• Caro (1975), The Power Broker

• Dawkins (1976), The Selfish Gene

• Alexander (1977), A Pattern Language

• Card (1977), Ender’s Game

• Hoftstader (1979), Gödel, Escher, Bach

• Papert (1980), Mindstorms

• Wolfe (1983), “The Tinkerings of Robert Noyce”

• Ableson and Sussman (1984), Structure and Interpretation of Computer Programs

• Feynman (1961-64, 1985), Feynman Lectures on Physics, Surely You’re Joking, Mr Feynman!

• Penrose (2007), Road to Reality: A Complete Guide to the Law of the Universe

• Rhodes (1985), The Making of the Atomic Bomb

• Carse (1986), Finite and Infinite Games

• Stephenson (1992-2015), Snow Crash, The Diamond Age, Cryptonomicon, Anathem, Seveneves.

• Rich (1996), Skunk Works

• Graham (1998-2024), Essays

• Scott (1998), Seeing Like a State

• Raymond (1999), The Cathedral and the Bazaar

• Hiltzik (1999), Dealers of Lightning

• Davidson and Rees-Mogg (1999), The Sovereign Individual

• Waldrop (2001), The Dream Machine

• Morris (2001), The Rise of Theodore Roosevelt

• Symonds (2003), Softwar

• Kushner (2003), Masters of Doom

• Cowen (2003-2025), Marginal Revolution

• Chernow (2004), Titan (sometimes Hamilton or Washington)

• Markoff (2006), What the Dormouse Said

• Livingston (2008), Founders at Work

• Meadows (2008), Thinking in Systems

• Reis (2011), The Lean Startup

• Herztfeld (2011), Revolution in the Valley

• Jackson (2012), The PayPal Wars

• Alexander (2013-2025), Slate Star Codex/Astral Codex Ten

• Bostrom (2014), Superintelligence

• Thiel (2014), Zero to One

• Horowitz (2014), The Hard Thing about Hard Things

• Gordon (2016), The Rise and Fall of American Growth

• Vance (2017), Elon Musk

• Isaacson (2005-2023), Benjamin Franklin: An American Life, Einstein: His Life and Universe, Steve Jobs, Leonardo Da Vinci, The Code Breaker, Elon Musk.

• Grove (1995), High Output Management (Worth reading with Schmidt’s “Trillion Dollar Coach.”)

• Kurzweil (2005), The Singularity is Near (or the new one, “The Singularity is Nearer”)

• Thorndike (2012), The Outsiders

• J.R.R. Tolkien (1954–1955), Lord of the Rings

• Douglas Adams (1979), Hitch-Hikers Guide to the Galaxy

• Andy Weir (2011), The Martian

• Iain Banks (1988), The Player of Games (sometimes Consider Phlebas)

• William Gibson (1984), Neuromancer

• Daniel Kahneman (2011), Thinking Fast and Slow

• Taleb (2012), Anti-Fragile

• Hargittai (2006), The Martians of Science

• Wensberg (1987), Land’s Polaroid: A Company and the Man Who Invented It

• Watts (2001), The Magic Kingdom: Walt Disney and the American Way of Life

• Hamming (2020 reprint), The Art of Doing Science and Engineering: Learning to Learn

• Aurelius (c.180), Meditations

• Seneca (c.65), Letters from a Stoic

• Epictetus (c.108), Discourses

• Eswaran (2009), Upanishads

• Callenbach (1975), Ecotopia

• Munger (2005/23), Poor Charlie’s Almanack: The Essential Wit and Wisdom of Charles T. Munger

• Bennett (2023), A Brief History of Intelligence

• Gertner (2012), The Idea Factory: Bell Labs and the Great Age of American Innovation

• Hiktzik (1999), Dealers of Lightning: Xerox PARC and the Dawn of the Computer Age

• Macrae (1992), John Von Neumann: The Scientific Genius Who Pioneered the Modern Computer, Game Theory, Nuclear Deterrence, and Much More (I recommend reading this with “The Neumann Compendium” edited by F. Brody and Tibor Vamos)

• Catmull (2023), Creativity, Inc (Expanded Edition)

• Westfall (1981), Never at Rest: A Biography of Isaac Newton

• Mahon and Forbes (2014), Faraday, Maxwell, and the Electromagnetic Field: How Two Men Revolutionized Physics (Read with Cropper’s “Great Physicists”).

• Watts (2006), Blindsight

• Simmons (1989), Hyperion

• Minsky (1986), The Society of Mind

• Borges (1944), Ficciones

• Tufte (1983), The Visual Display of Quantitative Information

• Csikszentmihalyi (1990), Flow: The Psychology of Optimal Experience

• Ferguson (2008), The Ascent of Money

Our Mission:

Variance is a podcast dedicated to spotlighting the vanguard of young talent—creators and thinkers who are excelling at the frontiers of their fields, from science, engineering, and AI to art, architecture, design, literature, and entrepreneurship. Our mission is to share the stories, variant insights, and journeys of these young innovators to inspire and expose listeners to truly original thinking.

The goal is to go beyond résumés and achievements. We’re curating conversations with guests who are not just brilliant, but also have a high degree of intellectual variance—people you might call quirky, weird, or non-conventional, who are just plain fun to talk to because they see the world from a unique angle.

In our episodes, we’ll dive deep into the heart of their creativity. What motivated them to act, think, and see differently? Where did they get the inspiration to make novel things? We will intentionally avoid ephemeral news and instead focus on the deeper philosophies, mental models, and creative processes that will make these conversations feel fresh and insightful a decade, or even 50 years, from now.

This is a personal passion project, not a new career. Our focus will be on quality over quantity, with 6 to 8 high-quality episodes published a year.

Our Third Episode: Algorithmic movies, AI Dogs, and the Afterlife (Miguel Novelo of Stanford)

How valuable are inanimate material objects, and can we have relationships with them - from computers to art? Why do we have dogs in our lives, which our laws treat as objects, when dogs have no say in the "relationship"? Will our new synthetic beings, our AI and robotic mind children, be more like our pets, or will we be their pets?

Join Miguel Novelo, an artist, researcher, and community organizer currently working on algorithmic movies on geological change, game engine storytelling based on dogs-human-technology afterlife, technoshamanisms, technology displacement, and technophobias. He is currently an independent artist and Lecturer at Stanford's Art and Art History Department and San Jose State University.

Show Notes:
Miguel's Personal Website
Interactive Installation: "Ｃｈｕｐａｆｌｏｒ： Ｗｈｉｓｔｌｉｎｇ； Ｄｏｇ Ｓｐｉｒｉｔ， Ａｑｕｉ" (2023)
Sonic Sculpture: Judgment Pelicano: Impact (2022)
Experimental Films: Super 8 and 16mm (2020s?)

Jake and I invite you to listen to our podcast, leave feedback and a star rating, and subscribe on your preferred platform. I'm incredibly excited to build this platform for young voices and share their journeys with you. If you like these first few episodes, please leave a star rating. If you didn’t like it, want something different, have a variant young guest we should talk to, or just want to troll us, drop us a note!

• Website: variancepod.com

• Apple Podcasts: Listen on Apple Podcasts

• Spotify: Listen on Spotify

• RSS: Libysn Feed

Hope you enjoy the conversation.

Arun & Jake
arun@variancepod.com
jake@variancepod.com

Our Mission:

Variance is a podcast dedicated to spotlighting the vanguard of young talent—creators and thinkers who are excelling at the frontiers of their fields, from science, engineering, and AI to art, architecture, design, literature, and entrepreneurship. Our mission is to share the stories, variant insights, and journeys of these young innovators to inspire and expose listeners to truly original thinking.

The goal is to go beyond résumés and achievements. We’re curating conversations with guests who are not just brilliant, but also have a high degree of intellectual variance—people you might call quirky, weird, or non-conventional, who are just plain fun to talk to because they see the world from a unique angle.

In our episodes, we’ll dive deep into the heart of their creativity. What motivated them to act, think, and see differently? Where did they get the inspiration to make novel things? We will intentionally avoid ephemeral news and instead focus on the deeper philosophies, mental models, and creative processes that will make these conversations feel fresh and insightful a decade, or even 50 years, from now.

This is a personal passion project, not a new career. Our focus will be on quality over quantity, with 6 to 8 high-quality episodes published a year.

Our Second Episode: AI Reliability and Humans Testing Language Models (Anastasios Angelopolous of LM Arena)

How fast is AI really improving, and how do we know? What guarantees can we expect from AI systems to be robust and reliable? What is AGI and have we gotten there? Can AI systems show creativity or even sentience?

Join Anastasios Angelopoulos as he lays out his thoughts to these hard questions, as he and his partners build the world's most sophisticated ways to test LLMs as they get better faster than everyone expects.

Show Notes:
Anastasios's Personal Website
Conformal Prediction (Science of AI reliability)
LM Arena (Humans testing LLMs)
DeepSeek and DeepSeek R1

Jake and I invite you to listen to our podcast, leave feedback and a star rating, and subscribe on your preferred platform. I'm incredibly excited to build this platform for young voices and share their journeys with you. If you like the first episode, please leave a star rating. If you didn’t like it or want something different, drop us a note (we really care)!

• Website: variancepod.com

• Apple Podcasts: Listen on Apple Podcasts

• Spotify: Listen on Spotify

• RSS: Libysn Feed

Hope you enjoy the conversation.

Arun & Jake
arun@variancepod.com
jake@variancepod.com

In a world saturated with hot takes and ephemeral news, I’ve been searching for conversations with more staying power. I wanted to create a space to explore the kind of fresh, original thinking that truly pushes our world forward.

That's why I'm thrilled to announce the launch of my new podcast, co-created with Jake Kraft, Variance.

Our Mission:

Variance is a podcast dedicated to spotlighting the vanguard of young talent—creators and thinkers who are excelling at the frontiers of their fields, from science, engineering, and AI to art, architecture, design, literature, and entrepreneurship. Our mission is to share the stories, variant insights, and journeys of these young innovators to inspire and expose listeners to truly original thinking.

The goal is to go beyond résumés and achievements. We’re curating conversations with guests who are not just brilliant, but also have a high degree of intellectual variance—people you might call quirky, weird, or non-conventional, who are just plain fun to talk to because they see the world from a unique angle.

In our episodes, we’ll dive deep into the heart of their creativity. What motivated them to act, think, and see differently? Where did they get the inspiration to make novel things? We will intentionally avoid ephemeral news and instead focus on the deeper philosophies, mental models, and creative processes that will make these conversations feel fresh and insightful a decade, or even 50 years, from now.

This is a personal passion project, not a new career. Our focus will be on quality over quantity, with 6 to 8 high-quality episodes published a year.

Our First Episode: A Conversation with Dan Spiegel of SAW Architects

To kick things off, our first episode features a conversation with Dan Spiegel, an architect and co-founder of the award-winning firm Spiegel Aihara Workshop (SAW). Dan is the perfect embodiment of what Variance is all about. He operates at the nexus of architecture, landscape, and public policy, challenging the very boundaries that typically define those fields. His work is driven by a search for "productive tension," where buildings and landscapes are not just complementary but are designed to be the same evolving entity. This is an architect who designs for unpredictability, creating frameworks that allow a place to emerge organically over decades rather than being frozen in time.

Our conversation explores the variant mind behind this philosophy. We dive into his speculative projects that tackle major urban challenges—like a proposal for collective housing that literally bridges property lines to fight the affordability crisis—and his built work, like a low-cost "prefabrication hybrid" home in Maui designed as an adaptable prototype. We don't just talk about buildings; we discuss the unconventional mental models and first principles he uses to solve problems, revealing a way of thinking that is essential for anyone interested in how creative and divergent minds see the world.

I invite you to listen to our debut and subscribe on your preferred platform. I'm incredibly excited to build this platform for young voices and share their journeys with you. If you like the first episode, please leave a star rating. If you didn’t like it or want something different, drop us a note (we really care)!

• Website: variancepod.com

• Apple Podcasts: Listen on Apple Podcasts

• Spotify: Listen on Spotify

Hope you enjoy the conversation.

Arun & Jake
arun@variancepod.com
jake@variancepod.com

The Philosophy of AI

A friend and mentor recently asked me:

“Who’s able to go meta on AI right now? What should I read for a philosophical take on the thing we call 'artificial intelligence'?”

I don't know of any good single source, or even a single approach. Here were my suggestions:

* The philosophy of Silicon Valley: Many thinker-doers here have been deeply influenced by Deutsch's "Beginning of Infinity." I read it a decade ago and still think a lot about it and will re-read it soon - it's perhaps the most important philosophical text written in the last 75 years. It pairs well with Kurzweil's books on the singularity (the most recent is "The Singularity Is Nearer").

* First and second wave AI researchers' final work: The initial groups that conceptualized and pushed AI forward (Minsky, Simon, Shannon, Newell, Turing, McCarthy, Weiner, Holland, Moravec, etc) became more philosophical toward the end of their careers. I've been slowly working my way through some of their books and papers to understand where they ended up and what they learned - they had great constructs, but not the required data and compute, and not the right mixture of connectionist math (stats, linear algebra, multi-variable calc, topology) that won the day. Minsky's "Society of Mind" and Simon's "Sciences of the Artificial" are good starting points (and these researchers have plenty of papers to dig through on Google Scholar). You could also take a historical approach and combine Nilsson's "The Quest for Artificial Intelligence - Stanford AI Lab" with Metz's recent "Genius Makers" book (or read the great historical footnotes in the Russell/Norvig AI textbook).

* Recent thinkers on intelligence: We don't really know what intelligence is, or even how to measure it, and I've been thinking about this non-stop for the last two years (and have talked to designers of the best human psychometric exams along the way, like WAIS and more). "A Brief History of Intelligence" by Bennett is a good adaptive/evolutionary perspective on intelligence, alongside Bostrum's books ("Superintelligence", "Deep Utopia") and Tegmark's "Life 3.0". I'm a fan of more technical work like Bang Liu's survey paper on agents.

* Widely read science fiction: When people get philosophical or literary here in the Valley (not too often, maybe late at night), we refer to the same set of sci-fi classics that touch on some key themes of AI: Asimov's Robot series, Dick's "Do Androids Dream of Electric Sheep?", Banks' Culture series, Gibson's "Neuromancer", Suarez's "Daemon", Hamilton's Commonwealth series, Egan's "Permutation City," Stross’s "Accelerando", Ishiguro's "Klara and the Sun", etc. This may be the closet set of works to get to philosophy, and they also pragmatically act as a roadmap for what to build.

Math Precedes Reality (The Pea and the Sun)

I recently gave a casual talk at Stanford (the ATOMIC series) about some emergent math (almost a century old) that is just starting to be applied to speculative questions in physics (interpretations of the Yukawa reactions or proton-antiproton pair production in particle physics).

The Pea and the Sun. The Banach Tarski Theorem - one of the wildest mathematical results I’ve ever seen, from non-measurable set theory.

Formal statements of the Banach Tarski Theorem:

A solid ball may be separated into a finite number of pieces and reassembled in such a way to create two solid balls, each identical in shape and volume to the original (with no stretching).

A solid ball in three-dimensional space can be decomposed into a finite number of non-overlapping pieces, which can then be rearranged using only rotations and translations (no stretching or distortion) into two identical copies of the original ball.

The unit ball B={(x,y,z): X^2 + y^2+z^2 <= 1} can be partitioned into two sets B1 and B2, such that B~B1 and B~B2 (with ~ means “is piecewise congruent to” or “is equidecomposable to”).

Here are my slides and here is the YouTube video of the talk (it was private but should be public soon). Here is a slicker video about the paradox.

Coming up soon:
* The first episode in a new podcast I started with a friend to talk to variant thinkers, called Variance.

* Some thoughts on fractal bush robots (inspired by work from Moravec and the artist Ruth Asawa, whose retrospective I saw today in the SF Moma).

As an insider working at companies building state-of-the-art AI, I have some non-consensus views about how we should adapt to and co-evolve with them.

First, almost everyone I know, including the smartest, most productive, and wealthiest people are not using AI enough. What is enough? Daily to hourly. At some point, that will be continuous. What does continuous mean? I once had a curmudgeon entrepreneur mentor point out that he built his TV network media company over 25 years and then walked away after putting it on autopilot. He checked in maybe monthly for a few hours, but the company continuously ran to create media services and sent him a large annual dividend (in excess of $100mn in good years). The machine did the work; he benefitted. Today we can use AIs hourly as large reasoning models. When current embryonic agentic systems start to get better, we can embark on continuous usage. I already see a skills gap forming. One interesting labor market study recently suggested that generative AI systems are most used by the wealthiest, best-educated workers.

Second, if you know how to use AI well, you can measurably improve your agency, that is, your ability to get things done with increased intelligence, productivity, and wealth. The catch is that in these early days, no one knows how to use AI well. So that means you need to constantly experiment with doing your daily tasks, chores, and jobs better with AI. Only you can become the expert in defining what “well”, “quality”, or “completion” means. Human taste and judgment matter. Intelligence can be defined in many ways (intellectual, social, artistic, etc), as can wealth (time, social connections, mental clarity, physical health, and finances). AI is a tool that can amplify human capabilities, giving individuals "superagency" – the ability to achieve more with greater intelligence, productivity, and reach.

Third, the upcoming AI age will be the biggest event in the history of homo sapiens as an alien species grows up. I expect it will be bigger than us transitioning to agriculture, forming cities, developing writing or the printing press, creating governments, machines, and industrial factories, developing electricity and electronics, advancing chemistry and biology into engineering and healthcare, or creating the internet and IT world we live. As one example: The printing press increased literacy rates from 30% to 60% in Renaissance Europe, leading to great technical and economic accomplishments (but also war), while some project that AI could automate 40% of cognitive tasks by 2030.

While many want to regulate AI or opine about what governments should do, I believe it will increase the agency and leverage of individuals by a massive amount - that we will have 10x to 100x professionals. We are early, as some recent studies suggest even today’s weak AI can assist coders to complete tasks 55% faster or that consultants using AI produce 40% higher-quality reports and 25% faster. My main goal is to figure out how to increase my own agency and help teach everyone else how to increase theirs. We are preparing for a new species to evolve and spread on our planet - one that is as smart and capable (or more so) than us. Our main goal should be to align it with our noblest and highest values and at the same time learn from it and partner with it to solve our greatest problems (eg poverty, disease, climate change, individual liberty, loneliness and lack of community, etc).

Here are three implementable strategies on how to “merge with AI”, that is, how to partner with it to increase your agency and enrich your life:

1) Use multiple, paid AI systems, and learn the ins and outs of each: Most people I know may use a single LLM like ChatGPT, and then rarely probe it deeply. What I suggest people do is use 3-5 systems and figure out what is the best for each task, chore, or job they’re trying to do. Pay the monthly fee to get the best system - it’s the cheapest subscription you will ever have. My main process is simple - for each task, I test at least 2-3 systems on variants of the task and learn which is the best one at the moment. I then use that one and may repeat the task in 2-3 months with another 3 systems. The ones I use the most and pay for are:

• OpenAI GPT-4o or o1/o3: For general answers to questions or discussions, either for fast responses with 4o or some deeper reasoning with o1/o3, or deep research

• Deepseek R1 on Perplexity: For searching for information on the web, or for a first cut on searching for products and services (Gemini often comes next)

• Gemini 2.0: To interact better with images, books, videos, maps, or audio - like summarizing a podcast or video or talking about it, or deep research

• Grok 2.0/3.0: For getting information on real-time news (which is often on X) or searching for variant opinions or creating divergent images.

• Midjourney: For general image creation

• ElevenLabs: For creating human speech and audio (and hopefully music soon)

• EmergentMind: For staying on top of cutting-edge AI research and papers, and asking specific questions to improve my knowledge on how AI works and is evolving

• Note that this list will change every 3 months as I experiment with new systems and swap out old ones that fall behind for newer ones.

2) Learn how far the AI can get, how to complement it, and be the quality bar enforcer: For most tasks and jobs, AI can’t do excellent end-to-end work yet, but can maybe get 40-90% of the way there. If you learn how to write detailed and very specific prompts based on what you want, the systems usually do much better. However, the catch today is if you’re an expert in the area, you still need to define and hold a quality bar for the AI systems, which act like junior colleagues or undergraduate student helpers. Current systems aren’t great at agentic planning and doing long sequence tasks, but I expect they will get vastly better every 6 months. You need to plan for that and grow with the improvements. Finally, note that AI can be used in every part of your life that requires intelligence: managing your time, creating a better social life with relationships and community, doing better at work or creating a business, improving your health, understanding the news, trends, and demographic changes around you, etc. Only your imagination and willingness to experiment are the limits.

3) Use AI to help identify and understand your ikigai. Ikigai (生き甲斐) is a Japanese concept that roughly translates to "a reason for being." It's often visualized as a Venn diagram with four overlapping circles:

• What you love: Your passions and interests.

• What you're good at: Your skills and talents.

• What the world needs: Problems you want to solve or contribute to.

• What you can be paid for: Work that provides financial stability.

The intersection of all four circles is your ikigai – your purpose in life, the thing that makes you want to get up in the morning. It's not just about finding a job you enjoy, but about finding something that brings you joy, utilizes your skills, contributes to the world, and provides for you. If you find your ikigai, you can focus your life more and push from high-agency into super-agency,

Here are some prompts to get you started:

• Life coach: I'm trying to find my ikigai, my reason for being. Could you act as a life coach and guide me through this process? I'd like to explore the four key areas: what I love, what I'm good at, what the world needs, and what I can be paid for. Let's start with what I love. What questions should I ask myself to identify my passions and interests?

• Feeling lost: I'm feeling a bit lost and unfulfilled. I've heard about ikigai, the Japanese concept of finding your purpose, and I'd like to explore it. Can you act as a life coach and help me on this journey? I'm not sure where to start. Maybe you could ask me some questions to get me thinking?

• Career focus: I'm interested in finding my ikigai. I'm a [Your Profession/Situation - e.g., software engineer, recent graduate, stay-at-home parent]. I enjoy [Mention a few general interests - e.g., working with my hands, creative writing, helping others]. Can you act as a life coach and help me explore how these interests might connect with my skills, what the world needs, and potential career paths?

• Values focus: I'm trying to discover my ikigai. I believe my values are important to this. Can you act as a life coach and help me identify my core values? What questions should I ask myself to understand what truly matters to me? How can these values guide me toward finding my purpose?

My final call to action is to take your personal use of AI seriously: Start tomorrow with one AI-augmented hour - use GPT 4o/o3 or Claude 3 to draft emails, Perplexity for research, and R1 for meeting prep. Measure your 30-day productivity gain against a baseline. Embrace AI and try to use it for everything you do that requires intelligence. My main tips:

1) Use multiple, paid AI systems, and learn the ins and outs of each

2) Learn how far the AI can get, how to complement it, and be the quality bar enforcer

3) Use AI to help identify and understand your ikigai.

Evolution favors the augmented.

This was a weak year of reading books for me. My reading was random, jumping from many memoirs and biographies to mathematical treatises, histories of art, education theory, Japanese history and culture, and more. I also wrote about how I decided what to read, in the Rao Reading Algorithm.

Much of my reading was of AI papers. 2024 was a regrouping year for artificial intelligence research and convergence of capabilities between many labs (eg. OpenAI, Google DeepMind, Anthropic, Meta Llama, X.AI, Amazon Nova, NVIDIA Nemotron, Alibaba Qwen). The earliest reasoning models, robotic foundation models, and agentic models are coming out, and these will change the world, though over a 5 to 10 year run.

These systems are a new alien species that will forever change human civilization. They are slowly progressing through rough versions of “laptop worker” human intelligence toward a hazy outline of superintelligence. I’ve been researching ways to make them benevolent and aligned with human flourishing, but it’s an open technical and humanistic problem if our mind children will share our values as a species. At the very least, the current tadpole AIs are fantastic at helping us read and do research better, albeit with complex problems around factuality, hallucination, and reasoning.

I had a three-way tie for the best books this year: Alexander’s “A Pattern Language” about creating harmonious environments; Darwin’s “Autobiography” about his life and influences; and Bennett’s “A Brief History of Intelligence”, which is a modern classic on par with these two. Below I continue my short section about my favorite technical papers, and audio and video, broadening from books to culture (hat tip to Piero Scaruffi, who makes great year end lists).

To motivate myself and create a trace for future people and AIs, I listed most of the books I read this year, versus just a subset (hat tip to Timothy Buck and Tyler Cowen). Overall I didn’t read much this year – it was a disappointing year. “Da mihi castitatem et continentiam a Tiktok et X, sed noli modo.“

BEST BOOKS OF 2024

Bennett, A Brief History of Intelligence: Evolution, AI, and the Five Breakthroughs That Made Our Brains. An exploration of the evolutionary milestones that shaped human intelligence, brilliantly explained, going from LUCA to bacteria and bilaterians, tetrapods to amniotes and therapsids, then cynodonts to mammals and hominids. It had insights into how an understanding of earthly intelligence can inform the development of modern attempts at AGI and then SI. It covers major breakthroughs in the evolution of intelligence, such as: steering, emoting, associating, and predicting; reinforcing, TD learning, and world models; simulations, generative models, and model-based RL; mentalizing and self/other modeling; language and large group coordination. I liked this book so much that I bought both a print and digital copy and re-read some chapters and sections multiple times. It itched at and etched on my brain and is a much discussed book in the Bay Area AI circles today.

Alexander, Ishikawa, Silverstein, et al., A Pattern Language: Towns, Buildings, Construction. A seminal guide offering 253 design patterns to create balanced and livable spaces, emphasizing that thoughtful design can enhance human flourishing. This was a favorite text from my undergraduate years that I dug up to re-read while doing a painful ADU construction project this year.

Darwin, The Autobiography of Charles Darwin: 1809-1882. A memoir revealing the life and intellectual journey of the naturalist. It shows the importance of curiosity, luck, and perseverance in scientific discovery. It is a glimpse into the mind of the most eminent Victorian I know (other than William Rowan Hamilton and the Faraday/Maxwell pair).

Pattern 106 (“Positive Outdoor Space”) – from Alexander et al, “A Pattern Language” (“Outdoor spaces which are merely ‘left over’ between buildings will, in general, not be used.”)

NON-FICTION

Vickers, By All Means Available: Memoirs in a Life of Intelligence, Special Operations, and Strategy. A memoir detailing a career in intelligence and special operations, highlighting the complexities and ethical considerations of strategic decision-making.

Johnson, Strange Beauty: Murray Gell-Mann and the Revolution in Physics. A biography of Murray Gell-Mann, tracing his groundbreaking work in physics and his influence on the understanding of fundamental particles.

Rhodes, Scientist: E.O. Wilson. A biography of E.O. Wilson, chronicling his life as a pioneer in entomology, sociobiology, and biodiversity conservation.

Zaleski and Zaleski, The Fellowship: The Literary Lives of the Inklings. An exploration of the lives and works of C.S. Lewis, J.R.R. Tolkien, and their literary circle, emphasizing the power of friendship and imagination.

Watts, The Magic Kingdom: Walt Disney and the American Way of Life. A spectacular biography of Walt Disney, examining his singular creativity and imagination, plus his influence on American culture and the entertainment industry.

Junger, In My Time of Dying. A dying journalist’s exploration of mortality and the human experience of death and his near death experience.

Neufeld, Von Braun: Dreamer of Space, Engineer of War. A biography of Wernher von Braun, exploring his contributions to rocket science and his complicated legacy.

Du Sautoy, Symmetry: A Journey into the Patterns of Nature. An exploration of the concept of symmetry in mathematics and nature, revealing its fundamental role in the universe.

Heller, Ayn Rand and the World She Made. A biography of Ayn Rand, examining her life, philosophy, and enduring influence on American thought. Learning about her high agency in escaping Communist Russia by walking from Moscow to Hamburg to find a way out was stricking.

Viroli, Niccolo’s Smile: A Biography of Machiavelli. An empathic and thoughtful biography of Machiavelli’s and his own works, as he remains the bad boy of political science even today.

Hamilton, American Caesars: Lives of the Presidents from FDR to George W. Bush. An attempt to replicate Suetonius for the modern empire-stage US presidency, told with wit and relish, but often with an naive, boy scout approach to politics and morality instead of the realpolitik the presidency deserves.

McCullough, The Wright Brothers. A spare and lovely account of the Wright brothers’ invention of the airplane, highlighting their ingenuity and determination.

Grant, Hidden Potential: The Science of Achieving Greater Things. An examination of how to unlock untapped potential, combining research and stories.

Rajan and Lamba, Breaking the Mould: Reimagining India’s Economic Future. An economic analysis of India’s challenges and some innovative policy solutions, offering insights into revitalizing the nation’s economic trajectory.

Wali, Chandra: A Biography of S. Chandrasekhar. A biography of Nobel laureate Subrahmanyan Chandrasekhar, detailing his contributions to astrophysics (Chandrasekhar, white dwarf stars, black holes) and his scientific journey.

Gleick, Genius: The Life and Science of Richard Feynman. A portrait of physicist Richard Feynman, capturing his brilliance, curiosity, and unconventional approach to science. The QED sections were just meh – you will have to read his book to learn more.

Reid, The Chip: How Two Americans Invented the Microchip and Launched a Revolution. The story of the invention of the microchip, highlighting the corporate competition that sparked the digital age.

Nevala-Lee, Astounding: John W. Campbell, Isaac Asimov, Robert A. Heinlein, L. Ron Hubbard, and the Golden Age of Science Fiction. An exploration of the Golden Age of science fiction, focusing on key figures who shaped the genre.

Inchauspé, Glucose Revolution: The Life-Changing Power of Balancing Your Blood Sugar. A guide to managing blood sugar levels for improved health and well-being, based on scientific research.

Hoban, Lucian Freud: Eyes Wide Open. A deep dive into the life and work of Lucian Freud, capturing his impact on portrait art through an examination of his acidic personality and amorality.

Schütte, Kraftwerk: Future Music from Germany. An exploration of Kraftwerk’s groundbreaking influence on industrial, electronic music, detailing their innovation and legacy.

Rubin, The Creative Act: A Way of Being. A reflection on the nature of creativity and how it can be harnessed in every aspect of life, penned by legendary music producer Rick Rubin.

Christensen, How Will You Measure Your Life? A candid guide applying strategy theories to one’s personal life, encouraging readers to define their success through values, family, and purpose and not the sugar rush of career and status.

Friedman, Capitalism and Freedom. A short treatise on applying libertarian principles to promote competitive, rules-based capitalism to restructure states to maximize human freedom.

Lillard, Montessori: The Science Behind the Genius. An analysis of Montessori education, presenting scientific research that supports its methods and effectiveness.

Stoner, Natural Education. A century-old philosophical treatise on how education should align with natural human development for holistic growth (I’m going to re-read Quintilian after this).

Minsky, Inventive Minds: Marvin Minsky on Education. A collection of writings from Marvin Minsky, offering his quirky insights into education and the future of learning.

Wise Bauer and Wise, The Well-Trained Mind: A Guide to Classical Education at Home (Fourth Edition). A comprehensive manual for providing a rigorous, classical education at home, guiding parents through each stage of their child’s learning journey.

Bloom, Developing Talent in Young People. A groundbreaking study revealing how exceptional talent is cultivated across various fields, emphasizing the roles of environment and deliberate practice (the neuroscience, sculpture, and math chapters were the best).

Cox and Terman, Genetic Studies of Genius, Vol 2. An in-depth analysis of the hereditary factors influencing high intelligence, contributing to the understanding of genius through longitudinal case studies.

Wollenberg, Berkeley: A City in History. A historical account of Berkeley, California, highlighting its evolution as a hub of culture, education, warfare, and activism.

Sloterdijk, You Must Change Your Life. A philosophical exploration of self-improvement, focusing on existential challenges and transformative practices, tied to some central European theorizing. I tried pairing this with Gebser’s “The Ever-Present Origin” but failed (Gebser was too tough for me).

Catmull, Creativity, Inc: Overcoming the Unseen Forces That Stand in the Way of True Inspiration. An insider’s guide to fostering creativity within organizations, drawing from the experiences of Pixar’s co-founder; re-reading this after a few years and still finding new insights.

Gaukroger, Descartes: An Intellectual Life. A biography of René Descartes, detailing his groundbreaking contributions to math, philosophy, and science.

Taylor, How Star Wars Conquered the Universe. A detailed account of the cultural phenomenon of Star Wars, exploring its creation and global impact.

Mason, A History of Japan (Revised Edition). A comprehensive overview of Japan’s historical evolution, offering insights into its cultural and political transformations. My favorite parts were about the Azuchi-Momoyama triumvirate (Nobunaga, Hideyoshi, and Ieyasu), but also about the Nara Court and the Meiji Revolution.

Weston, Giants of Japan: The Lives of Japan’s Greatest Men and Women. Profiles of influential Japanese figures, highlighting their contributions and the cultural contexts that shaped their achievements. Like Plutarch’s Lives for Japan.

Mogi, The Little Book of Ikigai: The Essential Japanese Way to Finding Your Purpose in Life. An exploration of ‘ikigai,’ the Japanese concept of finding purpose or minor delights. This paired welll with Miralles and García, “Ikigai: The Japanese Secret to a Long and Happy Life.”

Mogi, The Way of Nagomi: The Japanese Philosophy of Finding Balance and Peace in Everything You Do. A guide to achieving harmony and balance through the Japanese philosophy of ‘nagomi,’ applicable to various aspects of life.

Jodidio, Contemporary Japanese Architecture. A showcase of modern Japanese architectural designs, reflecting the country’s minimalist aesthetic sensibilities.

Pliny the Elder, Natural History. An ancient encyclopedia encompassing knowledge of the natural world, offering insights into Roman understanding of science and nature.

Mollick, Co-Intelligence: Harnessing the Power of Collective Thinking. An examination of how collaborative intelligence can drive innovation and problem-solving in organizations from the most thoughtful professors experimenting with AI for knowledge workers.

Carlile, Wild Life: 50 Projects to Rewild Your Life. A practical guide offering projects to reconnect with nature and promote biodiversity in daily life.

Rand, Philosophy: Who Needs It. A collection of essays arguing for the practical importance of philosophy in everyday life, with the main essay delivered at West Point as a striking and novel assertion of Rand’s views (beyond the novels).

Williams, Ethics and the Limits of Philosophy. A donnish analysis of moral philosophy, questioning the objectivity and applicability of ethical theories.

Ptahhotep, The Teaching of Ptahhotep. An ancient Egyptian text filled with moral and practical wisdom, emphasizing justice, humility, and effective leadership. Transliteration after ~Devaud 1916~, using the copy on Papyrus Prisse as principal source.

MATH, SCIENCE, & TECHNICAL

Yandell, The Honors Class: Hilbert’s Problems and Their Solvers. An engaging exploration of the 23 mathematical challenges posed by David Hilbert in 1900 and the mathematicians who endeavored to solve them, highlighting the evolution of modern mathematics.

McKenzie, Condensed Matter Physics: A Very Short Introduction. An introduction to the principles and phenomena of condensed matter physics, emphasizing the importance of understanding material properties in technological advancements.

Costandi, Neuroplasticity. An overview of the brain’s remarkable ability to reorganize itself, highlighting the potential for recovery and adaptation throughout life.

Baez, The Octonions. An examination of the octonions, an extension of complex numbers and quartenions, revealing their significance in higher-dimensional algebra and theoretical physics.

Gell-Mann, The Quark and the Jaguar: Adventures in the Simple and the Complex. A Nobel laureate’s exploration of the interplay between simplicity and complexity in nature, illustrating how fundamental particles and complex systems are interconnected.

Nash, Open Problems in Mathematics. A curated collection of unsolved problems across various mathematical disciplines, encouraging ongoing inquiry and discovery. Nash, Smale, and Langlands have competing agenda after Hilbert’s is put to rest.

Blundell, Superconductivity: A Very Short Introduction. An accessible guide to the phenomenon of superconductivity, explaining its principles and potential applications in technology.

Euler, Introduction to Analysis of the Infinite, Volumes I-II. Foundational texts presenting Euler’s approach to mathematical analysis, laying the groundwork for modern calculus.

Batterson, Stephen Smale: The Mathematician Who Broke the Dimension Barrier.
A biography of Stephen Smale, highlighting his contributions to topology and dynamical systems, and his impact on modern mathematics. Also see his “Mathematical Problems for the Next Century” [1998].

Earl, Topology. An introductory text on topology, emphasizing its fundamental concepts and applications in various fields.

Saveliev, Topology Illustrated. A visual guide to topology, using illustrations to explain complex concepts and their significance.

Albers et al., Mathematical People and More Mathematical People. Collections of interviews with prominent mathematicians, offering insights into their lives, work, and methods.

Maxwell, Treatise on Electricity and Magnetism. A seminal work laying the foundations of electromagnetic theory, unifying electricity, magnetism, and light; most physicists of his time thought this text was fantastical, and it took Hertz, Heaviside, and others for these strange invisible field theories to gain acceptance.

Mahon, The Man Who Changed Everything: The Life of James Clerk Maxwell.
A biography of James Clerk Maxwell, detailing his scientific achievements and their profound impact on physics.

Cropper, Great Physicists: The Life and Times of Leading Physicists from Galileo to Hawking. Profiles of influential physicists, explaining and exploring their contributions and the historical contexts of their work. This complements their actual papers and texts well, along with more modern treatments.

FICTION, POETRY, AND MEMOIR

Vendler, The Art of Shakespeare’s Sonnets. A comprehensive analysis of Shakespeare’s 154 sonnets, revealing intricate imaginative and stylistic features that enhance understanding of their emotional and dynamic effects. A book I can read and re-read many times.

Clarke, Rendezvous with Rama. A science fiction novel about a mysterious alien spacecraft entering the solar system, exploring themes of human curiosity and the unknown.

Weir, The Martian. A survival story of an astronaut stranded on Mars, highlighting human ingenuity and resilience. I couldn’t put it down and read it one reading (the book is just as fun as the movie).

Miyazaki, Starting Point: 1979-1996. A collection of essays, interviews, and memoirs by the renowned animator, offering insights into his creative process and philosophy.

Watson, The Double Helix: A Personal Account of the Discovery of the Structure of DNA. A candid memoir detailing the competitive and collaborative efforts leading to the groundbreaking discovery of DNA’s double-helix structure, underscoring the human aspects of scientific breakthroughs.

Olsson, The Weil Conjectures: On Math and the Pursuit of the Unknown. A blend of memoir and biography exploring the lives of the Weil siblings—philosopher Simone and mathematician André—and their intellectual pursuits.

Koren, Wabi-Sabi: For Artists, Designers, Poets & Philosophers. An exploration of the Japanese aesthetic of wabi-sabi, emphasizing beauty in imperfection and transience.

Garrison, Horace: Odes and Epodes. A translation and commentary on the lyrical poetry of Horace, offering insights into Roman literature and culture.

Browne, Translation of the Lilies Back into Lists. A poetic work exploring themes of language and perception.

Joyce, Finnegans Wake. A demented, experimental novel pushing the boundaries of representation, meaning, and language. No one actually reads it end to end – I chose passages to read to my toddler daughter to create some fun diversity and interference in her normal toddler fare.

Hsu, Stay True: A Memoir. A memoir reflecting on friendship, identity, and loss, situated in Berkeley.

Shatner, Star Trek Memories. A captain’s account of the making of the original Star Trek series, told with relish, pathos, and a hint of bitterness.

Steinhardt, The Second Kind of Impossible: The Extraordinary Quest for a New Form of Matter. The story of the discovery of quasicrystals, a new form of matter that defies traditional scientific understanding.

Johnson, Kelly: More Than My Share of It All. An autobiography by Clarence “Kelly” Johnson, legendary aviation engineer, detailing his pioneering work at Lockheed’s Skunk Works (like the SR-72) and his principles on running a frontier lab.

Allen, Apropos of Nothing. Memoirs of Woody Allen, offering a candid and reflective view of his career and personal life, including a defense of his family life.

PAPERS

I read 150+ AI papers this year, both professionally for work and for the sheer fun of it. It was the annus mirabilis for AI research and new models being released, and the only thing that scares me is whether is the first year of an exponential takeoff where every year is meaningfully better than the last or if the pace slows down and the magnificent future of the abundance of intelligence and material goods is just a mirage. As many AI researchers believe, if we can solve the problem of intelligence and manufacture cheap and widely available intelligence, we can then solve all of humanity’s problems in short order. I don’t have the time to leave a full trace of papers or summarize what I found notable about all the papers below, but these were the best of the batch that I read, and I present them in no particular order.

LLMs, Benchmarks, Etc.

McCarthy, “A Proposal for the Dartmouth Summer Research Project on Artificial Intelligence, August 31, 1955.” “Notes on formalizing context.” “Epistemological problems of artificial intelligence.” “Generality in artificial intelligence.” “Some expert systems need common sense.” “Varieties of contexts.” “What has AI in Common with Philosophy?” “Useful counterfactuals.” “Free will-even for robots.” “From here to human-level AI.” [1960s to 2000s]

Meta AI, Llama 3 Herd of Models [2024].

Benaich and Chalmers, State of AI Report 2024 [2024].

Nvidia, Nemotron-4 340B Technical Report [2024].

Valevski et al, Diffusion Models Are Real-Time Game Engines [2024].

Kumar et al, Training Language Models to Self-Correct via Reinforcement Learning [2024].

Chamelon Team, Chameleon: Mixed-Modal Early-Fusion Foundation Models [2024].

Ma et al, The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits [2024].

Pagnoni et al, Byte Latent Transformer: Patches Scale Better Than Tokens [2024].

Zhao et al, Measure Dataset Diversity, Don’t Just Claim It [2024]

Cahyavijaya, High Dimensional Value Representation in LLMs [2024].

Bang et al, Measuring Political Bias in LLMs [2024].

Feuer et al, Style Outweighs Substance: Failure Modes of LLM Judges in Alignment Benchmarking [2024]

Agents, Reasoning, Alignment and Safety, and Tools
Lambert, Interconnects Posts [2024].

Dalrylmple et al, Towards Guaranteed Safe AI: A Framework for Ensuring Robust and Reliable AI Systems [2024].

Wu et al, Thinking LLMs: General Instruction Following with Thought Generation [2024].

Golovneva, ROSCOE: A Suite of Metrics for Scoring Step-by-Step Reasoning [2022].

Yao et al, τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains [2024]

Wang et al, What Are Tools Anyway? A Survey from the Language Model Perspective [2024]

Chan et al, Visibility into AI Agents [2024].

OpenAI, Model Spec [2024]

Durante et al, Agent AI: Surveying the Horizons of Multimodal Interaction [2024].

Huh et al, The Platonic Representation Hypothesis [2024]

Lu et al, The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery [2024]

Brown et al, Large Language Monkeys: Scaling Inference Compute with Repeated Sampling [2024]

Si et al, Can LLMs Generate Novel Research Ideas? A Large-Scale Human Study with 100+ NLP Researchers [2024].

Rivera et al, Escalation Risks from Language Models in Military and Diplomatic Decision-Making [2024].

Li et al, More Agents Is All You Need [2024].

Mondorf and Plank, Beyond Accuracy: Evaluating the Reasoning Behavior of Large Language Models — A Survey [2024]

Asai et al, Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection [2024].

Brynjolfsson et al, Generative AI at Work [2024] (also see “AI and the Future of Work“)

Other

Dyson, Missed Opportunities [1972].

Feynman, Negative Probability [1984].

Smale, “Mathematical Problems for the Next Century” [1998]

Penrose, Before the Big Bang [2006].

Grandia et al, Design and Control of a Bipedal Robotic Character [2024]

Clough et al, What no one has seen before: gravitational waveforms from warp drive collapse [2024]

AUDIO AND VIDEO

I had some time for the video series, albums, podcasts, and playlists below (mixed among lots of junk Netflix and varied YouTube videos that I sampled). My tastes lean toward math and physics videos, intellectual dramas and discussions, and contemporary classical music – so if that’s your cup of tea, enjoy what’s below.

Theory of Everything: The Quest to Explain All Reality (The Great Courses).

The Holdovers.

Oppenheimer.

The Conclave.

“Tokyo Vice” (Max)

1923 Season One (Paramount Plus).

Feynman Podcasts (“The Curious, Brilliant, Vanishing Mr. Feynman”, series of 3).

Walt Disney American Experience (PBS documentary).

Turn Every Page: The Adventures of Robert Caro and Robert Gottlieb

Phillip Glass: The Complete Piano Etudes; Complete String Quartets

Tabakova, Kynance Cove.

John Luther Adams: Become Ocean.

Johannsson, Mandy; 12 Conversations with Thilo Heinzmann.

Mazzoli, Vespers for a New Dark Age.

David Senra Founders Podcast.

Conversations with Tyler, Podcast.

GRAPHS, DIAGRAMS, TABLES, AND CHARTS

Sheehan, The Elements According to Relative Abundance [1976]

Sargent, “Anger Transference” [1954] (a cover illustration for The Saturday Evening Post, published on March 20, 1954)

DaVinci, Notebooks (suggesting the human body is a scale model of Earth, or similar physics effects the operations of both) [15th-16th century]

Movies with the largest disparities in men vs women. Hickey, “What If Online Movie Ratings Weren’t Based Almost Entirely On What Men Think?”, Five-Thirty Eight (2018).

IN SUMMA

PAST LISTS

Best Books of 2023 – Philosophers and an Intelligence Takeoff

Best Books of 2022 (Lives and Machines)

Best Books of 2021 (Berkelians)

Best Books of 2020 (Reality)

Who, What, Where, How, and Why Do I Read - Why Reading Matters

Reading means my total consumption of ideas and media, learning via seeing or listening to symbols versus pure action. Reading involves books at the core, but also journal articles, news, blogs, music, video, maps, engineering and architectural drawings, code, patents, walking in cities, conversations with people, and viewing art. If it’s compressed info encoded into my brain and world models, I count it. The line between a reading algo and a learning algo is unclear. Reading is compressed learning, but obviously there are other ways to learn (direct trial and error, apprentice mentor teaching, Socratic dialectic, etc).

More subtly, reading is practice directing attention, causing your awareness to curate what comes into that awareness, for a purpose.

My reading goal is to use what I read, integrate it into a multidisciplinary whole in my world models, and connect it to problems I care about and am working on. It means updating my web of knowledge.

But mere usefulness is too strict a criterion. An equal goal is pleasure. I’ve rewired my brain so reading to learn is highly pleasurable. Often I learn fun and useless material that my curiosity takes me to. This material may be useful later (linear algebra, graph theory, topology, thermodynamics, contemporary classical music, minimalist art). Steve Jobs learning about calligraphy and fonts, and then later created the first great GUI and desktop publishing suite. He is an inspiration. Trivia can be world-changing. Curiosity is a blazing, angelic guide.

Generally I’m trying to become a learning machine - to get the best of compressed learning from many humans so I'm not limited to my own experience (like interactive fleet learning for robots and AI, who are early on this steep learning curve).

I'm guided by the wisdom of Charlie Munger:

In my whole life, I have known no wise people (over a broad subject matter area) who didn’t read all the time – none, zero. . . You’d be amazed at how much Warren reads – at how much I read. My children laugh at me. They think I’m a book with a couple of legs sticking out. Poor Charlie's Almanack (1st edition, 2005; 2nd edition, 2008).

Reading is highly personal. I don’t write about my reading algorithm to recommend it to you or for didactic purposes. Rather, this is descriptive, if my goal for sharing it is to get you to think deeper about your own reading, epistemology, and weltanshauung.

Core of the Algorithm

I’m looking to build semantic trees of knowledge. A semantic tree is an ordering of information that starts with fundamental principles and branches out into increasingly less fundamental details. For me, it's the most important compressed knowledge that I can act and build on: math, physics, and the core of humanities.

READ THE FULL ARTICLE HERE

Here are some paid publications I subscribed to for many years but have canceled in the last 2 years or the last few months:

• The New York Times (20+ year subscriber)

• The New Yorker (20+ year subscriber)

• The Financial Times (on and off, a 20+ year subscriber)

• Bloomberg News (5+ year subscriber)

• LA Times (5+ year subscriber)

• The Atlantic (on and off subscriber)

• Fortune Magazine (10+ year subscriber)

• MIT Tech Review (2+ year subscriber)

• Wired Magazine (4+ year subscriber)

• Der Spiegel (English) (3+ year subscriber)

• Guardian (5+ year subscriber)

I’m disappointed to have done this. The fundamental reason is that the quality of news has gone down. Surprisingly the high-cost subscription news products have gotten comparatively worse, vs some free ad-driven products.

Here are the handful of paid subscriptions I’ve kept:

• The Economist Magazine

• The Wall Street Journal

• Dwell Magazine

• Apple News Plus (which gives me partial access to many publications)

I’ve talked to many other well-informed and highly-educated Americans, and this seems to be part of a 5-year or even decade-long trend.

So what are the reasons I’ve canceled so many subscriptions when I would much rather keep them and pay for high-quality journalism?

Top reasons why I quit most mainstream media:

• The quality of news has decreased: standards for basic factual reporting (“the whole truth and nothing but the truth”) have declined compared to the prime 1980s and 1990s reporting. An older Walter Cronkite generation had a standard of accuracy, timeliness, and fairness that many modern publications have rejected, reverting back to a new age of Yellow Journalism that was common a century ago. James Bennet’s critique of the New York Times failing as an institution applies to many of the older publications - they are imploding.

• The rise of opinion journalism replacing fact-based reporting: Publications keep replacing objective journalistic reporting with editorials, commentary, and analysis from generalist pundits who traffic in politicized pablum (the New York Times and FT were recent offenders). I much prefer journalists who do investigative reporting, objectively present many sides, and link to hard evidence (e.g. reports, images, video, etc).

• Endless partisanship: Most of the paid subscriptions have decided via their publishers, senior editors, or hired journalists that they will be politically partisan and present subjectively slanted reporting. I don’t want to read Democrat, Republican, Tory, Labour, CDU, etc news. I want politically neutral, objective reporting, which was a standard in some high-quality publications from the 1970s to the 1990s. That has been lost. I mostly look for intelligent news, and prefer views from the center, or the rare smart views on the left and right - media political bias has hit extremes (see the chart below). There is some evidence that paid news and paywalls increase polarization and hyper-partisanship, specifically “evidence that the newsroom got more polarized after the paywall adoption: journalists who wrote more left-leaning news articles are more likely to get new byline assignments.”

• The lack of a global perspective, or hyper-local news: Most of these high-quality publications focus on national news, often sensational hangups, when I see value in having a global perspective (for large publications like the NYT, FT, etc), a hyper-local perspective (for say the LA Times or SF Chronicle), or reporting on important long-run trends such as demographic change, people and capital movements, institutional updates or decay, cultural diversity, etc.

• The lack of evidence: We live in a world where posting a report, transcript, image, video, court document, or so on should be a standard journalistic practice - if you cite something, link to it. Unfortunately, very few journalists do so.

• Clickbaity headlines and articles mixed with conflicts of interest: The quality of headlines has gone down as many editors have regressed to clickbait models. Often even the first few paragraphs of many articles read like clickbait to promote wild emotional reactions instead of a thoughtful hook or summary of the piece. Editorial coverage is suspect too. As one example, I think editors could cut the reporting on Elon, Trump, Israel/Palestine, and other sensational areas by half and free up that attention to important, long-run trends. Finally, I see journalists at top publications writing articles slamming politicians or companies in sensational weekly articles while they are on a book tour promoting their book on said politician or company - a clear conflict of interest in reporting that somehow is allowed. Other conflicts come from promoting a journalist’s personal brand and political views on social media, while hurting the credibility of their publication.

So where have I gone - what has replaced the mainstream media in my information diet? The trends I’ve noted are a movement to expert-driven reporting and commentary and paid newsletters.

What replaced the paid mainstream media in my information diet:

• Reuters, AP News, Politico, and The Hill - free, ad-supported mainstream media that still has fair quality levels

• Stratechery (Ben Thompson) - a tech newsletter, which along with Techmeme and the Verge has the best tech reporting

• Techmeme - a tech news aggregator

• Verge - a niche tech news site

• Hacker News - a niche aggregator site for coders

• Politico and the Hill - websites with relatively unbiased reporting on American politics and government

• Money Stuff by Matt Levine (Bloomberg) - a financial industry newsletter

• Parent Data by Emily Oster - a parenting newsletter

• Lawfare and Scotusblog - niche websites on legal matters

• Quanta Magazine - a niche math and science publication

• Podcasts: Founders, American Optimist, Freakonomics, No Priors, Dwarkesh Podcast, The Lex Fridman Podcast, Conversations with Tyler, The Ezra Klein Show, People I Mostly Admire, Stanford’s ETL Entrepreneurial Thought Leaders, Invest Like the Best, Big Technology Podcast, Goodfellows.

• The Great Courses: Online video courses in math, physics, history, etc.

• YouTube: Lectures from professors in areas I care about (robotics, AI, climate change, math, etc); Royal Institution science weekly Friday talks; interviews with key political, business, and cultural leaders; and clips from specialist reporters like Fareed Zakaria on geopolitics.

• Twitter/X and Threads: As a place to follow specialists (e.g. ML Twitter, Tech Twitter, books Twitter/Threads), and then link out to specialist websites or informed commentary that avoids the Dunning Kruger effect.

My consumption pattern suggests that a few, high-quality generalist publications that maintain high standards will do well (e.g. WSJ and the Economist), while the future is all about reporting by specialist journalists or experts who become journalists and who start their own newsletters and podcasts. They will monetize their high-value content without being held back by a legacy institution.

There remains a small minority of great journalists in mainstream media. Some that come to mind: Cade Metz, Thomas Friedman, Steven Erlanger, Keith Bradsher, Andrew Ross Sorkin, George Johnson, Roger Cohen, and Katrin Bennhold at the NY Times; Evan Osnos and Jiayang Fan at the New Yorker; Derek Thompson at the Atlantic; John Burn-Murdoch, Jamie Smyth, Edward White, Leila Abboud, Roula Kalaf at the FT; and so on.

I hope that these journalists start to push for a return to better institutional values in their publications (high-quality news, little or no editorial and opinion reporting, political neutrality, global or hyper-local views, plenty of attached evidence, neutral and not clickbaity language). If not, I hope they can leave to start a new institution, just like Barri Weiss did for The Free Press or Ben Smith did for Semafor.

And ultimately I hope to pay premium prices for high-quality news and reporting - that this recent trend reverses through the integrity and hard work of a new generation of editors and journalists. I especially hope to return to the NY Times and Financial Times one day in the future when they have new publishers and editors and have higher quality institutional values and reporting that hearkens back to or improves on their prior peak.

For the top books, I had a three-way tie for the best books ("Ageless" about biogerontology, "Three Laws of Nature" about thermodynamics, and the avant-garde novel "The Last Samurai"). I'm also adding a short section about audio and video, the works richer in bits, that struck a chord.

BEST BOOKS OF 2023

Steele, Ageless: The New Science of Getting Older Without Getting Old. This is the best layman's book on the new field of biogerontology that took off in the 1990s and especially in the last decade, which focuses on how to slow down, stop, or reverse aging.  If that sounds fantastic, it's because few understand the science and the author makes a case that this is one of the most important fields of scientific study today, given how much disease and death are tied to underlying aging factors.  Steele breaks down aging into 10 hallmark factors (DNA damage and mutations, trimmed telomeres, protein problems like autophagy, epigenetic alterations, the accumulation of senescent cells, malfunctioning mitochondria, signal failures like inflammation, changes in the microbiome, cellular exhaustion, malfunctions of the immune system).  He then goes into research on promising animal and human studies on each of these factors and the health and lifespan gains researchers have seen (from 10% to 600% in some species).  Alongside AI and clean energy (eg solar panels and nuclear fusion), biogerontology promises to be one of the most exciting and interesting fields in the next few decades.

Berry, Three Laws of Nature: A Little Book on Thermodynamics.   Einstein said of Thermodynamics: “It is the only physical theory of universal content, which I am convinced, that within the framework of applicability of its basic concepts will never be overthrown.” I never felt I understood the 3 laws in physics classes. Despite seeming simple, much didn't land. So this book by Stephen Berry (UChicago Chemist, known as a "Renaissance scientist" and "one of the most influential chemists of his generation") finally lit my fire and it walks through the history and development of the 3 laws. It turns out it took many of the brightest scientists nearly 150 years to work them out with careful experiments (never taught in today's schools), and it started with mining entrepreneurs trying to dig more coal in Northern England and that industry and practice was decades ahead of theory (similar to modern machine learning). The 3 laws, along with electromagnetism, are the basis of most technology supporting humans today, from electricity to mechanical gadgets, cars, planes, rockets, and computers - we would be savages without them. What's also strange and intriguing is modern physicists explain and interpret the 3 laws very differently, as I learned by finding and reading the explanations from Fermi, Feynman, Penrose, Weinberg, Suo, and Wolfram. For even more detail, check out Hanlon's Block by Block.

DeWitt, The Last Samurai. A short novel about a brilliant mother and her precocious son, who is searching for his father. I consider this one of the great novels of the 21st century and was delighted to learn that some others do too. Beyond compelling characters and a touching plot, DeWitt pushes forward what the form of the novel could be, in a rare way similar to Joyce, James, and Hemingway.

This year's full list is here (with all the books, papers, and more): https://raohacker.com/best-books-of-2023-philosophers-and-an-intelligence-takeoff/

How should a non-technical person or a machine learning engineer get up to speed on the fast-moving space of Generative AI? Below are the resources that I used and would recommend, mostly from my self-study, though I work with multiple teams of engineers at Meta building GenAI models at scale, with a focus on the LLMs that often act as the foundation and connective tissue.

I offer five stages to jump in:

1. Do some background reading on prior machine learning (deep learning, RL, etc);

2. Read some non-technical pieces on GenAI and play with current tools;

3. Learn the GenAI technical basics via courses, books, and papers;

4. Start reading papers in the GenAI field and take more advanced courses;

5. Build your own GenAI models from scratch.

I expect that over time the generative AI field will overtake traditional AI (for ranking, prediction, classification, etc) as generative use cases track a much wider area of human activity.

1) Background Reading for Machine Learning, Before GenAI

Below I’ve put together a collection of resources on older (pre-2023) machine learning and AI books, papers, videos, and podcasts. I recommend people start understanding the foundational material in this section before jumping into GenAI.

• Primer on AI and Machine Learning (Part 1) — Beginners Level (Non-Technical): Artificial Intelligence (AI) will be one of the greatest developments of science and civilization, as machines approach, augment, and exceed human performance on a wide range of cognitive tasks over the next few decades.

  This lays out how a smart, college-educated person with a fascination with AI but no technical background can get up to speed on it.

• Machine Learning (including Deep Learning and Reinforcement Learning) for Engineers — A Technical Primer (Part 2): This lays out how a software developer, hacker, or coding cross-functional partner (Engineering manager, product manager, project manager, etc) who is fascinated by AI but with no ML-specific background build a base.

2) Non-Technical Writing on GenAI, plus Current Tools

The technology of Gen AI

• A Generative AI Primer, from Its Origins (2023): Understanding the current state of technology requires understanding its origins. This reading list provides sources relevant to the form of generative AI that led to natural language processing (NLP) models such as ChatGPT.

• Google: A generative AI primer for busy executives (2023): A very high-level and simplified explanation of the field and why it matters. A place to start if you know nothing about this space.

• McKinsey: What’s the future of generative AI? An early view in 15 charts (2023): A quick take from many reports, with links to them.

• A guide to Generative AI terminology: A short list of some key terms in the field. Also see this decent Adobe GenAI Glossary.

• On the opportunities and risks of foundation models (2021): Stanford overview paper on Foundation Models. Long and opinionated, but this shaped the term.

• State of AI Report: An annual roundup of everything going on in AI, including technology breakthroughs, industry development, politics/regulation, economic implications, safety, and predictions for the future.

• GPTs are GPTs: An early look at the labor market impact potential of large language models: This paper from researchers at OpenAI, OpenResearch, and the UPenn predicts that “around 80% of the U.S. workforce could have at least 10% of their work tasks affected by the introduction of LLMs, while approximately 19% of workers may see at least 50% of their tasks impacted.”

• Navigating the Jagged Technological Frontier: Field Experimental Evidence of the Effects of AI on Knowledge Worker Productivity and Quality: An experiment involved 758 BCG consultants comprising about 7% of the individual contributor-level consultants at the company. After establishing a performance baseline on a similar task, subjects were randomly assigned to one of three conditions: no AI access, GPT-4 AI access, or GPT-4 AI access with a prompt engineering overview. For each one of a set of 18 realistic consulting tasks within the frontier of AI capabilities, consultants using AI were significantly more productive (they completed 12.2% more tasks on average, and completed task 25.1% more quickly), and produced significantly higher quality results (more than 40% higher quality compared to a control group).

• What Is ChatGPT Doing … and Why Does It Work? (2023): A very detailed explanation from Wolfram that starts simply and ends up getting into the details.

The Business of GenAI

• Behind the Hype: A Deep Dive into the AI Value Chain: We are near the local peak of a GenAI hype bubble in 2023, where ZIRP still affects AI startups. AI value chains and profit pools - what matters and who will win is variable. The current AI ecosystem (MAMANA, ML Ops, Startups) is growing fast, and this posits where the best investments are.

• Who owns the generative AI platform?: Our flagship assessment of where value is accruing, and might accrue, at the infrastructure, model, and application layers of generative AI.

• Navigating the high cost of AI compute: A detailed breakdown of why generative AI models require so many computing resources, and how to think about acquiring those resources (i.e., the right GPUs in the right quantity, at the right cost) in a high-demand market.

• The economic potential of generative AI | McKinsey: Estimates that generative AI could add the equivalent of $2.6 trillion to $4.4 trillion annually across the 63 use cases we analyzed—by comparison, the United Kingdom’s entire GDP in 2021 was $3.1 trillion. About 75% of the value that generative AI use cases could deliver falls across four areas: Customer operations, marketing and sales, software engineering, and R&D.

• Generative AI: A Creative New World (Sequoia): Certain functions may be completely replaced by generative AI, while others are more likely to thrive from a tight iterative creative cycle between human and machine—but generative AI should unlock better, faster and cheaper creation across a wide range of end markets. The dream is that generative AI brings the marginal cost of creation and knowledge work down towards zero, generating vast labor productivity and economic value—and commensurate market cap.

• Generative AI: Hype, or Truly Transformative (Goldman Sachs): GS economists assess the technology’s potentially large impact on productivity and growth, which their equity strategists estimate could translate into significant upside for US equities over the medium-to-longer term, though our strategists also warn that past productivity booms have resulted in equity bubbles that ultimately burst.

• AI Canon by a16z: This course has a mix of basic to advanced articles on AI, as curated by the VCs at a16z.

Legal Concerns

Currently, there is much uncertainty about the legality of GenAI tools, especially as they push the boundaries of copyright law when it comes to training models, along with other issues of liability.

The high-level copyright issues and ambiguities are presented by CRS GenAI and Copyright Primer (2023) and Samuelson GenAI Meets Copyright (2023). Three sets of ongoing litigation are detailed in TC Current GenAI Legal Cases, or you can read the complaints in Github Copilot Litigation and Silverman complaints against OpenAI. Prof. Sag highlights Copyright Safety for Generative AI, where memorization of the training data is more likely and copyright is problematic (e.g. models are trained on many duplicates of the same work; images are associated with unique text descriptions; and the ratio of the size of the model to the training data is relatively large).

Some solutions and technical directions to get around the copyright issues are presented in Lemley on Fair Learning, Foundation Models and Fair Use, and Provable Copyright for GenAI. There were some interesting discussions about this in ICML this year, per Gen AI and Law, ICML-23.

Best-in-Class GenAI Tools

I’ve used a lot of the existing tools on the market - these are all available (not vapor-ware) and are amongst the best.

Text/LLMs

• ChatGPT (esp GPT-4): GPT-4 is still the overall best LLM available right now.

• Claude 2: Very good at mimicking style, and approaching GPT-4; one of the top 3 LLMs.

• Bing AI (Sydney): Likely built off GPT-4, or something close, and is free; it’s great because it pairs it with search and citations.

• Github Copilot: Best code gen tool right now

• OpenAI CodeInterpreter: A close second to Github Copilot, but better for some tasks.

• Replit Ghostwriter: A tool that assists in writing code by providing suggestions based on your input.

Image

• MidJourney: The best text-to-image generation tool, but with a clunky Discord interface.

• Dall-E 2: A close second to MidJourney, decent interface

• Adobe Firefly: The first Big Tech image creator launched widely

• Stability AI (Stable Diffusion): An open-source image gen tool but with copyright issues

Video

• RunwayML (uses GPT-4): A video generation tool that utilizes advanced GPT-4 technology.

• Synthesia: A tool specialized in creating AI-generated video content.

• Invideo AI: A user experience design tool with capabilities for creating video prototypes.

Audio

• Splash: A tool facilitating the generation of audio content.

• Meta AudioCraft: A platform for crafting audio elements using AI technology.

• Murf.AI: A tool to make studio-quality voiceovers in minutes. Use Murf’s lifelike AI voices for podcasts, videos, and professional presentations.

Other / Productivity

• Tome (Slides Creator): A tool that aids in creating presentation slides efficiently.

• Microsoft Office 365 Copilot: An AI assistant for enhancing productivity within the Office 365 suite.

• Google Workspace Duet AI: Collaborative AI tool integrated within the Google Workspace for enhanced functionality.

• Copy.AI: A tool designed to assist in generating copy for various content types.

• Jasper AI: An AI writer and AI marketing software for enterprise teams. Creates blog posts, marketing copy, etc.

3) GenAI Basics (Technical)

This is a list of resources and guides to get coders started on learning the fundamental algos behind GPT, diffusion, and GAN models. The courses offer some of the best papers, readings, and videos together, while I’ve curated some of the best LLM papers.

Guides

• The illustrated transformer: A more technical overview of the transformer architecture by Jay Alammar.

• The annotated transformer: This in-depth post covers transformers at a source-code level. Requires some knowledge of PyTorch.

• Let’s build GPT: from scratch, in code, spelled out: For engineers, Karpathy does a video walkthrough of how to build a GPT model.

• The illustrated Stable Diffusion: Introduction to latent diffusion models, the most common type of generative AI model for images.

• The Illustrated VQ-GAN with CLIP : VQGAN stands for Vector Quantized Generative Adversarial Network, while CLIP stands for Contrastive Image-Language Pretraining. Whenever we say VQGAN-CLIP1, we refer to the interaction between these two networks. They’re separate models that work in tandem.

  In essence, the way they work is that VQGAN generates the images, while CLIP judges how well an image matches our text prompt.

• RLHF: Reinforcement Learning from Human Feedback: Chip Huyen explains RLHF, which can make LLMs behave in more predictable and human-friendly ways. This is one of the most important but least well-understood aspects of systems like ChatGPT.

• Reinforcement learning from human feedback: Computer scientist and OpenAI cofounder John Shulman goes deeper in this great talk on the current state, progress, and limitations of LLMs with RLHF.

Courses

• Stanford CS25: Transformers United, an online seminar on Transformers.

• Stanford CS324: Large Language Models with Percy Liang, Tatsu Hashimoto, and Chris Re, covering a wide range of technical and non-technical aspects of LLMs.

• GenAI Short Courses from DeepLearning.ai (2023) (e.g. Diffusion Models, LangChain, etc) 

LLM Background

• Eight Things to Know about Large Language Models (2023): Probably my favorite LLM survey position as it takes some strong stances and comprehensively cites the literature.

• Self-attention and transformer networks (2021)

• What are embeddings (2023)

• The Illustrated Word2vec - A Gentle Intro to Word Embeddings in Machine Learning (YouTube) (2022)

• Catching up on the weird world of LLMS (2023)

• GPT-4 technical report (2023): The latest and greatest paper from OpenAI, known mostly for how little it reveals! (blog post). The GPT-4 system card sheds some light on how OpenAI treats hallucinations, privacy, security, and other issues.

  • State of GPT, by Karpathy (2023)

• Alpaca: A strong, replicable instruction-following model (2023): Out of Stanford, this model demonstrates the power of instruction tuning, especially in smaller open-source models, compared to pure scale.

• Claude 2 Model Card (Anthropic)(2023)

• PaLM: Scaling language modeling with pathways (2022): PaLM, from Google, utilized a new system for training LLMs across thousands of chips and demonstrated larger-than-expected improvements for certain tasks as model size scaled up. (blog post). See also the PaLM-2 technical report.

• LLAMA 2 Open Foundation and Chat Models Report (Meta) (2023)

4) GenAI Papers and Advanced Courses (Technical)

There’s only one GenAI textbook worth reading, and that is Foster’s (for now). The scaling hypothesis and Chinchilla papers pair well with it. Some of these LLM courses are newer and I’m still going through them (there’s not much out on the market).

In-Depth Concepts

• Textbook: Foster, Generative Deep Learning (2nd Edition): Probably the single best technical book to get acquainted with the key GenAI models, from a great transformers overview to sections on VAEs, GANs, LLMs, audio models, and more.

• The scaling hypothesis: One of the most surprising aspects of LLMs is that scaling—adding more data and compute—just keeps increasing accuracy. GPT-3 was the first model to demonstrate this clearly, and Gwern’s post does a great job explaining the intuition behind it.

• Chinchilla’s scaling implications: Nominally an explainer of the important Chinchilla paper (see below), this post gets to the heart of the big question in LLM scaling: are we running out of data? This builds on the post above and gives a refreshed view on scaling laws.

Advanced Courses

• Princeton COS 597G (Fall 2022): Understanding Large Language Models 

• UC Berkeley CS294/194-196: Responsible GenAI and Decentralized Intelligence

• CMU LLMs Course - Large Language Models Methods and Applications (11-667) (2023)

• Generative AI with LLMs, Coursera/AWS (2023) 

• DataBricks LLM Course (2023)

LLM Papers

• Eight Things to Know about Large Language Models (2023): Probably my favorite LLM survey position as it takes some strong stances and comprehensively cites the literature.

• A survey of large language models (2023): Comprehensive breakdown of current LLMs, including development timeline, size, training strategies, training data, hardware, and more.

• Sparks of artificial general intelligence: Early experiments with GPT-4: Early analysis from Microsoft Research on the capabilities of GPT-4, the current most advanced LLM, relative to human intelligence.

• Attention is all you need (2017): The original transformer work and research paper from Google Brain that started it all. (blog post)

• BERT: Pre-training of deep bidirectional transformers for language understanding (2018): One of the first publicly available LLMs, with many variants still in use today. (blog post)

• Improving language understanding by generative pre-training (2018): The first paper from OpenAI covers the GPT architecture, which has become the dominant development path in LLMs. (blog post)

• Language models are few-shot learners (2020): The OpenAI paper that describes GPT-3 and the decoder-only architecture of modern LLMs.

• RLHF: Training language models to follow instructions with human feedback (2022): OpenAI’s paper explaining InstructGPT, which utilizes humans in the loop to train models and, thus, better follow the instructions in prompts. This was one of the key unlocks that made LLMs accessible to consumers (e.g., via ChatGPT). (blog post)

• LaMDA: language models for dialog applications (2022): A model from Google specifically designed for free-flowing dialog between a human and chatbot across a wide variety of topics. (blog post) 

• Training compute-optimal large language models (2022): The Chinchilla paper. It makes the case that most models are data-limited, not compute-limited, and changed the consensus on LLM scaling. (blog post)

• A Survey on Evaluation of Large Language Models (2023): Model evals will be crucial to helping improve models - this is the best survey yet.

• Summary of ChatGPT/GPT-4 (2023)

• NLP reasoning survey (2023)

• Theory of Mind in LLMs (2023)

• Predictability and Surprise in LLMs (2022)

• Emergent abilities of LLMs (2022)

• Multimodal LLMs (2023)

• Augmented LLMs with reasoning (2023)

Model enhancements (e.g. fine-tuning, retrieval, attention)

• Deep reinforcement learning from human preferences (2017): Research on reinforcement learning in gaming and robotics contexts, that turned out to be a fantastic tool for LLMs.

• Retrieval-augmented generation for knowledge-intensive NLP tasks (2020): Developed by Facebook, RAG is one of the two main research paths for improving LLM accuracy via information retrieval. (blog post)

• Improving language models by retrieving from trillions of tokens (2021): RETRO, for “Retrieval Enhanced TRansfOrmers,” is another approach—this one by DeepMind—to improve LLM accuracy by accessing information not included in their training data. (blog post)

• LoRA: Low-rank adaptation of large language models (2021): This research out of Microsoft introduced a more efficient alternative to fine-tuning for training LLMs on new data. It’s now become a standard for community fine-tuning, especially for image models.

• Constitutional AI (2022): The Anthropic team introduces the concept of reinforcement learning from AI Feedback (RLAIF). The main idea is that we can develop a harmless AI assistant with the supervision of other AIs.

• FlashAttention: Fast and memory-efficient exact attention with IO-awareness (2022): This research out of Stanford opened the door for state-of-the-art models to understand longer sequences of text (and higher-resolution images) without exorbitant training times and costs. (blog post)

• Hungry Hungry Hippos: Towards Language Modeling with State Space Models (2022): Again from Stanford, this paper describes one of the leading alternatives to attention in language modeling. This is a promising path to better scaling and training efficiency. (blog post)

Image generation models

• The Annotated Diffusion Model (2022): Takes a deeper look into Denoising Diffusion Probabilistic Models (also known as DDPMs, diffusion models, score-based generative models or simply autoencoders) as researchers have been able to achieve remarkable results with them for (un)conditional image/audio/video generation. Also see the Awesome Diffusion Models repo for papers, tutorials, and videos.

• The recent rise of diffusion-based models (2022): A very recent history of solving the text-to-image generation problem and explain the latest developments regarding diffusion models, which are playing a huge role in the new, state-of-the-art architectures.

• Introduction to Diffusion Models for Machine Learning (2022): Examines the theoretical foundations for Diffusion Models, and then demonstrate how to generate images with a Diffusion Model in PyTorch.

• What are Diffusion Models? (2021): Compares them to GANs and VAEs. Diffusion models are inspired by non-equilibrium thermodynamics. They define a Markov chain of diffusion steps to slowly add random noise to data and then learn to reverse the diffusion process to construct desired data samples from the noise. Unlike VAE or flow models, diffusion models are learned with a fixed procedure and the latent variable has high dimensionality (same as the original data).

• Learning transferable visual models from natural language supervision (2021): Paper that introduces a base model—CLIP—that links textual descriptions to images. One of the first effective, large-scale uses of foundation models in computer vision. (blog post)

• Zero-shot text-to-image generation (2021): This is the paper that introduced DALL-E, a model that combines the aforementioned CLIP and GPT-3 to automatically generate images based on text prompts. Its successor, DALL-E 2, would kick off the image-based generative AI boom in 2022. (blog post)

• High-resolution image synthesis with latent diffusion models (2021): The paper that described Stable Diffusion (after the launch and explosive open source growth).

• Photorealistic text-to-image diffusion models with deep language understanding (2022): Imagen was Google’s foray into AI image generation. More than a year after its announcement, the model has yet to be released publicly as of the publish date of this piece. (website)

• DreamBooth: Fine tuning text-to-image diffusion models for subject-driven generation (2022): DreamBooth is a system, developed at Google, for training models to recognize user-submitted subjects and apply them to the context of a prompt (e.g. [USER] smiling at the Eiffel Tower). (website)

• Adding conditional control to text-to-image diffusion models (2023): This paper from Stanford introduces ControlNet, a now very popular tool for exercising fine-grained control over image generation with latent diffusion models.

Agents

• A path towards autonomous machine intelligence (2022): A proposal from Meta AI lead and NYU professor Yann LeCun on how to build autonomous and intelligent agents that truly understand the world around them.

• ReAct: Synergizing reasoning and acting in language models (2022): A project out of Princeton and Google to test and improve the reasoning and planning abilities of LLMs. (blog post)

• Generative agents: Interactive simulacra of human behavior (2023): Researchers at Stanford and Google used LLMs to power agents, in a setting akin to “The Sims,” whose interactions are emergent rather than programmed.

• Reflexion: an autonomous agent with dynamic memory and self-reflection (2023): Work from researchers at Northeastern University and MIT on teaching LLMs to solve problems more reliably by learning from their mistakes and past experiences.

• Toolformer: Language models can teach themselves to use tools (2023): This Meta project trained LLMs to use external tools (APIs, in this case, pointing to things like search engines and calculators) in order to improve accuracy without increasing model size. 

• Auto-GPT: An autonomous GPT-4 experiment: An open-source experiment to expand on the capabilities of GPT-4 by giving it a collection of tools (internet access, file storage, etc.) and choosing which ones to use in order to solve a specific task.

• BabyAGI: This Python script utilizes GPT-4 and vector databases (to store context) in order to plan and execute a series of tasks that solve a broader objective.

Code generation

• Evaluating large language models trained on code (2021): This is OpenAI’s research paper for Codex, the code-generation model behind the GitHub Copilot product. (blog post)

• Competition-level code generation with AlphaCode (2021): This research from DeepMind demonstrates a model capable of writing better code than human programmers. (blog post)

• CodeGen: An open large language model for code with multi-turn program synthesis (2022): CodeGen comes out of the AI research arm at Salesforce, and currently underpins the Replit Ghostwriter product for code generation. (blog post)

• Code Llama: Open Foundation Models for Code (2023): The new open source LLAMA for code model from Meta.

Video generation

• Make-A-Video: Text-to-video generation without text-video data (2022): A model from Meta that creates short videos from text prompts, but also adds motion to static photo inputs or creates variations of existing videos. (blog post)

• Imagen Video: High-definition video generation with diffusion models (2022): A version of Google’s image-based Imagen model optimized for producing short videos from text prompts. (website)

• Gen-L-Video: Multi-Text to Long Video Generation via Temporal Co-Denoising (2023): This study explores the potential of extending the text-driven ability to the generation and editing of multi-text conditioned long videos. It introduces a novel paradigm dubbed as Gen-L-Video, which is capable of extending off-the-shelf short video diffusion models for generating and editing videos comprising hundreds of frames with diverse semantic segments without introducing additional training, all while preserving content consistency.

5) Build your own GenAI models from Scratch

Practical Guides to Building with LLMs

• Build a GitHub support bot with GPT3, LangChain, and Python: One of the earliest public explanations of the modern LLM app stack. Some of the advice in here is dated, but it kicked off widespread adoption and experimentation of new AI apps.

• Building LLM applications for production: Chip Huyen discusses many of the key challenges in building LLM apps, how to address them, and what types of use cases make the most sense.

• Prompt Engineering Guide: For anyone writing LLM prompts—including app devs—this is the most comprehensive guide, with specific examples for a handful of popular models. For a lighter, more conversational treatment, try Brex’s prompt engineering guide.

• Prompt injection: What’s the worst that can happen? Prompt injection is a potentially serious security vulnerability lurking for LLM apps, with no perfect solution yet. Simon Willison gives the definitive description of the problem in this post. Nearly everything Simon writes on AI is outstanding.

• OpenAI cookbook: For developers, this is the definitive collection of guides and code examples for working with the OpenAI API. It’s updated continually with new code examples.

• Vector Embeddings - Pinecone Learning Center: Many LLM apps are based around a vector search paradigm. Pinecone’s learning center—despite being branded vendor content—offers some of the most useful instructions on how to build with vector search.

• LangChain docs: As the default orchestration layer for LLM apps, LangChain connects to just about all other pieces of the stack. So their docs are a real reference for the full stack and how the pieces fit together.

Algos for GenAI

• The case for GZIP Classifiers and more on nearest neighbors algos

• Meta Recsys Using and extending Word2Vec

• The State of GPT (YouTube)

• How is LlamaCPP Possible?

• On Prompt Engineering

• Transformers from Scratch

Deployment of GenAI

• Building LLM Applications for Production

• Challenges and Applications of Large Language Models

• All the Hard Stuff Nobody talks about when building products with LLMs

• Scaling Kubernetes to run ChatGPT

• Numbers every LLM Developer should know

Evaluation

• Interpretable Machine Learning

• Evaluating ChatGPT

• ChatGPT: Jack of All Trades, Master of None

OpenAI needs to increase revenues in order to fund the eye-watering costs of computing power required to train and run large AI models. Its CEO Sam Altman recently described the company as on track to be “the most capital-intensive start-up in Silicon Valley history”.

While OpenAI has one of the most inventive and resourceful startup teams in recent Silicon Valley history, I’m both awed by them and still skeptical they can get their revenues to outpace costs. They still lack true product-market fit and a path to break even, let alone profitability, at their current trajectory. I don’t know if I would describe their run as heroic (“shooting for the moon”) or reckless (“the largest dumpster fire of VC funds ever”). The big question is if they can translate their technical and early product success into a business success, which merits an exploration of the AI value chain and where all the players in the ecosystem sit.

I will cover my main hypotheses in the following order:

• We are near the local peak of a GenAI hype bubble

• ZIRP still affects AI startups

• AI value chains and profit pools - what matters and who will win

• The current AI ecosystem (MAMANA, ML Ops, Startups)

• Where the best investments are

I think a lot about AI value chains since I support a large team of AI research scientists, MLEs, and product managers that help run one of the largest AI systems in the world, with fat margins - the Meta ads system (where 98% of Meta’s $116bn in 2022 revenue came from this system, which one study showed created huge value for advertisers). We’re constantly looking for ideas and investments in high, sustainable ROIs and NPVs, and generally, how to run profitable systems and not scaled ones at a loss.

We are near the local peak of a GenAI hype bubble

We’ve seen a number of tech bubbles in the last few years, with the most notable ones being SaaS valuations in the last decade and then more recently the crypto bubble of 2021. Per a recent Pitchbook report, “VCs have steadily increased their positions in generative AI, from $408 million in 2018 to $4.8 billion in 2021 to $4.5 billion in 2022. Angel and seed deals have grown, as well, with 107 deals and $358.3 million invested in 2022 compared with just 41 and $102.8 million in 2018.”

More recently, the French GenAI startup Mistral, raised $113M at 4 weeks old, with no product and no real team other than a handful of founders. The plans are hazy: “It’s very early to talk about what Mistral is doing or will be doing — it’s only around a month old — but from what Mensch said, the plan is to build models using only publicly available data to avoid legal issues that some others have faced over training data, he said; users will be able to contribute their own datasets, too. Models and data sets will be open-sourced, as well.”

I’ve seen dozens of deals happen at a smaller scale in the last 6 months (Glean, Stability, Mutiny, Descript, Adept, Runway). What’s been stunning is that VCs are funding research scientists with no product and no revenue at Series A to Series C levels. I have no doubt we are near the peak of a bubble.

Note that I think this is a localized bubble, similar to the Dot-com/internet bubble of 1999 and 2000, where dumb ideas were funded and it all popped within 18 months and 97%+ of the companies went bust. However, I don’t see many viable companies like Amazon and Paypal, who would live through that to succeed (maybe Jasper, Midjourney, Replit, and Copy.AI?). The AI trend will play out over decades and profoundly change tech, business, and society, but right now the capital chasing it is clearly the dumb money.

ZIRP still affects AI startups

Why is there so much dumb money sloshing around?

ZIRP. The Zero (nominal) Interest Rate Policy that central banks enacted from 2008 onwards basically continued till the end of 2021, a 13-year period that was unparalleled in 800 years of global interest rate history. Many countries had negative real rates, and that caused savers and investors to leave bank accounts and bonds and chase risky stocks, angel and VC investment, and even crypto. Basically, gravity was suspended for all financial assets for 13 years, and a generation of founders and VCs went to play with free money, and now gravity has returned (with the US Fed and other central banks raising interest rates from 2022 on). With zero/low-interest rates, the future/imaginary cash flows of tech stocks were worth a lot and valuations were high; at 5% or higher, those get discounted to be worth very little (it’s the basics of discounted cash flow valuation math).

Most traditional and crypto startups face a dire funding environment with a mass extinction event coming along, and so many VCs have decided to YOLO into AI startups for a final splurge of their remaining LP funds (often raised in 2021 or earlier, in more bountiful times).

The problem is that the already loose deal underwriting standards of the last decade have even been abandoned (e.g. $1mm ARR to do a Series A, $10mm for Series B, $50mm for Series C, etc). Note that these were low standards, focused on growth at all costs, and ignoring breakeven or profitability. The new AI investments are often made at no revenue or even massive losses, with an assumption of a deus ex machine (a Microsoft or Google-led funding round? the Second Coming via AGI?) to support the startups.

This could make sense if we see strong product-market fit, massive productivity gains, and zero interest rates again. I just doubt we see the first two in the next four years, and I doubt we see ZIRP again this decade, with financial experts like JP Morgan CEO Jamie Dimon expecting rates could go even higher, to 6% or 7%. I don’t know where interest rates go, but if inflation stays high and real rates remain negative or near zero, it’s unlikely we get back to ZIRP.

So ultimately, the test of strong AI companies is that of gravity vs rockets. That is, whether some rockets with strong business models can defy gravity (with skyrocketing revenues and costs in line) to get past the normal economics of earthbound companies. That will depend on the handful of players who “win”, that is, dominate the AI value chain and own most of the near-term and future profits generated by AI systems.

AI value chains and profit pools - what matters and who will win

A value chain is a series of consecutive steps that go into the creation of a finished product, from its initial design to its arrival at a customer's door. The chain identifies each step in the process at which value is added, including the sourcing, manufacturing, and marketing stages of its production. Generally, a different company owns different steps in the chain.

For example, in the music industry, artists create songs, labels aggregate and market artists, streaming services distribute music, concert and ticketing companies running musical events, etc. Most of the profits in this value chain that the music passes through go to the record labels - that’s where the profits aggregate and it’s the best place in the ecosystem.

A profit pool is the total profits earned in an industry at all points along the industry’s value chain. The pool will be deeper in some segments of the value chain than in others, and depths will vary within an individual segment as well. In most industries, a handful of companies get most of the profit pool, and the extreme may be something like smartphones, where Apple gets 48% of total revenue but 85% of the total industry operating profit pool.

Predicting exactly who will win the value chain in building and selling AI products is hard, but we can discuss the potential value each player might bring.

1. Model Builders: These include researchers, data scientists, and engineers who design, build, and refine AI models. Their value lies in their innovation and the effectiveness of their models. However, this area is prone to commoditization as more advanced models are frequently released in open-source, lowering their profitability over time. Still, model builders with proprietary, niche, or highly specialized models can capture significant value. One Google engineer recently wrote in a leaked memo that there is no moat in building models, so Google and OpenAI have no competitive edge here - this will likely be commoditized.

2. Infrastructure Providers: These players provide the foundational services like cloud storage and processing power needed to run AI systems. Companies like Amazon (AWS), Microsoft (Azure), and Google (Cloud) dominate this space. While competition is fierce, these services are essential, providing a steady profit stream. Their share of the value chain will depend on pricing dynamics and competition, but their ubiquity ensures a substantial piece of the pie.

3. Hardware and GPUs (“chips”): AI processing often requires specialized hardware. Companies like Nvidia and AMD that produce GPUs, and those like Google creating custom AI hardware (like Tensor Processing Units), are key players. They can capture significant value, especially as demand for AI computation rises. However, advances in software optimization and alternative hardware (like FPGAs and ASICs) could challenge their share, and some large companies are designing their own chips, like Google for TPUs, Meta for MTIA, Amazon’s Graviton, and Microsoft’s in-house GPU. Nvidia and AMD won’t be able to maintain a lock here, but Nvidia has a commendable lead.

4. Applications: This includes businesses that use AI models to provide specific services or products. It’s a broad category that could include everything from recommendation systems to autonomous vehicles. Their share of the value chain depends on how essential AI is to their value proposition and their market’s size. Given the potential for AI to disrupt many industries, the potential profitability here can be immense. However, very few of the current startups want to actually create and run specialized apps for specific customers in narrow industries - basically industry and enterprise solutions. Most want to just provide models. Corporates are more interesting but slower - they are just forming their GenAI teams, will take 18 months to prototype and get a sense of what products to launch, and then 2-3 years to scale it out.

5. Data Providers: AI models often require vast amounts of data for training. Companies that can provide high-quality, unique data can capture significant value, though often the best data is locked in the messy vaults of old-school Fortune 1000-type companies.

In terms of profitability in the AI value chain, it’s likely to be distributed unevenly, with some companies capturing more value due to strategic advantages, monopolistic behaviors, or superior products.

Ultimately, the “winners” in the AI value chain will probably be those who can control or influence multiple stages. For example, a company that creates AI models, has proprietary data, and builds applications could capture more of the value chain than a company operating in just one area. Similarly, companies providing the essential infrastructure for AI (like cloud providers or hardware manufacturers) could capture significant value due to the necessity of their offerings.

The current AI ecosystem (MAMANA, ML Ops, Startups)

MAMANA stands for Microsoft/OpenAI, Apple, Meta, Alphabet/Google, NVIDIA, and Amazon. They used to be called FAANG and are sometimes called MAGMA, but I stand by MAMANA as the best descriptor. These are the most important tech companies in the world (with TSMC and ASML supplying them and growing to be as important). Ben Thompson has a piece in January 2023 called “AI and the Big Five” where he is directionally right, but where I will state my own views for all six, with Microsoft, Meta, and Google as the strongest contenders. MAMANA will be the biggest winners in the AI value chain, simply because they own, control, of influence multiple stages in the value chain, and many have end-to-end production and relationships with customers (they have great distribution as aggregators).

Microsoft/OpenAI: Microsoft is in a great place to win by selling GPU compute and modeling services via Azure, its Fortune 1000 relationships, its incorporation of AI into apps like those in Microsoft Office, and its “free” usage of OpenAI models (after the hefty investment). I don’t see any downsides here; Nadella and his C-suite may just be the best executive team of any company in the world, and they are all in on AI when they have clear, profitable, business use cases on their enterprise apps and services (like seeing premium MS Office subscriptions with generative AI, at $30 per month, per user).

Apple: Apple has a few niche and proprietary apps and uses, like traditional machine learning models for things like recommendations and photo identification and voice recognition, but nothing that moves the needle for Apple’s business in a major way. Apple is likely is 3-5 years behind the other big tech companies in AI software and hardware, and its closeted and secretive culture repels all the AI talent, who like to publish papers and share open-source tools. While Apple is losing the AI war, it has such a strong ecosystem advantage with its superior hardware (iPhones, iPads, Macs, etc) that it will take a while for this to become evident.

Alphabet/Google: If I were forced to name the world’s most important AI institution, it would be Google. Brain/Research/DeepMind are the top repositories of AI talent, and all of Google’s key products (e.g. search, ads, G-suite, G-mail, Chrome, Android, etc) are thoroughly infused with AI. Google engineers came up with the transformers algorithmic discovery powering most GenAI. It has the best product distribution of any company in the world (esp with the search browser monopoly of Chrome, Firefox, and Safari) and more data than any organization in the world (likely even more than the CCP). And they make their own hardware with TPUs and have GCP - so they have everything to lose and the best advantages. The caveat is that they haven’t launched any major new products in 10 years - the culture may have gotten stagnant, similar to Microsoft in 2005.

Meta: Meta is underappreciated as an AI superpower (along with Google and Microsoft/OpenAI), but it too provides much open source infrastructure (the PyTorch stack, used by most of the AI researchers and startups), is making its own GPUs, and just recently open-sourced models like LLaMA to support startups and academia and weaken incumbents. All of Meta’s core products are infused with AI and couldn’t function without them (e.g Facebook, Instagram, Whatsapp, Messenger, Ads, etc). It also has more data than anyone else other than Google. Its only weakness is not having a paid cloud service or business customers (other than advertisers), and it needs to diversify away from ads.

NVIDIA: NVIDIA is riding high because it makes the best GPUs, the A100s and H100s and has a clear 1-3 year lead over others. Orders will be off the hook for these products and it’s run very well by CEO Huang who is one of the savviest hardware CEOs and founders in the world, along with a strong software stack of CUDA and other tools. The only downside is the coming competition from AMD, GCP, Amazon, Azure, etc with “good enough chips.”

Amazon: Amazon is in a strong position to capitalize via AWS for selling GPU usage and storage, and also using its retail business data to create amazing consumer products using AI. It also has a strong discipline to run products at break-even or profits, with the big exception being the Amazon Alexa fiasco and losses near $10bn (probably the highest loss-making AI product to date, with ChatGPT on track to beat it).

MLOps, short for Machine Learning Operations, refers to combining Machine Learning (ML), DevOps, and Data Engineering to standardize and streamline the lifecycle of machine learning projects. MLOps aims to increase automation and improve the quality of production ML while also focusing on business and regulatory requirements. It also allows for the consistent and reliable delivery of ML applications, by bridging the gap between data science (which deals with the design and development of models) and operations (which deals with the deployment and maintenance of models). They basically sell the “picks and shovels” to the application developers.

The top players in the MLOps industry include various tech companies, startups, and cloud service providers. Among tech giants, Google's Vertex AI offers MLOps services that allow for the end-to-end management of ML models, from building to deployment and maintenance. Similarly, Microsoft's Azure Machine Learning and Amazon's SageMaker provide comprehensive MLOps solutions as part of their cloud platforms.

Several startups and smaller companies also play a significant role in the MLOps space. Companies like DataRobot and Algorithmia provide platforms that incorporate various aspects of MLOps, from automated ML and data wrangling to model deployment and monitoring. Additionally, companies like Tecton, Weights and Biases, Pinecone, Grid.AI, Databricks, C3.ai, and H2O.ai offer solutions focusing on feature stores and data pipelines, crucial for large-scale ML deployments. Lastly, open-source tools like Kubeflow, MLflow, and Seldon are also significant players, providing MLOps capabilities to those who prefer or require customizable and flexible tools for their ML pipelines.

I expect there will be a few ML OPs winners in the AI value chain here, with the earliest and strongest winner being Databricks, with $1 billion of revenue in 2023 and strong growth. They will consolidate and buy smaller players to create a platform, but will also compete with AWS, Azure, and GCP, and that is a hard game to play (but I’m cheering them on from the sidelines - they make great products). One thing I fear is that too many startups want to play in ML Ops making the shovels, and not enough want to actually develop industry-specific apps to actually serve many customers, and so the ML Ops space may be over-invested in for now.

Next come the other startups, like the modeling startups of OpenAI, Anthropic, Mistral, Stability, etc. I expect most of these will fail as they don’t have any differentiated products, lack product market fit, and so lack profits - they are operating at huge losses and will likely fail. Stable Diffusion has a cloudy history, and many will have large lawsuits and losses around copyright and data usage. Finally, actually using the models to inference and serve users is very expensive, and many are choosing to scale losses like OpenAI.

What about the few startups that are defying the odds to generate actual revenue and who may be close to break-even or even profitability? Some I would put on that “success list” include Replit, Midjourney, Jasper, Copy.AI, etc.

What makes these startups more likely to succeed:

1. Their revenues are in line with costs (they don’t scale massive losses), and they are run in a lean manner (extra points to Midjourney);

2. They have some Product-Market-Fit, evidenced by growing revenue and profits for some - they don’t give away expensive free products;

3. They picked a niche vertical use case to serve via apps, like using LLMs for copywriting, or coding for teens;

4. They are trying to build unique datasets and niche models, and generally own more and more parts of the value chain (e.g their own data, models, infrastructure, and apps - it’s unlikely they can build their own GPUs for a while).

So what would it take for an OpenAI, Anthropic, Inflection, Adept, Mistral, HuggingFace, and others to improve their odds of success? They need to work on building strong businesses, grounded in specific products, and not just get carried away by hype. That means: 1) Getting costs in line with revenues and not shooting for the moon; 2) Showing actual product market fit with paid products, and not giving away lossy free products that burn cash due to inferencing costs as they scale (e.g. Paid ChatGPT plus vs free ChatGPT) - alternatively, they can try to develop other revenue models like pay by the query (buy a pack of queries) or advertising; 3) Diving into verticals, and realizing they cannot just serve LLMs or generic AI agents via APIs - they need to pick a few industries and use cases where they will compete directly and cultivate more direct customer relationships; 4) Generally trying to own more of the end-to-end stack, which means owning data that is valuable for verticals, their own infrastructure (and not outsourcing it to Azure and GCP), etc.

I believe the leaders of OpenAI, Anthropic, and these other teams are smart and creative, and encourage them to read up on the history of how the idealistic, loss-generating prior Web 2.0 companies (e.g. Google, Facebook, Booking/Priceline, Airbnb, etc) hired Chief Commercial/Business Officers and got their businesses in line (e.g find their Eric Schmidt, Omid Kordestani, Jay Walker, Sheryl Sandberg, Caroline Everson, Laurence Tosi, Belinda Johnson, etc).

Where the best investments are

The basics of investment don’t change much and that’s also true for AI companies - who can survive in the value chain and own and protect a profit pool - basically who can create a business moat.

The investor Warren Buffett explains business moats well:

"What we're trying to do, is we're trying to find a business with a wide and long-lasting moat around it, surround -- protecting a terrific economic castle with an honest lord in charge of the castle… we're trying to find is a business that, for one reason or another -- it can be because it's the low-cost producer in some area, it can be because it has a natural franchise because of surface capabilities, it could be because of its position in the consumers' mind, it can be because of a technological advantage, or any kind of reason at all, that it has this moat around it. But we are trying to figure out what is keeping -- why is that castle still standing? And what's going to keep it standing or cause it not to be standing five, 10, 20 years from now. What are the key factors? And how permanent are they? How much do they depend on the genius of the lord in the castle?"

So with that, I will end with some specific predictions of who I expect will win, given their moats or based on their execution:

1. MAMANA: Microsoft, Google, and Meta seem like the clear winners, as they own a large part of the value chain and have large profit pools to expand. AWS as a cloud provider should do extremely well. Apple is the loser so far, and NVIDIA will do well too, but will have more competition going forward.

2. MLOps: This area is harder to know - I don’t have a great sense of the ML cloud provider tools versus the smaller ones, but Databricks has stood out for its execution, and the company that creates the platform with the most useful tools will win.

3. Startups: The story is really early for this one, with most startups right now clearly as default dead, despite large teams and fundraising rounds. I would bet on Replit, Midjourney, and at least one copywriting company (whichever one grows carefully to be a marketing-tech company and can serve many verticals). But it’s also the most exciting place - who will own industry and use case specific apps, and then execute on the items above to win in the value chain and own the profit pools?

4. Corporates: It’s very early to see which of the traditional Fortune 1000 companies are utilizing AI well for their basic functions (we will know in 3-5 years). I’ve seen some promising work by Tesla, GM/Cruise, CapitalOne, Fidelity, Walmart, and Dell, but most seem quite behind and have a hard time attracting the right AI talent. Companies in the healthcare, retail, and energy industries seem to have the most to gain (or lose).

   

   
   
   

1. Deutsch's "Beginning of Infinity" - A 21st-century classic and the most discussed book in Silicon Valley. Dive into the infinite potential of human knowledge and progress; a world-class physicist lays out the philosophy of knowledge, science, engineering, and progress.

2. Dalio's "Principles" - Learn from Ray Dalio's approach to life and work, centered on radical transparency and meritocracy, embracing mistakes and learning from them, and defining clear principles and values. It pairs well with the philosophical parts of Isaacson’s biography of Elon Musk.

3. Koren's "Wabi-Sabi for Artists, Designers, Poets & Philosophers" - Embrace the beauty of imperfection and transience through the Japanese aesthetic of Wabi-Sabi (read Sen no Rikyu, Miyamoto Musashi, or Kevin Kelly’s “Excellent Advice” next).

4. Pinker’s “Enlightenment Now” - See why the world has progressed so much in the last 1000, 100, 50, and 20 years, why the news media is terrible with its negativity bias, and how to think about Enlightenment values now and in the future (the modern heirs are also Popper and Kuhn).

5. Nussbaum's "Fragility of Goodness" - Uncover the delicate balance between ethics, luck, and moral choices (follow up with Amartya Sen or Bernard Williams).

6. Munger’s “Poor Charlie's Almanack: The Wit and Wisdom of Charles T. Munger” - Get a brilliant polymath’s multiplexed ideas and frameworks. I haven’t enjoyed a book as much since Einstein’s “Ideas and Opinions.”

7. Friedman's "Free to Choose" - Explore the power of economic freedom and individual choice (best read with Von Mises, Schumpeter, and Hayek, then the Naval and Balaji readers).

8. Rand's "Fountainhead" - Understand the importance of individualism, creativity, and integrity as Roark strives to be a builder. Read Rand’s West Point essay “Philosophy: Who Needs It?” and then Karl Popper and Wittgenstein afterward (and Monk on Wittgenstein).

9. Rawls' "Theory of Justice" or "Political Liberalism" - Contemplate the principles of fairness and social justice in a pluralistic world where common values and culture may not intersect much. Read the Pogge biography of Rawls.

10. Frankl's "Man's Search for Meaning" - Walk in his shoes and learn the insights of a Holocaust survivor who found meaning and purpose amidst hellish suffering. Zoom out to Shirer’s “The Rise and Fall of the Third Reich” to get more context.

11. Hesse's "Siddhartha" - Watch the spiritual journey of a person who tries to develop self-knowledge and inner peace tin Siddhartha, including thinking, waiting, and fasting. Follow up with Easwaran’s “The Dhammapada”, Rahula’s “What the Buddha Taught,” or the Karen Armstrong biography.

12. Tolstoy's "War and Peace" - Delve into the complexities of human nature, love, and morality set against the backdrop of the Napoleonic Wars in Russia. Follow it up with Proust if you want to mess your life up.

13. Mill's "On Liberty and Essays" - Reflect on the significance of individual freedom and self-expression, and why they are crucial to a functioning society and republic. Strangely I think Mill pairs well with “Conversations with Jean-Paul Sartre.”

14. Darwin's "Origin of Species" - Read the original text with revolutionary ideas that shaped our understanding of evolution, and now the memes and forces shaping human society and AI. His other books are worth it too, including his “Autobiography” and “The Expression of the Emotions in Man and Animals.”

15. Benjamin Franklin's "Autobiography" - Delve into the life and wisdom of one of America's (and arguably the world’s) most influential figures, learning the pragmatic philosophy behind his curiosity, inventiveness, and pursuit of self-improvement. The Brands and Isaacson biographies of Franklin are wonderful too.

16. Smith’s “The Wealth of Nations” and “Theory of Moral Sentiments” - Read two of the most important Enlightenment books; they need to be read together. You can also add a Voltaire reader, selections from Locke/Montesquieu/Rousseau, and key pieces by Newton and Leibniz (eg. Principia and Monadology).

17. Kant's "Metaphysics of Morals" - Grasp the foundations of moral philosophy and ethical reasoning; this is Kant-lite and quite readable. More Kant comes in Want’s “Introducing Kant: A Graphic Guide” (the comic book is better than the original prose).

18. “Patanjali Yoga Sutra” (Bryant translation) - Explore the transformative eight-fold path of yoga and self-understanding in one of the earliest texts of applied psychology. Mallinson’s “Roots of Yoga” comes next.

19. “Bhagavad Gita” (Barbara Stoller-Miller or Easwaran translation) - Dwell on the timeless wisdom of Indian spirituality and philosophy. To go deeper, read Easwaran’s translation of the Upanishads or Davis’s biography of the Gita.

20. Confucius' "Analects" (Ames translation) - Delve into the principles of virtue, ethics, and social harmony, to understand Ren, Li, Xiao, Yi, and Zhi (weakly translated as humaneness, propriety, family respect, righteousness, wisdom). Read Laozi’s Tao Te Ching next.

21. "Chapters of the Sages / Pirkei Avot" (Bulka translation) - Gain insights from Jewish Talmudic wisdom and ethical teachings, like this one: “If I am not for me, who will be for me? And when I am for myself alone, what am I? And if not now, then when?”

22. Seneca's "Letters from a Stoic" (Penguin edition) - Absorb the teachings of Stoic philosophy for an integrated life, from a man who helped run the Roman Empire (and was superior to Marcus Aurelius).

23. Epictetus' "Handbook" (Oxford translation) - Master the art of resilience and inner strength through Stoicism, from a slave who was a master more than most consuls or emperors.

24. Sophocles' "Antigone" (Fagles translation) - Delve into themes of loyalty, morality, personal ethics, and the individual's struggle against authority in this poignant and thought-provoking play.

25. Aristotle’s “Nichomachean Ethics”, “Politics”, “Rhetoric”, and “Poetics” (McKeon Modern Library edition or Terence Irwin). The OG student of Plato and perhaps his opposite - see why his ghost still haunts all modern philosophy and much science.

26. Plato's "Complete Works" (Cooper translation) - Engage with the biting, painful, and sharp dialogues that shaped Western philosophy, especially Phaedo, Phaedrus, Meno, Symposium, Theaetetus, Parmenides, Protagoras, the Apology, and the flawed but magnificent Republic.

Some high-level thoughts:

ChatGPT is from OpenAI, which has a strategic alliance with its biggest investor, Microsoft. CEO Nadella is pushing hard to get ChatGPT technology into Bing Search and Azure APIs, and if frankly out-executing Google in a big way. Bing is the 2nd biggest player in search, with only a 9% market share.

Google is responding with its Red Team to look at launching Sparrow from its Deepmind subsidiary, or perhaps LaMDA from its Google AI subsidiary. Both make top-notch chat technology to compete with AI (Google is the top AI research org in the world), but have done a poor job launching useful products.

Meanwhile, 2 new search startups have joined the fray. First, You Chat from the former head of Salesforce AI (You.com / Richard Socher) offers a great new approach to search. Second, NeevaAI Chat from the former heads of Google Search (Neeva.com / Sridhar Ramaswamy (ex-SVP of Ads at Google) and Vivek Raghunathan (ex-VP of Monetization at YouTube)) are moving to get chat search into their very-competitive and disruptive subscription-based search service. Arguably, both companies’ regular organic search results are better than Google, and their recently launched chat search will make their products much more competitive (they’re already better than ChatGPT in that they include reference links in their answers and are less likely to make up plausible nonsense). I'm really excited to see these companies give both Google and Bing a run for their money, and push the boundaries of what search could be.

Google’s core business, search, is still growing and highly profitable ($258bn in revenues in 2021, $76bn in profit, with 81% of revenue from search and 92%+ from ads). The company prints money with every search results page it displays (and users are complaining more that the results are worse).

A search chatbot, however, likely needs a new monetization strategy, and it’s unclear whether it could be as profitable as Google’s current search. Google could launch a successful chatbot, and potentially make it profitable – but if the profits are lower than Google’s current search, the company would still lose. This self-cannibalization, often leading to demand-side disruption, is Google’s dilemma. Meta had a similar problem with short-form video, where Reels was less profitable and competed with its main feeds - it just leaned into it by taking some short-run cannibalization to build a strong business in the long run.

Finally, the main thing Google has going for it is locked-in distribution. It pays Apple ~$15bn annually to be the sole default search provider in iOS and Safari, and it pays Mozilla ~$350mm for that role in Firefox. It also controls Android and Chrome, the largest mobile operating system and browser in the world (with 71% and 65% market share in those markets). When you add what it gets from Firefox/Safari and iOS, that goes to 91% browser market share and 99% mobile OS share). This is frankly an impregnable position and may pose antitrust issues.

So how much greater does the search experience from chat have to be to overcome distribution? We shall see - I love the intensifying competition here.

My list this year was heavy on biographies and memoirs, along with machine learning papers (it was a burning hot year for advances in AI and ML); it’s where the fire of my curiosity burned.

BEST BOOK OF 2022

Dyson, Invention: A Life. Making things is hard, especially if you come from a place that looks down upon it, like the UK does for manufacturing. Dyson is a dogged inventor and entrepreneur whose company makes the best vacuum cleaners, hair dryers, air purifiers, and more. This memoir details his life story and process to build products and a company, focused on first principles thinking to identify human needs and engineer products in unconventional, innovative ways. I know how hard it is to manufacture machines, to build in the world of atoms, so I give Dyson a salute for sharing this story.

NON-FICTION

Schmidt and Rosenberg, How Google Works. This is a 2014 book that details the chaos of the older Google, when it was a powerhouse of creative engineers with the triumvirate of Schmidt, Brin, and Page leading it. The authors lay out their unique views on strategy, culture, talent, decision-making, communication, and innovation. Unfortunately, Google in 2022 seems to be a very different company. Other noteworthy tech histories I read this year included “No Filter (Instagram)” and “The Attention Factory (TikTok).”

Schlender and Tetzeli, Becoming Steve Jobs. Jobs was a great CEO who shone as a product genius and team builder with Cook, Jony, Johnson, Fadell, Schiller, Forstall, and more. They made the world’s most valuable company (Apple) and devices (iPhones and MacBooks). While the Isaacson biography gives a broader view of Jobs as a person, Schlender knew Jobs for much longer and gives the best story of his evolution from a bratty, man-child to a seasoned product creator, CEO, and Mensch. Other notable Apple biographies were Kahney’s “Tim Cook” and “Jony Ive”, both of who don’t get enough attention but are stars in their own right.

Slootman, Amp It Up. Slootman may be the best tech CEO after Nadella, Cook, and Zuck. This Dutchman has an unconventional background and style, but is the king of business software and has some great insights on running a top-notch company, with a focus on execution. Strong runner-ups for the CEO bios this year were Iger’s “The Ride of a Lifetime” (so good, he came back) and Dell’s “Play Nice But Win” (the rare founder who matures into a solid CEO).

Higgins, Power Play: Tesla, Elon Musk, and the Bet of the Century. Tesla, as an electric car company, is so technically difficult and ambitious that it shouldn’t exist. Musk took a whacky idea and made one of the most iconic and loved products of the last decade (despite his recent efforts at Twitter tarnishing Tesla and Musk). It’s a complex story of the car guys fighting with the software guys to build products that will one day roam on the moon and Mars.

Jackson and Delahanty, Eleven Rings: The Soul of Success. Phil Jackson is the former coach of the Chicago Bulls and the LA Lakers. He was a minister’s son who got pulled away into Eastern mysticism and major league basketball, who as a coach won the most NBA championships and had the 5th most wins when he retired. He also had difficult and prickly star players who fought often (Jordan and Pippen/Rodman, Kobe and Shaq), and he had to fuse them into high-performing teams.

Kolchin, American slavery, 1619-1877. This was the most painful book I read this year. The two greatest crimes of the American republic were the genocide against Native Americans and the systems of oppression via slavery, whose aftereffects still touch all American life. While I love the United States as a group of people, a culture, a constitutional republic, and a metaphysical entity with a set of evolving values, everyone needs to know the evil facts of its past, which this book presents well, unlike the historically inaccurate NYTimes 1619 project.

Roll, George Marshall: Defender of the Republic. Marshall was the Chief of Staff and General of the Army that oversaw the Allied coalition from DC in WW2, as the boss of General Eisenhower. He was a person of the highest ability and character, a leader who I place among Washington, Lincoln, and FDR as the best that the US can produce. His final great act as Secretary of State was the Marshall Plan to rebuild the European continent that he helped vanquish.

Ai Wei Wei, 1000 Years of Joys and Sorrows: A Memoir. Ai tells the story of his incredible life, from growing up in a camp during the Cultural revolution to making art in Beijing and New York, and fighting the Chinese authorities. A powerful passage: “Despite the ever-present threat of government intervention, I was now more than ready to resume my role as a provocateur. My war with power was a bit like an online game: each time I died, I came to life again. Power might use all kinds of tactics to attack me or monitor me, but I could turn those tactics into an advantage, through public activity and through creative ripostes, maintaining the role that they least wanted me to play—that of a mass-oriented activist and artist. Freedom of expression became a central meaning of my art, for personal freedom is the highest value that we can know.”

Whipple, The Gatekeepers: How the White House Chiefs of Staff Define Every Presidency. Most people don’t realize the second most powerful person in the US is generally the White House Chief of Staff, who I consider to be a co-consul running the republic with the President like in the old Roman days. Yes, the Speaker of the House, Senate Majority Leader, and the Chief Justice have official roles and their own rings of power, but the Chiefs act like COOs for the country, and are worth paying attention to. Another notable memoir for a civil servant was “Duty” by Bob Gates.

Perlroth, This Is How They Tell Me the World Ends: The Cyberweapons Arms Race. Our lives are all digital, and unfortunately, that world is not secure and is completely fragile in ways that few understand (unless you’ve taken courses in Operating Systems, Networking, Systems Security, etc., and studied hacking). Perlroth’s story of that underworld, and the cyber-arms race between world governments, reads like a dystopian novel. But it’s real, and it shows why we need projects like Bitcoin, Ethereum, Urbit, IPFS, and Uniswap as the testing grounds for something better.

Goldberg, The Path of Modern Yoga: The History of an Embodied Spiritual Practice. This is is the best history of modern global yoga, being generally balanced and thoughtful. It’s estimated that 300mm people in the world practice it, which traces its 20th-century heritage to a small group of people. The book pairs well with Singleton’s “Gurus of Yoga” and Mohan’s book “Krishnamacharya: His Life and Teachings”, about the pivotal teacher of Iyengar, Desikachar, Indra Devi, and Jois. For older yoga history, see Mallinson’s “Roots of Yoga” and Bryant’s “Yoga Sutras of Patañjali.”

Odell, Anna: The Biography. Wintour is arguably one of the most powerful people in global media, and certainly in fashion. She excelled due to her strong work ethic, impeccable taste, and skill at leading creative teams. She’s intensely loyal to her crew, picking and growing young talent with a fashion fund that backs new designers, along with Mets Costume Institute. Her weakness is that she’s made some mistakes over time: backing fit and leather; not having enough diversity in her models and staff, and; most importantly not taking the digital side seriously enough for a long time, and failing to prepare fast enough for a post-print world as Condé Nast fades away.

Van Der Kolk, The Body Keeps the Score: Brain, Mind, and Body in the Healing of Trauma. I was reading this at a cafe when a Berkeley undergrad struck up a conversation and told me it was one of the most read books of her class. It’s much more than a book by a psychiatrist and somehow goes deeper into the mysteries of human nature, describing how overwhelming experiences affect the development of the brain, mind, and body awareness, all of which are intertwined. We are all creatures forged on trauma, and some become stronger, others weaker.

Srinivasan, The Network State: How To Start a New Country. This is a wild book, at least comparable to Davidson’s “The Sovereign Individual” or Scott’s “Seeing like a State”, if not to older thinkers like Locke and Rousseau. The last 2 chapters are the best, and go into detail about the main idea: “A network state is a social network with a moral innovation, a sense of national consciousness, a recognized founder, a capacity for collective action, an in-person level of civility, an integrated cryptocurrency, a consensual government limited by a social smart contract, an archipelago of crowdfunded physical territories, a virtual capital, and an on-chain census that proves a large enough population, income, and real-estate footprint to attain a measure of diplomatic recognition.” My deeper thoughts on the Leviathan are here.

Hodges, Alan Turing: The Enigma. I read this book as a teenager in the 1990s when it seemed relevant in the PC Age, and loved it - Turing was a hero of mine. Revisiting it 25 years later, it’s even more relevant in the AI age, from the “On Computable Numbers” paper to his work on ACE and general computers, and his early work on AI. There’s also the controversy over his death: was it an accident, suicide, or murder? It pairs well with Copeland’s book “The Essential Turing” (which has the best intro and explanation to the “Computable Numbers” paper, ACE, and some of his AI papers and talks). The ACE work pairs well with reading Von Neumann’s EDVAC report, as together they laid the basis of modern computer engineering.

Gross-Loh and Puett, The Path: What Chinese Philosophers Can Teach Us About the Good Life. This book is based on one of the most popular classes at Harvard College. It provides sketches on Confucius, Mencius, Laozi, Zhuangzi, Xunzi, and others on relationships, decisions, influence, vitality, spontaneity, humanity, and more. It is concise and yet deep, with many wise views that are not often considered in the West.

Blattman, Why We Fight: The Roots of War and the Paths to Peace. War is hell, and it’s also deeply woven into the human psyche and society. So what does that say about us? Blattman delves into the root causes of war (unchecked interests, intangible incentives, uncertainty, commitment problems, misperception) and some paths to peace (interdependence, checks and balances, rules and enforcement, interventions). This book pairs well with Rudd’s sensible new book “The Avoidable War: The Dangers of a Catastrophic Conflict between the US and Xi Jinping’s China.”

Hargittal, Martians of Science: Five Physicists Who Changed the Twentieth Century. “The Martians” were a group of prominent Hungarian scientists of Jewish descent who emigrated from Europe to the United States around WWII. Leo Szilard was asked why there is no evidence of intelligent life beyond Earth despite the high probability of it existing, and he responded: “They are already here among us – they just call themselves Hungarians.” The main Martians were Von Karman, Von Neumann, Szilard, Teller, and Wigner, who gave the world modern aeronautical engineering, the modern computer and simulations, atomic fission theory, hydrogen bombs, and the theory of the atomic nucleus and the elementary particles.

FICTION

Tchaikovsky, Children of Time. A dark sci-fi novel about the remnants of humanity in ark ships traveling through space, who encounter a strange arachnid species after an experiment went wrong on a distant planet. The main plot twist comes from the winding passage of time, as the author plays it out on civilization and ark-ship scale over centuries.

Oliver, New and Selected Poems. Oliver is an under-appreciated New England poet, but beloved in the Bay Area yoga community for her poems on nature with imagery from her daily walks near her home. Some of her great poems are “Wild Geese”, “The Swan”, “The Summer Day”, “Invitation”, “Don’t Hesitate”, and “When Death Comes.” One poem starts: “Who made the world? / Who made the swan, and the black bear? / Who made the grasshopper? / This grasshopper, I mean— / the one who has flung herself out of the grass, / the one who is eating sugar out of my hand, / who is moving her jaws back and forth instead of up and down— / who is gazing around with her enormous and complicated eyes.”

Chiang, Stories of Your Life. Exhalation. Some thoughtful short stories that leave a chill. “Story of Your Life” gave us the movie “Arrival” - both are remarkable works of art, parallel and illuminating each other.

Ording, 99 Variations on a Proof. This is a strange book. The author picks a mundane cubic equation and finds 99 ways to prove it. Some are standard proofs (geometric, inductive, topological, algorithmic), but many are whacky and creative (blog, antiquity, tea, dialogue, hand-waving, screenplay, social media, etc). Besides showing how broad a mathematical proof could be, it tests the artificial line between fiction and non-fiction.

TECHNICAL

Devlin, The Millennium Problems: The Seven Greatest Unsolved Mathematical Puzzles Of Our Time. These are the seven most consequential math problems to be solved, per a committee in 2000 consisting of Alain Connes, Arthur Jaffe, Andrew Wiles, and Edward Witten, after consultation with other leading mathematicians. Only one has been solved, and six more would revolutionize their fields if they were (e.g. P=NP, the Riemann Hypothesis, Navier-Stokes, etc). It’s hard to explain them with relatively simple math and logic, and Devlin does that admirably. For the technical problem descriptions, see the CMI/AMS publication by Carlson, Jaffe, and Wiles called “The Millennium Prize Problems.”

Wells, Prime Numbers: The Most Mysterious Figures in Math. I’m obsessed with primes, the building blocks of all numbers and math. My sense is we only know some small percent of what we can know about them, and this book organizes centuries of prime research alphabetically, which gives it an irreverent randomness. Favorite sections were on abundant and amicable numbers, Andrica’s conjecture, formulae for primes, Gilbreath and Goldbach’s conjectures, permutable primes, and more.

Gauss, Disquisitiones Arithmeticae (trans. Clarke). The Prince of Mathematics wrote this book in Latin in 1801, and it’s one of the foundational texts of number theory. I’d call it one of the hardest books I’ve ever tried to read, though to be honest, it slayed me. I could follow the beginning of every chapter, and then like a wizard old Carl would take it up one or two notches and just flatten me. This book pairs well with Dunnington’s bio, “Carl Friedrich Gauss: Titan Of Science.” Disquisitiones is only for mathematical masochists, or if you want to feel extremely stupid.

Patterson and Hennessy, Computer Organization and Design. I never studied computer engineering and basics like instruction sets, arithmetic, CPUs, memory hierarchies, storage and I/O, etc. The closest I got were some popular books from Petzold and Danny Hillis, so I went straight to the best teachers to learn more. My favorite chapter was the one on GPUs.

PAPERS

Reimann, Manifolds on the hypotheses (Ueber die Hypothesen, welche der Geometrie zu Grunde liegen) (1851). This is a mind-bending paper that crosses the line between math and philosophy, has no equations, and yet extends Gauss’s theory of 2d surfaces to n-d space, where notions of distance, angles, and curvature can still be measured. It's relevant to gravity, general relativity, and modern machine learning (where neural networks learn manifolds in n-d space).

Metropolis and Ulam, The Monte Carlo Method (1949). A paper created to help model nuclear explosions that became important to the quantitative finance industry for decades (I learned the methods in school but never read the original paper or its history). Ulam was also the co-inventor of the Hydrogen bomb and [Ulam spiral](https://en.wikipedia.org/wiki/Ulam_spiral).

Alonso and KOE. Zero to Monero (Whitepaper)(2018). Monero may be the most interesting project in all of crypto, competing with Bitcoin and Ethereum, and a few others. It's basically Bitcoin with owner and transactional privacy. Monero is a standard one-dimensional distributed acyclic graph (DAG) cryptocurrency blockchain where transactions are based on elliptic curve cryptography using curve Ed25519.

Brynjolfsson et al. What Can Machines Learn and What Does It Mean for Occupations and the Economy? (2018). The authors investigate how machine learning (ML) will transform numerous occupations and industries at the task level. They apply a rubric evaluating task potential for ML and find that (“i) ML affects different occupations than earlier automation waves; (ii) most occupations include at least some SML tasks; (iii) few occupations are fully automatable using ML; and (iv) realizing the potential of ML usually requires a redesign of job task content.”

Aghion et al. Robots and automation (2021). A fascinating paper that uses a dataset of firms in France that suggests instead of destroying jobs, “automation has a positive direct effect on employment at the firm level” and could help workers, as firms with automation become much more productive, lower their quality-adjusted prices, and so benefit by higher market size and more employment.

Acemoglu, Harms of AI (2021). One of the world’s best research economists asking the right questions about where AI could go wrong, but with frustratingly simple and naive models to back up his hypotheses. I would like to see him take the brilliant intuition in this paper and gather datasets to prove or falsify them.

Lecun, A Path Towards Autonomous Machine Intelligence (2022). A paper by a leading ML researcher that aims to figure out what research directions could get us to the answer “How could machines learn as efficiently as humans and animals?”. He follows down the Minksy path of suggesting they need specific sub-models like a configurator, perception model, world model, cost model, actor model, and short-term memory (except Minsky suggested thousands of sub-models may be needed). This paper describes a roadmap for developing machines whose behavior is intrinsic rather than hard-wired or requiring supervision/rewards, with the key being self-supervised learning. My sense is that LeCun is right in the direction but wrong on the details, with reputations and fortunes to be made on getting the right details.

Habgood-Coote et al. Commentary on “Can a good philosophical contribution be made just by asking a question?”(2022). The paper itself is just the question, but thankfully the commentary goes into detail. It’s a good reminder that great questions are worth much more than their purported answers.

Manning, Human Language Understanding & Reasoning (2022). A clear and concise explanation about the “surprising breakthroughs in natural language processing through the use of simple artificial neural network computations, replicated on a very large scale and trained over exceedingly large amounts of data.” Basically the early stirrings of artificial general intelligence and foundation models that can connect sensory experiences, language, thinking, and feeling.

Hoffman et al. Training Compute-Optimal Large Language Models (the “Chinchilla’ paper) (2022). A stunning result from Google AI on how to better scale large language models. Keeping FLOPS constant, they find the ideal model size, # of tokens, and hints at dataset size. A huge accomplishment for massive model scaling.

Anil et al. On the Factory Floor: ML Engineering for Industrial-Scale Ads Recommendation Models (2022). The most important large-scale AI models in production in the world today are ads models, followed by search and content ranking models. Rarely do the ads engineers publish their thoughts, and this is a fun paper looking into what they do in this $500bn industry.

Fawzi et al. Discovering novel algorithms with AlphaTensor (2022). Matrix multiplication (matmul) is the heart of modern AI and ML - this paper uses AI to discover new matmul algorithms that are more efficient, and is an early landmark in AI pushing science while also making AI better (AI improving AI is the holy grail). A simple explanation.

Brown, Meta AI, et al. Human-level play in the game of Diplomacy by combining language models with strategic reasoning (2022). A simple yet also hard paper on how to combine a language model that negotiates with humans, a strategy module that makes decisions, and a filtering system that negates bad dialogue.

Chowdery et al. PaLM: Scaling Language Modeling with Pathways (2022). Google a 540-billion parameter, densely activated, Transformer language model on 6144 TPU v4 chips using Pathways, a new ML system that enables highly efficient training across multiple TPU Pods with near 100% utilization. This infrastructure achievement showed the benefits of model scaling by getting SOTA few-shot learning results on hundreds of language understanding and generation benchmarks. Basically one step closer to linguistic general intelligence. It uses many advances like SwiGLU Activation, Parallel formulations in transformer blocks, multi-query attention RoPE and Shared Input-output embeddings, and SentencePiece vocabulary.

IN SUMMA

And a final quote to end a tough year:

No single thing abides; but all things flow.

Fragment to fragment clings–the things thus grow

Until we know and name them. By degrees

They melt, and are no more the things we know.

Globed from the atoms falling slow or swift

I see the suns, I see the systems lift

Their forms; and even the systems and the suns

Shall go back slowly to the eternal drift.

(Lucretius, De Rerum Natura, Book 3).

PAST LISTS

Best Books of 2021 (Berkelians)

Best Books of 2020 (Reality)

Best Books of 2019 (Quantum Physics & Mystics)