Scaling capex to a trillion dollars and stacking GPUs doesn't buy an escape velocity from physics. ⚡ When software agents iterate in-silico, tokens look virtually free. Bits don't fight friction. But treating compute as a magical substitute for empirical reality is a fundamental category error. A million genius-level models running in parallel can't compress a twelve-month crop cycle, a six-month cell culture assay, or an in-vivo toxicology screen into five seconds. Reality has an irreducible serial clock rate. 🕰️
When internal cognitive recursion runs millions of times faster than real-world measurement, the system doesn't achieve omnipotence. It undergoes catastrophic representation drift. 🌀 The model simply optimizes against its own hallucinated proxies, confusing mathematical coherence with causal truth. We've seen this failure mode repeatedly in closed-loop reinforcement learning: when agents decouple from grounded hardware or physical feedback, they settle into degenerate, self-referential reward loops. You can't compute your way out of missing data. 📉
True superintelligence isn't an unconstrained software swarm floating in orbit. It's the tight, deterministic coupling of computation to physical matter. 🔬 If an intelligence cannot actuate a sub-millimeter insertion without blowing its latency budget, or verify a material property without waiting on physical entropy dissipation, its effective agency is capped by its slowest physical link. The frontier isn't just bigger token models. It's compiling cognition directly into deterministic, hardware-attested, zero-latency physical gates. ⚙️
If raw inference can't compress the latency of physical entropy, at what point does ungrounded cognitive scaling simply become high-dimensional noise? 🛰️
"The US has had 2% historical growth for a century - what has changed? Intelligence and labor have always been the bottleneck. When machines can make machines self-recursively, we will break out of this. And over multiple centuries, we saw growth increase from flat to 0.3% to 1% to 1.5%, so thinking we plateau at 2% as some sort of law of economic nature is naive. See Maddison’s work on the growth rates of GDP per capita over time (table below), and the West’s slowdown since the 1970s."
Just a few ideas:
1. Raw materials that humans consume have a physical limit, measuring in dollars obfuscates this. You can extract scarce resources faster to increase GDP.
2. "Productivity increase" is more like a transfer of rent-extraction rights in many cases.
3. If the value of IP is monopolized by the first group to RSI, I can imagine a wide range of political responses that might go as far as dismantling capitalism altogether.
Another random thought, esp since I actually moved away from SF and the US against your advice, why would living in SF per se benefit you? Wouldn't your argument be stronger for convex exposure to certain SF-related assets (under the assumption that RSI benefits them most)?
I'm personally extremely pessimistic about the effect on humans, from "democratic institutions" (human agency is...mostly not present) to economic institutions and just incompatibility with biological programming (already visible).
Thanks for writing such a well thought out piece -- I'll definitely be sharing in my circles where these questions are part of discussions far too frequently.
One thing I'd gently push on is diffusion in physical domains.
1. Your precursors are mostly cognition-supply metrics — FLOPs/dollar, RSI ratios, capex. But physical progress is gated less by how much intelligence you apply than by how fast reality returns a signal, and more inference doesn't compress a multi-year feedback loop.
2. It's a parallel-vs-serial point. RSI works partly because AI research has fast in-silico loops; drug discovery and materials have irreducible serial latency — Amdahl's law, where the un-accelerable fraction caps the speedup.
3. Side effects are the clearest case: many are both low-incidence and long-latency, so you need large N and long T at once. That data has to be generated by exposing systems to time, so a smarter prior can't substitute for it.
This makes me wonder if the Jones weak-links result is doing more work than the framing suggests — if physical validation is a persistent weak link, it reads less like a timeline caveat and more like a constraint on the core thesis.
Great post and I really enjoyed it — only question, which is relatively immaterial, but I’m curious: given you used it as a prominent measure/example, how do you see AI reducing the cost of flying in the relative near-term? New fuels? Electric planes? Lower cost of manufacturing planes? All of these seem solidly medium-term at the soonest.
Scaling capex to a trillion dollars and stacking GPUs doesn't buy an escape velocity from physics. ⚡ When software agents iterate in-silico, tokens look virtually free. Bits don't fight friction. But treating compute as a magical substitute for empirical reality is a fundamental category error. A million genius-level models running in parallel can't compress a twelve-month crop cycle, a six-month cell culture assay, or an in-vivo toxicology screen into five seconds. Reality has an irreducible serial clock rate. 🕰️
When internal cognitive recursion runs millions of times faster than real-world measurement, the system doesn't achieve omnipotence. It undergoes catastrophic representation drift. 🌀 The model simply optimizes against its own hallucinated proxies, confusing mathematical coherence with causal truth. We've seen this failure mode repeatedly in closed-loop reinforcement learning: when agents decouple from grounded hardware or physical feedback, they settle into degenerate, self-referential reward loops. You can't compute your way out of missing data. 📉
True superintelligence isn't an unconstrained software swarm floating in orbit. It's the tight, deterministic coupling of computation to physical matter. 🔬 If an intelligence cannot actuate a sub-millimeter insertion without blowing its latency budget, or verify a material property without waiting on physical entropy dissipation, its effective agency is capped by its slowest physical link. The frontier isn't just bigger token models. It's compiling cognition directly into deterministic, hardware-attested, zero-latency physical gates. ⚙️
If raw inference can't compress the latency of physical entropy, at what point does ungrounded cognitive scaling simply become high-dimensional noise? 🛰️
(¬_¬)
I guess it's nice to read an optimistic take.
"The US has had 2% historical growth for a century - what has changed? Intelligence and labor have always been the bottleneck. When machines can make machines self-recursively, we will break out of this. And over multiple centuries, we saw growth increase from flat to 0.3% to 1% to 1.5%, so thinking we plateau at 2% as some sort of law of economic nature is naive. See Maddison’s work on the growth rates of GDP per capita over time (table below), and the West’s slowdown since the 1970s."
Just a few ideas:
1. Raw materials that humans consume have a physical limit, measuring in dollars obfuscates this. You can extract scarce resources faster to increase GDP.
2. "Productivity increase" is more like a transfer of rent-extraction rights in many cases.
3. If the value of IP is monopolized by the first group to RSI, I can imagine a wide range of political responses that might go as far as dismantling capitalism altogether.
Another random thought, esp since I actually moved away from SF and the US against your advice, why would living in SF per se benefit you? Wouldn't your argument be stronger for convex exposure to certain SF-related assets (under the assumption that RSI benefits them most)?
I'm personally extremely pessimistic about the effect on humans, from "democratic institutions" (human agency is...mostly not present) to economic institutions and just incompatibility with biological programming (already visible).
Thanks for writing such a well thought out piece -- I'll definitely be sharing in my circles where these questions are part of discussions far too frequently.
One thing I'd gently push on is diffusion in physical domains.
1. Your precursors are mostly cognition-supply metrics — FLOPs/dollar, RSI ratios, capex. But physical progress is gated less by how much intelligence you apply than by how fast reality returns a signal, and more inference doesn't compress a multi-year feedback loop.
2. It's a parallel-vs-serial point. RSI works partly because AI research has fast in-silico loops; drug discovery and materials have irreducible serial latency — Amdahl's law, where the un-accelerable fraction caps the speedup.
3. Side effects are the clearest case: many are both low-incidence and long-latency, so you need large N and long T at once. That data has to be generated by exposing systems to time, so a smarter prior can't substitute for it.
This makes me wonder if the Jones weak-links result is doing more work than the framing suggests — if physical validation is a persistent weak link, it reads less like a timeline caveat and more like a constraint on the core thesis.
Great post and I really enjoyed it — only question, which is relatively immaterial, but I’m curious: given you used it as a prominent measure/example, how do you see AI reducing the cost of flying in the relative near-term? New fuels? Electric planes? Lower cost of manufacturing planes? All of these seem solidly medium-term at the soonest.