What computer use lacks is grindability. So because websites have like bot detectors, and also it takes a tremendous amount of compute to run parallel rollouts. It's very hard to just run 1,000 parallel rollouts of the same checkout flow on Amazon because you'll get shut down by Andy Jassy

It also has to be very grindable in the sense that you have to be able to run lots of parallel rollouts against a deterministic and replayable simulator.

the famous fact here is that an H100 costs more to rent now than it did 3 years ago, even though we have much superior technology. And we have much more compute in the world because as models get smarter, the opportunity cost of compute gets higher.

If you look at the end of the last Cold War, Cold War too. Sadly, we were into the second one. But look at how world trade just takes off. It was the miracle of your lifetimes. It lifted hundreds of millions of people out of poverty.

And also for China, with its geographical setup as it is, uh, the seas are going to be open to it only in peacetime. In wartime, this has nothing to do with anyone doing anything to them, it's just facts. It is surrounded by loads of neighbors, shallow seas, lots of islands, that are completely blocked.

Which is, as we were saying, is sort of unique, a capital that is uniquely ill-suited to be complementary to the production of AI or the serving of AI or to robots or the kinds of goods that the rich will bid up the prices of.

Maybe for the first time, this is no longer true, right? So the famous fact here is that an H100 costs more to rent now than it did 3 years ago, even though we have much superior technology and we have much more compute in the world. Um, because as models get smarter, the opportunity cost of compute gets higher.

in the fullness of time, compute kind of is the single most important determinant on how things work... as you scale up energy and compute and parameters, intelligence will just fall out of that.

The amount of information per flop has been decreasing as you had to unroll 2 days worth of thinking in order to see if you even did something correctly.

Because of memory bandwidth cost. Yeah. So I actually don't see a very good path to solving that. Like the memory, the HPM is where it's at where it is. It's not getting hugely better.