1. Video transcript

4:05 Grow, you don't change the number of neurons, but you change the neural pathways.
4:09 And the way you train them is that you feed it some text, and you encourage it to predict given a block of block of text, maybe the you know the some section of a book you read on the internet, uh predict having seen the first 100 words what the next word is likely to be.
4:27 And as I said, it'll just guess at random to begin with, but every time you guess right, you strengthen that synaptic that neural pathway.
4:34 And every time it guesses wrong, you punish that neural pathway.

Who decides what is right or wrong?

4:40 And so slowly over time, you buils up some predictive capability for it to be able to predict what the next word is with better and better accuracy.
4:49 And once it can predict the next word, you can then just take that next word, assume it's the next word, and then it'll just just start talking to you.
4:57 And that's how the chatbots work that I described.
4:58 It's a slow process.
5:02 Once it's seen about a million words, you've trained it on a million words, it's still spewing stuff that's pretty much indistinguishable from gibberish.
5:08 Once uh you're at the tens of millions and hundreds of millions and billions, it can string together completely coherent sentences.
5:14 It knows the rules of grammar.

How does a chatbot knows the rules of grammar?

5:16 It puts sentences together, but they're not particularly refined sentences.
5:23 And by the time it's read the entire internet, which is tens of trillions of words, it can do uh it can converse intelligently on pretty much any topic.
5:30 That's called pre-training, and that's what most of what you do is just training it to predict the next word on the internet.
5:37 There's a second stage to the process called post-training, in which you essentially send it to finishing school.
5:44 When it comes off pre-training, it is just trained to predict what the next word it's going to be in its in its training corpus.
5:55 Uh and it is, you know, somewhat uncouth and uh definitely disobedient.
5:58 You need to send it to finishing school called post-training, where you train it only be polite and you train it to try and be helpful to the user rather than just predict what the next word would be.
6:09 That's called post-training.
6:12 Um and that in brief is how you make a large language model.
6:15 Uh and the train uh the modern language models with a few trillion parameters, it takes a huge amount of computer power to produce them.

6:36 models, uh which is what we're going to be talking about.
6:37 And we're going to specifically talking about them doing theoretical science.
6:43 Before I begin, I should explain, you know, this is sounds like computer science, how physics got involved.
6:46 Well, physicists have been involved in every step of this process, but one particular pretty striking way in which they're involved at the start of the decade that launched the entire modern LLM boom was through scaling laws.
7:01 So, physicists just uh love scaling laws.
7:02 That's our that's our bread and butter
7:04 You know, some of the scaling laws are simple.
7:05

7:47 realtionship all the way from the tiny mouse to the mighty elephant.
7:50 And this, you know, like many of the scaling laws discovered by physics, well, this was first an empirical discovery in which physicists were not involved.
8:00 And it actually has a rather curious feature, common to many of the scaling laws that physicists deal with.
8:05 And the curious feature is you might imagine the power output of an animal should be proportional to its mass, that every kilogram of your flesh, uh you know, burns metabolically at the same rate.
8:17 But, that's not true.
8:18 Actually, the larger you are, the less each kilogram burns.
8:21 This was an empirical discovery first uh by Kleiber, only much later understood by phsicists as a consequence of the fractal dimension of our vascular system.
8:34 All of this is merely meant to be warm-up to the idea that uh we love scaling laws and So, what we did when we found large language models was to make scaling laws for them.

Why?

8:45 And this scaling law is the most famous contribution of theoretical physicists to computer science and also started the modern LLM boom.

Sentence not clear.

it appropriately in size and training length, uh how much better performance do you get?
9:11 So, better performance is down, and what
10:19 uh neural networks, large language models, furiously ever since.
10:22 Uh and uh and that has characterized the last 6 years.
10:26 Um you know, I'm going to show you a lot of uh straight lines on graphs and kind of invite you to imagine what happens if those straight lines have really have no businesss being straight, but nevertheless are, and invite you to imagine what will happen if those lines continue to be straight for just a little bit longer.
10:42 That's going to be part of my part of my talk.
10:47 Um the original straight line on a graph, of course, was Moore's law.
10:51 Um Moore's law says that over time, it's you know, it's slightly cheating cuz the x-axis isn't some you know, physical parameter, it's just date.
11:00 Uh but over a century now, the price of compute has been dropping exponentially, making a linear line on a logarithmic plot.
11:10 Um and there's really no reason why that should be so, and yet that has been a feature of our world for the last century.
11:17 I'm mainly showing this to you to emphasize how little this has to do with the subject of today's talk.
11:27 The entire era in which today's talk is going to focus on is just going to be the time over the last 5 years.
11:34 That's just the very rightward edge of this.
11:36 Compute has really not improved that much in terms of the orders of magnitude we're going to see.
11:45 It is only
12:45 number one most important contributor to the growth of large language models and improvement of LLMs over the last decade has been algorithmic progress.
12:52 It's been human ingenuity of figuring out how to build these machiness better than we previously knew how to build these machines.
12:58 Huge amounts of thought has gone in to shearing away all the inefficiencies in the way we train them, to better understand these systems better, and so to improve them more rapidly.
13:14 And then what this plot is meant to persuade you is that there's no reason to stop.
13:18 At least there's no reason to stop based on economics or on chips.
13:24 So a good rule of thumb, you know, these things take so many flops, so much computation to run that a good rule of thumb, you know, you have to measure them in Avogadro's numbers.
13:33 You know, moles worth of flops.
13:37 Some ginormous number.
13:39 A good rule of thumb is that an Avogadro flop costs about a million dollars in today's in today's money to train.
13:45 And as we see since 2020, the size of these training runs has been growing as I said exponentially up from about half a million 1n 2020 to about a third of a billion last year.
13:57 The point really and you know, what this on the left is a graph where people investigated this closely is that those numbers are still pretty small.
14:07 US GDP is approaching 30 trillion dollars per year.
14:11 Global GDP is even bigger.
15:31 school math benchmark.
15:31 The creators of this benchmark, who are these fellows on the right, just went and scraped all sorts of high school math problems from the internet.
15:43 And then gave them to the large language model and said, "LLM. are you able to solve these problems?"
15:49 And here's a sampling of them.
15:51 Level one, "What is 11% of the number 11% of what number is 77?"
15:56 I think I could do that one.
16:02 All the way up to really reasonably challenging level five problems.
16:07 And so, before they gave them to LLMs, they first gave them to a human.
16:55 Computers have been able to calculate 11% of what number is 77 for a very long time.
19:46 So here we go.
19:46 We can sort of draw a little line on this plot here as we zoomed from preschool to elementary school to high school over the course of those years.
19:53 A good rule of thumb is that we're moving about four times as fast as a human student moves.
19:58 For every year that passes, we advance four years into future.
20:04 Okay. That's, you know, that that's that.
20:07 Let's go harder.
20:07 Well, first of all, let's just look at the hardest tranche of thes math problems,
20:20 math problems are so-called level five math problems.
20:22 Again, back just three years ago, were really not doing well at all.
20:24 It was it was pretty close to random guessing.
20:26 Um and over the course of

Go Back to Youtube Review
Back to my home page Index