# Ilya Sutskever, NeurIPS 2024 Test of Time talk: 'Sequence to sequence learning with neural networks: what a decade' (13 December 2024)

- Speaker: Ilya Sutskever
- Event: NeurIPS 2024 Test of Time award talk, 'Sequence to sequence learning with neural networks: what a decade', Vancouver
- Date: 13 December 2024
- Length: About 24 min including Q&A
- Video URL: https://www.youtube.com/watch?v=1yvBqasHLZs ; official session recording: https://slideslive.com/39031672/test-of-time ; https://neurips.cc/virtual/2024/test-of-time/105032
- Slides: About 20 sparse slides, mostly one line or one chart each (the 2014 slide reprise, the 'age of pre-training' list, the mammal brain-to-body chart, 'what comes next').
- Transcript source: Whisper (faster-whisper, model medium) on the SlidesLive audio of the full Test of Time session (Goodfellow's GAN talk comes first). Cross-checked against the community transcript at github.com/shun-liang/readable-talks-transcriptions. Timestamps below are from the session audio; the host hands over at about 22:14 and Ilya's first transcribed line is at 22:34. Subtract about 22:20 to get the time in the standalone YouTube upload.
- Corrections: Corrections applied to the machine output, checked against the community transcript: 'intranet' to 'internet'; 'bit tensor' to 'Bittensor'; 'New Rips' to 'NeurIPS'; co-author names.
- Format: `[mm:ss]` is the time the line starts in the source recording. Machine transcript: expect small word errors; quotes used in the analysis were checked against the published text where one exists.

---

[00:14] Welcome to the Test of Time paper award session.
[00:20] Each year the program committee recognizes papers published 10 years earlier that have
[00:25] had a significant effect on shaping the field, standing the test of time.
[00:31] Ten years ago was NIPS at the time, 2014, in Montreal.
[00:37] The conference had a single oral track and the 2,000 or so attendees could all fit
[00:43] in one room. We're like half of the total crowd in this room. For the old-timers, 2014
[00:51] was also the first year that the workshops weren't held at a ski resort. There were
[00:55] about 1,600 submissions and 400 acceptances. Usually, one paper is selected for the award,
[01:04] but the program committee identified two papers deserving the honor given their transformational
[01:09] impact on the field.
[01:11] I guess they have to split the generous $0.00 prize.
[01:17] The first awarded paper is Generative Adversarial Nets.
[01:22] The authors are Ian Goodfellow, Jean-Poujet Abadi, Mehdi Mirza Bingju,
[01:29] David Ward Fairley, Chergil Ozer, Erin Courville, and Yashua Bengio.
[01:34] At least some of this, the authors are in the audience today.
[01:39] This paper has been cited over 85,000 times.
[01:44] It was a bold and creative approach to image synthesis
[01:48] that kick-started the interest in generative AI.
[01:51] And as the approach was refined, it became the first technique
[01:54] to produce crisp, realistic images.
[01:57] Here's a bit of trivia.
[01:59] The paper wasn't recognized at the time for an oral or a spotlight.
[02:03] It was just an ordinary poster.
[02:05] The benefit of hindsight, huh?
[02:08] Ian and David are going to present.
[02:10] Ian is going to start, and then David will take over.
[02:14] Ian is unfortunately remote.
[02:16] And before we start, at least for David,
[02:21] I'm asked to hand him the actual award.
[02:31] And so with that, I will turn this over to Ian.
[04:58] Thank you, Ian, for that important message
[05:03] and for the introduction.
[05:04] And thank you to the NeurIPS organizers
[05:06] for the recognition and opportunity
[05:09] to speak to you all today.
[05:11] And above all, thank you to the countless researchers
[05:14] who built upon this work in the past decade
[05:17] and have taken generative adversarial networks or GANs
[05:20] to places we never dreamed they'd go.
[05:24] Today, I'm gonna give you a personal account
[05:26] of how GANs came about,
[05:29] how they exploded in popularity
[05:31] and the lasting relevance of this work today.
[05:37] To understand how GANs became so influential,
[05:39] I think it helps to talk a little bit about
[05:41] the context in which they were developed.
[05:44] The year was 2014, and we were two years on
[05:47] from what I'll call the ImageNet moment,
[05:51] when Alex Krzyzewski, Ilya Sutskova, and Jeff Hinton
[05:54] demonstrated unprecedented object recognition performance
[05:57] with deep neural networks.
[05:59] As a PhD student in deep learning,
[06:00] you could feel the ground shift beneath your feet.
[06:04] It was an ear-defining moment for deep learning.
[06:07] Before, most research focused on unsupervised learning
[06:11] of feature extractors, which could then be leveraged
[06:13] or fine-tuned with supervised learning.
[06:18] After a renewed optimism took hold,
[06:22] and we all wanted to explore and expand the frontiers
[06:26] of what supervised deep network could do.
[06:30] Of particular interest to us today
[06:32] is the state of generative models in 2014
[06:35] and deep learning's relationship with generative models
[06:37] in that before era.
[06:40] In this community, generative model meant something subtly different than it does today.
[06:46] Generally it meant a probabilistic graphical model, usually the directed kind, that specified
[06:51] an often cartoonishly incorrect data generating process in terms of a set of latent variables.
[06:58] And that's not me throwing shade.
[06:59] The fact that it's cartoonishly incorrect is fine.
[07:03] All models are wrong, but some models are useful, as they say.
[07:07] These models became useful when you turned the crank of inference, latent Dirichlet allocation
[07:12] pictured top right of this slide, is not going to compose essays for you with its assumption
[07:17] that words are drawn at random, independently, conditional on a topic.
[07:22] But with the help of variational inference or Gibbs sampling, it will give you insights
[07:27] into a set of documents.
[07:29] In the deep learning world, by contrast, we saw generative modeling as a way to
[07:34] learn good representations of data.
[07:37] Flavors of restricted Boltzmann machine, an undirected graphical model, were a popular choice.
[07:43] Sparse coding, which can be viewed as map inference in a shallow directed model,
[07:49] underwent a renaissance of sorts. There was a probabilistic interpretation of certain kinds
[07:53] of autoencoders that was gaining steam. The variational autoencoder had just been
[07:58] proposed at ICLR. In either case, and this will sound strange to a subset of the room,
[08:05] generating models were not about generation. If you sampled from these
[08:11] models at all, which often required an unwieldy Markov chain in the case of
[08:16] models that deep learning practitioners were using, it was more for debugging or
[08:21] for curiosity's sake or for analysis than because you intended to use the
[08:25] samples for some downstream application. On that backdrop, we arrive at
[08:31] at the beginning of our story,
[08:34] 12 days before the New York's deadline in 2014,
[08:38] and a sizable fraction of the University of Montreal lab
[08:40] was at the pub, of course.
[08:45] We were having a send-off
[08:46] for several graduating students, including Ian.
[08:50] However, while you can take the grad student
[08:52] out of the lab,
[08:54] you can't always take the lab out of the grad student,
[08:57] and conversations quickly turn to research.
[09:00] There was ongoing work in the lab on generative models
[09:02] that were specifically designed as IID sample-generating
[09:05] machines, and it wasn't going well.
[09:11] Ian proposed an idea.
[09:14] What if we pitted a neural network sample generator
[09:16] against another neural network sample generator?
[09:20] No.
[09:21] What if we pitted a neural network sample generator
[09:23] against another neural network, this time classifier
[09:26] tasked with telling Rio from synthetic,
[09:30] and trained it all by gradient descent,
[09:32] backpropagated gradients from the loss
[09:35] through the discriminator in order to train the generator.
[09:39] Around the table, there was a lot of skepticism.
[09:41] The whole thing seemed degenerate and unstable.
[09:44] Ian, undeterred, obstinate, a little buzzed,
[09:51] went home that night and coded it up.
[09:55] The next day, he sent the entire lab
[09:57] a description of the idea and the very first GAN samples.
[10:02] With now 11 days remaining,
[10:05] a team quickly assembled to put together
[10:07] a nerve submission.
[10:11] The success of that effort, I think,
[10:14] boils down to a few elements.
[10:17] The first is the culture and intellectual environment
[10:19] at the lab.
[10:20] It was a friendly, vibrant place with a camaraderie
[10:25] that led to things like that pub outing.
[10:29] The steady flow of interns meant
[10:30] that there were a lot of capable hands around.
[10:34] Half of the student authors on this paper were interns.
[10:38] In particular, it was an incredibly lucky break
[10:40] that my co-author Jean was an intern at the time,
[10:44] as his game theory chops proved crucial
[10:46] to the proof in the paper.
[10:50] I'd also like to briefly acknowledge
[10:51] Theano, an ancestor of today's tools
[10:54] like PyTorch, TensorFlow, and Jax.
[10:58] Theano was an array mathematics library
[11:00] which allowed for the specification
[11:02] and differentiation of numerical programs
[11:05] and efficient execution of those programs on GPU.
[11:09] Also worthy of acknowledgement are the team
[11:11] that were developing and supporting it in those days.
[11:13] Frederic Bastien, Pascal Armblin, and Arnaud Beugeron
[11:18] Originally begun by James Berkstra and Olivier Breleux,
[11:22] Fianna was a linchpin of the entire lab's research
[11:25] and was an incredible force multiplier,
[11:27] flexible enough to easily accommodate
[11:29] wacky ideas like this one.
[11:31] And Yahshua, Yahshua deserves a lot of credit
[11:35] for having the long-term vision
[11:36] to support this kind of tool building work
[11:39] when it wasn't obvious this was a good thing to do.
[11:43] And this was particularly true
[11:44] in the more challenging research funding environment
[11:47] of the early 2010s and late aughts.
[11:53] After the submission, the team scattered as quickly
[11:56] as it had gathered.
[11:58] Ian departed for Google, as did I for an internship.
[12:02] Mehdi Mirza, while interning at Flickr
[12:05] with Simon Osindaro, pushed forward work
[12:08] on the conditional case, where generator and discriminator
[12:11] receive side information about the training case
[12:13] or desired sample, such as a class label.
[12:17] Ian published, and in my view,
[12:19] underappreciated workshop paper
[12:21] that made explicit some connections between GANs
[12:24] and noise contrast of estimation, among other things.
[12:28] The GAN paper was accepted at NURPS,
[12:30] and things were quiet for a little while.
[12:35] That is until LAPGAN, a paper from Remy Denton,
[12:38] Sumit Chintala, Arthur Slam, and Rob Fergus at NYU,
[12:44] scaling up GANs and class conditional GANs
[12:47] with a multi-scale cascade of GANs.
[12:51] It impressed the heck out of us,
[12:53] and even going from 28 by 28 MNIST samples
[12:56] to 48 by 48 Toronto face data set samples,
[13:01] we knew that things got harder as things got bigger.
[13:05] And this seemed like a very clever solution
[13:09] to that problem.
[13:12] A few months later, Alec Radford, Luke Metz,
[13:15] and Sumit Chintala published the result
[13:16] of a lot of very careful empirical architecture
[13:19] exploration that showcased a family of convolutional GANs
[13:23] that leveraged all the latest tricks
[13:24] like batch normalization,
[13:26] along with a lot of investigatory work.
[13:31] Particularly exciting, they demonstrated
[13:33] that the arithmetic properties found
[13:35] in were-to-vec embeddings could also be found in the learned
[13:38] latent space of GAN generators.
[13:42] And these two papers began a frankly unbelievable wave
[13:45] of interest and progress.
[13:47] I should point out that this figure, which was taken
[13:49] from the 2021 Stanford AI index report,
[13:52] understates things because it is not to scale.
[13:56] The last square image, for example,
[13:58] should be about 400 times the size of the first pixel
[14:01] for pixel.
[14:03] And while increased compute was certainly
[14:05] instrumental in this progress, it
[14:08] took a lot more than that to stabilize training
[14:09] at these scales.
[14:13] Resembling of today's text to image systems,
[14:16] Scott Reed and colleagues at U Michigan in 2018
[14:20] showcased, for the first time,
[14:22] highly realistic text-conditioned image
[14:24] generation.
[14:26] And this was just towards the end of the era
[14:28] where Ian and I remember people frequently asking,
[14:31] Who cares about good-looking samples?
[14:37] And while GANs are best known for image generation,
[14:39] they certainly have had an impact beyond that.
[14:44] And the most unexpected application
[14:45] that I am personally aware of, dentistry.
[14:50] It turns out that designing a dental crown
[14:52] is a highly labor-intensive process,
[14:54] and GANs can help produce better,
[14:56] more functional crowns more quickly and cheaply.
[14:59] Who knew?
[15:03] But I do think that there is something special
[15:05] about image generation as an application
[15:08] that led to GANs catching fire in the way that they did.
[15:12] While it's important to be rigorous
[15:14] and scientific in your work,
[15:15] and there's been a lot of fantastic work
[15:17] on quantitatively assessing the performance
[15:19] of models like GANs,
[15:21] there is something slightly magical
[15:23] and more than a little addictive
[15:25] about seeing your model dream up its own visual creations.
[15:29] It beats anxiously watching a loss curve bounce around.
[15:33] Of that, I can assure you,
[15:34] as someone who's done plenty of both.
[15:39] So where does that leave us today
[15:41] in the age of diffusion models and transformers?
[15:44] Well, it turns out that GANs are often still in the picture.
[15:49] The influential VQGAN method, pictured here,
[15:53] is actually not quite a GAN, but a hybrid method
[15:56] that augments the vector quantized
[15:57] variational autoencoders training criterion
[16:00] with an adversarial loss and a discriminator
[16:04] and a transformer prior.
[16:06] As another example, stable diffusion,
[16:09] one of the first household names in text-to-image synthesis
[16:12] also incorporates a discriminator and adversarial loss.
[16:18] But don't count GANs out just quite yet.
[16:22] GANs themselves, I mean.
[16:25] In the trade-off space of sample diversity
[16:27] versus quality versus speed,
[16:30] something that GANs excel at
[16:31] is training relatively compact generator networks
[16:35] that produce stunning samples
[16:36] and produce them very quickly.
[16:39] GigaGAN serves up 4K resolution text conditioned samples
[16:44] in under four seconds on last year's hardware
[16:48] through a combination of lower res generation
[16:50] and GAN-based super resolution, the latter pictured here.
[16:55] Just to remind you that this story started
[16:59] with an email full of 28 by 28 pixel grayscale digits.
[17:07] In terms of a lasting legacy, GANs
[17:09] have been one huge success story for scale,
[17:13] which I think it's arguable paved the way
[17:14] for other such stories.
[17:17] They showed the community how to operationalize
[17:19] perceptual quality using conceptually simple tools
[17:23] from supervised deep learning.
[17:25] And perhaps most significantly,
[17:27] they ushered in an era where the community took
[17:33] generative models seriously for synthesis tasks.
[17:36] While we were not the first to do so,
[17:39] we also demonstrated the effectiveness
[17:41] of turning the tools of supervised deep learning
[17:44] towards problems in unsupervised learning, which arguably foreshadowed the rise of contrastive
[17:50] self-supervised learning methods like CPC, AMDEM, and SIMCLR that were more directly
[17:57] concerned with learning representations, which, if you recall, is where we started.
[18:05] And with that, I'd like to thank you for listening and once again acknowledge
[18:07] the funding agencies and compute providers that made this work possible.
[18:11] Thank you.
[18:19] In your experiences since then,
[18:22] have you come across other teams today,
[18:26] other environments today, other research today
[18:29] that we might be talking about 10 years from now
[18:32] in the same way as Gantz?
[18:36] That is an excellent question
[18:39] and one that I don't think is very easy to answer
[18:45] is very easy to answer because in the view of the program committee, we weren't worth
[18:55] more than a poster.
[18:57] And in the view of the authors, frankly, we didn't think much of the paper either.
[19:02] We thought it was cool and interesting and we planned to pursue it ourselves, but we
[19:07] couldn't have imagined the influence that it would have.
[19:12] So I'm hesitant to pick and choose winners because hindsight is 20-20, foresight is anything
[19:25] but.
[19:26] I know you as not just an obviously accomplished researcher but somebody who takes the software
[19:30] side of research very seriously.
[19:33] And you made reference to the criticality of Seattle in giving rise to this work.
[19:40] then there have been several iterations of trying to, you know, come up with an ever
[19:45] better framework for general research.
[19:50] What would you say are some sort of wrong turns that you've noticed and right turns
[19:55] that you've noticed and what are some recurring mistakes or successful themes that you've
[20:01] observed in that iteration?
[20:03] Thank you.
[20:07] So I guess without wanting to, you know, point fingers and name names, one thing that I would
[20:19] remark on this topic is that Theano was very much designed as a generalist's tool.
[20:26] It wasn't even really a neural network library.
[20:29] It was a library for doing math, doing it quickly, and doing it on GPU.
[20:35] And I think that that was key to its success.
[20:39] I mentioned flexibility.
[20:41] You could bend the laws of reality with a tool like Theano.
[20:49] And I think that the most successful tools afford the same sorts of flexibility.
[20:57] And yeah, I think that trying to foresee what somebody will use your tool for is
[21:06] may be a fool there and what you should be aiming for is maximum flexibility and maximum
[21:16] span of usability.
[21:20] The second award paper is called Sequence to Sequence Learning with Neural Nets.
[21:27] All three authors are here, Ilya Setskevich, Oriel Vinyals, Kwak Lei.
[21:33] Just as the first paper was bold for its novelty, this work was bold for its ambition and scale.
[21:41] The paper has been cited almost 28,000 times.
[21:44] It offered an end-to-end sequence processing architecture that enabled the most impressive
[21:49] language model of its time.
[21:52] It showed that you could encode an entire sentence in an embedding, which surprised
[21:58] the heck out of me.
[22:00] This encoder-decoder framework served as a bridge that led the field from LSTM recurrent
[22:07] nets to modern foundation models.
[22:10] And Ilya's going to give the talk, but all three of these gentlemen are here.
[22:14] And I guess you have to fight over two sheets of paper here.
[22:34] Want to thank the organizers for choosing a paper for this award.
[22:39] It was very nice.
[22:43] And I also want to thank my incredible co-authors and collaborators, Oriol Vinyals and Quoc Le,
[22:51] who stood right before you a moment ago.
[22:55] And what you have here is an image, a screenshot from a similar talk 10 years ago at
[23:09] NeurIPS in 2014 in Montreal.
[23:13] And it was a much more innocent time.
[23:16] we are shown in the photos. This is the before. Here's the after, by the way. And now we've
[23:27] got my experienced, hopefully, visor. But here I'd like to talk a little bit about
[23:34] the work itself and maybe a 10-year retrospective on it. Because a lot of the things in this
[23:44] work were correct, but some not so much. And we can review them and we can see what
[23:51] happened and how it gently flowed to where we are today.
[23:56] So let's begin by talking about what we did.
[24:01] And the way we'll do it is by showing slides from the same talk 10 years ago.
[24:10] But the summary of what we did is the following three bullet points.
[24:14] It's an autoregressive model trained on text.
[24:17] It's a large neural network and it's a large data set.
[24:21] And that's it.
[24:22] Now let's dive into the details a little bit more.
[24:27] So this was a slide ten years ago.
[24:30] Not too bad.
[24:31] The deep learning hypothesis.
[24:34] And what we said here is that if you have a large neural network with ten layers,
[24:41] then it can do anything that a human being can do in a fraction of a second.
[24:47] Why did we have this emphasis on things that human beings can do in a fraction
[24:52] of a second?
[24:53] Why this thing specifically?
[24:55] Well, if you believe the deep learning dogma, so to say, that artificial neurons and biological
[25:02] neurons are similar, or at least not too different, and you believe that real neurons
[25:07] are slow, then anything that we can do quickly, by we I mean human beings, I even mean
[25:13] just one human in the entire world, if there is one human in the entire world that
[25:17] can do some task in a fraction of a second, then a 10 layer neural network can do
[25:21] it too, right?
[25:22] It follows.
[25:23] their connections and you embed them inside your neural net, the artificial one.
[25:28] So this was the motivation.
[25:30] Anything that a human being can do in a fraction of a second, a big 10 layer neural network
[25:33] can do too.
[25:34] We focused on 10 layer neural networks because this was the neural networks we knew
[25:39] how to train back in the day.
[25:42] If you could go beyond in your layers somehow, then you could do more.
[25:47] But back then we could only do 10 layers, which is why we emphasized whatever human
[25:52] beings can do in a fraction of a second. A different slide from the talk. A slide which
[25:58] says our main idea. And you may be able to recognize two things or at least one thing.
[26:03] You might be able to recognize that something autoregressive is going on here. What is
[26:07] it saying really? What does this slide really say? This slide says that if you
[26:12] have an autoregressive model and it predicts the next token well enough, then it will
[26:18] in fact, grab and capture and grasp the correct distribution over whatever, over sequences
[26:24] that come next.
[26:26] And this was a relatively new thing.
[26:27] It wasn't literally the first ever autoregressive neural network, but I would argue it was
[26:34] the first autoregressive neural network where we really believed that if you train
[26:39] it really well, then you will get whatever you want.
[26:42] In our case, back then was the humble, today humble, then incredibly audacious task
[26:47] of translation. Now I'm going to show you some ancient history that many of you
[26:52] may have never seen before. It's called the LSTM. To those unfamiliar, an LSTM is
[27:00] the things that poor deep learning researchers did before transformers and
[27:05] it's basically a ResNet but rotated at 90 degrees. So that's an LSTM and it
[27:13] came before it's like it's like kind of like a slightly more complicated
[27:18] ResNet you can see there is your integrator which is now called the
[27:23] residual stream but you've got some multiplication going on. It's a little
[27:27] bit more complicated but that's what we did. It was a ResNet rotated 90 degrees.
[27:32] Another cool feature from that old talk that I want to highlight is that
[27:37] we used parallelization but not just any parallelization. We used
[27:42] pipelining as witnessed by this one layer per GPU. Was it wise to pipeline? As we now
[27:51] know, pipelining is not wise. But we were not as wise back then. So we used
[27:56] that and we got a 3.5x speed up using eight GPUs. And the conclusion slide in
[28:06] some sense, the conclusion slide from the talk from back then, is the most
[28:10] important slide because it's spelled out what could arguably be the
[28:15] beginning of the scaling hypothesis, right? That if you have a very big data set and you
[28:20] train a very big neural network, then success is guaranteed. And one can argue, if one
[28:26] is charitable, that this indeed has been what's been happening.
[28:33] I want to mention one other idea. And this is, I claim, the idea that truly stood the
[28:39] test of time. It's the core idea of deep learning itself. It's the idea of connectionism.
[28:45] the idea that if you allow yourself to believe
[28:50] that an artificial neuron is kind of sort of
[28:56] like a biological neuron, right, if you believe
[29:00] that one is kind of sort of like the other,
[29:05] then it gives you the confidence to believe
[29:08] that very large neural networks, they don't need
[29:11] to be literally human brain scale,
[29:12] they might be a little bit smaller,
[29:14] but you could configure them to do pretty much all the things
[29:20] that we do, human beings.
[29:22] There's still a difference.
[29:23] Oh, I forgot to say then.
[29:25] There's still a difference because the human brain also
[29:28] figures out how to reconfigure itself.
[29:31] Whereas we are using the best learning algorithms
[29:33] that we have which require as many data points
[29:38] as there are parameters.
[29:40] Human beings are still better in this regard.
[29:43] But what this led, so I claim, arguably,
[29:49] is to the age of pre-training.
[29:51] And the age of pre-training is what we might say,
[29:54] the GPT-2 model, the GPT-3 model, the scaling laws.
[29:59] And I want to specifically call
[30:00] out my former collaborators, Alec Radford,
[30:04] also Jared Kaplan, Dario Moday,
[30:09] for really making this work.
[30:12] But that led to the age of pre-training.
[30:13] And this is what's been the driver of all of progress,
[30:17] all the progress that we see today.
[30:19] Extra-large neural networks.
[30:22] Extra-ordinary large neural networks.
[30:24] train on huge data sets.
[30:28] But pre-training, as we know it, will unquestionably end.
[30:32] Pre-training will end.
[30:34] Why will it end?
[30:36] Because while compute is growing through better hardware,
[30:40] better algorithms, and larger clusters, right,
[30:44] all those things keep increasing your compute.
[30:48] All these things keep increasing your compute.
[30:50] The data is not growing
[30:53] because we have but one internet.
[30:57] We have but one internet.
[30:59] You could even say, you can even go as far as to say
[31:02] that data is the fossil fuel of AI.
[31:06] It was like created somehow, and now we use it,
[31:12] and we've achieved peak data, and there'll be no more.
[31:17] We have to deal with the data that we have.
[31:20] Now it still let us go quite far,
[31:22] but this is, there's only one internet.
[31:29] So here, I'll take a bit of liberty to speculate
[31:33] about what comes next.
[31:35] Actually, I don't need to speculate,
[31:36] because many people are speculating too,
[31:38] and I'll mention their speculations.
[31:41] You may have heard the phrase agents.
[31:43] It's common, and I'm sure that eventually
[31:47] something will happen, but people feel
[31:49] like something agents is the future.
[31:53] More concretely, but also a little bit vaguely,
[31:55] synthetic data, but what does synthetic data mean?
[31:59] Figuring this out is a big challenge.
[32:03] And I'm sure that different people have all kinds
[32:05] of interesting progress there.
[32:07] And an inference time compute,
[32:09] or maybe what's been most recently,
[32:11] most vividly seen in O1, the O1 model,
[32:14] these are all examples of things
[32:16] of people trying to figure out
[32:19] what to do after pre-training.
[32:21] And those are all very good things to do.
[32:25] I want to mention one other example from biology,
[32:29] which I think is really cool.
[32:32] And the example is this.
[32:34] So about many, many years ago at this conference also,
[32:39] I saw a talk where someone presented this graph,
[32:42] but the graph showed the relationship
[32:45] between the size of the body of a mammal
[32:53] and the size of their brain.
[32:55] In this case, it's in mass.
[32:57] And that talk, I remember vividly,
[33:00] they were saying, look, it's in biology,
[33:02] everything is so messy, but here you have one rare example
[33:05] where there is a very tight relationship
[33:08] between the size of the body of the animal and their brain.
[33:11] And totally randomly, I became curious at this graph.
[33:15] And one of the early, so I went to Google
[33:19] to do research to look for this graph.
[33:21] And one of the images in Google Images was this.
[33:24] And the interesting thing in this image is,
[33:29] you see like, I don't know, is the mouse working?
[33:31] Oh yeah, the mouse is working great.
[33:33] So you've got this mammals, right?
[33:35] All the different mammals.
[33:38] Then you've got non-human primates.
[33:40] It's basically the same thing.
[33:42] But then you've got the hominids.
[33:44] And to my knowledge, hominids are like close relatives
[33:49] to the humans in evolution.
[33:52] Like the Neanderthals, there's a bunch of them.
[33:58] Like it's called homo habilis maybe.
[34:01] There's a whole bunch and they're all here.
[34:04] And what's interesting is that they have a different slope
[34:08] on their brain-to-body scaling exponent.
[34:13] So that's pretty cool.
[34:15] What that means is that there is a precedent.
[34:18] There is an example of biology figuring
[34:23] out some kind of different scaling.
[34:26] Something clearly is different.
[34:29] So I think that is cool.
[34:30] And by the way, I want to highlight.
[34:32] This x-axis is log scale.
[34:34] You see this is 100.
[34:35] This is 1,000, 10,000, 100,000, and likewise, in grams,
[34:40] one gram, 10 gram, 100 grams, 1,000 grams.
[34:46] So it is possible for things to be different.
[34:50] The things that we are doing,
[34:51] the things that we've been scaling so far
[34:52] is actually the first thing
[34:54] that we figured out how to scale.
[34:56] And without doubt, the field,
[34:59] everyone who's working here will figure out what to do.
[35:03] But I want to talk here,
[35:05] I want to take a few minutes
[35:07] and speculate about the longer term,
[35:10] the longer term, where are we all headed?
[35:12] Right, we're making all this progress.
[35:14] It's astounding progress.
[35:16] It's really, I mean, those of you who've been
[35:21] in the field 10 years ago and you remember
[35:23] just how incapable everything has been,
[35:28] like, yes, you can say, even if you kind of say,
[35:30] of course, deep learning, still to see it
[35:33] is just unbelievable.
[35:35] It's completely, I can't convey that feeling to you.
[35:40] You know, if you joined the field in the last two years,
[35:42] then, of course, you speak to computers,
[35:44] and they talk back to you, and they disagree,
[35:46] and that's what computers are.
[35:49] But that doesn't always been the case.
[35:51] But I want to talk to you a little bit
[35:53] about super intelligence, just a bit,
[35:56] because that is obviously where this field is headed.
[36:00] This is obviously what's being built here.
[36:03] And the thing about super intelligence
[36:05] is that it will be different qualitatively
[36:08] from what we have.
[36:10] And my goal in the next minute
[36:12] to try to give you some concrete intuition
[36:16] of how it will be different,
[36:18] so that you yourself could reason about it.
[36:22] So right now, we have our incredible language models
[36:24] and their unbelievable chatbots,
[36:25] and they can even do things,
[36:26] but they're also kind of strangely unreliable,
[36:30] and they get confused
[36:32] while also having dramatically superhuman performance
[36:36] on evals, so it's really unclear how to reconcile this.
[36:40] But eventually, sooner or later,
[36:44] the following will be achieved.
[36:46] Those systems are actually going to be
[36:48] agentic in real ways.
[36:49] Whereas right now, the systems are not agents
[36:52] in any meaningful sense.
[36:54] Just very, that might be too strong.
[36:56] They're very, very slightly agentic.
[36:58] Just beginning.
[37:00] It will actually reason.
[37:02] And by the way, I wanna mention something about reasoning
[37:05] is that a system that reasons,
[37:07] the more it reasons, the more unpredictable it becomes.
[37:11] The more it reasons, the more unpredictable it becomes.
[37:14] All the deep learning that we've been used to
[37:16] is very predictable because if you've been working
[37:18] on replicating human intuition essentially,
[37:21] it's like the gut feel.
[37:23] If you come back to the 0.1 second reaction time,
[37:26] what kind of processing we do in our brains?
[37:30] Well, it's our intuition.
[37:33] So we've endowed our AIs with some of that intuition.
[37:37] But reasoning, and you're seeing some early signs
[37:39] of that, reasoning is unpredictable.
[37:41] And one reason to see that is because the chess AIs,
[37:44] the really good ones, are unpredictable
[37:47] to the best human chess players.
[37:49] So we will have to be dealing with AI systems
[37:53] that are incredibly unpredictable.
[37:56] They will understand things from limited data.
[37:59] They will not get confused,
[38:00] all the things which are really big limitations.
[38:03] I'm not saying how, by the way,
[38:04] and I'm not saying when, I'm saying that it will.
[38:09] And when all those things will happen
[38:11] together with self-awareness, because why not?
[38:14] Self-awareness is useful.
[38:16] It is part, you ourselves are parts of our own world models.
[38:21] When all those things come together,
[38:22] we will have systems of radically different qualities
[38:26] and properties that exist today.
[38:28] And of course, they will have incredible
[38:30] and amazing capabilities,
[38:31] but the kind of issues that come up
[38:33] with systems like this,
[38:34] and I'll just leave it as an exercise just to imagine,
[38:39] it's very different from what we're used to.
[38:42] And I would say that it's definitely also impossible
[38:48] to predict the future.
[38:49] Really, all kinds of stuff is possible.
[38:54] But on this uplifting note, I will conclude.
[38:58] Thank you so much.
[39:08] Now in 2024, are there other biological structures
[39:12] that are part of human cognition
[39:14] that you think are worth exploring in a similar way
[39:18] or that you're interested in anyway?
[39:20] The way I'd answer this question is that
[39:24] if you are or someone is a person
[39:27] who has a specific insight about,
[39:29] hey, we are all being extremely silly
[39:33] because clearly the brain does something and we are not,
[39:37] and that's something that can be done,
[39:38] they should pursue it.
[39:40] I personally don't,
[39:45] well, depends on the level of abstraction
[39:48] you're looking at.
[39:49] Maybe I'll answer it this way.
[39:50] Like there's been a lot of desire
[39:52] to make biologically inspired AI.
[39:55] And you could argue on some level
[39:57] that biologically inspired AI is incredibly successful,
[39:59] which is all of deep learning is biologically inspired AI.
[40:02] But on the other hand, the biological inspiration
[40:05] was very, very, very modest.
[40:06] It's like, let's use neurons.
[40:08] This is the full extent of the biological inspiration.
[40:12] Let's use neurons.
[40:14] And more detailed biological inspiration
[40:16] has been very hard to come by,
[40:19] but I wouldn't rule it out.
[40:20] I think if someone has a special insight,
[40:23] they might be able to see something,
[40:25] and that would be useful.
[40:26] I have a question for you about sort of auto-correct.
[40:30] So here's the question.
[40:32] You mentioned reasoning as being one of the core aspects
[40:37] of maybe the modeling in the future
[40:40] and maybe a differentiator.
[40:43] What we saw in some of the poster sessions
[40:45] is that hallucinations in today's models,
[40:48] the way we're analyzing, maybe you correct me,
[40:51] you're the expert on this,
[40:52] but the way we're analyzing whether a model
[40:54] is hallucinating today without,
[40:56] because we know of the dangers of models
[40:58] not being able to reason,
[41:00] that we're using a statistical analysis,
[41:02] let's say some amount of standard deviations
[41:05] or whatever away from the mean.
[41:07] In the future, do you think that a model given reasoning
[41:13] will be able to correct itself,
[41:15] sort of auto-correct itself,
[41:17] and that will be a core feature of future models
[41:20] so that there won't be as many hallucinations
[41:22] because the model will recognize when,
[41:25] maybe that's too esoteric of a question,
[41:27] but the model will be able to reason
[41:28] and understand when a hallucination is occurring.
[41:30] Does the question make sense?
[41:32] Yes, and the answer is also yes.
[41:34] I think what you described is extremely highly plausible.
[41:39] I mean, you should check, I mean, for, yeah.
[41:42] I wouldn't rule out that it might already be happening
[41:45] with some of the, you know,
[41:47] early reasoning models of today, I don't know.
[41:50] but longer term, why not?
[41:53] Yeah, I mean, it's part of like Microsoft Word,
[41:55] like auto-correct, it's a core feature.
[41:58] Yeah, I just, I mean, I think calling it auto-correct
[42:02] is really doing it a disservice.
[42:03] I think you are, when you say auto-correct,
[42:06] you evoke like, it's far grander than auto-correct,
[42:11] but this point aside, the answer is yes.
[42:14] Hi, Ilya, I loved the ending,
[42:17] mysteriously leaving out, do they replace us?
[42:22] Or are they superior?
[42:23] Do they need rights?
[42:25] It's a new species of Homo sapien spawned intelligence.
[42:30] So maybe they need, I mean, I think the RL guy thinks
[42:34] they think that we need rights for these things.
[42:38] I have an unrelated question to that.
[42:41] How do you create the right incentive mechanisms
[42:44] for humanity to actually create it in a way
[42:48] that gives it the freedoms that we have as homo sapiens.
[42:56] You know, I feel like this, in some sense,
[42:59] those are the kind of questions
[43:01] that people should be reflecting on more.
[43:06] But to your question
[43:08] about what incentive structure should we create,
[43:11] I don't feel that I know.
[43:14] I don't feel confident answering questions like this
[43:16] because it's like you're talking about creating
[43:20] some kind of a top-down structure government thing.
[43:24] I don't know.
[43:25] It could be a cryptocurrency, too.
[43:28] I mean, there's Bittensor, you know, there's things.
[43:32] I don't feel like I am the right person
[43:35] to comment on cryptocurrency.
[43:37] But you know, there is a chance, by the way,
[43:44] what you're describing will happen,
[43:47] that indeed we will have, you know, in some sense,
[43:49] it's not a bad end result if you have AIs
[43:54] and all they want is to coexist with us
[43:58] and also just to have rights.
[44:00] Maybe that will be fine.
[44:01] It's, but I don't know.
[44:04] I mean, I think things are so incredibly unpredictable.
[44:07] I hesitate to comment, but I encourage the speculation.
[44:10] Thank you.
[44:11] And yeah, thank you for the talk.
[44:13] It's really awesome.
[44:16] Hiya there.
[44:17] Thank you for the great talk.
[44:19] My name is Shalev Lifshitz from University of Toronto,
[44:21] working with Sheila.
[44:23] Thanks for all the work you've done.
[44:25] I wanted to ask, do you think LLMs generalize multi-hop
[44:30] reasoning out of distribution?
[44:35] So OK, the question assumes that the answer is yes or no.
[44:40] But the question should not be answered with yes or no.
[44:44] Because what does it mean out of distribution
[44:47] generalization?
[44:48] What does it mean?
[44:49] What does it mean in distribution?
[44:51] And what does it mean out of distribution?
[44:53] Because it's a test of time talk, I'll say that long,
[44:57] long ago before people were using deep learning,
[45:02] they were using things like string matching and grams.
[45:06] For machine translation,
[45:07] people were using statistical phrase tables.
[45:10] Can you imagine?
[45:12] They had tens of thousands of code of complexity which was,
[45:16] I mean, it was truly unfathomable.
[45:20] And back then, generalization meant,
[45:24] is it literally not in the same phrasing as in the data set?
[45:28] Now, we may say, well, sure, my model achieves this high score
[45:33] on, I don't know, math competitions, but maybe the math,
[45:37] maybe some discussion in some forum on the internet was
[45:40] about the same ideas, and therefore, it's memorized.
[45:43] Well, okay, you could say maybe it's in distribution,
[45:45] maybe it's memorization, but I also think that our standards
[45:49] what counts as generalization have increased really quite substantially, dramatically,
[45:54] unimaginably if you keep track. And so I think the answer is to some degree probably not as well
[46:04] as human beings. I think it is true that human beings generalize much better, but at the same
[46:10] time they definitely generalize out of distribution to some degree. I think it's a useful
[46:17] tautological lens.