Transcript of YouTube Video: NVIDIA CEO Jensen Huang Keynote at COMPUTEX 2024

all right let's get

started good take good

take oh

je okay you guys ready yeah yeah

everybody thinks we make

gpus I didn't video is so much more than

[Music]

that this whole key you know it's going

to be about that okay so we'll start at

the top examples of the use cases and

then seeing it in action that's kind of

the flow it's such a compelling story

I'm super nervous about this just got to

get in the Rhythm we're 2 weeks away

from it you guys can go really really

makeing great we should take a swing at

it yeah that's the plan we need to get

daily status on that animation can you

mute cuz I hear myself sorry what's the

drop date for all the videos it needs to

be done on the

28th did you get all

that safe travels everybody super

excited to see everyone see you guys

soon okay bye

bye we're basically moving as fast as

the world can absorb technology so we've

got to Le problem ourselves

[Music]

isn't now the spine you just have to

figure out a way to make it pop you know

what I'm saying

yeah you know what I'm saying you want

to yeah that kind of

[Music]

[Applause]

[Music]

thing thank you I'm super late let's go

[Music]

W please welcome to the stage Nvidia

founder and CEO Jen w

[Music]

[Applause]

[Music]

T I am very happy to be

back thank you n for letting us use your

Stadium the last time I was

here I received a degree from

[Applause]

n and I I gave the Run don't walk

speech and today we have a lot to cover

so I cannot walk I must

run we have a lot to cover I have many

things to tell you I'm very happy to be

here in

Taiwan Taiwan is the home of our

treasured

Partners this is in

fact where everything Nvidia does begins

our partners and ourselves take it to

the

world

Taiwan and our

partnership has created the world's AI

infrastructure

today I want to talk to you about

several

things

one what is happening and the meaning of

the work that we do together what what

is generative

AI what is its impact on our industry

and on every

industry a blueprint for how we will go

forward and engage this incredible

opportunity and what's coming

next generative Ai and its impact our

blueprint and what comes next

these are really really exciting

times a

restart of our computer industry an

industry

that you have forged an industry that

you have

created and now you're

prepared for the next major

Journey but before we

start Nvidia lives at the

intersection of computer graphics

simulations and artificial

intelligence this is our

soul everything that I show you

today is

simulation it's math it's science it's

computer science it's amazing computer

architecture none of it's

animated and it's all

homemade this is invidious soul and we

put it all into this virtual world we

called Omniverse please enjoy

[Music]

[Music]

[Music]

oh

[Music]

[Music]

[Music]

go

[Music]

[Applause]

[Applause]

I want to speak to you in Chinese but I

have so much to tell you I have to think

too

hard to speak

Chinese so I have to speak to you in

English at the foundation of everything

that you saw was two fundamental

Technologies accelerated Computing and

artificial intelligence running inside

the

Omniverse those two technologies those

two fundamental forces of

computing are going to reshape the

computer

industry the computer

industry is now some 60 years

old in a lot of ways everything that we

do today was invented the year after my

birth in

1964 the IBM system 360 introduced

central processing units general purpose

Computing the separation of hardware and

software through an operating system

multitasking IO subsystems

dma all kinds of technologies that we

use today architectural

compatibility backwards compatibility

family compatibility all of the things

that we know today about Computing

largely described in

1964 of course the PC Revolution

democratized Computing and put it in the

hands and the houses of everybody and

then off in 2007 the iPhone introduced

mobile Computing and put the computer in

our

pocket ever since everything is

connected and running all the time

through the mobile

Cloud this last 60 years we saw

several just several not that many

actually two or three major technology

shifts two or

three tectonic shifts in Computing where

everything changed and we're about to

see that happen again there are two

fundamental things that are

happening the first is that the

processor the engine by which the

computer industry runs on the central

processing unit the performance scaling

has slowed tremendously and yet the

amount of computation we have to do is

still

doubling very quickly

exponentially if processing requirement

if the data that is that we need to

process continues to scale exponentially

but performance does not we will

experience computation inflation and in

fact we're seeing that right now as we

speak the amount of data center power

that's used all over the world is

growing quite

substantially the cost of computing is

growing we are seeing computation

inflation

this of course cannot

continue the data is going to continue

to increase

exponentially and CPU performance

scaling will never return there is a

better way for almost two decades now

we've been working on accelerated

Computing Cuda augments a CPU offloads

and accelerates the work that A

specialized processor can do much much

better in fact the performance is so

extraordinary that it is very clear now

as CPU scaling has slowed and event

substantially stopped we should

accelerate everything I predict that

every application that is processing

intensive will be accelerated and surely

every data center will be accelerated in

the near future now accelerated

Computing is very sensible it's very

common sense if you take a look at an

application and and here the 100t means

100 units of time it could be 100

seconds it could be 100 hours and in

many cases as you know we're now working

on artificial intelligence applications

that run for a 100

days the one t is code that is requires

sequential processing where

single-threaded CPUs are really quite

essential operating systems control

logic really essential to have one

instruction executed after another

instruction however there are many

algorithms computer Graphics is one that

you can operate completely in parallel

computer Graphics image processing

physics simulations combinatorial

optimizations graph processing database

processing and of course the very famous

linear algebra of deep learning there

are many types of algorithms that are

very conducive to acceleration

through parallel processing so we

invented an architecture to to do

that by adding the GPU to the

CPU the specialized processor can take

something that takes a great deal of

time and accelerate it down to something

that is in incredibly fast and because

the two processors can work side by side

they're both autonomous and they're both

separate independent that is we could

accelerate what used to take a 100 units

of time down to one unit of time well

the speed up is incredible it almost

sounds

unbelievable it almost sounds

unbelievable but today I'll demonstrate

many examples for

you the benefit is quite extraordinary a

100 times speed up but you only increase

the power by about a factor of three and

you increase the cost by only about

50% we do this all the time in the PC

industry we add a GPU a $500 GPU gForce

GPU to a $1,000 PC and the performance

increases tremendously we do this in a

data center a billion dooll data center

we add $500 million worth of gpus and

all of a sudden it becomes an AI

Factory this is happening all over the

world today well the savings are quite

EX extraordinary you're getting 60 times

performance per dollar 100 times speed

up you only increase your power by 3x

100 times speed up you only increase

your cost by 1.5x the savings are

incredible the savings are measured in

dollars it is very clear that many many

companies spend hundreds of millions of

dollars processing data in the the cloud

if it was accelerated it is not

unexpected that you could save hundreds

of millions of dollars now why is that

well the reason for that is very clear

we've been experiencing inflation for so

long in general purpose Computing now

that we finally came to we finally

determined to accelerate there's an

enormous amount of captured loss that we

can now regain a great deal of captured

retained waste that we can now relieve

out of the system and that will

translate into savings Savings in money

Savings in energy and that's the reason

why you've heard me

say the more you buy the more you

[Applause]

save and now I've shown you the

mathematics

it is not accurate but it is

correct okay that's called CEO

math CEO math is not accurate but it is

correct the more you buy the more you

save well accelerated Computing does

deliver extraordinary results but it is

not easy why is it that it saves so much

money but people haven't done it for so

long the reason for that is because it's

incredibly

hard there is no such thing as a

software that you can just run through a

c compiler and all of a

sudden that application runs a 100 times

faster that is not even logical if it

was possible to do that they would have

just changed the CPU to do that you in

fact have to rewrite the software that's

the hard part the software has to be

completely Rewritten so that you could

reactor

reexpress the algorithms that was

written on a CPU so that it could be

accelerated offloaded accelerated and

run in parallel that computer science

exercise is insanely hard well we've

made it easy for the world over the last

20 years of course the very famous CNN

the Deep learning library that processes

neural networks we have a library for AI

physics that you could use for fluid

dynamics and many other applications

where the neural network has to obey the

laws of physics we have a great new

library called aial that is a Cuda

accelerated 5G radio so that we can

software Define and accelerate the

Telecommunications networks the way that

we've soft software defined the world's

networking um uh internet and so the

ability for us to accelerate that allows

us to turn all of Telecom into

essentially the same type of platform a

Computing platform just like we have in

the cloud kitho is a comp comput

computational lithography platform that

allows us to uh process the most

computationally intensive parts of Chip

manufacturing making the mask tsmc is in

the process of going to production with

kitho saving enormous amounts of energy

and more enormous amounts of money but

the goal for tsmc is to accelerate their

stack so that they're prepared for even

further advances in algorithm and more

computation for deeper and deeper uh

narrow and narrow transistors parab

brakes is our Gene sequencing Library it

is the highest throughput library in the

world for Gene sequencing coop is an

incredible library for combinatorial

optimization route planning optimization

the traveling salesman problem

incredibly complicated people just

people have well scientists have largely

concluded that you needed a quantum

computer to do that we created an

algorithm that runs on accelerated

Computing that runs Lightning Fast 23

World Records we hold every single major

world record

today C Quantum is an emulation system

for a quantum computer if you want to

design a quantum computer you need a

simulator to do so if you want to design

Quantum algorithms you need a Quantum

emulator to do so how would you do that

how would you design these quantum

computers create these Quantum

algorithms if the quantum computer

doesn't exist while you use the fastest

computer in the world that exists today

and we call it of course Nvidia Cuda and

on that we have an emulator that

simulates quantum computers it is used

by several hundred thousand researchers

around the world it is integrated into

all the leading M leading Frameworks for

Quantum Computing and is used in

scientific super super Computing centers

all over the world CDF is an

unbelievable library for data

processing data processing cons consumes

the vast majority of cloud spend today

all of it should be accelerated qdf

accelerates the major libraries used in

the world spark many of you probably use

spark in your companies

pandas um a new one called polar and of

course uh networkx which is a graph

processing graph processing uh database

um library and so these are just some

examples there are so many more each one

of them had to be created so that we can

enable the ecosystem to take advantage

of accelerated Computing if we hadn't

created CNN Cuda alone wouldn't have

been able wouldn't have been possible

for all of the deep learning scientists

around the world to use because

Cuda and the algorithms that are used in

tensorflow and pytorch the Deep learning

algorithms the separation is too far

apart it's almost like trying to do

computer Graphics without openg it's

almost like doing data processing

without

SQL these domain specific libraries are

really The Treasure of our company we

have 350 of

them these libraries Is What It Takes

and what has made it possible for us to

have such open so many markets I'll show

you some other examples today well just

last week Google announced that they've

put CDF in the cloud and acceler pandas

pandas is the most popular data science

library in the world many of you in here

probably already use pandas it's used by

10 million data scientists in the world

downloaded 170 million times each

month it is the Excel that is the

spreadsheet of data scientists well with

just one click you can now use pandas in

collab which is Google's cloud data

centers platform accelerated by qdf the

speed up is really incredible let's take

a

[Music]

look that was a great demo right didn't

take

[Applause]

long when you accelerate data processing

that fast demos don't take long

okay well Cuda has now achieved what

people call a Tipping Point but it's

even better than that Cuda has now

achieved a virtuous cycle this rarely

happens if you look at history and all

the Computing architecture Computing

Platforms in the case of microprocessor

CPUs it has been here for 60

years it has not been changed for 60

years at this level this way of doing

Computing accelerated Computing has been

around

has creating a new platform is extremely

hard because it's a chicken and egg

problem if there are no

developers that use your platform then

of course there will be no users but if

there are no users there are no install

base if there no install based

developers aren't interested in it

developers want to write software for a

large installed base but a large

installed base requires a lot of

applications so that users would create

that install base this chicken or the

egg problem has rarely been broken and

has taken us now 20 years one domain

library after another one acceleration

Library another and now we have 5

million developers around the

world we serve every single industry

from Health Care Financial Services of

course the computer industry automotive

industry just about every major industry

in the world just about every field of

science because there are so many

customers for our architecture OEM and

cloud service providers are interested

in building our

systems system makers amazing system

makers like the ones here in Taiwan are

interested in building our systems which

then takes and offers more systems to

the

market which of course creates greater

opportunity for us which allows us to

increase our scale R&D scale which

speeds up the application even more well

every single time we speed up the

application the cost of computing goes

down this is that slide I was showing

you earlier 100x speed up translates to

97 96% 98% savings and so when we go

from 100 x speed up to 200 x speed up to

a th000 x speed up the savings the

marginal cost of computing continues to

fall well of course

we believe that by reducing the cost of

computing

incredibly the market developers

scientists inventors will continue to

discover new algorithms that consume

more and more and more Computing so that

one

day something

happens that a phase shift happens that

the marginal cost of computing is so low

that a new way of using computers

emerge in fact that's what we're seeing

now over the years we have driven down

the marginal cost of computing in the

last 10 years in one particular

algorithm by a million times well as a

result it is now very

logical and very common

sense to train large language models

with all of the data on the internet

nobody thinks

twice this idea that you could create a

computer that could process so much data

to write its own software the emergence

of artificial intelligence was made

possible because of this complete belief

that if we made Computing cheaper and

cheaper and cheaper somebody's going to

find a great use well today Cuda has

achieved the virtual cycle installed

bases growing Computing cost is coming

down which causes more developers to

come up with more more

ideas which drives more

demand and now we're on in the beginning

of something very very important but

before I show you that I want to show

you what is not possible if not for the

fact that we create a Cuda that we

created the modern version of gener the

modern Big Bang of AI generative AI what

I'm about to show you would not be

possible this is Earth two the idea that

we would create a digital twin of the

earth that we would go and simulate the

Earth so that we could predict the

future of our planet to better

avert disasters or better understand the

impact of climate change so that we can

adapt better so that we could change our

habits now this digital twin of Earth is

probably one of the most ambitious

projects that the world's ever

undertaken and we're taking step large

steps every single year and I'll show

you results every single year but this

year we made some great breakthroughs

let's take a

look on Monday the storm will Veer North

again and approach Taiwan there are big

uncertainties regarding its path

different paths will have different

levels of impact on

Taiwan

[Music]

for NVIDIA Earth 2

[Music]

C

[Music]

[Music]

[Music]

cordi

AI

[Music]

[Music]

for NVIDIA earth2 for

[Music]

[Music]

[Applause]

someday in the near future we will have

continuous weather

prediction at every at every square

kilometer on the

planet you will always know what the

climate's going to be you will always

know and this will run continuously

because we've trained the

AI and the AI requires so little

energy and so this is just an inedible

achievement I hope you you enjoyed it

and very

importantly

uh the truth is that was

a Jensen AI that was not

me I I wrote I wrote it but an AI Jensen

AI

had to say

it that is a miracle that is a miracle

indeed however in 2012 something very

important

happened because of our dedication to

advancing Cuda because of our dedication

to continuously improve the performance

of Drive the cost down

researchers discovered AI researchers

discovered Cuda in

2012 that was

nvidia's first contact with

AI this was a very important day we had

the good

wisdom to work with the the scientists

to make it possible for a deep learning

to happen and alexnet achieved of course

a tremendous computer vision

breakthrough but the great wisdom was to

take a step back and understanding what

was the background what is the

foundation of deep learning what is this

long-term impact what is its potential

and we realized that this technology has

great potential to scale an algorithm

that was invented and discovered decades

ago all of a sudden because of more

data larger networks and very

importantly a lot more

compute all of a sudden deep learning

was able to achieve what no human

algorithm was able to now imagine if we

were to scale up the architecture even

more larger networks more data and more

compute what could be possible so we

dedicated ourselves to reinvent

everything after 2012 we changed the

architecture of our GPU to add tensor

course we invented MV link that was 10

years ago

now

cdnn tensor

RT nickel we bought me enox tensor rtlm

the Triton inference server and all of

it came together on a brand new computer

nobody

understood nobody asked for it nobody

understood it and in fact I was certain

nobody wanted to buy it and so we

announced it at

GTC and open AI a small company in San

Francisco saw it and they asked me to

deliver one to them I delivered the

first dgx the F the world's first AI

supercomputer to open

AI in

2016 well after that we continued to

scale from one AI supercomputer one AI

Appliance we scaled it up to large

supercomputers even larger by

2017 the world discovered

Transformers so that we could train

enormous amounts of data and recognize

and learn patterns that are sequential

over large spans of time it is now

possible for for us to train these large

language models to understand and

achieve a breakthrough in natural

language

understanding and we kept going after

that we built even larger ones and then

in November

2022 trained on

thousands tens of thousands of Nvidia

gpus in a very large AI

supercomputer open AI announced chat GPT

1 million users after 5

Days 1 million after 5 days a 100

million after two months the fastest

growing application in history and the

reason for that is very simple it is

just so easy to use and it was so

magical to use to be able to interact

with a computer like it's

Human Instead of being clear about what

you want it's like the computer

understands your meaning it understands

your

intention oh I think here it it um asked

the the closest Night Market night as

you know um the night market is very

important to

me so when I was young uh I was I think

I was four and a half years old I used

to love going to the night market

because I I just love watching people

and and uh uh and so we went my parents

used to take us to the night market

uh

uh and and um uh and I love I love U I

love going and one day uh my face you

guys might might see that I have a large

scar on my face my face was cut because

somebody was washing their knife and I

was a little kid um but my memories of

of the Night Market is uh so deep

because of that and I used to love I I

just I still love going to the night

market and I just need to tell you guys

this the the Tona Night Market is is

really good because there's a lady uh

she's been working there for 43

years she's the fruit lady and it's in

the middle of the stre the middle

between the two go find her okay she

she's really

terrific I think it would be funny after

this all of you go to see

her she every year she's doing better

and better her cart has improved and

yeah I just love watching her succeed

anyways anyways uh Chad GPT came along

and um and something is very important

in this

slide here let me show you something

this

slide okay and this

Slide the fundamental difference is

this until chat

GPT revealed it to the

world AI was all about

perception natural language

understanding computer vision speech

recognition it's all about

perception and

detection this was the first time the

world saw a generative

AI It produced

tokens one token at a time and those

tokens were words some of the tokens of

course could now be images or charts or

tables songs words speech

videos those tokens could be anything

they anything that that you can learn

the meaning of it could be tokens of

chemicals tokens of proteins

genes you saw earlier in Earth 2 we were

generating

tokens of the

weather we can we can learn physics if

you can learn physics you could teach an

AI model physics the AI model could

learn the meaning of physics and it can

generate physics we were scaling down to

to 1 kilm not by using filtering it was

generating and so we can use this method

to generate tokens for almost

anything almost anything of value we can

generate steering wheel control for a

car we can generate articulation for a

robotic

arm everything that we can learn we can

now generate

we have now arrived not at the AI era

but a generative AI era but what's

really important is

this this computer that started out as a

supercomputer has now evolved into a

Data Center and it

produces one thing it produces

tokens it's an AI

Factory this AI Factory

is generating creating producing

something of Great Value a new

commodity in the late

1890s Nicola Tesla invented an AC

generator we invented an AI

generator the AC generator generated

electrons nvidia's AI generator

generates

tokens both of these things have large

Market opportunities

it's completely fungible in almost every

industry and that's why it's a new

Industrial

Revolution we have now a new Factory

producing a new commodity for every

industry that is of extraordinary value

and the methodology for doing this is

quite scalable and the methodology of

doing this is quite

repeatable notice how quickly so many

different AI models generative AI models

are being invented literally daily every

single industry is now piling

on for the very first

time the IT industry which is3

trillion $3 trillion IT industry is

about to create something that can

directly serve a hundred trillion dollar

of Industry no longer just an instrument

for information storage or

data processing but a factory for

generating intelligence for every

industry this is going to be a

manufacturing industry not a

manufacturing industry of computers but

using the computers in

manufacturing this has never happened

before quite an extraordinary thing what

led started with accelerated Computing

led to AI led to generative Ai and now

an industrial

revolution now the impact to our

industry is also quite

significant of course we could create a

new commodity a new product we call

tokens for many Industries but the

impact to ours is also quite profound

for the very first time as I was saying

earlier in 60 years every single layer

of computing has been changed from CPUs

general purpose Computing to accelerated

GPU

Computing where the computer needs

instructions now computers process llms

large language models AI models and

whereas the Computing model of the past

is retrieval

based almost every time you touch your

phone some pre-recorded text or

pre-recorded image or pre-recorded video

is retrieved for you and recomposed

based on a recommender system to present

it to you based on your habits but in

the

future your computer will generate as

much as possible retrieve only what's

necessary and the reason for that is

because generated generated data

requires less energy to go fetch

information generated data also is more

contextually relevant it will encode

knowledge it will your understanding of

you and instead of

get that inform information for me or

get that file for me you just

say ask me for an answer and instead of

a

tool instead of your computer being a

tool that we use the computer will now

generate

skills it performs tasks and instead of

an industry that is producing software

was which was a revolutionary idea in

the early

90s

remember the the idea that Microsoft

created for packaging software

revolutionize the PC industry without

packaged software what would we use the

PC to

do it drove this industry and now we

have a new Factory a new computer and

what we will run on top of this is a new

type of software and we call it Nims

Nvidia inference

microservices now what what happens is

the Nim runs inside this Factory and

this Nim is a pre-train model it's an AI

well this AI is of course quite complex

in itself but the the Computing stack

that runs AI are insanely complex when

you go and use chat GPT underneath their

stack is a whole bunch of software

underneath that prompt is a ton of

software and it's incredibly complex

because the models are large

billions to trillions of parameters it

doesn't run on just one computer it runs

on multiple computers it has to

distribute the workload across multiple

gpus tensor parallelism pipeline

parallelism data parallel all kinds of

parallelism expert parallelism all kinds

of parallelism Distributing the workload

across multiple gpus processing it as

fast as possible because if you are in a

factory if you run a factory you're

through put directly correlates to your

revenues your throughput directly

correlates to quality of service and

your throughput directly correlates to

the number of people who can use your

service we are now in a world where data

center throughput

utilization is vitally important it was

important in the past but not vially

important it was important in the past

but people don't measure it today every

parameter is measured start time uptime

utilization through put idle time you

name it because it's a factory when

something is a factory its operations

directly correlate to the financial

performance of the company and so we

realize that this is incredibly complex

for most companies to do so what we did

was we created this AI in a box and the

containers incredible months of

software inside this container is Cuda

CNN tensor RT Triton for inference

Services it is cloud native so that you

could Auto scale in a kubernetes

environment it has Management Services

and hooks so that you can monitor your

AIS it has common apis standard API so

that you could literally chat with this

box you download this Nim and you can

talk to it so long as you have Cuda on

your

computer which is now of course

everywhere it's in every cloud available

from every computer maker it is

available in hundreds of millions of PCS

when you download this you have an AI

and you can chat with it like chat GPT

all of the software is now integrated

400 dependencies all integrated into one

we tested this Nim each one of these

pre-trained models against all kind our

entire install base that's in the cloud

all the different versions of Pascal and

ampers and Hoppers

and all kinds of different versions I

even forget

some Nims incredible invention this is

one of my favorites and of course as you

know we now have the ability to create

large language models and pre-trained

models of all kinds and we we have all

of these various versions whether it's

language based or Vision based or are

Imaging based or we have versions that

are available for healthc care digital

biology we have versions that are

digital humans that I'll talk to you

about and the way you use this just come

to ai. nvidia.com and today we uh just

posted up in hugging face the Llama 3

Nim fully optimized it's available there

for you to try and you can even take it

with you it's available to you for free

and so you could run it in the cloud run

it in any Cloud you could download this

container put it into your own Data

Center and you could host it make it

available for your customers we have as

I mentioned all kinds of different

domains physics some of it is for

semantic retrieval called Rags Vision

languages all kinds of different

languages and the way that you use

it is connecting these microservices

into large applications one of the most

important applications in the coming

future of course is customer service

agents customer service agents are

necessary in just about every single

industry it represents trillions of

dollars of of customer service around

the world nurses are customer service

agents um in some ways some of them are

nonprescription or or non Diagnostics um

based nurses are essentially customer

service uh customer service for retail

for uh Quick Service Foods Financial

Services Insurance just tens and tens of

millions of customer service can now be

augmented by language models and

augmented by Ai and so these one these

boxes that you see are basically Nims

some of the NIMS are reasoning agents

given a task figure out what the mission

is break it down into a plan some of the

NIMS retrieve information some of the

NIMS might uh uh uh go and do search

some of the NIMS might use a tool like

Coop that I was talking about earlier it

could use a tool that uh could be

running on sap and so it has to learn a

particular uh language called abap maybe

some Nims have to uh uh do SQL queries

and so all of these Nims are experts

that are now assembled as a

team so what's happening

the application layer has been

changed what used to be applications

written with

instructions are now

applications that are assembling teams

assembling teams of

AIS very few people know how to write

programs almost everybody knows how to

break down a problem and assemble teams

very every company I believe in the

future will have a large collection of

Nims and you would bring down the

experts that you want you connect them

into a team and you you don't even have

to figure out

exactly how to connect

them you just give the mission to an

agent to a Nim to figure out who to

break the task down and who to give it

to and they that a that Central the

leader of the of the application if you

will the leader of the team would break

down the task and give it to the various

team members the team members would do

their perform their task bring it back

to the team leader the team leader would

reason about that and present an

information back to you just like humans

this is in our near future this is the

way applications are going to look now

of

course we could interact with these

large these AI services with text

prompts and speech prompts

however there are many applications

where we would like to interact with

what what is otherwise a humanlike form

we call them digital humans Nvidia has

been working on digital human technology

for some time let me show it to you and

well before I do that hang on a second

before I do that okay digital humans has

the potential of being a great interact

interactive agent with you they make

much more engaging they could be much

more empathetic

and of course um we have to uh cross

this incredible Chasm this uncanny Chasm

of realism so that the digital humans

would appear much more natural this is

of course our vision this is a vision of

where we love to go uh but let me show

you where we

are great to be in Taiwan before I head

out to the night market let's dive into

some exciting frontiers of digital

humans imagine a future where computers

interact with us just like humans can hi

my name is Sophie and I am a digital

human brand ambassador for Unique this

is the incredible reality of digital

humans digital humans will revolutionize

industries from customer service to

advertising and

gaming the possibilities for digital

humans are endless you the scans you

took of your current kitchen with your

phone they will be AI interior designers

helping generate beautiful

photorealistic suggestions and sourcing

the materials and Furniture we have

generated several design options for you

to choose from there'll also be AI

customer service agents making the

interaction more engaging and

personalized or digital healthcare

workers who will check on patients

providing timely personalized care um I

did forget to mention to the doctor that

I am allergic to pen in is it still okay

to take the medications the antibiotics

you've been prescribed cicin and

metronidazol don't contain penicillin so

it's perfectly safe for you to take them

and they'll even be AI brand ambassadors

setting the next marketing and

advertising Trends hi I'm IMA Japan's

first virtual

model new breakthroughs in generative Ai

and computer Graphics let digital humans

see understand and interact with us in

humanlike

ways H from what I can see it looks like

you're in some kind of recording or

production setup the foundation of

digital humans are AI models built on

multilingual speech recognition and

synthesis and llms that understand and

generate conversation

the AIS connect to another generative AI

to dynamically animate a lifelike 3D

mesh of a face and finally AI models

that reproduce lifelike appearances

enabling real-time path traced

subsurface scattering to simulate the

way light penetrates the skin scatters

and exits at various points giving skin

its soft and translucent

appearance Nvidia Ace is a suite of

digital human Technologies packaged as

easy to deploy fully optimized

microservices or Nims developers can

integrate Ace Nims into their existing

Frameworks engines and digital human

experiences neotron slm and llm Nims to

understand our intent and orchestrate

other models Reva speech Nims for

interactive speech and

translation audio to face and gesture

Nims for facial and body animation and

Omniverse RTX with dlss for neuro

rendering of skin and hair Ace Nims run

on Nvidia gdn a Global Network of Nvidia

accelerated infrastructure that delivers

low latency digital human processing to

over 100

[Music]

regions

Hama pretty incredible while those

those Ace runs in a cloud but it also

runs on PCS we had the good wisdom of

including tensor core gpus in all of RTX

so we've been shipping AI gpus for some

time preparing ourselves for this day

the reason for that is very simple we

always knew that in order to create a

new Computing platform you need an

install base first eventually the

application will come if you don't

create the install BAS

how could the application come and so if

you build it they might not

come but if you build it if you don't

build it they cannot come and so we

installed every single RTX GPU with

tensor core G tensor core processing and

now we have 100 million GeForce RTX AIP

PCS in the world and we're shipping 200

and this this copy Tex were featuring

four new amazing

laptops all of them are able to run AI

your future laptop your future PC will

become an AI it'll be constantly helping

you assisting you in the background the

PC will also run applications that are

enhanced by AI of course all your photo

editing your writing and your tools all

the things that you use will all be

enhanced by Ai and your PC will also

host applications with digital humans

that are AIS and so there are different

ways that AIS will manifest themselves

and become used in PCS but PCS will

become very important AI platform and so

where do we go from

here I spoke earlier about the scaling

of our data centers and every single

time we scaled we found a new phase

change when we scaled from dgx into

large AI supercomputers we enabled trans

Transformers to be able to train on

enormously large data data sets well

what happened was in the beginning the

data was human

supervised it required human labeling to

train AIS unfortunately there's only so

much you can human label Transformers

made it possible for unsupervised

learning to happen now Transformers just

look at an enormous amount of data or

look at enormous amount of video or look

at enormous amount of uh images and it

can learn from studying an enormous

amount of data find the patterns and

relationships

itself while the next generation of AI

needs to be physically based most of the

AIS today uh don't understand the laws

of physics it's not grounded in the

physical world in order for us to

generate uh uh images and videos and 3D

graphics and many physics phenomenons we

need AI that are physically based and

understand the laws of physics well the

way that you could do that is of course

learning from video is One Source

another way is synthetic data simulation

data and another way is using computers

to learn with each other this is really

no different than using alphago having

alphao play itsself self-play and

between the two

capabilities same capabil abilities

playing each other for a very long

period of time they emerge even smarter

and so you're going to start to see this

type of AI emerging well if the AI data

is synthetically generated and using

reinforcement learning it stands to

reason that the rate of data generation

will continue to advance and every

single time data generation grows the

amount of computation that we have to

offer needs to grow with it we are to

enter a phase where AIS can learn the

laws of physics and understand and be

grounded in physical world data and so

we expect that models will continue to

grow and we need larger gpus well

Blackwell was designed for this

generation this is Blackwell and it has

several very important Technologies uh

one of course is just the size of the

chip we took two of the largest a chip

that is as large as you can make it at

tsmc and we connected two of them

together with a 10 terabytes per second

link between the world's most advanced

cies connecting these two together we

then put two of them on a computer node

connected with a gray CPU the gray CPU

could be used for several things in the

training situation it could use it could

be used for fast checkpoint and restart

in the case of inference and generation

it could be used for storing context

memory so that the AI has memory and

understands uh the context of the the

conversation you would like to have it's

our second generation Transformer engine

Transformer engine allows us to adapt

dynamically to a lower precision based

on the Precision and the range necessary

for that layer of computation uh this is

our second generation GPU that has

secure AI so that you could you could

ask your service providers to protect

your AI from being either stolen from

theft or tampering this is our fifth

generation MV link MV link allows us to

connect multiple gpus together and I'll

show you more of that in a second and

this is also our first generation with a

reliability and availability engine this

system this Ras system allows us to test

every single transistor flip-flop memory

on chip memory off chip so that we

can in the field determine whether a

particular chip is uh failing the mtbf

the meantime between failure of a

supercomputer with 10,000

gpus is a measured in

hours the meantime between failure of a

supercomputer with 100,000 gpus is

measured in

minutes and so the ability for a

supercomputer to run for a long period

of time and train a model that could

last for several months is practically

impossible if we don't invent

Technologies to enhance its reliability

reliability would of course enhance its

uptime which directly affects the cost

and then lastly decompression engine

data processing is one of the most

important things we have to do we added

a data compression engine decompression

engine so that we can pull data out of

storage 20 times faster than what's

possible today well all of this

represents Blackwell and I think we have

one here that's in production during GTC

I showed you Blackwell in a prototype

State um the other

side this is why we

practice ladies and gentlemen this is

Blackwell

black black well is in

production incredible amounts of

Technology this is our production board

this is the most complex highest

performance computer the world's ever

made this is the gray

CPU and these are you could see each one

of these blackw dieses two of them

connected together you see that it is

the largest Dy the the largest chip the

world makes and then we connect two of

them together with a 10 terabyte per

second

link okay and that makes the Blackwell

computer and the performance is

incredible take a look at

this so

um you see you see our

uh the the computational the flops the

AI flops uh for each generation has

increased by a thousand times in eight

years Mo's law in eight

years is something along the lines of oh

I don't know maybe 40

60 and in the last eight years Mo's law

has gone a lot lot less and so just to

compare even Mo's law at its best of

times compared to what Blackwell could

do so the amount of computations is

incredible and when whenever we bring

the computation High the thing that

happens is the cost goes down and I'll

show you what we've done is we've

increased through its computational

capability the energy used to train a

gp4

2 trillion parameter 8 trillion

Tokens The amount of energy that is used

has gone down by 350 times Well Pascal

would have taken

1,000 gigatt hours 1,000 gwatt hours

means that it would take a gigawatt data

center the world doesn't have a gigawatt

data center but if you had a gigawatt

data center it would take a month if you

had a 100 watt 100 megawatt data center

it would take about a year and so nobody

would of course um uh create such a

thing and um and that's the reason why

these large language models chat GPT

wasn't possible only eight years ago by

us driving down the increasing the

performance the energy efficent while

keeping and improving energy efficent

efficiency along the way we've now taken

with Blackwell what used to be 1,000

gwatt hours to three and incredible

Advance uh 3 gwatt hours if it's a if

it's a um uh uh a 10,000 gpus for

example it would only take a cou 10,000

gpus I guess it would take a few days 10

days or so so the amount of advance in

just eight years is incredible well this

is for inference this is for token

generation Our token generation

performance has made it possible for us

to drive the energy down by three four

45,000

times 177,000 Jewels per token that was

Pascal 177,000 Jewels is kind of like

two light bulbs running for two days it

would take two light bulbs running for

two days amounts of energy 200 Watts

running for two days to generate one

token of GPT

4 it takes about three tokens to

generate one

word and so the amount of energy used

necessary for Pascal to generate gp4 and

have a chat GPT experience with you was

practically impossible but now we only

use 0.4 jewles per token and we can

generate tokens at incredible rates and

very little energy okay so Blackwell is

just an enormous leap well even so it's

not big enough and so we have to build

even larger

machines and so the way that we build it

is called dgx so this is this is our

Blackwell chips and it goes into djx

systems that's why we should

practice so this is a dgx Blackwell this

has this is air cooled has eight of

these gpus inside look at the size of

the heat sinks on these

gpus about 15 kilowatts 15,000 watts and

completely air cooled this version

supports x86 and it's it goes into the

infrastructure that we've been shipping

Hoppers into however if you would like

to have liquid cooling we have a new

system and this new

system is based on this board and we

call it mgx for

modular and this modular

system you won't be able to see this

this can they see this can you see this

you can the are

you

okay I

say and so this this is the mgx system

and here's the two uh black Blackwell

boards so this one node has four

Blackwell chips these four Blackwell

chips this is liquid

cooled nine of

them nine of them uh well 72 of these 72

of these gpus 72 of these G gpus are

then connected together with a new MV

link this is MV link switch

fifth

generation and the mvlink switch is a

technology Miracle this is the most

advanced switch the world's ever made

the data rate is

insane and these switches connect every

single one of these black Wells to each

other so that we have one giant 72 GPU

Blackwell well the

benefit the benefit of this is that in

one domain one GPU domain this now looks

like one GPU this one GPU has 72 versus

the last generation of eight so we

increased it by nine times the amount of

bandwidth we've increased by 18 times

the AI flops we've increased by 45 times

and yet the amount of power is only 10

times this is 100 kilow and that is 10

kilow and that's for one now of course

well you could always connect more of

these together and I'll show you to do

that in a second but what's the miracle

is this chip this MV link chip people

are starting to awaken to the importance

of this mvlink chip as it connects all

these different gpus together because

the large language models are so large

it doesn't fit on just one GPU doesn't

fit on one just just one node it's going

to take the entire rack of gpus like

this new dgx that I that I was just

standing next to uh to hold a large

language model that are tens of

trillions of parameters large mvlink

switch in itself is a technology Miracle

it's 50 billion transistors 74 ports at

400 gbits each four

links cross-sectional bandwidth of 7 7.2

terabytes per second but one of the

important things is that it has

mathematics inside the switch so that we

can do reductions which is really

important in deep learning right on the

Chip And so this is what this is what um

a dgx looks like now and a lot of people

ask us

um you know they say and there's this

there's this confusion about what Nvidia

does and and um how is it

possible that that Nvidia became so big

building

gpus and so there's an impression that

this is what a GPU looks like now this

is a GPU this is one of the most

advanced gpus in the world but this is a

gamer GPU but you and I know that this

is what a GPU looks like this is one

GPU ladies and gentlemen dgx

GPU you

know the back of this GPU is the MV link

spine the MV link spine is 5,000

wires two

miles and it's right here

this is an mvlink

spine and it connects

70 two gpus to each

other this is a electrical mechanical

Miracle the transceivers makes it

possible for us to drive the entire

length in Copper and as a result this

switch the Envy switch Envy link switch

driving the EnV link spine in Copper

makes it possible for us to save 20

kilowatt in one rack 20 kilowatt could

now be used for processing just an

incredible achievement so this is the

the MV links

[Applause]

spine

W I went down

today and if you and even this is not

big

enough even this is not big enough for

AI factories so we have to connect it

all together with very high-speed

networking well we have two types of

networking we have infiniband which has

been used uh in supercomputing and AI

factories all over the world and it is

growing incredibly fast for us however

not every data center can handle

infiniband because they've already

invested their ecosystem in Ethernet for

too long and it does take some specialty

and some expertise to manage infiniband

switches and infiniband networks and so

what we've done is we've brought the

capabilities of infiniband to the

ethernet architecture which is

incredibly hard and the reason for that

is

this ethernet was designed for high

average

throughput because every single note

every single computer is connected to a

different person on the internet and

most of the communications is the data

center with somebody on the other side

of the internet

however deep learning in AI

factories the gpus are not communicating

with people on the internet mostly it's

communicating with each

other they're communicating with each

other because they're all they're

collecting partial products and they

have to reduce it and then redistribute

it chunks of partial products reduction

redistribution that traffic is

incredibly

bursty and it is not the average through

put that matters it's the last arrival

that matters because if you're reducing

collecting partial products from

everybody if I'm trying to take all of

your so it's not the average throughput

is whoever gives me the answer

last okay ethernet has no provision for

that and so there are several things

that we have to create we created an

endtoend architecture so that the the

Nick and the switch can communicate and

we applied four different Technologies

to make this possible number one Nvidia

has the world's most advanced RDMA and

so now we have the ability to have a

network level RDMA for ethernet that is

incredibly great number two we have

congestion control the switch does

Telemetry at all times incredibly fast

and whenever the uh the uh the the gpus

or the the nxs are sending too much

information we can tell them to back off

so that it doesn't create hotspots

number three adaptive

routing ethernet needs to transmit and

receive in

order we see

congestions or we see

uh ports that are not currently being

used irrespective of the ordering we

will send it to the available ports and

Bluefield on the other end reorders it

so that it comes back in order that

adaptive routing incredibly powerful and

then lastly noise

isolation there's more than one model

being trained or something happening in

the data center at all times and their

noise and their traffic could get into

each other and causes Jitter and so when

it's when the noise of one training

model one model training causes the last

arrival to end up too late it really

slows down to training well overall

remember you have you have built A5

billion doll or3 billion Data Center and

you're using this for training if the

utilization Network

utilization

was 40% lower and as a result the

training time was 20%

longer

the5 billion data center is effectively

like a $6 billion data center so the

cost is incredible the cost impact is

quite

High ethernet with Spectrum X basically

allows us to improve the performance so

much that the network is basically free

and so this is really quite an

achievement we're very we have a whole

pipeline of ethernet products behind us

this is Spectrum x800

it is uh 51.2 terabits per second and

256

radic the next one coming is 512 Rix is

one year from now 512 Ric and that's

called Spectrum x800 Ultra and the one

after that is x1600 but the important

idea is this x800 is designed for tens

of

thousands tens of thousands of gpus x800

ultra is designed for hundreds of

thousands of gpus and x1600 is designed

for millions of gpus the days of

millions of GPU data centers are coming

and the reason for that is very simple

of course we want to train much larger

models but very importantly in the

future almost every interaction you have

with the Internet or with a computer

will likely have a generative AI running

in the cloud somewhere and that

generative AI is working with you

interacting with you generating videos

or images or text or maybe a digital

human and so you're interacting with

your computer almost all the time and

there's always a generative AI connected

to that some of it is on Prem some of it

is on your device and a lot of it could