“There’s no way to get there without a breakthrough,” OpenAI CEO Sam Altman said, arguing that AI will soon need even more energy.
So is AI the new Blockchain?
Unlike the Blockchain it has an actual use tho.
I bought pizza with bitcoin, haven’t bought anything with AI yet.
I get daily use out of LLMs but haven’t done anything with Bitcoin.
I want to use Bitcoin to buy cannabis and Soylent and have it arrive at my door via drone. Give me my better future
Give an LLM to a high school student and they could do something with it.
Try to explain block chain to an adult and watch them pretend to understand.
Always was.
Optimizing power consumption? Why?!
In fairness the computing world has seen unfathomable efficiency gains that are being pushed further with the sudden adoption of arm. We are doing our damnedest to make computers faster and more efficient, and we’re doing a really good job of it, but energy production hasn’t seen nearly those gains in the same amount of time. With the sudden widespread adoption of AI, a very power hungry tool (because it’s basically emulating a brain in a computer), it has caused a sudden spike in energy needed for computers that are already getting more efficient as fast as we can. Meanwhile energy production isn’t keeping up at the same rate of innovation.
The problem there is the paradox of efficiency, making something more efficient ends up using more of it not less as the increase in use stimulated by the greater efficiency outweighs the reduced input used.
It’s not so much the hardware as it is the software and utilisation, and by software I don’t necessarily mean any specific algorithm, because I know they give much thought to optimisation strategies when it comes to implementation and design of machine learning architectures. What I mean by software is the full stack considered as a whole, and by utilisation I mean the way services advertise and make use of ill-suited architectures.
The full stack consists of general purpose computing devices with an unreasonable number of layers of abstraction between the hardware and the languages used in implementations of machine learning. A lot of this stuff is written in Python! While algorithmic complexity is naturally a major factor, how it is compiled and executed matters a lot, too.
Once AI implementations stabilise, the theoretically most energy efficient way to run it would be on custom hardware made to only run that code, and that code would be written in the lowest possible level of abstraction. The closer we get to the metal (or the closer the metal gets to our program), the more efficient we can make it go. I don’t think we take bespoke hardware seriously enough; we’re stuck in this mindset of everything being general-purpose.
As for utilisation: LLMs are not fit or even capable of dealing with logical problems or anything involving reasoning based on knowledge; they can’t even reliably regurgitate knowledge. Yet, as far as I can tell, this constitutes a significant portion of its current use.
If the usage of LLMs was reserved for solving linguistic problems, then we wouldn’t be wasting so much energy generating text and expecting it to contain wisdom. A language model should serve as a surface layer – an interface – on top of bespoke tools, including other domain-specific types of models. I know we’re seeing this idea being iterated on, but I don’t see this being pushed nearly enough.[1]
When it comes to image generation models, I think it’s wrong to focus on generating derivative art/remixes of existing works instead of on tools to help artists express themselves. All these image generation sites we have now consume so much power just so that artistically wanting people can generate 20 versions (give or take an order of magnitude) of the same generic thing. I would like to see AI technology made specifically for integration into professional workflows and tools, enabling creative people to enhance and iterate on their work through specific instructions.[2] The AI we have now are made for people who can’t tell (or don’t care about) the difference between remixing and creating and just want to tell the computer to make something nice so they can use it to sell their products.
The end result in all these cases is that fewer people can live off of being creative and/or knowledgeable while energy consumption spikes as computers generate shitty substitutes. After all, capitalism is all about efficient allocation of resources. Just so happens that quality (of life; art; anything) is inefficient and exploiting the planet is cheap.
For example, why does OpenAI gate external tool integration behind a payment plan while offering simple text generation for free? That just encourages people to rely on text generation for all kinds of tasks it’s not suitable for. Other examples include companies offering AI “assistants” or even AI “teachers”(!), all of which are incapable of even remembering the topic being discussed 2 minutes into a conversation. ↩︎
I get incredibly frustrated when I try to use image generation tools because I go into it with a vision, but since the models are incapable of creating anything new based on actual concepts I only ever end up with something incredibly artistically compromised and derivative. I can generate hundreds of images based on various contortions of the same prompt, reference image, masking, etc and still not get what I want. THAT is inefficient use of resources, and it’s all because the tools are just not made to help me do art. ↩︎
It’s emulating a ridiculously simplified brain. Real brains have orders of magnitude more neurons, but beyond that they already have completely asynchronous evaluation of those neurons, as well as much more complicated connecting structure, as well as multiple methods of communicating with other neurons, some of which are incredibly subtle and hard to detect.
To really take AI to the next level I think you’d need a completely bespoke processor that can replicate those attributes in hardware, but it would be a very expensive gamble because you’d have no idea if it would work until you built it.
This dude al is the new florida man, wonder if it’s the same al from married with children
Some of the smartest people on the planet are working to make this profitable. It’s fucking hard.
You are dense and haven’t taking even a look at simple shit like hugging face. Power consumption is about the biggest topic you find with anyone in the know.
Some of the smartest people on the planet are working to make this profitable. It’s fucking hard.
[Take a look at] hugging face. Power consumption is about the biggest topic you find with anyone in the know.
^ fair comment
So much for hoping ai was going to solve energy breakthroughs.
Got to spend energy breakthroughs to make energy breakthroughs?
Nvidia Execs: Did you say the price of GPUs should go up?
OH NO, AI IS THE BLOCKCHAIN.
So why won’t he use the AI to discover one?
I’m sorry but as an AI language processing model I am unable to discover alternative energy sources. My training data concludes on June 21, 2021 and I am unable to understand requests that would require knowledge after that date.
All the “AI” can do is string together words actual humans have written. Those ChatGPT bots don’t think.
As opposed to humans who think all the time. Florida for example.
Just apply it to generating energy and not just storing it.
If only we could convert empty hype into energy.
Well we can, we had a “jumpstyle” wave going on in the Netherlands a couple of years ago. No clue if it ever got off the ground anywhere else seeing as it was a techno thing or something.
It’s like crypto but sliiiighly better
The human brain uses about 20W. Maybe AI needs to be more efficient instead?
Subsidize retrofitting the entire nation with solar and invert it back into the grid
Might be because it’s a LLM not an AI and requires massive amounts of data to be funneled into it to actually work. My admittedly limited understanding of it makes it seem like it’s just another buzzword for things like neural networks and machine learning.
LLMs are a specific application of neutral networks which utilize machine learning.
If those ‘buzzwords’ aren’t ai then what do you think ai is? It’s like saying circle and square aren’t shapes they’re buzzwords
Circles are just marketing hype. Now ovals are a thing, and you can buy some pretty neat ones.
We’ve only been doing nested if statements, but in a more complex manner.
If you think about it, a transistor is a bit like a hardware if statement, it’s if statements all the way down.
So AI can’t exist without stealing people’s content and it can’t exist without using too much energy. Why does it exist then?
Because the shareholders need more growth. They might create Ultron along the way, but think of the profits, man!
There’s no way these chatbots are capable of evolving into Ultron. That’s like saying a toaster is capable of nuclear fusion.
It’s the further research being done on top of the breakthrough tech enabling the chat bots applications people are worried about. It’s basically big tech’s mission now to build Ultron, and they aren’t slowing down.
What research? These bots aren’t that complicated beyond an optimisation algorithm. Regardless of the tasks you give it, it can’t evolve beyond what it is.
The models get more efficient and smaller very fast if you look just a year back. I bet we’ll run some small LLMs locally on our phones (I don’t really believe in the other form factors yet) sooner as we believe. I’d say prior 2030.
I can already locally host a pretty decent ai chatbot on my old M1 Macbook (llama v2 7B) which writes at the same speed I can read, its probably already possible with the top of the line phones.
Lol, “old M1 laptop” 3 to 4 years is not old, damn!
(I have running macbookpro5,3 (mid 2009) on Arch, lol)
But nice to hear that M1 (an thus theoretically even the iPad, if you are not talking about M1 pro / M1 max) can already run llamma v2 7B.
Have you tried the mistralAI already, should be a bit more powerful and a bit more efficient iirc. And it is Apache 2.0 licensed.
3 to 4 years is not old
Huh, nice. I got the macbook air secondhand so I thought it was older. Thanks for the suggestion, I’ll try mistralAI next, perhaps on my phone as a test.
Because it’s a miracle technology. Both of those things are also engineering problems - ones that have been massively mitigated already. You can run models almost as good as gpt3.5 on a phone, and individuals are pushing the limits on how efficiently we can train every week
It’s not just making a chatbot or a new tool for art - it’s also protein folding, coming up with unexpected materials, and being another pair of eyes that will assist a person do anything.
They literally promise the fountain of youth, autonomous robots, better materials, better batteries, better everything. It’s a path for our species to break our limits, and become more.
The downside is we don’t know how to handle it. We’re making a mess of it, but it’s not like we could stop… The AI alignment problem is dwarfed by the corporation alignment problem
🙄 iTS nOt stEAliNg, iTS coPYiNg
By your definition everything is stealing content. Nearly everything in human history is derivative of others work.
So will AI (or more accurately LLM) be the push needed to make limitless fusion energy a reality?
Obviously not. We’re being faced with an existential threat if we don’t secure alternative, sustainable forms of energy and even that threat isn’t enough to motivate our species.
Threat to existence is nothing compared to ability to hoard more money
Wow, I fucking hate this guy the more he opens his mouth. He can seriously fuck off right now, if he thinks AI realistically needs him at this point he’s sadly mistaken.
hey guys, theres this thing called nuclear energy. It’s pretty cool.
Thorium molten salt reactor
love a good thorium based metal cooled reactor
Can’t do it. Makes the boomers wet themselves. Jane Fonda made a movie about it after she got back from aiding and firing on US troops.
Massively subsidized and where do you put all the nuclear waste? Nuclear energy is dumb even without thinking about possible disasters. You are just falling for grifters who don’t want us to use renewable sources of energy. And before you say it: no, nuclear energy is not green. You would know that if you actually googled for like 5 seconds, but it’s easier to believe grifters promising “the one easy solution to solve all our problems”, right?
Massively subsidized
Nuclear energy is four times cheaper than renewables when externalities like baseline generation are imputed: https://www.sciencedirect.com/science/article/abs/pii/S0360544222018035?via%3Dihub
where do you put all the nuclear waste?
While more dangerous, the quantity of waste generated compared to all other forms of energy generation is very small. Storage is a solved problem, but you have probably read articles about a lack of storage in the U.S. This is entirely due to politicians’ failure to agree on where to store waste. Despite the relative safety, no one wants nuclear waste stored in their “back yard.”
And before you say it: no, nuclear energy is not green.
Nuclear energy generates zero CO2. Surely we can agree that this is the most pressing consideration in terms of climate change. If your concern is the nuclear waste, then I direct you to the growing problem of disposing of solar cells and wind turbines. Newer turbine blades, for example, are 40 meters long and weigh 2.5 tons. These cannot be recycled.
No matter how you cut the data, nuclear is an order of magnitude better than almost all other forms of energy generation. If our goal is to radically improve our environmental footprint while keeping the lights on even at night when it’s not windy, then nuclear absolutely must be part of the mix.
This guy knows his shit
Unfortunately he does only know how to misrepresent shit. This is of course all bullshit, and at best outdated information that does not take the massively falling price of renewable energy into account. Nuclear can be a transition helper, IF and only IF you already have running reactors.
We could’ve had those reactors if people didn’t say the same things you’re saying 30 years ago.
Let’s see your data
Let’s talk about the technology instead of the dumb word “nuclear”. Thorium fission > uranium fission.
Plant near me had a 1000% construction cost overrun and the company that owns it now is always threatening to shut it down if the state doesn’t give them more money.
If we look at just Europe, Slovakia, Finland, and Belarus all brought new reactors online last year alone. There are another six reactors currently under construction, and another 33 planned. France and Sweden recently announced their strategic commitment to nuclear power for a variety of reasons.
One major technological breakthrough is Small Modular Reactors (SMRs). These are far more cost effective, very safe (the reactor shuts down in the event of loss of power and coolant), and require a much smaller footprint. Rolls-Royce is on target to deliver the first of these in 2030.
The example you provide is an example of poor governance, not an inherent limitation of the technology. There are also examples of poor governance regarding renewable energy all over the world.
Will the reactors come with the little lady at the front?
Oh wow a government cost overrun. THIS HAS NEVER HAPPENED BEFORE!
mostly because nobody knows how to build a reactor properly anymore… Wonder why that happened.
Wierd spin you put on all of that. Burn the solar panels and blades. Reclaim the energy in heat and its still way safer than nuclear waste.
You can’t be serious, can you? First off you would need pretty higher temperatures to burn glass. Secondly the fumes and dust it would put out would be nasty.
Yeah, still not radioactive nasty though. Don’t get how you are all so naive. The only reason most countries have a nuclear program is so they have nuclear weapons.
You are right it isn’t very radioactive and a lot harder to control, not like I designed air scrubbers for 4 years of my life or something.
The only reason most countries have a nuclear program is so they have nuclear weapons.
Citation needed.
A pity decades of OPEC propaganda has worked so well.
Ahh you’re not naive you are biased. Anything you say is effectively propaganda. Jog on.
The only reason most countries have a nuclear program is so they have nuclear weapons.
The only reason most countries have a nuclear program capable of generating plutonium products is to build nuclear weapons*
FTFY
you put the nuclear waste in a hole, deep underground, after burning most of it up. Modern gen 4 designs can burn the vast majority of existing waste products down to a much more reasonable time span.
Nuclear energy is vastly more green than, coal, gas, petro, etc… Currently arguably more sustainable than massive amounts of solar and wind energy. Wind in particular has a massive waste issue, solar, it’s more complicated but there are a lot of precious metals involved and heavy refining done. It’s not a zero emissions industry either. The actual production of electricity IS net zero, unlike coal, petro, and gas, which still powers the majority of our grids. Please continue to explain to me how fossil fuels are better than funny green rock.
You’re also accusing me of knowing nothing about nuclear, which is funny, considering i have quite the autistic hyper-fixation on it. And know vastly more about it than the average person. Judging by your response, you’re probably not in the field of nuclear energy either.
Nuclear is a technology we know how to build, understand how to operate safely, and are capable of doing correctly. The only thing we need, is more nuclear plants.
The positive thing there is that it probably paces our development. If we can’t get to true AGI without way more energy than we can currently produce, then we don’t have true AGI risk right now.
There’s still risk because it might not be true or we might be able to get close enough to do damage. But slowing down AI is fine by me.
