Reading this makes me willing to bet that this capital intensive investment boom will be similar to other enormous capital investment booms in US history, such as the laying of the railroads in the 1800s, the proliferation of car companies in the early 1900s, and the telecom fiber boom in the late 1900s. In all of these cases there was an enormous infrastructure (over) build out, followed by a crash where nearly all the companies in the industry ended up in bankruptcy, but then that original infrastructure build out had huge benefits for the economy and society as that infrastructure was "soaked up" in the subsequent years. E.g. think of all the telecom investment and subsequent bankruptcies in the late 90s/early 00s, but then all that dark fiber that was laid was eventually lit up and allowed for the explosion of high quality multimedia growth (e.g. Netflix and the like).
I think that will happen here. I think your average investor who's currently paying for all these advanced chips, data centers and energy supplies will walk away sorely disappointed, but this investment will yield huge dividends down the road. Heck, I think the energy investment alone will end up accelerating the switch away from fossil fuels, despite AI often being portrayed as a giant climate warming energy hog (which I'm not really disputing, but now that renewables are the cheapest form of energy, I believe this huge, well-funded demand will accelerate the growth of non-carbon energy sources).
I'm sure you are right. At some point, the bubble will crash.
The question remains is when the bubble will crash. We could be in the 1995 equivalent of the dotcom boom and not 1999. If so, we have 4 more years of high growth and even after the crash, the market will still be much bigger in 2029 than in 2024. Cisco was still 4x bigger in 2001 than in 1995.
One thing that is slightly different from past bubbles is that the more compute you have, the smarter and more capable AI.
One gauge I use to determine if we are still at the beginning of the boom is this: Does Slack sell an LLM chatbot solution that is able to give me reliable answers to business/technical decisions made over the last 2 years in chat? We don't have this yet - most likely because it's probably still too expensive to do this much inference with such high context window. We still need a lot more compute and better models.
Because of the above, I'm in the camp that believe we are actually closer to the beginning of the bubble than at the end.
Another thing I would watch closely to see when the bubble might pop is if LLM scaling laws are quickly breaking down and that more compute no longer yields more intelligence in an economical way. If so, I think the bubble would pop. All eyes are on GPT5-class models for signs.
Glean.com does it for the enterprise I work at:
It consumes all of our knowledge sources including Slack, Google docs, wiki, source code and provides answers to complex specific questions in a way that’s downright magical.
I was converted into a believer when I described an issue to it, pointers to a source file in online git repo and it pointed me to another repository that my team did not own that controlled DNS configs that we were not aware about. These configs were the reason our code did not behave as we expected.
This is the main "killer feature" I've personally experienced from GPT things: a much better contextual "search engine-ish" tool for combing through and correlating different internal data sources (slack, wiki, jira, github branches, etc).
AI code assistants have been a net neutral for me (they get enough idioms in C++ slightly incorrect that I have to spend a lot of time just reading the generated code thoroughly), but being able to say "tell me what the timeline for feature X is" and have it comb through a bunch of internal docs / tickets / git commit messages, etc, and give me a coherent answer with links is amazing.
this is part of why I stay in python when doing ai-assisted programming; there's so much training information out there for python and I _generally_ don't care about if its slightly off-idiom, its still probably fine.
Yea, I was thumbs-down on ai-assisted programming because when I tested it out, I tried it by adding things to my existing C and C++ projects, and its suggestions were... kind of wild. Then, a few months later I gave it another chance when I was writing some Python and was impressed. Finally, I used it on a new-from-blank-text-file Rust project and was pretty much blown away.
The best I have ever seen were obscure languages with very strong type safety. Some researcher at a sibling org to my own told me to try it with the Lean language, and it basically gave flawless suggestions.
I'm guessing this is because the only training material was blogs from uber-nerdy CS researchers on a language where "mistakes" are basically impossible to write, and not a bunch of people flailing on forums asking about hello world-ish stuff and segfaulting examples.
As someone who doesn't generally program, it was pretty good at getting me an init.lua set up for nvim with a bunch of plugins and some functions that would have taken me ages to do by hand. That said...it still took a day or two of working with it and troubleshooting everything, and while it's been reliable so far, I worry that it's not exactly idiomatic. I don't know enough to really say.
What it's really good at is taking my description of something and pointing me in the right direction to do my own research.
(two things that helped me with getting decent code were to describe the problem and desired solution, followed by a "Does that make sense?". This seems to get it to restate the problem itself and produce better solutions. The other thing was to copy the output into a fresh session, ask for a description of what the code does and what improvements could be made)
Not saying that it’s a better way, but I started with vim by copying someone conf (on Github), removing all extraneous stuff, then slowly familiarizing myself with the rest. Then it was a matter of reading the docs when I wanted some configuration. I believe the first part is faster than dealing with an LLM, especially when dealing with an unfamiliar software.
I agree with this approach generally, but I needed to use some lua plugins to do something specific fairly quickly, and didn't feel like messing around with it for weeks on end to get it just right.
The downside of this nvim solution is the same downside as both pasting big blobs of ai code into a repo, and, pasting big vim configs you find online into your vimrc: inability to explain the pasted code.
When you need something fast for whatever reason sure, but later when you want to tweak or add something, you'll have to finally sit down and learn basically the whole thing or at least a major part of it to do so anyway. Imo it's better to do that from the start but sometimes that's not always ideal.
My data science friend tells me it's really good at writing bad pandas code because it's seen so much bad pandas code.
At the end of the day, it depends where you are in the hierarchy. Having it write code for me on a hobby project in react that's bad but works is one thing. I'm having a lot of fun with that. Having it write bad code for me professionally is another thing though. Either way, there's no going back to before ChatGPT, just like there's no going back to before Stack Overflow or Google. Or the Internet.
Maybe this is a case where more training data isn’t better. There is probably a lot of bad/old C++ out there in addition to new/modern C++, compared to Rust which is relatively all modern.
This is partly why I believe OS makers, Apple, Microsoft, Google, have a huge advantage in the future when it comes to LLMs.
They control the OS so they can combine and feed all your digital information to an LLM in a seamless way. However, in the very long term, I think their advantage will go away because at some point, LLMs could get so good that you don't need an OS like iOS anymore. An LLM could simply become standalone - and function without a traditional OS.
Therefore, I think the advantage for iOS, Android, Windows will increase in the next few years, but less powerful after that.
An LLM is an application that runs on an operating system like any other application. That the vendor of the operating system has tied it to the operating system is purely a marketing/force-it-onto-your-device/force-it-in-front-of-your-face play. It's forced bundling, just like Microsoft did with Internet Explorer 20 years ago.
I predict that OpenAI will try to circumvent iOS and Android by making their own device. I think it will be similar to Rabbit R1, but not a scam, and a lot more capable.
They recently hired Jony Ive on a project - it could be this.
I think it'll be a long term goal - maybe in 3-4 years, a device similar to the Rabbit R1 would be viable. It's far too early right now.
Even if this is true (and I'm not saying it's not), they probably won't create their own OS. They'd be smarter to do what Apple did and clone a BSD (or similar) rather than start afresh.
Yeah, Glean does this and there are a bunch of other competitors that do it as well.
I think you may be confused about the length of the context window. These tools don't pull all of your Slack history into the context window. They use a RAG approach to index all of your content into a vector DB, then when you make a query only the relevant document snippets are pulled into the context window. It's similar for example to how Cursor implements repository-wide AI queries.
I'm aware that one can't feed millions of messages into an LLM all at once. The only way to do this now is to use a RAG approach. But RAG approach has pros and cons and can miss crucial information. I think context window still matters a lot. The bigger the window, the more information you can feed in and the quality of answer should increase.
The point I'm trying to make is that increase context window will require more compute. Hence, we could still just be in the beginning of the compute/AI boom.
We might be even earlier — the 90s was a famous boom with a fast bust, but to me this feels closer to the dawn of the personal computer in the late 70s and early 80s: we can automate things now that were impossible to automate before. We might have a long time before seeing diminishing returns.
My workpalce uses Glean and since it was connected to Slack it has become significantly worse. It routinely gives incorrect or VERY incomplete information, misattributes work to developers who may have casually mentioned a project at some time and worst of all presents jokes or sarcastic responses as fact.
Not only is it an extremely poor source of information, it has ruined the company's Slack culture as people are no longer willing to (for lack of a better term) shitpost knowing that their goofy sarcasm will now be presented to Glean users as fact.
> the more compute you have, the smarter and more capable AI
Well, this is taken on faith by OpenAI/etc, but obviously the curve has to flatten at some point, and appears to already be doing so. OpenAI are now experimenting with scaling inference-time compute (GPT-O1), but have said that it takes exponential increases in compute to produce linear gains in performance, so it remains to be seen if customers find this a worthwhile value.
There is only so much that an approach like O1 can do, but anyways in terms of AI boom/bust the relevant question is whether this is a viable product. All sorts of consumer products could be improved by making them a lot more expensive, but there are cost/benefit limits to everything.
GPT-5 and Claude-4 will be interesting, assuming these are both pure transformer models (not COT), as they will be a measure how much benefit remains to be had from training set scaling. I'd expect gains will be more against narrow benchmarks, than in the overall feel of intelligence (LLM arena score?) one gets from the model.
I think OpenAI has already proven that it's a viable product. Their gross margins must be decent. I doubt they're making a loss for every token they inference.
I don't think they've broken out O1 revenue, but it must be very small at the moment since it was only just introduced. Their O1-preview pricing doesn't seem to reflect the exponential compute cost, so perhaps it is not currently priced to be profitable. Overall, across all models and revenue streams, their revenue does exceed inference costs ($4B vs $2B), but they still are projected to lose $5B this year, $14B next year, and not make a profit until 2029 (and only then if they've increased revenue by 100x ...).
Training costs are killing them, and it's obviously not sustainable to keep spending more on research and training than the revenue generated. Training costs are expected to keep growing fast, while revenue per token in/out is plummeting - they need massive inference volume to turn this into a profitable business, and need to pray that this doesn't turn into a commodity business where they are not the low cost producer.
The thing is that OpenAI can choose to spend less on training at any time.
We've seen this before, with for example Amazon where they made a deliberate effort to avoid profitability by spending as much as possible on infrastructure until the revenue became some much they couldn't spend it.
Being in a position where you are highly cash-flow positive and it's strategic investment that is the cost seems like a good position.
> Does Slack sell an LLM chatbot solution that is able to give me reliable answers to business/technical decisions made over the last 2 years in chat?
Note that the presence of such a feature isn't the same as whether it's secure enough for normal use.
In particular, anything anyone said in the last 2 years in chat could poison the LLM into exfiltrating your data or giving false results chosen by the attacker, because of the fundamental problems of LLMs.
You can never really tell, though following some market tea leaf readers they seem to think a few months from now, after a bit of a run up in the market. Here's one random datapoint, on mentions of "soft landing" in Bloomberg https://x.com/bravosresearch/status/1848047330794385494
Well yeah there may be a bit of that. I find them quite interesting for the data they bring up like the linked tweet but I don't really have an opinion as to whether they are any good at predicting things.
I was thinking re the data in the tweet, that there were a lot of mentions of "soft landing" before the dot com crash, before the 2006 property crash and now, it is quite likely there was an easy money policy preceding them and then government policy mostly focuses on consumer price inflation and unemployment, so they relax when those are both low and then hit the brakes when inflation goes up and then it moderates and things look good similar to now. But that ignores that easy money can also inflate asset prices, eg dot com stocks, houses in 06, or money losing AI companies like now. And then at some point that ends and the speculative asset prices go down rather than up, leaving people thinking oh dear we've borrowed to put loads of money into that dotcom/house/ai thing and now it's not worth much and we still have the debts...
> Reading this makes me willing to bet that this capital intensive investment boom will be similar to other enormous capital investment booms in US history, such as the laying of the railroads in the 1800s, the proliferation of car companies in the early 1900s, and the telecom fiber boom in the late 1900s. In all of these cases there was an enormous infrastructure (over) build out, followed by a crash where nearly all the companies in the industry ended up in bankruptcy, but then that original infrastructure build out had huge benefits for the economy and society as that infrastructure was "soaked up" in the subsequent years.
> I think that will happen here.
Why? The rail network, road network and fiber network that was laid could be used for decades after their original investors went bust.
The current datacenters full of AI compute can't really be used for anything else if AI companies go bust.
That's the problem with investing in compute infrastructure - you need to have a plan to use it all up in the next 5 years, because after that you wouldn't even be able to give it away.
The renewable power infrastructure for those datacenters will still exist.
People will be able to buy those used GPUs cheap and run small local LLMs perhaps. A 10 year old computer today won't do state of the art games or run models, but is entirely acceptable for moderate computing use.
> but now that renewables are the cheapest form of energy, I believe this huge, well-funded demand will accelerate the growth of non-carbon energy sources
I think the renewables would have been built at the same rate anyway precisely because they're so cheap; but nuclear power, being expensive, would not be built if this bubble had not happened, and somehow nuclear does seem to be getting some of this money.
I want to point out to anyone who's interested in the nuclear angle that even before the AI data center demand story arrived, the uranium market was facing a persistent undersupply for the first time in its many decades of history. As a result, the (long-term contract) price of uranium has been steadily rising for years: https://www.cameco.com/invest/markets/uranium-price
After Fukushima (https://news.ycombinator.com/item?id=41768726), Japanese reactors were shut down and there was a glut of uranium available in the spot market. Simultaneously, Kazatomprom flooded the market with cheap ISR uranium. The price of uranium fell far below the cost of production and the mining companies were obliterated. The few miners that survived via their long-term contracts (primarily Cameco) put their less efficient mines into care and maintenance.
Here is a discussion of the uranium mining situtation:
https://news.ycombinator.com/item?id=41661768 (including a very risky method of profiting from the undersupply of uranium, stock ticker SRUUF, not recommended). Note that Numerco's uranium spot price was put behind a paywall last week. You can still get the intra-day spot uranium price for free here: https://www.yellowcakeplc.com/
It is also important that the fuel is physically small so you can (and typically, do) store years of fuel on-site at the reactor. Nuclear is "secure" in the sense that it can provide "energy security".
I already linked this above, twice. I know it's a hassle to read, it's Sunday afternoon, so don't worry about it.
It's not important whether you as an individual get this right or not, as long as society reaches the correct conclusion. Thankfully, we're seeing that happen, a worldwide shift toward the adoption of nuclear power.
Not at all. Old paid of nuclear plants are competitive but new builds are insanely expensive leading to $140-220/MWh prices for the ratepayers before factoring in grid stability and transmission costs.[1]
The US has zero commercial reactors under construction and this is for one reason: economics.
The recent announcements from the hyperscalers are PPAs. If the company building the reactor can provide power at the agreed price they will take it off their hands. Thus creating a more stable financial environment to get funding.
They are not investing anything on their own. For a recent example NuScale another SMR developer essentially collapsed when their Utah deal fell through when nice renders and PowerPoints met real world costs and deadlines. [2]
That cost is excluding grid stability and transmission costs.
From what I’ve understood PG&E’s largest problem is the massive payouts and infrastructure upgrades needed from the wildfires, not the cost of the electricity itself.
Thanks! I always thought it is due to people's safety concerns here instead of economic reasons. After all, nuclear plant is quite 'popular' in Europe, and China too these days.
Basically, nuclear fission is clean baseload power. Wind and solar are not baseload power sources. They don't really compete. See discussion here: https://news.ycombinator.com/item?id=41858892
TRISO (HALEU) reactors use more than 1.5 times the natural uranium per unit of energy produced because the higher burnup is offset by higher enrichment inputs (see page 11 at https://fuelcycleoptions.inl.gov/SiteAssets/SitePages/Home/1...), and the fuel is even more expensive to manufacture, but they are completely safe. This is a technology from the 1960's but it's attractive now because so much money is chasing clean baseload nuclear fission for data centers.
These "impossible to melt down" TRISO small modular nuclear fission reactors are what Elon Musk was talking about on the campaign trail last week, when he said:
ELON MUSK: "The dangers of nuclear power are greatly
overstated. You can make a nuclear reactor that is
literally impossible to melt down even if you tried to
melt it down. You could try to bomb the place, and it
still wouldn't melt down. There should be no regulatory
issues with that. There should be significant nuclear
reform."
> Basically, nuclear fission is clean baseload power. Wind and solar are not baseload power sources. They don't really compete.
This means you don't understand how the grid works. California's baseload is ~15 GW while it peaks at 50 GW.
New built nuclear power is wholly unsuitable for load following duty due to the economics. It is an insane prospect when running at 100% 24/7, and even worse when it has to adapt.
Both nuclear power and renewables need storage, flexibility or other measures to match their inflexibility to the grid.
See the recent study where it was found that nuclear power needs to come down 85% in cost to be competitive with renewables, due to both options requiring dispatchable power to meet the grid load.
> The study finds that investments in flexibility in the electricity supply are needed in both systems due to the constant production pattern of nuclear and the variability of renewable energy sources. However, the scenario with high nuclear implementation is 1.2 billion EUR more expensive annually compared to a scenario only based on renewables, with all systems completely balancing supply and demand across all energy sectors in every hour. For nuclear power to be cost competitive with renewables an investment cost of 1.55 MEUR/MW must be achieved, which is substantially below any cost projection for nuclear power.
> These companies want clean baseload power, with no risk of meltdown, and they're willing to pay for it. Here's what Amazon has chosen
The recent announcements from the hyperscalers are PPAs. If the company building the reactor can provide power at the agreed price they will take it off their hands. Thus creating a more stable financial environment to get funding.
They are not investing anything on their own. For a recent example NuScale another SMR developer essentially collapsed when their Utah deal fell through when nice renders and PowerPoints met real world costs and deadlines.
Then we should be able to remove the enormous subsidy the Price Anderson act adds to the industry right? Let all new reactors buy insurance for a Fukushima level accident in the open market.
Nuclear powerplants are currently insured for ~0.05% of the cost of a Fukushima style accident and pooled together the entire US industry covers less than 5%.
Railroads and computer networks create network effects, I am not sure the same is true for data centers full of hardware that becomes outdated very quickly.
If they're building new power plants to support all those data centers than that power generation capacity might be put to good use doing something else.
>What will be the advantage of having a bunch of obsolete hardware? All I see is more e-waste.
The energy build out, data centers are not wasted. You can swap out A100 GPUs for BH200 GPUs in the same datacenter. A100s will be 5 years old when Blackwell is out - which is just about right for how long datacenter chips are expected to last.
I do, however, think that the industry will move to newer hardware faster to try to squeeze as much efficiency as possible due to the energy bottleneck. Therefore, I expect TSMC's N2 nodes to have huge demand. In fact, TSMC themselves have said designs for N2 far outnumber N3 at the same stage of the node. This is most likely due to AI companies that want to increase efficiency due to the lack of electricity.
Well, at least they are paving the way to more efficient hardware, GPU are way, way more energy efficient than CPU and the parallel architectures are the only remaining way to scale compute.
But yes, a lot of energy wasted in the growing phase.
>But yes, a lot of energy wasted in the growing phase.
Why exactly is energy wasted during this phase?
Are you expecting hardware to become obsolete much faster? But that only depends on TSMC's node cadence, which is still 2-3 years. Therefore, AI hardware will still be bound to TSMC's cadence.
In the case of dark fiber the hardware was fine, wavelength division multiplexing was created which increased capacity by 100x in some cases crashing demand for new fiber.
I think OP is suggesting AI algorithms and training methods will be improve resulting in enormous performance gains with existing hardware causing a similar surplus of infrastructure and crash in demand.
How much of current venture spending is going into reusable R&D that can be moved forward in time the way that physical infrastructure in their examples were able to be used in the future?
Even if the hardware quickly becomes outdated, I'm not sure it'll become worthless so quickly. And there's also the infrastructure of the data center and new electricity generation to power them. Another thing that might survive a crash and carry on to help the future is all the code used to support valuable use cases.
I don't think the parent was specifically referring to hardware alone. The 'rails' in this context are also the AI algorithms and the underlying software. New research and development could lead to breakthroughs that allow us to use significantly less hardware than we currently do. Just as the dot-com crash wasn’t solely about the physical infrastructure but also about protocols like HTTP, I believe the AI boom will involve advancements beyond just hardware. There may be short-term excess, but the long-term benefits, particularly on the software side, could be immense.
Maybe the bust will be so rough that TSMC will go out of business, and then these graphics cards will not go obsolete for quite a while.
Like Intel and Samsung might make a handful of better chips or whatever, but neither of their business models really involve being TSMC. So if the bubble pop took out TSMC, there wouldn’t be a new TSMC for a while.
Similarly, I like to compare AI (more specifially LLM) "investment" to the cost of building the channel tunnel between UK and France. The original investors lost their shirt, but once built it is profitable to operate.
Building robust LLM-based applications is token-intensive. You often have to plan for the parsing and digestion of a lot of tokens for summarization or even retrieval augmented generation. Even the mere generation of marketing blogposts consumes a lot of output tokens in most cases. Not to mention that all robust cognitive architectures often rely on the generation of several samples for each prompt, custom retry logics, feedback loops, and reasoning tokens to achieve state of the art performance, all solutions powerfully token-intensive.
Luckily, the cost of intelligence is quickly dropping. GPT-4, one of OpenAI’s most capable models, is now priced at $2.5 per million input tokens and $10 per million output tokens. At its initial release in March 2023, the cost was respectively $10/1M input tokens and $30/1M for output tokens. That’s a huge $7.5/1M input tokens and $20/1M output tokens reduction in price.
https://www.lycee.ai/blog/drop-o1-preview-try-this-alternati...
While I agree with your essential conclusions, I don't think the automobile companies really fit. Many of the early 1900s companies (e.g. Ford, GM, Mercedes, even Chrysler) are still among the largest auto companies in the world.
There were hundreds of failed automotive companies and parts suppliers though. I think the argument is that many will die, some will survive and take all (most)
There were huge winners in cars. Ford and GM have historically been huge companies. Then oil companies became the biggest companies in the world mostly due to cars.
GM went bankrupt. Ford would have without government intervention. Each have had periods of profitability but they weren't ever anything like microsoft/google etc. Ford has underperformed the stock market average since it went public like 70 odd years ago. GM got so big in the first place via acquisitions, not because the business of cars lent itself to a dominant player.
> While I agree with your essential conclusions, I don't think the automobile companies really fit.
Many of the early 1900s companies (e.g. Ford, GM, Mercedes, even Chrysler) are still among the largest auto companies in the world.
American Automative filled for Bankruptcy multiple times.
American Government had to step in to back them up and bail them out.
That phase is called consolidation. It’s part of the cycle. The speculative over leveraged and mismanaged companies get merged into the winners or disappear if they have nothing of value.
Something people forget is that a training cluster with tens of thousands of GPUs is a general purpose supercomputer also! They can be used for all sorts of numerical modelling codes, not just AI. Protein folding, topology optimisation, route planning, satellite image processing, etc…
We bought a lot of shovels. Even if we don’t find more gold, we can dig holes for industry elsewhere.
I think there is an LLM bubble for sure, but I’m very bullish on the ease with which one can generate new specialized models for various tasks that are not LLM.
For example, there’s a ton of room for developing all kinds of low latency, highly reliable, embedded classifiers in a number of domains.
It’s not as gee-whiz/sci-fi as an LLM demo, but I think potentially much bigger impact over time.
Not all of that infrastructure gets soaked up, plenty is abandoned. Look at the state of american passenger rail for example and how quickly the bottom of that industry dropped out. Many old rail right of ways sit abandoned today. Likewise with telecoms, e.g. the microwave network that also sits abandoned today.
There is a comment on this thread about this being like the railroads, but this is nothing like the railroads except insofar as it costs a lot of money.
The railroads have lasted decades and will remain relevant for many more decades. They slowly wear out, and they are the most efficient form of land transport.
These hardware investments will all be written off in 6 years time and won't be worth running given the power costs and relative output. They will be junked.
There's also the extra risk that for some reason future AI systems just don't run efficiently on current gen hardware.
Maybe, maybe not. Nvidia P40's GPUs were released 8 years ago, they are hot. Prices have doubled in the last year. Nothing will be written off in 6 years. Folks will be using A100 15 years from now. 7 yrs old V100 32gb GPU cards are still going for $1500 on ebay. We are more likely to invent a more efficient software architecture than invent a new type of better hardware and replace everything that exists.
Power plants and power infrastructure are probably an example of a positive consequence that comes from this.
We have been terrified to whisper the words "nuclear power" for decades now, but the AI boom is likely to put enough demand on the power grid that it forces us to face this reality and make appropriate buildouts.
Even if the AI Boom crashes, these power plants will have positive impacts on the country for decades, likely centuries to come. Keeping bountiful power available and likely low-cost.
So there is a boom in terms of investment for future capacity. It could be interesting to see if the demand is following. I suspect there will be over capacity built, resulting some bankruptcies + cheaper compute/ai costs, hopefully allowing the next generation of startups or companies to flourish later on.
It feels like there’s a big gap in this discussion. The focus is almost entirely on GPU and hardware investment, which is undeniably driving a lot of the current AI boom. But what’s missing is any mention of the software side and the significant VC investment going into AI-driven platforms, tools, and applications. This story could be more accurately titled ‘The GPU Investment Boom’ given how heavily it leans into the hardware conversation. Software investment deserves equal attention
I think a better analogy is the valuation of Intel vs. that of Microsoft. For a long time, Intel dominated the CPU market. So you'd expect them to be the most valuable company. Instead, a small startup called Microsoft started dominating the software market and eventually became the most valuable company on the planet. The combined value of the software market is orders of magnitudes larger than that of all chip makers combined and has been for quite some time. The only reason people buy hardware is to use software.
The same is going to happen with AI. Yes, Nvidia and their competitors are going to do well. But most of the value will be in software ultimately.
GPUs and data centers are just infrastructure. Same for the electricity generation needed to power all that. The demand for AI is causing there to be a lot of demand for that stuff. And that's driving the cost of all of it down. The cheapest way to add power generation is wind and solar. And both are dominating new power generation addition. Chip manufacturers are very busy making better, cheaper, faster etc. chips. They are getting better rapidly. It's hard to see how NVidia can dominate this market indefinitely.
AI is going to be very economical long term. Cheap chips. Cheap power. Lots of value. That's why all the software companies are busy ramping up their infrastructure. IMHO investing in expensive nuclear projects is a bit desperate. But I can see the logic of not wanting to fall behind for the likes of Amazon, Google, MS, Apple, etc. They can sure afford to lose some billions and it's probably more important to them to have the power available quickly than to get it cheaply.
Nvidia is a software company that also happens to make decent hardware. People buy their hardware because of their software. And it’s not just CUDA. Nvidia is building a whole bunch of potentially groundbreaking software products. Listen to the last GTC keynote to learn more about it.
The actual physical fiber was useful after the companies popped though.
GPUs are different, unless things go very poorly, these GPUs should be pretty much obsolete after 10 years.
The ecosystem for GPGPU software and the ability to design and manufacture new GPUs might be like fiber. But that is different because it doesn’t become a useful thing at rest, it only works while Nvidia (or some successor) is still running.
I do think that ecosystem will stick around. Whatever the next thing after AI is, I bet Nvidia has a good enough stack at this point to pivot to it. They are the vendor for these high-throughput devices: CPU vendors will never keep up with their ability to just go wider, and coders are good enough nowadays to not need the crutch of lower latency that CPUs provide (well actually we just call frameworks written by cleverer people, but borrowing smarts is a form of cleverness).
But we do need somebody to keep releasing new versions of CUDA.
But computer chips have always had limited usefulness because newer chips are simply faster and more efficient. The datacenter build outs and increase in electricity capacity will always be useful.
Is there a good estimate of the split? My impression from AI startups whose operations I know something about is that a majority of their VC raise is currently going back into paying for hardware directly or indirectly, even though they aren’t hardware startups per se, but I don’t have any solid numbers.
I think your analysis is spot on, based on my expertise of "reading too much hacker news since every day since March 2023". This Bain report[1] barely mentions "Independent Software Vendors" and when they do it's clearly an afterthought, and the only software focused takes I can find are[2] extremely speculative, e.g.
"For every dollar invested in hardware, we expect 8-to-20 times the amount to be spent on software... While the initial wave of AI investment lays the foundational infrastructure, the next wave is clearly set to capitalise on the burgeoning AI software market."
I'm hoping someone more knowledgeable about capital markets can fill us in here, I'd be curious to see some hard numbers still! Maybe this is what a Bloomberg terminal does...?
Regardless, I think this makes a lot of sense; there's no clear scientific consensus on the path forward for these models other than "keep going?", so building out preparatory infrastructure is seen as the clear, safe move. As the common refrain goes: "in a gold rush, sell shovels!"
As a big believer in the upcoming cognitive era of software and society, I would only add a short bit onto the end of that saying: "...until the hydraulic mining cannons[3] come online."
The AI bubble will pop in the next year. We are currently in 1998 of the dotcom bubble with another AI winter approaching. LLM and generative AI are this year’s “on the Internet” or “Uber for X” business plans.
The market PE is very high and has been high for a while. Tech and AI are fueling a lot of that. Look what happened to Tesla when the fundamentals started getting a little discouraging. However, I'm not comfortable predicting the "AI bubble" popping next year.
I'm curious how this will affect cloud costs for the rest of us. On the one hand, we may get some economies of scale, but on the other hand, cloud resources being used up by others may drive prices up. Does anyone have any guesses as to what will happen?
I doubt it will increase cost for traditional CPU-based clouds. Just take a look at Ampere 192 core and AMD 196 core CPUs. Their efficiency will continue to drive down traditional cloud $/perf.
I've yet to find an "AI" that doesn't seamlessly hallucinate, and I don't see how "AIs" that hallucinate will ever be useful outside niche applications.
Confabulations are decreasing with newer models. I tested confabulations based on provided documents (relevant for RAG) here: https://github.com/lechmazur/confabulations/.
Note the significant difference between GPT-4 Turbo and GPT-4o.
There's a camp of people who are hyper-fixated on LLM hallucinations as being a barrier for value creation.
I believe that is so far off the mark for a couple reasons:
1) It's possible to work around hallucinations in a more cost effective way than relying on humans to always be correct.
2) There are many use cases where hallucinations aren't such a bad thing (or even a good thing) for which we've never really had a system as powerful as LLMs to build for.
There's absolutely very large use cases for LLMs and it will be pretty disruptive. But it will also create net new value that wasn't possible before.
I say that as someone who thinks we have enough technology as it is and don't need any more.
For sure, sending customers into a never ending loop when they want support. That's been my experience with most AI support so far. It sucks. I like Amazon's approach where they have a basic chat bot (probably doesn't even use LLMs) that then escalates to a actual human being in some low cost country.
I kind of like the Chipotle approach. I have a problem with my order, it just refunds me instantly and sometimes gives me a add-on for free.
Honestly I only use LLM for one thing - I give it a set of TS definitions and user input, and ask it to fit those schemas if it can and to not force something if it isn't 100% confident.
I know some people whose whole company is based around the use of AI to send emails or messages, and in reality they're logged into their terminals real time fixing errors before actually sending out the emails. Basically, they are mechanical turks and they even say they're looking at labor in India or Africa to pay them peanuts to address these.
I believe that there is lot of content creation where quality really does not matter. And hallucinations don't really matter. Unless they are legally actionable, that is something like hate speech or libel.
Throwing dozens of articles, social media posts and why not even videos. Hallucinations really don't matter at scale. And enough content is already generating enough views to make it somewhat viable strategy.
I've never met a human that doesn't "hallucinate" either (either intentionally or unintentionally). Humans either intentionally lie or will fill in gaps in their knowledge with assumptions or inaccurate information. Most human generated content on social media is inaccurate, to an even higher percentage than what ChatGPT gives me.
I guess humans are worthless as well since they are notoriously unreliable. Or maybe it just means that artificial intelligence is more realistic than we want to admit, since it mimics humans exactly as we are, deficiencies and all.
This is kind of like the self-driving car debate. We don't want to allow self-driving cars until we can guarantee that they have a zero percent failure rate.
Meanwhile we continue to rely on human drivers which leads to 50,000 deaths per year in America alone, all because we refuse to accept a failure rate of even one accident from a self-driving car.
It's not quite the case with cars though - people are ok with Waymos which are not zero accident but probably safer than human drivers. The trouble with other systems like Tesla FSD is they are probably not safer than human yet if you don't have a human there nannying them.
Similarly I think people will be ok with other AI if it performs well.
Humans also confabulate (a better metaphor for AI errors than hallucination) when called on to respond without access to the ground truth, and most AI models have limited combination of access and ability to use that access when it comes to checking ground truth.
I remember listening to Dr Robert Martin who was leading Bell Labs in the late 90s and he talked about how the bandwidth capacity was pushing to infinity while cost per bit was pushing towards zero and we all know how that ended for the optical capacity builders of that time before the bubble popped. Is there a case for intelligence being inexhaustible to demand ? Is there a case for, as Sama says, the cost for intelligence as in input to a system converges with the price of electricity needed to power the gpu behind it in the daya center? Yes and yes. Still the same could be said for a bit of bandwidth.
I am very skeptical of the positive effects of infite intelligence on the living standards of knowledge workes.
On the more pessimistic end, AI will replace us and we'll be sent to the coal mines.
On the possibly most optimistic end, living standards are a composite of many things rooted in reality, so I'd say the actual cap is about doubling of life quality, which is not nothing, but not unprecendented if we look at the past century and a half.
The ideal scenario in my mind is that intelligent and benevolent AI takes over running things and then people have to figure out how to make their life meaningful with only leisure time.
There is no long-term time scale in which humans running things do not obliterate each other or end up sitting on a planet filled with trash.
Either we figure out how to colonize other planets or we hand over the reigns to something that can plan long term and not be irrational.
Maybe if we figure out immortality it might work, but with the short life span of a human there is no way to not be short-sited or eventually end up with the wrong person in charge the button.
Honestly I don't like these AI doom scenarios, mostly because I feel like they're used as a means by the people in charge to shift the Overton window of AI outcomes - as in, the AI might have built an ubiquitious police state with all the wealth being owned by the 0.001%, but at least it didn't kill us all!
Honestly, I feel like with the recent progress of AI, it's a realistic scenario to assume it will replace mpst knowledge workers in the next 5 to 10 years, probably won't replace researchers and other elite intellectuals, and won't even make a dent in the world of physical labor.
In that world, I see AI as harmful, but the people in charge won't, as they are directly benefiting from it.
We live in number-go-up capitalism. A good analog is the housing situation. The ever increasing price of real estate means that the total amount of wealth goes up, so it's seen as a beneficial process by the elites. The rest however will find that they need to dedicate a larger proportion of their income towards getting a roof over their heads, and think this process is bad.
Nowadays, the possibility building a life that would've been considered middle class half a century ago from scratch is available to like 10% of workers, working mainly intellectual jobs.
AI in the future will reduce the proportion of these people by taking away their high paying jobs.
> AI products are used ubiquitously to generate code, text, and images, analyze data, automate tasks, enhance online platforms, and much, much, much more—with usage expected only to increase going forward.
Why does every hype article start with this.
Personally my copilot usage has gone down while coding. I tried and tried but it always gets lost and starts spitting out subtle bugs that takes me more time to debug than if i had written it myself.
I always have this feeling of 'this might fail in production in unknown ways' because i might have missed checking the code throughly . I know i am not the only one, my coworkers and friends have expressed similar feelings.
I even tried the new 'chain of thought' model, which for some reason seems to be even worse.
This just reminded I forgot I have had a Copilot subscription. It has not made any useful code suggestions in months to the point of fading from my memory. I just logged in to cancel it. Now, I need to check my other subscriptions that I can cancel or reduce to a lower tier.
They have massively nerfed Copilot. I'm keeping my subscription for a couple more months but at this point, it has the same intelligence as the llma3.2 which I can run on my laptop.
My experience is similar. I used Claude for a coding task recently and it drove me into an infinite number of rabbit holes, each one seeming worse than the previous one. All the while being enable to stop and say: I’m sorry, I actually don’t know how to help you.
There is a market for AI. And it is exactly these articles and maybe pictures attached to them. Soon could be some videos as well. But how far beyond that. Is very good question.
From my experience, it is getting better over time, and I believe that there's still a lot of relatively low hanging fruit, particularly in terms of integrating the LLM with the language server protocol and other tooling. But having said that, at this point in time, it's just not good enough for independent work, so I would suggest using it only as you would pair-program with a mid-level human developer who doesn't have much context on the project, and has a short attention span. In particular, I generally only have the AI help me with one function/refactoring at a time, and in a way that is easy for me to test as we go, and am finding immense value.
The problem is all the most reliable code it can give you is stuff which ought to be (or already is) a documentation example or a reusable library, instead of "copy paste as a service".
I think some of the consternation we see from the anti LLM crowd (of which I'm one) is this line of reasoning. These LLMs produce fine code when the code you are asking for is in its training set. So they can be better than a mid level dev and much faster in narrow, unknown contexts. But with no feedback to warn you, if you ask it for code that it has no or only a bit of data on, it is much worse than a rubber duck.
That and tech's status inflation means when we are talking about "mid level" engineers, really we are talking about engineers with a couple years of experience who have just graduated to the training wheels phase of producing production code. LLMs are still broadly aimed at removing the need for what I would just call junior engineers.
That and the fact that code does not live in a standalone bubble, but in a complex setup of OSes, APIs, middleware and other languages. My experience trying to use Claude to help me with that was disappointing.
I see the same results as my TabNine+Template generator+language server as I do with things like CoPilot. I get TabNine issues when the code base isn’t huge. I think also tossing away language servers and template generators for just LLM will just lead to seeking “proper predictive path”. Most of the time the LLM will spit out the create-express/react-template for you, when you ask it to customize it will guess using the most common patterns. Do you need something to guess for you?
It’s also getting worse because people are poisoning the well.
It’s kinda true though. They are increasingly used for those things. Sure, the results are terrible and doing it without AI almost always yields better results but that doesn’t seem to stop people.
My feeling is (current) AI is more of a teacher than an implementor. It really does help when learning about something new or to give you ideas about directions to take. The actual code however still needs to be written by humans for the most part it seems.
AI is a great tool and does speed things up massively, it just doesn't align with the magical thought that we provide the ideas and AI does all of the grunt work. In general, always better to form mental models about things based on actual evidence as opposed to fantasy (and there is a lot of fantasy involved at the moment). This doesn't mean being pessimistic about potential future advancements however. It is just very hard to predict what the shape of those improvements will be.
Yeah, it's actually frustrating that even when writing Go code, which is statically typed, it keeps messing up the arguments order. That would seem to me a pretty easy thing to generate.
Although it's much better when writing standard REST and gRPC APIs
LLMs are great for simple, common tasks, i.e. CRUD apps, RESTful web endpoints, unit tests, for which there's an enormous amount of examples and not much unique complexity. There's a lot of developers whose day mostly involves these repetitive, simple tasks. There's also a lot of developers who work on things that are a lot more niche and complicated, where LLMs don't provide much help.
In my experience this underrates them. They can do pretty complex tasks that go well beyond your examples if prompted correctly.
The real limiting factor is not so much task complexity as the level of abstraction and indirection. If you have code that requires following a long chain of references to understand, LLMs will struggle to work with it.
For similar reasons, they also struggle with:
- generic types
- inheritance hierarchies
- long function call chains
- dependency injection
- deeply nested structures
They're also bad at counting, which can be an issue when dealing with concurrency—i.e. you started 5 operations concurrently at different points in your program and now need to block while waiting for 5 corresponding success or failure messages. Unless your code explicitly uses the number 5 somewhere, an LLM is often going to fail at counting the operations.
All in all, the main question I think in determining how well an LLM can do a task is whether the limiting factor for your task is knowledge or abstraction. If it's knowledge (the intricacies of some arcane OS API, for example), an LLM can do very well with good prompting even on quite large and complex tasks. If it's abstraction, it's likely to fail in all kinds of seemingly obvious ways.
> If it's knowledge (the intricacies of some arcane OS API, for example), an LLM can do very well
Only if that knowledge is sufficiently represented in the training data or on the web. If, on the other hand, it’s knowledge that isn’t well (or at all) represented, and instead requires experience or experimentation with the relevant system, LLMs don’t do very well. I regularly fail with applying LLMs to tasks that turn out to require such “hidden” knowledge.
And if it's really well represented, then it's hopefully already in a superior library or documentation/guide, and the LLM is acting as an (untrustworthy) middleman.
Yeah, exactly this. If I ask Cursor to write the serializer for a new Django model, it does it (although sometimes it invents fields that do not exist). It saves me 2 minute.
When I ask him to write a function that should do something much more complex, it usually do something so bad it takes me more time because it confuses me and now I have to back to my original reasoning (after trying to understand what it did).
What I found useful is to ask him to explain me what a function does in a new codebase I am exploring, although I have to be very careful because a lot of time invents or skips steps that are crucial.
See, I recently picked up the Ash framework for elixir and it does all that too, but in a declarative, precise language which codegens the implementation in a deterministic way.
It just does the job that cursor does there, but better.
Maybe us programmers should focus on making higher order programming tools instead of black box text generators for existing tools.
> LLMs are great for simple, common tasks, i.e. CRUD apps, RESTful web endpoints
i gave it a yaml and asked it to generate a json call to rest api . It missed a bunch of keys and made up a random new key. I threw out the whole thing and did it with awk/sed.
Seriously though, what are you doing?
Every single example everywhere throughout the internet that tries to show how good AI is at programming shows so mindbogglingly simplistic examples that it's getting annoying.
It sure is a great learning tool when you're trying to do something experimental in a new stack or completely new project, I'll give you that, but once you reach the skill level where someone would hire you to be an X developer (which most developers disagreeing with you are, mid+ developers of some stack X) the thing becomes a barely useful autocomplete.
Maybe that's the problem? It's just not a tool for professional developers?
I mean, let me just throw in an example here: I am currently working on https://guesspage.github.io , which is basically https://getguesstimate.com but for flowtext instead of a spreadsheet. The site is ... 99.9% Claude Sonnet written. I have literally only been debugging and speccing.
Sonnet can absolutely get very confused and break things. And there were tasks where I had a really hard time getting it to do the right thing, or understand what I wanted. But I need you to understand: Sonnet made this thing for me in two and a half days of part-time prompting. That is probably ten times faster than it would have taken me on my own, especially as I have absolutely no design ability.
Now, is this a big project? No, it's like 2kloc. But I don't think you can call it "simple" exactly. It's potentially useful technology. This sort of "just make this small tool exist for me" is where I see most of the value for AI in the next year. And the definition of "small tool" can stretch surprisingly far.
This is a simple project. Nobody is disputing that GenAI can automate a large chunk of the initial setup work, which dominates the time spent on small projects like this. But 99.999% of professional, paid software development is not working on the basic React infrastructure for a 2,000 loc javascript app.
Also your Google Drive API key is easily discoverable with about 15 seconds of looking at the JS source code -- this is something a professional software developer would (hopefully) have picked up without you asking, but an LLM isn't going to tell you that you shouldn't ship the `const API_KEY = ...` code as a file to the client, because you didn't ask.
I mean, it would have taken me a lot longer on my own. Sure it's not a huge project, I agree; I wouldn't call it entirely trivial.
> Also your Google Drive API key is easily discoverable with about 15 seconds of looking at the JS source code
No, I'm aware of that. That's deliberate. There's no way to avoid it for a serverless webapp. (Note that Guesspage is entirely hosted on Github Pages.) All the data stored is public anyways, the key is limited to only have permission to access the stored data, and you still have to log in and grab a token that is only stored in your browser and cannot be accessed from other sites. Literally the only unique thing you can do with it is trigger a login request on your own site that looks like it comes from Guesspage; and you can do that just as easily by creating a new API key and setting its name to "Guesspage".
The AI actually told me that was unsafe, and I corrected it. To the best of my understanding, the only thing that you can do with the API key is do Google Drive uploads to your own drive or that of someone who lets you that look to Google as if my app is triggering them. If there's a danger that can arise from that, and I don't think there is, then it's on me, not on Sonnet.
(It's also referer domain limited, but that's worthless. If only there was a way to cryptographically sign a referer...)
I am happy with the LLMs, but I only tried them on small projects done at my free time.
As a back end developer I am not familiar with the latest trends in JavaScript and CSS, and frankly I do not want to spend my time studying these. A LLM can generate an interactive web game based on my description. I review the code, it is usually okay, sometimes I suggest an improvement. I could have done all of that -- but it would take me a week, and the LLM does it in seconds. So it is a difference between a hobby project done or not done.
I also tried a LLM at work, not to code, but to explain some complex topics that were new to me. Once it provided a great high-level description that was very useful. And once it provided a great explanation... which was a total lie, as I found out when I tried to do a hello-world example. I still think the 50% success rate is great, as long as you can quickly verify it.
Shortly, we need to know the strengths and the weaknesses, and use the LLMs accordingly. Too much trust will get you burned. But properly used, they can save a lot of time.
For my I have a company subscription for Copilot and I just use the line based autocomplete. It’s mildly better than the built in autocomplete. I never have it do more than though and probably wouldn’t buy a license for myself.
* tests themselves in a staging environment, independent of any QA team or reviews
* monitors the changes after they’ve gone out
* has repeatedly found things in their own PRs and asked to hold off release to fix them
* is reviewing other people’s PRs and spotting things before they go out
yea, sure, i’ll release the changes. they’re doing the auditing work for me.
they clearly care about the software. and i’ve seen enough to trust them.
and if they got it wrong, well, shit, they did everything good enough. i’m sure they’ll be on the ball when it comes to rolling it back and/or fixing it.
an llm does not do those things. an llm *does not care about your software* and never will.
i’ll take people who give a shit any day of the week.
I'd say it depends more on "the production" than the human.
There are legal means to hold all people accountable for their actions ("Gross neglience" and all that).
So you can basically always trust that people will fix what they messed up given the possibility.
So if you can afford for the production to be broken (e.g. the downtime will just annoy some people) you might as well allow your team to deploy straight to prod without audits. It's not that rare actually.
I just asked Claude to generate some code using the SAP SuccessFactors API.
Every single example was completely useless. The code wouldn't compile, it would invent methods and variables and the instructions to go along with it were incoherent. All whilst gaslighting along with the way.
I have also previously tried using it with some Golang code and it would constantly add weird statements e.g. locking on non-concurrent operations.
LLMs are great when you are doing the same things as everyone else. Step outside of that and it's far more trouble than it's worth.
I wonder what lessons the current hardware-intensive AI boom could learn from the history of proof-of-work (PoW) mining, particularly regarding energy consumption, hardware specialization, and market dynamics.
Yes, I agree wholeheartedly and I actively dislike the concept of LLMs for anything real. But nothing will be worse than the flood of outright scams or attempts at rent seeking/middle man creation that crypto was. At least LLMs have some potential use cases (and line level auto complete is genuinely better now because of LLMs)
> I will take AI bubble over crypto bubble any day of the week.
I've been in the former since '21 and have seen every single cycle since 2011 in the latter, I can assure their is more dumb money in the former than in the latter; (just by scale alone) at least in the latter whether it was ICO or NFTs or whatever mal-investment was promptly punished (rugpulls/exit scams) companies like INTEL get to stay in Zombie mode because of the corpo-welfare that the US doles out while shaming everyone else to be prudent with their investments while these corps and banks spend like drunken sailors and tries to strangle the former out of existence (rightly so in most cases as most crypto is a total scam).
With that said, what you will see emerge are some incredibly established players in both fields that will have the staying power to change how the Industry is shaped around them: Nvidia and Bitcoin are comparable to one another in that respect.
Both have/had crazy volatility but the staying power and just the fact they remain firmly at the center of both Industry's is rather telling that you simply don't see what these technologies offer because of the hype and boom and bust cycles.
As a person who directly benefits from this: I can assure you most of these VCs are exit liquidity just as most foolish people are for the alt scams of yore, execpt the US economy (likely all of the Western World at this point) isn't entirely reliant on the promise of vapourware with 'crypto' in any capacity, weheras the same cannot be said about the theatrics of Jensen's Nvidia.
Source: I build data center infrastructure for these mega corps doing 'AI' and I'm doing an MSc in CS (Big Data) and been a Bitcoiner since Satoshi was still on BTF.
You might be right, but the crypto people were/are basically religious in their responses. I see a few loons for LLM's talking about how AGI is near, and of course there's the EA/LessWrong people talking about doomsday. But none of them were as staunchly dug in _and_ misinformed as the crypto folks.
edit: if it isn't clear I'm a staunch opponent of cryptocurrency in any form.
> AI people sound more dug in to be honest from my perspective. But I guess that's cause the crypto stuff tends to be less overtly religious and more overtly batshit crazy politics and economics, which I'm much more used to dealing with haha. And mostly, everyone has figured out the scam by now on the crypto side.
>>The AI people freak me out cause they are all talking eschatology and shit as if they have stumbled upon the literal ark of the covenant like in raiders of the lost ark or something.
>>>It's a really great act to be honest. They've been clearly studying a lot of the more dishonest American religious culture of the last couple of decades.
I'm going to commit a HN faux pas to prove a point and show you why I think Bitcoin has a valid use case here alone: I decided to repost what he said because there are valid points here and are worth discussing.
Had I the inclination I can hash this into the blockchain for all to see what was written from this poster for the aforementioned reasons for as long as the mainchain continues to be maintained, protected and supported .
This has great utility, and the mere suggestion that you cannot get over that is because those 'crazies offend me and my disposition' and stop there you fail to see why and what this technology can already do--create an actual immutable archive of all Human history if we desire it.
But to his point, yes it's roots in Crypto-Anarchism (which started in CA at the inception of the rise of modern SV btw) has many of you questioning the ''sanity' and 'motives' behind this technology, and you assume they are all the same but rest assure their is a reason for the brain drain from all of tech/STEM/finance during my era and time in Bitcoin.
Most of whom are now incredibly wealthier than they ever were working in academia or private industry if you think money is a measure of one's success--I don't, but most of you do.
The AI people strike me more as a range of the introduction corpos from banking and academia into bitcoin (Gavin's, Hearn) to total con men like Veer and sprinkled in there are the cult memebers you mentioned who honestly think that their techno-utopian trans-humanist dreams are being built one LLM update at a time. It's sad... it's the same thing just different names/faces.
No one cares that blockchains are immutable, that doesn't mean that information written there is correct. Just that it was written on X date and the content. You could find proof of the biggest scandal of all time and post it on the blockchain so "the man" can't stop the word from getting out but 99.99999999% of readers would read a version presented by a simple web server and with the value cached in a closed database or memory. Which if the government wants to take down, it can. And if that does go down, no one will save a link to the entry on the blockchain. And on the flip side I could write obvious falsehoods in the same way. Blockchain provides no value for legal attestation nor information distribution.
In practice, the vast majority of blockchain ledgers record the history of scams, penny stock style trading and money laundering attempts.
> edit: if it isn't clear I'm a staunch opponent of cryptocurrency in any form.
It's very clear, but your exposure to zealots, on both sides, shouldn't deter you from being objective and seeing what these technologies actually offer. Hence why i wrote that part in the 2nd to last paragraph.
HN has such misinfored vitriol for any technology it didn't ordain itself, you sem to be of that cohort, what's odd is that very same people who gave you VC.SV funded startup land are all major backers of this technology.
I can just summarize this in one phrase: you seem to collectively not know, what you don't know and make leaps in logic and misinformed judgments from that POV.
I have little interest in the ycombinator legacy of hustle culture, growth hacking, get-millions-for-glorified-todo-app, etc. As far as I can tell it is effectively the underlying reason for why crypto and AI get hyped up to the point where we can't have reasonable discussions about them in forums.
I'm just here because it happens to be where like-minded people (_sometimes_) hang out.
> I have little interest in the ycombinator legacy of hustle culture, growth hacking, get-millions-for-glorified-todo-app, etc. As far as I can tell it is effectively the underlying reason for why crypto and AI get hyped up to the point where we can't have reasonable discussions about them in forums.
Ohh, well... I'm guilty of drinking that kool-aid, unfortunately and was a boot-strapping fintech founder with the battle scars to show--those scars appear in this discussion thus far by the way.
But I get it, I'm trying to be amicable about this as much as I can, especially fbecause I know no one can deny us anymore in the BTC side at this point, and in the AI side... well, money-hype boom and AI-doomer pr0n cycles aside we are actually building amazing amounts of compute that hopefully can yield amazing results, something like Starship recovery system levels of advancement in many other Industries/Sectors one day, probably very far off in the future but that's the start, right?
Every forest starts with a sappling kind of a thing, and these two technologies emerged from a time when I was in my formidable development period so I saw them as something more than just the marketing side--AI is marketing, ML is really just stats with code to back it up after all. Bitcoin is just a FOSS network using token based cryptographic key encryption.
Stop using throwaways if you want real conversations, it might help you have those conversations you're looking for. :)
only 6 were AI, the highest being "OpenAI to become for-profit" coming at number 10. Top story was "Bop Spotter" followed by Starship and click to cancel.
I think it will burst when the Fed realizes inflation is not done and start raising again in 6 months. They can only feed the bubble for so long before the common people have had enough of rising prices.
No, raising rate would bring the economy to a slower pace and reduce private sector consumer demand. Private sector investment can continue to increase but at some point that too will hit a brick wall. The public sector spending depends on which type of big ego people get to make decisions. Given the extreme excesses so far it can go either way. The now extinct fiscal conservatives might just make a return finally but don’t hold your breath.
Raising rates works in the beginning with high fiscal deficit driven inflation by slowing demand and bank lending.
But raising interest rates and keeping them high in an environment where runaway government deficits and high government debts are causing inflation runs the risk of exacerbating inflation.
You have high interest rates on a large amount of government debt which continues to push _more_ money into the economy.
The Fed doesn’t have any real options at this point but to lower rates.
At some point we need to address the elephant in the room and ask people specifically what they mean by "high" rates, because 5% isn't particularly high by historical terms, it's only high for people who never paid attention to interest rates before 2010.
Public sector demand is a much smaller percentage of the overall economy. If raising rates did not slow the economy due to high government deficit spending, then we would certainly be living in a much different world of command economy with the government running everything. That’s not yet the world we live in.
Another possibility is that the rate is still not high enough and needs to be raised much much higher to stop inflation. I think rates need to be in the 6 to 7 percent to really stop inflation. The is just a pause. It will come back like a vengeance.
Wow! Rare to see someone get it. MMT follower ? Would add that the money printing is being distributed proportionaly to the wealthy under a Deomcrat regime. Pretty sad.
People are going to be talking about AI for the rest of your life, but feel free to go join an Amish community or live in the woods, maybe get a job as a Firewatch.
There is an interesting contrast between the phenomenal investment boom versus functionally zero job growth...
> Even the software publishers and computing infrastructure industries at the forefront of this AI boom have seen functionally zero net employment growth over the last year - the dismal job market that has beleaguered recent computer science graduates simply has not improved much.
... which may explain, in broad brush, the polarised HN attitude: bitter cynics on one side and aggresive zealots on the other.
Question:It seems like the requirements needed to become an investor in a startup is a multimillion pool of money...which is not what a salaried professional has at their disposal. Yet, a lot of the 100x opportunities come before public offerings, making a whole class of investment unavailable to modest but knowledgeable professionals. I know that sometimes, through personal connections, pools of investment money are built across individuals, but as a public service it seems like, no, it is not.
Reading this makes me willing to bet that this capital intensive investment boom will be similar to other enormous capital investment booms in US history, such as the laying of the railroads in the 1800s, the proliferation of car companies in the early 1900s, and the telecom fiber boom in the late 1900s. In all of these cases there was an enormous infrastructure (over) build out, followed by a crash where nearly all the companies in the industry ended up in bankruptcy, but then that original infrastructure build out had huge benefits for the economy and society as that infrastructure was "soaked up" in the subsequent years. E.g. think of all the telecom investment and subsequent bankruptcies in the late 90s/early 00s, but then all that dark fiber that was laid was eventually lit up and allowed for the explosion of high quality multimedia growth (e.g. Netflix and the like).
I think that will happen here. I think your average investor who's currently paying for all these advanced chips, data centers and energy supplies will walk away sorely disappointed, but this investment will yield huge dividends down the road. Heck, I think the energy investment alone will end up accelerating the switch away from fossil fuels, despite AI often being portrayed as a giant climate warming energy hog (which I'm not really disputing, but now that renewables are the cheapest form of energy, I believe this huge, well-funded demand will accelerate the growth of non-carbon energy sources).
I'm sure you are right. At some point, the bubble will crash.
The question remains is when the bubble will crash. We could be in the 1995 equivalent of the dotcom boom and not 1999. If so, we have 4 more years of high growth and even after the crash, the market will still be much bigger in 2029 than in 2024. Cisco was still 4x bigger in 2001 than in 1995.
One thing that is slightly different from past bubbles is that the more compute you have, the smarter and more capable AI.
One gauge I use to determine if we are still at the beginning of the boom is this: Does Slack sell an LLM chatbot solution that is able to give me reliable answers to business/technical decisions made over the last 2 years in chat? We don't have this yet - most likely because it's probably still too expensive to do this much inference with such high context window. We still need a lot more compute and better models.
Because of the above, I'm in the camp that believe we are actually closer to the beginning of the bubble than at the end.
Another thing I would watch closely to see when the bubble might pop is if LLM scaling laws are quickly breaking down and that more compute no longer yields more intelligence in an economical way. If so, I think the bubble would pop. All eyes are on GPT5-class models for signs.
Re: Slack chat:
Glean.com does it for the enterprise I work at: It consumes all of our knowledge sources including Slack, Google docs, wiki, source code and provides answers to complex specific questions in a way that’s downright magical.
I was converted into a believer when I described an issue to it, pointers to a source file in online git repo and it pointed me to another repository that my team did not own that controlled DNS configs that we were not aware about. These configs were the reason our code did not behave as we expected.
This is the main "killer feature" I've personally experienced from GPT things: a much better contextual "search engine-ish" tool for combing through and correlating different internal data sources (slack, wiki, jira, github branches, etc).
AI code assistants have been a net neutral for me (they get enough idioms in C++ slightly incorrect that I have to spend a lot of time just reading the generated code thoroughly), but being able to say "tell me what the timeline for feature X is" and have it comb through a bunch of internal docs / tickets / git commit messages, etc, and give me a coherent answer with links is amazing.
>they get enough idioms in C++ slightly incorrect
this is part of why I stay in python when doing ai-assisted programming; there's so much training information out there for python and I _generally_ don't care about if its slightly off-idiom, its still probably fine.
Yea, I was thumbs-down on ai-assisted programming because when I tested it out, I tried it by adding things to my existing C and C++ projects, and its suggestions were... kind of wild. Then, a few months later I gave it another chance when I was writing some Python and was impressed. Finally, I used it on a new-from-blank-text-file Rust project and was pretty much blown away.
The best I have ever seen were obscure languages with very strong type safety. Some researcher at a sibling org to my own told me to try it with the Lean language, and it basically gave flawless suggestions.
I'm guessing this is because the only training material was blogs from uber-nerdy CS researchers on a language where "mistakes" are basically impossible to write, and not a bunch of people flailing on forums asking about hello world-ish stuff and segfaulting examples.
As someone who doesn't generally program, it was pretty good at getting me an init.lua set up for nvim with a bunch of plugins and some functions that would have taken me ages to do by hand. That said...it still took a day or two of working with it and troubleshooting everything, and while it's been reliable so far, I worry that it's not exactly idiomatic. I don't know enough to really say.
What it's really good at is taking my description of something and pointing me in the right direction to do my own research.
(two things that helped me with getting decent code were to describe the problem and desired solution, followed by a "Does that make sense?". This seems to get it to restate the problem itself and produce better solutions. The other thing was to copy the output into a fresh session, ask for a description of what the code does and what improvements could be made)
Not saying that it’s a better way, but I started with vim by copying someone conf (on Github), removing all extraneous stuff, then slowly familiarizing myself with the rest. Then it was a matter of reading the docs when I wanted some configuration. I believe the first part is faster than dealing with an LLM, especially when dealing with an unfamiliar software.
I agree with this approach generally, but I needed to use some lua plugins to do something specific fairly quickly, and didn't feel like messing around with it for weeks on end to get it just right.
The downside of this nvim solution is the same downside as both pasting big blobs of ai code into a repo, and, pasting big vim configs you find online into your vimrc: inability to explain the pasted code.
When you need something fast for whatever reason sure, but later when you want to tweak or add something, you'll have to finally sit down and learn basically the whole thing or at least a major part of it to do so anyway. Imo it's better to do that from the start but sometimes that's not always ideal.
My data science friend tells me it's really good at writing bad pandas code because it's seen so much bad pandas code.
At the end of the day, it depends where you are in the hierarchy. Having it write code for me on a hobby project in react that's bad but works is one thing. I'm having a lot of fun with that. Having it write bad code for me professionally is another thing though. Either way, there's no going back to before ChatGPT, just like there's no going back to before Stack Overflow or Google. Or the Internet.
Wouldn't AI be worse at Rust than at C++ given the amount of code available in the respective languages?
Maybe this is a case where more training data isn’t better. There is probably a lot of bad/old C++ out there in addition to new/modern C++, compared to Rust which is relatively all modern.
This is partly why I believe OS makers, Apple, Microsoft, Google, have a huge advantage in the future when it comes to LLMs.
They control the OS so they can combine and feed all your digital information to an LLM in a seamless way. However, in the very long term, I think their advantage will go away because at some point, LLMs could get so good that you don't need an OS like iOS anymore. An LLM could simply become standalone - and function without a traditional OS.
Therefore, I think the advantage for iOS, Android, Windows will increase in the next few years, but less powerful after that.
An LLM is an application that runs on an operating system like any other application. That the vendor of the operating system has tied it to the operating system is purely a marketing/force-it-onto-your-device/force-it-in-front-of-your-face play. It's forced bundling, just like Microsoft did with Internet Explorer 20 years ago.
I predict that OpenAI will try to circumvent iOS and Android by making their own device. I think it will be similar to Rabbit R1, but not a scam, and a lot more capable.
They recently hired Jony Ive on a project - it could be this.
I think it'll be a long term goal - maybe in 3-4 years, a device similar to the Rabbit R1 would be viable. It's far too early right now.
Even if this is true (and I'm not saying it's not), they probably won't create their own OS. They'd be smarter to do what Apple did and clone a BSD (or similar) rather than start afresh.
I’m not even sure if they can make a website that takes text input to an executable and dumps the output.
Thanks. I didn't know that existed. But does it scale? Would it still work if large companies with many millions of Slack messages?
I suppose one reason Slack doesn't have a solution yet is because they're having a hard time getting it to work for large companies.
Yeah, Glean does this and there are a bunch of other competitors that do it as well.
I think you may be confused about the length of the context window. These tools don't pull all of your Slack history into the context window. They use a RAG approach to index all of your content into a vector DB, then when you make a query only the relevant document snippets are pulled into the context window. It's similar for example to how Cursor implements repository-wide AI queries.
I'm aware that one can't feed millions of messages into an LLM all at once. The only way to do this now is to use a RAG approach. But RAG approach has pros and cons and can miss crucial information. I think context window still matters a lot. The bigger the window, the more information you can feed in and the quality of answer should increase.
The point I'm trying to make is that increase context window will require more compute. Hence, we could still just be in the beginning of the compute/AI boom.
We might be even earlier — the 90s was a famous boom with a fast bust, but to me this feels closer to the dawn of the personal computer in the late 70s and early 80s: we can automate things now that were impossible to automate before. We might have a long time before seeing diminishing returns.
My workpalce uses Glean and since it was connected to Slack it has become significantly worse. It routinely gives incorrect or VERY incomplete information, misattributes work to developers who may have casually mentioned a project at some time and worst of all presents jokes or sarcastic responses as fact.
Not only is it an extremely poor source of information, it has ruined the company's Slack culture as people are no longer willing to (for lack of a better term) shitpost knowing that their goofy sarcasm will now be presented to Glean users as fact.
Maybe have some off limits to glean shit posting channels?
> One gauge I use to determine if we are still at the beginning of the boom is this
Has your barber/hairdresser recommended you buy NVDA?
There was an NVDA earnings watch party in NY this summer and Jensen signed some boobs earlier this year. There are some signs but still room to run
> the more compute you have, the smarter and more capable AI
Well, this is taken on faith by OpenAI/etc, but obviously the curve has to flatten at some point, and appears to already be doing so. OpenAI are now experimenting with scaling inference-time compute (GPT-O1), but have said that it takes exponential increases in compute to produce linear gains in performance, so it remains to be seen if customers find this a worthwhile value.
GPT-o1 does demonstrate my point: the more compute you have, the smarter the AI.
If you run chain of thoughts on an 8B model, it becomes a lot smarter too.
GPT-o1 isn't GPT5 though. I think OpenAI will have a chain of thoughts model for GPT5-class models as well. They're separate from normal models.
There is only so much that an approach like O1 can do, but anyways in terms of AI boom/bust the relevant question is whether this is a viable product. All sorts of consumer products could be improved by making them a lot more expensive, but there are cost/benefit limits to everything.
GPT-5 and Claude-4 will be interesting, assuming these are both pure transformer models (not COT), as they will be a measure how much benefit remains to be had from training set scaling. I'd expect gains will be more against narrow benchmarks, than in the overall feel of intelligence (LLM arena score?) one gets from the model.
I think OpenAI has already proven that it's a viable product. Their gross margins must be decent. I doubt they're making a loss for every token they inference.
I don't think they've broken out O1 revenue, but it must be very small at the moment since it was only just introduced. Their O1-preview pricing doesn't seem to reflect the exponential compute cost, so perhaps it is not currently priced to be profitable. Overall, across all models and revenue streams, their revenue does exceed inference costs ($4B vs $2B), but they still are projected to lose $5B this year, $14B next year, and not make a profit until 2029 (and only then if they've increased revenue by 100x ...).
Training costs are killing them, and it's obviously not sustainable to keep spending more on research and training than the revenue generated. Training costs are expected to keep growing fast, while revenue per token in/out is plummeting - they need massive inference volume to turn this into a profitable business, and need to pray that this doesn't turn into a commodity business where they are not the low cost producer.
https://x.com/ayooshveda/status/1847352974831489321
https://x.com/Gloraaa_/status/1847872986260341224
The thing is that OpenAI can choose to spend less on training at any time.
We've seen this before, with for example Amazon where they made a deliberate effort to avoid profitability by spending as much as possible on infrastructure until the revenue became some much they couldn't spend it.
Being in a position where you are highly cash-flow positive and it's strategic investment that is the cost seems like a good position.
> Does Slack sell an LLM chatbot solution that is able to give me reliable answers to business/technical decisions made over the last 2 years in chat?
Note that the presence of such a feature isn't the same as whether it's secure enough for normal use.
In particular, anything anyone said in the last 2 years in chat could poison the LLM into exfiltrating your data or giving false results chosen by the attacker, because of the fundamental problems of LLMs.
https://promptarmor.substack.com/p/data-exfiltration-from-sl...
You can never really tell, though following some market tea leaf readers they seem to think a few months from now, after a bit of a run up in the market. Here's one random datapoint, on mentions of "soft landing" in Bloomberg https://x.com/bravosresearch/status/1848047330794385494
I read through a few pages of tweets from this author and it looks just like another perpetual doomsday pundit akin to Zerohedge.
Well yeah there may be a bit of that. I find them quite interesting for the data they bring up like the linked tweet but I don't really have an opinion as to whether they are any good at predicting things.
I was thinking re the data in the tweet, that there were a lot of mentions of "soft landing" before the dot com crash, before the 2006 property crash and now, it is quite likely there was an easy money policy preceding them and then government policy mostly focuses on consumer price inflation and unemployment, so they relax when those are both low and then hit the brakes when inflation goes up and then it moderates and things look good similar to now. But that ignores that easy money can also inflate asset prices, eg dot com stocks, houses in 06, or money losing AI companies like now. And then at some point that ends and the speculative asset prices go down rather than up, leaving people thinking oh dear we've borrowed to put loads of money into that dotcom/house/ai thing and now it's not worth much and we still have the debts...
At least that's my guess.
> Reading this makes me willing to bet that this capital intensive investment boom will be similar to other enormous capital investment booms in US history, such as the laying of the railroads in the 1800s, the proliferation of car companies in the early 1900s, and the telecom fiber boom in the late 1900s. In all of these cases there was an enormous infrastructure (over) build out, followed by a crash where nearly all the companies in the industry ended up in bankruptcy, but then that original infrastructure build out had huge benefits for the economy and society as that infrastructure was "soaked up" in the subsequent years.
> I think that will happen here.
Why? The rail network, road network and fiber network that was laid could be used for decades after their original investors went bust.
The current datacenters full of AI compute can't really be used for anything else if AI companies go bust.
That's the problem with investing in compute infrastructure - you need to have a plan to use it all up in the next 5 years, because after that you wouldn't even be able to give it away.
The renewable power infrastructure for those datacenters will still exist.
People will be able to buy those used GPUs cheap and run small local LLMs perhaps. A 10 year old computer today won't do state of the art games or run models, but is entirely acceptable for moderate computing use.
> but now that renewables are the cheapest form of energy, I believe this huge, well-funded demand will accelerate the growth of non-carbon energy sources
I think the renewables would have been built at the same rate anyway precisely because they're so cheap; but nuclear power, being expensive, would not be built if this bubble had not happened, and somehow nuclear does seem to be getting some of this money.
I want to point out to anyone who's interested in the nuclear angle that even before the AI data center demand story arrived, the uranium market was facing a persistent undersupply for the first time in its many decades of history. As a result, the (long-term contract) price of uranium has been steadily rising for years: https://www.cameco.com/invest/markets/uranium-price
After Fukushima (https://news.ycombinator.com/item?id=41768726), Japanese reactors were shut down and there was a glut of uranium available in the spot market. Simultaneously, Kazatomprom flooded the market with cheap ISR uranium. The price of uranium fell far below the cost of production and the mining companies were obliterated. The few miners that survived via their long-term contracts (primarily Cameco) put their less efficient mines into care and maintenance.
Now we're seeing the uranium mining business wake up. But after a decade of bear-market conditions the miners cannot serve the demand: they've underinvested, they've lost skilled labor, they've shrunk. The rebound in uranium supply will be slow, much slower than the rebound in demand. This is because uranium mining is an extremely difficult process. Look at how long NexGen Energy's Rook 1 Arrow mine has taken to develop, and that's prime ore (https://s28.q4cdn.com/891672792/files/doc_downloads/2022/03/...). Look at Kazatomprom's slowing growth rate (https://world-nuclear-news.org/Articles/Kazatomprom-lowers-2...), look at the incredible complexity of Cameco's mining operations: https://www.petersenproducts.com/articles/an-inflatable-tunn...
Here is a discussion of the uranium mining situtation: https://news.ycombinator.com/item?id=41661768 (including a very risky method of profiting from the undersupply of uranium, stock ticker SRUUF, not recommended). Note that Numerco's uranium spot price was put behind a paywall last week. You can still get the intra-day spot uranium price for free here: https://www.yellowcakeplc.com/
Uranium, at least the un-enriched kind you can just buy, was never the problem.
Even the peak of that graph (136… er, USD per lb?) is essentially a rounding error compared to everything else.
0.00191 USD/kWh? Something like that, depends on the type of reactor it goes in.
You are correct. This is one of the advantages of nuclear power.
The fuel is a tiny fraction of the cost of running the plant. See discussion here, contrasting with natural gas: https://news.ycombinator.com/item?id=41858892
It is also important that the fuel is physically small so you can (and typically, do) store years of fuel on-site at the reactor. Nuclear is "secure" in the sense that it can provide "energy security".
It would only be an advantage if everything in the power plant else wasn't so expensive.
And I'm saying that as someone who finds all this stuff cool and would like to see it used in international shipping.
Discussed at length here: https://news.ycombinator.com/item?id=41863388
I already linked this above, twice. I know it's a hassle to read, it's Sunday afternoon, so don't worry about it.
It's not important whether you as an individual get this right or not, as long as society reaches the correct conclusion. Thankfully, we're seeing that happen, a worldwide shift toward the adoption of nuclear power.
Have a pleasant evening!
based on my reading nuclear power is much cheaper overall compared to wind solar etc?
Not at all. Old paid of nuclear plants are competitive but new builds are insanely expensive leading to $140-220/MWh prices for the ratepayers before factoring in grid stability and transmission costs.[1]
The US has zero commercial reactors under construction and this is for one reason: economics.
The recent announcements from the hyperscalers are PPAs. If the company building the reactor can provide power at the agreed price they will take it off their hands. Thus creating a more stable financial environment to get funding.
They are not investing anything on their own. For a recent example NuScale another SMR developer essentially collapsed when their Utah deal fell through when nice renders and PowerPoints met real world costs and deadlines. [2]
[1]: https://www.lazard.com/media/gjyffoqd/lazards-lcoeplus-june-...
[2]: https://iceberg-research.com/2023/10/19/nuscale-power-smr-a-...
> leading to $140-220/MWh prices for the ratepayers
I'm on PG&E, I wish I could get my electricity for only $0.14/kWh
That cost is excluding grid stability and transmission costs.
From what I’ve understood PG&E’s largest problem is the massive payouts and infrastructure upgrades needed from the wildfires, not the cost of the electricity itself.
Thanks! I always thought it is due to people's safety concerns here instead of economic reasons. After all, nuclear plant is quite 'popular' in Europe, and China too these days.
It’s not popular in Europe at all.
France derives about 70% of its electricity from nuclear energy.
For Europe overall is 22%.
Yes, that's right. See the recent discussion here:
https://news.ycombinator.com/item?id=41860341
Basically, nuclear fission is clean baseload power. Wind and solar are not baseload power sources. They don't really compete. See discussion here: https://news.ycombinator.com/item?id=41858892
Furthermore, we're seeing interest (from Google and Amazon and Dow Chemical) in expensive but completely safe TRISO (HALEU) reactors (https://www.energy.gov/ne/articles/triso-particles-most-robu...). These companies want clean baseload power, with no risk of meltdown, and they're willing to pay for it. Here's what Amazon has chosen: https://x-energy.com/fuel/triso-x
TRISO (HALEU) reactors use more than 1.5 times the natural uranium per unit of energy produced because the higher burnup is offset by higher enrichment inputs (see page 11 at https://fuelcycleoptions.inl.gov/SiteAssets/SitePages/Home/1...), and the fuel is even more expensive to manufacture, but they are completely safe. This is a technology from the 1960's but it's attractive now because so much money is chasing clean baseload nuclear fission for data centers.
These "impossible to melt down" TRISO small modular nuclear fission reactors are what Elon Musk was talking about on the campaign trail last week, when he said:
https://x.com/AutismCapital/status/1847452008502219111> Basically, nuclear fission is clean baseload power. Wind and solar are not baseload power sources. They don't really compete.
This means you don't understand how the grid works. California's baseload is ~15 GW while it peaks at 50 GW.
New built nuclear power is wholly unsuitable for load following duty due to the economics. It is an insane prospect when running at 100% 24/7, and even worse when it has to adapt.
Both nuclear power and renewables need storage, flexibility or other measures to match their inflexibility to the grid.
See the recent study where it was found that nuclear power needs to come down 85% in cost to be competitive with renewables, due to both options requiring dispatchable power to meet the grid load.
> The study finds that investments in flexibility in the electricity supply are needed in both systems due to the constant production pattern of nuclear and the variability of renewable energy sources. However, the scenario with high nuclear implementation is 1.2 billion EUR more expensive annually compared to a scenario only based on renewables, with all systems completely balancing supply and demand across all energy sectors in every hour. For nuclear power to be cost competitive with renewables an investment cost of 1.55 MEUR/MW must be achieved, which is substantially below any cost projection for nuclear power.
https://www.sciencedirect.com/science/article/pii/S030626192...
> These companies want clean baseload power, with no risk of meltdown, and they're willing to pay for it. Here's what Amazon has chosen
The recent announcements from the hyperscalers are PPAs. If the company building the reactor can provide power at the agreed price they will take it off their hands. Thus creating a more stable financial environment to get funding.
They are not investing anything on their own. For a recent example NuScale another SMR developer essentially collapsed when their Utah deal fell through when nice renders and PowerPoints met real world costs and deadlines.
https://iceberg-research.com/2023/10/19/nuscale-power-smr-a-...
> with no risk of meltdown
Then we should be able to remove the enormous subsidy the Price Anderson act adds to the industry right? Let all new reactors buy insurance for a Fukushima level accident in the open market.
Nuclear powerplants are currently insured for ~0.05% of the cost of a Fukushima style accident and pooled together the entire US industry covers less than 5%.
https://en.wikipedia.org/wiki/Price%E2%80%93Anderson_Nuclear...
Railroads and computer networks create network effects, I am not sure the same is true for data centers full of hardware that becomes outdated very quickly.
If they're building new power plants to support all those data centers than that power generation capacity might be put to good use doing something else.
What will be the advantage of having a bunch of obsolete hardware? All I see is more e-waste.
>What will be the advantage of having a bunch of obsolete hardware? All I see is more e-waste.
The energy build out, data centers are not wasted. You can swap out A100 GPUs for BH200 GPUs in the same datacenter. A100s will be 5 years old when Blackwell is out - which is just about right for how long datacenter chips are expected to last.
I do, however, think that the industry will move to newer hardware faster to try to squeeze as much efficiency as possible due to the energy bottleneck. Therefore, I expect TSMC's N2 nodes to have huge demand. In fact, TSMC themselves have said designs for N2 far outnumber N3 at the same stage of the node. This is most likely due to AI companies that want to increase efficiency due to the lack of electricity.
Well, at least they are paving the way to more efficient hardware, GPU are way, way more energy efficient than CPU and the parallel architectures are the only remaining way to scale compute.
But yes, a lot of energy wasted in the growing phase.
GPUs are different than CPUs.
They’re way more efficient at matmuls, but start throwing branching logic at them and they slow down a lot.
Literally a percentage of their cores will noop while others are executing a branch, since all cores are lockstep.
>But yes, a lot of energy wasted in the growing phase.
Why exactly is energy wasted during this phase?
Are you expecting hardware to become obsolete much faster? But that only depends on TSMC's node cadence, which is still 2-3 years. Therefore, AI hardware will still be bound to TSMC's cadence.
In the case of dark fiber the hardware was fine, wavelength division multiplexing was created which increased capacity by 100x in some cases crashing demand for new fiber.
I think OP is suggesting AI algorithms and training methods will be improve resulting in enormous performance gains with existing hardware causing a similar surplus of infrastructure and crash in demand.
How much of current venture spending is going into reusable R&D that can be moved forward in time the way that physical infrastructure in their examples were able to be used in the future?
Considering that models have been getting more powerful for the same number of parameters -- all of it.
All those future spare GPUs will make video game streaming dirt cheap.
Even poor people can enjoy 8k gaming on a phone soon.
Most data center GPUs do not have game rendering hardware in them.
Even if the hardware quickly becomes outdated, I'm not sure it'll become worthless so quickly. And there's also the infrastructure of the data center and new electricity generation to power them. Another thing that might survive a crash and carry on to help the future is all the code used to support valuable use cases.
Do you expect better hardware to suddenly start appearing on the market, fully-formed from the brow of Zeus?
I don't think the parent was specifically referring to hardware alone. The 'rails' in this context are also the AI algorithms and the underlying software. New research and development could lead to breakthroughs that allow us to use significantly less hardware than we currently do. Just as the dot-com crash wasn’t solely about the physical infrastructure but also about protocols like HTTP, I believe the AI boom will involve advancements beyond just hardware. There may be short-term excess, but the long-term benefits, particularly on the software side, could be immense.
Maybe the bust will be so rough that TSMC will go out of business, and then these graphics cards will not go obsolete for quite a while.
Like Intel and Samsung might make a handful of better chips or whatever, but neither of their business models really involve being TSMC. So if the bubble pop took out TSMC, there wouldn’t be a new TSMC for a while.
Similarly, I like to compare AI (more specifially LLM) "investment" to the cost of building the channel tunnel between UK and France. The original investors lost their shirt, but once built it is profitable to operate.
I was an original investor. I still have shirts and shares in it but they could have done better.
Building robust LLM-based applications is token-intensive. You often have to plan for the parsing and digestion of a lot of tokens for summarization or even retrieval augmented generation. Even the mere generation of marketing blogposts consumes a lot of output tokens in most cases. Not to mention that all robust cognitive architectures often rely on the generation of several samples for each prompt, custom retry logics, feedback loops, and reasoning tokens to achieve state of the art performance, all solutions powerfully token-intensive.
Luckily, the cost of intelligence is quickly dropping. GPT-4, one of OpenAI’s most capable models, is now priced at $2.5 per million input tokens and $10 per million output tokens. At its initial release in March 2023, the cost was respectively $10/1M input tokens and $30/1M for output tokens. That’s a huge $7.5/1M input tokens and $20/1M output tokens reduction in price. https://www.lycee.ai/blog/drop-o1-preview-try-this-alternati...
While I agree with your essential conclusions, I don't think the automobile companies really fit. Many of the early 1900s companies (e.g. Ford, GM, Mercedes, even Chrysler) are still among the largest auto companies in the world.
There were hundreds of failed automotive companies and parts suppliers though. I think the argument is that many will die, some will survive and take all (most)
But that happens in every bubble. Over investment, consolidation, huge winners in the end, and maybe eventually a single monopoly.
There isn't a rule as to how it plays out. No huge winners in cars, no huge winners in rail. Lots of huge winners in internet.
There were huge winners in cars. Ford and GM have historically been huge companies. Then oil companies became the biggest companies in the world mostly due to cars.
GM went bankrupt. Ford would have without government intervention. Each have had periods of profitability but they weren't ever anything like microsoft/google etc. Ford has underperformed the stock market average since it went public like 70 odd years ago. GM got so big in the first place via acquisitions, not because the business of cars lent itself to a dominant player.
Huge by itself isn't the same as huge winner.
> While I agree with your essential conclusions, I don't think the automobile companies really fit. Many of the early 1900s companies (e.g. Ford, GM, Mercedes, even Chrysler) are still among the largest auto companies in the world.
American Automative filled for Bankruptcy multiple times.
American Government had to step in to back them up and bail them out.
That phase is called consolidation. It’s part of the cycle. The speculative over leveraged and mismanaged companies get merged into the winners or disappear if they have nothing of value.
A couple of them went bankrupt and got bailouts.
Something people forget is that a training cluster with tens of thousands of GPUs is a general purpose supercomputer also! They can be used for all sorts of numerical modelling codes, not just AI. Protein folding, topology optimisation, route planning, satellite image processing, etc…
We bought a lot of shovels. Even if we don’t find more gold, we can dig holes for industry elsewhere.
I think there is an LLM bubble for sure, but I’m very bullish on the ease with which one can generate new specialized models for various tasks that are not LLM.
For example, there’s a ton of room for developing all kinds of low latency, highly reliable, embedded classifiers in a number of domains.
It’s not as gee-whiz/sci-fi as an LLM demo, but I think potentially much bigger impact over time.
Spot on.
I'm investing in undervalued businesses who are selling the shovels.
That's the funny thing about the AI boom. There's way more hype about shovel sellers than shovel buyers.
No one is getting excited about "AI slop", just the models that generate it. Funny situation.
The problem is that all that mal investment will get bailed about by us regular shmucks. Get ready for the hamster wheel to start spinning faster.
Not all of that infrastructure gets soaked up, plenty is abandoned. Look at the state of american passenger rail for example and how quickly the bottom of that industry dropped out. Many old rail right of ways sit abandoned today. Likewise with telecoms, e.g. the microwave network that also sits abandoned today.
That may be true now, but Europe has had a resurgence in rail usage growth recently, so I'm not sure it's true forever.
There is a comment on this thread about this being like the railroads, but this is nothing like the railroads except insofar as it costs a lot of money.
The railroads have lasted decades and will remain relevant for many more decades. They slowly wear out, and they are the most efficient form of land transport.
These hardware investments will all be written off in 6 years time and won't be worth running given the power costs and relative output. They will be junked.
There's also the extra risk that for some reason future AI systems just don't run efficiently on current gen hardware.
Maybe, maybe not. Nvidia P40's GPUs were released 8 years ago, they are hot. Prices have doubled in the last year. Nothing will be written off in 6 years. Folks will be using A100 15 years from now. 7 yrs old V100 32gb GPU cards are still going for $1500 on ebay. We are more likely to invent a more efficient software architecture than invent a new type of better hardware and replace everything that exists.
Some stuff like the buildings and power supplies will probably remain good. But year, probably new chips in a short while.
Power plants and power infrastructure are probably an example of a positive consequence that comes from this.
We have been terrified to whisper the words "nuclear power" for decades now, but the AI boom is likely to put enough demand on the power grid that it forces us to face this reality and make appropriate buildouts.
Even if the AI Boom crashes, these power plants will have positive impacts on the country for decades, likely centuries to come. Keeping bountiful power available and likely low-cost.
It is so bizarre that reducing pollution was not a sufficient driver to build more nuclear power, but training AI models is.
So there is a boom in terms of investment for future capacity. It could be interesting to see if the demand is following. I suspect there will be over capacity built, resulting some bankruptcies + cheaper compute/ai costs, hopefully allowing the next generation of startups or companies to flourish later on.
It feels like there’s a big gap in this discussion. The focus is almost entirely on GPU and hardware investment, which is undeniably driving a lot of the current AI boom. But what’s missing is any mention of the software side and the significant VC investment going into AI-driven platforms, tools, and applications. This story could be more accurately titled ‘The GPU Investment Boom’ given how heavily it leans into the hardware conversation. Software investment deserves equal attention
I think GPUs and datacenter is to AI is what fiber was to the dotcom boom.
A lot of LLM based software is uneconomical because we don't have enough compute and electricity for what they're trying to do.
I think a better analogy is the valuation of Intel vs. that of Microsoft. For a long time, Intel dominated the CPU market. So you'd expect them to be the most valuable company. Instead, a small startup called Microsoft started dominating the software market and eventually became the most valuable company on the planet. The combined value of the software market is orders of magnitudes larger than that of all chip makers combined and has been for quite some time. The only reason people buy hardware is to use software.
The same is going to happen with AI. Yes, Nvidia and their competitors are going to do well. But most of the value will be in software ultimately.
GPUs and data centers are just infrastructure. Same for the electricity generation needed to power all that. The demand for AI is causing there to be a lot of demand for that stuff. And that's driving the cost of all of it down. The cheapest way to add power generation is wind and solar. And both are dominating new power generation addition. Chip manufacturers are very busy making better, cheaper, faster etc. chips. They are getting better rapidly. It's hard to see how NVidia can dominate this market indefinitely.
AI is going to be very economical long term. Cheap chips. Cheap power. Lots of value. That's why all the software companies are busy ramping up their infrastructure. IMHO investing in expensive nuclear projects is a bit desperate. But I can see the logic of not wanting to fall behind for the likes of Amazon, Google, MS, Apple, etc. They can sure afford to lose some billions and it's probably more important to them to have the power available quickly than to get it cheaply.
Nuclear power is probably the least valid choice of all the power source choices when the goal is to "have power available quickly".
There is no kind of power plant that takes longer to build than a nuclear plant.
Nvidia is a software company that also happens to make decent hardware. People buy their hardware because of their software. And it’s not just CUDA. Nvidia is building a whole bunch of potentially groundbreaking software products. Listen to the last GTC keynote to learn more about it.
A lot of it is green washing. Some of it might be signals to the competition or other vendors. This AI buildup is going to be a knock-down-drag-out.
The actual physical fiber was useful after the companies popped though.
GPUs are different, unless things go very poorly, these GPUs should be pretty much obsolete after 10 years.
The ecosystem for GPGPU software and the ability to design and manufacture new GPUs might be like fiber. But that is different because it doesn’t become a useful thing at rest, it only works while Nvidia (or some successor) is still running.
I do think that ecosystem will stick around. Whatever the next thing after AI is, I bet Nvidia has a good enough stack at this point to pivot to it. They are the vendor for these high-throughput devices: CPU vendors will never keep up with their ability to just go wider, and coders are good enough nowadays to not need the crutch of lower latency that CPUs provide (well actually we just call frameworks written by cleverer people, but borrowing smarts is a form of cleverness).
But we do need somebody to keep releasing new versions of CUDA.
But computer chips have always had limited usefulness because newer chips are simply faster and more efficient. The datacenter build outs and increase in electricity capacity will always be useful.
Is there a good estimate of the split? My impression from AI startups whose operations I know something about is that a majority of their VC raise is currently going back into paying for hardware directly or indirectly, even though they aren’t hardware startups per se, but I don’t have any solid numbers.
I think your analysis is spot on, based on my expertise of "reading too much hacker news since every day since March 2023". This Bain report[1] barely mentions "Independent Software Vendors" and when they do it's clearly an afterthought, and the only software focused takes I can find are[2] extremely speculative, e.g.
I'm hoping someone more knowledgeable about capital markets can fill us in here, I'd be curious to see some hard numbers still! Maybe this is what a Bloomberg terminal does...?Regardless, I think this makes a lot of sense; there's no clear scientific consensus on the path forward for these models other than "keep going?", so building out preparatory infrastructure is seen as the clear, safe move. As the common refrain goes: "in a gold rush, sell shovels!"
As a big believer in the upcoming cognitive era of software and society, I would only add a short bit onto the end of that saying: "...until the hydraulic mining cannons[3] come online."
[1] https://www.bain.com/insights/ais-trillion-dollar-opportunit...
[2] https://www.privatebankerinternational.com/comment/is-ai-sof...
[3] https://en.wikipedia.org/wiki/Hydraulic_mining
>As the common refrain goes: "in a gold rush, sell shovels!"
What happens when everyone constructs a shovel factory, the shovels become dirt cheap, but there are no buyers?
Also they talk a lot about data centers and cloud compute, but do not mention chips for on decide inference.
Given where mobile sits in the hierarchy of interfaces, that is where I would be placing my bets if I were a VC.
The AI bubble will pop in the next year. We are currently in 1998 of the dotcom bubble with another AI winter approaching. LLM and generative AI are this year’s “on the Internet” or “Uber for X” business plans.
Will the AI bubble pop bring down the rest of the startup industry, or allow for more investment in non AI tech.
Climate related tech needs more money.
The market PE is very high and has been high for a while. Tech and AI are fueling a lot of that. Look what happened to Tesla when the fundamentals started getting a little discouraging. However, I'm not comfortable predicting the "AI bubble" popping next year.
I'm curious how this will affect cloud costs for the rest of us. On the one hand, we may get some economies of scale, but on the other hand, cloud resources being used up by others may drive prices up. Does anyone have any guesses as to what will happen?
I doubt it will increase cost for traditional CPU-based clouds. Just take a look at Ampere 192 core and AMD 196 core CPUs. Their efficiency will continue to drive down traditional cloud $/perf.
> I doubt it will increase cost for traditional CPU-based clouds.
Yeah I think you’re right about that. But what about GPU’s? Will they benefit from economies of scale or the opposite?
I've yet to find an "AI" that doesn't seamlessly hallucinate, and I don't see how "AIs" that hallucinate will ever be useful outside niche applications.
Confabulations are decreasing with newer models. I tested confabulations based on provided documents (relevant for RAG) here: https://github.com/lechmazur/confabulations/. Note the significant difference between GPT-4 Turbo and GPT-4o.
That’s very interesting! Thanks for the link.
Still useful for anything you can verify.
You don't really need to imagine this though - generative AI is already extremely useful in many non-nice applications.
There's a camp of people who are hyper-fixated on LLM hallucinations as being a barrier for value creation.
I believe that is so far off the mark for a couple reasons:
1) It's possible to work around hallucinations in a more cost effective way than relying on humans to always be correct.
2) There are many use cases where hallucinations aren't such a bad thing (or even a good thing) for which we've never really had a system as powerful as LLMs to build for.
There's absolutely very large use cases for LLMs and it will be pretty disruptive. But it will also create net new value that wasn't possible before.
I say that as someone who thinks we have enough technology as it is and don't need any more.
For sure, sending customers into a never ending loop when they want support. That's been my experience with most AI support so far. It sucks. I like Amazon's approach where they have a basic chat bot (probably doesn't even use LLMs) that then escalates to a actual human being in some low cost country.
I kind of like the Chipotle approach. I have a problem with my order, it just refunds me instantly and sometimes gives me a add-on for free.
Honestly I only use LLM for one thing - I give it a set of TS definitions and user input, and ask it to fit those schemas if it can and to not force something if it isn't 100% confident.
I know some people whose whole company is based around the use of AI to send emails or messages, and in reality they're logged into their terminals real time fixing errors before actually sending out the emails. Basically, they are mechanical turks and they even say they're looking at labor in India or Africa to pay them peanuts to address these.
Yeah, they just want it to go away. The same way they wish Windows and GUIs and people in general would just go away.
I believe that there is lot of content creation where quality really does not matter. And hallucinations don't really matter. Unless they are legally actionable, that is something like hate speech or libel.
Throwing dozens of articles, social media posts and why not even videos. Hallucinations really don't matter at scale. And enough content is already generating enough views to make it somewhat viable strategy.
It amazes me the level of nihilism needed to talk about this with casual indifference.
> quality really does not matter
What an inspiring vision for the future of news and entertainment.
I've never met a human that doesn't "hallucinate" either (either intentionally or unintentionally). Humans either intentionally lie or will fill in gaps in their knowledge with assumptions or inaccurate information. Most human generated content on social media is inaccurate, to an even higher percentage than what ChatGPT gives me.
I guess humans are worthless as well since they are notoriously unreliable. Or maybe it just means that artificial intelligence is more realistic than we want to admit, since it mimics humans exactly as we are, deficiencies and all.
This is kind of like the self-driving car debate. We don't want to allow self-driving cars until we can guarantee that they have a zero percent failure rate.
Meanwhile we continue to rely on human drivers which leads to 50,000 deaths per year in America alone, all because we refuse to accept a failure rate of even one accident from a self-driving car.
It's not quite the case with cars though - people are ok with Waymos which are not zero accident but probably safer than human drivers. The trouble with other systems like Tesla FSD is they are probably not safer than human yet if you don't have a human there nannying them.
Similarly I think people will be ok with other AI if it performs well.
https://github.com/stanford-oval/WikiChat
Aren't humans the main alternative to AI and they seamlessly hallucinate as well.
Humans also confabulate (a better metaphor for AI errors than hallucination) when called on to respond without access to the ground truth, and most AI models have limited combination of access and ability to use that access when it comes to checking ground truth.
I remember listening to Dr Robert Martin who was leading Bell Labs in the late 90s and he talked about how the bandwidth capacity was pushing to infinity while cost per bit was pushing towards zero and we all know how that ended for the optical capacity builders of that time before the bubble popped. Is there a case for intelligence being inexhaustible to demand ? Is there a case for, as Sama says, the cost for intelligence as in input to a system converges with the price of electricity needed to power the gpu behind it in the daya center? Yes and yes. Still the same could be said for a bit of bandwidth.
I am very skeptical of the positive effects of infite intelligence on the living standards of knowledge workes.
On the more pessimistic end, AI will replace us and we'll be sent to the coal mines.
On the possibly most optimistic end, living standards are a composite of many things rooted in reality, so I'd say the actual cap is about doubling of life quality, which is not nothing, but not unprecendented if we look at the past century and a half.
The ideal scenario in my mind is that intelligent and benevolent AI takes over running things and then people have to figure out how to make their life meaningful with only leisure time.
There is no long-term time scale in which humans running things do not obliterate each other or end up sitting on a planet filled with trash.
Either we figure out how to colonize other planets or we hand over the reigns to something that can plan long term and not be irrational.
Maybe if we figure out immortality it might work, but with the short life span of a human there is no way to not be short-sited or eventually end up with the wrong person in charge the button.
Honestly I don't like these AI doom scenarios, mostly because I feel like they're used as a means by the people in charge to shift the Overton window of AI outcomes - as in, the AI might have built an ubiquitious police state with all the wealth being owned by the 0.001%, but at least it didn't kill us all!
Honestly, I feel like with the recent progress of AI, it's a realistic scenario to assume it will replace mpst knowledge workers in the next 5 to 10 years, probably won't replace researchers and other elite intellectuals, and won't even make a dent in the world of physical labor.
In that world, I see AI as harmful, but the people in charge won't, as they are directly benefiting from it.
We live in number-go-up capitalism. A good analog is the housing situation. The ever increasing price of real estate means that the total amount of wealth goes up, so it's seen as a beneficial process by the elites. The rest however will find that they need to dedicate a larger proportion of their income towards getting a roof over their heads, and think this process is bad.
Nowadays, the possibility building a life that would've been considered middle class half a century ago from scratch is available to like 10% of workers, working mainly intellectual jobs.
AI in the future will reduce the proportion of these people by taking away their high paying jobs.
> people have to figure out how to make their life meaningful with only leisure time.
Like religion or nationalism fueling some wars.
Generated text =! Intelligence.
> AI products are used ubiquitously to generate code, text, and images, analyze data, automate tasks, enhance online platforms, and much, much, much more—with usage expected only to increase going forward.
Why does every hype article start with this. Personally my copilot usage has gone down while coding. I tried and tried but it always gets lost and starts spitting out subtle bugs that takes me more time to debug than if i had written it myself.
I always have this feeling of 'this might fail in production in unknown ways' because i might have missed checking the code throughly . I know i am not the only one, my coworkers and friends have expressed similar feelings.
I even tried the new 'chain of thought' model, which for some reason seems to be even worse.
This just reminded I forgot I have had a Copilot subscription. It has not made any useful code suggestions in months to the point of fading from my memory. I just logged in to cancel it. Now, I need to check my other subscriptions that I can cancel or reduce to a lower tier.
They have massively nerfed Copilot. I'm keeping my subscription for a couple more months but at this point, it has the same intelligence as the llma3.2 which I can run on my laptop.
My experience is similar. I used Claude for a coding task recently and it drove me into an infinite number of rabbit holes, each one seeming worse than the previous one. All the while being enable to stop and say: I’m sorry, I actually don’t know how to help you.
Because they all use AI to write the articles.
There is a market for AI. And it is exactly these articles and maybe pictures attached to them. Soon could be some videos as well. But how far beyond that. Is very good question.
AI trained on a web that's primarily about selling things
From my experience, it is getting better over time, and I believe that there's still a lot of relatively low hanging fruit, particularly in terms of integrating the LLM with the language server protocol and other tooling. But having said that, at this point in time, it's just not good enough for independent work, so I would suggest using it only as you would pair-program with a mid-level human developer who doesn't have much context on the project, and has a short attention span. In particular, I generally only have the AI help me with one function/refactoring at a time, and in a way that is easy for me to test as we go, and am finding immense value.
> better over time
The problem is all the most reliable code it can give you is stuff which ought to be (or already is) a documentation example or a reusable library, instead of "copy paste as a service".
I think some of the consternation we see from the anti LLM crowd (of which I'm one) is this line of reasoning. These LLMs produce fine code when the code you are asking for is in its training set. So they can be better than a mid level dev and much faster in narrow, unknown contexts. But with no feedback to warn you, if you ask it for code that it has no or only a bit of data on, it is much worse than a rubber duck.
That and tech's status inflation means when we are talking about "mid level" engineers, really we are talking about engineers with a couple years of experience who have just graduated to the training wheels phase of producing production code. LLMs are still broadly aimed at removing the need for what I would just call junior engineers.
That and the fact that code does not live in a standalone bubble, but in a complex setup of OSes, APIs, middleware and other languages. My experience trying to use Claude to help me with that was disappointing.
I see the same results as my TabNine+Template generator+language server as I do with things like CoPilot. I get TabNine issues when the code base isn’t huge. I think also tossing away language servers and template generators for just LLM will just lead to seeking “proper predictive path”. Most of the time the LLM will spit out the create-express/react-template for you, when you ask it to customize it will guess using the most common patterns. Do you need something to guess for you?
It’s also getting worse because people are poisoning the well.
It’s kinda true though. They are increasingly used for those things. Sure, the results are terrible and doing it without AI almost always yields better results but that doesn’t seem to stop people.
Look at this nonsense for example: https://intouch.family/en
That’s one of the saddest bits of AI enshittification yet.
holy shit, if that isn't satire... wow, just fucking wow.
My feeling is (current) AI is more of a teacher than an implementor. It really does help when learning about something new or to give you ideas about directions to take. The actual code however still needs to be written by humans for the most part it seems.
AI is a great tool and does speed things up massively, it just doesn't align with the magical thought that we provide the ideas and AI does all of the grunt work. In general, always better to form mental models about things based on actual evidence as opposed to fantasy (and there is a lot of fantasy involved at the moment). This doesn't mean being pessimistic about potential future advancements however. It is just very hard to predict what the shape of those improvements will be.
Yeah, it's actually frustrating that even when writing Go code, which is statically typed, it keeps messing up the arguments order. That would seem to me a pretty easy thing to generate.
Although it's much better when writing standard REST and gRPC APIs
Well I have the exact opposite experience. I don't know why people struggle to get good results with llms.
LLMs are great for simple, common tasks, i.e. CRUD apps, RESTful web endpoints, unit tests, for which there's an enormous amount of examples and not much unique complexity. There's a lot of developers whose day mostly involves these repetitive, simple tasks. There's also a lot of developers who work on things that are a lot more niche and complicated, where LLMs don't provide much help.
In my experience this underrates them. They can do pretty complex tasks that go well beyond your examples if prompted correctly.
The real limiting factor is not so much task complexity as the level of abstraction and indirection. If you have code that requires following a long chain of references to understand, LLMs will struggle to work with it.
For similar reasons, they also struggle with:
- generic types
- inheritance hierarchies
- long function call chains
- dependency injection
- deeply nested structures
They're also bad at counting, which can be an issue when dealing with concurrency—i.e. you started 5 operations concurrently at different points in your program and now need to block while waiting for 5 corresponding success or failure messages. Unless your code explicitly uses the number 5 somewhere, an LLM is often going to fail at counting the operations.
All in all, the main question I think in determining how well an LLM can do a task is whether the limiting factor for your task is knowledge or abstraction. If it's knowledge (the intricacies of some arcane OS API, for example), an LLM can do very well with good prompting even on quite large and complex tasks. If it's abstraction, it's likely to fail in all kinds of seemingly obvious ways.
> If it's knowledge (the intricacies of some arcane OS API, for example), an LLM can do very well
Only if that knowledge is sufficiently represented in the training data or on the web. If, on the other hand, it’s knowledge that isn’t well (or at all) represented, and instead requires experience or experimentation with the relevant system, LLMs don’t do very well. I regularly fail with applying LLMs to tasks that turn out to require such “hidden” knowledge.
And if it's really well represented, then it's hopefully already in a superior library or documentation/guide, and the LLM is acting as an (untrustworthy) middleman.
Yeah, exactly this. If I ask Cursor to write the serializer for a new Django model, it does it (although sometimes it invents fields that do not exist). It saves me 2 minute.
When I ask him to write a function that should do something much more complex, it usually do something so bad it takes me more time because it confuses me and now I have to back to my original reasoning (after trying to understand what it did).
What I found useful is to ask him to explain me what a function does in a new codebase I am exploring, although I have to be very careful because a lot of time invents or skips steps that are crucial.
See, I recently picked up the Ash framework for elixir and it does all that too, but in a declarative, precise language which codegens the implementation in a deterministic way.
It just does the job that cursor does there, but better.
Maybe us programmers should focus on making higher order programming tools instead of black box text generators for existing tools.
> LLMs are great for simple, common tasks, i.e. CRUD apps, RESTful web endpoints
i gave it a yaml and asked it to generate a json call to rest api . It missed a bunch of keys and made up a random new key. I threw out the whole thing and did it with awk/sed.
Seriously though, what are you doing? Every single example everywhere throughout the internet that tries to show how good AI is at programming shows so mindbogglingly simplistic examples that it's getting annoying. It sure is a great learning tool when you're trying to do something experimental in a new stack or completely new project, I'll give you that, but once you reach the skill level where someone would hire you to be an X developer (which most developers disagreeing with you are, mid+ developers of some stack X) the thing becomes a barely useful autocomplete. Maybe that's the problem? It's just not a tool for professional developers?
I mean, let me just throw in an example here: I am currently working on https://guesspage.github.io , which is basically https://getguesstimate.com but for flowtext instead of a spreadsheet. The site is ... 99.9% Claude Sonnet written. I have literally only been debugging and speccing.
Sonnet can absolutely get very confused and break things. And there were tasks where I had a really hard time getting it to do the right thing, or understand what I wanted. But I need you to understand: Sonnet made this thing for me in two and a half days of part-time prompting. That is probably ten times faster than it would have taken me on my own, especially as I have absolutely no design ability.
Now, is this a big project? No, it's like 2kloc. But I don't think you can call it "simple" exactly. It's potentially useful technology. This sort of "just make this small tool exist for me" is where I see most of the value for AI in the next year. And the definition of "small tool" can stretch surprisingly far.
This is a simple project. Nobody is disputing that GenAI can automate a large chunk of the initial setup work, which dominates the time spent on small projects like this. But 99.999% of professional, paid software development is not working on the basic React infrastructure for a 2,000 loc javascript app.
Also your Google Drive API key is easily discoverable with about 15 seconds of looking at the JS source code -- this is something a professional software developer would (hopefully) have picked up without you asking, but an LLM isn't going to tell you that you shouldn't ship the `const API_KEY = ...` code as a file to the client, because you didn't ask.
> This is a simple project.
I mean, it would have taken me a lot longer on my own. Sure it's not a huge project, I agree; I wouldn't call it entirely trivial.
> Also your Google Drive API key is easily discoverable with about 15 seconds of looking at the JS source code
No, I'm aware of that. That's deliberate. There's no way to avoid it for a serverless webapp. (Note that Guesspage is entirely hosted on Github Pages.) All the data stored is public anyways, the key is limited to only have permission to access the stored data, and you still have to log in and grab a token that is only stored in your browser and cannot be accessed from other sites. Literally the only unique thing you can do with it is trigger a login request on your own site that looks like it comes from Guesspage; and you can do that just as easily by creating a new API key and setting its name to "Guesspage".
The AI actually told me that was unsafe, and I corrected it. To the best of my understanding, the only thing that you can do with the API key is do Google Drive uploads to your own drive or that of someone who lets you that look to Google as if my app is triggering them. If there's a danger that can arise from that, and I don't think there is, then it's on me, not on Sonnet.
(It's also referer domain limited, but that's worthless. If only there was a way to cryptographically sign a referer...)
> I wouldn't call it entirely trivial.
It just doesn't represent a realistic codebase. It is significantly smaller than a lot of college projects.
The current software system I'm working on now is ~2 million lines of code split across a dozen services.
AI has been pretty good for search across the codebases and absolutely hopeless for code gen.
LLMs just aren't that good yet for writing code on a decent sized system.
This is a ludicrously simple app and also - the code[0] is of very poor quality.
[0] https://github.com/Guesspage/guesspage.github.io/blob/master...
I am happy with the LLMs, but I only tried them on small projects done at my free time.
As a back end developer I am not familiar with the latest trends in JavaScript and CSS, and frankly I do not want to spend my time studying these. A LLM can generate an interactive web game based on my description. I review the code, it is usually okay, sometimes I suggest an improvement. I could have done all of that -- but it would take me a week, and the LLM does it in seconds. So it is a difference between a hobby project done or not done.
I also tried a LLM at work, not to code, but to explain some complex topics that were new to me. Once it provided a great high-level description that was very useful. And once it provided a great explanation... which was a total lie, as I found out when I tried to do a hello-world example. I still think the 50% success rate is great, as long as you can quickly verify it.
Shortly, we need to know the strengths and the weaknesses, and use the LLMs accordingly. Too much trust will get you burned. But properly used, they can save a lot of time.
Would you feel comfortable pushing generated code to production unaudited?
For my I have a company subscription for Copilot and I just use the line based autocomplete. It’s mildly better than the built in autocomplete. I never have it do more than though and probably wouldn’t buy a license for myself.
Would you feel comfortable pushing human code to production unaudited?
depends on the human.
but i would never push llm generated code. never.
-
edit to add some substance:
if it’s someone who
* does a lot of manual local testing
* adds good unit / integration tests
* writes clear and well documented PRs
* knows the code style, and when to break it
* tests themselves in a staging environment, independent of any QA team or reviews
* monitors the changes after they’ve gone out
* has repeatedly found things in their own PRs and asked to hold off release to fix them
* is reviewing other people’s PRs and spotting things before they go out
yea, sure, i’ll release the changes. they’re doing the auditing work for me.
they clearly care about the software. and i’ve seen enough to trust them.
and if they got it wrong, well, shit, they did everything good enough. i’m sure they’ll be on the ball when it comes to rolling it back and/or fixing it.
an llm does not do those things. an llm *does not care about your software* and never will.
i’ll take people who give a shit any day of the week.
I'd say it depends more on "the production" than the human. There are legal means to hold all people accountable for their actions ("Gross neglience" and all that). So you can basically always trust that people will fix what they messed up given the possibility. So if you can afford for the production to be broken (e.g. the downtime will just annoy some people) you might as well allow your team to deploy straight to prod without audits. It's not that rare actually.
Only on Fridays before a three day weekend.
I just asked Claude to generate some code using the SAP SuccessFactors API.
Every single example was completely useless. The code wouldn't compile, it would invent methods and variables and the instructions to go along with it were incoherent. All whilst gaslighting along with the way.
I have also previously tried using it with some Golang code and it would constantly add weird statements e.g. locking on non-concurrent operations.
LLMs are great when you are doing the same things as everyone else. Step outside of that and it's far more trouble than it's worth.
"LLMs are great when you are doing the same things as everyone else. Step outside of that and it's far more trouble than it's worth."
If you're doing something in a way it's not in the training data set, maybe your way of approaching the problem is wrong?
for obscure API or SDK, upload docs and/or examples to Claude projects.
I wonder what lessons the current hardware-intensive AI boom could learn from the history of proof-of-work (PoW) mining, particularly regarding energy consumption, hardware specialization, and market dynamics.
I can't wait for the AI bubble to be over so HN can talk about something else
You can always join the Wordpress tantrum discussions if you're getting fatigued, that ones really jumping around here lately.
I'm designing an extension of datalog instead.
I will take AI bubble over crypto bubble any day of the week.
I disagree, at least one day of the week should be something else, for variety. :p
Yes, I agree wholeheartedly and I actively dislike the concept of LLMs for anything real. But nothing will be worse than the flood of outright scams or attempts at rent seeking/middle man creation that crypto was. At least LLMs have some potential use cases (and line level auto complete is genuinely better now because of LLMs)
> I will take AI bubble over crypto bubble any day of the week.
I've been in the former since '21 and have seen every single cycle since 2011 in the latter, I can assure their is more dumb money in the former than in the latter; (just by scale alone) at least in the latter whether it was ICO or NFTs or whatever mal-investment was promptly punished (rugpulls/exit scams) companies like INTEL get to stay in Zombie mode because of the corpo-welfare that the US doles out while shaming everyone else to be prudent with their investments while these corps and banks spend like drunken sailors and tries to strangle the former out of existence (rightly so in most cases as most crypto is a total scam).
With that said, what you will see emerge are some incredibly established players in both fields that will have the staying power to change how the Industry is shaped around them: Nvidia and Bitcoin are comparable to one another in that respect.
Both have/had crazy volatility but the staying power and just the fact they remain firmly at the center of both Industry's is rather telling that you simply don't see what these technologies offer because of the hype and boom and bust cycles.
As a person who directly benefits from this: I can assure you most of these VCs are exit liquidity just as most foolish people are for the alt scams of yore, execpt the US economy (likely all of the Western World at this point) isn't entirely reliant on the promise of vapourware with 'crypto' in any capacity, weheras the same cannot be said about the theatrics of Jensen's Nvidia.
Source: I build data center infrastructure for these mega corps doing 'AI' and I'm doing an MSc in CS (Big Data) and been a Bitcoiner since Satoshi was still on BTF.
You might be right, but the crypto people were/are basically religious in their responses. I see a few loons for LLM's talking about how AGI is near, and of course there's the EA/LessWrong people talking about doomsday. But none of them were as staunchly dug in _and_ misinformed as the crypto folks.
edit: if it isn't clear I'm a staunch opponent of cryptocurrency in any form.
zifpanachr23 said:
> AI people sound more dug in to be honest from my perspective. But I guess that's cause the crypto stuff tends to be less overtly religious and more overtly batshit crazy politics and economics, which I'm much more used to dealing with haha. And mostly, everyone has figured out the scam by now on the crypto side.
>>The AI people freak me out cause they are all talking eschatology and shit as if they have stumbled upon the literal ark of the covenant like in raiders of the lost ark or something.
>>>It's a really great act to be honest. They've been clearly studying a lot of the more dishonest American religious culture of the last couple of decades.
I'm going to commit a HN faux pas to prove a point and show you why I think Bitcoin has a valid use case here alone: I decided to repost what he said because there are valid points here and are worth discussing.
Had I the inclination I can hash this into the blockchain for all to see what was written from this poster for the aforementioned reasons for as long as the mainchain continues to be maintained, protected and supported .
This has great utility, and the mere suggestion that you cannot get over that is because those 'crazies offend me and my disposition' and stop there you fail to see why and what this technology can already do--create an actual immutable archive of all Human history if we desire it.
But to his point, yes it's roots in Crypto-Anarchism (which started in CA at the inception of the rise of modern SV btw) has many of you questioning the ''sanity' and 'motives' behind this technology, and you assume they are all the same but rest assure their is a reason for the brain drain from all of tech/STEM/finance during my era and time in Bitcoin.
Most of whom are now incredibly wealthier than they ever were working in academia or private industry if you think money is a measure of one's success--I don't, but most of you do.
The AI people strike me more as a range of the introduction corpos from banking and academia into bitcoin (Gavin's, Hearn) to total con men like Veer and sprinkled in there are the cult memebers you mentioned who honestly think that their techno-utopian trans-humanist dreams are being built one LLM update at a time. It's sad... it's the same thing just different names/faces.
No one cares that blockchains are immutable, that doesn't mean that information written there is correct. Just that it was written on X date and the content. You could find proof of the biggest scandal of all time and post it on the blockchain so "the man" can't stop the word from getting out but 99.99999999% of readers would read a version presented by a simple web server and with the value cached in a closed database or memory. Which if the government wants to take down, it can. And if that does go down, no one will save a link to the entry on the blockchain. And on the flip side I could write obvious falsehoods in the same way. Blockchain provides no value for legal attestation nor information distribution.
In practice, the vast majority of blockchain ledgers record the history of scams, penny stock style trading and money laundering attempts.
> In practice, the vast majority of blockchain ledgers record the history of scams, penny stock style trading and money laundering attempts.
It doesn't take long before they make themselves present. Thanks for proving my point.
> edit: if it isn't clear I'm a staunch opponent of cryptocurrency in any form.
It's very clear, but your exposure to zealots, on both sides, shouldn't deter you from being objective and seeing what these technologies actually offer. Hence why i wrote that part in the 2nd to last paragraph.
HN has such misinfored vitriol for any technology it didn't ordain itself, you sem to be of that cohort, what's odd is that very same people who gave you VC.SV funded startup land are all major backers of this technology.
I can just summarize this in one phrase: you seem to collectively not know, what you don't know and make leaps in logic and misinformed judgments from that POV.
I have little interest in the ycombinator legacy of hustle culture, growth hacking, get-millions-for-glorified-todo-app, etc. As far as I can tell it is effectively the underlying reason for why crypto and AI get hyped up to the point where we can't have reasonable discussions about them in forums.
I'm just here because it happens to be where like-minded people (_sometimes_) hang out.
> I have little interest in the ycombinator legacy of hustle culture, growth hacking, get-millions-for-glorified-todo-app, etc. As far as I can tell it is effectively the underlying reason for why crypto and AI get hyped up to the point where we can't have reasonable discussions about them in forums.
Ohh, well... I'm guilty of drinking that kool-aid, unfortunately and was a boot-strapping fintech founder with the battle scars to show--those scars appear in this discussion thus far by the way.
But I get it, I'm trying to be amicable about this as much as I can, especially fbecause I know no one can deny us anymore in the BTC side at this point, and in the AI side... well, money-hype boom and AI-doomer pr0n cycles aside we are actually building amazing amounts of compute that hopefully can yield amazing results, something like Starship recovery system levels of advancement in many other Industries/Sectors one day, probably very far off in the future but that's the start, right?
Every forest starts with a sappling kind of a thing, and these two technologies emerged from a time when I was in my formidable development period so I saw them as something more than just the marketing side--AI is marketing, ML is really just stats with code to back it up after all. Bitcoin is just a FOSS network using token based cryptographic key encryption.
Stop using throwaways if you want real conversations, it might help you have those conversations you're looking for. :)
Of the top 30 HN stories of the last month (https://hn.algolia.com/?dateRange=pastMonth&page=0&prefix=fa...)
only 6 were AI, the highest being "OpenAI to become for-profit" coming at number 10. Top story was "Bop Spotter" followed by Starship and click to cancel.
I think it will burst when the Fed realizes inflation is not done and start raising again in 6 months. They can only feed the bubble for so long before the common people have had enough of rising prices.
The Fed raising rates will increase inflation at this point (and further increase fiscal deficits), nothing stops that train.
Arguably if the investment here works out we’ll see deflation through extreme technical advancements.
No, raising rate would bring the economy to a slower pace and reduce private sector consumer demand. Private sector investment can continue to increase but at some point that too will hit a brick wall. The public sector spending depends on which type of big ego people get to make decisions. Given the extreme excesses so far it can go either way. The now extinct fiscal conservatives might just make a return finally but don’t hold your breath.
Raising rates works in the beginning with high fiscal deficit driven inflation by slowing demand and bank lending.
But raising interest rates and keeping them high in an environment where runaway government deficits and high government debts are causing inflation runs the risk of exacerbating inflation.
You have high interest rates on a large amount of government debt which continues to push _more_ money into the economy.
The Fed doesn’t have any real options at this point but to lower rates.
> keeping them high
At some point we need to address the elephant in the room and ask people specifically what they mean by "high" rates, because 5% isn't particularly high by historical terms, it's only high for people who never paid attention to interest rates before 2010.
Public sector demand is a much smaller percentage of the overall economy. If raising rates did not slow the economy due to high government deficit spending, then we would certainly be living in a much different world of command economy with the government running everything. That’s not yet the world we live in.
Another possibility is that the rate is still not high enough and needs to be raised much much higher to stop inflation. I think rates need to be in the 6 to 7 percent to really stop inflation. The is just a pause. It will come back like a vengeance.
Ask Argentina about that. They started finally reducing rates and its working some.
Wow! Rare to see someone get it. MMT follower ? Would add that the money printing is being distributed proportionaly to the wealthy under a Deomcrat regime. Pretty sad.
People are going to be talking about AI for the rest of your life, but feel free to go join an Amish community or live in the woods, maybe get a job as a Firewatch.
There is an interesting contrast between the phenomenal investment boom versus functionally zero job growth...
> Even the software publishers and computing infrastructure industries at the forefront of this AI boom have seen functionally zero net employment growth over the last year - the dismal job market that has beleaguered recent computer science graduates simply has not improved much.
... which may explain, in broad brush, the polarised HN attitude: bitter cynics on one side and aggresive zealots on the other.
I don't take financial advice from HN and it has served me well.
How do you know if you had have taken some advice, it might have served you better?
Indeed, downvoted advice has done great.
Question:It seems like the requirements needed to become an investor in a startup is a multimillion pool of money...which is not what a salaried professional has at their disposal. Yet, a lot of the 100x opportunities come before public offerings, making a whole class of investment unavailable to modest but knowledgeable professionals. I know that sometimes, through personal connections, pools of investment money are built across individuals, but as a public service it seems like, no, it is not.