One of the genuinely positive things about tools like Copilot and ChatGPT is that they empower people with minimal development experience to create their own programs. Little programs that do useful things - and that’s awesome. More power to the users.
This guy didn't just make a small program, he made a product that is attuned to some very stringent requirements. If you gave this project to experienced software developers it would be heavy on process and light on impact, take forever to make, and stupid expensive.
Yeah, the most important part of that video is where he says: "it works well enough". I think there's lots of room for this kind of bespoke software that's fundamentally unmaintainable -- and not something anyone could reasonably afford to make with professionals -- but good enough for purpose. This seems like a great, life-changing example, but it's still subject to the same thing the article is talking about.
I've had some free time recently so I've been trying to use the various AI tools to do a greenfield project with Rails. I know the stack intimately, so it's been a useful exercise as a check against hype. While I can occasionally trust the tooling to make big changes, most of the time I find that even the small changes are wrong in some important, fundamental way that requires re-writing everything at a later time. Asking the models to fix whatever is broken usually results in an endless cycle of (very junior-like!) random changes, and eventually I just have to go in and figure out the fundamental restructure that fixes the issue.
Would this be good enough for someone writing a one-time-use, bespoke piece of software? Maybe. Would they be able to maintain this code, without having a truly senior-level ability to work with the code itself? No [1].
I'm less fearful of LLM taking experienced programmers' jobs [2], because I fundamentally think this kind of stuff plays in the same end of the market as knockoff designer goods, Ikea furniture, and the like. It makes custom software available to an entire customer segment that couldn't have afforded it before, but nobody is going to confuse it with a product made by a team of skilled professionals [3,4].
[1] But to a really inexperienced developer, it looks the same.
[2] I am more fearful of clueless business types, who think this technology obviates the need for programmers.
[3] ...but will unskilled professionals be under pressure? Maybe!
[4] ...and what about skilled people, using the same tools? My current hypothesis is they're beneficial, but ask me again in six months.
As you said, I think also the key differences between skilled and unskilled devs in the context of AI tooling are :
1°) Know how to diagnose and fix critical problems by themselves, because there will be bugs in production for which AI won't be any help
2°) Write maintainable code, AI doesn't care at all of maintainability of code while devs should (must imho) consider pasting AI code as a merge/pull request.
For 2°) IIRC, I read somewhere that more and more code published on i.e. github is AI-generated. While this means the quality of code is decreasing, i.e. Copilot is also trained on more and more % of AI-generated code. So the quality of AI-generated code is decreasing globally.
As time passes, I think skilled devs will become more skilled and unskilled will become more unskilled.
I think it's already happening. I've been using Claude a lot since 3.5. For a while, I barely bothered to check what it was doing because the code it generated tended to be fairly straightforward and usually worked on the first try. But in the last few weeks, I've started reading everything line by line again, because now it's not only often incorrect but also written in a goofy, convoluted way. AI-generated code being constantly fed back to it in a feedback loop would explain a lot, I think.
My experience too, in many cases, is that I already know (even if in a vague sense) how to do the thing I'm trying to coax out of the model, but a quick prompt "split this string on $CHAR" or whatever is much faster than me fiddling with forgotten bash syntax.
This approach is different than blind trust in the output from a place of ignorance.
I love this so much. It's hacky, janky, and absolutely phenomenal and life changing. This is like perfectly in the spirit of the hacker ethos. This dude scraped together pieces around him and made a useful product that solves the problems he wants to solve, the way he wants to solve them.
The code is probably garbage. You can tell the UI is weird and jumpy. He's using timed delays to manipulate the browser with keyboard keys. The whole thing would be absolutely shredded here. But it's beautiful. It's crafted for function over form with duct tape and bubble gum but it's the difference between his brother being locked helplessly alone in his body, or being able to express himself and have some level of autonomy. It's just awesome.
LLMs rule for this kind of stuff. Solve your problems and move on. The code is not so precious. You'll learn many things along the way and will be able to iterate quickly with a tireless and semi competent but world knowledgeable assistant.
It sounds like you're thinking about a "hack", as in quick and dirty solution.
But that does not relate to "hacking", and most certainly does not relate to "hacker ethos". And while "hacking" is sort of ambiguous, I don't recall ever seeing "hacker ethos" defined by quick and dirty solutions.
No, though those concepts are of course related historically, I'm not confusing the two.
The hacker's ethos is about freedom of information, freedom of access (apropos to this video about vibe coding accessibility tools), community, collaboration and so on.
I've never seen or heard "elegance" associated with hackers, hacking, the hacking ethos etc. If I were to go further, elegance as it is used here has some anti-hacker connotations, i.e. it is a gatekeeping term to separate "true" coders from "amateurs" which is very anti-hacker.
Anyway, I don't think I'm conflating these terms, but it could be true that my understanding of the hacker ethos is wrong.
I didn't look at this video, but be vigilant when seeing one, as I was also surprised by someone demonstrating what they can do with Cursor and I went so far to install the exactly the same version of the app, use the same model and everything (prompt, word capitalization...) I could gather from the video and the results were nowhere near what was demonstrated in the video (recreating mobile web page from screenshot).
I know that LLMs are not deterministic machines, but IMO there is a lot of incentive to be "creative" with marketing of this stuff.
For the reference, this was less than two months ago.
The funny thing is someone somewhere will find a way to make millions off a vibe coded project. I just suspect it will be the outlier and they will be well positioned to make that happen.
And for the rest of the companies that embrace the 30%~ efficiency spike, it will just accelerate our work goals faster.
I like to use it on stuff that we wanna do to enhance the UX but rarely sees the light of day. Plus my wrists have never felt so good since replacing boring syntax choirs with LLMs.
Hence why software developers are out of job, eventually.
The day of only a few people around the factory floor to babysit the robots will come, but lets keep celebrating the day they start unloading them from the delivery trucks for installation.
It's not clear if the overall number of needed software developers will decrease or simply stop growing. The amount of software necessary in the future will only keep growing. Using the same folks to produce more is a large possibility. I imagine it will take decades to see a shrinking workforce that meaningfully displaces many existing software developers.
Not everywhere in the globe, the difference now is that those overseas teams can be reduced as well, as we outsource their work into AI managed systems.
Yes, the difference, like on those factories is that instead of the 100 that were on site when the trucks arrived at the factory, there will be 10 of them left.
I have a lot of uncertainty about the number of professional software developers there will be in the long run, but not about whether they'll be using LLMs.
Why do you not read the article? What you are saying is just total nonsense if you had actually read the article.
>If you gave this project to experienced software developers it would be heavy on process and light on impact, take forever to make, and stupid expensive.
No, it wouldn't. Every single software developer knows how to make programs which "work well enough". This claim is totally ridiculous, it also is not a "product" in any meaningful sense.
What the article is pointing out is that you can not sell software like this. It is not a "product" at all, it is a hobby project. And you can not develop a hobby project like you would develop a commercial product.
Just think for a second what it would take to sell this.
>they empower people with minimal development experience to create their own programs.
The author goes out of their way to play up the toy aspect of the LLMs when driven by the inexperienced. No mention is made of them being used by experienced developers, but I get the impression he feels they aren't useful to the experienced.
I'm just playing with a small client/server tool (rsync but for block devices), and vibe coding allowed me to figure out some rough edges of my design document, and experiment with "what would it look like built with threads? As async code? As procedural?) I would never have done that before because of the sheer amount of time it would take, I'd have picked what I thought was best and then lived with the sunk cost falacy until I had no other choice. It did a pretty good job of catching reasonable exceptions in the communication code which I probably would have just let throw a traceback until I got a bug report. It did a great job of adding logging and debugging options. And I had it take 4 attempts at making a "fancy" --progress display and picked the one I liked the best.
LLMs give a level of exploration to software development that we haven't seen since the good old days when HP set up several groups to each build their next generation workstation environment, and then picked the best one.
IMHO, the experienced software developers stand at a unique position to take advantage of these LLMs that far outstrip what an inexperienced developer can do.
> vibe coding allowed me to figure out some rough edges of my design document, and experiment with "what would it look like built with threads? As async code? As procedural?”
This doesn’t fall under my understanding of the phrase “vibe coding”. In the tweet from Karpathy which many point to for coining the phrase, he says that when vibe coding you essentially “forget the code exists”. I think it’s distinct from regular ol LLM-assisted development
I surely did a hybrid approach, but for a large swath of it I was pretty purely "vibe coding", where I wasn't looking at the produced code at all.
I initially had Gemini 2.5 build some code and then did a code review of it. I tweaked my design document and then had Claude and ChatGPT take a stab at it and at this point I wasn't looking at the code at all. These implementations had some deadlock or early termination problems so I asked for some different async and threaded implementations, but ultimately they went off into the weeds in ways I couldn't really reason about, so I went back to a straight select/procedural implementation and had it start tracking outstanding blocks, which got something working reliably. Then I asked for fancy progress displays, tried several attempts at that. Then at this point I started cleaning up the code some, had it add type annotations, etc...
Big swath of pure or nearly pure vibe coding in the middle there.
Running the first generated version to test, I realized I needed to specify: "Make sure that the client processes messages from the server while it is generating and sending hashes of the source file", because the first implementation would spin on generating the hashes and deadlock because the server end would write messages back to the client that the client wasn't reading and fill up the output buffers, which would fill up the client output buffers once the server stalled. This was obvious just from running it and knowing how all this stuff works (rather than by code review).
Then I started seeing issues with how I had phrased the terminal conditions, the client terminating immediately upon reaching the EOF on input file, and not reading any further server messages. I considered making a 3-way ACK to signal end, but instead changed the design to tell it to keep track of outstanding un-acked packets and only send the EOF/EOFACK when there were no outstanding messages.
The design is that the client generates a bunch of "xxhash64" messages for each block of data, and the server replies to each one with either "ok" (I already have that data) or "send" (send me that data). The "send" is replied to with either "data" (here's the data) or "zero" (the block is entirely NUL).
You can examine the code at any time, ask for an explanation, and even make changes yourself. In other words, you get out of it exactly what you put into it.
If you choose to let the AI do all the coding and never dig into the actual code, you'll learn more about how to communicate with an LLM than you will about how to write code.
That sounds like a similar failure rate for using newer models in technical contexts. Especially for popular technologies where there are enough resources such that someone other than the original creators could reasonably write a book about it.
I think that's an oversimplified metric when comparing the ability of an AI to analyze and describe code. Imaging you're reading a book on a challenging new topic. Even if the book is 100% right, there will be many passages you don't understand, and may even misinterpret. A book won't correct you, offer alternate explanations, or break down concepts any further than what was already written.
An LLM, for all its flaws, can do these things exceedingly well (provided it has good source material, of course). This gives you the ability to get answers to your questions and correct your mistakes. I'd argue that these benefits far outweigh the errors an LLM makes in breaking down and explaining material, and make an LLM better than a book in many (not all) cases.
> Even if the book is 100% right, there will be many passages you don't understand, and may even misinterpret.
Which is why no one reads only one book on a particular subject, especially if one is new to the domain. Often you bring in an alternate, but correct perspective from another book. The risks of hallucination with LLMs' responses outweighs the advantage of multiple perspectives (even if a broken clock is right twice a day, you still want a working one)
No one reads only one book on a subject? People act on mere headlines all the time. Reading one whole book would be a huge win over the status quo
> The risks of hallucination with LLMs' responses outweighs the advantage of multiple perspectives
You can ask a model a question multiple times. You can ask different models the same question. You can ask the same question different ways. You can do this automatically with agentic frameworks or reasoning models. "Multiple perspectives" are compatible with language models.
> You can ask a model a question multiple times. You can ask different models the same question. You can ask the same question different ways. You can do this automatically with agentic frameworks or reasoning models.
LLMs are for generating text, not producing knowledge. The actual knowledge come from the signal/noise ratio of the training data, not the training method.
It's like recording a violin in a noisy environment, encoding it with a low-bitrate lossy encoder and trying to get the PCM of only the violin back.
We've tried to use a better environment for recording specific notes from the violin (better training data), adjust the parameter of the lossy encoder (tuning the model), mix other violin samples to the respone (RAG, MCP), and redefining what a violin should sound like (LLMs marketing).
But the simpler solution is to record it correctly and play it later (AKA going to the source).
I would guess it's been around 1-2 years since you last tried seriously using a language model for any practical use cases, right? It makes sense to not use them if that is your experience, but it's hard to imagine anyone saying this who has actually been following along for the last few years to now. SOTA from 2 years ago now comfortably runs on a single GPU. A high end GPU can easily out perform SOTA from then both in (general!) knowledge reproduction, and capabilities (including saying things like "I don't know" when you ask beyond the encoded knowledge).
That is nonsense. A competent human will provide you with an explanation that is based on logic. LLMs just put together words that are statistically most likely to be used together given the current input. That is why you can get hallucinations with LLMs but not with humans.
That is a question that is very near and dear to my heart right now because my son is 15 and very interested in programming. I don't have any good answers yet, but you definitely need to know so many of the core concepts to be able to steer the LLM around pitfalls.
I agree they may not and that’s very interesting. As an experienced developer myself I’m starting to think these tools are likely going to increase our market value not decrease it.
It's not implausible that we'll wind up with something like the extremely long-lasting market for COBOL developers. A niche skill that's absolutely essential in a few places, and which can command high salaries as a result... for a few people. Only so many huge legacy banking systems, after all.
The problem, as with COBOL, would be the loss of the existing pipeline that created new competent high-level developers.
If I wanted to open a large world class fast food chain, I wouldn't be cooking home meals then. That would be silly.
Copilot can help me cook the equivalent of a McDonalds special in terms of software. It's good, I think McDonalds is delicious.
But it cannot help me cook a home meal software. It will insist that my home made fries must go in a little red box, and my fish sandwich needs only half a slice of cheese.
Deeply into that metaphor, maybe someone who has only worked for fast food chains might forget that a lot of good industrial dishes are variations of what previously were home cooked meals.
I am glad that copilot can help young cooks to become fast food workers. That really looks like something.
Well, I take pleasure in cooking home meal software. Can you make a copilot that helps with that?
You know what, nevermind. I don't need a copilot to cook home meals. I have a bunch of really good old books and I trust my taste.
It's not like some big company is going to be interested in some random amateur dish, is it? It was definitely not cooked for them.
Asking the LLM for something is like doing an "Ask the Audience" in Who Wants to be a Millionaire. You're basically polling the consensus answer to the question you're asking.
> You're basically polling the consensus answer to the question you're asking.
That's perfectly fine, because that's what you expect from human work to begin with. From natural text like blog article or tecnical reports to software changes, all output is expected to comply with patterns we are already familiar with. Heck, look at pull requests, where you ask your audience to evaluate your work hoping to reach a consensus.
You'd expect that from an average person, but not a genius in the subject. "Genius" is almost by definition a contrarian viewpoint or technique that happens to be better than the consensus or the best-known.
> You're basically polling the consensus answer to the question you're asking.
There is nothing wrong with going with the consensus answer. Sure, you can't invent an O(N) sorting algorithm with the consensus, but LLMs can definitely write good, maintainable code. Michelin star analogy might be an exaggeration, but it can help cook a good homemade meal, possibly even better than most wannabe home cooks.
>There is nothing wrong with going with the consensus answer.
For most things, yes. But you don't hire experts to tell you what everyone else knows. You hire experts to give you the angle.
For example, an LLM trained on all AWS documentation wouldn't be super useful (possibly less useful than traditional full-text search), because what you really want to know is the things that AWS didn't write, the writing "between the lines", all the things that DynamoDB can't do.
I found one of my license texts for a hobby project[0] in a Microsoft product[1]. I have no idea how they use it.
Copilot is very poor at understanding how the DSL arrangement of the project works or how to assemble validator trees. Before Copilot many other "enterprise" stuff like JetBrains IDEs had problems auto-completing it. But it was always good enough for me, for my own things.
It obviously have value. Maybe they're lazy and using it for something unrelated like the TLD list. I really don't know. What I know is that something of my amateur dish ended up being served in what I consider to be a fast food chain.
No, copilot is not endlessly variable. It works best with typed languages and an enterprise mindset. That's exactly the point against "vibe coding" they push. MS seems to despise anything that doesn't follow that.
Look, I love typescript and all that stuff. As I said, McDonalds meals are delicious.
But I don't want that approach for everything I code. This project is not the only one with that issue. I noticed the same in a portable shell compatibility layer, it can only help if I make it to look like other script languages.
"You're asking too much, how can copilot help with a language it was not trained on?"
I am not asking it to be good at that. Just don't make asshole posts saying anything other than MS pasteurization is "vibes".
If they paid me, then that would be another story. But they didn't, and used my "vibe" project anyways.
I am not complaining about software I released for free, before anyone tries to go for that.
I am making an observation about kinds of software that copilot can't help with. My kind of software development, in which I might take months to decide a name or redo everything several times just to try a new kind of sauce.
> No, copilot is not endlessly variable. It works best with typed languages and an enterprise mindset.
I don't think your assumption is true. The key factor is the corpus used to train it, and the context you fed it. I already experienced Copilot fumbling references to methods of a simple Java class, whereas it pulled off thinks like ARM templates flawlessly. You need to understand that internally LLMs do probability-based text completion. If you feed them enough context to maximize the probability they output things you expect, they do so.
Right, but that's exactly what static typing does in most cases - it provides model with more explicit context for what it's doing.
For the same reason, they tend to handle XML better than JSON - sure, both are trees, but XML is has redundancy, and that redundancy helps keep the model on the rails, so to speak.
I actually wonder if the perfect LLM language would be something a lot more like COBOL - not in a sense of being similarly high level, but rather verbosity. And perhaps also being closer to natural English, which is, after all, still a lot of its training set, especially for reasoning stuff. For query languages they seem to like SQL the most of all the things I've tried, and I strongly suspect that it's the same underlying cause.
Of course, a new language designed like that would have the fundamental problem that there's no existing corpus in such a language. Then again, if it is also designed such that one can reliably convert e.g. from Java to that language, then perhaps we can still pull that off.
> For query languages, at least, they seem to like SQL the most of all the things I've tried, and I strongly suspect that it's the same underlying cause.
The reason they like SQL more than other query languages is primarily that their training data has orders of magnitude more of it than any other query language, and that advantage is so huge that any other possible advantage would probably have comparatively negligible effect.
I'm not so sure about that. This was in comparison to e.g. walking object graphs in Python, C#, and JavaScript using the usual APIs (i.e. where querying one one-to-many relationship looks like Foo.Bars rather than joins, and using map/fold/filter or equivalents). I would expect there to be a lot more of that kinda stuff in the training set than SQL.
I am in the middle of my third AI assisted project. I disagree ~90%.
If you prompt an LLM like an architect and feed it code rather than expecting it to write your code, both ChatGPT 4o and Claude 3.7 Sonnet do a great job. Do they mess up? Regularly. But the key is to guide the LLM and not let the LLM guide you, otherwise you'll end up in purgatory.
It takes some time to get used to what types of prompts work. Remember, LLMs are just tools; used in a naive way, they can be a drain, but used effectively they can be great. The typing speed alone is something I could never match.
But just like anything, you have to know what you're doing. Don't go slapping together a bunch of source files that they spit out. Be specific, be firm, tell it what to review first, what's important and what is not. Mention specific algorithms. Detail exactly how you want something to fit together, or describe shortcomings or pitfalls. I'll be damned if they don't get it most of the time.
> Be specific, be firm, tell it what to review first, what's important and what is not [...] Detail exactly how you want something to fit together [...]
You mean like in a fast food chain?
I know how to use it. All of that was already implied in my original comment. Sometimes though, I want to cook without a rigid mindset.
They've been around forever and I believe number 2 or 3 in number of stores open in the US. You don't get that way without some people liking your product.
It's fine that some people don't like it, but is ridiculous to think your tastes apply to everyone.
Obtuse maybe, but I'm not missing the point. Their point is nonsensical, given that LLM generated code isn't "as good as fast food compared to home made meals" or even comparable to "food".
The point is, for most projects there is nothing wrong with writing McDonald's level code. It's not as healthy or tasty as a homemade meal, but it will perform its main function (i.e. prevent you from starving).
> Copilot can help me cook the equivalent of a McDonalds special in terms of software. It's good, I think McDonalds is delicious.
No. The likes of Copilot help you cook the meal you'd like, how you'd like it. In some cases it forgets to crack open eggs, in other cases it cooks a meal far better than whatever you'd be able to pull together in your home kitchen. And does it in seconds.
The "does it in seconds" is the key factor. Everyone heard of the "million monkeys with a typewriter" theorem. The likes of Copilot is like that, but it takes queues and gathers context and is supervised by you. The likes of Copilot is more in line with a lazy intern that you need to hand-hold to get them to finish their assignment. And with enough feedback and retries, it does finish the assignment. With flying colors.
Cook a meal, yes. "How you'd like it", no. You have to use the languages that have a large enough presence in its training set that it knows them well, and ditto for frameworks and libraries. Once you step out of that zone, quality rapidly plummets.
"Does it in seconds" is kinda true if you ignore the "supervised by you" part. That part - reading through the generated diffs to catch (sometimes very subtle) errors - can take a while.
And sure, you'd have the same problem with the intern. But the intern learns from you. The LLM will make the same mistakes again in the same circumstances. Yes, I know they do have memory now, but it my experience that RAG-based stuff is not particularly reliable. I suspect this might improve once hardware is fast enough that you can use a (smallish) helper LLM for RAG with reasonable perf, but we aren't quite there yet.
My main takeaway from vibe-coding is that nobody cared enough to fill that niche and expectation. And it was really frustrating, yet we're getting there through convolutated, inefficient and borderline barbaric means.
People are still lamenting after HyperCard. Automation on windows or macos didn't go anywhere. Shortcuts were a better step into that direction but I feel it got stuck in the middle as Apple wasn't going to push it further. Android had more freedom yet you won't see a "normal" user do automation either.
If we're going to point the middle finger at vibe-coding, I wish we had something to point to as a better tool for the general population to quickly make stuff they want/need.
(Doing it as a professional dev is to me another debate, still with nuance in it IMHO. I'd also love better prototyping tools to be honest.)
Python is not exactly a hard language to learn. But the truth is that people generally don’t like to devote time to what they deem unimportant even if the end result would be useful. I’ve seen smart people go though dense mathematics and physics books, but refuses to take an introductory course in programming.
Yeah, you tell it what you want and it gives you something, then you tell it how that's not good enough and it gives you another thing, and you repeat until you have something that works for you (but, as the author notes, is hardly a real product that could be maintained and shipped to real customers.)
The input is undefined char*, and that's the magic of it. It's kinda funny we're reaching the phase where some people throw Syntax Error on a missing colon, but AI be like "I want sfssfsfsffsfs" "ayyy got you covered". How the tables have turned.
Yeah this is how I interpret it. Type text into the chatbot, copy and paste the outputted code. If it fails to compile, paste the error message and get the resulting code. Similar if it fails at runtime. Supply the LLM with additional code files where necessary but don't really look at the code, just copy and paste in, copy and paste out.
Copy paste? Use something like desktop commander and just let it edit the files for you directly. It'll even run commands to test it out. Or go further and use Cline/RooCode and if you're building a webapp it'll load your page in a small browser, screenshot the contents, and send that to the model. The copy-paste stuff is beginner mode.
When I've been experimenting with Cursor I've found that one of the essential parts is the agent MUST be able to close the loop itself for it to be useful in any real sense.
So if you are having the agent write a script to call an endpoint and do something with the result. Have it write a script that does that and then ask it to run the script itself. It will then be able to close the loop by itself and iterate on the code until it does what you want (usually).
It is also very useful to ask the agent to write a test for something and then run that test to close the loop in the same way.
The vibes are much better if you just tell it to run a certain command, then it can see the error message and you don't have to paste back and forth so much. You can go make a cup of tea and it'll just iterate on changing files and running that command until it does the desired thing.
This is what AI-oriented IDEs like Cursor streamline - the LM gets immediate feedback from things like code completion, linter, build errors, test failures etc, and can immediately iterate without you having to do anything about it (sometimes overly aggressively even tbh).
Copy paste is so last year, just turn on agent mode in VS code or Cursor. It took me only a single prompt to convert a set of messy razor pages into a nicely structured React application.
How else are people using LLMs to code? Other than this?
Its so fast to do these things. Seems dumb not to use an LLM to interpret an error message if you don’t immediately know the problem. And if it doesn’t work you can alway abort and do it yourself. The whole code gen and debugging process might take 45 seconds.
Sure its dumb if you’re writing some short simple function. And you need to be in the loop and understand all the changes you are making in a professional setting. But that just sounds like the basic workflow of coding with an LLM.
It’s implied that programs are personal but product code goes through peer review, checking the premise of the change as well as the implementation.
When someone reviews vibe coded patches and gives feedback, what should the reviewer expect the author to do? Pass the feedback onto their agent and upload the result? That feels silly.
How has code review changed in our brave, vibey new world? Are we just reviewing ideas now, and leaving the code up to the model? Is our software now just an accumulation of deterministic prompts that can be replayed to get the same result?
With the Altair you had switches to input bits of CPU instructions. Then punchcards. Then Telex. Then assembly in a terminal. Then C, Pascal and Smalltalk. Then Java, C++, Python, PHP. Then the mountains of libraries and frameworks, realtime indexers, linters, autocomplete, suggestions.
The next step seems that we will write programs in prompts. Maybe we will get a language that is more structured than the current natural language prompts and of much higher order than than our current high level programming constructs and libraries.
You can’t eliminate formalism in programming. You can only abstract it away by creating a metaprogramming layer for the current one. And prompt isn’t that. Prompting is a blind shot hoping to hit something good.
The real problem with "vibe coding" (or any coding, or with any product what so ever) is that the end user probably is not a programmer, and therefore cannot have professional judgement on the quality of the implementation. The only thing typical end user care is the perceived behavior, not the implementation details.
As it is easier and cheaper to do anything, the result is low-quality products. This of course serves well for the real professionals, who can now sell premium quality products at premium pricing for those who know and want the quality.
It depends. Market has demonstrated time and again that premium quality is a rather niche product. Is the niche large enough for all the existing "real professionals" to fit into it?
I am happy that there will be work in security for good while. They might first pass run LLMs on that too. But it will miss things and that is where I can come in...
Right now there are thousands of apps getting built with AI that are going to work surprisingly well, will get embedded into all sorts of fundamental processes — and will start to perform unacceptably badly in two or three years. God knows what we do with this information but it seems clear we are setting up for a tsunami of horrible performance regressions
Actually even in the years before AI code generation started, I noticed an increase in code bases that seem very well architected and engineered. But once you start using the code for a week you notice it's full of bugs and the consistent architecture is just make believe. Surely this problem is getting way worse with generated code that is optimized to read like it's very well engineered
This article starts to touch on an important distinction between software and products, but seems to miss the larger picture.
With the cost of developing software dropping to near zero, there are a whole class of product features that may not be relevant for most users.
> You haven’t considered encoding, internationalization, concurrency, authentication, telemetry, billing, branding, mobile devices, deployment.
Software, up until now, was only really viable if you could get millions (or billions) of users.
Development costs are high, and as a product developer you're forced to make software that covers as many edge cases as possible, so that it can meet the needs of the most people.
I like to think of this type of software as "average" -- average in the sense that it's tailored not to one specific user or group of users, but necessarily accommodating a larger more amalgamous "ideal" user.
We've all seen software we love get worse over time, as it tries to expand it's scope to more users.
Salesforce may be the quintessential example of this, with each org only using a small fraction of it's total capabilities.
Vibe coding thus enables users to create programs (rather than products) that do exactly what they want, how they want, when they want. Users are no longer forced into the confines of the average user, and can instead tailor software directly to their needs.
Is that software a product? Maybe one day. For now, it's a solution.
The argument is that vibe coding is great for little personal programs but bad for full-blown products.
I don't think you can ship a fully baked product made exclusively with AI coding yet. But it's *really* useful for investigative product development - e.g. when you have a few different ideas for how something might work but aren't quite sure what's best. I used to regret that I never mastered Figma or other mockup tools - now I never will have to.
I agree with the article, but that's not how the vibe coders see themselves. From their perspective they can't see the gap between programming and product, and in my experience are pretty hostile to feedback from real software engineers.
This is about the problem with vibe coding your way to a product. But if the vibes are strong enough, maybe we don't need products at all?
They do tend to complicate things, with all of their moats and such. I never wanted a product that did the thing, I really just wanted to do the thing. "Works on my machine" might be good enough if you're unlikely to want to repeat yourself on a different machine.
> I never wanted a product that did the thing, I really just wanted to do the thing.
Would you vibe code your daily driver car's ECU or a high-frequency trading application to use for your 401k? If you were to do these things (more power to you), I rather suspect you'd still do a whole lot of research and critical thinking beforehand, which sort of obviates the "vibes".
It is hard to imagine trusting an AI that much. But plenty of things happening today would've been hard to imagine by the people who came before. So yeah, let's make that assumption.
At that point, I don't think we'd be using the AI to code something and then switch to a mode where we're now using the recently-coded thing.
We'd just ask the the car to go, and it would happen. Then we'd suggest a strategy for making trades, and that would happen. If the AI decided to write some code and fork a subprocess along the way... that would be an detail that we'd be unaware of.
Yeah I agree. That was my main takeaway... just because you have a running program doesn't mean you got a product... the vibe-coding thing is just the flashy headline
It throws away decades of software engineering principles in favour of unchecked AI output by clicking “accept all” on the output.
You would certainly NOT use ‘vibe coded’ slopware that powers key control systems in critical infrastructure such as energy, banking, hospitals and communications systems.
The ones pushing “vibe coding” are the same ones who are all invested in the AI companies that power it (and will never disclose that).
An incident involving using ‘vibe coded’ slopware is just waiting to happen.
‘OUTSOURCING’ isn’t software engineering.
It throws away decades of software engineering principles in favour of CHEAP output by clicking “I AGREE” on a contract.
You would certainly NOT use ‘OUTSOURCED’ slopware that powers key control systems in critical infrastructure such as energy, banking, hospitals and communications systems.
An incident involving using ‘OUTSOURCED’ slopware is just waiting to happen.
I mean that's not a great counterexample because outsourcing is famously difficult to get right, and from my own experience often results in exactly the kind of terrible code quality GP is talking about.
My point was that a blanket generalisation is bad because it's a blanket generalisation, the details are unimportant.
Yes, outsourcing is famously difficult to get right, and yet we've been doing it in every industry for decades now. And the famous examples become famous when something fails (boeing et. all) but there are also success stories out there. The trick, as always, is in the implementation.
My other point was that every "new thing" gets the same treatment. It doesn't work. Oh, it works but on toy problems. Well, it works on some complicated problems, but it's expensive. Ok, it works on a variety of problems, it's cheap, but it's unmaintainable. And so on, and so forth.
I am yet to be part of any outsourcing project that was actually a sucess story, where everyone came unarmed without battlefield scars, in a timeframe of the last 25 years that outsourcing started to be a common thing in enterprise consulting.
The real takeaway here is that programs you whip up for your own use, by any means, are not the same things as maintainable products you can deliver to customers.
Vibe coding, or VB tricks and hacks 25 years ago, or whatever, sure, do it if that works for you, but that's not a product you can maintain for a customer base. It's a program, not a product.
I think "maintainable" is carrying a lot of weight in there.
A lot of profitable or otherwise successful software has been built by people who reasonably couldn't be called software engineers or computer scientists or whatever academic title. With Excel, Access, VB, Delphi, Wordpress. I'm sure there's an astrologer somewhere that made OK money from a hack in Delphi or VB for divining the stars on a computer.
It shouldn't be called "vibe" coding, it seems more like glue coding, which some people have been doing and sometimes made careers out of for a very long time. Wordpress was for a long time the big thing in this area, it allowed (probably) millions of people to call themselves web designers and web developers without actually becoming competent in software design.
Many corporations selling software have developers that aren't much better than that. People that might do 'SELECT * FROM table WHERE foreign_id = ?' to check whether there exists any such row, or create full copies of data to add versioning to some product instead of keeping it as metadata, or generate and store hundreds or thousands of gigabytes of dumb data representing business processes (like bookings and such) for the coming century instead of inferring it from rules on the fly. The corporations where I've seen this are generally considered fine and good software companies by customers and competitors alike.
A new RAD, a new Wordpress, a new MICROS~1 Access or a new LotusNotes isn't particularly revolutionary. I suspect the 'hype' is more about getting people in IT to accept the technology as such and not revolt against the broader applications against other, non-technical, people; for control, disciplining, harassment, surveillance, war or whatever nefarious, tyrannical purpose.
The difference with VB or any RAD is AI generates so much code that a human won’t be able to maintain it manually. Like, 3 months into the vibe-coded project you discover there is a concurrency issue. But it is now all over the place in hundreds variations. How do you even fix it?
I have a fairly successful heuristic - the more words about "empowerment" or "democratization" are being spilled around, less power and democracy common people have. There is a reason corporate snakes love these words so much. But hey, enjoy your vibe-coded app and believe that this tech is hyped because of perspectives on betterment of human society and individual emancipation, not because it will allow our technocratic overlords tighten the leash.
If you haven't already read it, you'd probably enjoy "How big things get done" by Brent Flyvbjerg and Dan Gardner - what you said is the central thesis of the book.
I wish there were no "software products" at all. They usually just come in the way. I favor open source, and not imposing unnecessary constraints on users. Every piece of software should be swappable, and tying it to a vendor makes it less so.
similar thought if you were running a sport franchise as a GM and saying imma make a team where every player is swappable except of course swapping lebron might not be all that feasible :)
One of the genuinely positive things about tools like Copilot and ChatGPT is that they empower people with minimal development experience to create their own programs. Little programs that do useful things - and that’s awesome. More power to the users.
I've changed my outlook on vibe coding after seeing the results of this wholesome vibe coding: https://www.youtube.com/watch?v=4pJUXocn7aE .
This guy didn't just make a small program, he made a product that is attuned to some very stringent requirements. If you gave this project to experienced software developers it would be heavy on process and light on impact, take forever to make, and stupid expensive.
Yeah, the most important part of that video is where he says: "it works well enough". I think there's lots of room for this kind of bespoke software that's fundamentally unmaintainable -- and not something anyone could reasonably afford to make with professionals -- but good enough for purpose. This seems like a great, life-changing example, but it's still subject to the same thing the article is talking about.
I've had some free time recently so I've been trying to use the various AI tools to do a greenfield project with Rails. I know the stack intimately, so it's been a useful exercise as a check against hype. While I can occasionally trust the tooling to make big changes, most of the time I find that even the small changes are wrong in some important, fundamental way that requires re-writing everything at a later time. Asking the models to fix whatever is broken usually results in an endless cycle of (very junior-like!) random changes, and eventually I just have to go in and figure out the fundamental restructure that fixes the issue.
Would this be good enough for someone writing a one-time-use, bespoke piece of software? Maybe. Would they be able to maintain this code, without having a truly senior-level ability to work with the code itself? No [1].
I'm less fearful of LLM taking experienced programmers' jobs [2], because I fundamentally think this kind of stuff plays in the same end of the market as knockoff designer goods, Ikea furniture, and the like. It makes custom software available to an entire customer segment that couldn't have afforded it before, but nobody is going to confuse it with a product made by a team of skilled professionals [3,4].
[1] But to a really inexperienced developer, it looks the same.
[2] I am more fearful of clueless business types, who think this technology obviates the need for programmers.
[3] ...but will unskilled professionals be under pressure? Maybe!
[4] ...and what about skilled people, using the same tools? My current hypothesis is they're beneficial, but ask me again in six months.
As you said, I think also the key differences between skilled and unskilled devs in the context of AI tooling are :
1°) Know how to diagnose and fix critical problems by themselves, because there will be bugs in production for which AI won't be any help
2°) Write maintainable code, AI doesn't care at all of maintainability of code while devs should (must imho) consider pasting AI code as a merge/pull request.
For 2°) IIRC, I read somewhere that more and more code published on i.e. github is AI-generated. While this means the quality of code is decreasing, i.e. Copilot is also trained on more and more % of AI-generated code. So the quality of AI-generated code is decreasing globally.
As time passes, I think skilled devs will become more skilled and unskilled will become more unskilled.
I think it's already happening. I've been using Claude a lot since 3.5. For a while, I barely bothered to check what it was doing because the code it generated tended to be fairly straightforward and usually worked on the first try. But in the last few weeks, I've started reading everything line by line again, because now it's not only often incorrect but also written in a goofy, convoluted way. AI-generated code being constantly fed back to it in a feedback loop would explain a lot, I think.
There's a lot of subtle, pernicious stuff that "Sneaks In" too--not even stylistic either.
Things like a loss of context about the nature of some installed dependency or something.
You really have to know what you're doing and read carefully to avoid glossing over things that'll catch up to you later.
That's likely just you starting the read the "straightforward" code and not looking through Rose tinted glasses.
The cutoff date for Claude 3.5 is April 2024, that's currently slightly over one year ago.
My experience too, in many cases, is that I already know (even if in a vague sense) how to do the thing I'm trying to coax out of the model, but a quick prompt "split this string on $CHAR" or whatever is much faster than me fiddling with forgotten bash syntax.
This approach is different than blind trust in the output from a place of ignorance.
Clearly beneficial for skilled people imho, I wrote about that here [0] recently.
[0] https://manuel.kiessling.net/2025/03/31/how-seasoned-develop...
I love this so much. It's hacky, janky, and absolutely phenomenal and life changing. This is like perfectly in the spirit of the hacker ethos. This dude scraped together pieces around him and made a useful product that solves the problems he wants to solve, the way he wants to solve them.
The code is probably garbage. You can tell the UI is weird and jumpy. He's using timed delays to manipulate the browser with keyboard keys. The whole thing would be absolutely shredded here. But it's beautiful. It's crafted for function over form with duct tape and bubble gum but it's the difference between his brother being locked helplessly alone in his body, or being able to express himself and have some level of autonomy. It's just awesome.
LLMs rule for this kind of stuff. Solve your problems and move on. The code is not so precious. You'll learn many things along the way and will be able to iterate quickly with a tireless and semi competent but world knowledgeable assistant.
> perfectly in the spirit of the hacker ethos
I would strongly disagree on this count. Hacker ethos, at least as traditionally understood, emphasizes grokking and elegance.
I couldn't think of any two concepts less associated with "hacking".
I think you may have an understanding of that word that is not very common.
It sounds like you're thinking about a "hack", as in quick and dirty solution.
But that does not relate to "hacking", and most certainly does not relate to "hacker ethos". And while "hacking" is sort of ambiguous, I don't recall ever seeing "hacker ethos" defined by quick and dirty solutions.
No, though those concepts are of course related historically, I'm not confusing the two.
The hacker's ethos is about freedom of information, freedom of access (apropos to this video about vibe coding accessibility tools), community, collaboration and so on.
I've never seen or heard "elegance" associated with hackers, hacking, the hacking ethos etc. If I were to go further, elegance as it is used here has some anti-hacker connotations, i.e. it is a gatekeeping term to separate "true" coders from "amateurs" which is very anti-hacker.
Anyway, I don't think I'm conflating these terms, but it could be true that my understanding of the hacker ethos is wrong.
I didn't look at this video, but be vigilant when seeing one, as I was also surprised by someone demonstrating what they can do with Cursor and I went so far to install the exactly the same version of the app, use the same model and everything (prompt, word capitalization...) I could gather from the video and the results were nowhere near what was demonstrated in the video (recreating mobile web page from screenshot). I know that LLMs are not deterministic machines, but IMO there is a lot of incentive to be "creative" with marketing of this stuff. For the reference, this was less than two months ago.
While I agree that vibe coding holds a lot of potential, we should not fall into the trap of success bias here....
The funny thing is someone somewhere will find a way to make millions off a vibe coded project. I just suspect it will be the outlier and they will be well positioned to make that happen.
And for the rest of the companies that embrace the 30%~ efficiency spike, it will just accelerate our work goals faster.
I like to use it on stuff that we wanna do to enhance the UX but rarely sees the light of day. Plus my wrists have never felt so good since replacing boring syntax choirs with LLMs.
Yeah but if you want to remove those qualities from how a professional swe works, simply have them do it for free.
Hence why software developers are out of job, eventually.
The day of only a few people around the factory floor to babysit the robots will come, but lets keep celebrating the day they start unloading them from the delivery trucks for installation.
It's not clear if the overall number of needed software developers will decrease or simply stop growing. The amount of software necessary in the future will only keep growing. Using the same folks to produce more is a large possibility. I imagine it will take decades to see a shrinking workforce that meaningfully displaces many existing software developers.
No. Software development is here to stay, they just will not be writing much software.
Even before AI much of software development was managing overseas teams.
Not everywhere in the globe, the difference now is that those overseas teams can be reduced as well, as we outsource their work into AI managed systems.
I don't know whether they will be or they won't be, but 10 years from now all the employed ones will be using LLMs.
Yes, the difference, like on those factories is that instead of the 100 that were on site when the trucks arrived at the factory, there will be 10 of them left.
I have a lot of uncertainty about the number of professional software developers there will be in the long run, but not about whether they'll be using LLMs.
Yeah, but the number matters to many folks that instead being given the opportunity to use LLMs, will be queueing to get something to work on instead.
Why do you not read the article? What you are saying is just total nonsense if you had actually read the article.
>If you gave this project to experienced software developers it would be heavy on process and light on impact, take forever to make, and stupid expensive.
No, it wouldn't. Every single software developer knows how to make programs which "work well enough". This claim is totally ridiculous, it also is not a "product" in any meaningful sense.
What the article is pointing out is that you can not sell software like this. It is not a "product" at all, it is a hobby project. And you can not develop a hobby project like you would develop a commercial product.
Just think for a second what it would take to sell this.
This is literally the sort of empowerment that can be truly awesome. Thank you so much for sharing!
its also makes linux shell pipe coding graph node programable.
firefox("find cats") -> gimp ("carton filter cats")-> gimp("compose into partypic") -> google drive ("upload to cat folder")
>they empower people with minimal development experience to create their own programs.
The author goes out of their way to play up the toy aspect of the LLMs when driven by the inexperienced. No mention is made of them being used by experienced developers, but I get the impression he feels they aren't useful to the experienced.
I'm just playing with a small client/server tool (rsync but for block devices), and vibe coding allowed me to figure out some rough edges of my design document, and experiment with "what would it look like built with threads? As async code? As procedural?) I would never have done that before because of the sheer amount of time it would take, I'd have picked what I thought was best and then lived with the sunk cost falacy until I had no other choice. It did a pretty good job of catching reasonable exceptions in the communication code which I probably would have just let throw a traceback until I got a bug report. It did a great job of adding logging and debugging options. And I had it take 4 attempts at making a "fancy" --progress display and picked the one I liked the best.
LLMs give a level of exploration to software development that we haven't seen since the good old days when HP set up several groups to each build their next generation workstation environment, and then picked the best one.
IMHO, the experienced software developers stand at a unique position to take advantage of these LLMs that far outstrip what an inexperienced developer can do.
> vibe coding allowed me to figure out some rough edges of my design document, and experiment with "what would it look like built with threads? As async code? As procedural?”
This doesn’t fall under my understanding of the phrase “vibe coding”. In the tweet from Karpathy which many point to for coining the phrase, he says that when vibe coding you essentially “forget the code exists”. I think it’s distinct from regular ol LLM-assisted development
I surely did a hybrid approach, but for a large swath of it I was pretty purely "vibe coding", where I wasn't looking at the produced code at all.
I initially had Gemini 2.5 build some code and then did a code review of it. I tweaked my design document and then had Claude and ChatGPT take a stab at it and at this point I wasn't looking at the code at all. These implementations had some deadlock or early termination problems so I asked for some different async and threaded implementations, but ultimately they went off into the weeds in ways I couldn't really reason about, so I went back to a straight select/procedural implementation and had it start tracking outstanding blocks, which got something working reliably. Then I asked for fancy progress displays, tried several attempts at that. Then at this point I started cleaning up the code some, had it add type annotations, etc...
Big swath of pure or nearly pure vibe coding in the middle there.
I'm a little confused on how this helped your design doc? It seems like you picked the design that the LLM could best implement?
I was really curious about how it helped you iron out the problems but couldn't figure it out from your description
Running the first generated version to test, I realized I needed to specify: "Make sure that the client processes messages from the server while it is generating and sending hashes of the source file", because the first implementation would spin on generating the hashes and deadlock because the server end would write messages back to the client that the client wasn't reading and fill up the output buffers, which would fill up the client output buffers once the server stalled. This was obvious just from running it and knowing how all this stuff works (rather than by code review).
Then I started seeing issues with how I had phrased the terminal conditions, the client terminating immediately upon reaching the EOF on input file, and not reading any further server messages. I considered making a 3-way ACK to signal end, but instead changed the design to tell it to keep track of outstanding un-acked packets and only send the EOF/EOFACK when there were no outstanding messages.
The design is that the client generates a bunch of "xxhash64" messages for each block of data, and the server replies to each one with either "ok" (I already have that data) or "send" (send me that data). The "send" is replied to with either "data" (here's the data) or "zero" (the block is entirely NUL).
We used to call that "the zone", and it's not exclusive to coding.
Not in that sense. When you're coding and "in the zone", the code doesn't cease to exist - it's more like it becomes the extension of you.
https://en.wikipedia.org/wiki/Flow_(psychology)
I only forget that time exists when I'm in flow state, not code.
That's actually the term I prefer, but I couldn't remember it
How does the inexperienced developer become experienced under said circumstances?
You can examine the code at any time, ask for an explanation, and even make changes yourself. In other words, you get out of it exactly what you put into it.
If you choose to let the AI do all the coding and never dig into the actual code, you'll learn more about how to communicate with an LLM than you will about how to write code.
> ask for an explanation
That is risky. The AI might just hallucinate an explanation.
So can any human-based sources. A healthy learning experience with AI takes discipline, just as it does with humans.
The amount of errata in technical books is very small compare to the length of the book. I believe it ranges from none to a page or two.
That sounds like a similar failure rate for using newer models in technical contexts. Especially for popular technologies where there are enough resources such that someone other than the original creators could reasonably write a book about it.
I think that's an oversimplified metric when comparing the ability of an AI to analyze and describe code. Imaging you're reading a book on a challenging new topic. Even if the book is 100% right, there will be many passages you don't understand, and may even misinterpret. A book won't correct you, offer alternate explanations, or break down concepts any further than what was already written.
An LLM, for all its flaws, can do these things exceedingly well (provided it has good source material, of course). This gives you the ability to get answers to your questions and correct your mistakes. I'd argue that these benefits far outweigh the errors an LLM makes in breaking down and explaining material, and make an LLM better than a book in many (not all) cases.
> Even if the book is 100% right, there will be many passages you don't understand, and may even misinterpret.
Which is why no one reads only one book on a particular subject, especially if one is new to the domain. Often you bring in an alternate, but correct perspective from another book. The risks of hallucination with LLMs' responses outweighs the advantage of multiple perspectives (even if a broken clock is right twice a day, you still want a working one)
No one reads only one book on a subject? People act on mere headlines all the time. Reading one whole book would be a huge win over the status quo
> The risks of hallucination with LLMs' responses outweighs the advantage of multiple perspectives
You can ask a model a question multiple times. You can ask different models the same question. You can ask the same question different ways. You can do this automatically with agentic frameworks or reasoning models. "Multiple perspectives" are compatible with language models.
> You can ask a model a question multiple times. You can ask different models the same question. You can ask the same question different ways. You can do this automatically with agentic frameworks or reasoning models.
LLMs are for generating text, not producing knowledge. The actual knowledge come from the signal/noise ratio of the training data, not the training method.
It's like recording a violin in a noisy environment, encoding it with a low-bitrate lossy encoder and trying to get the PCM of only the violin back.
We've tried to use a better environment for recording specific notes from the violin (better training data), adjust the parameter of the lossy encoder (tuning the model), mix other violin samples to the respone (RAG, MCP), and redefining what a violin should sound like (LLMs marketing).
But the simpler solution is to record it correctly and play it later (AKA going to the source).
I would guess it's been around 1-2 years since you last tried seriously using a language model for any practical use cases, right? It makes sense to not use them if that is your experience, but it's hard to imagine anyone saying this who has actually been following along for the last few years to now. SOTA from 2 years ago now comfortably runs on a single GPU. A high end GPU can easily out perform SOTA from then both in (general!) knowledge reproduction, and capabilities (including saying things like "I don't know" when you ask beyond the encoded knowledge).
That is nonsense. A competent human will provide you with an explanation that is based on logic. LLMs just put together words that are statistically most likely to be used together given the current input. That is why you can get hallucinations with LLMs but not with humans.
This is incredibly meta
That is a question that is very near and dear to my heart right now because my son is 15 and very interested in programming. I don't have any good answers yet, but you definitely need to know so many of the core concepts to be able to steer the LLM around pitfalls.
I agree they may not and that’s very interesting. As an experienced developer myself I’m starting to think these tools are likely going to increase our market value not decrease it.
It's not implausible that we'll wind up with something like the extremely long-lasting market for COBOL developers. A niche skill that's absolutely essential in a few places, and which can command high salaries as a result... for a few people. Only so many huge legacy banking systems, after all.
The problem, as with COBOL, would be the loss of the existing pipeline that created new competent high-level developers.
IBM just launched a new mainframe with AI accelerators https://www.ibm.com/products/watsonx-code-assistant-z in part to help rewrite production COBOL into Java https://www.ibm.com/products/watsonx-code-assistant-z
"The value of 90% of my skills just dropped to $0. The leverage for the remaining 10% went up 1000x." -- Kent Beck
Sounds fantastic! And it might also help with ageism, when "learned programming long before LLMs" is recognized as a differentiator.
I, for one, welcome this change very much, disastrous as it sounds.
Sometimes I want to cook a home meal.
If I wanted to open a large world class fast food chain, I wouldn't be cooking home meals then. That would be silly.
Copilot can help me cook the equivalent of a McDonalds special in terms of software. It's good, I think McDonalds is delicious.
But it cannot help me cook a home meal software. It will insist that my home made fries must go in a little red box, and my fish sandwich needs only half a slice of cheese.
Deeply into that metaphor, maybe someone who has only worked for fast food chains might forget that a lot of good industrial dishes are variations of what previously were home cooked meals.
I am glad that copilot can help young cooks to become fast food workers. That really looks like something.
Well, I take pleasure in cooking home meal software. Can you make a copilot that helps with that?
You know what, nevermind. I don't need a copilot to cook home meals. I have a bunch of really good old books and I trust my taste.
It's not like some big company is going to be interested in some random amateur dish, is it? It was definitely not cooked for them.
Pretty weak analogy. Copilot can show you how to make either McDonalds or a Michelin starred restaurant meal. It is endlessly variable.
I can do neither.
McDonald’s is a system for delivering food fast and cheap at scale. It breaks down when making one burger.
If you’re not a trained chef, you will not pump out Michelin star meals, even using the same recipe.
Asking the LLM for something is like doing an "Ask the Audience" in Who Wants to be a Millionaire. You're basically polling the consensus answer to the question you're asking.
> You're basically polling the consensus answer to the question you're asking.
That's perfectly fine, because that's what you expect from human work to begin with. From natural text like blog article or tecnical reports to software changes, all output is expected to comply with patterns we are already familiar with. Heck, look at pull requests, where you ask your audience to evaluate your work hoping to reach a consensus.
You'd expect that from an average person, but not a genius in the subject. "Genius" is almost by definition a contrarian viewpoint or technique that happens to be better than the consensus or the best-known.
> You'd expect that from an average person, but not a genius in the subject.
That's perfectly fine. It doesn't take a genius to write a webpage or a report.
> You're basically polling the consensus answer to the question you're asking.
There is nothing wrong with going with the consensus answer. Sure, you can't invent an O(N) sorting algorithm with the consensus, but LLMs can definitely write good, maintainable code. Michelin star analogy might be an exaggeration, but it can help cook a good homemade meal, possibly even better than most wannabe home cooks.
>There is nothing wrong with going with the consensus answer.
For most things, yes. But you don't hire experts to tell you what everyone else knows. You hire experts to give you the angle.
For example, an LLM trained on all AWS documentation wouldn't be super useful (possibly less useful than traditional full-text search), because what you really want to know is the things that AWS didn't write, the writing "between the lines", all the things that DynamoDB can't do.
Dude, ask ChatGPT to explain the analogy for you. It understood well enough. It will save both of us some time.
Maybe AI is useful for some things after all.
Not sure about the reason for the personal attack, but my comment wasn't a reply to you. Michelin star restaurant was not even your analogy.
I guess you are right, though. AI is useful for some things. At least it reads the prompt before responding.
> Michelin star restaurant was not even your analogy
Precisely.
It's kind of true story actually.
I found one of my license texts for a hobby project[0] in a Microsoft product[1]. I have no idea how they use it.
Copilot is very poor at understanding how the DSL arrangement of the project works or how to assemble validator trees. Before Copilot many other "enterprise" stuff like JetBrains IDEs had problems auto-completing it. But it was always good enough for me, for my own things.
It obviously have value. Maybe they're lazy and using it for something unrelated like the TLD list. I really don't know. What I know is that something of my amateur dish ended up being served in what I consider to be a fast food chain.
No, copilot is not endlessly variable. It works best with typed languages and an enterprise mindset. That's exactly the point against "vibe coding" they push. MS seems to despise anything that doesn't follow that.
Look, I love typescript and all that stuff. As I said, McDonalds meals are delicious.
But I don't want that approach for everything I code. This project is not the only one with that issue. I noticed the same in a portable shell compatibility layer, it can only help if I make it to look like other script languages.
"You're asking too much, how can copilot help with a language it was not trained on?"
I am not asking it to be good at that. Just don't make asshole posts saying anything other than MS pasteurization is "vibes".
If they paid me, then that would be another story. But they didn't, and used my "vibe" project anyways.
I am not complaining about software I released for free, before anyone tries to go for that.
I am making an observation about kinds of software that copilot can't help with. My kind of software development, in which I might take months to decide a name or redo everything several times just to try a new kind of sauce.
[0]: https://github.com/Respect/Validation [1]: https://support.microsoft.com/en-us/office/microsoft-teams-t...
> No, copilot is not endlessly variable. It works best with typed languages and an enterprise mindset.
I don't think your assumption is true. The key factor is the corpus used to train it, and the context you fed it. I already experienced Copilot fumbling references to methods of a simple Java class, whereas it pulled off thinks like ARM templates flawlessly. You need to understand that internally LLMs do probability-based text completion. If you feed them enough context to maximize the probability they output things you expect, they do so.
Right, but that's exactly what static typing does in most cases - it provides model with more explicit context for what it's doing.
For the same reason, they tend to handle XML better than JSON - sure, both are trees, but XML is has redundancy, and that redundancy helps keep the model on the rails, so to speak.
I actually wonder if the perfect LLM language would be something a lot more like COBOL - not in a sense of being similarly high level, but rather verbosity. And perhaps also being closer to natural English, which is, after all, still a lot of its training set, especially for reasoning stuff. For query languages they seem to like SQL the most of all the things I've tried, and I strongly suspect that it's the same underlying cause.
Of course, a new language designed like that would have the fundamental problem that there's no existing corpus in such a language. Then again, if it is also designed such that one can reliably convert e.g. from Java to that language, then perhaps we can still pull that off.
> For query languages, at least, they seem to like SQL the most of all the things I've tried, and I strongly suspect that it's the same underlying cause.
The reason they like SQL more than other query languages is primarily that their training data has orders of magnitude more of it than any other query language, and that advantage is so huge that any other possible advantage would probably have comparatively negligible effect.
I'm not so sure about that. This was in comparison to e.g. walking object graphs in Python, C#, and JavaScript using the usual APIs (i.e. where querying one one-to-many relationship looks like Foo.Bars rather than joins, and using map/fold/filter or equivalents). I would expect there to be a lot more of that kinda stuff in the training set than SQL.
Please, think about what you just said.
How can I feed to it context about a DSL that does not exist yet?
> How can I feed to it context about a DSL that does not exist yet?
My example referred to ARM templates. They do exist, don't they?
I guess they do. But if that's the case then your comment brings nothing new to the discussion.
I am in the middle of my third AI assisted project. I disagree ~90%.
If you prompt an LLM like an architect and feed it code rather than expecting it to write your code, both ChatGPT 4o and Claude 3.7 Sonnet do a great job. Do they mess up? Regularly. But the key is to guide the LLM and not let the LLM guide you, otherwise you'll end up in purgatory.
It takes some time to get used to what types of prompts work. Remember, LLMs are just tools; used in a naive way, they can be a drain, but used effectively they can be great. The typing speed alone is something I could never match.
But just like anything, you have to know what you're doing. Don't go slapping together a bunch of source files that they spit out. Be specific, be firm, tell it what to review first, what's important and what is not. Mention specific algorithms. Detail exactly how you want something to fit together, or describe shortcomings or pitfalls. I'll be damned if they don't get it most of the time.
> Be specific, be firm, tell it what to review first, what's important and what is not [...] Detail exactly how you want something to fit together [...]
You mean like in a fast food chain?
I know how to use it. All of that was already implied in my original comment. Sometimes though, I want to cook without a rigid mindset.
I wonder how many lines of code you write in a day. Most PR I’ve seen has a diff in the lowe 2 digits, unless it’s refactoring or formatting.
[flagged]
They've been around forever and I believe number 2 or 3 in number of stores open in the US. You don't get that way without some people liking your product.
It's fine that some people don't like it, but is ridiculous to think your tastes apply to everyone.
Obtuse and missing the point on purpose.
Obtuse maybe, but I'm not missing the point. Their point is nonsensical, given that LLM generated code isn't "as good as fast food compared to home made meals" or even comparable to "food".
Go ahead and explain why I'm missing the point.
The point is, for most projects there is nothing wrong with writing McDonald's level code. It's not as healthy or tasty as a homemade meal, but it will perform its main function (i.e. prevent you from starving).
Eh, if that was the point then please, go ahead and show me this magical AI system that produce "edible" level of code then.
All I seen so far is badly generated boilerplate and enough template spaghetti to confuse some CS dropouts by the looks of it.
still a good advice
> Copilot can help me cook the equivalent of a McDonalds special in terms of software. It's good, I think McDonalds is delicious.
No. The likes of Copilot help you cook the meal you'd like, how you'd like it. In some cases it forgets to crack open eggs, in other cases it cooks a meal far better than whatever you'd be able to pull together in your home kitchen. And does it in seconds.
The "does it in seconds" is the key factor. Everyone heard of the "million monkeys with a typewriter" theorem. The likes of Copilot is like that, but it takes queues and gathers context and is supervised by you. The likes of Copilot is more in line with a lazy intern that you need to hand-hold to get them to finish their assignment. And with enough feedback and retries, it does finish the assignment. With flying colors.
Cook a meal, yes. "How you'd like it", no. You have to use the languages that have a large enough presence in its training set that it knows them well, and ditto for frameworks and libraries. Once you step out of that zone, quality rapidly plummets.
"Does it in seconds" is kinda true if you ignore the "supervised by you" part. That part - reading through the generated diffs to catch (sometimes very subtle) errors - can take a while.
And sure, you'd have the same problem with the intern. But the intern learns from you. The LLM will make the same mistakes again in the same circumstances. Yes, I know they do have memory now, but it my experience that RAG-based stuff is not particularly reliable. I suspect this might improve once hardware is fast enough that you can use a (smallish) helper LLM for RAG with reasonable perf, but we aren't quite there yet.
Sometimes it poisons your family to death. That’s the dangerous bit.
> To me, programs are “works on my machine” code.
My main takeaway from vibe-coding is that nobody cared enough to fill that niche and expectation. And it was really frustrating, yet we're getting there through convolutated, inefficient and borderline barbaric means.
People are still lamenting after HyperCard. Automation on windows or macos didn't go anywhere. Shortcuts were a better step into that direction but I feel it got stuck in the middle as Apple wasn't going to push it further. Android had more freedom yet you won't see a "normal" user do automation either.
If we're going to point the middle finger at vibe-coding, I wish we had something to point to as a better tool for the general population to quickly make stuff they want/need.
(Doing it as a professional dev is to me another debate, still with nuance in it IMHO. I'd also love better prototyping tools to be honest.)
Python is not exactly a hard language to learn. But the truth is that people generally don’t like to devote time to what they deem unimportant even if the end result would be useful. I’ve seen smart people go though dense mathematics and physics books, but refuses to take an introductory course in programming.
Python is comparatively easy, but you wouldn't say it's easier than the building mechanics in Zelda TOTK for instance.
I think what the target demographics want is that level of modularity and block building.
Also Visual Basic and Delphi.
Im so confused on what vibe coding is.
Is it:
- just pressing tab and letting copilot code whatever?
- asking an llm to do everything that it can and falling back on your own brain as needed or when its easier.
I guess probably more like the latter. I was just surprised to hear there was a special term for it. I thought everyone was doing that.
The pure definition is to have AI do everything for you without caring about what the actual code is.
That’s ambiguous. Whats the input?
Copilot anticipating based off what you’re typing? An explicit prompt about what you want?
Yeah, you tell it what you want and it gives you something, then you tell it how that's not good enough and it gives you another thing, and you repeat until you have something that works for you (but, as the author notes, is hardly a real product that could be maintained and shipped to real customers.)
> That’s ambiguous. Whats the input?
The input is undefined char*, and that's the magic of it. It's kinda funny we're reaching the phase where some people throw Syntax Error on a missing colon, but AI be like "I want sfssfsfsffsfs" "ayyy got you covered". How the tables have turned.
They oughta call it YOLO coding, in that case, though I suppose that slang is over a decade out of fashion.
YOLO has a misplaced modifier, they oughta call it YLOO coding (you live only once).
I like to think I learned this from the person who killed YOLO a decade ago (she teaches English).
Yeah this is how I interpret it. Type text into the chatbot, copy and paste the outputted code. If it fails to compile, paste the error message and get the resulting code. Similar if it fails at runtime. Supply the LLM with additional code files where necessary but don't really look at the code, just copy and paste in, copy and paste out.
Copy paste? Use something like desktop commander and just let it edit the files for you directly. It'll even run commands to test it out. Or go further and use Cline/RooCode and if you're building a webapp it'll load your page in a small browser, screenshot the contents, and send that to the model. The copy-paste stuff is beginner mode.
When I've been experimenting with Cursor I've found that one of the essential parts is the agent MUST be able to close the loop itself for it to be useful in any real sense.
So if you are having the agent write a script to call an endpoint and do something with the result. Have it write a script that does that and then ask it to run the script itself. It will then be able to close the loop by itself and iterate on the code until it does what you want (usually).
It is also very useful to ask the agent to write a test for something and then run that test to close the loop in the same way.
The vibes are much better if you just tell it to run a certain command, then it can see the error message and you don't have to paste back and forth so much. You can go make a cup of tea and it'll just iterate on changing files and running that command until it does the desired thing.
This is what AI-oriented IDEs like Cursor streamline - the LM gets immediate feedback from things like code completion, linter, build errors, test failures etc, and can immediately iterate without you having to do anything about it (sometimes overly aggressively even tbh).
Copy paste is so last year, just turn on agent mode in VS code or Cursor. It took me only a single prompt to convert a set of messy razor pages into a nicely structured React application.
How else are people using LLMs to code? Other than this?
Its so fast to do these things. Seems dumb not to use an LLM to interpret an error message if you don’t immediately know the problem. And if it doesn’t work you can alway abort and do it yourself. The whole code gen and debugging process might take 45 seconds.
Sure its dumb if you’re writing some short simple function. And you need to be in the loop and understand all the changes you are making in a professional setting. But that just sounds like the basic workflow of coding with an LLM.
[dead]
It’s implied that programs are personal but product code goes through peer review, checking the premise of the change as well as the implementation.
When someone reviews vibe coded patches and gives feedback, what should the reviewer expect the author to do? Pass the feedback onto their agent and upload the result? That feels silly.
How has code review changed in our brave, vibey new world? Are we just reviewing ideas now, and leaving the code up to the model? Is our software now just an accumulation of deterministic prompts that can be replayed to get the same result?
>When someone reviews vibe coded patches and gives feedback, what should the reviewer expect the author to do?
Quit his job or stop contributing. Nobody is helped by people blindly commiting code they don't understand.
With the Altair you had switches to input bits of CPU instructions. Then punchcards. Then Telex. Then assembly in a terminal. Then C, Pascal and Smalltalk. Then Java, C++, Python, PHP. Then the mountains of libraries and frameworks, realtime indexers, linters, autocomplete, suggestions.
The next step seems that we will write programs in prompts. Maybe we will get a language that is more structured than the current natural language prompts and of much higher order than than our current high level programming constructs and libraries.
You can’t eliminate formalism in programming. You can only abstract it away by creating a metaprogramming layer for the current one. And prompt isn’t that. Prompting is a blind shot hoping to hit something good.
The real problem with "vibe coding" (or any coding, or with any product what so ever) is that the end user probably is not a programmer, and therefore cannot have professional judgement on the quality of the implementation. The only thing typical end user care is the perceived behavior, not the implementation details.
As it is easier and cheaper to do anything, the result is low-quality products. This of course serves well for the real professionals, who can now sell premium quality products at premium pricing for those who know and want the quality.
It depends. Market has demonstrated time and again that premium quality is a rather niche product. Is the niche large enough for all the existing "real professionals" to fit into it?
I am happy that there will be work in security for good while. They might first pass run LLMs on that too. But it will miss things and that is where I can come in...
I doubt that vibe coded software is maintainable. Time will tell, so far no indication it is.
Right now there are thousands of apps getting built with AI that are going to work surprisingly well, will get embedded into all sorts of fundamental processes — and will start to perform unacceptably badly in two or three years. God knows what we do with this information but it seems clear we are setting up for a tsunami of horrible performance regressions
Actually even in the years before AI code generation started, I noticed an increase in code bases that seem very well architected and engineered. But once you start using the code for a week you notice it's full of bugs and the consistent architecture is just make believe. Surely this problem is getting way worse with generated code that is optimized to read like it's very well engineered
This article starts to touch on an important distinction between software and products, but seems to miss the larger picture.
With the cost of developing software dropping to near zero, there are a whole class of product features that may not be relevant for most users.
> You haven’t considered encoding, internationalization, concurrency, authentication, telemetry, billing, branding, mobile devices, deployment.
Software, up until now, was only really viable if you could get millions (or billions) of users.
Development costs are high, and as a product developer you're forced to make software that covers as many edge cases as possible, so that it can meet the needs of the most people.
I like to think of this type of software as "average" -- average in the sense that it's tailored not to one specific user or group of users, but necessarily accommodating a larger more amalgamous "ideal" user.
We've all seen software we love get worse over time, as it tries to expand it's scope to more users.
Salesforce may be the quintessential example of this, with each org only using a small fraction of it's total capabilities.
Vibe coding thus enables users to create programs (rather than products) that do exactly what they want, how they want, when they want. Users are no longer forced into the confines of the average user, and can instead tailor software directly to their needs.
Is that software a product? Maybe one day. For now, it's a solution.
The argument is that vibe coding is great for little personal programs but bad for full-blown products.
I don't think you can ship a fully baked product made exclusively with AI coding yet. But it's *really* useful for investigative product development - e.g. when you have a few different ideas for how something might work but aren't quite sure what's best. I used to regret that I never mastered Figma or other mockup tools - now I never will have to.
I agree with the article, but that's not how the vibe coders see themselves. From their perspective they can't see the gap between programming and product, and in my experience are pretty hostile to feedback from real software engineers.
This is about the problem with vibe coding your way to a product. But if the vibes are strong enough, maybe we don't need products at all?
They do tend to complicate things, with all of their moats and such. I never wanted a product that did the thing, I really just wanted to do the thing. "Works on my machine" might be good enough if you're unlikely to want to repeat yourself on a different machine.
> I never wanted a product that did the thing, I really just wanted to do the thing.
Would you vibe code your daily driver car's ECU or a high-frequency trading application to use for your 401k? If you were to do these things (more power to you), I rather suspect you'd still do a whole lot of research and critical thinking beforehand, which sort of obviates the "vibes".
It is hard to imagine trusting an AI that much. But plenty of things happening today would've been hard to imagine by the people who came before. So yeah, let's make that assumption.
At that point, I don't think we'd be using the AI to code something and then switch to a mode where we're now using the recently-coded thing.
We'd just ask the the car to go, and it would happen. Then we'd suggest a strategy for making trades, and that would happen. If the AI decided to write some code and fork a subprocess along the way... that would be an detail that we'd be unaware of.
https://www.pymnts.com/artificial-intelligence-2/2025/openai...
Honestly just good advice for engineers in general - understand the difference between programs and products
Yeah I agree. That was my main takeaway... just because you have a running program doesn't mean you got a product... the vibe-coding thing is just the flashy headline
That was my takeaway: The missing bit is "software engineering", which is not "coding".
‘Vibe coding’ isn’t software engineering.
It throws away decades of software engineering principles in favour of unchecked AI output by clicking “accept all” on the output.
You would certainly NOT use ‘vibe coded’ slopware that powers key control systems in critical infrastructure such as energy, banking, hospitals and communications systems.
The ones pushing “vibe coding” are the same ones who are all invested in the AI companies that power it (and will never disclose that).
An incident involving using ‘vibe coded’ slopware is just waiting to happen.
It's like building a decorative bridge over a decorative pond in your backyard out of scrap wood versus building an actual bridge for public traffic.
fun fact: nobody would never accept vibe coding software level for a 200$/month vibe coding software tools
‘OUTSOURCING’ isn’t software engineering. It throws away decades of software engineering principles in favour of CHEAP output by clicking “I AGREE” on a contract.
You would certainly NOT use ‘OUTSOURCED’ slopware that powers key control systems in critical infrastructure such as energy, banking, hospitals and communications systems.
An incident involving using ‘OUTSOURCED’ slopware is just waiting to happen.
... oh, wait...
I mean that's not a great counterexample because outsourcing is famously difficult to get right, and from my own experience often results in exactly the kind of terrible code quality GP is talking about.
My point was that a blanket generalisation is bad because it's a blanket generalisation, the details are unimportant.
Yes, outsourcing is famously difficult to get right, and yet we've been doing it in every industry for decades now. And the famous examples become famous when something fails (boeing et. all) but there are also success stories out there. The trick, as always, is in the implementation.
My other point was that every "new thing" gets the same treatment. It doesn't work. Oh, it works but on toy problems. Well, it works on some complicated problems, but it's expensive. Ok, it works on a variety of problems, it's cheap, but it's unmaintainable. And so on, and so forth.
I am yet to be part of any outsourcing project that was actually a sucess story, where everyone came unarmed without battlefield scars, in a timeframe of the last 25 years that outsourcing started to be a common thing in enterprise consulting.
There is a problem with vibe coding? You don't say!
The real takeaway here is that programs you whip up for your own use, by any means, are not the same things as maintainable products you can deliver to customers.
Vibe coding, or VB tricks and hacks 25 years ago, or whatever, sure, do it if that works for you, but that's not a product you can maintain for a customer base. It's a program, not a product.
I think "maintainable" is carrying a lot of weight in there.
A lot of profitable or otherwise successful software has been built by people who reasonably couldn't be called software engineers or computer scientists or whatever academic title. With Excel, Access, VB, Delphi, Wordpress. I'm sure there's an astrologer somewhere that made OK money from a hack in Delphi or VB for divining the stars on a computer.
It shouldn't be called "vibe" coding, it seems more like glue coding, which some people have been doing and sometimes made careers out of for a very long time. Wordpress was for a long time the big thing in this area, it allowed (probably) millions of people to call themselves web designers and web developers without actually becoming competent in software design.
Many corporations selling software have developers that aren't much better than that. People that might do 'SELECT * FROM table WHERE foreign_id = ?' to check whether there exists any such row, or create full copies of data to add versioning to some product instead of keeping it as metadata, or generate and store hundreds or thousands of gigabytes of dumb data representing business processes (like bookings and such) for the coming century instead of inferring it from rules on the fly. The corporations where I've seen this are generally considered fine and good software companies by customers and competitors alike.
A new RAD, a new Wordpress, a new MICROS~1 Access or a new LotusNotes isn't particularly revolutionary. I suspect the 'hype' is more about getting people in IT to accept the technology as such and not revolt against the broader applications against other, non-technical, people; for control, disciplining, harassment, surveillance, war or whatever nefarious, tyrannical purpose.
The difference with VB or any RAD is AI generates so much code that a human won’t be able to maintain it manually. Like, 3 months into the vibe-coded project you discover there is a concurrency issue. But it is now all over the place in hundreds variations. How do you even fix it?
I have a fairly successful heuristic - the more words about "empowerment" or "democratization" are being spilled around, less power and democracy common people have. There is a reason corporate snakes love these words so much. But hey, enjoy your vibe-coded app and believe that this tech is hyped because of perspectives on betterment of human society and individual emancipation, not because it will allow our technocratic overlords tighten the leash.
'The problem' is this species prefers doing instead of thinking.
'Vibe coding' is merely an instance.
If you haven't already read it, you'd probably enjoy "How big things get done" by Brent Flyvbjerg and Dan Gardner - what you said is the central thesis of the book.
Reading books is an instance of doing instead of thinking in many cases :)
I wish there were no "software products" at all. They usually just come in the way. I favor open source, and not imposing unnecessary constraints on users. Every piece of software should be swappable, and tying it to a vendor makes it less so.
similar thought if you were running a sport franchise as a GM and saying imma make a team where every player is swappable except of course swapping lebron might not be all that feasible :)
This was precisely the dilemma you'd have playing Speedball 2.
I love it how AI-sceptics just keep moving the goalposts.
To effectively vibe code, u must setup a really clean foundation first, so that llm knows their boundary. Following best practices is a must.
[flagged]