Re: The Cargo Cult Generation Has Arrived Now What? (Boilerplating++)

[LanX]

Generating Mandelbrot Sets are one of those vanilla tasks most hackers attempt when learning a programming language.

Like vocabulary trainer, hangman, Towers of Hanoi, a template system, (name it)

Point is, you'll find hundreds or even thousands of ready to use examples in the internet for training an LLM.

And LLMs are very good in replicating what already exists.

Trying to generate something more exotic will be much harder.

There is the story of an AI solving an Erdös problem, turned out that the solution was very similar to another Erdös problem.¹

So yes, AI is improving productivity, but I doubt it can replace the sentient designer who steers, understands and debugs the output.

And you already alluded to this by calling it "cargo cult" in the title.

Dumb programmers who need an IDE to write hello world might get into trouble at some point.

But experience shows that the IT industry has the tendency to create its own demand.

More generated code doesn't mean less programmers.

¹) https://theweeklythesis.substack.com/p/proof-profits-and-overconfidence

[ajt]

I've used several AI engines and found exactly what you describe here.

If the request is something well known it will steal an existing answer and spit it out. Often padded with lots of facts and explanations that are also widely available.

If the request is for something more marginal, then it still regurgitates known facts but often hallucinates creating an answer that would work in an ideal world but don't work in reality.

For example SAP's ERP platform has been in use for decades and has constantly evolved and changed, but is a programming language, ABAP, is less written about than Perl, and you can find more questions than answers with Google. When you ask AI to write ABAP, they often write gibberish, and invents standard functions or classes that should exist but don't, or fields in structures that should be there but again are absent. Strangely they can get quite "defensive and annoyed" when you complain that they are talking bollocks.

It's just expensive CARGO CULT! Sometimes it will be right, sometimes it won't, and unless you know what you're doing you may not be able to tell the difference...

[LanX]

Yes, that's how LLMs operate, they "remember" and "average".

This doesn't mean that AI in general can't improve to become more productive, like by including specialised agents. For instance checking if created ABAP fields or functions exist or if the code compiles could be delegated to a relatively dumb agent.

Saying this I haven't seen a viable "development model" yet which supports testing and maintenance.

The idea of "democratization", i.e. "cheap" non-experts being able to steer the AI to meaningful products is IMHO a pipe dream of non-expert BA majors - probably a tautology - who think they can get rid of the nerds.

[LanX]

So I asked DuckAI

> vibe coding limitations

Vibe coding has several limitations, including a lack of maintainability and scalability, difficulties with debugging, and potential security vulnerabilities in the generated code. It is often best suited for simple tasks or prototypes rather than complex, production-level applications.

Reference: https://www.glideapps.com/blog/vibe-coding-risks

From the TOC

Some issues with vibe coding include:
1. Security vulnerabilities
2. Lack of maintainability and scalability
3. Difficulties with debugging and troubleshooting
4. Limitations in custom logic and integration