My experience of experimenting with ChatGPT to build actual working code was meh. I'd asked it to do something very specific that as a human I knew how to do already, so that I could compare the response. The first response hallucinated methods in a module which did not exist. Upon informing the thing it responded with 'of course you're right.... here's a working solution' after which it proceeded to hallucinate another method in the same framework which again has never existed. I've had some success having it generate html/css for me, but it becomes very frustrating as it loses track of what it has been asked to do, starts missing things that it had previously made. The other day I read an interesting article about someone making a web app out of Graph::Easy & WebPerl, who decides to use 'Claude' to port the perl solution:
At this point I should have put final touches to the web app, shipped it and written this blog post. But I didn’t. I got greedy. One thing irked me. The WebPerl interpreter took a few seconds to initialise. “You know what?” I thought. “I bet Claude could do a good job simply porting the library over to a different language.” ......... And thus started a journey both fruitless and frustrating.
