For fun, I’ve decided to spend a few days and create a fairly complex client/server application (CRUD), using the technologies Python/FastAPI and Vue.js and the PostgreSQL database. But also for fun, I’ve decided to let ChatGPT do 100% of the programming, so my role is restricted 100% to prompts, putting the code into Visual Studio Code, and testing. Even in cases where I could easily do something by hand, I asked ChatGPT to do it. Think of the movie Supersize Me, but for IT.
I wanted to record my thoughts as I am working on this project!
What’s worked well
- ChatGPT exceeds where I want and need the help. I’m not a full-time application developer, so I have no desire to fill my brain with things I will never remember, like HTML and CSS and stuff like that. ChatGPT has done a great job: “Can you please make the box a bit longer and a slightly brighter shade of green?”
- ChatGPT doesn’t just give me the code; it explains to me what’s happening and highlights important topics. This is important for me, since me acting as “dumb scribe” together with ChatGPT does not work; see the next section for bad examples! This means I need to have as deep a knowledge of the code as possible.
- ChatGPT helps me learn. Before I started this project I had no experience with Vue.js. Thanks to ChatGPT’s constant inputs and help, after just 2 days I have a much better knowledge of this product.
What’s worked badly
- For whatever reason, ChatGPT seems to have good days and bad days. I have no logical explanation for it. I can only guess it has to do with my prompt history (e.g. “context”) – but that’s just a guess. One example: yesterday I had to stop our session many times: I’d give it a source code file, instruct it clearly to change only the business logic and not the look-and-feel, and ChatGPT would change both. But today, working with that exact same file, ChatGPT only makes the changes I request. Another example: yesterday, no matter what I tried, ChatGPT would also regurgitate the complete source file; but today, it (thankfully) only gives me the bits-and-pieces that have changed. Degree of frustration: *****
- Infinite debugging loops. Yesterday was a particularly bad day for this, but it happens frequently. ChatGPT gives me code that has an obscure bug. I feed the error message into ChatGPT, and it spits out essentially the same code as what created the bug. I call this an “infinite debugging loop” – and I’ve needed to “break out of this loop” many times – but sometimes telling ChatGPT to take a different approach (and have it really take a different approach) is easier typed than done! Degree of frustration: ***
- More than I need. Yesterday was also a particularly bad day for this. I needed a simple update to a single file (e.g. change the button text from “Save” to “SAVE”). ChatGPT would then not just give me the needed files, but also 4-5 other files (usually with the text “Please ensure file XZY looks like this:”. Degree of frustration: *
- Unable to debug some problems. There have been a number of bugs during development that ChatGPT was incapable of solving. After quite some frustrating time and back-and-forth, I carried out my own debugging. I found the need for this could be reduced by ensuring ChatGPT always took “baby-steps” that I could continually test. But I did want to point out: total elimination of the human factor in debugging is not yet possible.
- ChatGPT often goes into “guess mode.” I encountered this situation this morning where a Vue component with a dynamic table suddenly started adding duplicate entries into the table. ChatGPT: “Let’s see if this is race conditions…” It was not. ChatGPT: “OK, let’s see if they are added to the local array and not the server…” They were not. Guess after guess. I gave up. You get the idea. Frustration factor: ***
Other considerations
- I’ve done “extreme programming” or “pair programming” many times, and the good social interaction between programming partners can really keep the momentum going. Even with ChatGPT instructed to treat me nicely and call me Dr. Ken, I can only stomach about 90 minutes of work with ChatGPT without having to take a “frustration break.”
Advice for students and others
Naturally your mileage may vary, but I’ve found that programming with ChatGPT requires not only constant attention on my part and a deep knowledge of the code, but also the ability to carry out my own debugging. I can easily see that without some prior programming experience — and especially debugging experience — working in this way might be very slow and tedious. When you do enough debugging, you get a “gut instinct” of where the problem could lie – and that was essential for me to climb out of the holes ChatGPT would occasionally dig for me.
NOTE: This article was published on LinkedIn