Does GPT-3 threaten your engineering job?

AI and GPT-3 seem to be everywhere in the last few months.

With ChatGPT we finally seem to be one step closer to realising the type of artificial intelligence we have seen in movies.

Obviously, we are quite a way off from Terminator-like AI. Your life might not be at threat just yet but what about your job?

Let’s have a look at what ChatGPT can do and whether we should be worried just yet.

What is ChatGPT?

ChatGPT is an AI chatbot that has been trained on various programming languages as well as other sources to help developers write and debug their code by having a conversation with it.

The point of ChatGPT is to be an assistant rather than a replacement for programmers but some people are already saying that it may replace us all one day.

What can ChatGPT do?

So let’s have a look at an example and see what we can get ChatGPT to do for us.

I asked ChatGPT the following:

This was the reply I got:

Ok wow, it actually produced some comprehensible code. This is the code if you are interested:

import requests
from bs4 import BeautifulSoup

URL = "https://books.toscrape.com/"
page = requests.get(URL)

soup = BeautifulSoup(page.content, 'html.parser')
results = soup.find(class_='side_categories')
self_help_link = results.find_all('a')[1]['href']

URL = URL + self_help_link
page = requests.get(URL)
soup = BeautifulSoup(page.content, 'html.parser')

results = soup.find(class_='product_pod')

top_self_help_books = []
for result in results:
    book_title = result.h3.a['title']
    top_self_help_books.append(book_title)

print(top_self_help_books)

Does it work though? No.

I get the following error when I try and run it:

Traceback (most recent call last):
  File "/Users/alex/Projects/chatgpt.py", line 19, in <module>
    book_title = result.h3.a['title']
  File "/usr/local/lib/python3.10/site-packages/bs4/element.py", line 965, in __getattr__
    raise AttributeError(
AttributeError: 'NavigableString' object has no attribute 'h3'

Not off to a great start.

Let’s try asking it to fix its own code. So are asked the following:

This was its response:

When I make the change it suggests, the code actually runs and gives me an output:

["It's Only the Himalayas", 'Full Moon over Noah’s Ark: An Odyssey to Mount Ararat and Beyond', 'See America: A Celebration of Our National Parks & Treasured Sites', 'Vagabonding: An Uncommon Guide to the Art of Long-Term World Travel', 'Under the Tuscan Sun', 'A Summer In Europe', 'The Great Railway Bazaar', 'A Year in Provence (Provence #1)', 'The Road to Little Dribbling: Adventures of an American in Britain (Notes From a Small Island #2)', 'Neither Here nor There: Travels in Europe', '1,000 Places to See Before You Die']

Although these clearly aren’t self-help books. I can see from the code that it has just picked the second link in the list (which is Travel) rather than finding the words Self Help.

Let’s see if I can get it to fix itself again.

This was what it came back with:

So exactly the same code as before pretty much, which would still give me back the travel books.

Still, the fact that it came back with any code at all is still pretty amazing, even if it did require a bit of prompting to make it actually work.

Could ChatGPT replace programmers?

It is still early days for ChatGPT and what it can do is already promising.

I can see ChatGPT being a useful tool for beginners who don’t even know where to start. Even just as an aide for programmers to write code quicker.

It would have only taken me 20 minutes or so to write the above example myself after factoring in looking at the website’s source code. ChatGPT was still quicker though, even with the mistakes.

You do have to be quite specific with the requests though but I can see ChatGPT getting better with this over time.

ChatGPT is currently free to try out but it is costing OpenAI a staggering $3 million a month. Apparently each word generated costs $0.0003. This can soon add up, especially with the wordy examples I have tried.

As we have seen the accuracy is not that great either. It seems to generate code with errors and can’t always fix all the ones that it is given. This is why StackOverflow have banned responses generated with ChatGPT.

If the cost comes down and the responses are accurate then I can see some programmers being out of a job in a few more years.

The programmers that are going to be affected are the ones that aren’t solving problems themselves and are simply copying and pasting from StackOverflow.

If we ever do find ourselves in a world where all code is being written by AI then possibly there will be a new job title, “AI Prompt Architects”. These people will carefully construct questions for GPT so that it generates the correct code.

I am pretty sure QA and SDET engineers are safe for a while. If we don’t trust experienced engineers to write fault-free code (which we shouldn’t) then I doubt we are going to trust AI-generated code anymore.

🧠 Feed your brain with my weekly newsletter

Each week, I send out a newsletter with insights about productivity, life lessons and engineering. Join here for free →