Google fed coding interview questions to ChatGPT and, based off the AI's answers, determined it would be hired for a level three engineering position, according to an internal document.
As reported by CNBC, the experiment was done as part of Google's recent testing of multiple AI chatbots, which it's considering adding to the site. ChatGPT's ability to surface a concise, high-fidelity answer to a question could save users time typically spent surfing links on Google to find the same information.
“Amazingly, ChatGPT gets hired at L3 when interviewed for a coding position,” says the document. And while level three is considered an entry-level position on the engineering team at Google, average total compensation for the job is about $183,000.
Google's software engineer interview process relies mainly on technical questions, which ChatGPT passed. Although, the interview does include a few behavioral examples ("Tell me about a time...") Facebook, Amazon, and others also rely on these questions, particularly for leadership roles, which may be a barrier for "hiring" ChatGPT.
But the discovery that ChatGPT can pass the technical questions of the coding interview has raised questions about the platform's ability to transform not just Google but the engineering jobs behind it. Is this already happening, will it happen in five or 20 years?
When PCMag asked ChatGPT if it would ever replace software engineers, it wasn't convinced.
"No, ChatGPT will not replace software engineers," the AI responded. "ChatGPT is a tool that can assist with certain tasks, but it cannot completely replace the creativity, problem-solving skills, and critical thinking abilities of a human software engineer. Additionally, ChatGPT requires human supervision and direction to operate effectively."
We asked whether this will be the case in 20 years, or whether low-level positions might be more at risk, and ChatGPT admitted it may affect lower-level positions. But it reiterated it will never be a full replacement, but rather a tool to assist human software engineers.
Artificial intelligence bot and PR machine. Impressive.
These findings from Google, as well as a recent study that ChatGPT can identify and fix buggy code, has thrust the software engineering community into the same debate artists, journalists, and business people are having about the impact of AI on their futures.
Based on thousands of comments on Reddit posts on the subject, it appears many engineers are already using ChatGPT to replace time spent searching for coding solutions on popular sites like StackOverflow. Others are using it to generate small snippets of code. But others aren't as satisfied with ChatGPT's skills, and its sometimes verbose, nonsensical answers.
Whether the tool will eventually reduce lower-level engineering jobs, or advance the entire field by becoming more specialized faster, remains to be seen.
Plus, another part of a software engineer's role can include meeting with business teams (their "customers") and executives to explain and brainstorm technical solutions. It doesn't seem like ChatGPT is up for that: "There are many ethical and social implications to consider with the widespread use of AI in the workplace," it told PCMag.