It turns out OpenAI’s ChatGPT program is capable of passing an MBA exam —but just barely.
Christian Terwiesch, a professor at the University of Pennsylvania’s Wharton School, recently tested how well ChatGPT would fare on his own MBA exam by giving the AI-powered program five sample questions.
In the end, ChatGPT scored a “B to B-” across the five questions, or just enough to receive a passing mark. In a research paper, Terwiesch goes over the AI program’s performance, which ranged from providing stellar answers to others that were riddled with logic flaws.
For example, Terwiesch gave ChatGPT an “A+” on three of the test questions for supplying the correct answers and providing a succinct or superb explanation for them.
But on two other questions, the program received a “C” for providing flawed answers that showed “Chat GPT3 at times makes surprising mistakes in relatively simple calculations at the level of 6th grade math,” he wrote.
Looking at the results, Terwiesch concluded ChatGPT excels at basic operations management and process analysis questions. However, “the present version of ChatGPT is not capable of handling more advanced process analysis questions, even when they are based on fairly standard templates,” he added. Thus, if given different, and perhaps harder questions, the program might fail the exam.
Still, ChatGPT is powerful enough that Terwiesch plans on joining other schools and professors in banning its use for homework assignments and finals exams. But he doesn’t see the chatbot replacing business workers just yet.
“We have many reasons to believe that the technology is getting better over time. But, we are still far from an A+ for complex problems and we still need a human in the loop,” Terwiesch wrote in his paper. (Ironically though, he says ChatGPT could assist him in creating future MBA exam questions in about half the time it normally takes.)
The other constraint facing ChatGPT is how the AI program can only come up with answers based on existing knowledge. Hence, it lacks the creativity to produce innovative solutions that human entrepreneurs might dare to envision. That same creativity is what teachers should focus on in nurturing their students. “It is up to us as educators to reward, if not require, such out of the box thinking,” Terwiesch added.