Here’s why people say GPT-4 is good again

OpenAI appears to be busy cleaning up its GPT language models following allegations that GPT-4 is getting “lazy”, “stupid” and errors out of the norm for the ChatGPT chatbot that was making the rounds on social media in late November.

Some even speculate that GPT-4.5 has been secretly rolled out for some users based on some replies from ChatGPT itself. Regardless of whether this is true or not, there have definitely been some positive internal changes in the past under GPT-4.

More GPUs, better performance?

As early as last Thursday, posts started to appear that noted the performance improvement of GPT-4. Wharton professor Ethan Mollick, who previously commented on the sharp drop in GPT-4 performance in November, also noted a revitalization of the model without seeing any evidence of a transition to GPT-4.5 for himself. He consistently fixed his code using a code translator and described the change as “night and day, both for speed and quality of responses”, after experiencing ChatGPT-4 to be “unreliable and a bit boring for weeks”.

While this was happening, OpenAI quietly reopened its ChatGPT Plus subscription last Wednesday, which had a registration blackout since November 14. Altman stated in X’s post “Thank you for your patience while we find more GPUs.”

There’s no word if there’s a correlation between ChatGPT Plus registrations resuming and the GPT-4 upgrade, but the timing is interesting. Notably, registration for the paid version closed shortly after OpenAI’s first DevDay, where the company unveiled a number of new features for the paid version of the AI ​​chatbot. The company implemented a waiting list for ChatGPT Plus subscriptions as post-DevDay registrations exceeded the service’s capacity to handle features.

People complained that GPT-4 would explain how to run commands instead of doing the job.

Shortly after, users began reporting unusual behavior of GPT-4 beyond the traditional AI jokes already known. One common complaint was that GPT-4 would “talk back” to users or require more explanation of a command before it could execute a query. Another complaint was that the model would explain to users how to execute their command instead of executing the task.

The degradation of GPT-4 dates back to at least July, when the study observed a sharp drop in accuracy between March and June. Many, including OpenAI’s VP of Product Peter Welinder, have suggested that the quality of the answers may seem insufficient as a psychological phenomenon as the model is further updated. Some added that users may benefit from changing their queries to get the results they want.

Although OpenAI has been largely silent about its inner workings, Altman’s X post on GPUs was probably a big indicator of what’s going on behind the scenes. Reports in April indicated that OpenAI would need more than 30,000 GPU units to maintain its commercial performance for the rest of the year. That was before the spike in interest in November.

Secret testing of GPT-4.5 or just a hallucination?

In addition, speculation about GPT-4.5 has expanded with a few more details about a potential leak of a new version of GPT.

Newsletter founder @therundownai, Rowan Cheung, recently shared on X (formerly Twitter) leaked pricing details for the new GPT-4.5 model that OpenAI has in development. Details include new pricing tiers and information on advanced multimodal options.

Cheung asked OpenAI CEO Sam Altman via the social media platform about the validity of the leak, to which he replied, “No.”

GPT 4.5 speculation began on Thursday with a “leaked” image showing a new GPT-4.5 model with new advanced multi-modal capabilities and new pricing.

However, Sam Altman commented “nah”; when asked if the rumors were true.

But there is more to the story…

— Rowan Cheung (@rowancheung) December 18, 2023

However, the few users who managed the proposed update are convinced that they are using GPT-4.5 and that it is new and better than ever. Some asked the chatbot they thought was using GPT-4 what its model was – and it replied “GPT-4.5 Turbo”.

This has led many to believe that OpenAI is testing GPT-4.5, primarily on its mobile apps, in hopes of avoiding savvy users. However, not everyone has been able to recreate these results, and the model will tell them that the latest version is simply GPT-4. OpenAI employee Will Depue also commented on the matter, calling it “a very strange and strangely consistent hallucination”.

Cheung noticed a post from the official ChatGPT X page with a brain and head in the clouds emoji, which he believes is a vague way for OpenAI to reiterate that the responses are hallucinations.

In particular, OpenAI provides early and exclusive access to its paid users, including ChatGPT Plus users, developer API users, and enterprise users. When features are announced, be among the first to experience the latest GPT versions and their features. Many people who tinker with the models and notice subtle changes probably have some form of access to the developer API or use the service to test code and share their input with the public.

Although OpenAI is testing GPT-4.5, there’s no telling when the update might happen, especially as the company settles down from the destabilization of services and organization. Additionally, the company and the product are still new, and it’s not yet clear what a routine update cycle looks like. At this point I’m assuming it doesn’t matter what “version” of ChatGPT we’re on because the improvements seem real.

Editor’s recommendation