ChatGPT's abilities deteriorated – study
Date: 2023-07-21 Author: Karina Ziganova Categories: IN WORLD
OpenAI's artificial intelligence (AI)-powered chatbot – ChatGPT – provides more inaccurate answers to questions over time
Scientists at Stanford and the University of California, Berkeley, found that later versions of ChatGPT were much less likely to give accurate answers to the same questions for several months. However, they could not explain why this happened.
Due to the findings, analysts are asking anyone using ChatGPT to implement some form of monitoring analysis to ensure that the chatbot remains up to date.
How the study was conducted
To test how reliable the different ChatGPT models are, the researchers asked the ChatGPT-3.5 and ChatGPT-4 models to solve several math problems, answer sensitive questions, and also write lines of code.
As a result, it turned out that in March, ChatGPT-4 could give correct answers in 97.6% of cases. The same test in June showed that the accuracy of GPT-4 dropped to 2.4%. At the same time, an earlier chatbot model — GPT-3.5 — improved the identification of prime numbers over the same period of time. When it came to generating lines of new code, the capabilities of both models deteriorated significantly in the three months from March to June.
ChatGPT's responses to sensitive questions — with some examples emphasizing ethnicity and gender — later became more concise. An earlier version of the chatbot provided detailed explanations of why some sensitive questions cannot be answered
ChatGPT approved the wrong tokens for listing
Another study by the Coinbase cryptocurrency exchange showed that ChatGPT was unable to achieve the required level of analysis accuracy. In five out of eight cases, the chatbot identified high-risk assets as low-risk and approved them for listing on the platform. In addition, an AI bot cannot understand situations where it does not have enough data for qualitative analysis.
However, 75% of traders are still willing to believe ChatGPT's financial advice. A study by the Investor Index found that users trust the financial advice of an AI chatbot. At the same time, experts note that people rely less on the recommendations of professional consultants and prefer to study the situation on their own.
