ChatGPT's Behaviour is Changing Over Time

Submit
arxiv.org

ChatGPT's Behaviour is Changing Over Time

AI summary of the linked article

Researchers Lingjiao Chen, Matei Zaharia and James Zou report that the behavior of GPT-3.5 and GPT-4 changed substantially between their March 2023 and June 2023 versions, which they say highlights the need for continuous monitoring of LLMs. GPT-4 identified prime versus composite numbers with 84% accuracy in March but 51% in June, which the article partly attributes to a drop in its adherence to chain-of-thought prompting. GPT-4 became less willing to answer sensitive and opinion survey questions in June, while its multi-hop question performance improved and GPT-3.5's declined. The authors also provide evidence that GPT-4's ability to follow user instructions decreased over time.

Recommended by 2 curators
No comments yetWhat caught your eye? Start the discussion.