Erik Hoel in The Intrinsic Perspective:
Despite being the culmination of a century-long dream, no better word describes the much-discussed output of OpenAI’s ChatGPT than the colloquial “mid.”
I understand that this may be seen as downplaying its achievement. As those who’ve been paying attention to this space can attest, ChatGPT is by far the most impressive AI the public has had access to. It can basically pass the Turing test—conversationally, it acts much like a human. These new changes are from it having been given a lot of feedback and tutoring by humans themselves. ChatGPT was created by taking the original GPT-3 model and fine-tuning it on human ratings of its responses, e.g., OpenAI had humans interact with GPT-3, its base model, then rate how satisfied they were with the answer. ChatGPT’s connections were then shifted to give more weight to the ones that were important for producing human-pleasing answers.
Therefore, before we can discuss why ChatGPT is actually unimpressive, first we must admit that ChatGPT is impressive.
More here.