ChatGPT text can sound very knowledgeable until the topic is something you know well. Like tic-tac-toe.
Once I heard that ChatGPT can play tic-tac-toe I played several games against it and it confidently lost every single one.
Part of the problem seemed to be that it couldn’t keep track of which moves had already been made.
It very strongly prefers to place its marks in the corners, longwindedly explaining how its corner move is a superior strategy, all while I win game after game by placing my marks in the center rows and columns.
Might it have been trained mostly on example games where Player 1 does the expected corner opening move? Maybe. But it loses if my opening move is in the corners as well.
There is no end to its fluent pomposity. There is also no end to its incompetence. It has all the surface appearance of being an expert without any actual expertise.
Something to keep in mind the next time someone tries to sell you a large language model for expert advice.