One example I like to use is to ask it for the lyrics of an extremely well known song. It just makes shit up based on the title you give it.

The online ones (Claude, chatgpt, copilot, etc) now refuse to do it for ““copyright reasons”” but the offline ones still happily oblige. I assume the online ones added that block because it was such an obvious way to prove they don’t “know” shit.