Misinformation
I enjoyed the article in this week’s Saturday Globe & Mail in which the author, a journalist, asked GPT-4 to write his biography in 1,500 words which it did in a few seconds. If I didn’t know anything about him I would think it was quite impressive and credible. But he has annotated it, and these are the things it got wrong from the beginning:
Place and date of birth – both wrong.
Beginning of writing career – wrong year.
University from which he graduated – wrong one – there were actually two correct ones replacing the wrong one.
Graduating degree: wrong one.
That was just the first paragraph. The second went better. It was correct in naming him as a journalist, but starting with his first job at a publication that went out of business 11 years before his writing career began. Then we are told he was offered a job at a rival paper – which he was never offered and would never accepted, he says.
Paragraph three states he is the author of 20 books. He wished that were true. The description of his first book actually describes another one written by someone else in 1939. He wishes he had written one on the Canadian wilderness – but never did. He notes that by now the bot is struggling to find 1500 words with fill like “The book was a critical success and helped establish . . . . as a rising star in the world of Canadian literature”. – worthy of the kind of fill any keen grade nine student might produce.
In further paragraphs, GPT-4 expands his output to several books on noted Canadians - substituting books for a review and an article and it got a year wrong again. It went on to describe his teaching career at two universities – he never taught at either of them - and only one year as a lecturer at another one. But it ended with another nice filler platitude. “His courses were popular with his students and many went on to have successful careers as writers.” None of them were named.
Honours - and the lack of them, came next – a Governor General’s medal for writing – but the reality was a nomination for one book and the topic in its description was wrong. Awards supposedly for column writing also do not exist. Alas, he is also waiting for the Order of Canada mentioned in the GPT- Biography. To make it worse, he is reported to have died in 2016 – though he is still here to write the article. With a bit more commendation the bio finally reaches its 1500 words – “as his legacy as one of Canada’s most beloved authors and journalists lives on.” He does say amen to that.
We don’t need to fear GPT-4 for accuracy any time soon. But if I had read the bio without the annotations, I could well have believed at least some of it. That’s the danger. In the meantime, I’ll request my own 1500 word biography and see what happens. I just hope I am still alive.