I have been playing recently with the AI program "chatGPT" which has been causing quite a stir recently. Initially I tried fairly silly questions to trip it up, and attempts to get it to write poetry (it can't!). But then I hit on the idea of getting it to rewrite an serious article about a minor academic discovery I made in 1996, in the style of a tabloid newspaper, such as the Daily Mail, and to suggest a headline for it.
The details of the story are below in the text of the AI-generated article and the question I submitted to chatGPT.
chatGPT's "Daily Mail" style article about Shostakovich:
SHOCK DISCOVERY: Data Scientist Uncovers Shostakovich's Cryptic Musical Message Beyond the Grave!Iain Strachan, a data scientist with a love for mathematics and music, has made a startling discovery about the music of Russian composer Dmitri Shostakovich. While studying for his PhD in machine learning, Strachan uncovered a hidden pattern in Shostakovich's String Quartets that suggests the composer left a cryptic message for future generations.
According to a biography of the composer, Shostakovich had intended to write a full set of 24 string quartets, one in every major and minor key, just as he had written a set of 24 Preludes and Fugues for piano. Strachan, intrigued by this puzzle, looked for a logical sequence of keys in the existing set. He discovered that the sequence of String Quartets appeared to be the reverse of a well-known music theory diagram called the "Circle of Fifths," going anti-clockwise around the circle with some discrepancies where the sequence was broken.
Strachan's curiosity led him to a strange connection with a well-known sequence of notes that represented the composer's musical "signature" - a motif that he used many times in his works as a reference to himself. These notes were D, E flat, C, and B. When the German names for these notes are used, they become D-S-C-H, being part of his name in the German spelling D. SCHostakowitsch. Strachan wondered if the signature was somehow connected with the apparent inconsistencies in the sequence and found that the keys of D, E flat, C, and B would all have been assigned to quartet numbers, which were perfect squares: 4, 9, 1, and 16, had the composer lived to complete a 16th Quartet.
Strachan believed that Shostakovich left this message deliberately, shifting the E flat major quartet from number 7 to number 9 and wondering if someone would find it. He even joked, "I had the weird feeling of being spoken to from beyond the grave. And that person appeared to be me!"
Strachan wrote to several classical music magazines to ask if anyone had noticed this "digital signature of squares," but none replied. The BBC Music Magazine forwarded his letter to a Shostakovich expert, Professor X Y of Z.... University, who immediately wrote back to Strachan agreeing that this was probably intentional and congratulating him on a fascinating piece of detective work.
This discovery has caused a sensation in the classical music world, as scholars and fans alike marvel at the genius of Shostakovich's hidden message. Who knows what other mysteries are waiting to be uncovered in the great works of classical music?
The text submitted to chatGPT:
Re-write the following story as a news item in the style of a tabloid newspaper, and suggest a headline for the piece:Iain Strachan is a data scientist with a background in physics and mathematics.
While studying for a PhD in machine learning, he made an interesting musical discovery, concerning the music of Russian composer Dmitri Shostakovich (1906-1975); in particular his 15 String Quartets.
On reading the Elizabeth Wilson biography of the composer, he learnt that Shostakovich had intended to write a full set of 24 string quartets, one in every major and minor key, just as he had written a set of 24 Preludes and Fugues for piano. He had told a journalist: “haven’t you noticed I never repeat a key?”
Being interested in mathematical puzzles and sequences, he looked to see if there was a logical sequence of the keys in the existing set. The 24 Preludes and Fugues followed a well-known sequence, based on a well-known music theory diagram called the “Circle of Fifths” – progressing in a clockwise direction around the circle. Looking at the sequence of String Quartets, he noticed that it appeared to be the reverse of this, going anti-clockwise around the circle, but there were some discrepancies, where the sequence was broken.
Notably, the key of quartet number 7 should have been E flat major, but in fact was a very different key, F sharp minor – which is diametrically opposite E flat major in the circle of fifths diagram. The E flat major Quartet was placed at number 9 in the sequence, after which the anti-clockwise sequence continued. From this it was possible to guess what key the sixteenth quartet would have been in, had the composer lived to complete it.
This anomaly was puzzling for Strachan, who tried to find an explanation for this. He found a strange connection with a well-known sequence of notes that represented the composer’s musical “signature” – a motif that he used many times in his works as a reference to himself. These notes were D, E flat, C and B. When the German names for these notes are used they become D-S-C-H, being part of his name in the German spelling D. SCHostakowitsch. Strachan wondered if the signature was somehow connected with the apparent inconsistencies in the sequence, and then found, to his surprise that the keys of D,E flat, C and B would all have been assigned to quartet numbers which were perfect squares: 4, 9, 1 and 16, had the composer lived to complete a 16th Quartet.
Recounting the discovery to a friend at a later date he said: “I had the weird feeling of being spoken to from beyond the grave. I thought that he must have done this deliberately, because of the shifting of the E flat major quartet from number 7 to number 9, and he must have wondered if someone would find this. And that person appeared to be me!”
He subsequently wrote to four classical music magazines to ask if anyone had noticed this “digital signature of squares” but none replied. However, after sitting on it for several months, the BBC Music Magazine forwarded his letter to a Shostakovich expert, Prof. X... Y... of Z... University, who immediately wrote back to Strachan saying he agreed that this was probably intentional and congratulating him on a fascinating piece of detective work.
My thoughts:
Initial impressions
My initial impressions were that it was quite amazing. I have little respect for the tabloid press and their tendency to sensationalise things. The headline didn’t disappoint in that respect. The AI certainly knows how to generate a clickbait headline, particularly in picking up the alluring phrase “beyond the grave” to add to the sensationalism factor. Equally the tabloid-style hype in the final paragraph seemed spot on for the exercise.Does it pass the Turing Test? (is it indistinguishable from an article written by a human?)
Close, but not quite.
The eagle-eyed will spot that the program has moved the text about Shostakovich wondering if anyone would find the pattern from direct speech to the indirect speech preceding it, but it then leaves the nonsensical tautology in the direct speech. Compare chatGPT's version:
Strachan believed that Shostakovich left this message deliberately, shifting the E flat major quartet from number 7 to number 9 and wondering if someone would find it. He even joked, "I had the weird feeling of being spoken to from beyond the grave. And that person appeared to be me!"
with the original ...
“I had the weird feeling of being spoken to from beyond the grave. I thought that he must have done this deliberately, because of the shifting of the E flat major quartet from number 7 to number 9, and he must have wondered if someone would find this. And that person appeared to be me!”
So chatGPT has subtly changed the meaning from Shostakovich wondering if someone would find it (that turned out to be me), to Shostakovich directly speaking from beyond the grave to me. I think it will have done this simply because the algorithm has rearranged the text in a perfectly grammatical fashion, without considering that the meaning has changed, and that the quoted speech actually doesn't make logical sense. At the end of the day, it appears to be a clever algorithm that generates text on the fly but doesn't actually "think" about the meaning.
It is possible that a journalist might have made such an error as well, but one hopes that a sufficiently astute editor would correct this.
Is it reliable?
No. It has a tendency to make stuff up. The final paragraph might well be what would appear in a sensationalist tabloid article. However, there is nothing in the original text to suggest that this was the case. It has projected a very believable last paragraph in classic tabloid style, but it is also completely wrong.
In fact the only impression it made was that the discovery was mentioned in a footnote in Prof X Y's book on Shostakovich’s Eighth Quartet (which famously incorporates the DSCH motif).
Overall impression.
It was great fun to do the exercise and see what it could come up with. Bearing in mind it is only an algorithm based on associative memory, and has limited deductive powers, it is an incredible, albeit slightly imperfect result.
The headline and the final paragraph actually made me burst out laughing.
Should you use it to do your homework, or set essays? No. It makes stuff up out of nothing, and occasionally commits factual errors.
Should the Daily Mail use it as a journalist? Definitely! But articles it writes should be carefully checked by an editor.
Should the Daily Mail use it as a journalist? Definitely! But articles it writes should be carefully checked by an editor.
Comments