In the past couple of weeks, a new generation of computer language-generating tools has become available to the public. The main bit of news is about a product called "ChatGPT," but that's just the most recent iteration of a class of software "chat bots." (Whether it's appropriate to refer to these systems as "artificial intelligence" is a philosophical question.)
Within a few days of ChatGPT's release, Stack Overflow issued a temporary don't-use-this policy, stating
Overall, because the average rate of getting correct answers from ChatGPT is too low, the posting of answers created by ChatGPT is substantially harmful to the site and to users who are asking or looking for correct answers.
The primary problem is that while the answers which ChatGPT produces have a high rate of being incorrect, they typically look like they might be good and the answers are very easy to produce. There are also many people trying out ChatGPT to create answers, without the expertise or willingness to verify that the answer is correct prior to posting.
On Stack Overflow, the blanket ban has mostly been a volume problem. Physics is a much smaller community, and we have so far detected only a smattering of such posts. However, the ones we have found have been pretty terrible, ranging from low-information word salad to obvious physical errors. For example, the sentence
In the case you described, the Drapher's point [sic] corresponds to a temperature of approximately 3,631 K and a wavelength of approximately 3631 nanometers.
should raise the eyebrows of anyone whose physics education has gotten as far as Wien's Law. (It may not, however, raise the eyebrows of anyone who has tried to teach Wien's Law to reluctant intro-astronomy students.)
In another post, the asker ended their question with "I asked an AI, but it didn't help me," followed by a properly-quoted paragraph which hadn't helped them because it didn't make any sense. I had a little flashback to when my children were small, and would sometimes run excitedly up to me, saying, "this thing! i found it on the floor! it tastes so gross! you have to try it!"
Some posts have even crossed the line from well-intentioned to deceptive. On one now-deleted post, a commenter asked the user who posted the answer to include references. The post was edited to include
Some references for spin fluctuation are:
- "Pairing in Type-II Superconductors Induced by Spin Fluctuations" by D. J. Scalapino, E. Loh, Jr., and J. E. Hirsch, Physical Review Letters, Vol. 50, No. 4 (1983)
- "Spin Fluctuation-Mediated Pairing in Type-II Superconductors" by D. J. Scalapino, E. Loh, Jr., and J. E. Hirsch, Physical Review B, Vol. 34, No. 6 (1986)
- "Spin Fluctuation-Mediated Superconductivity: A Review" by D. J. Scalapino, Physics Reports, Vol. 250, No. 3 (1995)
It's instructive to compare these "references" to a search of this time period at the Physical Review, which should turn up the first two. Scalapino and Hirsch coauthored a number of papers on superconductivity in the 1980s, including one in PRL v50 (1983) and another in PRB v34 (1986). However, Loh doesn't seem to have joined the group until 1986, and none of the team's coauthored titles includes the phrase "spin fluctuations." Likewise, the best candidate for the third reference has a different issue number and title. Is it a good use of anyone's time to pursue this detective work into thirty- and forty-year-old literature to see whether these rhymes-with-correct citations address the question at hand? Almost certainly not.
Note that my request to "please don't do this" isn't a new fancy policy tailored to the existence of an exciting new chatbot which superficially appears intelligent. Our community has a number of established posting standards which are violated by these low-quality contributions:
Originality. User contributions on this site are expected to be primarily the poster's own original work. If properly cited, including a small passage from a third party is fine, but complete answers are not.
Attribution. Content which originally appeared elsewhere, including your own content, must be posted with attribution. Plagiarized content may be hidden until appropriate attributions are added, or may be removed altogether. It isn't common, but some serial plagiarists have found their site-use privileges suspended.
Respect for others. If a user posts a question or an answer, our community needs to be able to expect that the post is a good-faith effort to learn things, or to help other people to learn things. Note that the network-wide policy is that "abuse of the system or the community," including cat-on-keyboard gibberish posts, can reasonably be flagged using the "rude or abusive" option, where enough flags will automatically delete the post and apply a reputation penalty. Surreptitiously involving Physics users in your tests of some chatbot software is rude. Generating "citations" without any idea whether they refer to real documents or not, much less whether the cited documents are relevant, is an abuse of other people's time.
