210
$\begingroup$

We, the moderators of physics.SE, are deeply concerned about the recently announced "network policy" regarding computer-generated content on the SE network, which effectively attempts to forbid moderators from issuing suspensions or deleting posts for the undisclosed usage of computer-generated content.

When text generated by new GPT models became a pressing concern in December 2022, we explained that we consider the undisclosed use of such models to violate the standards of honesty and integrity physics.SE strives to uphold, and that we treat such content as violations of our pre-existing policies on plagiarism and interacting with other users in good faith, regardless of the quality of the content. In particular, while we are generally fairly cautious with moderator action towards posts that are merely of low quality, plagiarism in general and the use of computer-generated text specifically quickly lead to suspension of the user and deletion of the plagiarized posts.

Since December, we have issued 17 moderator warnings — most of these together with 7-day suspensions — for plagiarism via GPT-like models, and we have no evidence whatsoever that any of these warnings were in error. While we of course are fallible, we are convinced that these suspensions protected and continue to protect our site from a higher influx of plagiarized computer-generated texts, and we are prepared to explain our reasoning for each of them, if requested, in public or in private. We have received no such request from either the users involved or SE employees before this new network policy was instituted.

Therefore, we reject the implication in the network policy that there has been an unacceptably high false positive rate in the way we deal with computer-generated content, and we see no reason to change our processes. We owe this to you, the physics.SE community, since our promise to safeguard this site from computer-generated content on this meta was met with almost unanimous approval (it has received only a single downvote at the time of writing) — a rare thing for policy discussions on physics.SE, and a strong sign this community wants us to continue to act as we have before the announcement of the new network policy.

We call on all moderators of other sites in the network who value honesty and integrity and who value the core identity of our SE communities as places where amateurs and experts alike interact as humans, to do the same and rank the needs of their communities above this ill-considered network policy.

$\endgroup$
27
  • 46
    $\begingroup$ Bravo for the stance, I appreciate it! $\endgroup$
    – Jon Custer
    Commented Jun 1, 2023 at 18:41
  • 7
    $\begingroup$ Diamond is taken in 3... 2... 1... $\endgroup$
    – Shadow Wizard
    Commented Jun 1, 2023 at 20:00
  • 3
    $\begingroup$ @ShadowTheSpringWizard - well, the sooner the community finds out the better, for better or for worse. $\endgroup$
    – Jon Custer
    Commented Jun 1, 2023 at 20:33
  • 2
    $\begingroup$ I don't quite understand why this needs to be a system-wide policy rather than a specific subsite policy. $\endgroup$
    – ZeroTheHero
    Commented Jun 1, 2023 at 20:38
  • 1
    $\begingroup$ I propose going on strike for a day. SE needs to see that it needs its community and not just AI $\endgroup$
    – Starship
    Commented Jun 1, 2023 at 21:11
  • 1
    $\begingroup$ Just as a warning, more if nothing is done @JonCuster $\endgroup$
    – Starship
    Commented Jun 1, 2023 at 21:25
  • 13
    $\begingroup$ Fellow members: If you upvote this, please consider downvoting the new network policy. I am astonished that the net vote is not more negative than -163. $\endgroup$
    – Ghoster
    Commented Jun 2, 2023 at 0:22
  • 1
    $\begingroup$ @Ghoster - very few participate on the mother Meta. I read but not much more for example. $\endgroup$
    – Jon Custer
    Commented Jun 2, 2023 at 2:06
  • 4
    $\begingroup$ @Ghoster The visibility is not great since it doesn't appear on the front page once the score gets too low. The cynical part of me wonders if this is a strategy to reduce the blowback a bit. $\endgroup$
    – Chris Mod
    Commented Jun 2, 2023 at 2:20
  • 8
    $\begingroup$ @Ghoster need more than 101 rep to downvote, so vast majority of the users, including those with very high rep on other sires, can't downvote. On the other hand, anyone with as little as 200 rep in one site can upvote. If everyone could downvote, I'm pretty certain is was around -10000 by now. That said, I might start there a feature request to allow downvoting official posts like that even with less than the required reputation. :) $\endgroup$
    – Shadow Wizard
    Commented Jun 2, 2023 at 5:15
  • 4
    $\begingroup$ @kludg It looks like, this specific posting is talking about undisclosed use of AI. $\endgroup$
    – A. B.
    Commented Jun 2, 2023 at 12:23
  • 14
    $\begingroup$ @kludg There are other websites where humans can ask questions and get computer-generated answers. If the answers are memorable or valuable, there are other websites where they can be shared. Likewise there are other websites where people can ask about bicycle maintenance, or about parenting advice, or about how Tibby is dealing with her time in hospice. We don't do any of those things here; we are not the entire internet. (But since it came up, Tibby is in excellent spirits, and loves visitors and phone calls, even though they make her tired.) $\endgroup$
    – rob Mod
    Commented Jun 2, 2023 at 13:25
  • 5
    $\begingroup$ @ShadowTheSpringWizard While other sites require 125 to downvote, meta.se only requires 100. $\endgroup$
    – Sandejo
    Commented Jun 2, 2023 at 20:25
  • 10
    $\begingroup$ @Starshipisgoforlaunch An indefinite strike is already planned $\endgroup$
    – Zoe - Save the data dump
    Commented Jun 4, 2023 at 9:46
  • 6
    $\begingroup$ @Mazura I’m willing to bet users pasting GPT answers won’t engage with the site the way human users do: if it’s so easy to get an answer from GPT and post it on SE, it’s equally easy for posters to directly query GPT and get their answers directly from there. $\endgroup$
    – ZeroTheHero
    Commented Jun 9, 2023 at 1:31

3 Answers 3

Reset to default
73
$\begingroup$

I firmly applaud the stance taken by the moderators here. AI-generated text has many demonstrated problems that disqualify its use, without significant human intervention, for technical (e.g. physics!) uses.

First, as a recent BBC article points out:

When people use ChatGPT, they are shown a disclaimer warning that the content it generates may contain "inaccurate information about people, places, or facts".

And on its public blog about the tool, OpenAI says a limitation is that it "sometimes writes plausible-sounding but incorrect or nonsensical answers".

These are not the attributes of a solid scientific answer.

Furthermore, as a recent Physics Meta post pointed out, AI is perfectly happy to flat out invent "references" to make it sound more authoritative. Peer reviewing a journal article is one thing, but having to go look up references in Physics SE postings to see if the poster (or AI) invented them is asking too much.

If you want to use AI to take your human-generated text and improve the writing quality, well, OK, that isn't bad. Relying on AI to get physics right is asking way too much of it.

So, thanks to the moderators for trying to eliminate nonsense in physics.

$\endgroup$
10
  • 1
    $\begingroup$ i don't see the new "network policy". I don't believe that it promotes undisclosed use of AI. Actually after this post I don't believe mods. $\endgroup$
    – kludg
    Commented Jun 3, 2023 at 0:02
  • 25
    $\begingroup$ @kludg - start with meta.stackexchange.com/q/389582 and work forward in time. The mods know what they’ve been told, and corporate has not released what they told the mods but didn’t openly post. $\endgroup$
    – Jon Custer
    Commented Jun 3, 2023 at 0:44
  • 2
    $\begingroup$ I've read enough to form my opinion. I don't believe mods and don't want to take part in their games. $\endgroup$
    – kludg
    Commented Jun 3, 2023 at 3:56
  • 9
    $\begingroup$ @kludg I wouldn't say closing your eyes and covering your ears is the smartest way to form an opinion. $\endgroup$
    – Clonkex
    Commented Jun 7, 2023 at 4:20
  • $\begingroup$ "Relying on AI to get physics right is asking way too much of it." - "how many neutrons are in a liter of water" (note, only US spelling works) - Bing: none, CGTP3.5: 556, CGTP 4.0: 2x the correct number. I'm sure they will fix this, if they have not already, but that just means all the other answers are wrong. $\endgroup$
    – Maury Markowitz
    Commented Jun 7, 2023 at 14:05
  • $\begingroup$ Wow, 556??? Those are some mighty unusual oxygens there… $\endgroup$
    – Jon Custer
    Commented Jun 7, 2023 at 15:53
  • 1
    $\begingroup$ @MauryMarkowitz If it can't count the words in a sentence, perhaps asking it to count neutrons in a liter of water is too asking too much? :p $\endgroup$
    – Chris Mod
    Commented Jun 7, 2023 at 20:30
  • 1
    $\begingroup$ FWIW, Stephen Wolfram has written some excellent articles about how GPT works and its limitations, including why its computational abilities are very rudimentary. writings.stephenwolfram.com/2023/02/… $\endgroup$
    – PM 2Ring
    Commented Jun 8, 2023 at 22:51
  • 2
    $\begingroup$ From scottaaronson.blog/?p=7094#comment-1947593 It first gives some correct info about Fibonacci numbers, but then it says: "Two consecutive natural numbers that have the same parity are 8 and 9, both of which are odd" $\endgroup$
    – PM 2Ring
    Commented Jun 8, 2023 at 22:51
  • 1
    $\begingroup$ @PM2Ring Give it a sentence and ask it how many words are in it. It's impressively bad at numbers in general. Even if it gets it right by chance, you can just reply "Is it?" and it will apologize and change its mind. $\endgroup$
    – Chris Mod
    Commented Jun 14, 2023 at 1:46
27
$\begingroup$

In a comment, Braiam asks

How many of those users came back after the suspension?

I looked at 16 profiles with a plagiarism mod message since January 1. (One of the suspensions on the list of 17 wasn't actually ChatGPT-related. That's one of the perils of data analysis: typos.)

description count
users with mod message 16
users who visited after mod message 13
users who read mod message 10
users who returned after reading mod message 10
users who returned after end of suspension (>7 days later) 6
users with undeleted content since end of suspension 3
users with >100 reputation 3
users from outside usa/europe 6
users whose entire answer history occurred on one day 10
mean answers per single-day user 5.1
users who can't currently ask questions 9

It's dangerous to overinterpret a small dataset like this one. But a little more than half of these users joined the site, posted as many as fifteen low-quality answers on the same day, and were suspended for that. Those users are now mostly subject to the automated answer ban, and wouldn't be able to post answers again if they wanted to. Of the six users who aren't subject to the automated answer ban, one hasn't been back since about three weeks before their suspension (apparently their low-quality contribution flew under everyone's radar), while about half of the others have returned and posted new content. Two users actually replied to their moderator messages with apologies, which is much more understanding than usual. One of those has in fact returned regularly; the other is still suspended, but they did check in today.

$\endgroup$
5
  • 1
    $\begingroup$ Man, if you are going to do this, do not delete the comment. I was here wondering if you were going to do my suggestion or something, and didn't have a way to navigate back here. Also, as much as I expected, most of those users will not contribute anyways. I would suggest to encourage other SE sites to do this, since the community is in the blind about how much of a problem this is. $\endgroup$
    – Braiam
    Commented Jun 5, 2023 at 11:11
  • 3
    $\begingroup$ Sorry — it looks like another moderator was cleaning up comments, and took out both your request and my reply to it. I appreciated your question. $\endgroup$
    – rob Mod
    Commented Jun 5, 2023 at 12:32
  • 1
    $\begingroup$ So, less than one action a week… I’m sure there are more (many more) on the programming sites, but they have many more questions per day then we do. $\endgroup$
    – Jon Custer
    Commented Jun 5, 2023 at 13:23
  • 6
    $\begingroup$ I have a bunch of data questions, @JonCuster. But consider that Stack Overflow's volunteer moderator team is 4.5x ours (twenty-seven there to six here), while they have 50x the question traffic. If they've been doing fifty chatbot-related moderation actions a week, compared to our one-ish, that's a very different story. My suspicion is that the Stack Overflow audience is also much more likely than the Physics audience to include people who are interested in experimenting with new natural-language software. $\endgroup$
    – rob Mod
    Commented Jun 5, 2023 at 14:07
  • 4
    $\begingroup$ @rob - I agree completely. We (the non-programming sites all told) are just collateral damage. $\endgroup$
    – Jon Custer
    Commented Jun 5, 2023 at 14:31
-10
$\begingroup$

A possible explanation would be, if you want ChatGPT answers, then ask ChatGPT, and not the humans on the PSE.

I think, plagiarism is not a really effective argument, because the content of the ChatGPT is free and anyway we have our rules for citing other sources.

The more important question is the race between us and the machines. Or, between the content created by us, and the content created by the machines. Currently, ChatGPT can not create really high quality, useful answers, but odds are that soon it will.

If the site would have even a little bit of collective wish for the long-term survival, for the long-term success, we should think about, how to prepare for that world already now.

$\endgroup$
14
  • 31
    $\begingroup$ I don't think it matters that the output of ChatGPT is "free." If I were teaching a class and I learned that a student had let their mom write their papers, that student would fail, whether they paid their mom for her help or not. $\endgroup$
    – rob Mod
    Commented Jun 3, 2023 at 18:42
  • 32
    $\begingroup$ That ChatGPT is "free" is irrelevant: Plagiarism is not about copyright violation, it's about trying to pass off other people's (or, in this case, algorithms') work as your own. $\endgroup$
    – ACuriousMind Mod
    Commented Jun 3, 2023 at 18:43
  • 1
    $\begingroup$ Once such engines will be able to synthesize a high quality answer to a Physics question, almost certainly in some cases it will come from being trained on data from a site like Physics.SE! :) Consequently, I see a certain parallel between a possibly good AI answer and a duplicate question: if an AI was able to produce it, it means it was addressed somewhere, ofc not necessarily on Physics.SE. I think we are still a good time interval away from the point when such engines can give high quality answers to what is roughly called "novel" questions -- but that of course is up for debate :) $\endgroup$
    – Amit
    Commented Jun 3, 2023 at 18:44
  • $\begingroup$ rob & acm: okay, but there are rules, how can external sources be cited. What if someone answers this: "I asked chatgpt here (link to chatlog), and his answer is that [...]". I think that is rule-conformant. $\endgroup$
    – peterh
    Commented Jun 3, 2023 at 18:45
  • 14
    $\begingroup$ @peterh: Yes, if you disclose the usage of language models, that is not plagiarism. That's why both this and our earlier meta post focus largely on the undisclosed use of such models. We generally discourage answers that are completely made up of quotes from other sources, but that situation is not my primary concern here. $\endgroup$
    – ACuriousMind Mod
    Commented Jun 3, 2023 at 18:47
  • 19
    $\begingroup$ @peterh Saying you asked ChatGPT does not allow any confidence that the answer is correct. To consider posting a ChatGPT answer as valid the poster must also stand by the answer themselves and is (in effect) claiming they verified it. If they've verified the answer themself they could just as simply have written an answer without ChatGPT. $\endgroup$
    – StephenG - Help Ukraine
    Commented Jun 3, 2023 at 23:46
  • $\begingroup$ @Amit Yes. And after that, there will be yet another step. As the AI can answer physics questions what no one can. $\endgroup$
    – peterh
    Commented Jun 4, 2023 at 10:17
  • $\begingroup$ @peterh That will be exciting for the field of physics, although maybe less exciting to some physicists :) But until that happens, I was trying to suggest, perhaps this intermediate stage can be a pretext to raise the quality and novelty of questions on the site, i.e. maybe at some point a new flag will be created under the name of "Answerable by AI" :) $\endgroup$
    – Amit
    Commented Jun 4, 2023 at 10:30
  • $\begingroup$ @Amit Well, after that, there will be yet another step. It will be able to answer questions, for which no living human can understand the answer. $\endgroup$
    – peterh
    Commented Jun 4, 2023 at 18:18
  • 1
    $\begingroup$ As far as I am concerned this enters the area of wild speculation :) I like thinking about futuristic technology very much, but I don't have enough confidence to assert that such a thing will happen, and even less confidence in guessing what will happen in the foreseeable future $\endgroup$
    – Amit
    Commented Jun 4, 2023 at 18:21
  • $\begingroup$ @Amit I think, things will become completely above us from the point, that only the AI will understand even the question. $\endgroup$
    – peterh
    Commented Jun 5, 2023 at 2:36
  • 1
    $\begingroup$ Plagiarism is not the right way to go about this. This is an internet forum and not a research paper. Nobody is making any money or getting real life benefits off of posting here. People come here to get answers to their problems, quickly if possible without any other added fluff. If someone wants to link to their original source in their post for others to read up on, that's great. But it shouldn't have to be mandatory, regardless of whether his original post is generated by ai or not. Only thing that should matter is whether the thing posted is correct and actually helps the asker. $\endgroup$
    – user13267
    Commented Jun 16, 2023 at 3:28
  • $\begingroup$ @user13267 ...and your last sentence is imho a strong argument against the strikers. But also I am not sure, what to do. I think the problem is not the ChatGPT of the today, but of the tomorrow. Odds are that it will be able to answer questions what no one can. $\endgroup$
    – peterh
    Commented Jun 16, 2023 at 4:39
  • $\begingroup$ @peterh personally I think chatgpt is not a problem at all. It's just a tool. Select few users here should not get to decide what tools anyone wants to use to create their post. Does this tool make spamming easier? Possibly. But deal with that spamming for what it is, instead of going after the tool used for making it. The same argument can be made for anything else. Any kind of tool might be used for spamming, but the spamming is what needs to be dealt with, not the tool. $\endgroup$
    – user13267
    Commented Jun 16, 2023 at 4:46

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .