10
$\begingroup$

The SE team has made several promising changes to the "Hot Network Questions" system. If you have previously found HNQs to be problematic, you are invited to participate in the discussion over there.

The purpose of this discussion is a new moderator superpower:

Moderators have the ability to remove questions from the HNQ List.

There are times when the hotness formula selects a question that a site would rather not have featured. Up until now, the only recourse that was available was to close the question (which may be appropriate anyway but isn't ideal when done purely to manage traffic), or to do nothing. We're putting the power in the hands of our moderators to remove questions that don't set a good example for their sites. I recommend each site have a meta discussion with guidance for moderators about when - if ever - a question should be removed.

I'd like to propose that we not invent a new policy on these questions de novo. I would prefer instead that we think as a community about the kinds of standards that we already have in place and communicate via voting, close-voting, and flagging, and how this new tool fits into our kit.

I'm imagining a process like this:

  1. Over the next few weeks you will, like usual, notice some unusually active questions and think "well, that shouldn't be HNQ."

  2. When you notice this, raise a custom moderator flag on the question and try to articulate what your reasoning is.

  3. The moderation team will deal with your flag promptly --- that's the point of flags. But we'll also save your reasoning someplace, and as we start to get enough flags to divine a pattern, we'll try to summarize it either here or in a new post.

You could also link to posts in answers to this question, and explain how you think that the Hot Network Questions list has affected them. Those kinds of answers shouldn't be limited to bad posts that should be un-HNQ-ed: part of deciding what our policy should be will be deciding what we do want to keep. I'm especially interested in articulating, as a community, what types of posts get better responses after exposure via HNQ.

I also suggest flagging (or linking) current questions, because there will be some temptation to base any policy we develop on remembered examples from the past. For deciding how we think the site ought to work, it's usually better to notice how we are responding when we are using the site organically. I would hate for us to have a contentious discussion about a problematic question from the past and lock ourselves into some "policy" based on behaviors that don't really occur on the site any more. That's why I've titled this post "data collection": I'm proposing that we spend a reasonable amount of time just looking for patterns, and try to come to a conclusion around May-ish.

$\endgroup$
  • 1
    $\begingroup$ What does point 3 mean exactly? If I flag something and say "Well, that shouldn't be there!" does that imply it could be removed if a mod agrees? Or is this just purely data-collection until something is decided and "deal with your flag" just means dismiss it? $\endgroup$ – tpg2114 Mar 12 at 22:03
  • 1
    $\begingroup$ I don't fully understand your last paragraph. What happens if there is a class of questions where we get one on HNQ every 6 months, say, with that rate constant over the past 3-5 years and ongoing, but with an outsized effect compared to other 'that shouldn't be there' questions? Is 'the next few weeks' a sufficient timespan to adequately sample a representative cross-section of the relevant question types? $\endgroup$ – Emilio Pisanty Mar 12 at 23:29
  • $\begingroup$ Does anybody have any nice ideas about how we can implement something related to the number of close votes and downvotes on a question? SE hasn't changed the algorithm for 'hotness' to consider these factors, but I'm sure that we could put together our own makeshift version. $\endgroup$ – user191954 Mar 13 at 4:11
  • $\begingroup$ If you "look for patterns" in a fairly small dataset, with no predefined criteria, you will find them, because that's what human brains do. It doesn't mean they have any significance, or that you would find the same patterns in the next time-slice of data. For example (to state the obvious!) if you look at the tags, or number of votes, or whatever, you are guaranteed to find that some questions score highest and other score lowest on whatever criterion you pick - but so what? $\endgroup$ – alephzero Mar 13 at 10:20
  • 1
    $\begingroup$ As a note: Users can now keep track of the Physics HNQs (day-wise) from the Backup Room transcript. $\endgroup$ – S.D. Mar 13 at 20:49
  • $\begingroup$ @tpg2114 You're right, that was vague. What I meant was that, in response to a flag, the moderation team will have a conversation and use our collective judgement to decide whether to act using our new tool out of a sense of urgency, or whether instead to watch patiently, or whether instead to take some more traditional action (like closing or improving the question). We will be cautious about using this new tool before we have a good community consensus, but I don't want to tie our hands with a "we would never"-style promise. $\endgroup$ – rob Mar 14 at 20:52
  • $\begingroup$ @EmilioPisanty You make a good point, and the classes of questions in your first two posts below are good examples. $\endgroup$ – rob Mar 14 at 20:55
  • 1
    $\begingroup$ Relevant context: the equivalent thread on Meta MathOverflow. $\endgroup$ – Emilio Pisanty Jun 27 at 23:48
13
$\begingroup$

Questions about perpetual-motion machines should meet a stricter-than-usual bar in terms of clarity and prior research to get on HNQ.

Questions about perpetual-motion seem to have a particularly easy time hitching a ride on the HNQ list (this is one recent example), and it's easy to see why: they are instantly appealing to a lot of people, they are generally easy to understand, and they often serve as ways to either make fun of OP (for being "dumb enough" to ask such a question in the first place) or to rail at the physics "establishment" for insisting on trivial such things as having a consistent physical theory of nature.

However, they get very tiring, very fast. Basically, the only questions on the subject that make the bar for being on-topic are variations on the theme of

I came up with this device and I analyzed it in (insert flawed analysis that concludes with perpetual motion). Where did I make a mistake?

and after seeing two or three of those on the HNQ as Representative Questions of the site, they start to grate very fast. (And, as with the physics-in-fiction vein, they very quickly become a skewed presentation of this site's subject matter.) Frankly, I think we could do with a heck of a lot fewer of those.

I'm not quite sure I'd go as far as proposing that perpetual-motion machines should be completely off-limits for this site's HNQ output, but I would say that for such questions to remain on the list they should be asked in exceptionally clear way, they should have a very clear physics through-line, and they should clearly demarcate themselves from previous iterations of the subject on this site.

$\endgroup$
  • $\begingroup$ I agree here, but I think a more direct route is to simply be stricter with closing these questions as duplicates. For example, nobody was voting to close the question you linked, so I initiated a duplicate close for another magnetic perpetual motion machine, the point being that though they aren't the exact same setup, they come from the same misconception. $\endgroup$ – knzhou Mar 13 at 14:11
  • $\begingroup$ If we are inclusive in closing these things as duplicates, then almost all of the low-effort questions will be duplicates. The remaining ones often have some neat physics to teach, I recall one got a great answer explaining Johnson noise. $\endgroup$ – knzhou Mar 13 at 14:12
  • $\begingroup$ The Johnson-noise question would be this one? $\endgroup$ – Emilio Pisanty Mar 13 at 14:54
  • 1
    $\begingroup$ Frankly, I rather disagree with that closure. I was OK with it until it cooled down, but I was pondering whether to de-dupe-hammer it. I don't see why we need to penalize the askers of these questions by denying them the chance of getting an answer that is specific to the configuration they have in mind. "They come from a similar misconception" is not a closure reason broadly speaking. The only concern is HNQ, which can now be addressed separately, so it should. $\endgroup$ – Emilio Pisanty Mar 13 at 14:58
  • $\begingroup$ @EmilioPisanty Yeah, I also agree with that logic. I see no reason to punish someone asking about perpetual motion in good faith. The people who will argue against the physics until the heat death of the universe, I could care less about leaving the question open. But if it's an honest attempt to understand conceptually why a perpetual motion setup doesn't work as they expect, that's not inherently off topic at all. I do agree that keeping them off of HNQ is for the best though. As amusing as the answers can sometimes be... they're amusing for the wrong reasons here. $\endgroup$ – JMac Mar 13 at 19:26
6
$\begingroup$

OK, I'll start.

Questions about physics in works of fiction shouldn't be on HNQ.

This has been a point of contention for quite some time, and it has a strong track record of leaving people confused and upset, even several years after the question has had its heyday (example, example, example). I'm referring, in particular, to questions like "Could Sam have pierced Shelob's carapace?" or "Could Legolas actually see that far?", particularly when the titles are that explicit in their mentions of the fictional context.

To be clear, this is independent of the decision on whether such a question is on- or off-topic here, which can have both positive and negative answers. As examples, "how does the warp drive in Star Wars work?" is clearly off-topic, while something like "is orbital mechanics being portrayed correctly in (insert sci-fi story)?" has a much better chance of being a legitimate question about physics, for which the fictional context is relevant and worth mentioning but not overwhelming. The decision of whether a given question along that vein is on- or off-topic is (in a nutshell) "is it about physics or not".

However, I think we need to be significantly more strict when it comes to allowing that type of question on the HNQ list. The reason is simple: upvotes on those threads, on both the question and its answers, are of extremely low quality. The Q&A engine does a (reasonably) good job at vetting the quality of content, both questions and answers, and for regular questions it does an OK job (though with plenty of room for improvement) at vetting the quality of votes. However, for questions about works of fiction, that completely breaks down.

  • For the question, upvotes rarely seem to come from a position of "this question is carefully researched, well-posed and well thought through, and it is a useful framework that allows the physics to shine through" and seem to be more along the lines of "this question is about a work of fiction that I like".
  • For answers, upvotes have a much stronger bias to favour truthiness rather than actual quality in the ways that it is recognized elsewhere on the site.

Moreover, those problems compound themselves, as the wider-than-usual opening on the upvote tap further reinforces the positive feedback loop that keeps these questions in the list, to the detriment of real physics questions that could be getting that spot on the spotlight. And while the three-day cap definitely helps, the feedback-loop problems remain, so I don't see how this class of questions will behave differently in the currently-live scheme.

Furthermore, the fact that questions like these are so likely to explode in popularity means that they present a very skewed vision of this site's scope on that SE-network-facing window. We get maybe one on-topic question about physics-in-works-of-fiction for every 5-10,000 questions, at best? Then, until the mechanisms are tempered such that the natural rate of these questions appearing on the list is one such question per 10,000 Physics SE questions on the HNQ list, they should be prevented from appearing there.

$\endgroup$
  • 1
    $\begingroup$ This is an excellent starting point. Here's an even easier reason why I think we don't want those to be HNQs: they aren't representative of the site's content. If the HNQ system is supposed to aid discovery and it brings people here for LotR, it's certainly doing a bad job (I guess your last paragraph is about something similar, but I'd like to emphasize that bit.). $\endgroup$ – user191954 Mar 13 at 12:48
  • $\begingroup$ I agree with the idea of this; but I'm not sure if I would take it to quite the extent that you do. Questions with titles that obviously reference a work of fiction should definitely be removed from HNQ, because clearly a lot of potentially undeserved attention will quickly come to it. If the question title only references the physical concept that they have a question about, and the body clarifies that some fictional work inspired them to question it, I don't think we should necessarily remove that from HNQ. If you take it away from the title it should be no worse than any pop-sci HNQ. $\endgroup$ – JMac Mar 13 at 13:08
  • 1
    $\begingroup$ I agree here, but I've always thought such questions should instead just be migrated to Worldbuilding.SE or Scifi.SE. A big fraction of all of their questions are along these lines. Last time I proposed this I got downvoted to hell, though. $\endgroup$ – knzhou Mar 13 at 14:15
  • 2
    $\begingroup$ I need to disagree a bit with the second example (Legolas; full disclosure: I wrote the highest voted answer). The question is drawn less from the Tolkien fantasy setting and more from a reasonably well-known outreach channel that discussed it. Nearly all the upvoted answers took this as an excuse to share some (often very technical) expertise on imaging hardware and/or techniques in a way that made it accessible to non-experts. Perhaps others disagree, but if a question like this can give someone on (random other site) EL&U an accessible intro to diffraction limits or interferometry, why not? $\endgroup$ – Kyle Oman Mar 13 at 14:43
  • $\begingroup$ [con't] I still agree it's clickbait, and can sometimes (often?) go wrong, but if good answers are written, why not? $\endgroup$ – Kyle Oman Mar 13 at 14:44
  • $\begingroup$ @Kyle Maybe you're right - maybe these questions do deserve a chance to keep that status until they start veering off-topic? If that's the route we take, though, I would argue that they'd need a strict single-strike-and-you're-out policy, with even a single off-topic answer being cause for knocking the question out of HNQ. (That's the beginning of avalanche, particularly since the number of answers counts multiplicatively in the hotness score. Wait any later and you risk a serious degradation in quality.) $\endgroup$ – Emilio Pisanty Mar 13 at 15:05
  • $\begingroup$ @EmilioPisanty fair enough. The line for a given question may be a bit fuzzy, but I'm often pleasantly surprised by the spin an answerer can put on an otherwise mediocre question. So I agree giving it at least that one chance is worthwhile (sometimes). $\endgroup$ – Kyle Oman Mar 13 at 18:11
3
$\begingroup$

Let's talk specifics.

And by that, I mean, "let me be the first to shoot my own self-interest in the foot", and argue that this question, which I answered, was rightly kicked off the HNQ list.

What's the deal? Well, the question starts off with the title

Is the Born rule indeed wrong?

and it is basically a request to verify the correctness of an unpublished arXiv eprint. As pointed out in the comments, the consensus view on peer-review questions leans towards classing this question as Too Broad, but I answered it because the literature it was referring to was complex enough for an outsider to puzzle out on their own (in short: a second preprint commented on the paper to point out flaws in the argument, and the authors responded by adding an appendix) but still containable enough that an answer in our format can explain what the deal is. And, also, the paper is awful enough that its flaws can be explained reasonably quickly.

With that in mind, I think that question is on-topic and should remain open, but I do not think it's a good fit for the type of questions we want representing our site on HNQ.

  • For one (though I didn't realize it before posting), the title is clickbait, pure and simple. It can be improved, sure, but I'm not sure I see how far up the slope from "terrible" back towards "vaguely reasonable" it can be brought.

    Moreover, the contentiousness of the title (which cannot really be removed, even if it is softened to something more accurate like "Is this paper, which claims the Born rule is incorrect, right?") simply isn't backed up by a particularly well-thought-through question.

  • More to the point, keeping it on HNQ is just giving free advertisement to a truly awful paper, which really doesn't deserve it.

Though then again, those are pretty squishy reasons; I wish I could firm it down more but that's the best I've got. (For full disclosure: it got on HNQ and then kicked off before I had time to react - but my first reaction to finding out about the former was that I needed to flag it so it got off.) I'm not sure how much of this can be distilled into a clear policy going forward, and I'm looking forward for people's reaction to this to see how wild off the bat it is, and whether a more clear fingerprint of what made this a bad fit (which can then be used as a benchmark to compare against in future cases) can be hammered out.

Still, though, I think the core point is pretty strong: this was not a question that we want to represent the site, if nothing else because "yeah, we'll fact-check your shallow questions about fifteen-year-old abandoned preprints" isn't really the kind of message I think we should be sending.

$\endgroup$
  • 1
    $\begingroup$ When I removed the question from HNQ, I did so because physics.SE should not serve as PR for otherwise undistinguished arXiv preprints, particularly not if they are contentious as the rebuttal, also on arXiv, shows. I'm not even sure this paper clears the "non-mainstream" bar we set elsewhere - it is just an arXiv preprint, with no attached journal publication, and the authors don't seem to be otherwise known as established physicists. It is also, I might add, from 2005, so the "recently published" in the question is at best misinformed and at worst deceptive. $\endgroup$ – ACuriousMind Apr 22 at 21:22
  • $\begingroup$ As for that last bit - it's definitely misinformed rather than deceptive (there's a legitimate issue with how arXiv compiles dates), and it's worth removing (i.e., I've removed it already). I'm not sure how much that affects the issue (the form of the question at the time is still relevant), but it's ultimately only a very minor part of the discussion there. $\endgroup$ – Emilio Pisanty Apr 22 at 21:44
1
$\begingroup$

This initiative appears to have petered out, but this question is an excellent example of a bad HNQ question, for a wide variety of reasons.

The question was characterized, in answers, comments, and chat, as a "John Rennie vs. safesphere" wrestling match, or even a battle of relativity vs. crackpots, when it was really about a much more boring technical point where both users were technically right. For example, we have this appealing exchange in the comments:

I dont get the answer yes or no?

So there are currently 36 votes for Yes, 0 votes for No (because there was no answer daring to argue that point of view, other than an answer, not by safesphere, that got deleted after receiving -11 votes), and 0 votes for “what does ‘bend spacetime’ even mean?”. Yes has won.

This seems to imply that basic facts of general relativity need defending, even though neither JR nor safesphere said anything in contradiction with them, and that the "winner" is decided by random coders that come over from StackOverflow and can't even write a complete sentence. This does not give a good impression to practicing physicists; it tells them that posting here is a waste of time.

Even more unappealing is that, in the process, many non-mainstream commentators did come out of the woodwork, filling the comments with nonsense. About half were regulars here, and half were random coders. At the same time, I got a pile of downvotes, presumably from users who thought that I was siding "against" John Rennie.

The quality of the discussion is just quite low, and brought lower by the HNQ list, and a good indicator of why we have so many fewer practicing physicists than MathOverflow has practicing mathematicians. It shouldn't deserve so much attention, but instead it's on track to being the most viewed question this week.

$\endgroup$
  • 1
    $\begingroup$ I would argue that this answer is missing a set of (proposed) actionable criteria that mods should use to decide that a given question falls in this class and should be removed from the list. $\endgroup$ – Emilio Pisanty May 24 at 21:39
  • $\begingroup$ @EmilioPisanty I wish I knew a good objective-sounding criterion to pick out such questions! $\endgroup$ – knzhou May 24 at 21:45
  • $\begingroup$ Yeah, it's a tricky business. I agree with you that the heightened attention doesn't seem to have done that thread any good, but it'd be better to have a sense as to what it was about the question that sent it careening downhill like that. $\endgroup$ – Emilio Pisanty May 24 at 22:50

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .