1
$\begingroup$

TL;DR if you don't want background information: Should we allow answers which only provide a single experimental measurement to remain posted as answers?

The other day, an old question What is the temperature of the clear night sky from the surface of Earth?, was bumped by an answer which contained a single measurement, and minimal information about the controls or procedure used to take the measurement.

This is the second such answer on the question, the first of which was posted in 2014, and I believe was +2 before the question was bumped by the new answer. This would explain why a second user found it appropriate to post a similar answer with their own measurements.

I assumed this was inherently off topic, but Kyle Kanos mentioned in Chat that his review got quickly declined by users in the review queue. I decided to give it a moderator flag, because I felt such answers blatantly did not fit on the site. My flag read for each answer read:

"Single point experimental data with minimal details of setup or reasoning for results doesn't meet quality standards for answers on this site."

which were declined for the reason:

"flags should not be used to indicate technical inaccuracies, or an altogether wrong answer"

I thought my flag was quite clear that I was not trying to point out a technical inaccuracy, or an incorrect answer, but it seems that is not the case. Because of this, I would like to get some sort of community consensus on this going forward.

I will be posting my own answer, because I have thoughts on how this should be, and shoehorning them into my question more than I already have doesn't seem fair.

$\endgroup$
5
  • $\begingroup$ I don't understand why people are so worked up about that thread. The OP explicitly solicits such answers: "Anyone ever pointed a pyrometer or similar at the night sky?". You can argue that this is awful questioning, take advantage of the absentee OP, change the question, and then go on to argue that the answers don't answer the question, but as they stand they're perfectly reasonable responses to what was asked. $\endgroup$
    – Emilio Pisanty
    Apr 26, 2019 at 16:44
  • $\begingroup$ I don't think that's a good thing though. I see questions that request single points of data to be an issue as well. That's why I thought it would be nice to clear up with the community how we can deal with single points of data as answers. Leaving the answers around seems like non-scientific clutter to me, nothing beneficial to this site's purpose. It seems the current policy (based on declined flags) is to leave answers like that around. If that's not the policy, then obviously a question asking for such answers should also change. $\endgroup$
    – JMac
    Apr 26, 2019 at 16:54
  • $\begingroup$ For background, the review tickets for the two answers in question are here and here. The 'current policy' that you're referring to (the fact that the flags were declined) comes down to two reviews, each by two users. $\endgroup$
    – Emilio Pisanty
    Apr 26, 2019 at 17:00
  • $\begingroup$ @EmilioPisanty I also had the mod flags declined, as mentioned in this question. That is what prompted me to clarify the policy in meta, since I did everything else I thought I should. $\endgroup$
    – JMac
    Apr 26, 2019 at 17:02
  • 1
    $\begingroup$ Gotcha. Those moderator flags do end up requiring a moderator to make a technical judgement call about the quality of an answer, and I for one don't think that that's the type of thing that should be delegated to that layer. $\endgroup$
    – Emilio Pisanty
    Apr 26, 2019 at 17:04

2 Answers 2

Reset to default
3
$\begingroup$

I strongly disagree with the existing answer.

You seem to be worried about people just going out and doing measurements themselves without adequate formal quality controls. If we applied the same standards to theoretical physics, the entire site would grind to a halt. Often when answering a question, I do a small calculation or derivation myself. Other people present numeric computations or derivations by Mathematica. None of these come with formal "controls" or "procedures". It would be ridiculously limiting if every one of these answers had to come with textbook references for every result and be checked to the standards of a published paper.

We already have a perfectly good quality control system for both "uncontrolled" derivations and "uncontrolled" measurements: if you don't think the answer is good, then downvote it. If we deleted all answers with measurements, we would lose some of the greatest answers on this site (such as the top three answers of all time: 1, 2, 3). Moreover, it would send the message that even everyday physics is something dangerous and inaccessible, beyond the purview of all but certified professionals, which is quite counterproductive.

$\endgroup$
5
  • 2
    $\begingroup$ I think you misunderstood my concerns here. All of your examples detail their experimental setup well, and provide more than one piece of data (besides maybe #3, which gives extensive conceptual support for the results). I don't see how not accepting single measurements from each user as an answer would grind theoretical physics to a halt. The point is that these experiments have a single sample and don't detail enough controls to reproduce experiments. That's not conceptual, and it's not really experimental either. It's just measuring something once. I don't consider that physics. $\endgroup$
    – JMac
    Apr 26, 2019 at 12:46
  • 2
    $\begingroup$ @JMad I simply don’t see how a single person’s sloppy measurement is any worse than a single person’s sloppy derivation. In the latter case we downvote, so why crack down harder on measurements? Honestly, if I had to distinguish them I would go easier on measurements, because talk is cheap. $\endgroup$
    – knzhou
    Apr 26, 2019 at 12:53
  • 2
    $\begingroup$ Because derivations are useful in general as answers, and with an incorrect derivation, you can at least point out why it is incorrect, which may have value. Single points of data aren't even wrong, they just aren't beneficial to science without more data or at least analysis of their own data. I don't see how leaving them around to encourage more of it has any value to this site. $\endgroup$
    – JMac
    Apr 26, 2019 at 13:04
  • $\begingroup$ IMO, two of those three answers you posit as "greatest" are among the worst. Just because it's highly voted does not mean it's good, it just happened to hit HNQ because it was low-level physics that 90+% of SE network users were capable of understanding (and in one case, a link provided by xkcd author led to even more people voting). There are plenty of <5 vote answers that are significantly better than any of those three. $\endgroup$
    – Kyle Kanos
    Apr 26, 2019 at 13:05
  • 1
    $\begingroup$ @KyleKanos I think it's a bit more than that. Doing a structured experiment to answer a specific question at least shows a scientific approach. Generally, most people here just answer conceptually (which is great), but experimental data has a novelty factor to it. Novel answers are usually well received, across the whole network. $\endgroup$
    – JMac
    Apr 26, 2019 at 13:14
1
$\begingroup$

In my opinion, these answers do not belong on the site.

The primary issue that I see is that many questions on this site could be answered by taking a single measurement and posting the results. It is my opinion that such answers based around a single measurement are inherently low quality. Leaving those answers on questions seems likely to encourage others to answer the same question, or new questions, using this method.

Where it is so easy to take (bad/uncontrolled) measurements for some variables, I think by not deleting such answers we would be allowing for extra clutter that has no benefit. A single piece of data with limited experimental controls is not a good representation of what physics actually is, and doesn't belong as an answer on a Physics Q&A site.

$\endgroup$
5
  • $\begingroup$ By looking sometimes at the review queue, I suspect that this site suffers from overflagging of answers. If an answer is bad, like the one you linked, downvote, comment if you wish and that's it. See also these guidelines on the main meta. $\endgroup$
    – Massimo Ortolano
    Apr 25, 2019 at 20:00
  • 1
    $\begingroup$ @MassimoOrtolano My issue isn't that the answer is just bad; it's that anyone could post such answers with minimal effort, and minimal backup. I don't see the benefit in allowing everyone to post their singular result from a home-experiment as an answer. It would lead to a lot of bad answers, and doesn't give us anything beneficial. $\endgroup$
    – JMac
    Apr 25, 2019 at 20:06
  • 1
    $\begingroup$ Extending the above comment, such answers don't explain anything. They just report a value, which is not particularly useful (due to weak knowledge of controls, set up, etc + highly localized to a single location). $\endgroup$
    – Kyle Kanos
    Apr 25, 2019 at 21:51
  • 1
    $\begingroup$ Protecting the question is one of the solutions (which Kyle Kanos has done). However, clarifying the question by explicitly mentioning that "single, uncontrolled measurement is discouraged" might be another solution... $\endgroup$
    – Andrew T.
    Apr 26, 2019 at 3:03
  • $\begingroup$ @AndrewT In my opinion it should apply to all answers, not just that specific question. We shouldn't have to protect questions to prevent people from posting single points of questionable data. $\endgroup$
    – JMac
    Apr 26, 2019 at 11:30

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .