12
$\begingroup$

There has been some small discussion in the comments of a recent post about Physics SE questions appearing in the list of Hot Network Questions (HNQ). It was also brought up that Physics SE can choose to be removed from this list, but even if this is not a choice to be made, I think a discussion about the pros and cons of Physics SE in the HNQ would still be a good one to have.

From what I can tell, the good side of the HNQ is that it gets more people to view the site. If the HNQ question they have arrived at is a good question with good answers, then they will most likely learn some physics, which is awesome. Additionally, these new viewers might explore other questions on the site. So the HNQ essentially brings in more viewers and users, and helps spread physics knowledge to the community. I think everyone can agree these are positive outcomes.

On the other hand, there are instances where HNQ physics questions are not very good questions, at least by the standards of typical Physics SE posts. The question could be poorly written, or it could be about a "pop-sci" topic that many people think they know a lot about, thus garnering many poor quality answers. This is not necessarily bad for regular users and physics experts on the site (although it can be annoying sometimes), but it could be bad for new users to the site and to physics, as they will have an incorrect view of how Physics SE, or physics in general, works.

In my opinion, I think the good outweighs the bad here. Even if there are some "bad apples", I think more people coming here and learning about physics is always good. Plus, we have measures in place to mitigate the bad side. Users can vote on questions and answers, vote to close questions, and even flag questions to be removed from the HNQ should moderators agree to remove them. Of course these measures might not always be enough, but they are what we have at the moment.

What do other, regular Physics SE users think about the HNQ? What are some other pros and cons, and does the bad really outweigh the good?

Please note that this post is not intended to be a discussion on how the HNQ could be better handled on PSE. This post is just intended to be a discussion on how the HNQ affects the site in its current state, and whether it is a net good or net bad effect.

$\endgroup$
21
  • 8
    $\begingroup$ It's more con than pro for me. (my opinion) $\endgroup$
    – user208739
    Commented Nov 12, 2019 at 17:05
  • $\begingroup$ @AbhasKumarSinha Would you care to explain in an answer to this post? $\endgroup$
    – BioPhysicist
    Commented Nov 12, 2019 at 17:05
  • $\begingroup$ @AbhasKumarSinha Ah ok. Well I am not really talking about personal preferences here as to which questions an individual would want to see or not see. I am discussing more about in the context of the site as a whole. Thanks anyway. $\endgroup$
    – BioPhysicist
    Commented Nov 12, 2019 at 17:18
  • 2
    $\begingroup$ @all: Proposing improvements to the way the software works is not a constructive use of this thread, which is about how the influx of traffic from HNQ affects this site and whether we're happy with the questions that are making the list right now. If you have thoughts on how the software should run, take them to chat or to Meta Stack Exchange ─ just be sure to read the announcement on the latest batch of changes to the feature first, at the very least. $\endgroup$
    – Emilio Pisanty
    Commented Nov 13, 2019 at 16:03
  • $\begingroup$ Is there an easy way to automatically protect HNQs so that occasional users don't disproportionately upvote the question? $\endgroup$
    – ZeroTheHero
    Commented Nov 14, 2019 at 15:48
  • 3
    $\begingroup$ @ZeroTheHero At the moment, protecting a question just prevents lower-rep users from answering it, but doesn't impact voting. Locking a question prevents everything including voting, but applies to everybody. There isn't a tool in the system at the moment that would lock only for lower-rep users. $\endgroup$
    – tpg2114 Mod
    Commented Nov 14, 2019 at 16:04
  • $\begingroup$ @tpg2114 right... Thanks for reminding me of this. $\endgroup$
    – ZeroTheHero
    Commented Nov 14, 2019 at 16:14
  • $\begingroup$ @ZeroTheHero No, but you can help support this proposal (and other related ones in its Linked sidebar) if you think that that's something that still requires changing. $\endgroup$
    – Emilio Pisanty
    Commented Nov 15, 2019 at 13:33
  • $\begingroup$ @EmilioPisanty I did just that. Thanks for the heads up. $\endgroup$
    – ZeroTheHero
    Commented Nov 15, 2019 at 14:13
  • $\begingroup$ Disagree. Arbitrarily driving traffic to the site only increases the noise to signal. It's the same or worse as being pinned, which is why I'm here and you just wasted one minute of your life reading what I just wrote. $\endgroup$
    – Mazura
    Commented Nov 16, 2019 at 1:09
  • 1
    $\begingroup$ Currently there's, Why are the rain clouds darker?.... posted 13h ago from a user who already quit. - Coupled with the fact that you'd need to raise a custom flag to a moderator, instead of an actual flag for it, I don't see how that question or what would have to be manually done to it by a moderator 'helps' the site. $\endgroup$
    – Mazura
    Commented Nov 16, 2019 at 1:23
  • 1
    $\begingroup$ @Mazura It would be helpful to post your thoughts as well as any evidence that HNQ doesn't do any good for the site. $\endgroup$
    – BioPhysicist
    Commented Nov 16, 2019 at 5:56
  • 1
    $\begingroup$ No evidence, only logic. If a Q goes on the list, it gets more attention (which is unideal when predominantly from outside the group of 'experts') then it would have otherwise. "People don't DV enough is all I'm seeing in the data." ... so all that's happening is more noise via UVs from random people on the internet. That skews the votes between all Qs into a binary list of those that have been on, and those that have not been on, the list. $\endgroup$
    – Mazura
    Commented Nov 16, 2019 at 21:35
  • 1
    $\begingroup$ @Mazura Unlike the main site, the meta site is often used for discussions. Your answer does not have to be a post that is correct or incorrect. It just needs to contribute to the discussion. Given the amount of things you have said in the comments, I think an answer putting your comments in one place would be helpful $\endgroup$
    – BioPhysicist
    Commented Nov 16, 2019 at 23:01
  • 1
    $\begingroup$ Can we do better at reducing bike-shedding? "Personally I came to the conclusion that unless this [("All they can do is upvote"]) [HNQ] voting asymmetry is somehow fixed..." versus a comment here : "Proposing improvements to the way the software works is not a constructive use of this thread". - Same problem, same N/A solution. $\endgroup$
    – Mazura
    Commented Nov 22, 2019 at 23:21

2 Answers 2

Reset to default
15
$\begingroup$

I think the main effect by which the HNQ questions harm our site is completely disconnected from their quality: They drown out smaller topics, and they favour questions with non-technical answers.

Their topics are either classical/quantum mechanics at (at most) undergraduate level, questions about everyday life situations or something some pop-science publication wrote about. Additionally, the highest voted answers are usually - and understandably, given that a large share of viewers of these questions are not regular users - those most accessible (and most authoritative-sounding) to non-physicists.

Now, there's nothing inherently wrong with such questions, but it means that people who like to ask and answer questions of this type get disproportionally more upvotes than those who prefer smaller subfields or technical questions. This is in addition to the voting pool for these things being smaller to begin with because the fraction of regular users who are interested in them and can judge correctness there is of course also smaller for smaller subfields.

Now, I can't prove the following assertion, but I believe that many (not all!) experts naturally are of the second kind of users: They want to answer detailed questions about the subfield they're proficient in, not "general physics" questions. The HNQ questions therefore do us a disservice because they have the potential to attract a lot more askers whose physics knowledge is minimal, but they don't really attract (or might even deter) users competent beyond a general physics education.

From personal experience I can certainly say that I find it extremely frustrating when, thanks to HNQ, my second-highest voted answer is a one-liner on basic special relativity, while many answers I poured actual effort into don't even make a double-digit number of votes. As I said, less votes for more specialized posts are to be expected anyway, but HNQ magnifies this effect considerably.

$\endgroup$
15
  • $\begingroup$ Yeah I see that my highest voted answers are the shorter ones that ended up on HNQ, and there are times where I put so much effort into an answer and it gets very minimal attention. But does the HNQ prevent the more specific topics from being looked at? I guess really what needs to be (somehow) looked at is how "self-contained" the HNQ is. Do people who visit and interact with these questions then go move onto the rest of the site, and if so, what effect does that have? I don't think there would be a way to determine this though. $\endgroup$
    – BioPhysicist
    Commented Nov 17, 2019 at 20:58
  • $\begingroup$ "Do people who visit and interact with these questions then go move onto the rest of the site, and if so, what effect does that have? I don't think there would be a way to determine this though" $-$ this can be determined, just not by us. That data isn't public, but SE does have a team of data scientists on staff to do precisely this kind of data crunching (cf. this recent blog post). Whether SE management has put HNQ high up on their priorities list, on the other hand, is a completely different matter... $\endgroup$
    – Emilio Pisanty
    Commented Nov 18, 2019 at 1:14
  • 5
    $\begingroup$ This is presenting a false dichotomy though. It's not like we can choose between having popular questions and having lots of experts. As we saw with the theoretical physics SE and PhysicsOverflow, attracting expert attention is just hard, because experts have a lot to do, and collaborators they can discuss physics with in person. None of the 50+ physicists I know at my university use Physics.SE, and I don't imagine any of them would start if I told them the HNQ was abolished. $\endgroup$
    – knzhou
    Commented Nov 18, 2019 at 5:17
  • 6
    $\begingroup$ I post answers for the enjoyment of helping others, to help clarify my own thinking, and to counter popular incorrect statements. The first two are completely independent of the HNQ, while the third benefits from it. $\endgroup$
    – knzhou
    Commented Nov 18, 2019 at 5:19
  • 4
    $\begingroup$ @knzhou I'm not saying we shouldn't have this kind of questions - I'm saying HNQ rewards them disproportionately beyond what they would get from being "popular" among the regular userbase already $\endgroup$
    – ACuriousMind Mod
    Commented Nov 18, 2019 at 7:03
  • $\begingroup$ @ACuriousMind But is that necessarily bad for the site overall? $\endgroup$
    – BioPhysicist
    Commented Nov 18, 2019 at 13:35
  • $\begingroup$ I don't think HNQ are helpful but I'm not sure if they are harmful. "drown out" is too strong for my own taste but I do regret some HNQ are disproportionately prominent: thankfully there aren't enough terrible HNQ IMO to cause serious harm to the site. @EmilioPisanty if there a way to query the system and obtain the number of upvotes on HNQ before and after the change in policy? i.e. is there a way to adjust your script to see if the new policy somehow reduced the prominence of HNQ (by using the proxy of counting upvotes)? $\endgroup$
    – ZeroTheHero
    Commented Nov 18, 2019 at 16:20
  • $\begingroup$ @ZeroTheHero Almost but not quite. Voting data has timestamps, but in the public SEDE they're rounded down to the date (with no time) due to strong and valid privacy considerations. Given that most HNQs make the list within a day of posting, the public data would be too noisy to be useful. In any case, though, without a good control, I'm not sure what we'd learn from such a query. Obviously the bulk of the upvotes will come after the questions gets advertised, but what do we learn from that? $\endgroup$
    – Emilio Pisanty
    Commented Nov 18, 2019 at 16:46
  • $\begingroup$ (Note also, regarding "to see if the new policy somehow reduced the prominence of HNQ", that this data ─ which questions made it to HNQ and when ─ only started getting logged at all when the policy changed. There are no data available of the 2013-2019 period before the policy change ─ it simply wasn't ever logged. When challenged (cf. the link in my answer), SE devs cited weak "performance" arguments to justify that, though evidently the problems were solvable.) $\endgroup$
    – Emilio Pisanty
    Commented Nov 18, 2019 at 16:48
  • $\begingroup$ @EmilioPisanty I was curious to see if allowing some community intervention to "delist" HNQ broadly reduced the total number of upvotes to these questions, but one would need some timing data which I gather is not available. I have a sense that the truly terrible HNQ are "delisted" reasonably quickly and that the scale of the problem of upvoting bad questions (and associated answers) has been reduced, but this could be confirmation biais. $\endgroup$
    – ZeroTheHero
    Commented Nov 18, 2019 at 16:53
  • 1
    $\begingroup$ Well, this is why the lack of logging pre-HNQcalypse, and the decision to roll out the functional changes without allowing for a few months of logging on the old system, is so frustrating. There's probably a fair amount that can be gotten from the behaviour of questions that get delisted (and if you can sharpen the general question into a specific query, I can write it for you), but none of that can be compared to the pre-change status quo. gah. We complained for years pointing out that this is precisely what would happen, and got nowhere. And now we have data we can't use. Great! $\endgroup$
    – Emilio Pisanty
    Commented Nov 18, 2019 at 17:00
  • 1
    $\begingroup$ @AaronStevens I think it makes the site a bit more "pop-sci" oriented (for lack of a better word) than I'd like it to be, so I see it as "harmful". I frankly just chose "harmful" because your question asked about "help or harm", This isn't such a major harm that I'd necessarily argue we'd need to stop being listed on HNQs or whatever - I just wanted to give a perspective that doesn't rely on hashing out whether or not the questions appearing there are "good" by whatever measure. $\endgroup$
    – ACuriousMind Mod
    Commented Nov 18, 2019 at 17:18
  • $\begingroup$ Yes, I agree. Thank you for your input $\endgroup$
    – BioPhysicist
    Commented Nov 18, 2019 at 17:25
  • 1
    $\begingroup$ 'might even deter users competent beyond a general physics education' - that's bathwater I'm willing to throw out with the baby, +1. $\endgroup$
    – Mazura
    Commented Nov 20, 2019 at 20:08
  • $\begingroup$ my second-highest voted answer is a one-liner on basic special relativity, while many answers I poured actual effort into don't even make a double-digit number of votes Are aware that you'd not have different number of votes on the less popular topics if HNQ were not a thing though? $\endgroup$
    – Tomáš Zato
    Commented Nov 26, 2019 at 12:12
8
$\begingroup$

Whatever our opinion, it should be based on actual data, i.e., on the full set of questions from this site that get promoted on the HNQ sidebar.

Luckily, the update to the HNQ mechanisms from March of this year finally (after years of asking for it) allowed us to track what does and does not get on that sidebar, exposing this aspect of question history both to the question revisions list and timeline as well as to the Data Explorer.

So, to that effect, a couple of months ago I wrote a SEDE query,

Upvotes and downvotes on HNQ questions,

which lists this site's HNQs as well as some statistics on their scores.

So: before you make some categorical statement about things which are Good For The Site, or Bad For The Site, or The End Of Stack Exchange As We Know It, or something, go and have a look and see whether it holds up against the actual data. (Say: if your impression is negative, are you sure that this isn't just because of a "vocal minority" of bad questions that are easy to notice, and that various human biases bring to further prominence than they really should have?)

$\endgroup$
15
  • $\begingroup$ Yes I agree. I think the data is extremely important as well. Thank you $\endgroup$
    – BioPhysicist
    Commented Nov 12, 2019 at 17:40
  • 7
    $\begingroup$ People don't DV enough is all I'm seeing in the data. But that's a problem we've had for a long while (see this, this, etc) and not particular to HNQ $\endgroup$
    – Kyle Kanos
    Commented Nov 13, 2019 at 1:15
  • 1
    $\begingroup$ @KyleKanos And the number of votes on the question does not reflect the quality of answers either. $\endgroup$
    – BioPhysicist
    Commented Nov 13, 2019 at 14:00
  • 1
    $\begingroup$ @KyleKanos I agree that some of these don't get downvoted enough but for my part I find it eventually mentally exhausting to constantly downvote. It's like constantly bitching: it gets to you. $\endgroup$
    – ZeroTheHero
    Commented Nov 13, 2019 at 14:13
  • 1
    $\begingroup$ Bitching is basically all I'm here to do and I can't DV either. Votes are supposed to signal where it is in all the noise; the only currency SE has. Votes from the list devalue that currency. If we wanted answers and votes to be all over the place, then we should've called it Yahoo. If specific sites are for experts and enthusiasts, I expect the people voting on them to have a stake, so to speak, not just a passing interest, which is why at <125 rep, it doesn't count. However that's a trivial amount for an 11yo site... $\endgroup$
    – Mazura
    Commented Nov 16, 2019 at 21:17
  • 1
    $\begingroup$ Top posts divided by their age, versus HNQ posts divided by age, is a ratio of what, and where does that leave standard questions? The problem with that is the subjective opinion of what a 'really good question' is and whether it should have as many votes as a top Q that was never on the list. There's no control group so there's nothing to do with the data. $\endgroup$
    – Mazura
    Commented Nov 16, 2019 at 21:24
  • 7
    $\begingroup$ @Mazura Your two comments here are basically word salad. If you have specific metrics that you're interested in that you're having trouble fishing out of SEDE then I'm happy to help you build a query for it or to write one for you. If you have constructive criticism, then I'm happy to hear it, but if the only thing you're going to post is unconstructive negativity - maybe that's a thing on Home Improvement, but it's not appropriate or constructive here. $\endgroup$
    – Emilio Pisanty
    Commented Nov 16, 2019 at 22:07
  • $\begingroup$ I prefer the term stone soup (admittedly, I've no idea what I'm talking about - and you've admitted not knowing what to do with the data either). "without a good control, I'm not sure what we'd learn from such a query." - "We complained for years pointing out that this is precisely what would happen, and got nowhere. And now we have data we can't use." - I have an extremely specific metric I'd like to know (that we can't, and it's the only one that matters) : what puts a Q on the list? Why would that knowledge need to be kept hidden if the list was a good thing? $\endgroup$
    – Mazura
    Commented Nov 20, 2019 at 20:01
  • $\begingroup$ There is a 'thing' at Home Improvement that's worth mentioning. Basically you get one or two upvotes on answers (or none, which is how I got tenacious) which signals to the community that this checks out, move along. But if it goes to the HNQ, you'll get 10 UVs for linking to wiki and adding a picture of a glass of water showing nucleation points. That doesn't help anyone build a house.... $\endgroup$
    – Mazura
    Commented Nov 20, 2019 at 20:13
  • 1
    $\begingroup$ @Mazura The data is provided as-is, and it is intended as an index with various ways to sort it, and not (by itself) as a source of statistical information, which it obviously cannot provide. (You could have found this out, if it was previously unclear to you, by politely asking what the data was meant to do, instead of blithely assuming otherwise and launching unwarranted attacks.) $\endgroup$
    – Emilio Pisanty
    Commented Nov 22, 2019 at 0:17
  • 2
    $\begingroup$ As for "Why would that knowledge need to be kept hidden if the list was a good thing?", I have no idea why you think the mechanism for selecting questions for the list is secret. The algorithm is perfectly public and has been documented here and here for the past eleven years, with some minor tweaks documented here as soon as they were implemented. $\endgroup$
    – Emilio Pisanty
    Commented Nov 22, 2019 at 0:25
  • $\begingroup$ @KyleKanos People not downvoting enough is particular to HNQ, because many people who visit those questions because they see them in the hot bar cannot downvote them. In fact, I do put effort to get at least a bit of reputation on QAs that I visit in HNQ for this very reason, to be able to downvote bad content that reaches HNQ. $\endgroup$
    – Tomáš Zato
    Commented Nov 26, 2019 at 12:15
  • $\begingroup$ @TomášZato I recognize the disparity in abilities for upvotes vs downvotes on 101 rep users, but I think that even regular users downvote too infrequently. $\endgroup$
    – Kyle Kanos
    Commented Nov 26, 2019 at 12:31
  • $\begingroup$ @EmilioPisanty In my 5 years of using this site, I can safely say that many questions make the HNQ every day. From what I can tell, as long as it has an answer with 4 or more score within 8h (and is not closed), it makes the list. But whether they get views and upvotes is a different matter. Many of my (highest-voted) answers to questions that made the list peter out at 500 views and 5-10 votes. I'd estimate one in ten get to 2k views and above within 24h and one in twenty HNQs stay for the entire 72h and rack up thousands of views and >25 votes. $\endgroup$
    – Vincent Thacker
    Commented May 3 at 12:46
  • $\begingroup$ (cont.) I've also observed a large correlation between the technicality of a question and how "successful" it is on the HNQ. So the conclusion is: focus on the nature of the question rather than whether it is "labelled" as HNQ. $\endgroup$
    – Vincent Thacker
    Commented May 3 at 12:52

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .