8
$\begingroup$

This question is an offshoot of Ben Crowell's question Have we lost the necessary critical mass of professional physicists?, and I'm asking it separately to help avoid multi-threaded comment discussions.

I'm particularly interested in a comment of dmckee:

I don't think that we (that is the mods) should be in the business of closing a lot of questions with our super-powers anymore. There are enough 3k users (81) that they should be able to carry the bulk of the load. My impression, however, is that a relatively small fraction of the eligible users apply their power with any regularity.

As I said in that thread, I think this is a specific enough question that we can find the relevant statistics to give an answer to it. I must add that I agree with dmckee's impression.

So, the specific question: Is it possible to confirm this statement from publicly available data, hopefully through the data explorer? If so, what, exactly, are the relevant statistics to ask for?

To kick off the discussion, and to answer Manishearth's comment, I think the relevant tables to look at are not the per-question close-votes, but the publicly available per-user log of the review activity.


Edit:

OK, so it looks we may need to ask the dev team for these statistics. In this case, what, exactly, are the relevant statistics to ask for? I'll make a proposal shortly but please try and think of others.

$\endgroup$
  • $\begingroup$ Note that scraping the activity log won't be easy either (involves hitting the server with tons of AJAX, the data explorer is useless for that). $\endgroup$ – Manishearth Oct 28 '13 at 13:21
  • $\begingroup$ Yeah, I was just looking at that, the data explorer's Users table doesn't seem to have that data available. Is there some specific reason for that? $\endgroup$ – Emilio Pisanty Oct 28 '13 at 13:22
  • 6
    $\begingroup$ 96 more rep and I'll gladly do my share of reviewing! Maybe I need to answer some HW questions. $\endgroup$ – tpg2114 Oct 28 '13 at 15:06
  • $\begingroup$ (a) The data explorer runs on older code and probably hasn't been updated to the review queue age (b) I think they don't want people to harass others over their closing choices too. $\endgroup$ – Manishearth Oct 28 '13 at 15:10
  • 1
    $\begingroup$ @tpg2114 answer 5 more, and then promptly close them :P Now that's what I call a feedback mechanism! $\endgroup$ – Manishearth Oct 28 '13 at 15:11
  • $\begingroup$ @Manishearth I can understand that, but as is the case here the public review data are most useful if they are accessible in bulk. SO has a huge review queue problem; why have analytics on that not surfaced up? $\endgroup$ – Emilio Pisanty Oct 28 '13 at 16:15
  • $\begingroup$ @EmilioPisanty I suggest you ask for it on Meta Stack Exchange. Note that analytics on that are available to the community team (they have a data explorer type thing that can query the full database) $\endgroup$ – Manishearth Oct 28 '13 at 16:19
  • $\begingroup$ Note: Some of this data is available here $\endgroup$ – Manishearth Oct 30 '13 at 19:51
4
$\begingroup$

To test the hypothesis "there is only a small core of users that use the review queues regularly", my first naive attempt would be

From the users with close-vote powers, during month=October 2013, how many users made range_1=0, range_2=[1,5], range_3=[6,20], range_4=[21,50], and range_5=[51,∞] reviews in the review_queue=close_votes?

where the ranges are of course adjustable, and month should then be varied over the past six months or so.

If it is in fact the case that the hypothesis is true, then the data above doesn't get us very far as to why some people don't use the review queues, or how the site might motivate them to.

One possible reason I can think of this is activity on the site (the dependence on which must in any case be substracted from any other possible explanations). I can think of two queries to try and get at this:

What's the average reputation earned during month=October 2013 by the users that made range_1=0, range_2=[1,5], range_3=[6,20], range_4=[21,50], and range_5=[51,∞] reviews in the review_queue=close_votes in that period?

and

How are the reputation earned during month=October 2013 and the number of reviews made in the review_queue=close_votes in that period correlated? Is there a positive correlation? How does a scatter plot look like?

Just to be clear, I don't want to put off users who are simply using the site without going into the review-queue quagmire.* But if we find that there's some specific measures we can do to encourage close-voters to review and to close-vote low-level questions to get them in the queue in the first place, then we should do it.

*Not really! It's not that bad!

$\endgroup$
  • $\begingroup$ A note (I think I've said this before), if ever you feel that the queue is becoming a quagmire, feel free to ping me or another mod in chat, we can pitch in to clear it up. $\endgroup$ – Manishearth Oct 29 '13 at 5:53
  • 1
    $\begingroup$ That's the thing: we shouldn't be needing that any more. $\endgroup$ – Emilio Pisanty Oct 29 '13 at 11:53
  • 2
    $\begingroup$ Of course we shouldn't, but the current situation is not the ideal one. Just letting you know that this is a major option :) $\endgroup$ – Manishearth Oct 29 '13 at 16:56

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .