The secret to getting rewarded for high-quality answers is ___

Question

This is a follow-up post on John Rennie's topic The secret to getting a massive reputation is ___, and my answer therein.

I think many of us have at some point been dispirited, when an answer on an advanced topic that took a lot of effort to write received very few upvotes; and answers on basic subjects that took much less effort are upvoted far more.

So I've been wondering if it is possible to implement an additional reputation score that rewards high-quality answers more than basic answers. I have such a proposal, and I'm interested in what everyone thinks about it.

The idea is as follows: the standard reputation system treats every vote the same: each vote counts as 10 points. That's all very nice and democratic, but it doesn't reflect the quality of an answer; the score depends heavily on the popularity of the question. But perhaps we can also give answers a weighted vote, such that the score you get not only depends on how many votes you received, but also on who has given the upvotes. In particular, a vote from a high-rep user will give you a higher score than a vote from a low-rep user. Now, I'm not in favour of having this 'quality score' displayed on every individual answer, because that might influence other voters. Instead, I'd prefer for each user to have an 'overall quality score', calculated from all their received votes combined, updated once a day, and displayed next to their total reputation.

The rationale behind this is that the high-rep users are knowledgeable on a broad range of subjects, including specialized topics. So they will likely also read and vote on these more advanced subjects. Therefore, while these topics attract less votes, those votes are more likely to come from high-rep users (though not always; there are also specialists who have not built up a high reputation. No system is perfect). On the other hand, more basic questions will attract more casual users with lower reputations. Also, high-rep users have more experience with the site, and will more often recognize and appreciate a high-quality answer.

The next problem is then to decide how the votes should be weighted. There are of course a lot of possible ways, so I'll just give a simple example. Consider the following distribution: The horizontal axis has all the users who have voted at least once, sorted according to their reputation, from highest to lowest. The vertical axis has the total amount of upvotes given by each user. Then divide this distribution into 9 areas, representing 9 user groups, such that the total amount of votes that each group has given is roughly the same (some rounding off is needed so that each user falls into one group). The upvotes of the group of highest-rep users are worth a score of 18; the votes of users of group 2 are worth a score of 16; the votes of users in groups 3 to 9 are worth 14, 12, 10, 8, 6, 4, and 2, respectively. In this manner, the average score of all the votes of the entire site will be 10 (rounded off).

In other words, if you get an upvote from someone in group 1 (the top users), you get 18 points. If you get an upvote from someone in group 9 (the lowest-rep users), you get 2 points, etc.

One can refine this system further:

• Assign an additional quality score to a received bounty;
• Assign a higher quality score to votes from people who have posted an answer to the same question. After all, a vote from a 'competitor' is an acknowledgement that your answer is of high quality;
• Downvotes can also be taken into account.

Would there be any interest in this feature? Is it worth doing? Is it actually doable? Do you have alternative ideas? I look forward to your feedback.

Answer 1

The rationale behind this is that the high-rep users are knowledgeable on a broad range of subjects, including specialized topics.

This is where I think the argument breaks. High reputation comes from experience using the site, not necessarily knowledge about the subject area, so I wouldn't expect that high-rep users are more qualified to judge the correctness of an answer than low- or moderate-rep users. They may be more qualified than the average new user (someone who has posted perhaps one or two times, if that), but most of that group probably doesn't vote anyway so I don't think this proposal would make a difference in that respect.

If you want to weight votes by the expertise of the person casting them, I think the statistic that would make sense to look at is the number of highly upvoted answers, and to some extent the level to which they are upvoted, on questions that share a tag with the question being answered. That way people who have demonstrated expertise in the topic(s) of the question get to cast the most influential votes. But that would be a complicated calculation, and I don't know if SE's servers could handle it in real time or near real time, and in any case who knows if it would really do any better. It'd be an interesting quantity to compute offline, but you can't do it without access to individual vote data (which is not released).

Incidentally, what you're proposing is basically applying PageRank to answers. Google's history shows that it's not difficult for users to adjust their voting and posting behavior to game this system, and that leads to a whole new set of problems which become considerably more complex to deal with.

Answer 2

I think many of us have at some point been dispirited, when an answer on an advanced topic that took a lot of effort to write received very few upvotes; and answers on basic subjects that took much less effort are upvoted far more.

To be honest, I stopped reading right there and perhaps you'll stop reading right here - or maybe not.

Look, when you focus your mind and put in the effort to write an answer on an advanced topic that you're proud of, isn't that the most valuable reward?

It is my opinion that there are actually relatively few here whose upvotes are meaningful in any significant way.

And, when you write an answer on an advanced topic, relatively few will be able to judge if it is a helpful answer or not.

So, if you think carefully about this, you should expect that advanced topic answers, no matter how wonderful, will receive less upvotes than 'off-the-cuff' answers to basic questions that happen to get a lot of page views and upvotes from those that could not begin to appreciate the advanced topics.

Thus, one should not be dispirited by this perfectly understandable state of affairs.

Answer 3

I think a great answer (note I'm not saying the great answer since there can be more than one) to a question has a few qualities.

It's distinct, in the sense either it goes about answering the question in a novel way other answers don't, or it just outperforms other people's explanations using the same methodologies.

It's accessible, if many people can look at your answer and pull something of value from it then it's a useful answer.

It's organized, and people can extract the information they need in an organized manner without much extra effort.

It's hard to ignore, it's got some kind of eye-catching flair to it that draws in the reader.

and most importantly

It enriches the conversation, you may even ending up deviating some from the actual question but for the sake of enriching the conversation, providing greater context, or providing a new insight that the question would benefit from.

At least that's my five cents...

Theres always a constant tug of war between these angles but ultimately popular questions will usually have some kind of ratio of these different ingredients that makes them appealing to large numbers of people.