9
$\begingroup$

Recently David Z said in chat that

It's good to keep in mind that people are less likely to read a longer answer.

So I was wondering: How true is it that length can work against you in an answer? While we can't directly measure the number of users who do read individual answers, surely there is some data we can dig up.

I'll post my own findings. Everyone else should feel free to contradict me with better statistics (not a hard thing to do).

$\endgroup$
7
  • 1
    $\begingroup$ Of course, one can't necessarily assume that likelihood of being read correlates with vote count. ;-) $\endgroup$
    – David Z
    Jul 24, 2014 at 9:46
  • 1
    $\begingroup$ It would also be interesting to replace in the investigations below the length of the post by some measure for the "amount of LaTex"... $\endgroup$
    – Dilaton
    Jul 24, 2014 at 10:12
  • 1
    $\begingroup$ @Dilaton count '$' characters perhaps? (and also '[', '('... a bit tricky to do). $\endgroup$
    – Kyle Oman
    Jul 24, 2014 at 17:04
  • 1
    $\begingroup$ @Kyle One would use regular expressions in the statement to count the number of characters in sections extracted between $ $ markers and $$ $$ markers. I don't know enough SQL or regex to do it myself though. $\endgroup$
    – tpg2114
    Jul 24, 2014 at 17:11
  • $\begingroup$ Err tried to say '\[' rather than '[' and '\(' rather than '('. Apparently I need to escape my backslashes. $\endgroup$
    – Kyle Oman
    Jul 24, 2014 at 17:14
  • $\begingroup$ @tpg2114 and then there are those like me who regularly use the little-known fact that \begin{align}...\end{align} works in mathjax without $ signs. $\endgroup$
    – user10851
    Jul 24, 2014 at 17:15
  • $\begingroup$ Regex involving backslashes shiver. $\endgroup$
    – Kyle Oman
    Jul 24, 2014 at 17:18

1 Answer 1

Reset to default
12
$\begingroup$

Raw Scores

First, here are some queries asking how questions and answers score as a function of their lengths (in raw input characters, no sophisticated parsing here), with the associated graphs replicated here for convenience:

In both cases the mean score in each bin is shown together with +1-sigma and -1-sigma scores. There might be a slight upward trend for answers, but there is not really any trend to speak of for questions.

Normalized Scores

Perhaps a better measure is (score) / (page views). At least this way we are closer to answering the question "of those who looked at this question/answer, what fraction thought it was good?"

Voters don't seem to care much about a post's length.

Accepted vs. Not Accepted Answers

Could there be a difference in voting patterns as a function of post length between these two populations of answers?

It's a busy plot, but I don't see a trend in either population.

Conclusion

So, at least in this rough analysis, it seems post length really doesn't matter when it comes to votes. Of course, some of my answers I'm most proud of are rather long expositions on rich topics, so I'm not saying we should all write one-sentence answers to maximize votes per unit effort.

Finally, I leave this last graph that has nothing to do with post length. It's not really surprising; it shows that answer score is correlated with (the absolute value of) question score.

$\endgroup$
8
  • $\begingroup$ If anyone spots any egregious error in my SQL that invalidates these queries, please point out my shortcomings. $\endgroup$
    – user10851
    Jul 24, 2014 at 6:58
  • $\begingroup$ I'm curious about your Answer score as a function of length plot, which could be interpreted as saying that there's some 'critical spot' at around 10k chars where answer ratings can be much better or much worse than average. What's the shape of the distribution around that point? Does it "grow a bump upwards"? Or is it simply broader? (It's unlikely, for instance, for answers with score <-10 to last long without being automatically deleted.) $\endgroup$
    – Emilio Pisanty
    Jul 24, 2014 at 10:54
  • $\begingroup$ Chris, this SEDE query is an attempt at doing the quartiles. What are your thoughts? $\endgroup$
    – Emilio Pisanty
    Jul 24, 2014 at 12:33
  • $\begingroup$ How does your answer score as a function of length chart incorporate number of answers of that length? If it simply averages over the number of answers in that bin, that might explain why there's a dramatic increase after 18000; very few answers of that length so the average can easily be offset by one particularly high-voted answer $\endgroup$
    – Jim
    Jul 24, 2014 at 14:04
  • 1
    $\begingroup$ @EmilioPisanty I like the quartiles a lot better than standard deviations for such non-Gaussian data. I also would never have been able to figure out how to write that query. $\endgroup$
    – user10851
    Jul 24, 2014 at 16:20
  • 1
    $\begingroup$ @ChrisWhite Yeah, it was a lot harder than I thought it was going to be. (Usually is, too.) And yeah, the data is not gaussian at all. $\endgroup$
    – Emilio Pisanty
    Jul 24, 2014 at 16:26
  • $\begingroup$ It's also notable that the 'big-hit' answers (i.e. the top of the fourth quartile) is very high for shorter answers. $\endgroup$
    – Emilio Pisanty
    Jul 24, 2014 at 16:30
  • $\begingroup$ +1. Using the standard equation "1 picture = 1000 words", this is a 6343 word long answer, and it's a pretty good answer. $\endgroup$
    – David Hammen
    Aug 4, 2014 at 20:28

You must log in to answer this question.