It seems obvious--the choice to upvote or downvote a question or answer should be based solely on the content. But it's sometimes difficult to ignore what appear to be unfair tallies. I sometimes have an urge to "cancel" what appears to be a mean downvote on a question or an answer that has no real problem but that ordinarily would not attract a vote either way from me. Beyond the obvious voting bias this introduces, it doesn't really cancel, in terms of reputation points, because they are not assigned symmetrically.

So: is it always wrong to take current tally into account when deciding if/how to vote?