Yesterday I had 1693 rep, but when I logged on today I had only 1689 left. I went in to see if I'd gotten any downvotes. There weren't any, but I did have a "post was removed" notice - I ticked the "show removed posts" box at the bottom of the page because I've come across this before (this and this meta question helped me figure things out then, I've already done the rep recalculation trigger).
However, that only accounts for a loss of 2 rep points and I seem to have lost 4. Is there anything else that can explain this?
Note: the amount of rep we're talking about is ludicrously small. So it's not the rep itself that bothers me, but the fact that I have no idea where it went.