What the site will do if a user with little/no knowledge of physics down-votes me when I answer his/her question correctly, but that user assumes that my answer is not helpful-answer to the question posed by him/her?
The site won't do anything. Generally speaking, if the user assumes your answer is not helpful, then it's not helpful to them, and they have every right (perhaps even a responsibility) to downvote it. Note that being correct is not the same as being helpful, although I would hope the two are usually related.
If you write answers that are helpful to many more people than not, you will get more upvotes than downvotes over time.