When I set a bounty, I lose my reputation to someone else.
If I lose my reputation because of a bounty, will it be the same as losing reputation because of downvotes, bad questions etc.
I mean will I lose privileges and badges as well?
From the Physics SE help page on bounties:
If your new reputation brings you below the requirement for any privileges, you will lose access to those privileges.
Badges are not linked to your rep and won't be lost just because your rep decreases when you set a bounty.