What exactly is the point of having badges? Do we get any special privileges with certain badges? Or is it a feature that may be used in the future?
No real point; there isn't any rep or anything. It's just something to strive for. You see all those badges and think "I should get some more". Getting a badge on asking/answering a question (which is the first thing you do when you join) is a way to introduce the concept of badges. A new user sees that he got a badge, and investigates. He feels like getting more badges, so he works towards some of them. It benefits the community.
Badges are used by many online communities. It's a way of setting goals for the users; and helps retention.
See How do "badges" work? as well.