I agree with you. There are many good (or even great) questions which are for various reasons off-topic as per current site policy or community guidance. I feel$^\dagger$ that while answering them may be satisfying, it sends the message that asking more questions of that type is to be encouraged.
If a question is very good but not a good fit for the site, I think the OP should be encouraged to reshape the question if at all possible. This is one of the few things that comments are actively for in this community, but I rarely see proposed content edits in comment sections to off-topic posts. If the question cannot be made on-topic through edits, then no answers should be posted and it should be closed. Providing answers to off-topic questions adds a wink and a nudge to the question closure which I would personally prefer to avoid.
If I were desperate to answer a question I believe to be fundamentally off-topic, I would create a chat to start a discussion. Unfortunately I find the chat feature on SE to be abysmal, which I would guess is a barrier toward getting this to be a more widely-adopted policy.
There are of course reasonable people who may disagree as to whether a question is on- or off-topic, so I don't downvote good answers to questions I think should be closed. However, I do think that the behavior you describe is contradictory, and I think a vote-to-close should preclude a user from then answering the question anyway and vice-versa.
$^\dagger$That begin said, I must plead guilty to occasionally answering questions which probably should be closed - in my case, typically because they should probably be migrated to the Math SE. I can only ask the forgiveness of the community for my occasional lapses in judgement, and that they accept this friendly emoji as a token of my regret :)