Trying to preserve "substantive discussions" in comments goes against the grain of how Stack Exchange is designed. Stack Exchange is designed specifically to make discussions difficult as part of its core functionality. The Physics Stack Exchange is a Q&A site, not a discussion forum.
If some intriguing point is raised in the discourse of a comment thread, sure, you can pursue the question… but it should be ask as part of another post. Stack Exchange is not about on-going, unbounded, threaded discussion… and certainly not in comments.
The moderators (and community) are acting properly when deleting those comment discussions. But to understand why comments are so transient and expendable, you have to understand the core purpose of Stack Exchange's behavior.
After someone asks a question, members of the community propose answers. Others vote on those answers. Very quickly, the best answers rise to the top. You don’t have to read through a lot of discussion to find the best answer. If an answer can be improved, users can edit the post.
Comments help facilitate that wiki-style editing by allowing us to ask for a bit of clarification or otherwise help improve that post. That's what comments are for. Period.
When users start adding important, useful, and interesting information in the commentary, you break down that structure of "one question, best answer." You have to comb through all the commentary for import addendums and corrections and partial answers and important bits of information spread throughout. That entirely defeats the purpose of having a Stack Exchange site.
There's nothing inherently wrong with communities that that want free, open-ended discussion and back-and-forth debate. If traditional forums are your choice, there are certainly enough of them out there.
But on Stack Exchange, routine cleanup of comments helps enforce the purpose of comments. Comments are there to discuss improvements to the post. When their purpose has been served, they are deleted. When comments drift into conversations, they should be deleted. And, as @David quoted in his post, it's all part of discouraging comments from becoming mini chat systems.
What you are asking simply is not what we do here… and most like it that way.