Analyzing "Can't"

I have had, as we all have, many experiences where I'm told I "can't do that."

"Can't" seems like the kind of word we should use when it's a physical impossibility, or it really is outside of our capacity.

You have permission to tell me that I can't if I say:
A) I'm going to fly off a tall building
B) I'm going to declare all world conflicts will stop at midnight.

For everything else, "can't" is a pretty strong word. I think it would be more helpful if those saying "can't" would say "Well, you can, but there's some pretty significant implications. Let's go through them."

It seems to me if organizations started reserving "can't" for legitimate non-starters, we'd get a lot better at examining conventions.