Here's the deal.
You can pound on them to, say, 100-200 watts for short periods of time, on a regular basis, and it won't "blow" anything. The 500 watts is fraction-of-a-second by the way.
However, over time, you can cause melted spots between the voice coil wires, which will, over time, lead to lowering resistance/impedance. A little bit at a time like that can result in a pretty terribly melted voice coil in the end, that didn't come on all at once.
Like I said, depends on how hard you're pounding them & how often. Personally, I'd try to stay under 100 watts if that's the RMS rated value.