That is exactly why I cancelled my CC when I cancelled earthlink. Let me tell you, when you switch from them to another provider, they give you the 'you have a 30 day remorse period, you know' salespitch.
When you mention that you're getting a much better price (as I am paying $20 a month less for the same bandwidth), they start offering free time and lower prices for less features (email address's, etc).
I've also read of horror stories like yours, where they don't stop billing until 4 months later, and refuse to refund the money. Just google "Cancel earthlink".
The only advice I can give you (or anyone) when cancelling earthlink is document EVERYTHING, and make sure they know that you want it cancelled, and are keeping track. That seems to be the best way to FORCE them to actually cancel. They rely on people not looking back.
Write down the incident number, and check back within 10 days of the actual end date. Verify that it is indeed being shut off, and that you are not going to be billed.
If you do recieve an invoice, COMPLAIN RIGHT AWAY, do not ignore it.
Use a card that you don't mind killing. Kill it the instant they finalize the last payment that you want authorized.
And lastly; DON'T LOOK BACK! No matter what they offer you with, keep a firm stance, and brace yourself for numerous cross sells. They will all but pay for a call girl to keep you on their service.