Linen is pretty tough stuff, so you should be able to give it all you've got.
But if you can't get back to the white, there's always dye.