Will my nipples ever be able to touch fabric again?

I'm 5 weeks pp and EBF. We've had latch issues on and off here and there but we are definitely on the upswing now. However my nipples are incredibly sensitive and all I can wear are silverettes or milk collectors--and the moment my nipples touch fabric my teeth clench. Please tell me it gets better!!

Update: I'm 8 weeks pp and everything is SO MUCH BETTER. If you're experiencing this there is light at the end of the tunnel!