Remove Silence on track with multiple takes?

I'm trying to use the remove silence feature but when I use it on tracks that have multiple takes it won't let me use it and says "command not available because there are no audio regions selected". Even though I've got the comp selected. Even when I manually select specific takes in the drop down it won't let me do it.

I know I could export the comp to a new take and remove the silence that way but then it's a pain if I wanna change something to the comp later.

Any way around this? Am I doing something wrong?