I think the reason for having the permissions for all sites is for when you want to use the instant replay feature on the off chance and you realise that there were no permissions enabled for the specific domain.

I'd be pretty annoyed if the bug is not reproducible and I lose that chance of generating a bug report.