that sounds bizarre. why would they order them to do that? what if they re trying to block spammers or sth. What about pages that users want public, but not indexable, e.g. dropbox shared links