I somewhat support being unscientific in this case.
It's not as though food dyes are being used to make healthy foods more palatable. It's quite the opposite, that food colorings are used to make processed foods more stimulating, which in turn makes non-processed foods less attractive.
I support this in the same way that I'd support an outright ban on processed food advertising (especially to children). There is a health crisis in the US (and much of the western world), with one factor being the prevalence and poor nutrition provided by processed foods.
"Some consumer advocacy groups argue the dyes aren’t worth the potential risk because they lack nutritional value."
It's the precautionary principle in action. Is it scientific? Not really. But does it make rational sense? Possibly.