Too many people aren't aware that you can say no, and not do something if you don't want to or if it is a bad idea. Especially if nobody is paying you.
Even for people that are paying you for a product, sometimes you really need to say no, that is how it works, and no, we are not going to customize it for your particular workflow.
Just saying "no" already can cost too much effort, because it involves making a decision that you'll want to justify. It takes effort to report an issue and even more to make a pull-request, so you don't want to reject them without a good reason, without really understanding it. Except for some obviously sloppy reports that would be just rude.
The easier option is to simply not respond. Even this leaves some level of stress, because we've all had our own bug report or pull request ignored. We don't want to this uncaring person on the other side. But sadly, ignoring other people's requests is the best option for involuntary maintainers to protect themselves.
The final option is to just hand out the keys to whoever is the most active contributor, and stop trying to control the direction of the project. This will get the stress-level down. You may not like this option, but it's much better than burning out.
Even for people that are paying you for a product, sometimes you really need to say no, that is how it works, and no, we are not going to customize it for your particular workflow.