As a security professional as well as developer, I agree with the OP. Simply put, you're going to screw things up no matter what you do. Be smart about it, plan for the worst, and seek outside feedback into your security processes; you're never going to find all the edge cases, even if that's your day job.