I spin up a whole linux VM with a passed through nvidia GPU for these and I still spend the majority of the time fighting the python and figuring out the missing steps in the setup instructions.

Glad for this thread though since it looks like there's some tricks I haven't tried, plus since it seems a lot of other people have similar issues I feel less dumb.