A tip for anyone who suffers with the slow training times of the sklearn logistic regression: you can write it with skorch in no time and get _much_ faster training times.
I wonder if sklearn will have a pytorch backend one day
GPyTorch also absolutely crushes the Scikit implementation for Gaussian processes in my experience. Scikit is a treasure, but maybe not my first choice for performance.
Consider also GPU accelerating the whole thing if you have a GPU around. cuML matches the sklearn API https://github.com/rapidsai/cuml/. Pays off very quickly if you have large datasets.
Btw., going on a tangent, you might like Hummingbird (https://github.com/microsoft/hummingbird). It allows you trained scikit-learn tree-based models to PyTorch. I watched the SciPy talk last year, and it's a super smart & elegant idea.
I wonder if sklearn will have a pytorch backend one day