ValueError: n_components=1000 must be between 0 and min(n_samples, n_features)=100 with svd_solver='full'

[krassowski]

For a dataset with n_features=1001 and n_samples=999 we will get:

ValueError: n_components=1000 must be between 0 and min(n_samples, n_features)=999 with svd_solver='full'

The default value of preprocess_with_pca_dim causes problems when trying to transform_fit() as:

1. it cannot be changed (unless manually in fit)
2. it uses hardcoded 1000 (but when n < p, then SVD will not work), so maybe it could use min(1000, n_samples)

[krassowski]

And a very interesting idea, btw - thank you for sharing code here!