Added in profile to enable shims for SNAPSHOT releases

[revans2]

This fixes #602

I didn't update any of the Jenkins files because they either already use a profile for databricks that would disable the snapshot-shims profile or they have it enabled by default. I could update the premerge to try and have this off by default, but I would need to add a second one that has it enabled too, so we know if we are missing something.

[revans2]

build

[tgravescs]

so to clarify, this is active by default but if you specify another profile it disables this one? I would think we want this profile specified for everything except the release build.

[jlowe]

so to clarify, this is active by default but if you specify another profile it disables this one?

Yes, specifying any other profile explicitly on the command-line will disable the snapshot-shims profile.

I would think we want this profile specified for everything except the release build.

The problem with that setup is we would never be regularly testing what we're planning to ship.

From a cursory glance, it appears the premerge CI is specifying some profiles which would turn this off (mirror-apache-to-urm, include-databricks). Rather than the implicit behavior, we may want to make an explicit profile, either to include or exclude, so we don't get some surprising behaviors when someone activates a separate profile that has nothing to do with shims.

Also how would this interact with the spark*tests profiles? If someone specifies the spark301tests profile, will it turn off the snapshot shims and then fail the tests against 3.0.1-SNAPSHOT because it cannot find the shim?

[revans2]

I agree that we should probably be specific about all of the profiles for the builds.

I missed the $MVN_URM_MIRROR having a profile in it too, so I'll try to make it clearer what is happening there.

Also how would this interact with the spark*tests profiles?

From my experience with maven there is no way to have one profile activate/deactivate another profile. We could, however, have a property that activates multiple profiles. So they would both be independent of each other.

[jlowe]

We could, however, have a property that activates multiple profiles

+1 for this. That way we can ask for spark versions to test against with property settings, and those will enable both the test profile for a spark version along with the shim if that shim version happens to be a snapshot.

[revans2]

I would have to do some testing to see if we can make it work mixing properties and explicit setting of profiles.

[revans2]

build

[jlowe]

Looks good to me, would be good to hear from @tgravescs.

[tgravescs]

this will deploy the nightly version without the snapshot shims to urm. Which means devs and integration tests can't use it. Our integration test builds rely on this now.

[tgravescs]

if we want to keep integration testing nightly what we actually push, then we would need 2 separate jars - one with shims and one without, otherwise we aren't integration testing the shims which I think is a bad idea.

otherwise we push the version with all the shims and then just test the one without shims differently. This doesn't seem high risk to me to have to test it on every premerge, I would rather just test nightly or something before a release.

[revans2]

build

[revans2]

@tgravescs I addressed your comments on the nightly deploy.