add JULIA_CONDAPKG_ENV for centralized environment

[t-bltg]

Possible implementation for #52.

Fix #52.

Tests passing locally.

• needs to be documented, if agreed for.

[cjdoris]

Looks good so far, thanks!

On my phone right now so can't review properly but the environment written to needs to be included in the metadata, and the skip resolve check should check if the environment has changed.

[codecov]

Codecov Report

Merging #53 (9943c8f) into main (75ef6ed) will decrease coverage by 0.06%.
The diff coverage is 90.24%.

@@            Coverage Diff             @@
##             main      #53      +/-   ##
==========================================
- Coverage   87.66%   87.60%   -0.06%     
==========================================
  Files           7        7              
  Lines         900      912      +12     
==========================================
+ Hits          789      799      +10     
- Misses        111      113       +2     

Impacted Files	Coverage Δ
src/resolve.jl	92.47% <88.57%> (-0.59%)	⬇️
src/deps.jl	74.51% <100.00%> (ø)
src/env.jl	98.21% <100.00%> (+1.66%)	⬆️
src/meta.jl	90.90% <100.00%> (+0.10%)	⬆️

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

[t-bltg]

Thanks,

the environment written to needs to be included in the metadata

Does that imply adding a conda_env field in the struct Meta, and bumping META_VERSION ?

[t-bltg]

@cjdoris, can you remove the workflows approval please ?
It's cumbersome and wasting everybody's time.

[cjdoris]

Thanks,

the environment written to needs to be included in the metadata

Does that imply adding a conda_env field in the struct Meta, and bumping META_VERSION ?

Yes that's right. There should be a special value (the empty string?) for the default project location, in case the user moves the project.

I'll take a closer look at this in the new year, thank you.

[cjdoris]

@cjdoris, can you remove the workflows approval please ? It's cumbersome and wasting everybody's time.

Uh how? I only get a button to run the workflows (which I've just done).

[t-bltg]

I'll take a closer look at this in the new year, thank you.

Thanks, have some nice afk holidays !

Uh how? I only get a button to run the workflows (which I've just done).

Settings (the wheel) -> Actions (left side bar) -> General -> Change "Require approval for first-time contributors" to "Require approval for first-time contributors who are new to Github".

[cjdoris]

As you've gathered from Discourse I haven't forgotten about this PR! I'm just struggling to find enough laptop time to take a proper look.

Any thoughts on those CI failures?

One observation is you need to reset STATE.conda_env in test/setup.jl.

[t-bltg]

One observation is you need to reset STATE.conda_env in test/setup.jl.

Thanks, I've changed that.

Can you apply the changes suggested in #53 (comment) ?

CI is waiting again for approval: I can't thus move forward on this PR. Let's not waste time.

[cjdoris]

Ok I've changed that setting thanks.

[cjdoris]

The code looks great now, thanks for your effort on this.

One thing that concerns me is that it's possible to end up with a Conda environment that is not consistent with your CondaPkg.toml. For example, in one project I specify somepackage==1.0.0, which is installed, then in another I specify somepackage==2.0.0, which is installed, then when I switch back to the first project, CondaPkg does not resolve again, so is still on 2.0.0. This is because the CondaPkg metadata is still stored per-project.

Some solutions to this:

1. Just document it as a current drawback of using a shared environment, leaving it up to the user to re-resolve when they change project (or ignore the problem, since dependencies are probably pretty loose anyway).
2. Don't allow resolve() to bail out early when in a shared environment. This will slow down time-to-first-resolve even when the environment is already OK.
3. Add a metadata file to the shared Conda environment specifying which Julia projects have installed stuff into it. Then CondaPkg can take dependencies from all of these projects. This could be a fair bit of work to implement.
4. Add a metadata file to the shared Conda environment specifying which Julia project was the last to resolve there. Then if you start Julia again in the same project, it can skip resolving. Downside is that it will do a full resolve whenever you swap projects.

The third option I think is the only one which would reliably always have the right dependencies, without any needless resolving.

I'd be happy to leave this as future work and merge this PR as-is, maybe adding a note to the documentation as in option 1.

[t-bltg]

Yes I first took the approach of writing a metadata file in the shared env (without considering one in a per-julia-version .CondaPkg directory), but resumed to an orthogonal direction after the first review in #52 (comment).

So I think I'll probably document 1., and leave this enhancement for another PR.

Also, for my usage, having different specified conda packages versions never came up (kind of orthogonal to having a shared environment).

[t-bltg]

As a side note, the tests are hard to follow in CI, and IMO it would be useful to print the test name to split the CondaPkg / MicroMamba stdout output into per-test sections (left for another PR, because of the non-standard TestItemRunner usage).

[t-bltg]

Hum, I fail to understand why windows CI spuriously fails.

[t-bltg]

I'm pretty sure we hit an incorrect fast path, either

https://github.com/cjdoris/CondaPkg.jl/blob/4ecd7a2d16c3da1f95966c5a5570a026023dd700/src/resolve.jl#L383-L386
or
https://github.com/cjdoris/CondaPkg.jl/blob/4ecd7a2d16c3da1f95966c5a5570a026023dd700/src/resolve.jl#L415-L419

, causing sporadic / random failures on windows only.

[cjdoris]

This is great, thanks for all your work!