Use condaforge/linux-anvil container (CentOS 6, same as used by conda-forge) #2274
Conversation
|
@johanneskoester maybe we can use this branch: #2136 We could just rerun the other test PR. |
…enable conda-forge for now (let's do things in little steps).
|
Absolutely, good thought! I'll do that once this branch builds successfully. |
|
Johannes and all; |
|
@chapmanb it's not just |
|
@chapmanb I really hear you. And I would very much like to keep CentOS 5. The thing is that even Continuum switches to CentOS 6, since they will be using the conda-forge setup (or even do this already). So staying with CentOS 5 means that we would need to start having our own copies of r- and anaconda-channel packages soon. The gcc package seems to be only the first example of a package from the anaconda channel that does not work any more on CentOS 5. If there is a way to provide this support, I would be happy to cancel the move. But that would mean that somebody else has to make the effort, because I can't afford the time to support all the duplicate packages then. Also, CentOS 5 will receive updates only until March 2017. So any serious admin will consider an upgrade in the next months anyway. |
|
Normally I'd advocate maintaining compatibility over upgrading, but with the Continuum gcc build causing issues (we must not be alone...), moving to CentOS 6 is not unreasonable. CentOS 5 is long in the teeth. For perspective, Windows Vista was released in the same year as CentOS 5, and mainstream support by Microsoft ended in 2012 (extended support, mainly for large corporate customers, ends next year). Clusters running CentOS 5 are also likely running dated hardware, leaving users who care about performance moving to newer systems or cloud compute like AWS. Then there's the issue of CentOS 5 being EOL in 2017. As an alternative to upgrading the build environment, could we fork the Continuum gcc recipe and get it to work in CentOS 5 for our channel? As Johannes mentions, that could lead to maintaining many parallel recipes. :( My concern with moving to use the condaforge container is what it means for the future direction of bioconda. Will we seek greater integration with condaforge proper? (I haven't looked into their system very much.) If so, would that be a win for users? What about developers? If we will join condaforge in some capacity, will the ease of adding and updating recipes be diminished? How much of an effort would it be to join them? Do we have a strategic plan? |
|
@tomkinsc just my opinion regarding conda-forge. I think we only gain benefits in depending on the conda-forge channel. bioconda will still exists, but specific to bioinformatics and continue to be very easy to contribute. All libraries that are some kine of |
|
@johanneskoester I have seen this perl error before. This is I think a conda bug. Somewhere perl, is hardcoded and our perl-threaded is not taken into account. |
|
@tomkinsc my feeling is that conda-forge gradually becomes something like a staging area for the anaconda channel. Continuum's strong support for conda-forge might even let them activate it as default channel at some point. For them, this would be reasonable, because they don't have to support the recipes in there on their own. Bioconda will always be separate of conda-forge, because it is good to have full control over the recipes that really matter for us (the bioinformatics specific). With channel priorities, we can always choose to overwrite certain packages if we disagree with conda-forge at some point. This gives us the best of both worlds: we stay a small, flexible and focused community and can delegate some of the maintenance work for general purpose packages to conda-forge. |
|
Another possibility, in the very distant future, is that we eventually use their channel and build system, but Bioconda becomes something like a subchannel that owns the bioinformatics related recipes. I'll leave it to others to worry about that. Obviously for the near future (~10 months) it totally makes sense to proceed with separate channels but some shared infrastructure. |
|
Another long-term consideration might be exploring their permissions / merging model, which might be useful for assigning ownership to sub-groups of recipes. |
|
Sure, this can be reconsidered at some point, although I don't see the immediate benefits of an even tighter integration. Our build system has its advantages as well, and the conda-forge approach has yet to prove that it is superior. At the moment, let's just try to use the same container. |
|
One last question: how does this change relate to #1388, in which you proposed including conda-forge in the search path for recipes. That is discussion is still in the future IIRC, right? In some sense, even if we don't advise users users to include |
|
@kyleabeauchamp we want to include conda-forge as soon as possible in our build-process. |
…e fixes to skip-existing appear upstream.
bioconda utils perl test
bioconda utils python test
Update bioconductor annotate - bioconda_uilts bioc-test
| fn: annotate_1.48.0.tar.gz | ||
| url: https://bioarchive.galaxyproject.org/annotate_1.48.0.tar.gz | ||
| md5: f42ea2231d1858f3d0f587cc44562898 | ||
| fn: nnotate_1.50.0.tar.gz |
There was a problem hiding this comment.
good catch!
bioconda compile test
|
Ok, all tests pass. I will merge this now. The consequence is that we build on CentOS 6 from now on. The next step is to enable conda-forge as a channel dependency. Watch out for a follow up PR, maybe tomorrow, once we are sure that this one is working perfectly. |
|
Fingers crossed! |
Dear @bioconda/core, this PR consolidates our build system with conda-forge by using the same, CentOS 6 based docker container. This will solve recent breakages with the gcc package from Continuum which no longer works on CentOS 5 (see e.g. PR #2269). @bgruening @kyleabeauchamp @daler agree with me that this is a good point to finally move to CentOS 6. The result will be that packages built after this has been merged will require GLIBC 2.12 instead of 2.5.
I'll keep this open today, and I would ask you to raise any remaining objections.
Thank you!
Tests