TypeError when using Anaconda API

[amueller]

Hey.
I'm still trying to use conda-smithy to build a RC for our own channel.
However, I got

TEST END: scikit-learn-0.18rc-np110py27_blas_openblas_202
INFO:/opt/conda/lib/python3.5/site-packages/conda_build/config.py:--keep-old-work flag not specified.  Removing source and build files.

# If you want to upload this package to anaconda.org later, type:
#
# $ anaconda upload /feedstock_root/build_artefacts/linux-64/scikit-learn-0.18rc-np110py27_blas_openblas_202.tar.bz2
#
# To have conda build upload to anaconda.org automatically, use
# $ conda config --set anaconda_upload yes

Using Anaconda API: https://api.anaconda.org
Traceback (most recent call last):
  File "/feedstock_root/ci_support/upload_or_check_non_existence.py", line 124, in <module>
    main()
  File "/feedstock_root/ci_support/upload_or_check_non_existence.py", line 104, in main
    exists = built_distribution_already_exists(cli, meta, owner)
  File "/feedstock_root/ci_support/upload_or_check_non_existence.py", line 24, in built_distribution_already_exists
    fname = bldpkg_path(meta)
TypeError: bldpkg_path() missing 1 required positional argument: 'config'

./ci_support/run_docker_build.sh returned exit code 1

The built is here: https://circleci.com/gh/scikit-learn/scikit-learn-feedstock/3

Any help would be much appreciated.

[ogrisel]

Note the scikit-learn feedstock was setup by @jakirkham a while ago. Maybe it needs an update:

https://github.com/conda-forge/scikit-learn-feedstock

[amueller]

I tried conda smithy init again and it created the same upload_or_check_non_existence.py maybe I get an old conda or something because I didn't subscribe to the conda-forge channel?

[jakirkham]

Hey guys, so few thoughts here.

1. The issue is a conda-build API break that occurs in 2.x so are using 1.x for the interim.
2. A workaround is in the latest conda-smithy/conda-forge-build-setup. Working on a proper fix.
3. To update an existing feedstock, follow these directions.
4. There is a trick to control deployment label that is used.
5. In the long run, we want to provide a mechanism to do dev releases at conda-forge. So it should get easier.

[jakirkham]

Also if you want to try a PR to the feedstock here for re-rendering, am more than happy to review/merge. Alternatively I can just do it here and you can pull it into your feedstock.

[jakirkham]

Take a look at PR ( conda-forge/scikit-learn-feedstock#4 ) to see what the net result of the re-rendering is.

[amueller]

thanks. So we should merge that? Or just use your PR as basis for our feedstock?

[amueller]

I put it here:
https://github.com/scikit-learn/scikit-learn-feedstock

And I only tried to change the target channel to scikit-learn/ci
and the recipe to 0.18rc https://github.com/scikit-learn/scikit-learn-feedstock/blob/master/recipe/meta.yaml

[jakirkham]

thanks. So we should merge that? Or just use your PR as basis for our feedstock?

It should definitely be merged at conda-forge. How you wish to use it I'll leave up to you.

[jakirkham]

Ok, so there are probably some more tweaks that you will need.

First, you will need some new tokens. They all live in ~/.conda-smithy. Basically you will need github.token, travis.token, circle.token, and anaconda.token. Not sure worth setting up AppVeyor is worth it. I don't recall the token permissions for GitHub, but it would need certainly need access to public repos and probably repo hooks at least.

Second try running through steps 2 and 3 in this section. May need to re-render after this.

If you run into issues, let me know and we can take a closer look.

[amueller]

yeah I added all these tokens and tried running these steps. I''ll give it another go, thanks!

[jakirkham]

Sounds good. Also if you want to include Python 2.6 builds, feel free to try making this tweak.

[amueller]

@jakirkham I don't think we care about 2.6 conda packages enough to do an RC. not sure though.

[ogrisel]

-1 for wasting time on Python 2.6 ;)

[amueller]

Ok I think we're nearly there. There is now a failure uploading to anaconda.org:
https://circleci.com/gh/scikit-learn/scikit-learn-feedstock/8

subprocess.CalledProcessError: Command '['anaconda', '--quiet', '-t', 'binstar.token', 'upload', '/feedstock_root/build_artefacts/linux-64/scikit-learn-0.18rc-np110py27_blas_openblas_202.tar.bz2', '--user=conda-forge', '--channel=main']' returned non-zero exit status 1

It seems to try to use user conda-forge and channel main. However, the target is ['scikit-learn', 'rc']:

https://github.com/scikit-learn/scikit-learn-feedstock/blob/master/conda-forge.yml

Is there anything else I need to change?

[jakirkham]

Did you use the --organization flag in steps 2 and 3?

Also try adding this to conda-forge.yml.

github:
  user_or_org: scikit-learn

Finally make sure to do a final re-render.

[amueller]

@jakirkham I did. Though I get this error on step 3:

 * scikit-learn/scikit-learn-feedstock already enabled on appveyor
Traceback (most recent call last):
  File "/home/andy/anaconda3/bin/conda-smithy", line 11, in <module>
    load_entry_point('conda-smithy==1.2.0', 'console_scripts', 'conda-smithy')()
  File "/home/andy/anaconda3/lib/python3.5/site-packages/conda_smithy/cli.py", line 236, in main
    args.subcommand_func(args)
  File "/home/andy/anaconda3/lib/python3.5/site-packages/conda_smithy/cli.py", line 166, in __call__
    ci_register.appveyor_configure(owner, repo)
  File "/home/andy/anaconda3/lib/python3.5/site-packages/conda_smithy/ci_register.py", line 149, in appveyor_configure
    raise ValueError(response)
ValueError: <Response [404]>

[amueller]

If I do a rerender, it removes the built matrix.

[amueller]

Yeah after rerender it doesn't run anymore at all:
https://circleci.com/gh/scikit-learn/scikit-learn-feedstock/10

[jakirkham]

The matrix removal is interesting. If you have a reproducer, please share in issue ( #223 ). We have been trying very hard to reproduce and have been unable to for awhile now.

[jakirkham]

I'm guessing the above error means we didn't add an AppVeyor token. It is a bit annoying as we don't actually care about or are using AppVeyor ATM. Maybe it doesn't matter. 😕

Honestly am starting to wonder if it wouldn't be better to do the RC at conda-forge. While we haven't ironed out the process, I think it is going to be less work all around. Plus I'm more familiar with how that works.

[amueller]

conda-forge/scikit-learn-feedstock#6
that's what I figured ;)

[jakirkham]

Going to close this out, as the initial issue is resolved with a re-render. Also we seem to have sorted out what we want to do with the RC, which was the other thing worth discussing.