Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix #825: add matplotlib to fix missing module error under docker image #827

Merged

Conversation

jimthompson5802
Copy link
Contributor

@jimthompson5802 jimthompson5802 commented Jan 21, 2019

Fix #825: ImportError: No module named matplotlib.pyplot in examples/sklearn_elasticnet_diabetes/train_diabetes.py in mlflow docker image

Overview of change:

  • Added Python package matplotlib to test-requirements.txt
  • Added support fo C compiler to Docker image to support install of one of matplotlib dependencies: subprocess32. pip install of this package is via source code and requires code compilation. As a result add to Dockerfile apt-get install -y build-essential command.

@jimthompson5802 jimthompson5802 changed the title Fix Issue #825: add matplotlib to fix missing module error under docker image Fix #825: add matplotlib to fix missing module error under docker image Jan 22, 2019
@jimthompson5802
Copy link
Contributor Author

jimthompson5802 commented Jan 22, 2019

Failure in Travis CI (Job 2921) is on Python 3 keras related tests. AFAICT, these changes do not affect anything re: Keras. Two PR submitted after mine passed their CI checks. Possible one-time fluke. Trying to restart CI check by closing PR and re-opening.

@jimthompson5802
Copy link
Contributor Author

reopening to try and restart CI tests based on this posting

@jimthompson5802
Copy link
Contributor Author

Travis CI failed (Job 2926) again but this time on the R tests. The Python 3 Keras test passed w/o issue. Trying one more time to restart CI tests.

@jimthompson5802
Copy link
Contributor Author

Restarting CI tests.

@dbczumar
Copy link
Collaborator

@jimthompson5802 Thank you for the contribution. After discussing it with the team, we feel that it may make sense to introduce an MLproject specification with a conda.yaml specification that enumerates the required dependencies, including matplotlib (just like the example you tweaked in #829).

Ideally, the Dockerfile included in the repository root is intended to include the minimal set of dependencies for developing MLflow and running existing tests, so I'd like to avoid extending the set of test requirements (which are also installed for every Travis build) as much as possible.

Would you be willing to add an MLproject with a Conda environment for this example? Users could then execute mlflow run ... inside the Docker container to achieve the desired result.

@jimthompson5802
Copy link
Contributor Author

@dbczumar Yes I can convert to MLProject structure.

One process question: Should I do this work on this current pull request or should I close this pull request and submit a new pull request when I've the MLProject in place?

@dbczumar
Copy link
Collaborator

@jimthompson5802 Thanks so much! I think it's fine to perform this work in the same PR, no reason to submit a new one.

@jimthompson5802
Copy link
Contributor Author

@dbczumar I've completed conversion to MLProject structure. Ready for your review.

This is the log for a test run:
sklearn_diabetes_mlproject_testrun.txt

@codecov-io
Copy link

Codecov Report

Merging #827 into master will not change coverage.
The diff coverage is n/a.

Impacted file tree graph

@@           Coverage Diff           @@
##           master     #827   +/-   ##
=======================================
  Coverage   65.75%   65.75%           
=======================================
  Files          22       22           
  Lines         800      800           
=======================================
  Hits          526      526           
  Misses        274      274

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 820eff1...ce163fc. Read the comment docs.

@jimthompson5802
Copy link
Contributor Author

prep to restart travis ci build

@jimthompson5802
Copy link
Contributor Author

restart travis ci build

Copy link
Collaborator

@dbczumar dbczumar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@dbczumar
Copy link
Collaborator

@jimthompson5802 I tried the MLproject on OSX and Ubuntu. It ran properly on Ubuntu, but I encountered a matplotlib issue on OSX. It looks like there are extra dependencies for the conda environment that must be installed on OSX and are unavailable on Linux systems. They are documented here: https://matplotlib.org/faq/osx_framework.html#conda.

I've updated the PR to divide the example into two subdirectories, one for each platform. Please let me know if this makes sense!

@jimthompson5802
Copy link
Contributor Author

jimthompson5802 commented Jan 25, 2019

@dbczumar No problem. Glad you guys have the resources to find these platform specific issues. Thank you.

@dbczumar dbczumar merged commit 561a365 into mlflow:master Jan 28, 2019
@jimthompson5802 jimthompson5802 deleted the fix_examples_sklearn_elasticnet_diabetes branch January 28, 2019 21:25
marcusrehm pushed a commit to marcusrehm/mlflow that referenced this pull request Feb 18, 2019
…er image (mlflow#827)

* add matplotlib to fix missing module error under docker

* restored prior version

* Initial check-in

* adapted to MLProject structure

* Split example into platform-specific subdirectories

* Add README explaining platform differences

* README links

* readme link fix

* Reveert formatting changes to java readme
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants