Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update for Travis CI to test against spark 2.0 #360

Merged
merged 2 commits into from
Sep 7, 2016

Conversation

boazmohar
Copy link
Contributor

@boazmohar boazmohar commented Sep 6, 2016

Address #359

@jwittenbach
Copy link
Contributor

Well look at that -- awesome! 👍

@freeman-lab
Copy link
Member

Well that was straightforward 🎉 thanks @boazmohar ! Can you add a note to the README as part of this PR saying that it works with Spark 2.0.0?

@boazmohar
Copy link
Contributor Author

Yep, was very easy. I guess the parts that changed a lot in 2.0 are DataFrame and Dataset not RDD's.

@jwittenbach
Copy link
Contributor

I've been thinking that it would be test whether there might be any speed-up in switching things to DataFrames -- though I haven't had any time to play with it.

@freeman-lab
Copy link
Member

Yeah agreed @jwittenbach, we should definitely look into that, I think the Dataset API may be what we want to look at. But in the meantime let's merge this in, good to make clear it's working with the latest version.

@freeman-lab freeman-lab merged commit e8d06ad into thunder-project:master Sep 7, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants