Skip to content

Commit

Permalink
Review feedback on PR
Browse files Browse the repository at this point in the history
  • Loading branch information
ScrapCodes committed Feb 27, 2014
1 parent 46ed2ad commit 9b29e34
Show file tree
Hide file tree
Showing 2 changed files with 4 additions and 4 deletions.
2 changes: 1 addition & 1 deletion docs/building-with-maven.md
Original file line number Diff line number Diff line change
Expand Up @@ -79,4 +79,4 @@ The debian package can then be found under assembly/target. We added the short c

## A note about Hadoop version 0.23.x

For building spark with hadoop 0.23.x and also yarn, you will have to provide a dependency on avro manually.
For building spark with hadoop 0.23.x and also yarn, you will have to manually add a dependency on avro (org.apache.avro, avro, 1.7.4).
6 changes: 3 additions & 3 deletions project/SparkBuild.scala
Original file line number Diff line number Diff line change
Expand Up @@ -87,7 +87,7 @@ object SparkBuild extends Build {
case Some(v) => v.toBoolean
}
lazy val hadoopClient = if (hadoopVersion.startsWith("0.20.") || hadoopVersion == "1.0.0") "hadoop-core" else "hadoop-client"
val isAvroNeeded = hadoopVersion.startsWith("0.23.") && isYarnEnabled
val maybeAvro = if (hadoopVersion.startsWith("0.23.") && isYarnEnabled) Seq("org.apache.avro" % "avro" % "1.7.4") else Seq()
// Conditionally include the yarn sub-project
lazy val yarnAlpha = Project("yarn-alpha", file("yarn/alpha"), settings = yarnAlphaSettings) dependsOn(core)
lazy val yarn = Project("yarn", file("yarn/stable"), settings = yarnSettings) dependsOn(core)
Expand Down Expand Up @@ -286,8 +286,8 @@ object SparkBuild extends Build {
"com.twitter" %% "chill" % "0.3.1",
"com.twitter" % "chill-java" % "0.3.1",
"com.clearspring.analytics" % "stream" % "2.5.1"
) ++ (if (isAvroNeeded) Seq(
"org.apache.avro" % "avro" % "1.7.4") else Seq())
),
libraryDependencies ++= maybeAvro
)

def rootSettings = sharedSettings ++ Seq(
Expand Down

0 comments on commit 9b29e34

Please sign in to comment.