[SPARK-7262][ML] Binary LogisticRegression with L1/L2 (elastic net) using OWLQN in new ML package #5967

dbtsai · 2015-05-07T06:23:25Z

Handle scaling and addBias internally.
L1/L2 elasticnet using OWLQN optimizer.

SparkQA · 2015-05-07T08:50:20Z

Test build #32083 has finished for PR 5967 at commit 34705bc.

This patch passes all tests.
This patch does not merge cleanly.
This patch adds no public classes.

SparkQA · 2015-05-08T09:25:23Z

Test build #32221 has finished for PR 5967 at commit 8ec65d2.

This patch fails Scala style tests.
This patch does not merge cleanly.
This patch adds no public classes.

SparkQA · 2015-05-08T12:03:49Z

Test build #32222 timed out for PR 5967 at commit a784321 after a configured wait of 150m.

mengxr · 2015-05-08T15:58:44Z

test this please

mengxr · 2015-05-08T16:15:41Z

mllib/src/main/scala/org/apache/spark/ml/classification/LogisticRegression.scala

+ * Two MultilabelSummarizer can be merged together to have a statistical summary of the
+ * corresponding joint dataset.
+ */
+class MultiClassSummarizer private[ml] extends Serializable {


private class?

I want to test it in the suite.

Then we can mark it package private. This shouldn't be a public class.

mengxr · 2015-05-08T17:02:10Z

@dbtsai I made a pass and it looks good to except some minor inline comments. Could you address the comments today? It should be good to go then.

SparkQA · 2015-05-08T18:21:34Z

Test build #32244 has finished for PR 5967 at commit a784321.

This patch passes all tests.
This patch does not merge cleanly.
This patch adds no public classes.

SparkQA · 2015-05-08T19:49:45Z

Test build #32249 has finished for PR 5967 at commit f98e711.

This patch fails Spark unit tests.
This patch does not merge cleanly.
This patch adds no public classes.

dbtsai · 2015-05-08T19:54:54Z

test this please

SparkQA · 2015-05-08T20:23:13Z

Test build #32257 has finished for PR 5967 at commit 5c31824.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
- class Binarizer(JavaTransformer, HasInputCol, HasOutputCol):
- class IDF(JavaEstimator, HasInputCol, HasOutputCol):
- class IDFModel(JavaModel):
- class Normalizer(JavaTransformer, HasInputCol, HasOutputCol):
- class OneHotEncoder(JavaTransformer, HasInputCol, HasOutputCol):
- class PolynomialExpansion(JavaTransformer, HasInputCol, HasOutputCol):
- class RegexTokenizer(JavaTransformer, HasInputCol, HasOutputCol):
- class StandardScaler(JavaEstimator, HasInputCol, HasOutputCol):
- class StandardScalerModel(JavaModel):
- class StringIndexer(JavaEstimator, HasInputCol, HasOutputCol):
- class StringIndexerModel(JavaModel):
- class Tokenizer(JavaTransformer, HasInputCol, HasOutputCol):
- class VectorIndexer(JavaEstimator, HasInputCol, HasOutputCol):
- class Word2Vec(JavaEstimator, HasStepSize, HasMaxIter, HasSeed, HasInputCol, HasOutputCol):
- class Word2VecModel(JavaModel):
- class HasSeed(Params):
- class HasTol(Params):
- class HasStepSize(Params):
- case class UnresolvedExtractValue(child: Expression, extraction: Expression)
- trait ExtractValue extends UnaryExpression
- case class GetStructField(child: Expression, field: StructField, ordinal: Int)
- case class GetArrayStructFields(
- abstract class ExtractValueWithOrdinal extends ExtractValue
- case class GetArrayItem(child: Expression, ordinal: Expression)
- case class GetMapValue(child: Expression, ordinal: Expression)

SparkQA · 2015-05-08T20:36:06Z

Test build #32252 has finished for PR 5967 at commit c84e931.

This patch passes all tests.
This patch does not merge cleanly.
This patch adds no public classes.

mengxr · 2015-05-08T20:58:22Z

test this please

dbtsai · 2015-05-08T21:24:00Z

test this please

SparkQA · 2015-05-08T21:28:00Z

Test build #32273 has finished for PR 5967 at commit fa029bb.

This patch fails MiMa tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
- class Binarizer(JavaTransformer, HasInputCol, HasOutputCol):
- class IDF(JavaEstimator, HasInputCol, HasOutputCol):
- class IDFModel(JavaModel):
- class Normalizer(JavaTransformer, HasInputCol, HasOutputCol):
- class OneHotEncoder(JavaTransformer, HasInputCol, HasOutputCol):
- class PolynomialExpansion(JavaTransformer, HasInputCol, HasOutputCol):
- class RegexTokenizer(JavaTransformer, HasInputCol, HasOutputCol):
- class StandardScaler(JavaEstimator, HasInputCol, HasOutputCol):
- class StandardScalerModel(JavaModel):
- class StringIndexer(JavaEstimator, HasInputCol, HasOutputCol):
- class StringIndexerModel(JavaModel):
- class Tokenizer(JavaTransformer, HasInputCol, HasOutputCol):
- class VectorIndexer(JavaEstimator, HasInputCol, HasOutputCol):
- class Word2Vec(JavaEstimator, HasStepSize, HasMaxIter, HasSeed, HasInputCol, HasOutputCol):
- class Word2VecModel(JavaModel):
- class HasSeed(Params):
- class HasTol(Params):
- class HasStepSize(Params):
- case class UnresolvedExtractValue(child: Expression, extraction: Expression)
- trait ExtractValue extends UnaryExpression
- case class GetStructField(child: Expression, field: StructField, ordinal: Int)
- case class GetArrayStructFields(
- abstract class ExtractValueWithOrdinal extends ExtractValue
- case class GetArrayItem(child: Expression, ordinal: Expression)
- case class GetMapValue(child: Expression, ordinal: Expression)

SparkQA · 2015-05-08T21:39:02Z

Test build #32274 has finished for PR 5967 at commit fa029bb.

This patch fails MiMa tests.
This patch merges cleanly.
This patch adds no public classes.

SparkQA · 2015-05-08T21:39:23Z

Test build #32261 has finished for PR 5967 at commit 5c31824.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
- case class UnresolvedExtractValue(child: Expression, extraction: Expression)
- trait ExtractValue extends UnaryExpression
- case class GetStructField(child: Expression, field: StructField, ordinal: Int)
- case class GetArrayStructFields(
- abstract class ExtractValueWithOrdinal extends ExtractValue
- case class GetArrayItem(child: Expression, ordinal: Expression)
- case class GetMapValue(child: Expression, ordinal: Expression)

SparkQA · 2015-05-08T22:31:18Z

Test build #32268 has finished for PR 5967 at commit 5c31824.

This patch fails PySpark unit tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
- public class EnumUtil
- class Binarizer(JavaTransformer, HasInputCol, HasOutputCol):
- class IDF(JavaEstimator, HasInputCol, HasOutputCol):
- class IDFModel(JavaModel):
- class Normalizer(JavaTransformer, HasInputCol, HasOutputCol):
- class OneHotEncoder(JavaTransformer, HasInputCol, HasOutputCol):
- class PolynomialExpansion(JavaTransformer, HasInputCol, HasOutputCol):
- class RegexTokenizer(JavaTransformer, HasInputCol, HasOutputCol):
- class StandardScaler(JavaEstimator, HasInputCol, HasOutputCol):
- class StandardScalerModel(JavaModel):
- class StringIndexer(JavaEstimator, HasInputCol, HasOutputCol):
- class StringIndexerModel(JavaModel):
- class Tokenizer(JavaTransformer, HasInputCol, HasOutputCol):
- class VectorIndexer(JavaEstimator, HasInputCol, HasOutputCol):
- class Word2Vec(JavaEstimator, HasStepSize, HasMaxIter, HasSeed, HasInputCol, HasOutputCol):
- class Word2VecModel(JavaModel):
- class HasSeed(Params):
- class HasTol(Params):
- class HasStepSize(Params):
- case class UnresolvedExtractValue(child: Expression, extraction: Expression)
- trait ExtractValue extends UnaryExpression
- case class GetStructField(child: Expression, field: StructField, ordinal: Int)
- case class GetArrayStructFields(
- abstract class ExtractValueWithOrdinal extends ExtractValue
- case class GetArrayItem(child: Expression, ordinal: Expression)
- case class GetMapValue(child: Expression, ordinal: Expression)

mengxr · 2015-05-08T22:37:31Z

test this please

SparkQA · 2015-05-09T00:29:24Z

Test build #32278 has finished for PR 5967 at commit fa029bb.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

…using OWLQN in new ML package 1) Handle scaling and addBias internally. 2) L1/L2 elasticnet using OWLQN optimizer. Author: DB Tsai <dbt@netflix.com> Closes #5967 from dbtsai/lor and squashes the following commits: fa029bb [DB Tsai] made the bound smaller 0806002 [DB Tsai] better initial intercept and more test 5c31824 [DB Tsai] fix import c387e25 [DB Tsai] Merge branch 'master' into lor c84e931 [DB Tsai] Made MultiClassSummarizer private f98e711 [DB Tsai] address feedback a784321 [DB Tsai] fix style 8ec65d2 [DB Tsai] remove new line f3f8c88 [DB Tsai] add more tests and they match R which is good. fix a bug 34705bc [DB Tsai] first commit (cherry picked from commit 86ef4cf) Signed-off-by: Xiangrui Meng <meng@databricks.com>

mengxr · 2015-05-09T04:43:59Z

Merged into master and branch-1.4. Thanks!

…using OWLQN in new ML package 1) Handle scaling and addBias internally. 2) L1/L2 elasticnet using OWLQN optimizer. Author: DB Tsai <dbt@netflix.com> Closes apache#5967 from dbtsai/lor and squashes the following commits: fa029bb [DB Tsai] made the bound smaller 0806002 [DB Tsai] better initial intercept and more test 5c31824 [DB Tsai] fix import c387e25 [DB Tsai] Merge branch 'master' into lor c84e931 [DB Tsai] Made MultiClassSummarizer private f98e711 [DB Tsai] address feedback a784321 [DB Tsai] fix style 8ec65d2 [DB Tsai] remove new line f3f8c88 [DB Tsai] add more tests and they match R which is good. fix a bug 34705bc [DB Tsai] first commit

first commit

34705bc

DB Tsai added 2 commits May 8, 2015 02:16

add more tests and they match R which is good. fix a bug

f3f8c88

remove new line

8ec65d2

fix style

a784321

mengxr reviewed May 8, 2015
View reviewed changes

DB Tsai added 2 commits May 8, 2015 10:53

address feedback

f98e711

Made MultiClassSummarizer private

c84e931

DB Tsai added 2 commits May 8, 2015 12:02

Merge branch 'master' into lor

c387e25

fix import

5c31824

DB Tsai added 2 commits May 8, 2015 14:04

better initial intercept and more test

0806002

made the bound smaller

fa029bb

asfgit closed this in 86ef4cf May 9, 2015

dbtsai deleted the lor branch May 10, 2015 09:24

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[SPARK-7262][ML] Binary LogisticRegression with L1/L2 (elastic net) using OWLQN in new ML package #5967

[SPARK-7262][ML] Binary LogisticRegression with L1/L2 (elastic net) using OWLQN in new ML package #5967

dbtsai commented May 7, 2015

SparkQA commented May 7, 2015

SparkQA commented May 8, 2015

SparkQA commented May 8, 2015

mengxr commented May 8, 2015

mengxr May 8, 2015

dbtsai May 8, 2015

mengxr May 8, 2015

mengxr commented May 8, 2015

SparkQA commented May 8, 2015

SparkQA commented May 8, 2015

dbtsai commented May 8, 2015

SparkQA commented May 8, 2015

SparkQA commented May 8, 2015

mengxr commented May 8, 2015

dbtsai commented May 8, 2015

SparkQA commented May 8, 2015

SparkQA commented May 8, 2015

SparkQA commented May 8, 2015

SparkQA commented May 8, 2015

mengxr commented May 8, 2015

SparkQA commented May 9, 2015

mengxr commented May 9, 2015

[SPARK-7262][ML] Binary LogisticRegression with L1/L2 (elastic net) using OWLQN in new ML package #5967

[SPARK-7262][ML] Binary LogisticRegression with L1/L2 (elastic net) using OWLQN in new ML package #5967

Conversation

dbtsai commented May 7, 2015

SparkQA commented May 7, 2015

SparkQA commented May 8, 2015

SparkQA commented May 8, 2015

mengxr commented May 8, 2015

mengxr May 8, 2015

Choose a reason for hiding this comment

dbtsai May 8, 2015

Choose a reason for hiding this comment

mengxr May 8, 2015

Choose a reason for hiding this comment

mengxr commented May 8, 2015

SparkQA commented May 8, 2015

SparkQA commented May 8, 2015

dbtsai commented May 8, 2015

SparkQA commented May 8, 2015

SparkQA commented May 8, 2015

mengxr commented May 8, 2015

dbtsai commented May 8, 2015

SparkQA commented May 8, 2015

SparkQA commented May 8, 2015

SparkQA commented May 8, 2015

SparkQA commented May 8, 2015

mengxr commented May 8, 2015

SparkQA commented May 9, 2015

mengxr commented May 9, 2015