Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[SPARK-3798][SQL] Store the output of a generator in a val #2656

Closed
wants to merge 1 commit into from
Closed

[SPARK-3798][SQL] Store the output of a generator in a val #2656

wants to merge 1 commit into from

Conversation

marmbrus
Copy link
Contributor

@marmbrus marmbrus commented Oct 4, 2014

This prevents it from changing during serialization, leading to corrupted results.

@marmbrus
Store the output of a generator in a val.
efa32eb 
This prevents it from changing during serialization.
@marmbrus marmbrus mentioned this pull request Oct 4, 2014
Closed
@SparkQA
Copy link

SparkQA commented Oct 4, 2014

QA tests have started for PR 2656 at commit efa32eb.

  • This patch merges cleanly.

@mateiz
Copy link
Contributor

mateiz commented Oct 4, 2014

Is there any way to add a unit test for this?

@marmbrus
Copy link
Contributor Author

marmbrus commented Oct 4, 2014

Unfortunately, I haven't found a way to reproduce it deterministically.

@SparkQA
Copy link

SparkQA commented Oct 4, 2014

Tests timed out for PR 2656 at commit efa32eb after a configured wait of 120m.

@AmplabJenkins
Copy link

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/21296/Test FAILed.

@SparkQA
Copy link

SparkQA commented Oct 4, 2014

QA tests have started for PR 2656 at commit efa32eb.

  • This patch merges cleanly.

@SparkQA
Copy link

SparkQA commented Oct 5, 2014

Tests timed out for PR 2656 at commit efa32eb after a configured wait of 120m.

@SparkQA
Copy link

SparkQA commented Oct 9, 2014

QA tests have started for PR 2656 at commit efa32eb.

  • This patch merges cleanly.

@SparkQA
Copy link

SparkQA commented Oct 9, 2014

QA tests have finished for PR 2656 at commit efa32eb.

  • This patch fails Spark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Oct 9, 2014

QA tests have started for PR 2656 at commit efa32eb.

  • This patch does not merge cleanly!

@SparkQA
Copy link

SparkQA commented Oct 9, 2014

QA tests have finished for PR 2656 at commit efa32eb.

  • This patch passes unit tests.
  • This patch does not merge cleanly!

@asfgit asfgit closed this in 2837bf8 Oct 10, 2014
asfgit pushed a commit that referenced this pull request Oct 13, 2014
@marmbrus
[SQL] Add type checking debugging functions
371321c 
Adds some functions that were very useful when trying to track down the bug from #2656.  This change also changes the tree output for query plans to include the `'` prefix to unresolved nodes and `!` prefix to nodes that refer to non-existent attributes.

Author: Michael Armbrust <michael@databricks.com>

Closes #2657 from marmbrus/debugging and squashes the following commits:

654b926 [Michael Armbrust] Clean-up, add tests
763af15 [Michael Armbrust] Add typeChecking debugging functions
8c69303 [Michael Armbrust] Add inputSet, references to QueryPlan. Improve tree string with a prefix to denote invalid or unresolved nodes.
fbeab54 [Michael Armbrust] Better toString, factories for AttributeSet.
@marmbrus marmbrus deleted the generateBug branch November 19, 2014 02:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@marmbrus @SparkQA @mateiz @AmplabJenkins