Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

bigquery: try getting job if create fails #2887

Merged
merged 3 commits into from
Feb 12, 2018
Merged

bigquery: try getting job if create fails #2887

merged 3 commits into from
Feb 12, 2018

Conversation

pongad
Copy link
Contributor

@pongad pongad commented Feb 9, 2018

If we create a job with randomly generated ID, it's highly unlikely
that it will with another job created by someone else.
When create fails, it is possible that the job was created successfully
anyway.
Thus, we can be more failure-tolerant by trying to retrieve the job,
if create throws.

Fixes #2867.

@pongad
bigquery: try getting job if create fails
c408bd0 
If we create a job with randomly generated ID, it's highly unlikely
that it will with another job created by someone else.
When create fails, it is possible that the job was created successfully
anyway.
Thus, we can be more failure-tolerant by trying to retrieve the job,
if create throws.

Fixes #2867.
@googlebot googlebot added the cla: yes This human has signed the Contributor License Agreement. label Feb 9, 2018
pongad
pongad commented Feb 9, 2018
View reviewed changes
google-cloud-bigquery/src/test/java/com/google/cloud/bigquery/BigQueryImplTest.java
@@ -874,71 +877,95 @@ public void testListTableDataWithOptions() {
assertArrayEquals(TABLE_DATA.toArray(), Iterables.toArray(page.getValues(), List.class));
}

@Test
public void testCreateQueryJob() {

This comment was marked as spam.

Sign in to view

tswast
tswast suggested changes Feb 9, 2018
View reviewed changes
Copy link
Contributor

@tswast tswast left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! Looking great. A few nits / questions.

google-cloud-bigquery/src/main/java/com/google/cloud/bigquery/BigQueryImpl.java
@@ -590,7 +641,8 @@ public Boolean call() {
@Override
public TableResult query(QueryJobConfiguration configuration, JobOption... options)
throws InterruptedException, JobException {
return query(configuration, JobId.of(), options);
Job.checkNotDryRun(configuration, "query");

This comment was marked as spam.

Sign in to view

This comment was marked as spam.

Sign in to view

google-cloud-bigquery/src/main/java/com/google/cloud/bigquery/BigQueryImpl.java Outdated
try {
return getJob(jobInfo.getJobId());
} catch (BigQueryException e) {
System.err.println("rethrow");

This comment was marked as spam.

Sign in to view

google-cloud-bigquery/src/test/java/com/google/cloud/bigquery/BigQueryImplTest.java
bigqueryRpcMock.create(
JobInfo.of(LOAD_JOB_CONFIGURATION_WITH_PROJECT).toPb(), EMPTY_RPC_OPTIONS))
.andReturn(COMPLETE_LOAD_JOB.toPb());
bigqueryRpcMock.create(EasyMock.capture(jobCapture), EasyMock.eq(EMPTY_RPC_OPTIONS)))

This comment was marked as spam.

Sign in to view

google-cloud-bigquery/src/test/java/com/google/cloud/bigquery/BigQueryImplTest.java Outdated
bigquery = options.getService();
Job job = bigquery.create(COPY_JOB);
assertEquals(new Job(bigquery, new JobInfo.BuilderImpl(COMPLETE_COPY_JOB)), job);
Job job = bigquery.create(QUERY_JOB, JOB_OPTION_FIELDS);

This comment was marked as spam.

Sign in to view

google-cloud-bigquery/src/test/java/com/google/cloud/bigquery/BigQueryImplTest.java Outdated
String selector = (String) capturedOptions.getValue().get(JOB_OPTION_FIELDS.getRpcOption());
assertThat(selector.split(","))
.asList()
.containsExactly("jobReference", "configuration", "user_email");

This comment was marked as spam.

Sign in to view

This comment was marked as spam.

Sign in to view

google-cloud-bigquery/src/main/java/com/google/cloud/bigquery/BigQueryImpl.java
new Supplier<JobId>() {
@Override
public JobId get() {
return JobId.of();

This comment was marked as spam.

Sign in to view

This comment was marked as spam.

Sign in to view

This comment was marked as spam.

Sign in to view

This comment was marked as spam.

Sign in to view

This comment was marked as spam.

Sign in to view

@pongad
Copy link
Contributor Author

pongad commented Feb 12, 2018

@tswast PTAL

tswast
tswast approved these changes Feb 12, 2018
View reviewed changes
Copy link
Contributor

@tswast tswast left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM once Circle is happy

@pongad pongad merged commit e2f0d20 into googleapis:master Feb 12, 2018
@pongad pongad deleted the create-retry branch February 12, 2018 19:30
@BenBirt BenBirt mentioned this pull request Sep 18, 2020
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@tswast tswast tswast approved these changes

@vam-google vam-google Awaiting requested review from vam-google

Assignees
No one assigned
Labels
cla: yes This human has signed the Contributor License Agreement.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@pongad @tswast @googlebot