Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: [gapic-generator-java] handle response and metadata type ambiguity in LRO parsing #1726

Merged
merged 19 commits into from
Jul 5, 2023
Merged
Show file tree
Hide file tree
Changes from 11 commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
8096dbd
test: uncomment showcase-extended setup
emmileaf May 24, 2023
c5e0b30
test: adjustments for showcase-extended setup
emmileaf May 25, 2023
ca740ba
test: add proto to reproduce type collision issue
emmileaf May 25, 2023
7ce4084
test: add golden generated files for wicked and collisions protos
emmileaf May 31, 2023
76d8fba
fix: update LRO parsing to check package for short-form response and …
emmileaf May 31, 2023
2e1c69e
tests: exclude Location edge case from test proto and update goldens
emmileaf May 31, 2023
e4c4916
Merge branch 'main' into showcase-extended-explore
emmileaf May 31, 2023
7c759a9
test: update showcase-extended goldens after merging changes from main
emmileaf May 31, 2023
67fecec
fix: recursively write generics on methodInvocationExpr to account fo…
emmileaf Jun 1, 2023
30d8a0d
test: add Location edge case back to test proto and update goldens
emmileaf Jun 1, 2023
b73d72a
test: add service description to new test proto
emmileaf Jun 1, 2023
de5e9c3
chore: refactor type name matching into private helper
emmileaf Jun 13, 2023
b7952a4
test: turn off showcase-extended setup
emmileaf Jun 13, 2023
69e3a27
test: remove golden files previously generated by showcase-extended s…
emmileaf Jun 13, 2023
1ff61f4
test: add unit tests to TypeParserTest
emmileaf Jun 13, 2023
3703269
test: add writer unit test for generics import collision fix
emmileaf Jun 14, 2023
06dbf1f
test: undo changes to showcase-extended setup
emmileaf Jun 14, 2023
ba60786
test: update unit tests for lro type parsing
emmileaf Jun 14, 2023
ba1d1af
test: update unit tests for writer import collision fix
emmileaf Jun 14, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -362,7 +362,8 @@ public void visit(MethodInvocationExpr methodInvocationExpr) {
leftAngle();
int numGenerics = methodInvocationExpr.generics().size();
for (int i = 0; i < numGenerics; i++) {
buffer.append(methodInvocationExpr.generics().get(i).name());
Reference r = methodInvocationExpr.generics().get(i);
r.accept(this);
if (i < numGenerics - 1) {
buffer.append(COMMA);
space();
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -824,39 +824,34 @@ static LongrunningOperation parseLro(
boolean isResponseTypeNameShortOnly = lastDotIndex < 0;
String responseTypeShortName =
lastDotIndex >= 0 ? responseTypeName.substring(lastDotIndex + 1) : responseTypeName;
// When only shortname is provided, match on same proto package as method (See
// https://aip.dev/151)
String responseTypeFullName =
isResponseTypeNameShortOnly
? methodDescriptor.getFile().getPackage() + "." + responseTypeShortName
: responseTypeName;
emmileaf marked this conversation as resolved.
Show resolved Hide resolved

lastDotIndex = metadataTypeName.lastIndexOf('.');
boolean isMetadataTypeNameShortOnly = lastDotIndex < 0;
String metadataTypeShortName =
lastDotIndex >= 0 ? metadataTypeName.substring(lastDotIndex + 1) : metadataTypeName;
// When only shortname is provided, match on same proto package as method (See
// https://aip.dev/151)
String metadataTypeFullName =
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What if the short name is invalid? Can we throw an exception in the generator? Or we are going to generate code that is not compilable?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note from discussion offline: if the short name is invalid (no match) then the exception is handled by existing logic here:

Preconditions.checkNotNull(
responseMessage,
String.format(
"LRO response message %s not found on method %s",
responseTypeName, methodDescriptor.getName()));
// TODO(miraleung): Check that the packages are equal if those strings are not empty.
Preconditions.checkNotNull(
metadataMessage,
String.format(
"LRO metadata message %s not found in method %s",
metadataTypeName, methodDescriptor.getName()));

But this behavior should be tested (through true unit tests rather than goldens). Will look into replacing showcase-extended setup in this PR with true unit testing for Parser.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reverted changes to showcase-extended setup, and added unit tests separately for respective Parser and Writer fixes (more in PR description).

isMetadataTypeNameShortOnly
? methodDescriptor.getFile().getPackage() + "." + metadataTypeShortName
: metadataTypeName;

// The messageTypes map keys to the Java fully-qualified name.
for (Map.Entry<String, Message> messageEntry : messageTypes.entrySet()) {
String messageKey = messageEntry.getKey();
int messageLastDotIndex = messageEntry.getKey().lastIndexOf('.');
String messageShortName =
messageLastDotIndex >= 0 ? messageKey.substring(messageLastDotIndex + 1) : messageKey;
if (responseMessage == null) {
if (isResponseTypeNameShortOnly && responseTypeName.equals(messageShortName)) {
responseMessage = messageEntry.getValue();
} else if (!isResponseTypeNameShortOnly && responseTypeShortName.equals(messageShortName)) {
// Ensure that the full proto name matches.
Message candidateMessage = messageEntry.getValue();
if (candidateMessage.fullProtoName().equals(responseTypeName)) {
responseMessage = candidateMessage;
}
}
Message candidateMessage = messageEntry.getValue();
if (responseMessage == null
&& candidateMessage.fullProtoName().equals(responseTypeFullName)) {
responseMessage = candidateMessage;
}
if (metadataMessage == null) {
if (isMetadataTypeNameShortOnly && metadataTypeName.equals(messageShortName)) {
metadataMessage = messageEntry.getValue();
} else if (!isMetadataTypeNameShortOnly && metadataTypeShortName.equals(messageShortName)) {
// Ensure that the full proto name matches.
Message candidateMessage = messageEntry.getValue();
if (candidateMessage.fullProtoName().equals(metadataTypeName)) {
metadataMessage = candidateMessage;
}
}
if (metadataMessage == null
&& candidateMessage.fullProtoName().equals(metadataTypeFullName)) {
metadataMessage = candidateMessage;
}
}

Expand Down
26 changes: 13 additions & 13 deletions showcase/BUILD.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ proto_library_with_info(
name = "showcase_proto_with_info",
deps = [
"@com_google_gapic_showcase//schema/google/showcase/v1beta1:showcase_proto",
# "//showcase/gapic-showcase-extended/proto:showcase_proto_extended",
"//showcase/gapic-showcase-extended/proto:showcase_proto_extended",
"@com_google_googleapis//google/cloud:common_resources_proto",
"@com_google_googleapis//google/cloud/location:location_proto"
],
Expand All @@ -27,7 +27,7 @@ java_proto_library(
name = "showcase_java_proto",
deps = [
"@com_google_gapic_showcase//schema/google/showcase/v1beta1:showcase_proto",
# "//showcase/gapic-showcase-extended/proto:showcase_proto_extended"
"//showcase/gapic-showcase-extended/proto:showcase_proto_extended"
],
)

Expand All @@ -37,13 +37,13 @@ java_grpc_library(
deps = [":showcase_java_proto"],
)

#java_grpc_library(
# name = "showcase_java_grpc_extended",
# srcs = [
# "//showcase/gapic-showcase-extended/proto:showcase_proto_extended",
# ],
# deps = [":showcase_java_proto"],
#)
java_grpc_library(
name = "showcase_java_grpc_extended",
srcs = [
"//showcase/gapic-showcase-extended/proto:showcase_proto_extended",
],
deps = [":showcase_java_proto"],
)

java_gapic_library(
name = "showcase_java_gapic",
Expand All @@ -56,7 +56,7 @@ java_gapic_library(
service_yaml = "@com_google_gapic_showcase//schema/google/showcase/v1beta1:showcase_v1beta1.yaml",
test_deps = [
":showcase_java_grpc",
# ":showcase_java_grpc_extended",
":showcase_java_grpc_extended",
"@com_google_googleapis//google/cloud/location:location_java_grpc"
],
transport = "grpc+rest",
Expand Down Expand Up @@ -121,10 +121,10 @@ sh_binary(
# GRPC Showcase : Update and Verify
GRPC_DATA = [
"libshowcase_java_grpc-src.jar",
# "libshowcase_java_grpc_extended-src.jar",
"libshowcase_java_grpc_extended-src.jar",
":grpc_gapic_showcase_files",
":showcase_java_grpc",
# ":showcase_java_grpc_extended"
":showcase_java_grpc_extended"
]

sh_binary(
Expand All @@ -146,7 +146,7 @@ PROTO_DATA = [
"proto-google-cloud-showcase-v1beta1-java.tar.gz",
":proto_gapic_showcase_files",
":showcase_java_proto",
# ":showcase_java_proto_extended",
# ":showcase_java_proto_extended",
]

sh_binary(
Expand Down
19 changes: 13 additions & 6 deletions showcase/gapic-showcase-extended/proto/BUILD.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,8 @@
Provides proto_library target
Exports grpc service config
"""
load ("@rules_proto//proto:defs.bzl", "proto_library")

load("@rules_proto//proto:defs.bzl", "proto_library")

# This is an API workspace, having public visibility by default makes perfect sense.
package(default_visibility = ["//visibility:public"])
Expand All @@ -27,8 +28,14 @@ package(default_visibility = ["//visibility:public"])
# gapic-showcase project is used to test the generated client behavior with a showcase server
# gapic-showcase-extension project is used to test the generator's behavior

#proto_library(
# name = "showcase_proto_extended",
# srcs = [],
# deps = []
#)
proto_library(
name = "showcase_proto_extended",
srcs = [
":collisions.proto",
":wicked.proto",
],
deps = [
"@com_google_googleapis//google/api:client_proto",
"@com_google_googleapis//google/longrunning:operations_proto",
],
)
51 changes: 51 additions & 0 deletions showcase/gapic-showcase-extended/proto/collisions.proto
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
// Copyright 2023 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// https://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

syntax = "proto3";

import "google/api/client.proto";
import "google/longrunning/operations.proto";

package google.showcase.v1beta1;

option java_package = "com.google.showcase.v1beta1";
option java_multiple_files = true;

// This service exercises scenarios where short names of types
// exhibit ambiguity or collide with other types
service Collisions {

option (google.api.default_host) = "localhost:7469";

rpc doSomething(Request) returns (google.longrunning.Operation) {
option (google.longrunning.operation_info) = {
response_type: "Annotation"
metadata_type: "Location"
};
}
}

message Request {
string name = 1;
Annotation annotation = 2;
Location location = 3;
}

message Annotation {
string name = 1;
}

message Location {
string name = 1;
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
/*
* Copyright 2023 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

package com.google.showcase.v1beta1.samples;

// [START localhost7469_v1beta1_generated_Collisions_Create_SetCredentialsProvider_sync]
import com.google.api.gax.core.FixedCredentialsProvider;
import com.google.showcase.v1beta1.CollisionsClient;
import com.google.showcase.v1beta1.CollisionsSettings;
import com.google.showcase.v1beta1.myCredentials;

public class SyncCreateSetCredentialsProvider {

public static void main(String[] args) throws Exception {
syncCreateSetCredentialsProvider();
}

public static void syncCreateSetCredentialsProvider() throws Exception {
// This snippet has been automatically generated and should be regarded as a code template only.
// It will require modifications to work:
// - It may require correct/in-range values for request initialization.
// - It may require specifying regional endpoints when creating the service client as shown in
// https://cloud.google.com/java/docs/setup#configure_endpoints_for_the_client_library
CollisionsSettings collisionsSettings =
CollisionsSettings.newBuilder()
.setCredentialsProvider(FixedCredentialsProvider.create(myCredentials))
.build();
CollisionsClient collisionsClient = CollisionsClient.create(collisionsSettings);
}
}
// [END localhost7469_v1beta1_generated_Collisions_Create_SetCredentialsProvider_sync]
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
/*
* Copyright 2023 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

package com.google.showcase.v1beta1.samples;

// [START localhost7469_v1beta1_generated_Collisions_Create_SetCredentialsProvider1_sync]
import com.google.showcase.v1beta1.CollisionsClient;
import com.google.showcase.v1beta1.CollisionsSettings;

public class SyncCreateSetCredentialsProvider1 {

public static void main(String[] args) throws Exception {
syncCreateSetCredentialsProvider1();
}

public static void syncCreateSetCredentialsProvider1() throws Exception {
// This snippet has been automatically generated and should be regarded as a code template only.
// It will require modifications to work:
// - It may require correct/in-range values for request initialization.
// - It may require specifying regional endpoints when creating the service client as shown in
// https://cloud.google.com/java/docs/setup#configure_endpoints_for_the_client_library
CollisionsSettings collisionsSettings = CollisionsSettings.newHttpJsonBuilder().build();
CollisionsClient collisionsClient = CollisionsClient.create(collisionsSettings);
}
}
// [END localhost7469_v1beta1_generated_Collisions_Create_SetCredentialsProvider1_sync]
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
/*
* Copyright 2023 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

package com.google.showcase.v1beta1.samples;

// [START localhost7469_v1beta1_generated_Collisions_Create_SetEndpoint_sync]
import com.google.showcase.v1beta1.CollisionsClient;
import com.google.showcase.v1beta1.CollisionsSettings;
import com.google.showcase.v1beta1.myEndpoint;

public class SyncCreateSetEndpoint {

public static void main(String[] args) throws Exception {
syncCreateSetEndpoint();
}

public static void syncCreateSetEndpoint() throws Exception {
// This snippet has been automatically generated and should be regarded as a code template only.
// It will require modifications to work:
// - It may require correct/in-range values for request initialization.
// - It may require specifying regional endpoints when creating the service client as shown in
// https://cloud.google.com/java/docs/setup#configure_endpoints_for_the_client_library
CollisionsSettings collisionsSettings =
CollisionsSettings.newBuilder().setEndpoint(myEndpoint).build();
CollisionsClient collisionsClient = CollisionsClient.create(collisionsSettings);
}
}
// [END localhost7469_v1beta1_generated_Collisions_Create_SetEndpoint_sync]
Loading