-
Notifications
You must be signed in to change notification settings - Fork 591
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
28857 openapi include exclude #29769
base: integration
Are you sure you want to change the base?
28857 openapi include exclude #29769
Conversation
Code analysis and actionsDO NOT DELETE THIS COMMENT.
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ID review comments attached; there appears to be a placeholder reminder in a couple of spots, so when that is completed I will review that text.
If you have questions or concerns about any of the suggested changes, please let me know.
I will add the "ID reviewed" label when we agree on changes. Thanks!
|
||
include=MicroProfile OpenAPI included modules. | ||
include=A comma separated list of modules that will be included in the OpenAPI document. Modules are in the format of {ApplicationName}/{ModuleName} or {ApplicationName} which refer to to <TODO, sort out the confusion of names>. An ApplicationName which does not specify a module will include all modules in the application. The special value "all" will include all available modules. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Following appears to be a note that further work needs to be done:
to to <TODO, sort out the confusion of names>
Same appears in line 32
Change
comma separated
to
comma-separated
Change
that will be included
to
that are to be included
Change
An ApplicationName which
to
An ApplicationName that
Change
will include all modules
to
includes all modules
Change
The special value "all" will include
to
The "all" special value includes
|
||
exclude=MicroProfile OpenAPI excluded modules. | ||
exclude=A comma separated list of that will be excluded from the OpenAPI document. Modules are in the format of {ApplicationName}/{ModuleName} or {ApplicationName} which refer to to <TODO, sort out the confusion of names>. An ApplicationName which does not specify a module will exclude all modules in the application. An exclude takes precedent over an include. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Following appears to be a note that further work needs to be done:
to to <TODO, sort out the confusion of names>
Change
A comma separated list of that will be excluded
to
A comma-separated list of modules that are excluded
Change
An exclude takes precedent over an include.
to
Exclusion supersedes inclusion.
4595c11
to
6af9c5b
Compare
#build |
Your personal pipeline request is at https://libh-proxy1.fyre.ibm.com/cognitive/pipelineAnalysis.html?uuid=8008f055-9474-4534-83f0-158272300a50 Target locations of links might be accessible only to IBM employees. |
269270e
to
5ce8c23
Compare
404b48c
to
e5127e8
Compare
#build |
Your personal pipeline request is at https://libh-proxy1.fyre.ibm.com/cognitive/pipelineAnalysis.html?uuid=b9f42711-27e0-4f75-ba46-0a8d69789982 Target locations of links might be accessible only to IBM employees. |
Your personal build request is at https://wasrtc.hursley.ibm.com:9443/jazz/resource/itemOid/com.ibm.team.build.BuildResult/_IoGZUIbrEe-vDLerZarrlA Target locations of links might be accessible only to IBM employees. |
e5127e8
to
199d8a3
Compare
...0.internal/src/io/openliberty/microprofile/openapi20/internal/ModuleSelectionConfigImpl.java
Show resolved
Hide resolved
...0.internal/src/io/openliberty/microprofile/openapi20/internal/ModuleSelectionConfigImpl.java
Show resolved
Hide resolved
...0.internal/src/io/openliberty/microprofile/openapi20/internal/ModuleSelectionConfigImpl.java
Show resolved
Hide resolved
...0.internal/src/io/openliberty/microprofile/openapi20/internal/ModuleSelectionConfigImpl.java
Show resolved
Hide resolved
...api.2.0.internal/src/io/openliberty/microprofile/openapi20/internal/merge/ModelEquality.java
Show resolved
Hide resolved
...napi.2.0.internal/src/io/openliberty/microprofile/openapi20/internal/utils/OpenAPIUtils.java
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Remove this.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would suggest moving this change into a separate PR as it's the only change in fattest.simplicity.
Changing any of the core test framework projects will mean your personal builds need to run every bucket, rather than only the ones which use your feature and therefore take a while longer to complete.
I wouldn't shy away from making changes to the test framework if they're needed for your PR, but if it's just doc changes like this I'd move it to its own PR to avoid the larger PB.
public static interface OpenAPIAppConfigListener extends Comparable<OpenAPIAppConfigListener> { | ||
public void processConfigUpdate(); | ||
|
||
public int getConfigListenerPriority(); | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This interface needs documented.
If it implements Comparable
, it needs to also document how two objects are compared and whether that comparison is consistent with equals
. However, in this case I think it would make more sense not to make this implement Comparable
, and instead use a Comparator
which compares based on the value returned by getConfigListenerPriority()
.
In addition, please document whether listeners are called with the largest priority first or the smallest priority first.
Your personal build request is at https://wasrtc.hursley.ibm.com:9443/jazz/resource/itemOid/com.ibm.team.build.BuildResult/_NFwGMIc0Ee-vDLerZarrlA Target locations of links might be accessible only to IBM employees. |
The build benjamin-confino-29769-20241010-0344 For help analyzing your personal build, go to https://libh-proxy1.fyre.ibm.com/cognitive/buildAnalysis.html?uuid=_IoGZUIbrEe-vDLerZarrlA |
# Copyright (c) 2020, 2024 IBM Corporation and others. | ||
# Copyright (c) 2020, 2022 IBM Corporation and others. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Incorrect copyright date change?
@SuppressWarnings("unused") | ||
private OpenAPIAppConfigProvider openAPIAppConfigProvider; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Remove SuppressWarnings
@SuppressWarnings("unused") | ||
private OpenAPIAppConfigProvider openAPIAppConfigProvider; | ||
|
||
@Reference(cardinality = ReferenceCardinality.MANDATORY, policy = ReferencePolicy.STATIC, unbind = "unbindAppConfigListener") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
unbind
is not necessary here, it gets inferred based on the method names.
public void bindAppConfigListener(OpenAPIAppConfigProvider openAPIAppConfigProvider) { | ||
this.openAPIAppConfigProvider = openAPIAppConfigProvider; | ||
openAPIAppConfigProvider.registerAppConfigListener(this); | ||
} | ||
|
||
public void unbindAppConfigListener(OpenAPIAppConfigProvider openAPIAppConfigProvider) { | ||
openAPIAppConfigProvider.unregisterAppConfigListener(this); | ||
this.openAPIAppConfigProvider = null; | ||
} | ||
|
||
@Reference | ||
public void bindOpenAPIAppConfigProvider(OpenAPIAppConfigProvider openAPIAppConfigProvider) { | ||
this.openAPIAppConfigProvider = null; | ||
openAPIAppConfigProvider.registerAppConfigListener(this); | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This looks like you have two references to OpenAPIAppConfigProvider
, but the first one is named appConfigListener
.
@Reference(cardinality = ReferenceCardinality.MANDATORY, policy = ReferencePolicy.STATIC, unbind = "unbindAppConfigListener") | ||
public void bindAppConfigListener(OpenAPIAppConfigProvider openAPIAppConfigProvider) { | ||
this.configFromServerXML = openAPIAppConfigProvider; | ||
openAPIAppConfigProvider.registerAppConfigListener(this); | ||
} | ||
|
||
public void unbindAppConfigListener(OpenAPIAppConfigProvider openAPIAppConfigProvider) { | ||
openAPIAppConfigProvider.unregisterAppConfigListener(this); | ||
this.configFromServerXML = null; | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Since this is a static reference, it might be easier to annotate the field and register as a listener in activate/deactivate.
Technically here we're registering as a listener before we're activated.
@Override | ||
public synchronized void processConfigUpdate() { | ||
//Reset everything for the next lazyInit | ||
hasBeenInit = false; | ||
isAll = false; | ||
isFirst = false; | ||
serverxmlmode = false; | ||
included = null; | ||
excluded = null; | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We have an issue with thread safety because we have this method to reset everything and re-initialize on next use.
If lazyInit
was only called once it would be sufficient for only the lazyInit
method to be synchronized as long as every method which needs to access any of these values calls lazyInit
before doing so.
However, since we have the ability to reset, every use of any of these variables must be synchronized, otherwise they could be reset in the middle of a method which uses them.
To avoid this, I would suggest:
- add a new static inner class to hold these values - let's call it
ConfigValues
- rather than having a
lazyInit
method which sets all these fields, have a method which creates, caches and returns aConfigValues
object- all other code which needs the
ConfigValues
object must retrieve it via this method
- all other code which needs the
processConfigUpdate
then only needs to delete the cachedConfigValues
object- this method and the method to get
ConfigValues
may need synchronization or correct use ofvolatile
, but everything else should be correct without synchronization
|
||
System.out.println("GREPGREP " + a + " " + b + " " + comparator.getClass().toGenericString()); | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Remove this (or make it proper entry trace)
@Override | ||
public String toString() { | ||
lazyInit(); | ||
StringBuilder sb = new StringBuilder("Module Selection Config["); | ||
if (isFirst) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm a bit concerned about toString()
doing initialization here. That's the sort of thing that can cause enabling trace to change behaviour...
Could we make it so that toString()
reports "Uninitialized Config` or something similar if initialization hasn't happened yet?
if (serverxmlmode && ProductInfo.getBetaEdition()) { | ||
for (Iterator<ModuleName> iterator = includedNotYetSeen.iterator(); iterator.hasNext();) { | ||
ModuleName moduleName = iterator.next(); | ||
for (ModuleInfo moduleInfo : moduleInfos) { | ||
if (matches(moduleName, moduleInfo, false)) { | ||
includedNotYetSeenButSeenUnderOldNaming.add(moduleName); | ||
iterator.remove(); | ||
break; | ||
} | ||
} | ||
} | ||
|
||
for (String unmatchedInclude : includedNotYetSeenButSeenUnderOldNaming.stream().map(ModuleName::toString).collect(toList())) { | ||
Tr.warning(tc, MessageConstants.OPENAPI_USING_WRONG_NAME_SOURCE_CWWKO1680W, Constants.MERGE_INCLUDE_CONFIG, unmatchedInclude); | ||
} | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we do this check that the may have used the wrong name for excluded apps as well as included apps please?
(The other check for a name that's missing should remain as only checking the included apps).
....2.0.internal/src/io/openliberty/microprofile/openapi20/internal/utils/MessageConstants.java
Show resolved
Hide resolved
Co-authored-by: Andrew Rouse <anrouse@uk.ibm.com>
Map<String, ApplicationRecord> oldApps = applications; | ||
applications = new LinkedHashMap<>(); | ||
for (ApplicationRecord record : oldApps.values()) { | ||
//Add application uses config to decide if it creates and registers any providers in ApplicationInfo | ||
//Rather than map from the old state to the new state when the config changes, KISS and start again. | ||
addApplication(record.info); | ||
} | ||
cachedProvider = null; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In general, we use synchronized (this)
in the method body rather than synchronized
in the signature so that the generated trace is outside the synchronized block, making it easier to debug lock contention.
Since addApplication
can take a while to complete, I think we should do that here.
public int getConfigListenerPriority() { | ||
return 2; | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Comment that this is to ensure we're called after the ModuleSelectionConfig
@Modified | ||
protected void modified(Map<String, Object> properties) { | ||
if (ProductInfo.getBetaEdition()) { | ||
if (TraceComponent.isAnyTracingEnabled() && tc.isEventEnabled()) { | ||
Tr.event(tc, "Processing update to server.xml"); | ||
} | ||
processModuleConfig(properties); | ||
for (OpenAPIAppConfigListener listener : openAPIAppConfigListeners) { | ||
listener.processConfigUpdate(); | ||
} | ||
} | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This class only cares about a subset of properties from the <mpOpenAPI>
config element. If someone changes a config value that isn't related to module inclusion/exclusion then we shouldn't call the listeners since re-generating all the OpenAPI documents is a relatively expensive operation.
<?xml version="1.0" encoding="UTF-8"?> | ||
<!-- Used for tests that need a non-default deployment name for a war --> | ||
<web-app version="3.0" | ||
xmlns="http://java.sun.com/xml/ns/javaee" | ||
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" | ||
xsi:schemaLocation="http://java.sun.com/xml/ns/javaee web-app_3_0.xsd" | ||
id="testID"> | ||
<module-name>nameFromWar</module-name> | ||
</web-app> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please use WebXmlAsset
instead of having a web.xml file included manually.
server.swapInServerXMLFromPublish(SERVER_XML_NAME); | ||
server.waitForStringInLogUsingMark("CWWKG0017I"); //The server configuration was successfully updated |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
server.swapInServerXMLFromPublish(SERVER_XML_NAME); | |
server.waitForStringInLogUsingMark("CWWKG0017I"); //The server configuration was successfully updated | |
server.setMarkToEndOfLog(); | |
server.swapInServerXMLFromPublish(SERVER_XML_NAME); | |
server.waitForConfigUpdateInLogUsingMark(null); |
I think we need to manually set the mark ourselves before starting the update if we want to use it afterwards.
Also there's a specific method for waiting for config updates.
But also, I would prefer it if we had a method to update the config that made it more obvious in the test what the value of the config is so you don't have to jump between lots of files to understand the test.
E.g. something that let you make calls like this:
updateConfig(server, includeApp("all"));
updateConfig(server, includeApp("app1"));
updateConfig(server,
includeApp("app1"),
includeApp("app2"),
excludeModule("app2/war1"));
or like this:
openapiConfig(server).includeApp("all").apply();
openapiConfig(server).includeApp("app1").apply();
openapiConfig(server).includeApp("app1")
.includeApp("app2")
.excludeModule("app2/war1")
.apply();
Edit: I've since commented on MergeServerXmlTest
that we should probably cover the same cases that are covered in MergeConfigTest
. Those tests could be conducted using dynamic updates, though I'm unsure whether it would cover the same cases you have in this class where there are apps deployed and you update the config.
I wanted to update this message though in case you were working through and implemented this suggestion before seeing the other comment.
server.swapInServerXMLFromPublish(APP1_XML_NAME); | ||
server.waitForStringInLogUsingMark("CWWKG0017I"); //The server configuration was successfully updated |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
server.swapInServerXMLFromPublish(APP1_XML_NAME); | |
server.waitForStringInLogUsingMark("CWWKG0017I"); //The server configuration was successfully updated | |
server.setMarkToEndOfLog(); | |
server.swapInServerXMLFromPublish(APP1_XML_NAME); | |
server.waitForConfigUpdateInLogUsingMark(null); |
...rty.microprofile.openapi.2.0.internal_fat/publish/servers/OpenAPIMergeTestServer/jvm.options
Show resolved
Hide resolved
@Test | ||
public void tesWarningWhenMatchesOldNamingScheme() throws Exception { | ||
|
||
EnterpriseArchive ear = null; | ||
WebArchive war1 = ShrinkWrap.create(WebArchive.class, "test1.war") | ||
.addClasses(DeploymentTestApp.class, DeploymentTestResource.class); | ||
|
||
WebArchive war2 = ShrinkWrap.create(WebArchive.class, "test2.war") | ||
.addClasses(DeploymentTestApp.class, DeploymentTestResource.class); | ||
|
||
WebArchive war3 = ShrinkWrap.create(WebArchive.class, "test3.war") | ||
.addClasses(DeploymentTestApp.class, DeploymentTestResource.class) | ||
.addAsWebInfResource(DeploymentTestApp.class.getPackage(), "web.xml", "web.xml"); | ||
|
||
ear = ShrinkWrap.create(EnterpriseArchive.class, "testEar.ear") | ||
.addAsModules(war1, war2, war3); | ||
ShrinkHelper.exportAppToServer(server, ear, SERVER_ONLY, DISABLE_VALIDATION); | ||
|
||
server.setAdditionalSystemProperties( | ||
Collections.singletonMap("mp_openapi_extensions_liberty_merged_include", "testEar")); | ||
server.startServer(); | ||
|
||
//Test that a warning message saying we cant find the app but could under the pre OpenAPI 4.0 rules | ||
assertThat( | ||
server.findStringsInLogs( | ||
"CWWKO1680W"), | ||
hasSize(1)); | ||
|
||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This test method contains lots of things are aren't relevant and hides or doesn't explain everything important about the test.
I don't think it's relevant that the app is an ear, or that it contains three modules, or that one of the modules contains a web.xml. Please simplify this.
I'm unsure if the mp_openapi_extensions_liberty_merged_include
property is important - this looks to be identical to the config in the server.xml? Should it be removed?
The server.xml gives a custom name to the application and adds configuration which includes the app using the wrong name. These details should either be explained in a comment or be moved into the test method.
The error message appears because the server.xml name is "nameFromServerXml" while the deployment descriptor name is "testEar" because there's no deployment descriptor so it defaults to the archive name without the extension. This detail should be explained in a comment.
There's no description of what the CWWKO1680W
message is or why we expect it.
//Test that a warning message saying we cant find the app but could under the pre OpenAPI 4.0 rules | ||
assertThat( | ||
server.findStringsInLogs( | ||
"CWWKO1680W"), | ||
hasSize(1)); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The warning message is wrong, the rules differ depending on whether the config came from server.xml or MP Config, not whether we're using mpOpenAPI-4.0
or an earlier feature.
We need to ensure the message has been substituted correctly as the name of the message also includes CWWKO1680W
. Since this message has placeholders, we should include the expected values for those placeholders in the regex.
I'm not sure if this message is guaranteed to have been written by the time that server.startServer()
returns. We should use server.waitForStringInLogs
here unless we're certain that waiting is never necessary.
ear = ShrinkWrap.create(EnterpriseArchive.class, "testEar.ear") | ||
.addAsModules(war1, war2, war3); | ||
ShrinkHelper.exportAppToServer(server, ear, SERVER_ONLY, DISABLE_VALIDATION); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We're disabling validation here because the name of the app doesn't match the archive name. That's probably worth a comment.
Since we do expect the app to start, we should do the validation manually by calling server.addInstalledAppForValidation("nameFromServerXml");
ourselves after exporting the app.
@RunWith(FATRunner.class) | ||
public class MergeServerXMLTest { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This test class appears to have been adapted from MergeTest
?
MergeTest
is meant to test that we correctly merge documents, all tests run with a simple config which merges all apps.
I don't think we need to repeat the same tests with with the same basic config in server.xml.
MergeConfigTest
is our main test for the merging config - we should include all the include and exclude cases from there in our server.xml config testing. Since the server.xml config is dynamic, we won't have to restart the server between each test like we do in MergeConfigTest
.
The new cases that you've covered in this class where you're updating the config are obviously still valuable.
@@ -48,6 +48,7 @@ | |||
import io.openliberty.microprofile.openapi20.internal.cache.ConfigSerializer; | |||
import io.openliberty.microprofile.openapi20.internal.services.ConfigFieldProvider; | |||
import io.openliberty.microprofile.openapi20.internal.services.ModelGenerator; | |||
import io.openliberty.microprofile.openapi20.internal.services.ModuleSelectionConfig; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
spurious import
for (Iterator<ModuleName> iterator = includedNotYetSeen.iterator(); iterator.hasNext();) { | ||
ModuleName moduleName = iterator.next(); | ||
for (ModuleInfo moduleInfo : moduleInfos) { | ||
if (matches(moduleName, moduleInfo, false)) { | ||
includedNotYetSeenButSeenUnderOldNaming.add(moduleName); | ||
iterator.remove(); | ||
break; | ||
} | ||
} | ||
} | ||
|
||
for (String unmatchedInclude : includedNotYetSeenButSeenUnderOldNaming.stream().map(ModuleName::toString).collect(toList())) { | ||
Tr.warning(tc, MessageConstants.OPENAPI_USING_WRONG_NAME_SOURCE_CWWKO1680W, Constants.MERGE_INCLUDE_CONFIG, unmatchedInclude); | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This code needs updating because the placeholders in the message have been updated.
release bug
label if applicable: https://github.com/OpenLiberty/open-liberty/wiki/Open-Liberty-Conventions).resolves #28857
This PR