Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Xamarin.Android.Build.Tasks] Fix @(JavaDocJar) #5479

Merged
merged 1 commit into from
Jan 9, 2021

Conversation

jonpryor
Copy link
Member

@jonpryor jonpryor commented Jan 7, 2021

Fixes: #2745
Fixes: #5474

Issue #2745 noted that @(JavaDocJar) didn't work; attempting to use
it would result in an MSB3375 error:

…/Xamarin.Android.Bindings.Documentation.targets(31,5): error MSB3375: The file "obj/Debug/javadocs/glide-3.7.0-javadoc.stamp" does not exist.

This issue was subsequently hidden by 380e95e, which disabled the
_ExtractJavaDocJars target unless JDK 1.8 was being used.

There were two problems with 380e95e:

  1. The _ExtractJavaDocJars target didn't need to be disabled!
    It just unzips the .jar; it does not require JDK 1.8.

  2. The check for JDK 1.8 was bad, and was never True.

The bad JDK 1.8 check was later removed in a7413a2, which "fixed"
(2), but a7413a2 continued to disable _ExtractJavaDocJars,
preventing @(JavaDocJar) from being used.

The only documentation-related functionality that needed to be
disabled because of JDK 11 was @(JavaSourceJar), which got a better
fix in commits a7413a2 and 0e95ec7.

Fix the _ExtractJavaDocJars target so that it properly runs when
there are any @(JavaDocJar) items, then fix the _ExtractJavaDocJars
target so that it doesn't emit an MSB3375 error.

Note: @(JavaDocJar) is still problematic: it involves parsing
Javadoc HTML, which contains numerous "dialects" (it's why the JDK 11
commit skipped @(JavaSourceJar) support: trying to update our HTML
scrapers was "too fiddly").

Deprecate @(JavaDocJar), and remove
Xamarin.Android.Bindings.Documentation.targets from the .NET 6
installation packages. Support for $(_UseLegacyJavadocImport)=False
won't exist for .NET 6; @(JavaSourceJar) support in .NET t will
only use java-source-utils.jar (a7413a2).

Update the BindingBuildTest.cs infrastructure so that instead of
having .jar files encoded in Base64, we instead use
@(EmbeddedResource)s for the .jar files. A new UpdateResources
target will update the appropriate .jar files.

Comment on lines 40 to 48
using var s = typeof (InlineData).Assembly.GetManifestResourceStream (name);
var data = new List<byte> ();
var buf = new byte[1024];
int r;
while ((r = s.Read (buf, 0, buf.Length)) > 0) {
for (int i = 0; i < r; ++i)
data.Add (buf [i]);
}
return data.ToArray ();
Copy link
Member

@jonathanpeppers jonathanpeppers Jan 7, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could this do something like this instead?

using var s = typeof (InlineData).Assembly.GetManifestResourceStream (name);
var buf = new byte [s.Length];
s.Read (buf, 0, buf.Length);
return buf;

I think a List<T> would potentially resize the underlying array a couple times.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That may work, but I don't know what the semantics of the stream returned by GetManifestResourceStream() is. Is it possible that for "large" files Stream.Read() won't read in everything?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe this one then?

using var s = typeof (InlineData).Assembly.GetManifestResourceStream (name);
using var m = new MemoryStream ();
s.CopyTo (m);
return m.ToArray ();

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We might want to pass stream length to memory stream constructor if we want to avoid resizes during the process.

Comment on lines 12 to 14
public static byte[] JavaSourceJarTestJar = GetResourceData ("javasourcejartest.jar");
public static byte[] JavaSourceJarSourcesTestJar = GetResourceData ("javasourcejartest-sources.jar");
public static byte[] JavaSourceJarJavadocTestJar = GetResourceData ("javasourcejartest-javadoc.jar");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we make these properties, so they don't load the files unless the property is called? I'm thinking if I Debug a single test, it might load all these files.

It could do something like:

static byte[] javaSourceJarJavadocTestJar;
public  static  byte[]  JavaSourceJarJavadocTestJar => javaSourceJarJavadocTestJar ??= GetResourceData ("javasourcejartest-javadoc.jar");

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to worry about thread safety?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I guess many of the tests use NUnit's parallel feature.

@@ -8,60 +9,11 @@ namespace Xamarin.Android.Build.Tests
{
static class InlineData
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bit of a nit pick, but should this still be called InLineData? cos its not really inline its embedded ?

{
using var s = typeof (InlineData).Assembly.GetManifestResourceStream (name);
var data = new byte [s.Length];
s.Read (data, 0, data.Length);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This can return after reading less than data.Length bytes. https://docs.microsoft.com/en-us/dotnet/api/system.io.stream.read?view=net-5.0

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixes: dotnet#2745
Fixes: dotnet#5474

Issue dotnet#2745 noted that `@(JavaDocJar)` didn't work; attempting to use
it would result in an MSB3375 error:

	…/Xamarin.Android.Bindings.Documentation.targets(31,5): error MSB3375: The file "obj/Debug/javadocs/glide-3.7.0-javadoc.stamp" does not exist.

This issue was subsequently hidden by 380e95e, which disabled the
`_ExtractJavaDocJars` target unless JDK 1.8 was being used.

There were two problems with 380e95e:

 1. The `_ExtractJavaDocJars` target didn't need to be disabled!
    It just unzips the `.jar`; it does not require JDK 1.8.

 2. The check for JDK 1.8 was bad, and was *never* True.

The bad JDK 1.8 check was later removed in a7413a2, which "fixed"
(2), but a7413a2 continued to disable `_ExtractJavaDocJars`,
preventing `@(JavaDocJar)` from being used.

The only documentation-related functionality that needed to be
disabled because of JDK 11 was `@(JavaSourceJar)`, which got a better
fix in commits a7413a2 and 0e95ec7.

Fix the `_ExtractJavaDocJars` target so that it properly runs when
there are any `@(JavaDocJar)` items, then fix the `_ExtractJavaDocJars`
target so that it doesn't emit an MSB3375 error.

Note: `@(JavaDocJar)` is still problematic: it involves parsing
Javadoc HTML, which contains numerous "dialects" (it's why the JDK 11
commit skipped `@(JavaSourceJar)` support: trying to update our HTML
scrapers was "too fiddly").

Deprecate `@(JavaDocJar)`, and remove
`Xamarin.Android.Bindings.Documentation.targets` from the .NET 6
installation packages.  Support for `$(_UseLegacyJavadocImport)`=False
won't exist for .NET 6; `@(JavaSourceJar)` support in .NET t will
*only* use `java-source-utils.jar` (a7413a2).

Update the `BindingBuildTest.cs` infrastructure so that instead of
having `.jar` files encoded in Base64, we instead use
`@(EmbeddedResource)`s for the `.jar` files.  A new `UpdateResources`
target will update the appropriate `.jar` files.
@@ -239,7 +239,7 @@
<_MSBuildFiles Include="$(MSBuildSrcDir)\Xamarin.Android.Bindings.targets" ExcludeFromAndroidNETSdk="true" />
<_MSBuildFiles Include="$(MSBuildSrcDir)\Xamarin.Android.Bindings.ClassParse.targets" />
<_MSBuildFiles Include="$(MSBuildSrcDir)\Xamarin.Android.Bindings.Core.targets" />
<_MSBuildFiles Include="$(MSBuildSrcDir)\Xamarin.Android.Bindings.Documentation.targets" />
<_MSBuildFiles Include="$(MSBuildSrcDir)\Xamarin.Android.Bindings.Documentation.targets" ExcludeFromAndroidNETSdk="true" />
Copy link
Member

@jonathanpeppers jonathanpeppers Jan 8, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, this file wasn't even being imported in .NET 6. So this is safe to remove.

It is only imported for "legacy" binding projects:

https://github.com/xamarin/xamarin-android/blob/a7413a2389886082c3d3422c50a7e6cc84f43d8f/src/Xamarin.Android.Build.Tasks/Xamarin.Android.Bindings.targets#L154

@jonpryor jonpryor merged commit 16e0aa2 into dotnet:master Jan 9, 2021
jonpryor added a commit to jonpryor/xamarin-android that referenced this pull request Jan 15, 2021
Fixes: dotnet#2745

Issue dotnet#2745 noted that `@(JavaDocJar)` didn't work; attempting to use
it would result in an MSB3375 error:

	…/Xamarin.Android.Bindings.Documentation.targets(31,5): error MSB3375: The file "obj/Debug/javadocs/glide-3.7.0-javadoc.stamp" does not exist.

This issue was subsequently hidden by 380e95e, which disabled the
`_ExtractJavaDocJars` target unless JDK 1.8 was being used.

There were two problems with 380e95e:

 1. The `_ExtractJavaDocJars` target didn't need to be disabled!
    It just unzips the `.jar`; it does not require JDK 1.8.

 2. The check for JDK 1.8 was bad, and was *never* True.

The only documentation-related functionality that needed to be
disabled because of JDK 11 was `@(JavaSourceJar)`, which got a better
fix in commits a7413a2 and 0e95ec7 [not relevant for d16-9].

Fix the `_ExtractJavaDocJars` target so that it properly runs when
there are any `@(JavaDocJar)` items, then fix the `_ExtractJavaDocJars`
target so that it doesn't emit an MSB3375 error.

Note: `@(JavaDocJar)` is still problematic: it involves parsing
Javadoc HTML, which contains numerous "dialects" (it's why the JDK 11
commit skipped `@(JavaSourceJar)` support: trying to update our HTML
scrapers was "too fiddly").

Deprecate `@(JavaDocJar)`.

Fix the `$(_JavadocSupported)` test so that it is no longer always
false, by moving it's definition into a `_GetJavadocSupported`
target.  This allows `$(_JdkVersion)` to be interpreted *after*
it is set.

Update the `BindingBuildTest.cs` infrastructure so that instead of
having `.jar` files encoded in Base64, we instead use
`@(EmbeddedResource)`s for the `.jar` files.  A new `UpdateResources`
target will update the appropriate `.jar` files.
jonpryor added a commit to jonpryor/xamarin-android that referenced this pull request Jan 16, 2021
Fixes: dotnet#2745

Issue dotnet#2745 noted that `@(JavaDocJar)` didn't work; attempting to use
it would result in an MSB3375 error:

	…/Xamarin.Android.Bindings.Documentation.targets(31,5): error MSB3375: The file "obj/Debug/javadocs/glide-3.7.0-javadoc.stamp" does not exist.

This issue was subsequently hidden by 380e95e, which disabled the
`_ExtractJavaDocJars` target unless JDK 1.8 was being used.

There were two problems with 380e95e:

 1. The `_ExtractJavaDocJars` target didn't need to be disabled!
    It just unzips the `.jar`; it does not require JDK 1.8.

 2. The check for JDK 1.8 was bad, and was *never* True.

The only documentation-related functionality that needed to be
disabled because of JDK 11 was `@(JavaSourceJar)`, which got a better
fix in commits a7413a2 and 0e95ec7 [not relevant for d16-9].

Fix the `_ExtractJavaDocJars` target so that it properly runs when
there are any `@(JavaDocJar)` items, then fix the `_ExtractJavaDocJars`
target so that it doesn't emit an MSB3375 error.

Note: `@(JavaDocJar)` is still problematic: it involves parsing
Javadoc HTML, which contains numerous "dialects" (it's why the JDK 11
commit skipped `@(JavaSourceJar)` support: trying to update our HTML
scrapers was "too fiddly").

Deprecate `@(JavaDocJar)`.

Fix the `$(_JavadocSupported)` test so that it is no longer always
false, by moving it's definition into a `_GetJavadocSupported`
target.  This allows `$(_JdkVersion)` to be interpreted *after*
it is set.

Update the `BindingBuildTest.cs` infrastructure so that instead of
having `.jar` files encoded in Base64, we instead use
`@(EmbeddedResource)`s for the `.jar` files.  A new `UpdateResources`
target will update the appropriate `.jar` files.
jonpryor added a commit that referenced this pull request Jan 17, 2021
Fixes: #2745

Issue #2745 noted that `@(JavaDocJar)` didn't work; attempting to use
it would result in an MSB3375 error:

	…/Xamarin.Android.Bindings.Documentation.targets(31,5): error MSB3375: The file "obj/Debug/javadocs/glide-3.7.0-javadoc.stamp" does not exist.

This issue was subsequently hidden by 380e95e, which disabled the
`_ExtractJavaDocJars` target unless JDK 1.8 was being used.

There were two problems with 380e95e:

 1. The `_ExtractJavaDocJars` target didn't need to be disabled!
    It just unzips the `.jar`; it does not require JDK 1.8.

 2. The check for JDK 1.8 was bad, and was *never* True.

The only documentation-related functionality that needed to be
disabled because of JDK 11 was `@(JavaSourceJar)`, which got a better
fix in commits a7413a2 and 0e95ec7 [not relevant for d16-9].

Fix the `_ExtractJavaDocJars` target so that it properly runs when
there are any `@(JavaDocJar)` items, then fix the `_ExtractJavaDocJars`
target so that it doesn't emit an MSB3375 error.

Note: `@(JavaDocJar)` is still problematic: it involves parsing
Javadoc HTML, which contains numerous "dialects" (it's why the JDK 11
commit skipped `@(JavaSourceJar)` support: trying to update our HTML
scrapers was "too fiddly").

Deprecate `@(JavaDocJar)`.

Fix the `$(_JavadocSupported)` test so that it is no longer always
false, by moving it's definition into a `_GetJavadocSupported`
target.  This allows `$(_JdkVersion)` to be interpreted *after*
it is set.

Update the `BindingBuildTest.cs` infrastructure so that instead of
having `.jar` files encoded in Base64, we instead use
`@(EmbeddedResource)`s for the `.jar` files.  A new `UpdateResources`
target will update the appropriate `.jar` files.
@github-actions github-actions bot locked and limited conversation to collaborators Jan 26, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
4 participants