Skip to content

Commit

Permalink
Merge remote-tracking branch 'upstream/main' into mod-opt-fix
Browse files Browse the repository at this point in the history
  • Loading branch information
TIHan committed May 18, 2022
2 parents 574c17d + 0a62b65 commit 7628c7b
Show file tree
Hide file tree
Showing 617 changed files with 10,115 additions and 39,112 deletions.
453 changes: 399 additions & 54 deletions .github/fabricbot.json

Large diffs are not rendered by default.

7 changes: 5 additions & 2 deletions docs/design/coreclr/botr/dac-notes.md
Original file line number Diff line number Diff line change
Expand Up @@ -137,12 +137,15 @@ Even better, in a DAC build, the DAC will automatically marshal variables, data

`PTR` types defined with `DPTR` are the most common in the runtime, but we also have `PTR` types for global and static pointers, restricted-use arrays, pointers to variable-sized objects, and pointers to classes with virtual functions that we may need to call from mscordacwks.dll (msdaccore.dll). Most of these are rare and you can refer to [daccess.h][daccess.h] to learn more about them if you need them.

The `GPTR` and `VPTR` macros are common enough to warrant special mention here. Both the way we use these and their external behavior is quite similar to `DPTR`s. Again, marshaling is automatic and transparent. The `VPTR` macro declares a marshaled pointer type for a class with virtual functions. This special macro is necessary because the virtual function table is essentially an implicit extra field. The DAC has to marshal this separately, since the function addresses are all target addresses that the DAC must convert to host addresses. Treating these classes in this way means that the DAC automatically instantiates the correct implementation class, making casts between base and derived types unnecessary. When you declare a `VPTR` type, you must also list it in [vptr_list.h][vptr_list.h]. `__GlobalPtr` types provide base functionality to marshal both global variables and static data members through the `GPTR`, `GVAL`, `SPTR` and `SVAL` macros. The implementation of global variables is almost identical to that of static fields (both use the `__GlobalPtr` class) and require the addition of an entry in [dacvars.h][dacvars.h]. The comments in [daccess.h][daccess.h] and [dacvars.h][dacvars.h] provide more details about declaring these types.
The `GPTR` and `VPTR` macros are common enough to warrant special mention here. Both the way we use these and their external behavior is quite similar to `DPTR`s. Again, marshaling is automatic and transparent. The `VPTR` macro declares a marshaled pointer type for a class with virtual functions. This special macro is necessary because the virtual function table is essentially an implicit extra field. The DAC has to marshal this separately, since the function addresses are all target addresses that the DAC must convert to host addresses. Treating these classes in this way means that the DAC automatically instantiates the correct implementation class, making casts between base and derived types unnecessary. When you declare a `VPTR` type, you must also list it in [vptr_list.h][vptr_list.h]. `__GlobalPtr` types provide base functionality to marshal both global variables and static data members through the `GPTR`, `GVAL`, `SPTR` and `SVAL` macros. The implementation of global variables is almost identical to that of static fields (both use the `__GlobalPtr` class) and require the addition of an entry in [dacvars.h][dacvars.h]. Global functions used in the DAC do not require macros at the implementation site, but they must be declared in the [gfunc_list.h][gfunc_list.h] header to have their addresses be automatically available to the DAC. The comments in [daccess.h][daccess.h] and [dacvars.h][dacvars.h] provide more details about declaring these types.

[dacvars.h]: https://github.com/dotnet/runtime/blob/main/src/coreclr/inc/dacvars.h
[vptr_list.h]: https://github.com/dotnet/runtime/blob/main/src/coreclr/inc/vptr_list.h
[gfunc_list.h]: https://github.com/dotnet/runtime/blob/main/src/coreclr/inc/gfunc_list.h

Global and static values and pointers are interesting because they form the entry points to the target address space (all other uses of the DAC require you to have a target address already). Many of the globals in the runtime are already DACized. It occasionally becomes necessary to make a previously un-DACized (or a newly introduced) global available to the DAC. By using the appropriate macros and [dacvars.h][dacvars.h] entry, you enable a post-build step (DacTableGen.exe run by the build in ndp\clr\src\dacupdatedll) to save the address of the global (from clr.pdb) into a table that is embedded into mscordacwks.dll. The DAC uses this table at run-time to determine where to look in the target address space when the code accesses a global.
Global and static values and pointers are interesting because they form the entry points to the target address space (all other uses of the DAC require you to have a target address already). Many of the globals in the runtime are already DACized. It occasionally becomes necessary to make a previously un-DACized (or a newly introduced) global available to the DAC. By using the appropriate macros and [dacvars.h][dacvars.h] entry, you enable the dac table machinery (in [dactable.cpp]) to save the address of the global into a table that is exported from coreclr.dll. The DAC uses this table at run-time to determine where to look in the target address space when the code accesses a global.

[dactable.cpp]: https://github.com/dotnet/runtime/blob/main/src/coreclr/debug/ee/dactable.cpp

### VAL Types

Expand Down
3 changes: 3 additions & 0 deletions docs/workflow/building/coreclr/nativeaot.md
Original file line number Diff line number Diff line change
Expand Up @@ -77,6 +77,9 @@ Sometimes it's handy to be able to rebuild the managed test manually or run the

For more advanced scenarios, look for at [Building Test Subsets](../../testing/coreclr/windows-test-instructions.md#building-test-subsets) and [Generating Core_Root](../../testing/coreclr/windows-test-instructions.md#generating-core_root)

### Running library tests
Build library tests by passing the `libs.tests` subset together with the `/p:TestNativeAot=true` to build the libraries, i.e. `clr.alljits+clr.tools+clr.nativeaotlibs+clr.nativeaotruntime+libs+libs.tests /p:TestNativeAot=true` together with the full arguments as specified [above](#building). Then, to run a specific library, go to the tests directory of the library and run the usual command to run tests for the library (see [Running tests for a single library](../../testing/libraries/testing.md#running-tests-for-a-single-library)) but add the `/p:TestNativeAot=true`, i.e. `dotnet.cmd build /t:Test /p:TestNativeAot=true`.

## Design Documentation

- [ILC Compiler Architecture](../../../design/coreclr/botr/ilc-architecture.md)
Expand Down
4 changes: 4 additions & 0 deletions eng/Signing.props
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,10 @@
<!-- apphost and comhost template files are not signed, by design. -->
<FileSignInfo Include="apphost.exe;singlefilehost.exe;comhost.dll" CertificateName="None" />

<!-- We don't need to code sign .js files because they are not used in Windows Script Host. -->
<!-- WARNING: Needs to happed outside of any target -->
<FileExtensionSignInfo Update=".js" CertificateName="None" />

<!-- Third-party components which should be signed. -->
<FileSignInfo Include="Newtonsoft.Json.dll" CertificateName="3PartySHA2" />
<FileSignInfo Include="Mono.Cecil.dll" CertificateName="3PartySHA2" />
Expand Down
1 change: 0 additions & 1 deletion eng/Tools.props
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,6 @@

<!-- excluded from source build -->
<ItemGroup Condition="'$(DotNetBuildFromSource)' != 'true'">
<PackageReference Include="Microsoft.DotNet.Build.Tasks.Feed" Version="$(MicrosoftDotNetBuildTasksFeedVersion)" />
<PackageReference Include="Microsoft.DotNet.VersionTools.Tasks" Version="$(MicrosoftDotNetVersionToolsTasksVersion)" />
</ItemGroup>

Expand Down
76 changes: 38 additions & 38 deletions eng/Version.Details.xml
Original file line number Diff line number Diff line change
Expand Up @@ -54,77 +54,77 @@
</Dependency>
</ProductDependencies>
<ToolsetDependencies>
<Dependency Name="Microsoft.DotNet.Arcade.Sdk" Version="7.0.0-beta.22255.2">
<Dependency Name="Microsoft.DotNet.Arcade.Sdk" Version="7.0.0-beta.22266.1">
<Uri>https://github.com/dotnet/arcade</Uri>
<Sha>ba1c3aff4be864c493031d989259ef92aaa23fc3</Sha>
<Sha>70d269dfe645525adb6836d25d8a97d7960eda1a</Sha>
</Dependency>
<Dependency Name="Microsoft.DotNet.Helix.Sdk" Version="7.0.0-beta.22255.2">
<Dependency Name="Microsoft.DotNet.Helix.Sdk" Version="7.0.0-beta.22266.1">
<Uri>https://github.com/dotnet/arcade</Uri>
<Sha>ba1c3aff4be864c493031d989259ef92aaa23fc3</Sha>
<Sha>70d269dfe645525adb6836d25d8a97d7960eda1a</Sha>
</Dependency>
<Dependency Name="Microsoft.DotNet.ApiCompat" Version="7.0.0-beta.22255.2">
<Dependency Name="Microsoft.DotNet.ApiCompat" Version="7.0.0-beta.22266.1">
<Uri>https://github.com/dotnet/arcade</Uri>
<Sha>ba1c3aff4be864c493031d989259ef92aaa23fc3</Sha>
<Sha>70d269dfe645525adb6836d25d8a97d7960eda1a</Sha>
</Dependency>
<Dependency Name="Microsoft.DotNet.GenAPI" Version="7.0.0-beta.22255.2">
<Dependency Name="Microsoft.DotNet.GenAPI" Version="7.0.0-beta.22266.1">
<Uri>https://github.com/dotnet/arcade</Uri>
<Sha>ba1c3aff4be864c493031d989259ef92aaa23fc3</Sha>
<Sha>70d269dfe645525adb6836d25d8a97d7960eda1a</Sha>
</Dependency>
<Dependency Name="Microsoft.DotNet.GenFacades" Version="7.0.0-beta.22255.2">
<Dependency Name="Microsoft.DotNet.GenFacades" Version="7.0.0-beta.22266.1">
<Uri>https://github.com/dotnet/arcade</Uri>
<Sha>ba1c3aff4be864c493031d989259ef92aaa23fc3</Sha>
<Sha>70d269dfe645525adb6836d25d8a97d7960eda1a</Sha>
</Dependency>
<Dependency Name="Microsoft.DotNet.XUnitExtensions" Version="7.0.0-beta.22255.2">
<Dependency Name="Microsoft.DotNet.XUnitExtensions" Version="7.0.0-beta.22266.1">
<Uri>https://github.com/dotnet/arcade</Uri>
<Sha>ba1c3aff4be864c493031d989259ef92aaa23fc3</Sha>
<Sha>70d269dfe645525adb6836d25d8a97d7960eda1a</Sha>
</Dependency>
<Dependency Name="Microsoft.DotNet.XUnitConsoleRunner" Version="2.5.1-beta.22255.2">
<Dependency Name="Microsoft.DotNet.XUnitConsoleRunner" Version="2.5.1-beta.22266.1">
<Uri>https://github.com/dotnet/arcade</Uri>
<Sha>ba1c3aff4be864c493031d989259ef92aaa23fc3</Sha>
<Sha>70d269dfe645525adb6836d25d8a97d7960eda1a</Sha>
</Dependency>
<Dependency Name="Microsoft.DotNet.Build.Tasks.Archives" Version="7.0.0-beta.22255.2">
<Dependency Name="Microsoft.DotNet.Build.Tasks.Archives" Version="7.0.0-beta.22266.1">
<Uri>https://github.com/dotnet/arcade</Uri>
<Sha>ba1c3aff4be864c493031d989259ef92aaa23fc3</Sha>
<Sha>70d269dfe645525adb6836d25d8a97d7960eda1a</Sha>
</Dependency>
<Dependency Name="Microsoft.DotNet.Build.Tasks.Packaging" Version="7.0.0-beta.22255.2">
<Dependency Name="Microsoft.DotNet.Build.Tasks.Packaging" Version="7.0.0-beta.22266.1">
<Uri>https://github.com/dotnet/arcade</Uri>
<Sha>ba1c3aff4be864c493031d989259ef92aaa23fc3</Sha>
<Sha>70d269dfe645525adb6836d25d8a97d7960eda1a</Sha>
</Dependency>
<Dependency Name="Microsoft.DotNet.Build.Tasks.Installers" Version="7.0.0-beta.22255.2">
<Dependency Name="Microsoft.DotNet.Build.Tasks.Installers" Version="7.0.0-beta.22266.1">
<Uri>https://github.com/dotnet/arcade</Uri>
<Sha>ba1c3aff4be864c493031d989259ef92aaa23fc3</Sha>
<Sha>70d269dfe645525adb6836d25d8a97d7960eda1a</Sha>
</Dependency>
<Dependency Name="Microsoft.DotNet.Build.Tasks.Templating" Version="7.0.0-beta.22255.2">
<Dependency Name="Microsoft.DotNet.Build.Tasks.Templating" Version="7.0.0-beta.22266.1">
<Uri>https://github.com/dotnet/arcade</Uri>
<Sha>ba1c3aff4be864c493031d989259ef92aaa23fc3</Sha>
<Sha>70d269dfe645525adb6836d25d8a97d7960eda1a</Sha>
</Dependency>
<Dependency Name="Microsoft.DotNet.Build.Tasks.Workloads" Version="7.0.0-beta.22255.2">
<Dependency Name="Microsoft.DotNet.Build.Tasks.Workloads" Version="7.0.0-beta.22266.1">
<Uri>https://github.com/dotnet/arcade</Uri>
<Sha>ba1c3aff4be864c493031d989259ef92aaa23fc3</Sha>
<Sha>70d269dfe645525adb6836d25d8a97d7960eda1a</Sha>
</Dependency>
<Dependency Name="Microsoft.DotNet.CodeAnalysis" Version="7.0.0-beta.22255.2">
<Dependency Name="Microsoft.DotNet.CodeAnalysis" Version="7.0.0-beta.22266.1">
<Uri>https://github.com/dotnet/arcade</Uri>
<Sha>ba1c3aff4be864c493031d989259ef92aaa23fc3</Sha>
<Sha>70d269dfe645525adb6836d25d8a97d7960eda1a</Sha>
</Dependency>
<Dependency Name="Microsoft.DotNet.Build.Tasks.TargetFramework" Version="7.0.0-beta.22255.2">
<Dependency Name="Microsoft.DotNet.Build.Tasks.TargetFramework" Version="7.0.0-beta.22266.1">
<Uri>https://github.com/dotnet/arcade</Uri>
<Sha>ba1c3aff4be864c493031d989259ef92aaa23fc3</Sha>
<Sha>70d269dfe645525adb6836d25d8a97d7960eda1a</Sha>
</Dependency>
<Dependency Name="Microsoft.DotNet.RemoteExecutor" Version="7.0.0-beta.22255.2">
<Dependency Name="Microsoft.DotNet.RemoteExecutor" Version="7.0.0-beta.22266.1">
<Uri>https://github.com/dotnet/arcade</Uri>
<Sha>ba1c3aff4be864c493031d989259ef92aaa23fc3</Sha>
<Sha>70d269dfe645525adb6836d25d8a97d7960eda1a</Sha>
</Dependency>
<Dependency Name="Microsoft.DotNet.Build.Tasks.Feed" Version="7.0.0-beta.22255.2">
<Dependency Name="Microsoft.DotNet.Build.Tasks.Feed" Version="7.0.0-beta.22266.1">
<Uri>https://github.com/dotnet/arcade</Uri>
<Sha>ba1c3aff4be864c493031d989259ef92aaa23fc3</Sha>
<Sha>70d269dfe645525adb6836d25d8a97d7960eda1a</Sha>
</Dependency>
<Dependency Name="Microsoft.DotNet.VersionTools.Tasks" Version="7.0.0-beta.22255.2">
<Dependency Name="Microsoft.DotNet.VersionTools.Tasks" Version="7.0.0-beta.22266.1">
<Uri>https://github.com/dotnet/arcade</Uri>
<Sha>ba1c3aff4be864c493031d989259ef92aaa23fc3</Sha>
<Sha>70d269dfe645525adb6836d25d8a97d7960eda1a</Sha>
</Dependency>
<Dependency Name="Microsoft.DotNet.SharedFramework.Sdk" Version="7.0.0-beta.22255.2">
<Dependency Name="Microsoft.DotNet.SharedFramework.Sdk" Version="7.0.0-beta.22266.1">
<Uri>https://github.com/dotnet/arcade</Uri>
<Sha>ba1c3aff4be864c493031d989259ef92aaa23fc3</Sha>
<Sha>70d269dfe645525adb6836d25d8a97d7960eda1a</Sha>
</Dependency>
<Dependency Name="Microsoft.NET.Test.Sdk" Version="16.9.0-preview-20201201-01">
<Uri>https://github.com/microsoft/vstest</Uri>
Expand Down Expand Up @@ -254,9 +254,9 @@
<Uri>https://github.com/dotnet/xharness</Uri>
<Sha>79dc54a9932d26ad0deaf82562b48c4bc8fd525a</Sha>
</Dependency>
<Dependency Name="Microsoft.DotNet.PackageTesting" Version="7.0.0-beta.22255.2">
<Dependency Name="Microsoft.DotNet.PackageTesting" Version="7.0.0-beta.22266.1">
<Uri>https://github.com/dotnet/arcade</Uri>
<Sha>ba1c3aff4be864c493031d989259ef92aaa23fc3</Sha>
<Sha>70d269dfe645525adb6836d25d8a97d7960eda1a</Sha>
</Dependency>
<Dependency Name="optimization.windows_nt-x64.MIBC.Runtime" Version="1.0.0-prerelease.22217.3">
<Uri>https://dev.azure.com/dnceng/internal/_git/dotnet-optimization</Uri>
Expand Down
32 changes: 16 additions & 16 deletions eng/Versions.props
Original file line number Diff line number Diff line change
Expand Up @@ -53,22 +53,22 @@
<!-- SDK dependencies -->
<MicrosoftDotNetCompatibilityVersion>2.0.0-alpha.1.21525.11</MicrosoftDotNetCompatibilityVersion>
<!-- Arcade dependencies -->
<MicrosoftDotNetApiCompatVersion>7.0.0-beta.22255.2</MicrosoftDotNetApiCompatVersion>
<MicrosoftDotNetBuildTasksFeedVersion>7.0.0-beta.22255.2</MicrosoftDotNetBuildTasksFeedVersion>
<MicrosoftDotNetCodeAnalysisVersion>7.0.0-beta.22255.2</MicrosoftDotNetCodeAnalysisVersion>
<MicrosoftDotNetGenAPIVersion>7.0.0-beta.22255.2</MicrosoftDotNetGenAPIVersion>
<MicrosoftDotNetGenFacadesVersion>7.0.0-beta.22255.2</MicrosoftDotNetGenFacadesVersion>
<MicrosoftDotNetXUnitExtensionsVersion>7.0.0-beta.22255.2</MicrosoftDotNetXUnitExtensionsVersion>
<MicrosoftDotNetXUnitConsoleRunnerVersion>2.5.1-beta.22255.2</MicrosoftDotNetXUnitConsoleRunnerVersion>
<MicrosoftDotNetBuildTasksArchivesVersion>7.0.0-beta.22255.2</MicrosoftDotNetBuildTasksArchivesVersion>
<MicrosoftDotNetBuildTasksInstallersVersion>7.0.0-beta.22255.2</MicrosoftDotNetBuildTasksInstallersVersion>
<MicrosoftDotNetBuildTasksPackagingVersion>7.0.0-beta.22255.2</MicrosoftDotNetBuildTasksPackagingVersion>
<MicrosoftDotNetBuildTasksTargetFrameworkVersion>7.0.0-beta.22255.2</MicrosoftDotNetBuildTasksTargetFrameworkVersion>
<MicrosoftDotNetBuildTasksTemplatingVersion>7.0.0-beta.22255.2</MicrosoftDotNetBuildTasksTemplatingVersion>
<MicrosoftDotNetBuildTasksWorkloadsPackageVersion>7.0.0-beta.22255.2</MicrosoftDotNetBuildTasksWorkloadsPackageVersion>
<MicrosoftDotNetRemoteExecutorVersion>7.0.0-beta.22255.2</MicrosoftDotNetRemoteExecutorVersion>
<MicrosoftDotNetVersionToolsTasksVersion>7.0.0-beta.22255.2</MicrosoftDotNetVersionToolsTasksVersion>
<MicrosoftDotNetPackageTestingVersion>7.0.0-beta.22255.2</MicrosoftDotNetPackageTestingVersion>
<MicrosoftDotNetApiCompatVersion>7.0.0-beta.22266.1</MicrosoftDotNetApiCompatVersion>
<MicrosoftDotNetBuildTasksFeedVersion>7.0.0-beta.22266.1</MicrosoftDotNetBuildTasksFeedVersion>
<MicrosoftDotNetCodeAnalysisVersion>7.0.0-beta.22266.1</MicrosoftDotNetCodeAnalysisVersion>
<MicrosoftDotNetGenAPIVersion>7.0.0-beta.22266.1</MicrosoftDotNetGenAPIVersion>
<MicrosoftDotNetGenFacadesVersion>7.0.0-beta.22266.1</MicrosoftDotNetGenFacadesVersion>
<MicrosoftDotNetXUnitExtensionsVersion>7.0.0-beta.22266.1</MicrosoftDotNetXUnitExtensionsVersion>
<MicrosoftDotNetXUnitConsoleRunnerVersion>2.5.1-beta.22266.1</MicrosoftDotNetXUnitConsoleRunnerVersion>
<MicrosoftDotNetBuildTasksArchivesVersion>7.0.0-beta.22266.1</MicrosoftDotNetBuildTasksArchivesVersion>
<MicrosoftDotNetBuildTasksInstallersVersion>7.0.0-beta.22266.1</MicrosoftDotNetBuildTasksInstallersVersion>
<MicrosoftDotNetBuildTasksPackagingVersion>7.0.0-beta.22266.1</MicrosoftDotNetBuildTasksPackagingVersion>
<MicrosoftDotNetBuildTasksTargetFrameworkVersion>7.0.0-beta.22266.1</MicrosoftDotNetBuildTasksTargetFrameworkVersion>
<MicrosoftDotNetBuildTasksTemplatingVersion>7.0.0-beta.22266.1</MicrosoftDotNetBuildTasksTemplatingVersion>
<MicrosoftDotNetBuildTasksWorkloadsPackageVersion>7.0.0-beta.22266.1</MicrosoftDotNetBuildTasksWorkloadsPackageVersion>
<MicrosoftDotNetRemoteExecutorVersion>7.0.0-beta.22266.1</MicrosoftDotNetRemoteExecutorVersion>
<MicrosoftDotNetVersionToolsTasksVersion>7.0.0-beta.22266.1</MicrosoftDotNetVersionToolsTasksVersion>
<MicrosoftDotNetPackageTestingVersion>7.0.0-beta.22266.1</MicrosoftDotNetPackageTestingVersion>
<!-- NuGet dependencies -->
<NuGetBuildTasksPackVersion>6.0.0-preview.1.102</NuGetBuildTasksPackVersion>
<!-- Installer dependencies -->
Expand Down
11 changes: 6 additions & 5 deletions eng/common/init-tools-native.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -93,7 +93,7 @@ try {
$ToolVersion = ""
}
$ArcadeToolsDirectory = "C:\arcade-tools"
if (Test-Path $ArcadeToolsDirectory -eq $False) {
if (-not (Test-Path $ArcadeToolsDirectory)) {
Write-Error "Arcade tools directory '$ArcadeToolsDirectory' was not found; artifacts were not properly installed."
exit 1
}
Expand All @@ -103,13 +103,14 @@ try {
exit 1
}
$BinPathFile = "$($ToolDirectory.FullName)\binpath.txt"
if (Test-Path -Path "$BinPathFile" -eq $False) {
if (-not (Test-Path -Path "$BinPathFile")) {
Write-Error "Unable to find binpath.txt in '$($ToolDirectory.FullName)' ($ToolName $ToolVersion); artifact is either installed incorrectly or is not a bootstrappable tool."
exit 1
}
$BinPath = Get-Content "$BinPathFile"
Write-Host "Adding $ToolName to the path ($(Convert-Path -Path $BinPath))..."
Write-Host "##vso[task.prependpath]$(Convert-Path -Path $BinPath)"
$ToolPath = Convert-Path -Path $BinPath
Write-Host "Adding $ToolName to the path ($ToolPath)..."
Write-Host "##vso[task.prependpath]$ToolPath"
}
}
exit 0
Expand Down Expand Up @@ -188,7 +189,7 @@ try {
Write-Host "##vso[task.prependpath]$(Convert-Path -Path $InstallBin)"
return $InstallBin
}
else {
elseif (-not ($PathPromotion)) {
Write-PipelineTelemetryError -Category 'NativeToolsBootstrap' -Message 'Native tools install directory does not exist, installation failed'
exit 1
}
Expand Down
Loading

0 comments on commit 7628c7b

Please sign in to comment.