Remove BuildAndTest and RebuildAndTest target in favor of Test (dotne…

…t#33151)

* Implicitly build when invoking the test target

To get closer to the VSTest behavior this changes the Test target to
implicitly invoke the Build target unless /p:TestNoBuild is passed in.
(VSTest uses VSTestNoBuild which is controlled by the
dotnet test --no-build flag)

* Fix error logging for test outputs

* Update dotnet msbuild to build docs

* Update ReportGenerator global tool version

Updating ReportGenerator's version to 4.5.0 which removes unnecessary
logging (args passed in).

.config/dotnet-tools.json

@@ -9,7 +9,7 @@
       ]
     },
     "dotnet-reportgenerator-globaltool": {
-      "version": "4.4.2",
+      "version": "4.5.0",
       "commands": [
         "reportgenerator"
       ]

docs/coding-guidelines/adding-api-guidelines.md

@@ -33,12 +33,12 @@ should be added to `netcoreapp5.0`. [More Information on TargetFrameworks](https
  - If changing the target framework
     - Update `SupportedFramework` metadata on the ref ProjectReference to declare the set of concrete platforms you expect your library to support. (see [Specific platform mappings][net-standard table]). Generally will be a combination of netcoreapp2.x, netfx46x, and/or `$(AllXamarinFrameworks)`.
   - If assembly or package version is updated the package index needs to be updated by running
-    `dotnet msbuild <Library>/pkg/<Library>.pkgproj /t:UpdatePackageIndex`
+    `dotnet build <Library>/pkg/<Library>.pkgproj /t:UpdatePackageIndex`
 
 **Update tests**
   - Add new `TargetFramework` to the ```TargetFrameworks```.
   - Add new test code following [conventions](project-guidelines.md#code-file-naming-conventions) for new files to that are specific to the new target framework.
-  - To run just the new test targetFramework run `dotnet build <Library>.csproj -f <TargetFramework> /t:RebuildAndTest`. TargetFramework should be chosen only from supported TargetFrameworks.
+  - To run just the new test targetFramework run `dotnet build <Library>.csproj -f <TargetFramework> /t:Test`. TargetFramework should be chosen only from supported TargetFrameworks.
 
 ## Documentation
 

docs/coding-guidelines/interop-guidelines.md

@@ -134,7 +134,7 @@ TargetsUnix is true for both OSX and Linux builds and can be used to include cod
 You should not test the value of the OSGroup property directly, instead use one of the values above.
 
 #### Project Files
-Whenever possible, a single .csproj should be used per assembly, spanning all target platforms, e.g. System.Console.csproj includes conditional entries for when targeting Windows vs when targeting Linux. A property can be passed to dotnet msbuild to control which flavor is built, e.g. `dotnet msbuild /p:OSGroup=OSX System.Console.csproj`.
+Whenever possible, a single .csproj should be used per assembly, spanning all target platforms, e.g. System.Console.csproj includes conditional entries for when targeting Windows vs when targeting Linux. A property can be passed to dotnet build to control which flavor is built, e.g. `dotnet build /p:OSGroup=OSX System.Console.csproj`.
 
 ### Constants
 - Wherever possible, constants should be defined as "const". Only if the data type doesn't support this (e.g. IntPtr) should they instead be static readonly fields.

docs/workflow/building/libraries/README.md

@@ -8,23 +8,22 @@ Here is one example of a daily workflow for a developer working mainly on the li
 :: From root:
 git clean -xdf
 git pull upstream master & git push origin master
-build -subsetCategory coreclr -c Release
 :: Build Debug libraries on top of Release runtime:
-build -subsetCategory libraries -runtimeConfiguration Release
-
+build -subsetCategory coreclr-libraries -runtimeConfiguration Release
 :: The above you may only perform once in a day, or when you pull down significant new changes.
 
 :: Switch to working on a given library (RegularExpressions in this case)
 cd src\libraries\System.Text.RegularExpressions
 
 :: If you use Visual Studio, you might open System.Text.RegularExpressions.sln here.
+build -vs System.Text.RegularExpressions
 
 :: Change to test directory
 cd tests
 
 :: Then inner loop build / test
 :: (If using Visual Studio, you might run tests inside it instead)
-pushd ..\src & dotnet build & popd & dotnet build /t:buildandtest
+pushd ..\src & dotnet build & popd & dotnet build /t:test
 ```
 
 The instructions for Linux and macOS are essentially the same:
@@ -33,10 +32,8 @@ The instructions for Linux and macOS are essentially the same:
 # From root:
 git clean -xdf
 git pull upstream master & git push origin master
-./build.sh -subsetCategory coreclr -c Release
 # Build Debug libraries on top of Release runtime:
-./build.sh -subsetCategory libraries -runtimeConfiguration Release
-
+./build.sh -subsetCategory coreclr-libraries -runtimeconfiguration Release
 # The above you may only perform once in a day, or when you pull down significant new changes.
 
 # Switch to working on a given library (RegularExpressions in this case)
@@ -46,7 +43,7 @@ cd src/libraries/System.Text.RegularExpressions
 cd tests
 
 # Then inner loop build / test:
-pushd ../src & dotnet build & popd & dotnet build /t:buildandtest
+pushd ../src & dotnet build & popd & dotnet build /t:test
 ```
 
 The steps above may be all you need to know to make a change. Want more details about what this means? Read on.

docs/workflow/building/libraries/code-coverage.md

@@ -36,20 +36,20 @@ This runs the tests and generates the full code coverage report. The resulting i
 
 You can also build and test with code coverage for a particular test project rather than for the whole repo with the ```/p:Coverage=true``` argument:
 
-    dotnet msbuild /t:BuildAndTest /p:Coverage=true
+    dotnet build /t:Test /p:Coverage=true
 
 The results for this one library will then show up in the aforementioned index.htm file. For example, to build, test, and get code coverage results for the System.Diagnostics.Debug library, from the root of the repo one can do:
 
     cd src\System.Diagnostics.Debug\tests\
-    dotnet msbuild /t:BuildAndTest /p:Coverage=true
+    dotnet build /t:Test /p:Coverage=true
 
 And then once the run completes:
 
     $(OutDir)\report\index.htm
 
 **Note:** If you only want to measure the coverage of your local changes (that haven't been pushed to git), run:
 
-    dotnet msbuild /t:BuildAndTest /p:Coverage=true /p:CoverageSourceLink=false
+    dotnet build /t:Test /p:Coverage=true /p:CoverageSourceLink=false
 
 
 ## Code coverage with System.Private.CoreLib code

docs/workflow/debugging/libraries/debugging-packages.md

@@ -4,23 +4,23 @@ Debugging CoreFX build issues
 ## MSBuild debug options
 
 * Enable MSBuild diagnostics log (msbuild.log):
-`dotnet msbuild my.csproj /flp:v=diag /t:rebuild`
+`dotnet build my.csproj /flp:v=diag`
 * Generate a flat project file (out.pp):
-`dotnet msbuild my.csproj /pp:out.pp`
+`dotnet build my.csproj /pp:out.pp`
 * Generate a binary log usable by the [MSBuild Binary and Structured Log Viewer](http://msbuildlog.com/):
-`dotnet msbuild my.csproj /bl`
+`dotnet build my.csproj /bl`
 
 ## Steps to debug packaging build issues
 
 (This documentation is work in progress.)
 
 I found the following process to help when investigating some of the build issues caused by incorrect packaging. 
 
-To quickly validate if a given project compiles on all supported configurations use `dotnet msbuild /t:RebuildAll`. This applies for running tests as well. For more information, see [Building individual libraries](../../building/libraries/README.md#building-individual-libraries) 
+To quickly validate if a given project compiles on all supported configurations use `dotnet build /t:RebuildAll`. This applies for running tests as well. For more information, see [Building individual libraries](../../building/libraries/README.md#building-individual-libraries) 
 
 Assuming the current directory is `\src\contractname\`:
 
-1. Build the `\ref` folder: `dotnet msbuild /t:rebuild` 
+1. Build the `\ref` folder: `dotnet build` 
 
 
 Check the logs for output such as:
@@ -50,11 +50,11 @@ CopyFilesToOutputDirectory:
 
 Using your favourite IL disassembler, ensure that each platform contains the correct APIs. Missing APIs from the contracts is likely caused by not having the right `DefineConstants` tags in the csproj files.
 
-2. Build the `\src` folder: `dotnet msbuild /t:rebuild`
+2. Build the `\src` folder: `dotnet build`
 
 Use the same technique above to ensure that the binaries include the correct implementations.
 
-3. Build the `\pkg` folder: `dotnet msbuild /t:rebuild`
+3. Build the `\pkg` folder: `dotnet build`
 
 Ensure that all Build Pivots are actually being built. This should build all .\ref and .\src variations as well as actually creating the NuGet packages.
 
@@ -72,7 +72,7 @@ Build:
 
 To validate the content of the nupkg, change the extension to .zip. As before, use an IL disassembler to verify that the right APIs are present within `ref\<platform>\contractname.dll` and the right implementations within the `lib\<platform>\contractname.dll`.
 
-4. Run the tests from `\tests`: `dotnet msbuild /t:rebuildandtest`
+4. Run the tests from `\tests`: `dotnet build /t:test`
 
 Ensure that the test is referencing the correct pkg. For example:
 ```
@@ -96,7 +96,7 @@ ResolvePkgProjReferences:
 To run a test from a single Build Pivot combination, specify all properties and build the `csproj`:
 
 ```
-dotnet build System.Net.ServicePoint.Tests.csproj -f netcoreapp2.0 /t:rebuildandtest /p:OuterLoop=true /p:xunitoptions=-showprogress
+dotnet build System.Net.ServicePoint.Tests.csproj -f netcoreapp2.0 /t:test /p:OuterLoop=true /p:xunitoptions=-showprogress
 ```
 Will run the test using the following pivot values:
 * Architecture: AnyCPU

docs/workflow/debugging/libraries/unix-instructions.md

@@ -5,7 +5,7 @@ CoreFX can be debugged on unix using both lldb and visual studio code
 
 ## Using lldb and SOS
 
-- Run the test using msbuild at least once with `/t:BuildAndTest`.
+- Run the test using msbuild at least once with `/t:Test`.
 - [Install version 3.9 of lldb](../coreclr/debugging.md#debugging-core-dumps-with-lldb) and launch lldb with dotnet as the process and arguments matching the arguments used when running the test through msbuild.
 - Load the sos plugin using `plugin load libsosplugin.so`.
 - Type `soshelp` to get help. You can now use all sos commands like `bpmd`.

docs/workflow/debugging/libraries/windows-instructions.md

@@ -173,4 +173,4 @@ Helper scripts are available at https://github.com/dotnet/runtime/tree/master/sr
 * `*System.Threading.Tasks.Dataflow.DataflowEventSource {16F53577-E41D-43D4-B47E-C17025BF4025}`: Provides an event source for tracing Dataflow information.
 
 ## Notes 
-* You can find the test invocation command-line by looking at the logs generated after the `dotnet msbuild /t:rebuildandtest` within the test folder.
+* You can find the test invocation command-line by looking at the logs generated after the `dotnet build /t:test` within the test folder.

docs/workflow/testing/coreclr/testing.md

@@ -31,7 +31,7 @@ Note:  CoreCLR must be built prior to building an individual test. See the first
 * Native Test: Build the generated CMake projects
   * Projects are auto-generated when the `build-test.sh`/`build-test.cmd` script is run
     * It is possible to explicitly run only the native test build with `build-test.sh/cmd skipmanaged`
-* Managed Test: Invoke `dotnet msbuild` on the project directly. `dotnet` can be the `dotnet.sh` or `dotnet.cmd` script in the repo root.
+* Managed Test: Invoke `dotnet build` on the project directly. `dotnet` can be the `dotnet.sh` or `dotnet.cmd` script in the repo root.
 ```
 <runtime-repo-root>/dotnet.sh msbuild <runtime-repo-root>/src/coreclr/tests/src/JIT/CodegenBringupTests/Array1_d.csproj /p:__BuildType=Release
 ```

docs/workflow/testing/libraries/filtering-tests.md

@@ -7,11 +7,11 @@ The tests can be filtered based on xunit trait attributes defined in [`Microsoft
 ```cs
 [OuterLoop()]
 ```
-Tests marked as `OuterLoop` are for scenarios that don't need to run every build. They may take longer than normal tests, cover seldom hit code paths, or require special setup or resources to execute. These tests are excluded by default when testing through `dotnet msbuild` but can be enabled manually by adding the `-testscope outerloop` switch or `/p:TestScope=outerloop` e.g.
+Tests marked as `OuterLoop` are for scenarios that don't need to run every build. They may take longer than normal tests, cover seldom hit code paths, or require special setup or resources to execute. These tests are excluded by default when testing through `dotnet build` but can be enabled manually by adding the `-testscope outerloop` switch or `/p:TestScope=outerloop` e.g.
 
 ```cmd
 build -test -testscope outerloop
-cd src/System.Text.RegularExpressions/tests && dotnet msbuild /t:RebuildAndTest /p:TestScope=outerloop
+cd src/System.Text.RegularExpressions/tests && dotnet build /t:Test /p:TestScope=outerloop
 ```
 
 #### PlatformSpecificAttribute
@@ -28,11 +28,11 @@ Use this attribute on test methods to specify that this test may only be run on
 
 When running tests by building a test project, tests that don't apply to the `OSGroup` are not run. For example, to run Linux-specific tests on a Linux box, use the following command line:
 ```sh
-dotnet msbuild <csproj_file> /t:BuildAndTest /p:OSGroup=Linux
+dotnet build <csproj_file> /t:Test /p:OSGroup=Linux
 ```
 To run all Linux-compatible tests that are failing:
 ```sh
-dotnet msbuild <csproj_file> /t:BuildAndTest /p:OSGroup=Linux /p:WithCategories=failing
+dotnet build <csproj_file> /t:Test /p:OSGroup=Linux /p:WithCategories=failing
 ```
 
 #### ActiveIssueAttribute
@@ -148,9 +148,9 @@ _**A few common examples with the above attributes:**_
 
 - Run all tests acceptable on Windows that are not failing:
 ```cmd
-dotnet msbuild <csproj_file> /t:BuildAndTest /p:OSGroup=Windows_NT
+dotnet build <csproj_file> /t:Test /p:OSGroup=Windows_NT
 ```
 - Run all outer loop tests acceptable on OS X that are currently associated with active issues:
 ```sh
-dotnet msbuild <csproj_file> /t:BuildAndTest /p:OSGroup=OSX /p:WithCategories="OuterLoop;failing""
+dotnet build <csproj_file> /t:Test /p:OSGroup=OSX /p:WithCategories="OuterLoop;failing""
 ```

docs/workflow/testing/libraries/testing.md

@@ -12,25 +12,31 @@ libraries -buildtests
 
 - The following builds and runs all tests for .NET Core in release configuration.
 ```
-libraries -buildtests -test -c Release
+libraries -test -c Release
 ```
 
 - The following example shows how to pass extra msbuild properties to ignore tests ignored in CI.
 ```
 libraries -test /p:WithoutCategories=IgnoreForCI
 ```
 
+Unless you specifiy `/p:TestNoBuild=true`, test assemblies are implicitly built when invoking the `Test` target.
+- The following shows how to only test the libraries without building them
+```
+libraries -test /p:TestNoBuild=true
+```
+
 ## Running tests on the command line
 
 To build tests you need to pass the `-buildtests` flag to build.cmd/sh or if you want to build both src and tests you pass `-buildtests` flag (`libraries -restore -build -buildtests`). Note that you need to specify -restore and -build additionally as those are only implicit if no action is passed in.
 
 If you are interested in building and running the tests only for a specific library, then there are two different ways to do it:
 
-The easiest (and recommended) way to do it, is by simply building the test .csproj file for that library.
+The easiest (and recommended) way to build and run the tests for a specific library, is to invoke the `Test` target on that library:
 
 ```cmd
 cd src\libraries\System.Collections.Immutable\tests
-dotnet build /t:BuildAndTest   ::or /t:Test to just run the tests if the binaries are already built
+dotnet build /t:Test
 ```
 
 It is possible to pass parameters to the underlying xunit runner via the `XUnitOptions` parameter, e.g.:
@@ -44,10 +50,10 @@ There may be multiple projects in some directories so you may need to specify th
 
 To quickly run or debug a single test from the command line, set the XunitMethodName property, e.g.:
 ```cmd
-dotnet build /t:BuildAndTest /p:XunitMethodName={FullyQualifiedNamespace}.{ClassName}.{MethodName}
+dotnet build /t:Test /p:XunitMethodName={FullyQualifiedNamespace}.{ClassName}.{MethodName}
 ```
 
-#### Running tests in a different target framework
+#### Running tests on a different target framework
 
 Each test project can potentially have multiple target frameworks. There are some tests that might be OS-specific, or might be testing an API that is available only on some target frameworks, so the `TargetFrameworks` property specifies the valid target frameworks. By default we will build and run only the default build target framework which is `netcoreapp5.0`. The rest of the targetframeworks will need to be built and ran by specifying the BuildTargetFramework option.
 ```cmd

docs/workflow/testing/mono/testing.md

@@ -5,13 +5,13 @@ We currently only support running tests against coreclr.  There are additional m
 have not been moved over yet. Simply run the following command:
 
 ``` 
-dotnet msbuild /t:RunCoreClrTests $(REPO_ROOT)/src/mono/mono.proj
+dotnet build /t:RunCoreClrTests $(REPO_ROOT)/src/mono/mono.proj
 ```
 
 If you want to run individual tests, execute this command:
 
 ```
-dotnet msbuild /t:RunCoreClrTest /p:CoreClrTest="<TestName>" $(REPO_ROOT)/src/mono/mono.proj
+dotnet build /t:RunCoreClrTest /p:CoreClrTest="<TestName>" $(REPO_ROOT)/src/mono/mono.proj
 ```
 
 ## Running Library Tests
@@ -32,7 +32,7 @@ build.cmd /p:RuntimeFlavor=mono
 3. Run the tests
 
 ```
-dotnet msbuild /t:BuildAndTest /p:RuntimeFlavor=mono
+dotnet build /t:Test /p:RuntimeFlavor=mono
 ```
 
 # Patching Local dotnet (.dotnet-mono)
@@ -42,7 +42,7 @@ test programs and get a glimpse of how mono will work with the dotnet tooling.
 To generate a local .dotnet-mono, execute this command:
 
 ```
-dotnet msbuild /t:PatchLocalMonoDotnet $(REPO_ROOT)/src/mono/mono.proj
+dotnet build /t:PatchLocalMonoDotnet $(REPO_ROOT)/src/mono/mono.proj
 ```
 
 You can then, for example, run our HelloWorld sample via:

eng/build.ps1

@@ -39,7 +39,7 @@ function Get-Help() {
   Write-Host "  -build                  Build all source projects (short: -b)"
   Write-Host "  -buildtests             Build all test projects"
   Write-Host "  -rebuild                Rebuild all source projects"
-  Write-Host "  -test                   Run all unit tests (short: -t)"
+  Write-Host "  -test                   Build and run tests (short: -t)"
   Write-Host "  -pack                   Package build outputs into NuGet packages"
   Write-Host "  -sign                   Sign build outputs"
   Write-Host "  -publish                Publish artifacts (e.g. symbols)"

eng/build.sh

@@ -35,7 +35,7 @@ usage()
   echo "  --build                    Build all source projects (short: -b)"
   echo "  --buildtests               Build all test projects"
   echo "  --rebuild                  Rebuild all source projects"
-  echo "  --test                     Run all unit tests (short: -t)"
+  echo "  --test                     Build and run tests (short: -t)"
   echo "  --pack                     Package build outputs into NuGet packages"
   echo "  --sign                     Sign build outputs"
   echo "  --publish                  Publish artifacts (e.g. symbols)"

eng/pipelines/libraries/enterprise/linux.yml

@@ -26,7 +26,7 @@ variables:
   - name: enterpriseTestsSetup
     value: $(sourcesRoot)/Common/tests/System/Net/EnterpriseTests/setup
   - name: containerRunTestsCommand
-    value: /repo/.dotnet/dotnet msbuild /t:rebuildandtest
+    value: /repo/.dotnet/dotnet build /t:test --no-restore
   - name: containerLibrariesRoot
     value: /repo/src/libraries
 

eng/testing/tests.props

@@ -2,6 +2,9 @@
   <PropertyGroup>
     <TestProjectName Condition="'$(TestProjectName)' == ''">$(MSBuildProjectName)</TestProjectName>
     <TestFramework Condition="'$(TestFramework)' == ''">xunit</TestFramework>
+    <!-- Implicit test build support. -->
+    <TestDependsOn Condition="'$(TestNoBuild)' != 'true'">Build</TestDependsOn>
+    <TestDependsOn>$(TestDependsOn);GenerateRunScript;RunTests</TestDependsOn>
   </PropertyGroup>
 
   <!-- Set env variable to use the local netfx assemblies instead of the ones in the GAC. -->