[Perf] Linux/arm64: 373 Improvements on 4/12/2023 2:59:13 PM #16460

performanceautofiler · 2023-04-18T16:21:28Z

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	7a57f6dd800e77508a4ba405e623059935c82f43
Compare	887c043eb94be364188e2b23a87efa214ea57f1e
Diff	Diff
Configs	CompilationMode:tiered, LLVM:false, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Improvements in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
GreaterThanOrEqualAllBenchmark - Duration of single invocation	4.81 ns	0.02 ns	0.01	0.28	False
LessThanOrEqualBenchmark - Duration of single invocation	15.20 ns	1.14 ns	0.07	0.08	False
MaxBenchmark - Duration of single invocation	14.38 ns	0.97 ns	0.07	0.05	True
GreaterThanBenchmark - Duration of single invocation	13.17 ns	1.13 ns	0.09	0.05	False
AndNotBenchmark - Duration of single invocation	13.57 ns	1.28 ns	0.09	0.05	False
GreaterThanAllBenchmark - Duration of single invocation	4.26 ns	0.00 ns	0.00	0.26	False
ConditionalSelectBenchmark - Duration of single invocation	15.72 ns	1.24 ns	0.08	0.06	False
GreaterThanOrEqualBenchmark - Duration of single invocation	13.97 ns	1.19 ns	0.09	0.06	False
GreaterThanOrEqualAnyBenchmark - Duration of single invocation	3.48 ns	0.00 ns	0.00	0.24	False
LessThanAllBenchmark - Duration of single invocation	3.14 ns	0.00 ns	0.00	0.41	False
GreaterThanAnyBenchmark - Duration of single invocation	3.01 ns	0.00 ns	0.00	0.36	False
MinBenchmark - Duration of single invocation	14.36 ns	1.14 ns	0.08	0.03	True
EqualsBenchmark - Duration of single invocation	5.95 ns	0.95 ns	0.16	0.17	False
EqualsAnyBenchmark - Duration of single invocation	4.33 ns	0.00 ns	0.00	0.25	False
LessThanOrEqualAnyBenchmark - Duration of single invocation	4.99 ns	0.02 ns	0.00	0.28	False
EqualsStaticBenchmark - Duration of single invocation	12.38 ns	0.91 ns	0.07	0.04	False
LessThanBenchmark - Duration of single invocation	12.64 ns	1.10 ns	0.09	0.07	False
LessThanOrEqualAllBenchmark - Duration of single invocation	3.43 ns	0.00 ns	0.00	0.19	False
LessThanAnyBenchmark - Duration of single invocation	4.64 ns	0.00 ns	0.00	0.32	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

Repro Steps

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
AOT MONO build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false
Dotnet SDK installed for dotnet commands
Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create aot directory 
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt64&gt;*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt64&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt64&gt;*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt64&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200

Payloads

Baseline
Compare

Histogram

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>.GreaterThanOrEqualAllBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.02473169897391581 < 4.752713901652048.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 89.02385879989981 (T) = (0 -0.006182924743478952) / Math.Sqrt((2.034866072616266 / (299)) + (0.00015291423353409707 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9991611691088178 = (7.370883462297354 - 0.006182924743478952) / 7.370883462297354 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>.LessThanOrEqualBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.1365049368012576 < 12.644165651267807.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 201.13835105517728 (T) = (0 -1.0741481028167497) / Math.Sqrt((1.122996683000519 / (299)) + (0.003074887756173666 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9264468249282617 = (14.603694562051267 - 1.0741481028167497) / 14.603694562051267 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>.MaxBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.9671983649563634 < 13.591981710809595.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 229.49760017878248 (T) = (0 -1.1011779307938565) / Math.Sqrt((0.4967317716922627 / (299)) + (0.008363018422825599 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9273574089034828 = (15.15884709193216 - 1.1011779307938565) / 15.15884709193216 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>.GreaterThanBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.1325297887562202 < 12.509234450678372.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 164.5649172008554 (T) = (0 -1.1327879088517798) / Math.Sqrt((1.3193406374158982 / (299)) + (0.012273290264995433 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.9233048970387132 = (14.770016143319792 - 1.1327879088517798) / 14.770016143319792 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>.AndNotBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.276185312122473 < 12.88686196403954.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 4/12/2023 8:39:35 PM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 125.98568456785107 (T) = (0 -1.1808317206614993) / Math.Sqrt((0.4026233983126058 / (299)) + (0.01818461480901254 / (2))) is greater than 1.9679296690655894 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (2) - 2, .975) and 0.9159723269014372 = (14.052890876514061 - 1.1808317206614993) / 14.052890876514061 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>.GreaterThanAllBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 4.139463430650372.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 76.1992626334594 (T) = (0 -0) / Math.Sqrt((2.352811120008207 / (299)) + (0 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 1 = (6.759408704569653 - 0) / 6.759408704569653 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>.ConditionalSelectBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.2372350133836898 < 14.832924630676638.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 4/12/2023 8:39:35 PM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 260.157590866413 (T) = (0 -1.1922352408257426) / Math.Sqrt((0.38369075163302346 / (299)) + (0.0040499590605339525 / (2))) is greater than 1.9679296690655894 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (2) - 2, .975) and 0.9262037348375887 = (16.155766666536067 - 1.1922352408257426) / 16.155766666536067 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>.GreaterThanOrEqualBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.1900088960642317 < 13.198235668751158.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 187.54225532388793 (T) = (0 -1.175524384483629) / Math.Sqrt((1.202713086114938 / (299)) + (0.006381632513072025 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9228266778873223 = (15.23226358931771 - 1.175524384483629) / 15.23226358931771 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>.GreaterThanOrEqualAnyBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 3.3273874887821147.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 75.15765479281342 (T) = (0 -0.018697258528037473) / Math.Sqrt((1.6504707923268571 / (299)) + (0.00032870567127481643 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.9966953097484262 = (5.657794560060046 - 0.018697258528037473) / 5.657794560060046 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>.LessThanAllBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 2.9217997064176466.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 69.10114428203778 (T) = (0 -0) / Math.Sqrt((1.656382492020901 / (299)) + (0 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 1 = (5.143159118210431 - 0) / 5.143159118210431 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>.GreaterThanAnyBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 2.9236955186197386.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 68.87699888188719 (T) = (0 -0) / Math.Sqrt((1.6716940692076054 / (299)) + (0 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 1 = (5.150116134010372 - 0) / 5.150116134010372 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>.MinBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.1388178170766778 < 13.687466660363487.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 312.21235738590315 (T) = (0 -1.1060084579824265) / Math.Sqrt((0.45619750336430986 / (299)) + (0.0021018542053539042 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9274567194043452 = (15.246187502149896 - 1.1060084579824265) / 15.246187502149896 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>.EqualsBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.9507187428349194 < 5.7272232100419815.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 49.97784573239463 (T) = (0 -0.8687811549243992) / Math.Sqrt((4.796556706481057 / (299)) + (0.04709830661918599 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9056107328997199 = (9.204236685103165 - 0.8687811549243992) / 9.204236685103165 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>.EqualsAnyBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 4.1863764454565295.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 82.49999480062378 (T) = (0 -5.798748612979259E-05) / Math.Sqrt((1.9697678798577278 / (299)) + (1.681274273826444E-08 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.9999913402658964 = (6.696220165269109 - 5.798748612979259E-05) / 6.696220165269109 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>.LessThanOrEqualAnyBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.02022729956687926 < 4.663423959170314.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 87.4520568199451 (T) = (0 -0.014335778533137743) / Math.Sqrt((2.062042019675009 / (299)) + (0.00013492598560545452 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9980347208924032 = (7.294525483796252 - 0.014335778533137743) / 7.294525483796252 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>.EqualsStaticBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.9142065767681371 < 12.080349993930671.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 186.86312987474076 (T) = (0 -0.9920968221799316) / Math.Sqrt((1.2846465444005521 / (299)) + (0.003361811436945345 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.9299568342749062 = (14.16407742153922 - 0.9920968221799316) / 14.16407742153922 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>.LessThanBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0999372495620554 < 12.05658767586407.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 164.0422561418951 (T) = (0 -1.040376616947537) / Math.Sqrt((1.3113365448090784 / (299)) + (0.008016940323039182 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9264932072752543 = (14.153475867779207 - 1.040376616947537) / 14.153475867779207 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>.LessThanOrEqualAllBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 3.2421105156754852.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 76.39371717205371 (T) = (0 -0.0018961384993286035) / Math.Sqrt((1.6574256383454449 / (299)) + (1.4381364834544516E-05 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9996668456606708 = (5.691471715921829 - 0.0018961384993286035) / 5.691471715921829 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>.LessThanAnyBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 4.151242950033223.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/22/2023 9:02:35 AM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 81.5848596714104 (T) = (0 -0) / Math.Sqrt((2.0046933409980645 / (299)) + (0 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 1 = (6.680330522768292 - 0) / 6.680330522768292 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

The text was updated successfully, but these errors were encountered:

performanceautofiler · 2023-04-18T16:21:33Z

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	7a57f6dd800e77508a4ba405e623059935c82f43
Compare	887c043eb94be364188e2b23a87efa214ea57f1e
Diff	Diff
Configs	CompilationMode:tiered, LLVM:false, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Improvements in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Single>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
GreaterThanBenchmark - Duration of single invocation	33.98 ns	1.22 ns	0.04	0.03	False
SquareRootBenchmark - Duration of single invocation	75.80 ns	1.15 ns	0.02	0.54	False
EqualsBenchmark - Duration of single invocation	64.05 ns	1.39 ns	0.02	0.01	False
AndNotBenchmark - Duration of single invocation	13.45 ns	1.54 ns	0.11	0.04	False
LessThanAnyBenchmark - Duration of single invocation	16.94 ns	0.00 ns	0.00	0.03	True
GreaterThanOrEqualBenchmark - Duration of single invocation	33.75 ns	1.22 ns	0.04	0.02	False
LessThanOrEqualBenchmark - Duration of single invocation	34.92 ns	1.14 ns	0.03	0.02	False
MaxBenchmark - Duration of single invocation	39.76 ns	1.22 ns	0.03	0.04	False
LessThanBenchmark - Duration of single invocation	32.89 ns	1.24 ns	0.04	0.04	False
SumBenchmark - Duration of single invocation	29.18 ns	0.01 ns	0.00	0.03	True
GreaterThanAllBenchmark - Duration of single invocation	8.27 ns	1.00 ns	0.12	0.08	False
EqualsStaticBenchmark - Duration of single invocation	32.67 ns	1.30 ns	0.04	0.03	False
GreaterThanOrEqualAnyBenchmark - Duration of single invocation	17.43 ns	0.05 ns	0.00	0.04	True
LessThanOrEqualAnyBenchmark - Duration of single invocation	17.73 ns	0.00 ns	0.00	0.05	True
ConditionalSelectBenchmark - Duration of single invocation	15.65 ns	1.15 ns	0.07	0.04	False
EqualsAnyBenchmark - Duration of single invocation	17.50 ns	0.00 ns	0.00	0.04	False
GreaterThanOrEqualAllBenchmark - Duration of single invocation	9.22 ns	0.00 ns	0.00	0.11	False
GreaterThanAnyBenchmark - Duration of single invocation	16.69 ns	0.00 ns	0.00	0.02	True
LessThanOrEqualAllBenchmark - Duration of single invocation	9.12 ns	0.14 ns	0.02	0.08	False
MinBenchmark - Duration of single invocation	39.68 ns	1.15 ns	0.03	0.03	False
AbsBenchmark - Duration of single invocation	79.03 ns	1.20 ns	0.02	0.03	False
LessThanAllBenchmark - Duration of single invocation	8.56 ns	0.00 ns	0.00	0.08	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

Repro Steps

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
AOT MONO build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false
Dotnet SDK installed for dotnet commands
Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create aot directory 
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Single&gt;*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Single&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Single&gt;*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Single&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200

Payloads

Baseline
Compare

Histogram

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Single>.GreaterThanBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.2174815995011121 < 32.09364714272069.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 129.32919174440062 (T) = (0 -1.3155473688513433) / Math.Sqrt((22.498080883672046 / (299)) + (0.01496199535123699 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9650701494185921 = (37.66255357392141 - 1.3155473688513433) / 37.66255357392141 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Single>.SquareRootBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.1477465922646004 < 68.24464717483066.
IsChangePoint: Marked as a change because one of 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 191.351629945023 (T) = (0 -1.2905590636980253) / Math.Sqrt((32.12642110066876 / (299)) + (0.019606046244329236 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9802752461239949 = (65.42839884395075 - 1.2905590636980253) / 65.42839884395075 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Single>.EqualsBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.3898925556153252 < 60.88030290719239.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 862.7918321003209 (T) = (0 -1.3685136307414287) / Math.Sqrt((1.5343733836567381 / (299)) + (0.0019167866530944077 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9792638873858752 = (65.996633805376 - 1.3685136307414287) / 65.996633805376 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Single>.AndNotBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.5387584480683516 < 12.861829888195546.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 4/12/2023 8:39:35 PM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 107.43472820655639 (T) = (0 -1.3337929322528914) / Math.Sqrt((0.4043222489862636 / (299)) + (0.037890076828726166 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.9049842026161172 = (14.037591316148207 - 1.3337929322528914) / 14.037591316148207 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Single>.LessThanAnyBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 16.19986913666128.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 279.6393844911575 (T) = (0 -0.05903745465166802) / Math.Sqrt((1.0355973849157147 / (299)) + (0.003965496595446338 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9968468962998889 = (18.723600701616043 - 0.05903745465166802) / 18.723600701616043 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Single>.GreaterThanOrEqualBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.2193022711640458 < 33.029968992261544.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 124.0347957214372 (T) = (0 -1.230369803465554) / Math.Sqrt((26.232485434727142 / (299)) + (0.022912556587771016 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9685727510708095 = (39.14977751433902 - 1.230369803465554) / 39.14977751433902 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Single>.LessThanOrEqualBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.1407140626905405 < 32.572310179777666.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 128.3620793470017 (T) = (0 -1.1603435349233968) / Math.Sqrt((25.88227689233125 / (299)) + (0.002759731214516475 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9703059774856022 = (39.07667054407271 - 1.1603435349233968) / 39.07667054407271 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Single>.MaxBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.2197082347212163 < 37.99728347806631.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 184.46766092456704 (T) = (0 -1.1663265326114942) / Math.Sqrt((14.237237522966012 / (299)) + (0.014877394272427508 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9728520100163065 = (42.96180060888663 - 1.1663265326114942) / 42.96180060888663 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Single>.LessThanBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.2422537851896933 < 32.01528511158599.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 125.30722453049891 (T) = (0 -1.2157460774939686) / Math.Sqrt((22.17888622635432 / (299)) + (0.03948107646776355 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9676175995162969 = (37.543420479461076 - 1.2157460774939686) / 37.543420479461076 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Single>.SumBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.012576441918246816 < 27.70398641767929.
IsChangePoint: Marked as a change because one of 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 268.32891482445075 (T) = (0 -0.09944986210793155) / Math.Sqrt((0.6881910047331272 / (299)) + (0.044449476340145476 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.9965088140614032 = (28.485982659492205 - 0.09944986210793155) / 28.485982659492205 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Single>.GreaterThanAllBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.9993646247179121 < 8.155542257796853.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 39.755643404961326 (T) = (0 -0.2526571318376675) / Math.Sqrt((1.5523352553063205 / (299)) + (0.24783800785544918 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9760622561357258 = (10.554759599326523 - 0.2526571318376675) / 10.554759599326523 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Single>.EqualsStaticBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.2984183140885024 < 31.57473874322328.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 127.85229365234441 (T) = (0 -1.163879032486437) / Math.Sqrt((22.558062748333903 / (299)) + (0.02332495306282567 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9690566989788059 = (37.613279581556355 - 1.163879032486437) / 37.613279581556355 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Single>.GreaterThanOrEqualAnyBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.049799882542833226 < 16.926728392584803.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 310.7944935683916 (T) = (0 -0.029891759577800357) / Math.Sqrt((1.0412960004267087 / (299)) + (0.0011716962494577503 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9984371742607425 = (19.12673871880546 - 0.029891759577800357) / 19.12673871880546 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Single>.LessThanOrEqualAnyBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 16.76052215471468.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 332.06351905942626 (T) = (0 -0.004900686985384124) / Math.Sqrt((0.9875178835713269 / (299)) + (6.550962603888798E-05 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9997438976556975 = (19.135658436592895 - 0.004900686985384124) / 19.135658436592895 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Single>.ConditionalSelectBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.1478341342360758 < 14.886911799989235.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 4/12/2023 8:39:35 PM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 386.77525561048077 (T) = (0 -1.17606212233562) / Math.Sqrt((0.38609258131887697 / (299)) + (0.0006324207891939028 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.9272518702967941 = (16.166217978849197 - 1.17606212233562) / 16.166217978849197 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Single>.EqualsAnyBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 16.219748003272073.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 308.98314987133307 (T) = (0 -0) / Math.Sqrt((1.0918380034923634 / (299)) + (0 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 1 = (18.671461151490668 - 0) / 18.671461151490668 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Single>.GreaterThanOrEqualAllBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 8.759110539821892.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 145.10969233694078 (T) = (0 -0.028631828162800403) / Math.Sqrt((1.6033062823428086 / (299)) + (0.002564222265562457 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9974598833965337 = (11.271855836746873 - 0.028631828162800403) / 11.271855836746873 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Single>.GreaterThanAnyBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 16.09736310570515.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 312.16789348527226 (T) = (0 -0) / Math.Sqrt((1.0725695120107057 / (299)) + (0 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 1 = (18.69671739198984 - 0) / 18.69671739198984 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Single>.LessThanOrEqualAllBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.14153576006207214 < 8.6768219410983.
IsChangePoint: Marked as a change because one of 3/21/2023 3:33:39 AM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 140.48551562213135 (T) = (0 -0.03633671360647766) / Math.Sqrt((1.5513207234937603 / (299)) + (0.004919880826983609 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9967818737559719 = (11.29126418639064 - 0.03633671360647766) / 11.29126418639064 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Single>.MinBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.1457518776890105 < 38.1219886970183.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 116.0504459085135 (T) = (0 -1.582872759824273) / Math.Sqrt((14.200630560097638 / (299)) + (0.3168858466103924 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9630975584601492 = (42.893442649720924 - 1.582872759824273) / 42.893442649720924 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Single>.AbsBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.197431792609512 < 74.94312880003991.
IsChangePoint: Marked as a change because one of 3/8/2023 4:26:04 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 81.17862078320528 (T) = (0 -1.1883259500464645) / Math.Sqrt((162.62740681695385 / (299)) + (0.03070506304775067 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.9806444190136575 = (61.39448621485232 - 1.1883259500464645) / 61.39448621485232 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Single>.LessThanAllBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 8.086161139598591.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 146.65645446022728 (T) = (0 -0.001717801847116594) / Math.Sqrt((1.5472214761170855 / (299)) + (6.686629255884249E-06 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9998372239896279 = (10.553163474085592 - 0.001717801847116594) / 10.553163474085592 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

performanceautofiler · 2023-04-18T16:21:34Z

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	7a57f6dd800e77508a4ba405e623059935c82f43
Compare	887c043eb94be364188e2b23a87efa214ea57f1e
Diff	Diff
Configs	CompilationMode:tiered, LLVM:false, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Improvements in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Double>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
AbsBenchmark - Duration of single invocation	18.43 ns	1.17 ns	0.06	0.02	False
LessThanBenchmark - Duration of single invocation	16.89 ns	1.14 ns	0.07	0.04	True
LessThanOrEqualAllBenchmark - Duration of single invocation	4.69 ns	0.00 ns	0.00	0.10	False
GreaterThanAnyBenchmark - Duration of single invocation	5.02 ns	0.00 ns	0.00	0.22	False
MaxBenchmark - Duration of single invocation	22.46 ns	1.36 ns	0.06	0.02	True
LessThanOrEqualBenchmark - Duration of single invocation	16.82 ns	1.30 ns	0.08	0.04	True
AndNotBenchmark - Duration of single invocation	13.41 ns	1.47 ns	0.11	0.03	False
EqualsStaticBenchmark - Duration of single invocation	16.45 ns	1.33 ns	0.08	0.05	True
GreaterThanAllBenchmark - Duration of single invocation	3.68 ns	0.01 ns	0.00	0.26	False
GreaterThanOrEqualAllBenchmark - Duration of single invocation	4.43 ns	0.00 ns	0.00	0.19	False
ConditionalSelectBenchmark - Duration of single invocation	15.71 ns	1.25 ns	0.08	0.05	False
EqualsBenchmark - Duration of single invocation	31.70 ns	1.45 ns	0.05	0.02	False
GreaterThanBenchmark - Duration of single invocation	16.45 ns	1.11 ns	0.07	0.04	True
LessThanOrEqualAnyBenchmark - Duration of single invocation	5.83 ns	0.06 ns	0.01	0.19	False
SumBenchmark - Duration of single invocation	3.34 ns	0.11 ns	0.03	0.56	False
SquareRootBenchmark - Duration of single invocation	36.48 ns	1.13 ns	0.03	0.56	False
LessThanAllBenchmark - Duration of single invocation	3.92 ns	0.00 ns	0.00	0.26	False
MinBenchmark - Duration of single invocation	20.92 ns	1.15 ns	0.05	0.04	True
GreaterThanOrEqualAnyBenchmark - Duration of single invocation	6.12 ns	0.00 ns	0.00	0.22	False
EqualsAnyBenchmark - Duration of single invocation	4.87 ns	0.00 ns	0.00	0.24	False
GreaterThanOrEqualBenchmark - Duration of single invocation	17.18 ns	1.24 ns	0.07	0.03	True
LessThanAnyBenchmark - Duration of single invocation	5.05 ns	0.00 ns	0.00	0.23	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

Repro Steps

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
AOT MONO build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false
Dotnet SDK installed for dotnet commands
Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create aot directory 
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Double&gt;*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Double&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Double&gt;*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Double&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200

Payloads

Baseline
Compare

Histogram

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Double>.AbsBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.1685316639371406 < 17.470292970613553.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 265.9810408683974 (T) = (0 -1.125143914428803) / Math.Sqrt((0.08509872053184261 / (299)) + (0.018580129296502563 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.9373130031470912 = (17.94860131948714 - 1.125143914428803) / 17.94860131948714 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Double>.LessThanBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.142000376578214 < 15.840118834626063.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 209.40788551281096 (T) = (0 -1.2321039072525821) / Math.Sqrt((1.546529464166659 / (299)) + (0.004667444648504812 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.93118648782309 = (17.90497052504768 - 1.2321039072525821) / 17.90497052504768 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Double>.LessThanOrEqualAllBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 4.305675104309217.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 91.79638508080087 (T) = (0 -0) / Math.Sqrt((1.6001858470818762 / (299)) + (0 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 1 = (6.715451198956031 - 0) / 6.715451198956031 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Double>.GreaterThanAnyBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 4.744075767129367.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 82.78144551874597 (T) = (0 -0.0029785296375820086) / Math.Sqrt((2.30799729207082 / (299)) + (3.548655520781765E-05 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9995908715122913 = (7.280181476149838 - 0.0029785296375820086) / 7.280181476149838 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Double>.MaxBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.3605792487300599 < 19.627191578939733.
IsChangePoint: Marked as a change because one of 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 376.38607023880115 (T) = (0 -1.3759301081154351) / Math.Sqrt((0.5345241100594975 / (299)) + (0.0033599414477547065 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9334330262649663 = (20.669861207635037 - 1.3759301081154351) / 20.669861207635037 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Double>.LessThanOrEqualBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.2997433658844217 < 16.02799495400571.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 225.82297700326987 (T) = (0 -1.325626117404055) / Math.Sqrt((1.6121048005933185 / (299)) + (0.0003870618084399684 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9265803471068222 = (18.05546696512964 - 1.325626117404055) / 18.05546696512964 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Double>.AndNotBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.4741281951900653 < 12.892040128695468.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 4/12/2023 8:39:35 PM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 129.1913817870566 (T) = (0 -1.320713497363827) / Math.Sqrt((0.37875252625983835 / (299)) + (0.025279891640892033 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.9059336560703589 = (14.040234181437757 - 1.320713497363827) / 14.040234181437757 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Double>.EqualsStaticBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.3270145743368293 < 15.897497603022776.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 173.52414532169524 (T) = (0 -1.180435445948409) / Math.Sqrt((1.5614675911197218 / (299)) + (0.01637330297888975 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9341589002746374 = (17.92854996153252 - 1.180435445948409) / 17.92854996153252 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Double>.GreaterThanAllBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.009128725022929335 < 3.5028204099748264.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 76.73554116774783 (T) = (0 -0.0022885258134701537) / Math.Sqrt((1.8891630693842538 / (299)) + (2.079495433439516E-05 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.999625097203895 = (6.104317805165769 - 0.0022885258134701537) / 6.104317805165769 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Double>.GreaterThanOrEqualAllBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.0014448330713390952 < 4.253216253409204.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 89.78636831822624 (T) = (0 -0.01465509015498996) / Math.Sqrt((1.6207566397700464 / (299)) + (0.0006076288855713984 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.997818252070712 = (6.717132606502237 - 0.01465509015498996) / 6.717132606502237 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Double>.ConditionalSelectBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.2523132826739753 < 14.933815291136703.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 4/12/2023 8:39:35 PM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 272.60237744958295 (T) = (0 -1.307699891933211) / Math.Sqrt((0.3793161320055893 / (299)) + (0.005095740816529537 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.9190623998825433 = (16.156889876095622 - 1.307699891933211) / 16.156889876095622 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Double>.EqualsBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.4498933628645945 < 29.740215524798316.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 301.3209824311946 (T) = (0 -1.3583193065388914) / Math.Sqrt((3.1246301086663073 / (299)) + (0.006069699084624476 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9604241408050584 = (34.321915788312296 - 1.3583193065388914) / 34.321915788312296 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Double>.GreaterThanBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.1084553436282536 < 15.776961840898894.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 154.58694157114715 (T) = (0 -1.242835216378329) / Math.Sqrt((1.5511024658092705 / (299)) + (0.025851833792248206 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9306787263235766 = (17.92862638646273 - 1.242835216378329) / 17.92862638646273 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Double>.LessThanOrEqualAnyBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.05908028860789203 < 5.5103909126709265.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 82.31189621533728 (T) = (0 -0.014770072151973008) / Math.Sqrt((2.7523343628373036 / (299)) + (0.0008726201254979542 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9981549133892333 = (8.005083374289669 - 0.014770072151973008) / 8.005083374289669 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Double>.SumBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.10719423974321259 < 3.206694092060105.
IsChangePoint: Marked as a change because one of 1/5/2023 9:23:01 PM, 2/12/2023 11:53:55 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 76.2424535908256 (T) = (0 -0.070377058765335) / Math.Sqrt((0.4685059492260352 / (299)) + (0.0039287937463019255 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.9813247403719945 = (3.7684648121194995 - 0.070377058765335) / 3.7684648121194995 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Double>.SquareRootBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.1318558670604966 < 34.62319694715093.
IsChangePoint: Marked as a change because one of 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 192.1560170477975 (T) = (0 -1.1357253001720213) / Math.Sqrt((8.50233504286301 / (299)) + (0.006989723215640398 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9670991799471887 = (34.51966541712313 - 1.1357253001720213) / 34.51966541712313 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Double>.LessThanAllBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 3.7341155349887654.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 78.14108972286881 (T) = (0 -0) / Math.Sqrt((1.8434887337245138 / (299)) + (0 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 1 = (6.135699945927029 - 0) / 6.135699945927029 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Double>.MinBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.14879150903285 < 19.90201587434318.
IsChangePoint: Marked as a change because one of 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 241.055472671324 (T) = (0 -1.2909146124814421) / Math.Sqrt((0.5428673244327807 / (299)) + (0.018638090307507467 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9376014542388306 = (20.68821631552793 - 1.2909146124814421) / 20.68821631552793 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Double>.GreaterThanOrEqualAnyBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 5.555860695901354.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 85.87352354096113 (T) = (0 -9.63055408400098E-05) / Math.Sqrt((2.6363097141514813 / (299)) + (3.709902878594718E-08 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9999880567106179 = (8.063569236084383 - 9.63055408400098E-05) / 8.063569236084383 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Double>.EqualsAnyBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 4.797907928782618.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/21/2023 8:47:40 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 84.9278443871613 (T) = (0 -0) / Math.Sqrt((2.215768044476282 / (299)) + (0 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 1 = (7.3109978182391595 - 0) / 7.3109978182391595 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Double>.GreaterThanOrEqualBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.2409686451640094 < 16.289334372278375.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 134.2536308557938 (T) = (0 -1.2286472625124079) / Math.Sqrt((1.5343363827798964 / (299)) + (0.04227690727752132 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9319346720556658 = (18.051000408272927 - 1.2286472625124079) / 18.051000408272927 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Double>.LessThanAnyBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 4.795096569389167.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 70.94460576252517 (T) = (0 -0.05938911367402159) / Math.Sqrt((2.2278511973372175 / (299)) + (0.011651795736444032 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9918441707794856 = (7.28179956547397 - 0.05938911367402159) / 7.28179956547397 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

performanceautofiler · 2023-04-18T16:21:35Z

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	7a57f6dd800e77508a4ba405e623059935c82f43
Compare	887c043eb94be364188e2b23a87efa214ea57f1e
Diff	Diff
Configs	CompilationMode:tiered, LLVM:false, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Improvements in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int64>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
EqualsAnyBenchmark - Duration of single invocation	4.67 ns	0.00 ns	0.00	0.25	False
GreaterThanAllBenchmark - Duration of single invocation	4.47 ns	0.03 ns	0.01	0.30	False
EqualsBenchmark - Duration of single invocation	6.27 ns	1.01 ns	0.16	0.21	False
LessThanOrEqualAllBenchmark - Duration of single invocation	3.63 ns	0.00 ns	0.00	0.23	False
LessThanOrEqualAnyBenchmark - Duration of single invocation	5.00 ns	0.00 ns	0.00	0.20	False
ConditionalSelectBenchmark - Duration of single invocation	15.83 ns	1.42 ns	0.09	0.03	False
LessThanAnyBenchmark - Duration of single invocation	4.42 ns	0.00 ns	0.00	0.30	False
GreaterThanOrEqualAnyBenchmark - Duration of single invocation	3.40 ns	0.00 ns	0.00	0.21	False
GreaterThanOrEqualAllBenchmark - Duration of single invocation	4.93 ns	0.00 ns	0.00	0.29	False
AbsBenchmark - Duration of single invocation	11.30 ns	1.20 ns	0.11	0.04	True
LessThanBenchmark - Duration of single invocation	15.12 ns	1.22 ns	0.08	0.07	False
GreaterThanBenchmark - Duration of single invocation	13.20 ns	1.09 ns	0.08	0.07	False
MaxBenchmark - Duration of single invocation	14.31 ns	1.35 ns	0.09	0.04	True
AndNotBenchmark - Duration of single invocation	13.51 ns	1.10 ns	0.08	0.03	False
GreaterThanOrEqualBenchmark - Duration of single invocation	14.00 ns	0.76 ns	0.05	0.07	False
GreaterThanAnyBenchmark - Duration of single invocation	3.19 ns	0.04 ns	0.01	0.35	False
LessThanOrEqualBenchmark - Duration of single invocation	14.23 ns	1.38 ns	0.10	0.07	False
EqualsStaticBenchmark - Duration of single invocation	13.08 ns	1.36 ns	0.10	0.08	False
LessThanAllBenchmark - Duration of single invocation	2.98 ns	0.00 ns	0.00	0.36	False
MinBenchmark - Duration of single invocation	14.47 ns	1.31 ns	0.09	0.06	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

Repro Steps

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
AOT MONO build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false
Dotnet SDK installed for dotnet commands
Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create aot directory 
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int64&gt;*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int64&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int64&gt;*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int64&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200

Payloads

Baseline
Compare

Histogram

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int64>.EqualsAnyBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 4.197314446931763.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 80.12754552033738 (T) = (0 -0.011371701622663054) / Math.Sqrt((2.063134364896877 / (299)) + (0.0006465779889743877 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.998310146599913 = (6.729401273552785 - 0.011371701622663054) / 6.729401273552785 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int64>.GreaterThanAllBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.02912612655281604 < 4.25156287586361.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 76.85789511121953 (T) = (0 -0.019126288511037478) / Math.Sqrt((2.252162110268659 / (299)) + (0.0005773632911324818 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.9971624714620315 = (6.740474414657515 - 0.019126288511037478) / 6.740474414657515 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int64>.EqualsBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0133406598281498 < 5.807345201759325.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 59.914027606933104 (T) = (0 -0.8334789022403855) / Math.Sqrt((4.348047211082819 / (299)) + (0.01671909891572176 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9077128006317796 = (9.031359797959128 - 0.8334789022403855) / 9.031359797959128 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int64>.LessThanOrEqualAllBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 3.4856926580200285.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 75.20713227766132 (T) = (0 -0.0006664541796438702) / Math.Sqrt((1.6975535834817224 / (299)) + (1.776644694259136E-06 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9998824109026226 = (5.6676528224815925 - 0.0006664541796438702) / 5.6676528224815925 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int64>.LessThanOrEqualAnyBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 4.731375243275568.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 88.15519901877477 (T) = (0 -0) / Math.Sqrt((2.035633083623399 / (299)) + (0 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 1 = (7.2738122396014875 - 0) / 7.2738122396014875 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int64>.ConditionalSelectBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.418000481221709 < 14.891225442173377.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 4/12/2023 8:39:35 PM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 247.23091072261752 (T) = (0 -1.3695539007835165) / Math.Sqrt((0.37076040735928634 / (299)) + (0.004694142312308534 / (2))) is greater than 1.9679296690655894 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (2) - 2, .975) and 0.9153381336893327 = (16.17675064896312 - 1.3695539007835165) / 16.17675064896312 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int64>.LessThanAnyBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 4.21690232069728.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 72.32495804953238 (T) = (0 -0.037165941150002824) / Math.Sqrt((2.1389809148174943 / (299)) + (0.004845583199623637 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.994412904440446 = (6.652104076947361 - 0.037165941150002824) / 6.652104076947361 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int64>.GreaterThanOrEqualAnyBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 3.2698729364279076.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 75.33140116436685 (T) = (0 -0) / Math.Sqrt((1.6942162729554906 / (299)) + (0 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 1 = (5.670545971378855 - 0) / 5.670545971378855 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int64>.GreaterThanOrEqualAllBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 4.812966282803528.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 86.58017583322788 (T) = (0 -0.009689370003283092) / Math.Sqrt((2.120004362116339 / (299)) + (0.00037553556424208874 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9986813946163167 = (7.348195391267817 - 0.009689370003283092) / 7.348195391267817 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int64>.AbsBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.204816484221732 < 10.802218505453629.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 133.4112477233397 (T) = (0 -1.2847850488333064) / Math.Sqrt((0.13459349612744662 / (299)) + (0.020039101824055498 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.884697282086697 = (11.14271260976993 - 1.2847850488333064) / 11.14271260976993 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int64>.LessThanBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.2165497913239582 < 12.030245190480969.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 163.58090649250812 (T) = (0 -1.1710978410619204) / Math.Sqrt((1.3108362249816963 / (299)) + (0.007755054089992555 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9174028849196888 = (14.178435165964725 - 1.1710978410619204) / 14.178435165964725 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int64>.GreaterThanBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0867286398942324 < 12.54680585515374.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 160.45620221702612 (T) = (0 -1.1161314215926608) / Math.Sqrt((1.2774499084850957 / (299)) + (0.014863723477112546 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.9244525696830552 = (14.773916424558 - 1.1161314215926608) / 14.773916424558 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int64>.MaxBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.3537987674291558 < 13.59729119535039.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 156.7106433340816 (T) = (0 -1.2533572095201448) / Math.Sqrt((0.5001129946907542 / (299)) + (0.0246770783357956 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9171650628310113 = (15.130779986748996 - 1.2533572095201448) / 15.130779986748996 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int64>.AndNotBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0954332956684076 < 12.842683023907593.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 4/12/2023 8:39:35 PM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 277.9172268913727 (T) = (0 -1.0650259880032902) / Math.Sqrt((0.37319774091918706 / (299)) + (0.001849208718882208 / (2))) is greater than 1.9679296690655894 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (2) - 2, .975) and 0.9240327365090795 = (14.019538667870796 - 1.0650259880032902) / 14.019538667870796 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int64>.GreaterThanOrEqualBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.7614344217257397 < 13.207210162806357.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 125.16196204313987 (T) = (0 -0.9943322281942135) / Math.Sqrt((1.173935492019073 / (299)) + (0.03616950153727161 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9347882060535982 = (15.247736153547107 - 0.9943322281942135) / 15.247736153547107 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int64>.GreaterThanAnyBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.04483428970947636 < 2.943141256736572.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 69.30005641050018 (T) = (0 -0.008966857941895272) / Math.Sqrt((1.6357870045313316 / (299)) + (0.0004020227067506516 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.9982663668753917 = (5.172292692504592 - 0.008966857941895272) / 5.172292692504592 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int64>.LessThanOrEqualBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.3801969098199638 < 12.402260175382272.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 89.47353770898744 (T) = (0 -1.2485265185891965) / Math.Sqrt((1.1137266342119365 / (299)) + (0.07370389035226872 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9142790246181453 = (14.565005974645992 - 1.2485265185891965) / 14.565005974645992 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int64>.EqualsStaticBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.359384675578795 < 12.047243666512449.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 109.78478814378342 (T) = (0 -1.256134143378807) / Math.Sqrt((1.3301052019022037 / (299)) + (0.04680475072380615 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.9112733407690105 = (14.157347456400988 - 1.256134143378807) / 14.157347456400988 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int64>.LessThanAllBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.0009486296696070769 < 2.8739667968343605.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 69.65516415073624 (T) = (0 -0.00023715741740176923) / Math.Sqrt((1.6432107129091968 / (299)) + (2.2497456251470803E-07 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.999954074891624 = (5.164003434905789 - 0.00023715741740176923) / 5.164003434905789 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int64>.MinBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.308868655264499 < 13.715455465265158.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 252.92791050064545 (T) = (0 -1.3183765782726722) / Math.Sqrt((0.45245018149520333 / (299)) + (0.006061723274639676 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9134797678543309 = (15.237783644096082 - 1.3183765782726722) / 15.237783644096082 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

performanceautofiler · 2023-04-18T16:21:37Z

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	7a57f6dd800e77508a4ba405e623059935c82f43
Compare	887c043eb94be364188e2b23a87efa214ea57f1e
Diff	Diff
Configs	CompilationMode:tiered, LLVM:false, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Improvements in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Byte>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
MinBenchmark - Duration of single invocation	65.38 ns	1.03 ns	0.02	0.03	True
MaxBenchmark - Duration of single invocation	63.92 ns	1.18 ns	0.02	0.02	True
GreaterThanOrEqualAllBenchmark - Duration of single invocation	36.56 ns	0.03 ns	0.00	0.02	False
LessThanOrEqualAnyBenchmark - Duration of single invocation	36.48 ns	0.00 ns	0.00	0.02	True
AndNotBenchmark - Duration of single invocation	13.51 ns	1.30 ns	0.10	0.06	False
GreaterThanOrEqualBenchmark - Duration of single invocation	65.12 ns	1.15 ns	0.02	0.02	True
SumBenchmark - Duration of single invocation	22.68 ns	0.49 ns	0.02	0.02	True
EqualsAnyBenchmark - Duration of single invocation	32.76 ns	0.12 ns	0.00	0.03	False
GreaterThanAllBenchmark - Duration of single invocation	32.79 ns	0.00 ns	0.00	0.03	False
LessThanOrEqualBenchmark - Duration of single invocation	62.17 ns	1.04 ns	0.02	0.02	True
LessThanAnyBenchmark - Duration of single invocation	32.73 ns	0.00 ns	0.00	0.02	False
GreaterThanOrEqualAnyBenchmark - Duration of single invocation	3.84 ns	0.00 ns	0.00	0.21	False
EqualsStaticBenchmark - Duration of single invocation	56.56 ns	1.12 ns	0.02	0.02	True
LessThanBenchmark - Duration of single invocation	56.28 ns	1.33 ns	0.02	0.02	True
LessThanOrEqualAllBenchmark - Duration of single invocation	3.81 ns	0.00 ns	0.00	0.20	False
LessThanAllBenchmark - Duration of single invocation	3.50 ns	0.32 ns	0.09	0.31	False
GreaterThanAnyBenchmark - Duration of single invocation	3.38 ns	0.00 ns	0.00	0.26	False
GreaterThanBenchmark - Duration of single invocation	59.80 ns	1.33 ns	0.02	0.02	True
EqualsBenchmark - Duration of single invocation	6.03 ns	0.82 ns	0.14	0.23	False
ConditionalSelectBenchmark - Duration of single invocation	15.52 ns	1.09 ns	0.07	0.02	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

Repro Steps

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
AOT MONO build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false
Dotnet SDK installed for dotnet commands
Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create aot directory 
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Byte&gt;*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Byte&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Byte&gt;*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Byte&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200

Payloads

Baseline
Compare

Histogram

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Byte>.MinBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0265872892210974 < 62.7106619626753.
IsChangePoint: Marked as a change because one of 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 750.6894346500455 (T) = (0 -1.1170289570309044) / Math.Sqrt((1.9272318719685395 / (299)) + (0.004684356663843583 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.9829765858335314 = (65.61721086661623 - 1.1170289570309044) / 65.61721086661623 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Byte>.MaxBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.177027860465191 < 60.61538114192385.
IsChangePoint: Marked as a change because one of 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 639.8193189140269 (T) = (0 -1.0591092146852326) / Math.Sqrt((2.5115241426180885 / (299)) + (0.006417686671740009 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.9834565063578046 = (64.01968275817488 - 1.0591092146852326) / 64.01968275817488 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Byte>.GreaterThanOrEqualAllBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.027900336781856563 < 34.3850525872527.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 205.74161573387238 (T) = (0 -0.10984630806863568) / Math.Sqrt((9.874315756554905 / (299)) + (0.025499106634500793 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9973174194051847 = (40.947999206785504 - 0.10984630806863568) / 40.947999206785504 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Byte>.LessThanOrEqualAnyBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 34.63227182947145.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 229.92248077068746 (T) = (0 -0) / Math.Sqrt((9.389079149902019 / (299)) + (0 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 1 = (40.74340703086422 - 0) / 40.74340703086422 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Byte>.AndNotBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.3046606926945505 < 12.87877046816437.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 4/12/2023 8:39:35 PM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 82.60967280658188 (T) = (0 -1.152739593282662) / Math.Sqrt((0.37548961423977933 / (299)) + (0.0461600408930338 / (2))) is greater than 1.9679296690655894 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (2) - 2, .975) and 0.9178949047196137 = (14.039805804330326 - 1.152739593282662) / 14.039805804330326 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Byte>.GreaterThanOrEqualBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.151147718182235 < 61.896438023137634.
IsChangePoint: Marked as a change because one of 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 645.8025302449868 (T) = (0 -1.0990819375871688) / Math.Sqrt((2.6632238877067693 / (299)) + (0.004095797374636306 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9832089251125494 = (65.45631801145797 - 1.0990819375871688) / 65.45631801145797 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Byte>.SumBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.48921775901202685 < 21.541844572153817.
IsChangePoint: Marked as a change because one of 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 184.7011983069481 (T) = (0 -0.13120804617281953) / Math.Sqrt((0.6086721281428809 / (299)) + (0.05724673250908585 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9944746417127868 = (23.746522732554823 - 0.13120804617281953) / 23.746522732554823 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Byte>.EqualsAnyBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.11526276725751598 < 31.135279287127208.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 201.64054147239315 (T) = (0 -0.03067741561015594) / Math.Sqrt((9.81329258588967 / (299)) + (0.0031898163692352485 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9991709211068719 = (37.00180509289151 - 0.03067741561015594) / 37.00180509289151 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Byte>.GreaterThanAllBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 31.38027206926111.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 211.55245569152024 (T) = (0 -0) / Math.Sqrt((9.173956714175558 / (299)) + (0 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 1 = (37.05619474711515 - 0) / 37.05619474711515 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Byte>.LessThanOrEqualBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0371074467415615 < 59.25863405498676.
IsChangePoint: Marked as a change because one of 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 569.7092118972317 (T) = (0 -1.044128328852748) / Math.Sqrt((3.1611432703229188 / (299)) + (0.006164517790369662 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.9834118572332264 = (62.94425744539409 - 1.044128328852748) / 62.94425744539409 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Byte>.LessThanAnyBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 31.327395836981633.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 202.1262167650613 (T) = (0 -0.005563938438949628) / Math.Sqrt((10.022486748856185 / (299)) + (0.00010566049475113814 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9998497303765451 = (37.02636841049424 - 0.005563938438949628) / 37.02636841049424 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Byte>.GreaterThanOrEqualAnyBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.00024514106686756014 < 3.7012572967722606.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 81.25744951254846 (T) = (0 -0.0061720389766051886) / Math.Sqrt((1.6494341081589838 / (299)) + (7.056996969959209E-05 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9989800073983104 = (6.051062494356542 - 0.0061720389766051886) / 6.051062494356542 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Byte>.EqualsStaticBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.1158914486956368 < 53.68653757164844.
IsChangePoint: Marked as a change because one of 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 400.20501430389874 (T) = (0 -1.0617823396803245) / Math.Sqrt((5.2697424325236435 / (299)) + (0.008578297516864305 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.981480014482973 = (57.33170464437679 - 1.0617823396803245) / 57.33170464437679 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Byte>.LessThanBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.3291432496538131 < 53.27751121504436.
IsChangePoint: Marked as a change because one of 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 368.5695801271828 (T) = (0 -1.2142524804791863) / Math.Sqrt((5.345122944939153 / (299)) + (0.021253799407576022 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9788240601692607 = (57.34113763945252 - 1.2142524804791863) / 57.34113763945252 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Byte>.LessThanOrEqualAllBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 3.631253246537694.
IsChangePoint: Marked as a change because one of 3/22/2023 1:13:01 AM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 80.44225851592506 (T) = (0 -0.0013749466776625352) / Math.Sqrt((1.7114330765121906 / (299)) + (5.5386036617598295E-06 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.9997741515478711 = (6.087917205993888 - 0.0013749466776625352) / 6.087917205993888 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Byte>.LessThanAllBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.31640670020624184 < 3.088130760809301.
IsChangePoint: Marked as a change because one of 3/21/2023 12:49:42 AM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 53.46011657007868 (T) = (0 -0.06481866932928104) / Math.Sqrt((1.7411802589234495 / (299)) + (0.019791246250201323 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.9878891783624445 = (5.352128143666046 - 0.06481866932928104) / 5.352128143666046 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Byte>.GreaterThanAnyBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 3.150135518456894.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 71.6500095011699 (T) = (0 -0.004392695364599492) / Math.Sqrt((1.6706219881568578 / (299)) + (2.7643326916201503E-05 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9991809940081802 = (5.363447164579742 - 0.004392695364599492) / 5.363447164579742 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Byte>.GreaterThanBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.3259431979219813 < 56.76067939639708.
IsChangePoint: Marked as a change because one of 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 456.5520949872018 (T) = (0 -1.1051896526672433) / Math.Sqrt((3.372508277019584 / (299)) + (0.02308478884682078 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9817987784181693 = (60.72063062901737 - 1.1051896526672433) / 60.72063062901737 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Byte>.EqualsBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.8215588514020351 < 5.846971376142552.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 68.17656797526209 (T) = (0 -0.8228990386305134) / Math.Sqrt((4.356719387844839 / (299)) + (0.004536299547607211 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9121469822053773 = (9.366770308952109 - 0.8228990386305134) / 9.366770308952109 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Byte>.ConditionalSelectBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0896118748042345 < 14.809505729591605.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 4/12/2023 8:39:35 PM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 394.12746445136474 (T) = (0 -1.104563899609968) / Math.Sqrt((0.37024710315900883 / (299)) + (0.0004471260915825491 / (2))) is greater than 1.9679296690655894 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (2) - 2, .975) and 0.9317060838674927 = (16.173679328431497 - 1.104563899609968) / 16.173679328431497 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

performanceautofiler · 2023-04-18T16:21:38Z

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	7a57f6dd800e77508a4ba405e623059935c82f43
Compare	887c043eb94be364188e2b23a87efa214ea57f1e
Diff	Diff
Configs	CompilationMode:tiered, LLVM:false, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Improvements in System.Runtime.Intrinsics.Tests.Perf_Vector128Float

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
EqualsBenchmark - Duration of single invocation	63.94 ns	1.42 ns	0.02	0.02	False
MaxBenchmark - Duration of single invocation	39.63 ns	1.14 ns	0.03	0.03	False
LessThanBenchmark - Duration of single invocation	33.74 ns	1.03 ns	0.03	0.06	False
LessThanOrEqualAllBenchmark - Duration of single invocation	9.27 ns	0.00 ns	0.00	0.13	False
GreaterThanBenchmark - Duration of single invocation	32.41 ns	1.04 ns	0.03	0.04	False
AbsBenchmark - Duration of single invocation	78.60 ns	1.20 ns	0.02	0.03	False
LessThanOrEqualAnyBenchmark - Duration of single invocation	17.69 ns	0.02 ns	0.00	0.09	True
SquareRootBenchmark - Duration of single invocation	72.48 ns	1.04 ns	0.01	0.58	False
EqualsStaticBenchmark - Duration of single invocation	33.69 ns	1.15 ns	0.03	0.07	False
GreaterThanAnyBenchmark - Duration of single invocation	17.43 ns	0.00 ns	0.00	0.10	False
GreaterThanOrEqualAllBenchmark - Duration of single invocation	9.30 ns	0.00 ns	0.00	0.12	False
EqualsAnyBenchmark - Duration of single invocation	16.82 ns	0.05 ns	0.00	0.08	False
AndNotBenchmark - Duration of single invocation	13.43 ns	1.07 ns	0.08	0.09	False
LessThanAllBenchmark - Duration of single invocation	8.07 ns	0.00 ns	0.00	0.15	False
MinBenchmark - Duration of single invocation	39.59 ns	1.04 ns	0.03	0.04	False
ConditionalSelectBenchmark - Duration of single invocation	15.45 ns	1.12 ns	0.07	0.08	False
GreaterThanAllBenchmark - Duration of single invocation	8.53 ns	0.00 ns	0.00	0.16	False
GreaterThanOrEqualAnyBenchmark - Duration of single invocation	17.44 ns	0.00 ns	0.00	0.06	True
LessThanOrEqualBenchmark - Duration of single invocation	33.59 ns	1.03 ns	0.03	0.06	False
GreaterThanOrEqualBenchmark - Duration of single invocation	34.82 ns	1.04 ns	0.03	0.04	False
SumBenchmark - Duration of single invocation	29.28 ns	0.04 ns	0.00	0.05	False
LessThanAnyBenchmark - Duration of single invocation	16.78 ns	0.03 ns	0.00	0.08	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

Repro Steps

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
AOT MONO build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false
Dotnet SDK installed for dotnet commands
Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create aot directory 
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Float*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Runtime.Intrinsics.Tests.Perf_Vector128Float* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Float*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Runtime.Intrinsics.Tests.Perf_Vector128Float* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200

Payloads

Baseline
Compare

Histogram

System.Runtime.Intrinsics.Tests.Perf_Vector128Float.EqualsBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.418692323164142 < 60.79306590508124.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 768.7837823488813 (T) = (0 -1.3818524748166259) / Math.Sqrt((1.4513523640705883 / (299)) + (0.00886924744137406 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9790722961818279 = (66.02981802603324 - 1.3818524748166259) / 66.02981802603324 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Float.MaxBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.1404584555276236 < 37.744662362762945.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 189.80626700361353 (T) = (0 -1.155869604690052) / Math.Sqrt((13.779767877582731 / (299)) + (0.008030216816252475 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9729816577611397 = (42.780922473754565 - 1.155869604690052) / 42.780922473754565 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Float.LessThanBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.03188822892308 < 31.70267265182251.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 132.7231537504239 (T) = (0 -1.0301830732216761) / Math.Sqrt((22.08169831292192 / (299)) + (0.007035242987618407 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9725471822640263 = (37.525586011950274 - 1.0301830732216761) / 37.525586011950274 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Float.LessThanOrEqualAllBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 8.80888140919075.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 141.3367524228207 (T) = (0 -0.03347258652733678) / Math.Sqrt((1.549016064254783 / (299)) + (0.00448165619532019 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9970253681155171 = (11.252681954344146 - 0.03347258652733678) / 11.252681954344146 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Float.GreaterThanBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0423688111537528 < 31.98339613512628.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 134.45572506470936 (T) = (0 -1.1465444638073088) / Math.Sqrt((21.133947561567314 / (299)) + (0.009572782534566312 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9694199447256361 = (37.49321096775417 - 1.1465444638073088) / 37.49321096775417 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Float.AbsBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.1976555536930589 < 74.68205825621803.
IsChangePoint: Marked as a change because one of 3/8/2023 4:26:04 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 81.31690803146111 (T) = (0 -1.2048151771999212) / Math.Sqrt((163.0971782951037 / (299)) + (0.010769655372754058 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.9803715113853659 = (61.380944852863905 - 1.2048151771999212) / 61.380944852863905 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Float.LessThanOrEqualAnyBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.022517650932647206 < 16.67980272963609.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 285.9199362513877 (T) = (0 -0.060924691622854486) / Math.Sqrt((1.0639333931912325 / (299)) + (0.0035274270111000706 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9968124066813333 = (19.113069181716934 - 0.060924691622854486) / 19.113069181716934 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Float.SquareRootBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0351303773230085 < 61.94102310883152.
IsChangePoint: Marked as a change because one of 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 193.23890336447178 (T) = (0 -1.201236028215467) / Math.Sqrt((32.33608269698363 / (299)) + (0.013777917073154608 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.9816755860172908 = (65.55385778497154 - 1.201236028215467) / 65.55385778497154 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Float.EqualsStaticBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.1547941024128563 < 32.128148146500514.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 134.27109459492812 (T) = (0 -1.134679050478927) / Math.Sqrt((21.87177611818811 / (299)) + (0.00048247194030125455 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9697256251244442 = (37.47985070354307 - 1.134679050478927) / 37.47985070354307 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Float.GreaterThanAnyBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 16.12317360122377.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 207.06398901670042 (T) = (0 -0.0898752342774425) / Math.Sqrt((1.1407466508088822 / (299)) + (0.01708734325977169 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9951965918589207 = (18.710721978592858 - 0.0898752342774425) / 18.710721978592858 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Float.GreaterThanOrEqualAllBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 8.833755152655199.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 146.93805002196092 (T) = (0 -0.04600111697319458) / Math.Sqrt((1.5290795125442362 / (299)) + (0.0028806787068165894 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9959180377602365 = (11.269363671492146 - 0.04600111697319458) / 11.269363671492146 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Float.EqualsAnyBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.05365938880178752 < 16.113008196055574.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 244.89465917304227 (T) = (0 -0.06632148301420325) / Math.Sqrt((1.140713175148538 / (299)) + (0.007793183992116468 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9964454108292861 = (18.657988259409084 - 0.06632148301420325) / 18.657988259409084 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Float.AndNotBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0703821028686098 < 12.761128745064862.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 4/12/2023 8:39:35 PM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 220.67705138220657 (T) = (0 -1.116142727670421) / Math.Sqrt((0.39604284862210853 / (299)) + (0.004188069564504277 / (2))) is greater than 1.9679296690655894 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (2) - 2, .975) and 0.9203827209807612 = (14.018850448289204 - 1.116142727670421) / 14.018850448289204 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Float.LessThanAllBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 8.099726249955642.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 145.92345203514913 (T) = (0 -0.00933254271863905) / Math.Sqrt((1.5389354493947642 / (299)) + (0.00028589753958582564 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9991154523523968 = (10.5506387857413 - 0.00933254271863905) / 10.5506387857413 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Float.MinBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0366628247339984 < 37.72549496178934.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 193.26805623628323 (T) = (0 -1.0482947171356192) / Math.Sqrt((13.86171396582334 / (299)) + (0.001852116079802441 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.975522909989267 = (42.82758762074878 - 1.0482947171356192) / 42.82758762074878 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Float.ConditionalSelectBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.1170253169113622 < 14.721672468084574.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 4/12/2023 8:39:35 PM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 392.1595918601623 (T) = (0 -1.1256164825441206) / Math.Sqrt((0.4163178643412197 / (299)) + (0.00014761625385897853 / (2))) is greater than 1.9679296690655894 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (2) - 2, .975) and 0.9302664206180631 = (16.14167080652809 - 1.1256164825441206) / 16.14167080652809 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Float.GreaterThanAllBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.0006317359598446823 < 8.199064639738376.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 104.46473541771799 (T) = (0 -0.12961591336767878) / Math.Sqrt((1.5117139610235144 / (299)) + (0.01967679202615741 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9877293379874235 = (10.563074203725328 - 0.12961591336767878) / 10.563074203725328 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Float.GreaterThanOrEqualAnyBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 16.68780206471084.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 233.42925541074632 (T) = (0 -0.0674595265529945) / Math.Sqrt((1.0908362529282378 / (299)) + (0.01199137705733608 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.996467624124232 = (19.097493847062974 - 0.0674595265529945) / 19.097493847062974 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Float.LessThanOrEqualBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0271236229472627 < 32.43472252850435.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 127.09628738684285 (T) = (0 -1.04609252031126) / Math.Sqrt((26.24098194726355 / (299)) + (0.008618172469527622 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.973284932921972 = (39.15739822984122 - 1.04609252031126) / 39.15739822984122 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Float.GreaterThanOrEqualBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.041996825035418 < 32.91818890144715.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 123.83441325346178 (T) = (0 -1.1655210134464522) / Math.Sqrt((26.482564196222064 / (299)) + (0.02180469459789225 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9702193070248316 = (39.136799617734845 - 1.1655210134464522) / 39.136799617734845 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Float.SumBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.04381993752062968 < 27.740780347420735.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 593.693640156406 (T) = (0 -0.02445692535928955) / Math.Sqrt((0.6114394494884284 / (299)) + (0.0012715470993283164 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.9991416335785711 = (28.492406912395413 - 0.02445692535928955) / 28.492406912395413 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Float.LessThanAnyBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.034834334782723034 < 16.058528944075988.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 297.43638394324233 (T) = (0 -0.026106895902684998) / Math.Sqrt((1.0946306067522407 / (299)) + (0.0011101237024995294 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9986033468027866 = (18.692468505979413 - 0.026106895902684998) / 18.692468505979413 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

performanceautofiler · 2023-04-18T16:21:39Z

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	7a57f6dd800e77508a4ba405e623059935c82f43
Compare	887c043eb94be364188e2b23a87efa214ea57f1e
Diff	Diff
Configs	CompilationMode:tiered, LLVM:false, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Improvements in System.Numerics.Tests.Perf_VectorOf<Single>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
EqualsAnyBenchmark - Duration of single invocation	13.09 ns	0.00 ns	0.00	0.07	True
LessThanAllBenchmark - Duration of single invocation	12.89 ns	0.00 ns	0.00	0.06	True
GreaterThanOrEqualAllBenchmark - Duration of single invocation	5.55 ns	0.16 ns	0.03	0.14	False
LessThanOrEqualBenchmark - Duration of single invocation	40.67 ns	1.03 ns	0.03	0.02	False
GreaterThanOrEqualAnyBenchmark - Duration of single invocation	13.56 ns	0.00 ns	0.00	0.05	True
LessThanOrEqualAnyBenchmark - Duration of single invocation	5.65 ns	0.08 ns	0.01	0.14	False
EqualsStaticBenchmark - Duration of single invocation	27.98 ns	1.16 ns	0.04	0.02	True
MinBenchmark - Duration of single invocation	27.77 ns	1.12 ns	0.04	0.04	True
GreaterThanOrEqualBenchmark - Duration of single invocation	29.01 ns	1.05 ns	0.04	0.02	True
GreaterThanBenchmark - Duration of single invocation	27.63 ns	1.15 ns	0.04	0.02	True
AbsBenchmark - Duration of single invocation	72.01 ns	0.97 ns	0.01	0.03	False
GreaterThanAnyBenchmark - Duration of single invocation	12.82 ns	0.00 ns	0.00	0.06	False
EqualsBenchmark - Duration of single invocation	14.65 ns	0.01 ns	0.00	0.08	False
LessThanAnyBenchmark - Duration of single invocation	4.92 ns	0.00 ns	0.00	0.19	False
LessThanOrEqualAllBenchmark - Duration of single invocation	13.54 ns	0.00 ns	0.00	0.06	True
LessThanBenchmark - Duration of single invocation	40.61 ns	1.05 ns	0.03	0.02	False
GreaterThanAllBenchmark - Duration of single invocation	4.73 ns	0.04 ns	0.01	0.22	False
MaxBenchmark - Duration of single invocation	28.16 ns	1.15 ns	0.04	0.04	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

Repro Steps

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
AOT MONO build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false
Dotnet SDK installed for dotnet commands
Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create aot directory 
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_VectorOf<Single>.EqualsAnyBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 12.292989818443322.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 297.9467039333178 (T) = (0 -0.005028943802420911) / Math.Sqrt((0.6885665069631141 / (299)) + (4.7516909193015814E-05 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.9996491235866408 = (14.332521682704279 - 0.005028943802420911) / 14.332521682704279 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Single>.LessThanAllBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 12.241049713282626.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 115.4419727179337 (T) = (0 -0.11298275785990283) / Math.Sqrt((0.6961246767387036 / (299)) + (0.05106041429451774 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9920966610284296 = (14.295572828942404 - 0.11298275785990283) / 14.295572828942404 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Single>.GreaterThanOrEqualAllBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.16154105448813508 < 5.293467736085792.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 77.80023385895481 (T) = (0 -0.049228310772488) / Math.Sqrt((2.681867355906501 / (299)) + (0.005701681265924283 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9938320991063959 = (7.981371883510057 - 0.049228310772488) / 7.981371883510057 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Single>.LessThanOrEqualBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0345547536360273 < 38.65291158257902.
IsChangePoint: Marked as a change because one of 3/8/2023 4:26:04 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 156.6115326130026 (T) = (0 -1.0635420900291106) / Math.Sqrt((13.658216393700139 / (299)) + (0.009207557495917798 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9699299461322597 = (35.36881226442017 - 1.0635420900291106) / 35.36881226442017 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Single>.GreaterThanOrEqualAnyBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 12.841008407430795.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 307.1909917650733 (T) = (0 -0.002539369684065823) / Math.Sqrt((0.7059136815376167 / (299)) + (2.5793593569410236E-05 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.99983013215331 = (14.949089739736639 - 0.002539369684065823) / 14.949089739736639 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Single>.LessThanOrEqualAnyBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.08341290589989453 < 5.352013771964093.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 76.0078004285278 (T) = (0 -0.06570099501248634) / Math.Sqrt((2.7130432447040733 / (299)) + (0.0072907957909502255 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9917872173778473 = (7.9998458543475826 - 0.06570099501248634) / 7.9998458543475826 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Single>.EqualsStaticBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.161253598171172 < 26.48617181424411.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 266.96830574730535 (T) = (0 -1.056472043640446) / Math.Sqrt((1.8816058663261512 / (299)) + (0.010193757480180986 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.9584473698387063 = (25.424913887269394 - 1.056472043640446) / 25.424913887269394 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Single>.MinBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.1198606762607575 < 26.437945716791763.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 254.3078712299934 (T) = (0 -1.1697789095491913) / Math.Sqrt((1.4903324492525816 / (299)) + (0.017884179631783837 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9548320648601554 = (25.89843671018908 - 1.1697789095491913) / 25.89843671018908 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Single>.GreaterThanOrEqualBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0497792886713324 < 27.548788985101986.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 268.483581674961 (T) = (0 -1.10177339953223) / Math.Sqrt((1.7617295949613752 / (299)) + (0.012788224630048027 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9587322521228441 = (26.698171240454997 - 1.10177339953223) / 26.698171240454997 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Single>.GreaterThanBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.146230111973555 < 26.261075137770092.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 281.8879272147045 (T) = (0 -1.0380097407882736) / Math.Sqrt((1.8217226370446244 / (299)) + (0.005537740708359839 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9591543944293491 = (25.41300897088725 - 1.0380097407882736) / 25.41300897088725 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Single>.AbsBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.9709418660696281 < 68.22445361892265.
IsChangePoint: Marked as a change because one of 3/8/2023 4:26:04 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 62.265324015706874 (T) = (0 -1.1516874954916447) / Math.Sqrt((201.36559231213258 / (299)) + (0.03315829958237157 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9780894433247451 = (52.56313258313163 - 1.1516874954916447) / 52.56313258313163 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Single>.GreaterThanAnyBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.003583796802820326 < 12.311870433744534.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 299.0659414645279 (T) = (0 -0.001194598934273442) / Math.Sqrt((0.6817085359464942 / (299)) + (4.28119984130173E-06 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.9999163782595472 = (14.285745881436757 - 0.001194598934273442) / 14.285745881436757 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Single>.EqualsBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.013154737999979795 < 13.914470339279369.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 152.86648958971696 (T) = (0 -0.0032886844999949487) / Math.Sqrt((3.935068433245308 / (299)) + (4.326178296202811E-05 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9998125827378754 = (17.547393781746315 - 0.0032886844999949487) / 17.547393781746315 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Single>.LessThanAnyBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 4.529989377760601.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 72.54295044165077 (T) = (0 -0.04669769757048146) / Math.Sqrt((2.1768272179872983 / (299)) + (0.0087226998335366 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9934254468252345 = (7.10279411074152 - 0.04669769757048146) / 7.10279411074152 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Single>.LessThanOrEqualAllBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.0018269328043338205 < 12.870318536900054.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 305.1871729137103 (T) = (0 -0.001557623270677771) / Math.Sqrt((0.7149071740426246 / (299)) + (4.341421861141803E-06 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9998956570644181 = (14.927922642693542 - 0.001557623270677771) / 14.927922642693542 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Single>.LessThanBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0486368954141059 < 38.51546234510024.
IsChangePoint: Marked as a change because one of 3/8/2023 4:26:04 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 154.76317386119084 (T) = (0 -1.1564294306452105) / Math.Sqrt((13.882338607454836 / (299)) + (0.006873754522756556 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9670677946447913 = (35.11545668356846 - 1.1564294306452105) / 35.11545668356846 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Single>.GreaterThanAllBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.036202064392858606 < 4.507375095128238.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/22/2023 1:13:01 AM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 81.71677167422958 (T) = (0 -0.03542387391373594) / Math.Sqrt((2.153020054185471 / (299)) + (0.0012436139846091993 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9950231942581614 = (7.117793169208459 - 0.03542387391373594) / 7.117793169208459 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Single>.MaxBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.14645871987686 < 26.701070557212716.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 285.5901363666154 (T) = (0 -1.0708837937155358) / Math.Sqrt((1.5725979924797822 / (299)) + (0.009095413732098582 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9585870830837384 = (25.858690318310604 - 1.0708837937155358) / 25.858690318310604 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

performanceautofiler · 2023-04-18T16:21:40Z

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	7a57f6dd800e77508a4ba405e623059935c82f43
Compare	887c043eb94be364188e2b23a87efa214ea57f1e
Diff	Diff
Configs	CompilationMode:tiered, LLVM:false, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Improvements in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int16>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
EqualsStaticBenchmark - Duration of single invocation	37.67 ns	1.37 ns	0.04	0.03	True
LessThanOrEqualAllBenchmark - Duration of single invocation	4.12 ns	0.00 ns	0.00	0.19	False
MinBenchmark - Duration of single invocation	44.53 ns	1.04 ns	0.02	0.04	True
EqualsAnyBenchmark - Duration of single invocation	18.42 ns	0.00 ns	0.00	0.05	False
EqualsBenchmark - Duration of single invocation	6.08 ns	0.77 ns	0.13	0.23	False
ConditionalSelectBenchmark - Duration of single invocation	15.65 ns	1.33 ns	0.09	0.04	False
SumBenchmark - Duration of single invocation	10.81 ns	0.00 ns	0.00	0.05	True
GreaterThanBenchmark - Duration of single invocation	39.53 ns	1.29 ns	0.03	0.02	True
LessThanOrEqualBenchmark - Duration of single invocation	40.48 ns	1.17 ns	0.03	0.02	True
GreaterThanOrEqualAnyBenchmark - Duration of single invocation	3.82 ns	0.00 ns	0.00	0.22	False
GreaterThanAllBenchmark - Duration of single invocation	18.83 ns	0.00 ns	0.00	0.05	True
MaxBenchmark - Duration of single invocation	43.10 ns	1.03 ns	0.02	0.02	True
AndNotBenchmark - Duration of single invocation	13.63 ns	1.18 ns	0.09	0.05	False
GreaterThanOrEqualAllBenchmark - Duration of single invocation	20.70 ns	0.00 ns	0.00	0.03	True
LessThanAnyBenchmark - Duration of single invocation	18.67 ns	0.00 ns	0.00	0.03	True
LessThanOrEqualAnyBenchmark - Duration of single invocation	20.44 ns	0.06 ns	0.00	0.05	False
GreaterThanOrEqualBenchmark - Duration of single invocation	41.98 ns	1.02 ns	0.02	0.02	True
AbsBenchmark - Duration of single invocation	35.22 ns	1.20 ns	0.03	0.01	True
LessThanAllBenchmark - Duration of single invocation	3.05 ns	0.00 ns	0.00	0.22	False
GreaterThanAnyBenchmark - Duration of single invocation	3.32 ns	0.00 ns	0.00	0.22	False
LessThanBenchmark - Duration of single invocation	37.11 ns	1.20 ns	0.03	0.02	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

Repro Steps

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
AOT MONO build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false
Dotnet SDK installed for dotnet commands
Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create aot directory 
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int16&gt;*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int16&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int16&gt;*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int16&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200

Payloads

Baseline
Compare

Histogram

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int16>.EqualsStaticBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.3660310014997843 < 36.02440641916293.
IsChangePoint: Marked as a change because one of 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 389.63920307883126 (T) = (0 -1.148468034316466) / Math.Sqrt((0.9676738830551882 / (299)) + (0.022997668859418837 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9698435683700803 = (38.08368471477286 - 1.148468034316466) / 38.08368471477286 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int16>.LessThanOrEqualAllBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 3.651543609395123.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 80.40272141972149 (T) = (0 -0) / Math.Sqrt((1.6879066733504067 / (299)) + (0 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 1 = (6.041007427033596 - 0) / 6.041007427033596 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int16>.MinBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.035490457256608 < 42.30510617448029.
IsChangePoint: Marked as a change because one of 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 518.9774097887945 (T) = (0 -1.085749564710631) / Math.Sqrt((0.723556591542833 / (299)) + (0.016699574262084224 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.974884994484442 = (43.23110994493084 - 1.085749564710631) / 43.23110994493084 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int16>.EqualsAnyBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.0014607087541930349 < 17.784725795924178.
IsChangePoint: Marked as a change because one of 3/21/2023 8:47:40 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 156.88995237572217 (T) = (0 -0.09627963700538447) / Math.Sqrt((3.218623758804021 / (299)) + (0.03202601864524501 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9955408389861486 = (21.59142419533942 - 0.09627963700538447) / 21.59142419533942 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int16>.EqualsBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.76714822214762 < 5.752063283896412.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 40.02800054980103 (T) = (0 -0.9960306270251514) / Math.Sqrt((4.375590443575416 / (299)) + (0.1401614783973075 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.8924854912311946 = (9.26415084281297 - 0.9960306270251514) / 9.26415084281297 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int16>.ConditionalSelectBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.3338073947284792 < 14.816972921694557.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 4/12/2023 8:39:35 PM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 177.15537109706614 (T) = (0 -1.2580225552803612) / Math.Sqrt((0.396012892848062 / (299)) + (0.011486683780354038 / (2))) is greater than 1.9679296690655894 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (2) - 2, .975) and 0.9221111552850981 = (16.15151129645235 - 1.2580225552803612) / 16.15151129645235 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int16>.SumBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 10.291796601574424.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 241.16911752027391 (T) = (0 -0) / Math.Sqrt((0.7754546608537304 / (299)) + (0 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 1 = (12.281864603957814 - 0) / 12.281864603957814 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int16>.GreaterThanBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.2891069753006694 < 37.250258970498805.
IsChangePoint: Marked as a change because one of 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 127.58917931031719 (T) = (0 -1.4225239815957527) / Math.Sqrt((1.9151237829569467 / (299)) + (0.33989904955754463 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9644294188010971 = (39.99158668904816 - 1.4225239815957527) / 39.99158668904816 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int16>.LessThanOrEqualBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.1709273009784986 < 38.63824355113291.
IsChangePoint: Marked as a change because one of 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 537.1217278438871 (T) = (0 -1.1158392883477477) / Math.Sqrt((0.7551246661690815 / (299)) + (0.012042729223763806 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9728378481442099 = (41.08066600437224 - 1.1158392883477477) / 41.08066600437224 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int16>.GreaterThanOrEqualAnyBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 3.6357241307246033.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 80.38172690655496 (T) = (0 -0.005318678494357911) / Math.Sqrt((1.7017821875060115 / (299)) + (8.452749132351524E-05 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.999125004513296 = (6.078521061168103 - 0.005318678494357911) / 6.078521061168103 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int16>.GreaterThanAllBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 17.89110830596938.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 171.1296193540671 (T) = (0 -0.07820538052865116) / Math.Sqrt((3.30223442959116 / (299)) + (0.024529473935694074 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.9963945431093719 = (21.69083777757393 - 0.07820538052865116) / 21.69083777757393 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int16>.MaxBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0303482108371473 < 41.568948114110455.
IsChangePoint: Marked as a change because one of 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 606.8050847492174 (T) = (0 -1.1393759347748866) / Math.Sqrt((0.45494696227595866 / (299)) + (0.012692120971521807 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9733266526744097 = (42.715896166574346 - 1.1393759347748866) / 42.715896166574346 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int16>.AndNotBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.1836985401589974 < 12.867988263044165.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 4/12/2023 8:39:35 PM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 334.4120923591401 (T) = (0 -1.173726890921916) / Math.Sqrt((0.4125434777165157 / (299)) + (0.00019886757701477584 / (2))) is greater than 1.9679296690655894 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (2) - 2, .975) and 0.9163728853447785 = (14.035243183517286 - 1.173726890921916) / 14.035243183517286 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int16>.GreaterThanOrEqualAllBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 19.862209217799567.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 225.01011725929163 (T) = (0 -0.0037282698197539143) / Math.Sqrt((3.341878350861212 / (299)) + (6.949997924443942E-05 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.9998433942653686 = (23.806726034193733 - 0.0037282698197539143) / 23.806726034193733 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int16>.LessThanAnyBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 17.70910536574174.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 204.17481758072398 (T) = (0 -0) / Math.Sqrt((3.340987117780002 / (299)) + (0 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 1 = (21.582610972322996 - 0) / 21.582610972322996 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int16>.LessThanOrEqualAnyBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.06100923142038542 < 19.573205689666036.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 219.40346575168965 (T) = (0 -0.020336410473461807) / Math.Sqrt((3.32966100932465 / (299)) + (0.0012407087728353808 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.999138265073127 = (23.599380551112244 - 0.020336410473461807) / 23.599380551112244 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int16>.GreaterThanOrEqualBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0167379726008703 < 39.94715015557373.
IsChangePoint: Marked as a change because one of 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 546.658782728534 (T) = (0 -1.0974238173496556) / Math.Sqrt((1.5418205173389543 / (299)) + (0.0033689466071071903 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.9743823641681968 = (42.8386063630139 - 1.0974238173496556) / 42.8386063630139 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int16>.AbsBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.1958762394470996 < 33.41233973585393.
IsChangePoint: Marked as a change because one of 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 611.5441801576259 (T) = (0 -1.092786060802359) / Math.Sqrt((0.2787789643707561 / (299)) + (0.008872532938512121 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9691463442863194 = (35.41836568552289 - 1.092786060802359) / 35.41836568552289 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int16>.LessThanAllBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 2.954032233292274.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 70.90026226704443 (T) = (0 -0.009744643211055459) / Math.Sqrt((1.657642865420442 / (299)) + (0.0004747903565538462 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.9981730519395966 = (5.333837027038225 - 0.009744643211055459) / 5.333837027038225 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int16>.GreaterThanAnyBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 3.1148747419933476.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 68.75262335354051 (T) = (0 -0.0170437080135616) / Math.Sqrt((1.7060222247564802 / (299)) + (0.0011619519314061758 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9968088522918248 = (5.340933598873724 - 0.0170437080135616) / 5.340933598873724 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int16>.LessThanBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.2027578115867992 < 36.01967455839621.
IsChangePoint: Marked as a change because one of 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 557.4546720353605 (T) = (0 -1.0937626852817246) / Math.Sqrt((0.9524903015665485 / (299)) + (0.006062720366349798 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.9712646123784796 = (38.06326539554274 - 1.0937626852817246) / 38.06326539554274 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

performanceautofiler · 2023-04-18T16:21:42Z

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	7a57f6dd800e77508a4ba405e623059935c82f43
Compare	887c043eb94be364188e2b23a87efa214ea57f1e
Diff	Diff
Configs	CompilationMode:tiered, LLVM:false, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Improvements in System.Numerics.Tests.Perf_VectorOf<Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
AbsBenchmark - Duration of single invocation	11.02 ns	1.06 ns	0.10	0.22	False
GreaterThanAnyBenchmark - Duration of single invocation	4.34 ns	0.01 ns	0.00	0.17	False
GreaterThanBenchmark - Duration of single invocation	12.80 ns	1.11 ns	0.09	0.06	True
GreaterThanOrEqualBenchmark - Duration of single invocation	14.45 ns	1.15 ns	0.08	0.05	True
MinBenchmark - Duration of single invocation	13.58 ns	1.01 ns	0.07	0.04	True
GreaterThanOrEqualAnyBenchmark - Duration of single invocation	5.65 ns	0.00 ns	0.00	0.17	False
EqualsBenchmark - Duration of single invocation	4.60 ns	0.00 ns	0.00	0.16	False
LessThanBenchmark - Duration of single invocation	12.85 ns	1.10 ns	0.09	0.05	True
LessThanOrEqualAllBenchmark - Duration of single invocation	5.86 ns	0.00 ns	0.00	0.14	False
LessThanAllBenchmark - Duration of single invocation	4.26 ns	0.00 ns	0.00	0.15	False
LessThanOrEqualBenchmark - Duration of single invocation	14.16 ns	1.11 ns	0.08	0.04	True
EqualsAnyBenchmark - Duration of single invocation	4.71 ns	0.00 ns	0.00	0.19	False
MaxBenchmark - Duration of single invocation	13.83 ns	1.15 ns	0.08	0.03	True
EqualsStaticBenchmark - Duration of single invocation	12.90 ns	1.13 ns	0.09	0.07	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

Repro Steps

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
AOT MONO build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false
Dotnet SDK installed for dotnet commands
Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create aot directory 
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_VectorOf<Int32>.AbsBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.060115744642584 < 10.18921875601527.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 241.45650502103624 (T) = (0 -1.0571160278771903) / Math.Sqrt((0.32278526412977776 / (299)) + (0.002329095339594081 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.8997904510817814 = (10.549054848454679 - 1.0571160278771903) / 10.549054848454679 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Int32>.GreaterThanAnyBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.005341274917560139 < 4.079776854360187.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 78.52551212815483 (T) = (0 -0.005190676582722336) / Math.Sqrt((2.2463525828802617 / (299)) + (5.2859090345095225E-05 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9992386270837542 = (6.817521968494344 - 0.005190676582722336) / 6.817521968494344 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Int32>.GreaterThanBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.1076902628462122 < 12.207063732516175.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 188.35063980660004 (T) = (0 -1.0870994317493072) / Math.Sqrt((0.5569889983760135 / (299)) + (0.01141228818272777 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9224697014823351 = (14.021607713810306 - 1.0870994317493072) / 14.021607713810306 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Int32>.GreaterThanOrEqualBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.1529795348455256 < 13.712219695260057.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 292.5317633729714 (T) = (0 -1.1916684134838773) / Math.Sqrt((0.23619894413610523 / (299)) + (0.005812350805301133 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9207994995107183 = (15.046223270333353 - 1.1916684134838773) / 15.046223270333353 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Int32>.MinBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0103883999815007 < 12.939782272768882.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 207.14135053595737 (T) = (0 -1.0320838691869885) / Math.Sqrt((0.29776888056954787 / (299)) + (0.016276965278159704 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.9290082127039113 = (14.538074170219572 - 1.0320838691869885) / 14.538074170219572 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Int32>.GreaterThanOrEqualAnyBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 5.377292057024303.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 95.7586046691953 (T) = (0 -0.005817022620483135) / Math.Sqrt((2.0216263531920986 / (299)) + (0.0001353510086688499 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9992636163513332 = (7.899445663974243 - 0.005817022620483135) / 7.899445663974243 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Int32>.EqualsBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 4.34811976261102.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 29.618740395696683 (T) = (0 -0.20845567706837684) / Math.Sqrt((3.8131366640926756 / (299)) + (0.17381507720814163 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9711697692100962 = (7.230454677503909 - 0.20845567706837684) / 7.230454677503909 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Int32>.LessThanBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.1029394167339164 < 12.006096329011797.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 169.1784193875529 (T) = (0 -1.1576035296684402) / Math.Sqrt((0.35001438734985163 / (299)) + (0.017024472720635852 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9150093573757091 = (13.620364476895833 - 1.1576035296684402) / 13.620364476895833 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Int32>.LessThanOrEqualAllBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 5.535688639986659.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 96.70219535175077 (T) = (0 -0) / Math.Sqrt((2.0546595298430415 / (299)) + (0 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 1 = (8.016239293414015 - 0) / 8.016239293414015 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Int32>.LessThanAllBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 4.333089455511219.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 77.58502046643837 (T) = (0 -0.004068820299928053) / Math.Sqrt((2.357199199465345 / (299)) + (6.622119453242644E-05 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9994103200130008 = (6.900048144135262 - 0.004068820299928053) / 6.900048144135262 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Int32>.LessThanOrEqualBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.1121017733861396 < 13.501244089770791.
IsChangePoint: Marked as a change because one of 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 249.59850758799107 (T) = (0 -1.0234309182524637) / Math.Sqrt((0.22980897629583374 / (299)) + (0.009168637224612924 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9309995377917121 = (14.832232792341143 - 1.0234309182524637) / 14.832232792341143 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Int32>.EqualsAnyBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.0021360963893761057 < 4.170544333916974.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 48.98252650472354 (T) = (0 -0.11083481322008229) / Math.Sqrt((2.267808691713966 / (299)) + (0.044342374122214 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9837097754129944 = (6.803762135268095 - 0.11083481322008229) / 6.803762135268095 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Int32>.MaxBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.1480832756299497 < 13.12586349357325.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 282.0903973669955 (T) = (0 -1.079642533468326) / Math.Sqrt((0.28396182546099713 / (299)) + (0.005536955277998246 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9265933110570567 = (14.707686029913395 - 1.079642533468326) / 14.707686029913395 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Int32>.EqualsStaticBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.1288855579691521 < 12.239997753231586.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 162.46529392039756 (T) = (0 -0.9700720701093033) / Math.Sqrt((0.5114440360955748 / (299)) + (0.018818846628789725 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.930623645578918 = (13.982747842606704 - 0.9700720701093033) / 13.982747842606704 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

performanceautofiler · 2023-04-18T16:21:43Z

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	7a57f6dd800e77508a4ba405e623059935c82f43
Compare	887c043eb94be364188e2b23a87efa214ea57f1e
Diff	Diff
Configs	CompilationMode:tiered, LLVM:false, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Improvements in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
GreaterThanOrEqualBenchmark - Duration of single invocation	42.17 ns	1.06 ns	0.03	0.01	True
MinBenchmark - Duration of single invocation	44.48 ns	1.22 ns	0.03	0.03	True
GreaterThanOrEqualAnyBenchmark - Duration of single invocation	3.76 ns	0.00 ns	0.00	0.16	False
LessThanAllBenchmark - Duration of single invocation	3.41 ns	0.00 ns	0.00	0.19	False
GreaterThanAnyBenchmark - Duration of single invocation	3.10 ns	0.09 ns	0.03	0.24	False
SumBenchmark - Duration of single invocation	10.73 ns	0.00 ns	0.00	0.05	True
EqualsBenchmark - Duration of single invocation	5.99 ns	0.83 ns	0.14	0.24	False
GreaterThanAllBenchmark - Duration of single invocation	18.78 ns	0.01 ns	0.00	0.03	True
ConditionalSelectBenchmark - Duration of single invocation	15.75 ns	3.40 ns	0.22	0.02	False
LessThanOrEqualBenchmark - Duration of single invocation	40.53 ns	1.17 ns	0.03	0.02	True
LessThanOrEqualAnyBenchmark - Duration of single invocation	20.51 ns	0.00 ns	0.00	0.03	False
GreaterThanBenchmark - Duration of single invocation	39.52 ns	1.12 ns	0.03	0.02	True
AndNotBenchmark - Duration of single invocation	13.74 ns	1.30 ns	0.09	0.04	False
EqualsStaticBenchmark - Duration of single invocation	38.04 ns	1.05 ns	0.03	0.02	True
LessThanOrEqualAllBenchmark - Duration of single invocation	3.73 ns	0.00 ns	0.00	0.25	False
MaxBenchmark - Duration of single invocation	43.65 ns	1.29 ns	0.03	0.01	True
GreaterThanOrEqualAllBenchmark - Duration of single invocation	20.88 ns	0.00 ns	0.00	0.05	True
LessThanAnyBenchmark - Duration of single invocation	18.84 ns	0.00 ns	0.00	0.05	True
LessThanBenchmark - Duration of single invocation	38.00 ns	1.20 ns	0.03	0.02	True
EqualsAnyBenchmark - Duration of single invocation	18.69 ns	0.03 ns	0.00	0.03	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

Repro Steps

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
AOT MONO build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false
Dotnet SDK installed for dotnet commands
Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create aot directory 
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt16&gt;*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt16&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt16&gt;*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt16&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200

Payloads

Baseline
Compare

Histogram

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>.GreaterThanOrEqualBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0560488889942856 < 40.10634927090094.
IsChangePoint: Marked as a change because one of 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 693.2522555180066 (T) = (0 -1.030601473514113) / Math.Sqrt((0.9814117556680987 / (299)) + (0.0014427664194586881 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.9757260452781072 = (42.457089721132576 - 1.030601473514113) / 42.457089721132576 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>.MinBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.2154490236223745 < 42.26310782713885.
IsChangePoint: Marked as a change because one of 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 600.0946942408149 (T) = (0 -1.099691159853939) / Math.Sqrt((0.7861971949669785 / (299)) + (0.009139083460761388 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9745247920363503 = (43.16711217522058 - 1.099691159853939) / 43.16711217522058 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>.GreaterThanOrEqualAnyBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 3.5886826483515537.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 81.45969039008878 (T) = (0 -0) / Math.Sqrt((1.679592353060537 / (299)) + (0 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 1 = (6.105329469408691 - 0) / 6.105329469408691 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>.LessThanAllBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 3.080238230957833.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 72.08791025664966 (T) = (0 -0.0020107762720504917) / Math.Sqrt((1.6221113500057258 / (299)) + (2.0216106081206368E-05 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.9996215831831492 = (5.313654632962893 - 0.0020107762720504917) / 5.313654632962893 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>.GreaterThanAnyBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.08867917473925963 < 3.0122458956703264.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 67.7935409670637 (T) = (0 -0.025684031212847465) / Math.Sqrt((1.714859539700582 / (299)) + (0.0017877258116100175 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9952047185935544 = (5.356105103305129 - 0.025684031212847465) / 5.356105103305129 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>.SumBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 10.17297228586088.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 232.68351096308731 (T) = (0 -0.010133384422672862) / Math.Sqrt((0.817960797617043 / (299)) + (0.00013480952886770378 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9991731244234898 = (12.255029306151084 - 0.010133384422672862) / 12.255029306151084 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>.EqualsBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.8332229356953914 < 5.590173810850937.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 69.15557462853539 (T) = (0 -0.8077725900430492) / Math.Sqrt((4.324438435596605 / (299)) + (0.0018682915046415415 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.9124965766260242 = (9.231325574437891 - 0.8077725900430492) / 9.231325574437891 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>.GreaterThanAllBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.01156355893569357 < 17.958754136899152.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 160.35764010656513 (T) = (0 -0.08908811405535244) / Math.Sqrt((3.2537193924555834 / (299)) + (0.036122714103808816 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.9958882859015308 = (21.66690385611203 - 0.08908811405535244) / 21.66690385611203 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>.ConditionalSelectBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 3.3965298202124106 < 14.925286274247728.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 4/12/2023 8:39:35 PM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 12.267910000826305 (T) = (0 -2.2642949521281137) / Math.Sqrt((0.3633929150151421 / (299)) + (2.5639115930117313 / (2))) is greater than 1.9679296690655894 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (2) - 2, .975) and 0.8598916955874396 = (16.16103314947494 - 2.2642949521281137) / 16.16103314947494 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>.LessThanOrEqualBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.1670627337553663 < 38.35680387885664.
IsChangePoint: Marked as a change because one of 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 437.71768267646974 (T) = (0 -1.1115819625623204) / Math.Sqrt((0.7482891367965068 / (299)) + (0.023275224649806043 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9729153595363068 = (41.04104553473352 - 1.1115819625623204) / 41.04104553473352 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>.LessThanOrEqualAnyBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 19.52060836034417.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 226.13020606700428 (T) = (0 -0.0045380537002835805) / Math.Sqrt((3.260766202238166 / (299)) + (2.3105276583878108E-05 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.9998079341756069 = (23.62759597977279 - 0.0045380537002835805) / 23.62759597977279 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>.GreaterThanBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.1227305823775426 < 37.49296064382966.
IsChangePoint: Marked as a change because one of 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 535.5367654102981 (T) = (0 -1.0952156125306571) / Math.Sqrt((1.1680202001330582 / (299)) + (0.004975692949818158 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9722930114504648 = (39.52849695565448 - 1.0952156125306571) / 39.52849695565448 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>.AndNotBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.2987357738016827 < 12.986105518713371.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 4/12/2023 8:39:35 PM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 99.54781602817106 (T) = (0 -1.1745156099210061) / Math.Sqrt((0.3780477298660642 / (299)) + (0.03086129822908424 / (2))) is greater than 1.9679296690655894 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (2) - 2, .975) and 0.916327231002917 = (14.037011371787534 - 1.1745156099210061) / 14.037011371787534 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>.EqualsStaticBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.046130033326077 < 36.06360955655575.
IsChangePoint: Marked as a change because one of 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 523.2577890764622 (T) = (0 -1.0449945383600814) / Math.Sqrt((0.8804392950786644 / (299)) + (0.008238555368546103 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9725439550281298 = (38.06063617067648 - 1.0449945383600814) / 38.06063617067648 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>.LessThanOrEqualAllBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 3.7765940871765795.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 82.61109291350391 (T) = (0 -0.0013370980564030176) / Math.Sqrt((1.6032192931138949 / (299)) + (7.151324849746909E-06 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9997790491589795 = (6.051563552453031 - 0.0013370980564030176) / 6.051563552453031 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>.MaxBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.2949994890972698 < 41.498162250387765.
IsChangePoint: Marked as a change because one of 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 164.82318779718727 (T) = (0 -1.3629254403506441) / Math.Sqrt((0.5465071345668577 / (299)) + (0.2442411386928698 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.968078717207283 = (42.69644955063021 - 1.3629254403506441) / 42.69644955063021 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>.GreaterThanOrEqualAllBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 19.7684273016671.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 226.71400345304247 (T) = (0 -0.00750795124800162) / Math.Sqrt((3.271015658071202 / (299)) + (0.00022129907503079326 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.9996841190582079 = (23.768294489074208 - 0.00750795124800162) / 23.768294489074208 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>.LessThanAnyBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 17.813232278901648.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 208.19032874480308 (T) = (0 -0) / Math.Sqrt((3.214881337589039 / (299)) + (0 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 1 = (21.58775251451627 - 0) / 21.58775251451627 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>.LessThanBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.1993890882386569 < 35.98430111195692.
IsChangePoint: Marked as a change because one of 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 560.0106265063885 (T) = (0 -1.1146388601461958) / Math.Sqrt((0.9088270869765421 / (299)) + (0.006529504462360507 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.9706909857697196 = (38.030581697101574 - 1.1146388601461958) / 38.030581697101574 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>.EqualsAnyBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.02669692100260258 < 17.888779907845528.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 196.71073969688362 (T) = (0 -0.038949849712380046) / Math.Sqrt((3.304944963263653 / (299)) + (0.003770604068921695 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9981954228796102 = (21.583920837900862 - 0.038949849712380046) / 21.583920837900862 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

performanceautofiler · 2023-04-18T16:21:44Z

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	7a57f6dd800e77508a4ba405e623059935c82f43
Compare	887c043eb94be364188e2b23a87efa214ea57f1e
Diff	Diff
Configs	CompilationMode:tiered, LLVM:false, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Improvements in System.Numerics.Tests.Perf_VectorOf<SByte>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
GreaterThanBenchmark - Duration of single invocation	42.19 ns	1.16 ns	0.03	0.03	True
GreaterThanOrEqualBenchmark - Duration of single invocation	46.57 ns	1.16 ns	0.02	0.01	True
LessThanOrEqualAllBenchmark - Duration of single invocation	31.51 ns	0.01 ns	0.00	0.03	True
EqualsStaticBenchmark - Duration of single invocation	41.49 ns	1.14 ns	0.03	0.03	True
LessThanOrEqualBenchmark - Duration of single invocation	48.19 ns	1.04 ns	0.02	0.01	True
AbsBenchmark - Duration of single invocation	33.76 ns	1.21 ns	0.04	0.02	False
GreaterThanAnyBenchmark - Duration of single invocation	28.61 ns	0.00 ns	0.00	0.02	True
EqualsBenchmark - Duration of single invocation	4.22 ns	0.00 ns	0.00	0.16	False
MaxBenchmark - Duration of single invocation	47.69 ns	1.02 ns	0.02	0.02	True
MinBenchmark - Duration of single invocation	47.40 ns	1.10 ns	0.02	0.01	True
LessThanBenchmark - Duration of single invocation	43.71 ns	0.97 ns	0.02	0.01	True
EqualsAnyBenchmark - Duration of single invocation	29.35 ns	0.00 ns	0.00	0.01	True
LessThanAllBenchmark - Duration of single invocation	28.22 ns	0.00 ns	0.00	0.03	True
GreaterThanOrEqualAnyBenchmark - Duration of single invocation	31.49 ns	0.00 ns	0.00	0.02	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

Repro Steps

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
AOT MONO build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false
Dotnet SDK installed for dotnet commands
Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create aot directory 
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_VectorOf<SByte>.GreaterThanBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.16191906698198 < 39.74874063746855.
IsChangePoint: Marked as a change because one of 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 507.2564480739783 (T) = (0 -1.0776627175077718) / Math.Sqrt((1.419749841174971 / (299)) + (0.008583075102761975 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9750515148952226 = (43.19551720202076 - 1.0776627175077718) / 43.19551720202076 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<SByte>.GreaterThanOrEqualBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.1564107900715441 < 43.53366701947293.
IsChangePoint: Marked as a change because one of 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 435.685812757645 (T) = (0 -1.1634607297397008) / Math.Sqrt((0.8581064016936393 / (299)) + (0.032860951413175125 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9752639961103606 = (47.0351126613066 - 1.1634607297397008) / 47.0351126613066 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<SByte>.LessThanOrEqualAllBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.008826799561746245 < 30.215494230129266.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 348.0263331549614 (T) = (0 -0.0030602830477838394) / Math.Sqrt((2.8442474881183464 / (299)) + (1.736957615904743E-05 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9999098712919224 = (33.95458686866479 - 0.0030602830477838394) / 33.95458686866479 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<SByte>.EqualsStaticBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.1394540607067531 < 39.45525156819947.
IsChangePoint: Marked as a change because one of 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 592.7728611184723 (T) = (0 -1.1200026701708639) / Math.Sqrt((1.4099812856580252 / (299)) + (0.001193107308148799 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9740100812561894 = (43.093734967431715 - 1.1200026701708639) / 43.093734967431715 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<SByte>.LessThanOrEqualBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0364647192181349 < 45.472935635438674.
IsChangePoint: Marked as a change because one of 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 467.2732846069435 (T) = (0 -1.064506706294221) / Math.Sqrt((2.6716456070719254 / (299)) + (0.010044674715638928 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.9786877403942876 = (49.94809213045173 - 1.064506706294221) / 49.94809213045173 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<SByte>.AbsBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.2105291152804711 < 31.677373558792.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 466.9475013340218 (T) = (0 -1.1322965972278731) / Math.Sqrt((0.6942700720379675 / (299)) + (0.008421978101616595 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.9672506892398978 = (34.57466954105557 - 1.1322965972278731) / 34.57466954105557 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<SByte>.GreaterThanAnyBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.0006194182112546465 < 27.159038924850584.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 407.5887673356839 (T) = (0 -0.009208584061664868) / Math.Sqrt((1.6297368626616189 / (299)) + (0.0003242372951828436 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9996963242918776 = (30.32374278009059 - 0.009208584061664868) / 30.32374278009059 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<SByte>.EqualsBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 4.069383141410053.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 66.84220277890458 (T) = (0 -0.004931573597987119) / Math.Sqrt((3.5186992620889046 / (299)) + (9.728167260945447E-05 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9993210527648797 = (7.2635594386256 - 0.004931573597987119) / 7.2635594386256 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<SByte>.MaxBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.016667586449273 < 45.22734628728847.
IsChangePoint: Marked as a change because one of 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 667.7839576019742 (T) = (0 -1.1349458274249842) / Math.Sqrt((0.16751426984731388 / (299)) + (0.022002512097518186 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.976438107773061 = (48.168704639407835 - 1.1349458274249842) / 48.168704639407835 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<SByte>.MinBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.100321000081951 < 44.52133063055503.
IsChangePoint: Marked as a change because one of 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 388.4180223236356 (T) = (0 -0.9663310781731358) / Math.Sqrt((0.14824981614165514 / (299)) + (0.06877060761495082 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.9795844108593547 = (47.33299987161616 - 0.9663310781731358) / 47.33299987161616 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<SByte>.LessThanBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.9664934968708349 < 41.55115795682418.
IsChangePoint: Marked as a change because one of 3/10/2023 12:53:36 AM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 396.3784935913938 (T) = (0 -0.9948439058366182) / Math.Sqrt((2.903853720757718 / (299)) + (0.012689905318118562 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9783670754082888 = (45.98749011577469 - 0.9948439058366182) / 45.98749011577469 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<SByte>.EqualsAnyBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 27.098131225790027.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 405.72885471516975 (T) = (0 -0.007865795834604731) / Math.Sqrt((1.6432339399000548 / (299)) + (0.00024748297644674056 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9997400147057003 = (30.25477212390415 - 0.007865795834604731) / 30.25477212390415 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<SByte>.LessThanAllBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 27.235845876145305.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 403.4847985077745 (T) = (0 -0.0008715863417228894) / Math.Sqrt((1.6933940144726132 / (299)) + (3.038651004311557E-06 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9999712989203842 = (30.367719730025836 - 0.0008715863417228894) / 30.367719730025836 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<SByte>.GreaterThanOrEqualAnyBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 30.13165049699328.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 432.7363315140791 (T) = (0 -0.003549808075962471) / Math.Sqrt((1.8036448343200169 / (299)) + (5.040454950467352E-05 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9998945023856128 = (33.64823078302464 - 0.003549808075962471) / 33.64823078302464 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

performanceautofiler · 2023-04-18T16:21:45Z

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	7a57f6dd800e77508a4ba405e623059935c82f43
Compare	887c043eb94be364188e2b23a87efa214ea57f1e
Diff	Diff
Configs	CompilationMode:tiered, LLVM:false, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Improvements in System.Numerics.Tests.Perf_VectorOf<Byte>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
MinBenchmark - Duration of single invocation	46.78 ns	1.02 ns	0.02	0.01	True
EqualsBenchmark - Duration of single invocation	4.27 ns	0.00 ns	0.00	0.16	False
LessThanOrEqualAllBenchmark - Duration of single invocation	31.93 ns	0.00 ns	0.00	0.04	True
GreaterThanOrEqualAnyBenchmark - Duration of single invocation	31.24 ns	0.00 ns	0.00	0.04	True
EqualsStaticBenchmark - Duration of single invocation	41.17 ns	1.23 ns	0.03	0.02	True
GreaterThanBenchmark - Duration of single invocation	41.32 ns	1.15 ns	0.03	0.02	True
MaxBenchmark - Duration of single invocation	47.60 ns	1.00 ns	0.02	0.01	True
LessThanBenchmark - Duration of single invocation	43.77 ns	1.16 ns	0.03	0.06	True
LessThanAllBenchmark - Duration of single invocation	28.42 ns	0.00 ns	0.00	0.03	True
LessThanOrEqualBenchmark - Duration of single invocation	47.78 ns	1.10 ns	0.02	0.01	True
EqualsAnyBenchmark - Duration of single invocation	27.95 ns	0.00 ns	0.00	0.01	True
GreaterThanAnyBenchmark - Duration of single invocation	28.86 ns	0.00 ns	0.00	0.02	True
GreaterThanOrEqualBenchmark - Duration of single invocation	45.74 ns	1.04 ns	0.02	0.02	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

Repro Steps

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
AOT MONO build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false
Dotnet SDK installed for dotnet commands
Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create aot directory 
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_VectorOf<Byte>.MinBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0151591695427975 < 44.44313732303929.
IsChangePoint: Marked as a change because one of 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 734.3607698862082 (T) = (0 -1.1249518505997012) / Math.Sqrt((0.12862485183497466 / (299)) + (0.014123390053659346 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9762383704604177 = (47.343211404156825 - 1.1249518505997012) / 47.343211404156825 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Byte>.EqualsBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 4.063583918924831.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 67.52072309173145 (T) = (0 -0) / Math.Sqrt((3.429200558200219 / (299)) + (0 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 1 = (7.2309928698012875 - 0) / 7.2309928698012875 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Byte>.LessThanOrEqualAllBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 30.08062540920818.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 397.04237127572696 (T) = (0 -0.003377517758375253) / Math.Sqrt((2.1682569223189696 / (299)) + (5.297530727863085E-05 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.9999001884894614 = (33.83896045804898 - 0.003377517758375253) / 33.83896045804898 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Byte>.GreaterThanOrEqualAnyBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 29.865658573398708.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 392.0250816560123 (T) = (0 -0.0004982346451176355) / Math.Sqrt((2.204177132404643 / (299)) + (9.929510463819847E-07 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9999851980587293 = (33.66008795767726 - 0.0004982346451176355) / 33.66008795767726 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Byte>.EqualsStaticBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.2349382063186793 < 39.62440318397986.
IsChangePoint: Marked as a change because one of 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 482.95214357375045 (T) = (0 -1.1000666102837586) / Math.Sqrt((1.3910830063188562 / (299)) + (0.011564485983523458 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9744445537933477 = (43.046268939628355 - 1.1000666102837586) / 43.046268939628355 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Byte>.GreaterThanBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.14965754495814 < 39.32942869461154.
IsChangePoint: Marked as a change because one of 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 495.3671353032115 (T) = (0 -1.0811694578283153) / Math.Sqrt((1.402360556437453 / (299)) + (0.010045886419898715 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9749260627724471 = (43.11925359054721 - 1.0811694578283153) / 43.11925359054721 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Byte>.MaxBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0045578561683564 < 45.40763380047897.
IsChangePoint: Marked as a change because one of 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 659.5103217898725 (T) = (0 -1.1273148309852026) / Math.Sqrt((0.16499837554035718 / (299)) + (0.018123570662841825 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9765854669199461 = (48.14594538917032 - 1.1273148309852026) / 48.14594538917032 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Byte>.LessThanBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.1639067323470451 < 41.53616910828074.
IsChangePoint: Marked as a change because one of 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 477.81441632206554 (T) = (0 -1.0882671814786709) / Math.Sqrt((2.093676144750017 / (299)) + (0.007583150778282421 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.9759177014901221 = (45.189506351825024 - 1.0882671814786709) / 45.189506351825024 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Byte>.LessThanAllBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 26.909273093667.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 403.4004825496736 (T) = (0 -0.0004491335823809088) / Math.Sqrt((1.6973269718976691 / (299)) + (1.0086048741115429E-06 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.9999852232876266 = (30.39468936203115 - 0.0004491335823809088) / 30.39468936203115 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Byte>.LessThanOrEqualBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.1019797380047323 < 45.37988469997421.
IsChangePoint: Marked as a change because one of 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 593.7522994395911 (T) = (0 -1.1768198796661804) / Math.Sqrt((0.9375606754652968 / (299)) + (0.016736291220218677 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.9759752752112301 = (48.98369866930897 - 1.1768198796661804) / 48.98369866930897 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Byte>.EqualsAnyBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 26.602987210447626.
IsChangePoint: Marked as a change because one of 3/21/2023 12:49:42 AM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 402.4233537437328 (T) = (0 -0.001366959729695039) / Math.Sqrt((1.688676447018786 / (299)) + (7.474315610431738E-06 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9999548098838886 = (30.249086466750583 - 0.001366959729695039) / 30.249086466750583 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Byte>.GreaterThanAnyBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 26.6071677011858.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 411.3699580736695 (T) = (0 -0.00010070326788769231) / Math.Sqrt((1.6200100159515778 / (299)) + (4.056459265304128E-08 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9999966742784465 = (30.280126061570492 - 0.00010070326788769231) / 30.280126061570492 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Byte>.GreaterThanOrEqualBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.044612753019817 < 43.549239259061096.
IsChangePoint: Marked as a change because one of 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 345.5890930150691 (T) = (0 -1.1872599002023614) / Math.Sqrt((0.9358260408444171 / (299)) + (0.05797034024841522 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9747735288334047 = (47.06404999580458 - 1.1872599002023614) / 47.06404999580458 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

performanceautofiler · 2023-04-18T16:21:47Z

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	7a57f6dd800e77508a4ba405e623059935c82f43
Compare	887c043eb94be364188e2b23a87efa214ea57f1e
Diff	Diff
Configs	CompilationMode:tiered, LLVM:false, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Improvements in System.Runtime.Intrinsics.Tests.Perf_Vector128Int

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
GreaterThanAllBenchmark - Duration of single invocation	7.03 ns	0.00 ns	0.00	0.10	False
LessThanBenchmark - Duration of single invocation	16.62 ns	1.15 ns	0.07	0.08	False
ConditionalSelectBenchmark - Duration of single invocation	16.55 ns	1.22 ns	0.07	0.11	False
GreaterThanOrEqualAllBenchmark - Duration of single invocation	9.68 ns	0.00 ns	0.00	0.11	False
LessThanAnyBenchmark - Duration of single invocation	6.95 ns	0.00 ns	0.00	0.11	False
GreaterThanBenchmark - Duration of single invocation	16.82 ns	1.03 ns	0.06	0.05	False
LessThanAllBenchmark - Duration of single invocation	2.92 ns	0.00 ns	0.00	0.41	False
EqualsAnyBenchmark - Duration of single invocation	7.18 ns	0.00 ns	0.00	0.13	False
LessThanOrEqualAnyBenchmark - Duration of single invocation	8.13 ns	0.00 ns	0.00	0.11	False
GreaterThanOrEqualAnyBenchmark - Duration of single invocation	3.33 ns	0.00 ns	0.00	0.19	False
LessThanOrEqualAllBenchmark - Duration of single invocation	3.36 ns	0.05 ns	0.01	0.22	False
MaxBenchmark - Duration of single invocation	18.08 ns	1.14 ns	0.06	0.07	True
LessThanOrEqualBenchmark - Duration of single invocation	17.68 ns	0.73 ns	0.04	0.03	True
MinBenchmark - Duration of single invocation	18.04 ns	0.94 ns	0.05	0.06	True
AndNotBenchmark - Duration of single invocation	13.45 ns	1.10 ns	0.08	0.12	False
EqualsBenchmark - Duration of single invocation	6.06 ns	0.69 ns	0.11	0.18	False
AbsBenchmark - Duration of single invocation	15.35 ns	1.25 ns	0.08	0.07	False
EqualsStaticBenchmark - Duration of single invocation	16.60 ns	1.16 ns	0.07	0.06	True
GreaterThanOrEqualBenchmark - Duration of single invocation	17.87 ns	1.24 ns	0.07	0.06	False
GreaterThanAnyBenchmark - Duration of single invocation	2.96 ns	0.00 ns	0.00	0.38	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

Repro Steps

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
AOT MONO build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false
Dotnet SDK installed for dotnet commands
Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create aot directory 
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Int*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Runtime.Intrinsics.Tests.Perf_Vector128Int* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Int*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Runtime.Intrinsics.Tests.Perf_Vector128Int* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200

Payloads

Baseline
Compare

Histogram

System.Runtime.Intrinsics.Tests.Perf_Vector128Int.GreaterThanAllBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 6.665936322801749.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 97.9424325508709 (T) = (0 -0.0008047316946009693) / Math.Sqrt((3.016353255484294 / (299)) + (3.2379655014767392E-06 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.999918205283767 = (9.838431278467302 - 0.0008047316946009693) / 9.838431278467302 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Int.LessThanBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.1521437407355288 < 15.769321897051531.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 147.539148904302 (T) = (0 -1.123442140305099) / Math.Sqrt((1.6983591882359248 / (299)) + (0.03356204275757712 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9396793299235611 = (18.62449702368128 - 1.123442140305099) / 18.62449702368128 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Int.ConditionalSelectBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.2241166861566952 < 14.924668542476075.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 4/12/2023 8:39:35 PM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 137.23000438223028 (T) = (0 -1.1216253948482433) / Math.Sqrt((0.4397326952401317 / (299)) + (0.021008929588147856 / (2))) is greater than 1.9679296690655894 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (2) - 2, .975) and 0.9305014919547574 = (16.138841342005207 - 1.1216253948482433) / 16.138841342005207 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Int.GreaterThanOrEqualAllBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.001824122968405497 < 7.845847371664349.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 114.94048707971214 (T) = (0 -0.0035775635948997165) / Math.Sqrt((2.720640891813578 / (299)) + (2.120945677105084E-06 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9996738180105996 = (10.967998574895777 - 0.0035775635948997165) / 10.967998574895777 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Int.LessThanAnyBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 6.6487136381490375.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 84.16675397447601 (T) = (0 -0.08815078412543234) / Math.Sqrt((2.8610316528578497 / (299)) + (0.013640322638820597 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9908904933604116 = (9.676790150450424 - 0.08815078412543234) / 9.676790150450424 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Int.GreaterThanBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.034155875781319 < 16.006603852638705.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 182.9833281989517 (T) = (0 -1.1005046080563377) / Math.Sqrt((2.0243049905724493 / (299)) + (0.013651025924249376 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.9418827620147385 = (18.935941318054798 - 1.1005046080563377) / 18.935941318054798 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Int.LessThanAllBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 2.8778281345979666.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 63.70476177553769 (T) = (0 -0.043525241013227925) / Math.Sqrt((1.758754074765663 / (299)) + (0.002526240435031618 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9916054971771807 = (5.184969489188876 - 0.043525241013227925) / 5.184969489188876 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Int.EqualsAnyBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 6.565610637988081.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 97.43836768565536 (T) = (0 -0.010287249429424059) / Math.Sqrt((2.9293633546607496 / (299)) + (0.0002059638862166968 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.9989367261546684 = (9.675070514141012 - 0.010287249429424059) / 9.675070514141012 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Int.LessThanOrEqualAnyBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 7.732018591179261.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 108.62194123068483 (T) = (0 -0.028196113426378837) / Math.Sqrt((2.775908720507672 / (299)) + (0.0021674817237156267 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9973881339064578 = (10.795390122063553 - 0.028196113426378837) / 10.795390122063553 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Int.GreaterThanOrEqualAnyBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 3.237142623804856.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 62.45387392795546 (T) = (0 -0.04824927443145175) / Math.Sqrt((1.737445757999178 / (299)) + (0.006983977449484632 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.9915092372865867 = (5.682560690952998 - 0.04824927443145175) / 5.682560690952998 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Int.LessThanOrEqualAllBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.04529751243254255 < 3.2574002320757613.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 63.02256041369 (T) = (0 -0.09463484847083414) / Math.Sqrt((1.8055385406595654 / (299)) + (0.007496797359966092 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9833995357823149 = (5.7007350655902576 - 0.09463484847083414) / 5.7007350655902576 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Int.MaxBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.1370585033569722 < 17.1597169907922.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 251.13085146739846 (T) = (0 -1.202464459875967) / Math.Sqrt((1.2344761889453435 / (299)) + (0.005462517857504075 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9393221649531097 = (19.81719451504377 - 1.202464459875967) / 19.81719451504377 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Int.LessThanOrEqualBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.7254771678207164 < 16.697728877450537.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 143.69514840402366 (T) = (0 -1.0133254603740407) / Math.Sqrt((1.4529513914672094 / (299)) + (0.046503075533839366 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9479357681262394 = (19.462986851146425 - 1.0133254603740407) / 19.462986851146425 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Int.MinBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.9411780133109184 < 17.189893080100845.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 186.23121294810812 (T) = (0 -1.0592811535009887) / Math.Sqrt((1.3396711050912788 / (299)) + (0.023056434907980938 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9467933204226385 = (19.908800209207108 - 1.0592811535009887) / 19.908800209207108 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Int.AndNotBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.102841968990184 < 12.785644173514125.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 4/12/2023 8:39:35 PM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 337.9128265218397 (T) = (0 -1.1080402933818192) / Math.Sqrt((0.43011676880664695 / (299)) + (5.4045152961337805E-05 / (2))) is greater than 1.9679296690655894 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (2) - 2, .975) and 0.9211030475266151 = (14.044145669068854 - 1.1080402933818192) / 14.044145669068854 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Int.EqualsBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.6899718332688494 < 5.720462884963817.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 62.58715658948636 (T) = (0 -0.7856112638568383) / Math.Sqrt((4.8757453788568155 / (299)) + (0.006393374172688911 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9142384602105389 = (9.160414630911033 - 0.7856112638568383) / 9.160414630911033 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Int.AbsBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.2459610138610309 < 14.726799836370159.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 85.17175156640809 (T) = (0 -1.30552399712943) / Math.Sqrt((0.2865008841599765 / (299)) + (0.09638433624685183 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9117112028948067 = (14.786972299259439 - 1.30552399712943) / 14.786972299259439 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Int.EqualsStaticBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.163687573070258 < 15.793039619237646.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 206.30863365269948 (T) = (0 -1.077018280685706) / Math.Sqrt((1.712670947974173 / (299)) + (0.007486934069484306 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.9421384636054904 = (18.613717294722623 - 1.077018280685706) / 18.613717294722623 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Int.GreaterThanOrEqualBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.242235908599654 < 16.98512045154699.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 233.31965747491782 (T) = (0 -1.215032969409727) / Math.Sqrt((1.8503558851112298 / (299)) + (0.0008413002923918335 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9388781510598181 = (19.878864767308368 - 1.215032969409727) / 19.878864767308368 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Int.GreaterThanAnyBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 2.8090051903626434.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 68.34892638365938 (T) = (0 -0.019982922703599908) / Math.Sqrt((1.6585635130305134 / (299)) + (0.0007378678834627892 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.996140633506546 = (5.177772760760897 - 0.019982922703599908) / 5.177772760760897 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

performanceautofiler · 2023-04-18T16:21:48Z

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	7a57f6dd800e77508a4ba405e623059935c82f43
Compare	887c043eb94be364188e2b23a87efa214ea57f1e
Diff	Diff
Configs	CompilationMode:tiered, LLVM:false, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Improvements in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
MaxBenchmark - Duration of single invocation	18.41 ns	1.01 ns	0.06	0.04	True
AndNotBenchmark - Duration of single invocation	13.67 ns	1.06 ns	0.08	0.04	False
MinBenchmark - Duration of single invocation	18.13 ns	1.17 ns	0.06	0.05	True
GreaterThanOrEqualAllBenchmark - Duration of single invocation	8.21 ns	0.02 ns	0.00	0.06	False
LessThanOrEqualBenchmark - Duration of single invocation	17.61 ns	1.26 ns	0.07	0.03	False
LessThanAllBenchmark - Duration of single invocation	2.89 ns	0.00 ns	0.00	0.41	False
LessThanBenchmark - Duration of single invocation	16.72 ns	1.14 ns	0.07	0.05	False
EqualsAnyBenchmark - Duration of single invocation	6.89 ns	0.01 ns	0.00	0.13	False
GreaterThanBenchmark - Duration of single invocation	16.94 ns	1.15 ns	0.07	0.03	False
GreaterThanOrEqualBenchmark - Duration of single invocation	18.01 ns	1.03 ns	0.06	0.04	False
GreaterThanAllBenchmark - Duration of single invocation	7.02 ns	0.00 ns	0.00	0.10	False
LessThanOrEqualAnyBenchmark - Duration of single invocation	7.97 ns	0.01 ns	0.00	0.09	False
LessThanAnyBenchmark - Duration of single invocation	6.81 ns	0.00 ns	0.00	0.13	False
LessThanOrEqualAllBenchmark - Duration of single invocation	3.59 ns	0.02 ns	0.01	0.19	False
EqualsStaticBenchmark - Duration of single invocation	16.80 ns	1.13 ns	0.07	0.03	False
EqualsBenchmark - Duration of single invocation	6.04 ns	1.42 ns	0.23	0.17	False
GreaterThanAnyBenchmark - Duration of single invocation	3.07 ns	0.00 ns	0.00	0.40	False
ConditionalSelectBenchmark - Duration of single invocation	15.74 ns	1.12 ns	0.07	0.05	False
GreaterThanOrEqualAnyBenchmark - Duration of single invocation	3.33 ns	0.02 ns	0.01	0.25	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

Repro Steps

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
AOT MONO build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false
Dotnet SDK installed for dotnet commands
Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create aot directory 
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt32&gt;*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt32&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt32&gt;*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt32&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200

Payloads

Baseline
Compare

Histogram

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt32>.MaxBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0129578655562546 < 17.31831611452061.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 216.85836434245684 (T) = (0 -1.11904365151117) / Math.Sqrt((1.1859890506816169 / (299)) + (0.013921199463838949 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9435761860061954 = (19.83282540301231 - 1.11904365151117) / 19.83282540301231 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt32>.AndNotBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0577426797308283 < 12.890801259070269.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 4/12/2023 8:39:35 PM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 348.83405687743505 (T) = (0 -1.0472659905484136) / Math.Sqrt((0.38180085299099736 / (299)) + (0.00021952203244984713 / (2))) is greater than 1.9679296690655894 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (2) - 2, .975) and 0.9253937088753025 = (14.0372343238187 - 1.0472659905484136) / 14.0372343238187 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt32>.MinBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.1704268037442818 < 17.309302617130545.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 222.86537260783487 (T) = (0 -1.105864679655841) / Math.Sqrt((1.329309117113897 / (299)) + (0.013446553237040314 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.9445161118411164 = (19.93127584153959 - 1.105864679655841) / 19.93127584153959 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt32>.GreaterThanOrEqualAllBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.02375535426481136 < 7.778986020064934.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 105.29320000543503 (T) = (0 -0.044303072528022666) / Math.Sqrt((2.790808918569751 / (299)) + (0.005420766358063247 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9959467714069835 = (10.930316786068703 - 0.044303072528022666) / 10.930316786068703 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt32>.LessThanOrEqualBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.258533080907396 < 16.795962813207343.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 177.0592872941015 (T) = (0 -1.1007957097607235) / Math.Sqrt((1.417823345250553 / (299)) + (0.023915114759798944 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9433561352739228 = (19.433626485128425 - 1.1007957097607235) / 19.433626485128425 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt32>.LessThanAllBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 2.750902929785464.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 57.769966016485874 (T) = (0 -0.046278059821176616) / Math.Sqrt((1.6486674871088918 / (299)) + (0.008566635283249607 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9909275329940799 = (5.1009344857334105 - 0.046278059821176616) / 5.1009344857334105 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt32>.LessThanBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.1436231336054477 < 15.88902547522244.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 126.6583922877781 (T) = (0 -1.2575107539306756) / Math.Sqrt((1.610130275097664 / (299)) + (0.0538134398097893 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9325433699728748 = (18.64176661990101 - 1.2575107539306756) / 18.64176661990101 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt32>.EqualsAnyBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.012487104731861516 < 6.5463930630535.
IsChangePoint: Marked as a change because one of 12/30/2022 6:06:02 AM, 3/6/2023 4:10:52 PM, 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 98.70860146166484 (T) = (0 -0.003121776182965379) / Math.Sqrt((2.8337047447623935 / (299)) + (3.898194614611957E-05 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9996754056998604 = (9.617470736921717 - 0.003121776182965379) / 9.617470736921717 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt32>.GreaterThanBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.1505650866348698 < 16.11835521556654.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 165.04001559573334 (T) = (0 -1.0333004947016375) / Math.Sqrt((1.991007908688172 / (299)) + (0.020560997751172153 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9455026983293565 = (18.96058085492799 - 1.0333004947016375) / 18.96058085492799 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt32>.GreaterThanOrEqualBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.029416886162171 < 17.119299147449514.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 219.09048053593324 (T) = (0 -1.0504535990314228) / Math.Sqrt((1.8118656145082541 / (299)) + (0.005225253285508792 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9470912582323374 = (19.85406501716224 - 1.0504535990314228) / 19.85406501716224 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt32>.GreaterThanAllBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 6.6805275214751365.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 98.76016156570962 (T) = (0 -0.010332293825815717) / Math.Sqrt((2.9148858249938874 / (299)) + (0.0004270251828119584 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9989472691626282 = (9.814753647390887 - 0.010332293825815717) / 9.814753647390887 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt32>.LessThanOrEqualAnyBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.007224733052397895 < 7.70474209293683.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 111.82124879041336 (T) = (0 -0.003106117216169267) / Math.Sqrt((2.7584662334970442 / (299)) + (1.3547387851192228E-05 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9997109386663129 = (10.745529941861758 - 0.003106117216169267) / 10.745529941861758 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt32>.LessThanAnyBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 6.499920246054059.
IsChangePoint: Marked as a change because one of 12/30/2022 6:06:02 AM, 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 99.3917215361748 (T) = (0 -0) / Math.Sqrt((2.814440029214199 / (299)) + (0 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 1 = (9.64296561868561 - 0) / 9.64296561868561 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt32>.LessThanOrEqualAllBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.020745682750309517 < 3.3564637036252902.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 75.37172093229505 (T) = (0 -0.005186420687577379) / Math.Sqrt((1.677423122658285 / (299)) + (0.00010759583819412248 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9990843340472727 = (5.6640969036031885 - 0.005186420687577379) / 5.6640969036031885 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt32>.EqualsStaticBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.1299906478561674 < 15.885903282544728.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 175.9911750415711 (T) = (0 -1.13912750620434) / Math.Sqrt((1.660093446592541 / (299)) + (0.017371319096458074 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9389063298761978 = (18.64558969686998 - 1.13912750620434) / 18.64558969686998 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt32>.EqualsBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.4171753951467372 < 5.694615351744458.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 43.139115835102 (T) = (0 -0.9948022668668739) / Math.Sqrt((4.40379923571299 / (299)) + (0.07972419994703141 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.889785246495783 = (9.026035401229757 - 0.9948022668668739) / 9.026035401229757 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt32>.GreaterThanAnyBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 2.804739182927838.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 69.10754446204582 (T) = (0 -0.0005240217564562267) / Math.Sqrt((1.6571390079693138 / (299)) + (1.098395204957876E-06 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9998981584523398 = (5.145461439812391 - 0.0005240217564562267) / 5.145461439812391 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt32>.ConditionalSelectBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.1233390784646171 < 14.959111751164308.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 4/12/2023 8:39:35 PM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 433.4710904961389 (T) = (0 -1.1251895552375473) / Math.Sqrt((0.3597532224500586 / (299)) + (6.8485285743071725E-06 / (2))) is greater than 1.9679296690655894 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (2) - 2, .975) and 0.9304682383397008 = (16.18238238712714 - 1.1251895552375473) / 16.18238238712714 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt32>.GreaterThanOrEqualAnyBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.02091108275530297 < 3.2654741454680227.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 74.38549682435192 (T) = (0 -0.018948048893107153) / Math.Sqrt((1.6802562745405674 / (299)) + (0.0006498575822527712 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9966612579912595 = (5.675206063691895 - 0.018948048893107153) / 5.675206063691895 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

performanceautofiler · 2023-04-18T16:21:50Z

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	7a57f6dd800e77508a4ba405e623059935c82f43
Compare	887c043eb94be364188e2b23a87efa214ea57f1e
Diff	Diff
Configs	CompilationMode:tiered, LLVM:false, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Improvements in System.Numerics.Tests.Perf_VectorOf<UInt32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
MaxBenchmark - Duration of single invocation	13.81 ns	1.12 ns	0.08	0.05	True
GreaterThanAnyBenchmark - Duration of single invocation	4.28 ns	0.00 ns	0.00	0.16	False
LessThanOrEqualAllBenchmark - Duration of single invocation	5.65 ns	0.00 ns	0.00	0.13	False
LessThanAllBenchmark - Duration of single invocation	3.94 ns	0.17 ns	0.04	0.19	False
GreaterThanOrEqualAnyBenchmark - Duration of single invocation	5.28 ns	0.00 ns	0.00	0.10	False
EqualsStaticBenchmark - Duration of single invocation	13.57 ns	1.14 ns	0.08	0.06	True
MinBenchmark - Duration of single invocation	13.65 ns	1.05 ns	0.08	0.04	True
GreaterThanBenchmark - Duration of single invocation	12.78 ns	1.01 ns	0.08	0.06	True
EqualsAnyBenchmark - Duration of single invocation	4.38 ns	0.00 ns	0.00	0.14	False
EqualsBenchmark - Duration of single invocation	4.51 ns	0.00 ns	0.00	0.15	False
LessThanOrEqualBenchmark - Duration of single invocation	14.11 ns	1.46 ns	0.10	0.04	True
LessThanBenchmark - Duration of single invocation	12.54 ns	1.04 ns	0.08	0.05	True
GreaterThanOrEqualBenchmark - Duration of single invocation	14.35 ns	1.26 ns	0.09	0.05	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

Repro Steps

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
AOT MONO build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false
Dotnet SDK installed for dotnet commands
Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create aot directory 
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_VectorOf<UInt32>.MaxBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.1232677921774377 < 13.120561825497663.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 281.5055937119781 (T) = (0 -1.069100984400542) / Math.Sqrt((0.3046797825141886 / (299)) + (0.005273894068952398 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9271687887587486 = (14.679159747311836 - 1.069100984400542) / 14.679159747311836 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<UInt32>.GreaterThanAnyBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 4.117395353677227.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 76.59102929508065 (T) = (0 -0.010481249895235785) / Math.Sqrt((2.318585895035574 / (299)) + (0.0004394263974655205 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9984592406978079 = (6.802652354799134 - 0.010481249895235785) / 6.802652354799134 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<UInt32>.LessThanOrEqualAllBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 5.287387850882226.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 26.3734708473493 (T) = (0 -0.31439318691015444) / Math.Sqrt((2.0968312776595646 / (299)) + (0.31252442317623425 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9607500216486344 = (8.010021918883844 - 0.31439318691015444) / 8.010021918883844 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<UInt32>.LessThanAllBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.17421283767639129 < 4.0381991755724025.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 69.27374084776558 (T) = (0 -0.04370941451108832) / Math.Sqrt((2.341369557544305 / (299)) + (0.007569483848048109 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9936417852093995 = (6.874479071657758 - 0.04370941451108832) / 6.874479071657758 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<UInt32>.GreaterThanOrEqualAnyBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 5.095302343299436.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 95.02433998448932 (T) = (0 -0.008372468578512834) / Math.Sqrt((2.054979177404595 / (299)) + (0.0002803929203927389 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9989436981911397 = (7.926208691762288 - 0.008372468578512834) / 7.926208691762288 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<UInt32>.EqualsStaticBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.1414595392686706 < 12.281751253645929.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 180.0619772867456 (T) = (0 -1.0769974765530983) / Math.Sqrt((0.5446122387858173 / (299)) + (0.013266203733051055 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.922982530070287 = (13.983807537899864 - 1.0769974765530983) / 13.983807537899864 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<UInt32>.MinBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0485854104122794 < 12.983186857699335.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 202.8226196994472 (T) = (0 -1.102678707352987) / Math.Sqrt((0.2998631739211494 / (299)) + (0.01353023800925638 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9241318269891771 = (14.534140781216456 - 1.102678707352987) / 14.534140781216456 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<UInt32>.GreaterThanBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.013821291714212 < 12.17678176844706.
IsChangePoint: Marked as a change because one of 3/7/2023 1:52:33 AM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 170.42082284874834 (T) = (0 -1.0454429710441993) / Math.Sqrt((0.5618765004013855 / (299)) + (0.015523140850734416 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9252156828248108 = (13.979441285733103 - 1.0454429710441993) / 13.979441285733103 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<UInt32>.EqualsAnyBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 4.086739771832967.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 77.6361582719024 (T) = (0 -0.009746754059909131) / Math.Sqrt((2.2618797093645937 / (299)) + (0.0003799968588174206 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9985675999709871 = (6.804491666078936 - 0.009746754059909131) / 6.804491666078936 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<UInt32>.EqualsBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 4.408096000454717.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 63.43505722717493 (T) = (0 -0) / Math.Sqrt((3.942061774166192 / (299)) + (0 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 1 = (7.283757103467383 - 0) / 7.283757103467383 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<UInt32>.LessThanOrEqualBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.4588714879190559 < 13.459948833400523.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 115.76844912382714 (T) = (0 -1.176223823497345) / Math.Sqrt((0.24843279641515328 / (299)) + (0.05218949211054167 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9206028880455983 = (14.814440910304874 - 1.176223823497345) / 14.814440910304874 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<UInt32>.LessThanBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0363773746320346 < 11.989834919153493.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 254.25603429926244 (T) = (0 -1.07770591427693) / Math.Sqrt((0.3713189501536693 / (299)) + (0.0047364692529291786 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9207622232976042 = (13.600910564724924 - 1.07770591427693) / 13.600910564724924 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<UInt32>.GreaterThanOrEqualBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.2597631895835237 < 13.707872602287472.
IsChangePoint: Marked as a change because one of 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 232.93706444543136 (T) = (0 -1.1280696025439902) / Math.Sqrt((0.23780364099696574 / (299)) + (0.011074408011254777 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9249659568362156 = (15.034103921091376 - 1.1280696025439902) / 15.034103921091376 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	7a57f6dd800e77508a4ba405e623059935c82f43
Compare	887c043eb94be364188e2b23a87efa214ea57f1e
Diff	Diff
Configs	CompilationMode:tiered, LLVM:false, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Improvements in System.Memory.Span<Byte>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
IndexOfAnyFiveValues - Duration of single invocation	683.51 ns	469.08 ns	0.69	0.01	True
IndexOfAnyThreeValues - Duration of single invocation	449.32 ns	380.92 ns	0.85	0.01	False
LastIndexOfAnyValues - Duration of single invocation	361.24 ns	320.07 ns	0.89	0.01	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

Repro Steps

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
AOT MONO build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false
Dotnet SDK installed for dotnet commands
Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create aot directory 
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Memory.Span&lt;Byte&gt;*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Memory.Span&lt;Byte&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Memory.Span&lt;Byte&gt;*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Memory.Span&lt;Byte&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200

Payloads

Baseline
Compare

Histogram

System.Memory.Span<Byte>.IndexOfAnyFiveValues(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 469.0760958084139 < 648.7957417508766.
IsChangePoint: Marked as a change because one of 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 345.1325207466473 (T) = (0 -470.0411789103622) / Math.Sqrt((21.074837755581367 / (299)) + (1.3191486441209324 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.3171938568722695 = (688.3962360930794 - 470.0411789103622) / 688.3962360930794 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Memory.Span<Byte>.IndexOfAnyThreeValues(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 380.9206471379465 < 432.69158355013803.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 1/5/2023 4:05:11 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 54.08694588408392 (T) = (0 -379.3795869610783) / Math.Sqrt((20.959500296815666 / (299)) + (6.725830292005971 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.15873213024525734 = (450.961697933002 - 379.3795869610783) / 450.961697933002 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Memory.Span<Byte>.LastIndexOfAnyValues(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 320.0692282361499 < 344.1476848771163.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 3/10/2023 12:53:36 AM, 4/12/2023 8:39:35 PM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 95.29743552679189 (T) = (0 -319.80359875486715) / Math.Sqrt((70.7114968711727 / (299)) + (0.14111804265312458 / (2))) is greater than 1.9679296690655894 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (2) - 2, .975) and 0.14172053468379325 = (372.6101015792628 - 319.80359875486715) / 372.6101015792628 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

performanceautofiler · 2023-04-18T16:21:51Z

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	7a57f6dd800e77508a4ba405e623059935c82f43
Compare	887c043eb94be364188e2b23a87efa214ea57f1e
Diff	Diff
Configs	CompilationMode:tiered, LLVM:false, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Improvements in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<SByte>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
SumBenchmark - Duration of single invocation	22.46 ns	0.10 ns	0.00	0.02	True
GreaterThanAllBenchmark - Duration of single invocation	32.74 ns	0.21 ns	0.01	0.03	False
EqualsBenchmark - Duration of single invocation	6.22 ns	0.67 ns	0.11	0.24	False
AbsBenchmark - Duration of single invocation	50.10 ns	1.27 ns	0.03	0.02	True
LessThanAnyBenchmark - Duration of single invocation	33.21 ns	0.00 ns	0.00	0.03	False
LessThanOrEqualAnyBenchmark - Duration of single invocation	36.17 ns	0.00 ns	0.00	0.02	True
GreaterThanOrEqualBenchmark - Duration of single invocation	64.77 ns	1.03 ns	0.02	0.02	True
LessThanOrEqualAllBenchmark - Duration of single invocation	3.82 ns	0.08 ns	0.02	0.18	False
MinBenchmark - Duration of single invocation	65.88 ns	1.49 ns	0.02	0.02	True
GreaterThanAnyBenchmark - Duration of single invocation	3.15 ns	0.00 ns	0.00	0.33	False
ConditionalSelectBenchmark - Duration of single invocation	15.55 ns	1.40 ns	0.09	0.05	False
GreaterThanOrEqualAnyBenchmark - Duration of single invocation	3.86 ns	0.05 ns	0.01	0.17	False
GreaterThanBenchmark - Duration of single invocation	60.36 ns	1.08 ns	0.02	0.01	True
GreaterThanOrEqualAllBenchmark - Duration of single invocation	36.26 ns	0.00 ns	0.00	0.03	True
AndNotBenchmark - Duration of single invocation	13.71 ns	1.33 ns	0.10	0.03	False
EqualsStaticBenchmark - Duration of single invocation	56.25 ns	1.03 ns	0.02	0.02	True
EqualsAnyBenchmark - Duration of single invocation	32.82 ns	0.00 ns	0.00	0.03	False
LessThanOrEqualBenchmark - Duration of single invocation	61.90 ns	1.33 ns	0.02	0.02	True
MaxBenchmark - Duration of single invocation	63.90 ns	1.53 ns	0.02	0.02	True
LessThanBenchmark - Duration of single invocation	55.37 ns	1.02 ns	0.02	0.02	True
LessThanAllBenchmark - Duration of single invocation	2.91 ns	0.00 ns	0.00	0.29	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

Repro Steps

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
AOT MONO build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false
Dotnet SDK installed for dotnet commands
Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create aot directory 
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;SByte&gt;*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;SByte&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;SByte&gt;*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;SByte&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200

Payloads

Baseline
Compare

Histogram

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<SByte>.SumBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.10113015051841304 < 21.344090750934694.
IsChangePoint: Marked as a change because one of 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 446.5430177957088 (T) = (0 -0.04624874923365599) / Math.Sqrt((0.6265930568482628 / (299)) + (0.0029016108618603793 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9980538036114367 = (23.763659980788166 - 0.04624874923365599) / 23.763659980788166 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<SByte>.GreaterThanAllBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.20576790903352832 < 31.11276833361364.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 197.31947568666433 (T) = (0 -0.09597523370892092) / Math.Sqrt((9.586732181554991 / (299)) + (0.01240894177584459 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9974129214465476 = (37.09792019296953 - 0.09597523370892092) / 37.09792019296953 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<SByte>.EqualsBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.6694232178676656 < 5.7510568021941975.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 69.75901884704302 (T) = (0 -0.7322618221822434) / Math.Sqrt((4.078376791872346 / (299)) + (0.005518763196489403 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9211059404660042 = (9.281583765716977 - 0.7322618221822434) / 9.281583765716977 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<SByte>.AbsBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.2662807975221064 < 47.68140369213748.
IsChangePoint: Marked as a change because one of 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 375.97059108730366 (T) = (0 -1.2077089491371844) / Math.Sqrt((3.248651477135806 / (299)) + (0.030233545538994128 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9768812263487647 = (52.239317160867415 - 1.2077089491371844) / 52.239317160867415 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<SByte>.LessThanAnyBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 31.28625870536464.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 203.3674689326492 (T) = (0 -0.006032108415164963) / Math.Sqrt((9.89362641878503 / (299)) + (0.00014554532772921585 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9998370568642326 = (37.019714802686025 - 0.006032108415164963) / 37.019714802686025 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<SByte>.LessThanOrEqualAnyBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.0047085812572027754 < 34.303285985559114.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 227.02500075949285 (T) = (0 -0.0009417162514405551) / Math.Sqrt((9.65094169014083 / (299)) + (4.434147491136254E-06 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.9999769122753295 = (40.78861234182271 - 0.0009417162514405551) / 40.78861234182271 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<SByte>.GreaterThanOrEqualBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0285521011320813 < 61.584830296023576.
IsChangePoint: Marked as a change because one of 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 462.8921091736083 (T) = (0 -1.1486695981438775) / Math.Sqrt((4.103724045376175 / (299)) + (0.024336554174861663 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9826741733557913 = (66.2981121612362 - 1.1486695981438775) / 66.2981121612362 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<SByte>.LessThanOrEqualAllBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.08259835866486889 < 3.6746626599011005.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 76.98024748144955 (T) = (0 -0.018758004374194788) / Math.Sqrt((1.7622101881503631 / (299)) + (0.0012971071347344138 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.9969031990037622 = (6.057219820383312 - 0.018758004374194788) / 6.057219820383312 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<SByte>.MinBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.4882726163097562 < 62.63912496910894.
IsChangePoint: Marked as a change because one of 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 608.3474959290515 (T) = (0 -1.3042544938876273) / Math.Sqrt((1.8453420785615886 / (299)) + (0.024920245603799483 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.9801054641182187 = (65.55842778328 - 1.3042544938876273) / 65.55842778328 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<SByte>.GreaterThanAnyBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 2.9813535598117595.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 71.85595889220521 (T) = (0 -0.005358646111044523) / Math.Sqrt((1.6674860494455304 / (299)) + (0.00010009910423168451 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9990046127837113 = (5.383478934986578 - 0.005358646111044523) / 5.383478934986578 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<SByte>.ConditionalSelectBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.3971197930848922 < 14.884471206501441.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 4/12/2023 8:39:35 PM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 100.12615595944308 (T) = (0 -1.253086018478977) / Math.Sqrt((0.40939540460258816 / (299)) + (0.041491456454455206 / (2))) is greater than 1.9679296690655894 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (2) - 2, .975) and 0.9223756045405798 = (16.14294077348471 - 1.253086018478977) / 16.14294077348471 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<SByte>.GreaterThanOrEqualAnyBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.047339585786150044 < 3.7528372124620804.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 78.81284974928668 (T) = (0 -0.015663277362425145) / Math.Sqrt((1.7183408456443814 / (299)) + (0.0004980624528231138 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9974130558708189 = (6.054741262380244 - 0.015663277362425145) / 6.054741262380244 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<SByte>.GreaterThanBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0774203797760256 < 56.41789234632807.
IsChangePoint: Marked as a change because one of 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 400.21433386424223 (T) = (0 -1.091153106860029) / Math.Sqrt((6.38590745489592 / (299)) + (0.006681866185829576 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9823505458344104 = (61.82361769506726 - 1.091153106860029) / 61.82361769506726 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<SByte>.GreaterThanOrEqualAllBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.001583433138413948 < 34.4412941765879.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 226.1853452929633 (T) = (0 -0.004750329716022857) / Math.Sqrt((9.790447585563705 / (299)) + (3.6701367001621354E-05 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9998839667711853 = (40.939390936111884 - 0.004750329716022857) / 40.939390936111884 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<SByte>.AndNotBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.33282641210315 < 12.888436343901153.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 4/12/2023 8:39:35 PM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 270.42065331633194 (T) = (0 -1.3033429454908974) / Math.Sqrt((0.40230126208964123 / (299)) + (0.0017385496069516337 / (2))) is greater than 1.9679296690655894 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (2) - 2, .975) and 0.907101052537773 = (14.029684739117624 - 1.3033429454908974) / 14.029684739117624 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<SByte>.EqualsStaticBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0302679225587261 < 53.28239200872912.
IsChangePoint: Marked as a change because one of 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 362.8672918692053 (T) = (0 -1.1353159510779847) / Math.Sqrt((6.002228253228566 / (299)) + (0.016401666334535932 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9802741648062075 = (57.554772202256814 - 1.1353159510779847) / 57.554772202256814 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<SByte>.EqualsAnyBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.004994815922224176 < 31.19158332239263.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 203.89181900180319 (T) = (0 -0.006081276926279675) / Math.Sqrt((9.871097792275943 / (299)) + (8.35602167641125E-05 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9998359262046581 = (37.06427899475435 - 0.006081276926279675) / 37.06427899475435 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<SByte>.LessThanOrEqualBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.3262933351470418 < 59.025987146784566.
IsChangePoint: Marked as a change because one of 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 467.4928236461123 (T) = (0 -1.1982890251003906) / Math.Sqrt((3.220477997803017 / (299)) + (0.03327777322783282 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.9809528016516104 = (62.91156332719677 - 1.1982890251003906) / 62.91156332719677 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<SByte>.MaxBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.5253860955148109 < 61.113507420645334.
IsChangePoint: Marked as a change because one of 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 449.2219368050595 (T) = (0 -1.202862131206085) / Math.Sqrt((2.4731556806887887 / (299)) + (0.056270344925308066 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.9811977237923434 = (63.974282577353684 - 1.202862131206085) / 63.974282577353684 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<SByte>.LessThanBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.020369590378034 < 53.63208818814862.
IsChangePoint: Marked as a change because one of 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 354.4903632104546 (T) = (0 -1.085686427829301) / Math.Sqrt((5.977199433032375 / (299)) + (0.021156425423081337 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9811014880747508 = (57.44824947718664 - 1.085686427829301) / 57.44824947718664 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<SByte>.LessThanAllBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 3.0128969797308987.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 71.41753974046384 (T) = (0 -0.002090308573562619) / Math.Sqrt((1.6895396661095887 / (299)) + (1.0485977293272577E-05 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.9996108587208818 = (5.371593006783996 - 0.002090308573562619) / 5.371593006783996 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

performanceautofiler · 2023-04-18T16:21:52Z

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	7a57f6dd800e77508a4ba405e623059935c82f43
Compare	887c043eb94be364188e2b23a87efa214ea57f1e
Diff	Diff
Configs	CompilationMode:tiered, LLVM:false, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Improvements in System.Numerics.Tests.Perf_VectorOf<UInt16>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
LessThanBenchmark - Duration of single invocation	26.80 ns	1.03 ns	0.04	0.02	True
EqualsAnyBenchmark - Duration of single invocation	14.28 ns	0.00 ns	0.00	0.04	False
EqualsStaticBenchmark - Duration of single invocation	24.68 ns	1.02 ns	0.04	0.01	True
LessThanOrEqualAllBenchmark - Duration of single invocation	16.66 ns	0.02 ns	0.00	0.03	True
GreaterThanOrEqualAnyBenchmark - Duration of single invocation	16.11 ns	0.00 ns	0.00	0.05	True
LessThanAllBenchmark - Duration of single invocation	14.78 ns	0.07 ns	0.00	0.05	True
MinBenchmark - Duration of single invocation	28.58 ns	1.16 ns	0.04	0.02	True
EqualsBenchmark - Duration of single invocation	4.35 ns	0.00 ns	0.00	0.24	False
GreaterThanAnyBenchmark - Duration of single invocation	14.77 ns	0.00 ns	0.00	0.07	False
GreaterThanOrEqualBenchmark - Duration of single invocation	26.82 ns	1.10 ns	0.04	0.02	True
GreaterThanBenchmark - Duration of single invocation	24.68 ns	1.02 ns	0.04	0.01	True
LessThanOrEqualBenchmark - Duration of single invocation	28.20 ns	1.22 ns	0.04	0.01	True
MaxBenchmark - Duration of single invocation	28.89 ns	1.08 ns	0.04	0.02	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

Repro Steps

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
AOT MONO build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false
Dotnet SDK installed for dotnet commands
Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create aot directory 
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_VectorOf<UInt16>.LessThanBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.03312182761144 < 24.650548008071738.
IsChangePoint: Marked as a change because one of 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 480.68241191202367 (T) = (0 -0.9938994071016025) / Math.Sqrt((0.782569213339627 / (299)) + (0.001310573930494203 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.963296574915454 = (27.07920050546136 - 0.9938994071016025) / 27.07920050546136 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<UInt16>.EqualsAnyBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 13.896046072558084.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 212.8743982537679 (T) = (0 -0) / Math.Sqrt((1.8528730923010732 / (299)) + (0 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 1 = (16.757555884786395 - 0) / 16.757555884786395 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<UInt16>.EqualsStaticBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0197087428993978 < 23.4462683413558.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 319.99690621331155 (T) = (0 -1.1240417129634466) / Math.Sqrt((1.0579077717820518 / (299)) + (0.010196440962574957 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9569175755195185 = (26.090493432483004 - 1.1240417129634466) / 26.090493432483004 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<UInt16>.LessThanOrEqualAllBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.01733928453146087 < 15.681350938599456.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 238.07708319645482 (T) = (0 -0.00401232095084182) / Math.Sqrt((1.8494501894784439 / (299)) + (5.689205459883027E-05 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.9997859573621436 = (18.745428439046883 - 0.00401232095084182) / 18.745428439046883 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<UInt16>.GreaterThanOrEqualAnyBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 15.383264421343625.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 235.51412973894404 (T) = (0 -0.0017651851239583547) / Math.Sqrt((1.8575254331631574 / (299)) + (8.181733736792748E-06 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.9999049301033364 = (18.56723511757123 - 0.0017651851239583547) / 18.56723511757123 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<UInt16>.LessThanAllBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.0722280901756011 < 14.013546143317651.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 210.58659905634062 (T) = (0 -0.01454886653587191) / Math.Sqrt((1.851723030208286 / (299)) + (0.0010397039823360958 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.9991372204494667 = (16.862785548034672 - 0.01454886653587191) / 16.862785548034672 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<UInt16>.MinBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.158563999789745 < 27.320267775149734.
IsChangePoint: Marked as a change because one of 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 620.5139024038654 (T) = (0 -1.2180492197823205) / Math.Sqrt((0.17288254096729264 / (299)) + (0.005807849023774654 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9582527607147012 = (29.176760922039048 - 1.2180492197823205) / 29.176760922039048 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<UInt16>.EqualsBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 4.288669993247314.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 70.38978607411235 (T) = (0 -0.006834710549828049) / Math.Sqrt((2.9721161740923385 / (299)) + (0.00018685307319972336 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9990293261204077 = (7.0412016780540805 - 0.006834710549828049) / 7.0412016780540805 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<UInt16>.GreaterThanAnyBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 14.012675973592438.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 206.95400143184887 (T) = (0 -0.018520017040976486) / Math.Sqrt((1.8542576627586325 / (299)) + (0.0012783408053733364 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9988930561531089 = (16.73076470228384 - 0.018520017040976486) / 16.73076470228384 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<UInt16>.GreaterThanOrEqualBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0985380347682216 < 25.495994059183282.
IsChangePoint: Marked as a change because one of 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 442.9931560928361 (T) = (0 -1.0303163376566808) / Math.Sqrt((0.7774503984493059 / (299)) + (0.004521298302039133 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.963317420992266 = (28.087347332897558 - 1.0303163376566808) / 28.087347332897558 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<UInt16>.GreaterThanBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.023139070979359 < 23.40362396553847.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 412.01797691515986 (T) = (0 -1.0489620368033912) / Math.Sqrt((1.0523840854764057 / (299)) + (0.0007032723220822757 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9598033888976774 = (26.09578290401653 - 1.0489620368033912) / 26.09578290401653 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<UInt16>.LessThanOrEqualBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.2181461410612004 < 26.79356127502554.
IsChangePoint: Marked as a change because one of 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 402.65243235515277 (T) = (0 -1.121168320566969) / Math.Sqrt((0.7581487535542883 / (299)) + (0.01190991774137424 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.9618096951603431 = (29.357406945931128 - 1.121168320566969) / 29.357406945931128 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<UInt16>.MaxBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.084059758602459 < 27.376248201147707.
IsChangePoint: Marked as a change because one of 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 435.91308234619993 (T) = (0 -1.0972090872644256) / Math.Sqrt((0.16101029575253328 / (299)) + (0.018293967783513568 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.9626017172290445 = (29.33848845371446 - 1.0972090872644256) / 29.33848845371446 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	7a57f6dd800e77508a4ba405e623059935c82f43
Compare	887c043eb94be364188e2b23a87efa214ea57f1e
Diff	Diff
Configs	CompilationMode:tiered, LLVM:false, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Improvements in System.Tests.Perf_String

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
Replace_Char - Duration of single invocation	7.27 μs	2.58 μs	0.36	0.06	True
Replace_Char - Duration of single invocation	228.06 ns	184.95 ns	0.81	0.04	False
Replace_Char - Duration of single invocation	692.56 ns	366.42 ns	0.53	0.13	False
Replace_String - Duration of single invocation	494.23 ns	322.30 ns	0.65	0.07	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

Repro Steps

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
AOT MONO build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false
Dotnet SDK installed for dotnet commands
Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create aot directory 
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Tests.Perf_String*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Tests.Perf_String* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Tests.Perf_String*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Tests.Perf_String* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_String.Replace_Char(text: "yfesgj0sg1ijslnjsb3uofdz3tbzf6ysgblu3at20nfab2wei1kxfbvsbpzwhanjczcqa2psra3aacxb67qnwbnfp2tok6v0a58lzfdql1fehvs91yzkt9xam7ahjbhvpd9edll13ab46i74ktwwgkgbi792e5gkuuzevo5qm8qt83edag7zovoe686gmtw730kms2i5xgji4xcp25287q68fvhwszd3mszht2uh7bchlgkj5qnq1x9m4lg7vwn8cq5l756akua6oyx9k71bmxbysnmhvxvlxde4k9maumfgxd8gxhxx4mwpph2ttyox9zilt3ylv1q9s4bopfuoa8qlrzodg2q67sh85wx4slcd6w7ufnendaxai633ove2ktbaxdt2sz6y6mo42473xd274gz833p6hj3mu77c4m4od9e5s8btxleh0efqnu9zj9rwtbk5758lio35b3q426j5fwwq1qyknfedrsmqyfw1m38mkkotdf7n0vr6p3erhy8dkzntr9fwjrslxjgrbegih0n6bpb5bfuy55bu65ce9kejcfifxwpcs05umrsb8kvd64q2iwugbbi7vd35g5ho0rff9rhombgzzaniyq7bbjbqr88jyw4ccgnoyl31of3a5thv0vg08gnrqzxas800hewtw8tnwgw5pav81ntdpdd62689x3iqpc317y82b3e2trbpdzieoxldaz009tz37gqmh4bdp1bv9lnl5s58udb11z0h7i2sdl5nbyhjyfzxwzezmp4qx0i3eyvsd3fg8sryq9jhlvkonnfcvb4snl4mcbimdzg49tzdhqjmfxfcq3p1st6b9x2xyevo17evpqp4yc4f2rm0f26ivr3t2f5m0boc44vituxaovcqy1jrkcs6im2kdu3jvcexx2k76egve63aon5a6nbxss4rcke90npmqp35qluf571ms160y2nhaqef835wah41qru8tauu362v0r8konl8", oldChar: 'b', newChar: '+')

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 2.5840135830944884 < 6.89681145518524.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 346.9510605822912 (T) = (0 -2586.8984731103674) / Math.Sqrt((48826.73905257054 / (299)) + (27.37179685559163 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.6362879345101611 = (7112.49012216956 - 2586.8984731103674) / 7112.49012216956 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Tests.Perf_String.Replace_Char(text: "This is a very nice sentence", oldChar: 'i', newChar: 'I')

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 184.95293126788127 < 216.16357016730817.
IsChangePoint: Marked as a change because one of 12/30/2022 6:06:02 AM, 3/6/2023 6:03:21 AM, 4/11/2023 11:54:05 PM, 4/12/2023 8:39:35 PM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 75.46522231430193 (T) = (0 -185.2520698254923) / Math.Sqrt((55.60176611276047 / (299)) + (0.17896775329919845 / (2))) is greater than 1.9679296690655894 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (2) - 2, .975) and 0.1761383463997582 = (224.8582259120185 - 185.2520698254923) / 224.8582259120185 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Tests.Perf_String.Replace_Char(text: "yfesgj0sg1ijslnjsb3uofdz3tbzf6ysgblu3at20nfab2wei1kxfbvsbpzwhanjczcqa2psra3aacxb67qnwbnfp2tok6v0a58l", oldChar: 'b', newChar: '+')

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 366.42232323352914 < 665.5700779858105.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 4/12/2023 8:39:35 PM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 132.57831867296585 (T) = (0 -368.22194122946127) / Math.Sqrt((606.7668595131727 / (299)) + (6.4772498625655075 / (2))) is greater than 1.9679296690655894 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (2) - 2, .975) and 0.4524709195830874 = (672.5157702109283 - 368.22194122946127) / 672.5157702109283 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Tests.Perf_String.Replace_String(text: "This is a very nice sentence. This is another very nice sentence.", oldValue: "a", newValue: "b")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 322.30165771952073 < 469.9249365689921.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 4/12/2023 8:39:35 PM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 224.60615075687608 (T) = (0 -322.01703554263054) / Math.Sqrt((141.84769176027353 / (299)) + (0.16201956715539517 / (2))) is greater than 1.9679296690655894 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (2) - 2, .975) and 0.3420269880554911 = (489.4076652034298 - 322.01703554263054) / 489.4076652034298 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

performanceautofiler · 2023-04-18T16:21:54Z

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	7a57f6dd800e77508a4ba405e623059935c82f43
Compare	887c043eb94be364188e2b23a87efa214ea57f1e
Diff	Diff
Configs	CompilationMode:tiered, LLVM:false, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Improvements in System.Numerics.Tests.Perf_VectorOf<Double>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
EqualsBenchmark - Duration of single invocation	12.09 ns	0.00 ns	0.00	0.11	False
GreaterThanBenchmark - Duration of single invocation	7.72 ns	1.10 ns	0.14	0.06	False
MaxBenchmark - Duration of single invocation	9.13 ns	1.17 ns	0.13	0.06	False
MinBenchmark - Duration of single invocation	9.02 ns	1.10 ns	0.12	0.06	False
LessThanOrEqualBenchmark - Duration of single invocation	9.05 ns	1.05 ns	0.12	0.04	False
EqualsStaticBenchmark - Duration of single invocation	7.22 ns	1.20 ns	0.17	0.06	False
GreaterThanOrEqualBenchmark - Duration of single invocation	7.83 ns	1.11 ns	0.14	0.04	False
LessThanBenchmark - Duration of single invocation	8.71 ns	1.09 ns	0.13	0.03	False
AbsBenchmark - Duration of single invocation	11.10 ns	1.13 ns	0.10	0.07	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

Repro Steps

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
AOT MONO build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false
Dotnet SDK installed for dotnet commands
Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create aot directory 
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_VectorOf<Double>.EqualsBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 11.424536913138555.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 9:06:14 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 144.93709600190616 (T) = (0 -0.0024569512425617057) / Math.Sqrt((3.112500451412558 / (299)) + (9.035767736288572E-06 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9998338965683644 = (14.791694658979686 - 0.0024569512425617057) / 14.791694658979686 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Double>.GreaterThanBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0964118801015594 < 6.830086719583521.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 86.63630235691174 (T) = (0 -1.0151711771734302) / Math.Sqrt((1.9079123498185442 / (299)) + (0.008522191840394567 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.8873040920292706 = (9.00805712872114 - 1.0151711771734302) / 9.00805712872114 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Double>.MaxBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.1682576556186435 < 8.454028638552234.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 85.55857107690295 (T) = (0 -1.1548729253219048) / Math.Sqrt((1.6754808792719738 / (299)) + (0.025696682561608605 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.8904106876150356 = (10.538189356139746 - 1.1548729253219048) / 10.538189356139746 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Double>.MinBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.095652339380036 < 8.52583844106852.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 65.09699347232963 (T) = (0 -1.1997161236289706) / Math.Sqrt((1.6630354344472713 / (299)) + (0.060848423585674404 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.8866298243040087 = (10.582290415127156 - 1.1997161236289706) / 10.582290415127156 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Double>.LessThanOrEqualBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0509483325138234 < 8.557392415493572.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 111.44349749273107 (T) = (0 -0.995574910702056) / Math.Sqrt((2.1913762320351577 / (299)) + (0.003275249055245882 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9099439529124118 = (11.055058964932833 - 0.995574910702056) / 11.055058964932833 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Double>.EqualsStaticBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.201278087024121 < 6.8523363574264335.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 80.40365743537062 (T) = (0 -1.087913394070121) / Math.Sqrt((1.952975444202531 / (299)) + (0.009648564429134945 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.8794726027323786 = (9.026274678897249 - 1.087913394070121) / 9.026274678897249 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Double>.GreaterThanOrEqualBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.1112830108148122 < 7.581636507409645.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 96.12885341424166 (T) = (0 -1.1311957279173643) / Math.Sqrt((1.5016286369240537 / (299)) + (0.00982027535348008 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.880215069373517 = (9.443556230329945 - 1.1311957279173643) / 9.443556230329945 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Double>.LessThanBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.094821716886946 < 8.355238499983415.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 106.34201373479989 (T) = (0 -1.1256259447854287) / Math.Sqrt((2.2154631933772433 / (299)) + (0.0039584010493743625 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.8964613624589299 = (10.87155453768582 - 1.1256259447854287) / 10.87155453768582 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Double>.AbsBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.1349334131486806 < 10.562155308947961.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 138.40778996739638 (T) = (0 -1.1106799363786095) / Math.Sqrt((0.3981970649397952 / (299)) + (0.014497247825615709 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.8901817160615713 = (10.113797962835854 - 1.1106799363786095) / 10.113797962835854 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

performanceautofiler · 2023-04-18T16:21:55Z

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	7a57f6dd800e77508a4ba405e623059935c82f43
Compare	887c043eb94be364188e2b23a87efa214ea57f1e
Diff	Diff
Configs	CompilationMode:tiered, LLVM:false, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Improvements in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
GreaterThanAnyBenchmark - Duration of single invocation	2.91 ns	0.02 ns	0.01	0.32	False
GreaterThanAllBenchmark - Duration of single invocation	7.11 ns	0.00 ns	0.00	0.11	False
EqualsBenchmark - Duration of single invocation	5.89 ns	0.73 ns	0.12	0.20	False
EqualsStaticBenchmark - Duration of single invocation	16.76 ns	1.23 ns	0.07	0.03	False
ConditionalSelectBenchmark - Duration of single invocation	15.52 ns	1.25 ns	0.08	0.05	False
AndNotBenchmark - Duration of single invocation	13.32 ns	1.04 ns	0.08	0.05	False
GreaterThanOrEqualBenchmark - Duration of single invocation	18.10 ns	1.01 ns	0.06	0.03	False
AbsBenchmark - Duration of single invocation	15.57 ns	1.39 ns	0.09	0.06	True
GreaterThanBenchmark - Duration of single invocation	17.01 ns	1.57 ns	0.09	0.05	False
GreaterThanOrEqualAnyBenchmark - Duration of single invocation	6.49 ns	0.05 ns	0.01	0.25	False
EqualsAnyBenchmark - Duration of single invocation	6.79 ns	0.17 ns	0.02	0.15	False
LessThanAllBenchmark - Duration of single invocation	3.01 ns	0.00 ns	0.00	0.37	False
LessThanOrEqualBenchmark - Duration of single invocation	17.99 ns	1.15 ns	0.06	0.03	True
LessThanOrEqualAnyBenchmark - Duration of single invocation	8.20 ns	0.00 ns	0.00	0.08	False
MaxBenchmark - Duration of single invocation	18.73 ns	1.33 ns	0.07	0.03	True
LessThanAnyBenchmark - Duration of single invocation	7.13 ns	0.00 ns	0.00	0.12	False
LessThanOrEqualAllBenchmark - Duration of single invocation	3.42 ns	0.00 ns	0.00	0.16	False
GreaterThanOrEqualAllBenchmark - Duration of single invocation	8.18 ns	0.00 ns	0.00	0.06	False
LessThanBenchmark - Duration of single invocation	16.57 ns	1.32 ns	0.08	0.04	False
MinBenchmark - Duration of single invocation	18.19 ns	1.13 ns	0.06	0.05	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

Repro Steps

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
AOT MONO build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false
Dotnet SDK installed for dotnet commands
Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create aot directory 
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int32&gt;*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int32&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int32&gt;*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int32&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200

Payloads

Baseline
Compare

Histogram

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int32>.GreaterThanAnyBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.019007893286052523 < 2.806934622985855.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 68.73599458350354 (T) = (0 -0.004881392153480463) / Math.Sqrt((1.6629522654836306 / (299)) + (8.875201249826756E-05 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9990505369460274 = (5.141213376399182 - 0.004881392153480463) / 5.141213376399182 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int32>.GreaterThanAllBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 6.706968762086716.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 99.94705470004753 (T) = (0 -0) / Math.Sqrt((2.8708093364457197 / (299)) + (0 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 1 = (9.793469641766904 - 0) / 9.793469641766904 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int32>.EqualsBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.7316050067923329 < 5.707234374820109.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 58.103623611795264 (T) = (0 -0.776268822216201) / Math.Sqrt((4.754119421145546 / (299)) + (0.018837166977950164 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9148604122240622 = (9.117601370811318 - 0.776268822216201) / 9.117601370811318 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int32>.EqualsStaticBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.2267873644549074 < 15.866846830619622.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 152.969807156414 (T) = (0 -1.2935809138883894) / Math.Sqrt((1.683039649415584 / (299)) + (0.02881892406723359 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9305379974685528 = (18.62285662298254 - 1.2935809138883894) / 18.62285662298254 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int32>.ConditionalSelectBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.2496808872262994 < 14.802787231642016.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 4/12/2023 8:39:35 PM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 252.97038926740228 (T) = (0 -1.296566531605532) / Math.Sqrt((0.37626096898684414 / (299)) + (0.004396527297711709 / (2))) is greater than 1.9679296690655894 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (2) - 2, .975) and 0.9198143133359321 = (16.169550770792835 - 1.296566531605532) / 16.169550770792835 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int32>.AndNotBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0355707708009574 < 12.873204315052037.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 4/12/2023 8:39:35 PM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 186.03849818246846 (T) = (0 -1.0942868650601514) / Math.Sqrt((0.409887948005268 / (299)) + (0.0068951594501091345 / (2))) is greater than 1.9679296690655894 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (2) - 2, .975) and 0.921882125399522 = (14.008149487639232 - 1.0942868650601514) / 14.008149487639232 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int32>.GreaterThanOrEqualBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.014394615365394 < 17.091529378191122.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 211.00822720421007 (T) = (0 -1.1428769113564228) / Math.Sqrt((1.7830990279306544 / (299)) + (0.007617961999784657 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9424524987828751 = (19.859713926489803 - 1.1428769113564228) / 19.859713926489803 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int32>.AbsBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.3855586966520719 < 14.77645482972224.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 165.5349273782905 (T) = (0 -1.2002362892218041) / Math.Sqrt((0.30131171456474515 / (299)) + (0.02866254158239647 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.9188507670475603 = (14.790482245534513 - 1.2002362892218041) / 14.790482245534513 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int32>.GreaterThanBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.565855413757601 < 16.159639999391832.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 124.71544011617289 (T) = (0 -1.3366022300811702) / Math.Sqrt((1.9301594123252683 / (299)) + (0.05393870228678991 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9294578870823124 = (18.947578613654372 - 1.3366022300811702) / 18.947578613654372 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int32>.GreaterThanOrEqualAnyBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.05185611171917622 < 3.303688831302934.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 67.84209635014926 (T) = (0 -0.07353912853616537) / Math.Sqrt((1.6125687356975258 / (299)) + (0.005693393040953253 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9870409864642277 = (5.67474741292354 - 0.07353912853616537) / 5.67474741292354 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int32>.EqualsAnyBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.16830323522354937 < 6.573123975677058.
IsChangePoint: Marked as a change because one of 12/30/2022 6:06:02 AM, 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 90.13167395793668 (T) = (0 -0.044801134875886356) / Math.Sqrt((2.862433194555399 / (299)) + (0.006799933429103064 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9953403113661317 = (9.614619858986952 - 0.044801134875886356) / 9.614619858986952 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int32>.LessThanAllBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 2.8052248682282417.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 68.71526881184698 (T) = (0 -0.0021603523477280754) / Math.Sqrt((1.64684189548218 / (299)) + (1.866848906533683E-05 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9995767341841405 = (5.104008561005096 - 0.0021603523477280754) / 5.104008561005096 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int32>.LessThanOrEqualBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.148228491930056 < 16.869394242055808.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 184.30816743543636 (T) = (0 -1.1798272207167537) / Math.Sqrt((1.406315125402103 / (299)) + (0.02047213284496215 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9393259708112783 = (19.445341548803274 - 1.1798272207167537) / 19.445341548803274 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int32>.LessThanOrEqualAnyBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 7.672828480482539.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 110.35651747571009 (T) = (0 -0.004008206651127815) / Math.Sqrt((2.831971774723748 / (299)) + (6.426288223258102E-05 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.999627253716728 = (10.753176707604757 - 0.004008206651127815) / 10.753176707604757 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int32>.MaxBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.3337781525698162 < 17.329671845690005.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 188.58609816312352 (T) = (0 -1.1672966249228942) / Math.Sqrt((1.1777954279499256 / (299)) + (0.02353897326941065 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9412210099917295 = (19.85907931998576 - 1.1672966249228942) / 19.85907931998576 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int32>.LessThanAnyBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 6.538090392302095.
IsChangePoint: Marked as a change because one of 12/30/2022 6:06:02 AM, 3/6/2023 4:10:52 PM, 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 97.72198651721999 (T) = (0 -0.00044630787253206733) / Math.Sqrt((2.8899605331438876 / (299)) + (7.967628683364002E-07 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9999535476970818 = (9.607873980264353 - 0.00044630787253206733) / 9.607873980264353 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int32>.LessThanOrEqualAllBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 3.2237240965932883.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 75.25120651397228 (T) = (0 -0) / Math.Sqrt((1.7007742556439296 / (299)) + (0 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 1 = (5.675461866090245 - 0) / 5.675461866090245 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int32>.GreaterThanOrEqualAllBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 7.857739947873561.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 113.10109101063935 (T) = (0 -0) / Math.Sqrt((2.7875228291261434 / (299)) + (0 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 1 = (10.92044702781338 - 0) / 10.92044702781338 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int32>.LessThanBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.3247701602487119 < 15.884277685272304.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 145.86330316152512 (T) = (0 -1.1814801261128216) / Math.Sqrt((1.6740658094583405 / (299)) + (0.034904329188089965 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9366138774434888 = (18.639413146931084 - 1.1814801261128216) / 18.639413146931084 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int32>.MinBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.12941883256255 < 17.296553199810425.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 215.4364109200783 (T) = (0 -1.1601945210865274) / Math.Sqrt((1.23875869670624 / (299)) + (0.017356005107963843 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.9418712587681357 = (19.959051176744666 - 1.1601945210865274) / 19.959051176744666 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

performanceautofiler · 2023-04-18T16:21:56Z

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	7a57f6dd800e77508a4ba405e623059935c82f43
Compare	887c043eb94be364188e2b23a87efa214ea57f1e
Diff	Diff
Configs	CompilationMode:tiered, LLVM:false, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Improvements in System.Numerics.Tests.Perf_VectorOf<Int16>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
LessThanOrEqualBenchmark - Duration of single invocation	28.12 ns	1.03 ns	0.04	0.02	True
MinBenchmark - Duration of single invocation	29.34 ns	1.13 ns	0.04	0.01	True
MaxBenchmark - Duration of single invocation	28.76 ns	1.08 ns	0.04	0.02	True
GreaterThanOrEqualAnyBenchmark - Duration of single invocation	16.32 ns	0.00 ns	0.00	0.04	True
LessThanAllBenchmark - Duration of single invocation	14.53 ns	0.00 ns	0.00	0.06	False
LessThanBenchmark - Duration of single invocation	25.69 ns	1.17 ns	0.05	0.02	True
LessThanOrEqualAllBenchmark - Duration of single invocation	16.54 ns	0.03 ns	0.00	0.05	False
EqualsBenchmark - Duration of single invocation	4.14 ns	0.00 ns	0.00	0.20	False
AbsBenchmark - Duration of single invocation	20.50 ns	1.05 ns	0.05	0.02	True
GreaterThanBenchmark - Duration of single invocation	24.67 ns	1.16 ns	0.05	0.02	True
EqualsAnyBenchmark - Duration of single invocation	14.46 ns	0.01 ns	0.00	0.03	False
GreaterThanAnyBenchmark - Duration of single invocation	14.33 ns	0.00 ns	0.00	0.06	False
GreaterThanOrEqualBenchmark - Duration of single invocation	26.90 ns	1.24 ns	0.05	0.03	True
EqualsStaticBenchmark - Duration of single invocation	24.62 ns	1.15 ns	0.05	0.02	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

Repro Steps

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
AOT MONO build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false
Dotnet SDK installed for dotnet commands
Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create aot directory 
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_VectorOf<Int16>.LessThanOrEqualBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0299094699093367 < 26.70434809332945.
IsChangePoint: Marked as a change because one of 3/10/2023 12:53:36 AM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 376.8406320746397 (T) = (0 -1.0795970165643052) / Math.Sqrt((1.369422425942183 / (299)) + (0.004901659404778049 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9637625953398162 = (29.792338239678806 - 1.0795970165643052) / 29.792338239678806 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Int16>.MinBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.1334116145263868 < 27.206829700228827.
IsChangePoint: Marked as a change because one of 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 936.7620576038373 (T) = (0 -1.0798415191886952) / Math.Sqrt((0.14802563331092364 / (299)) + (0.0016109428622093983 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9629536709294091 = (29.1484081224642 - 1.0798415191886952) / 29.1484081224642 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Int16>.MaxBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0785638542827618 < 27.314789686731473.
IsChangePoint: Marked as a change because one of 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 926.6927391830106 (T) = (0 -1.1318635397038808) / Math.Sqrt((0.1761268032119002 / (299)) + (0.0013468856588188872 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9614066157299265 = (29.32791619887974 - 1.1318635397038808) / 29.32791619887974 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Int16>.GreaterThanOrEqualAnyBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 15.479409176155082.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 230.4587501126549 (T) = (0 -0.004672700749923831) / Math.Sqrt((1.9337243775251114 / (299)) + (0.00010917066149169363 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.9997483645026358 = (18.569322686468606 - 0.004672700749923831) / 18.569322686468606 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Int16>.LessThanAllBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 13.857865659428539.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 209.04956493031344 (T) = (0 -0.0002428994092422092) / Math.Sqrt((1.9483588821012925 / (299)) + (2.9500061505107115E-07 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.9999856063781202 = (16.875489106755968 - 0.0002428994092422092) / 16.875489106755968 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Int16>.LessThanBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.169680063759951 < 24.511806560028518.
IsChangePoint: Marked as a change because one of 3/9/2023 2:25:45 AM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 297.4831803356212 (T) = (0 -1.0315870050652) / Math.Sqrt((1.4937662664836013 / (299)) + (0.014690670844448351 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.9625276985873571 = (27.529320756294695 - 1.0315870050652) / 27.529320756294695 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Int16>.LessThanOrEqualAllBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.031117017389331002 < 15.689205713119978.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 188.25223476412162 (T) = (0 -0.07423197865226712) / Math.Sqrt((1.8743277950205994 / (299)) + (0.01056979542339872 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.9960309205771428 = (18.702568213873608 - 0.07423197865226712) / 18.702568213873608 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Int16>.EqualsBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 4.17563676038021.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 60.849179993726025 (T) = (0 -0.08919449809116378) / Math.Sqrt((3.0220821611205793 / (299)) + (0.014876525397764257 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.9873466388975167 = (7.04907552773935 - 0.08919449809116378) / 7.04907552773935 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Int16>.AbsBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0466052367317749 < 19.502175789947653.
IsChangePoint: Marked as a change because one of 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 336.5834538565701 (T) = (0 -1.0529401442014308) / Math.Sqrt((0.23600787478456617 / (299)) + (0.011294484767237016 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9505295503676405 = (21.284224259661514 - 1.0529401442014308) / 21.284224259661514 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Int16>.GreaterThanBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.1645323833774957 < 23.43398927216669.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 111.6965237638202 (T) = (0 -1.2090166396238742) / Math.Sqrt((0.9507345030604225 / (299)) + (0.23072784206340383 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.9535280084252368 = (26.01602812048272 - 1.2090166396238742) / 26.01602812048272 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Int16>.EqualsAnyBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.006088940838078353 < 13.810997896025448.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 191.6521112876766 (T) = (0 -0.03703121341680759) / Math.Sqrt((1.9025939132171357 / (299)) + (0.004908677555085161 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.997787103805296 = (16.734274976580743 - 0.03703121341680759) / 16.734274976580743 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Int16>.GreaterThanAnyBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 13.754119669400334.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 192.82902544403117 (T) = (0 -0.0317314428302135) / Math.Sqrt((1.9381235555147236 / (299)) + (0.0050344223204355385 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.9981020565860459 = (16.71885610335775 - 0.0317314428302135) / 16.71885610335775 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Int16>.GreaterThanOrEqualBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.2403327394563022 < 25.52555908614525.
IsChangePoint: Marked as a change because one of 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 243.69990296157422 (T) = (0 -1.1810573419868744) / Math.Sqrt((0.7665398416346988 / (299)) + (0.048052149280992086 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.957924588795693 = (28.07001305945591 - 1.1810573419868744) / 28.07001305945591 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Int16>.EqualsStaticBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.1499306024038798 < 23.39643778570623.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 302.7045746501268 (T) = (0 -1.0736892994704876) / Math.Sqrt((0.9681770309298324 / (299)) + (0.014224590320937787 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9587434456663713 = (26.024696361889603 - 1.0736892994704876) / 26.024696361889603 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	7a57f6dd800e77508a4ba405e623059935c82f43
Compare	887c043eb94be364188e2b23a87efa214ea57f1e
Diff	Diff
Configs	CompilationMode:tiered, LLVM:false, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Improvements in System.Memory.Span<Char>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
IndexOfAnyFiveValues - Duration of single invocation	783.24 ns	651.32 ns	0.83	0.00	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

Repro Steps

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
AOT MONO build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false
Dotnet SDK installed for dotnet commands
Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create aot directory 
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Memory.Span&lt;Char&gt;*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Memory.Span&lt;Char&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Memory.Span&lt;Char&gt;*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Memory.Span&lt;Char&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200

Payloads

Baseline
Compare

Histogram

System.Memory.Span<Char>.IndexOfAnyFiveValues(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 651.3155600594047 < 745.3859131810973.
IsChangePoint: Marked as a change because one of 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 261.84582345915055 (T) = (0 -651.9158234346651) / Math.Sqrt((10.595326244095041 / (299)) + (0.9413112199902175 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.1728711433284138 = (788.1671860150202 - 651.9158234346651) / 788.1671860150202 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	7a57f6dd800e77508a4ba405e623059935c82f43
Compare	887c043eb94be364188e2b23a87efa214ea57f1e
Diff	Diff
Configs	CompilationMode:tiered, LLVM:false, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Improvements in System.Numerics.Tests.Perf_Plane

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
EqualsBenchmark - Duration of single invocation	21.48 ns	3.40 ns	0.16	0.03	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

Repro Steps

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
AOT MONO build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false
Dotnet SDK installed for dotnet commands
Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create aot directory 
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_Plane*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_Plane* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_Plane*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_Plane* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_Plane.EqualsBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 3.401939759562811 < 20.386133830325402.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 154.64780114898318 (T) = (0 -3.498678461045658) / Math.Sqrt((1.2175423620593036 / (299)) + (0.04890155299779784 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.8494624563717896 = (23.241235220939355 - 3.498678461045658) / 23.241235220939355 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

performanceautofiler · 2023-04-18T16:21:57Z

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	7a57f6dd800e77508a4ba405e623059935c82f43
Compare	887c043eb94be364188e2b23a87efa214ea57f1e
Diff	Diff
Configs	CompilationMode:tiered, LLVM:false, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Improvements in System.Text.Json.Tests.Utf8JsonReaderCommentsTests

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
Utf8JsonReaderCommentParsing - Duration of single invocation	3.75 μs	2.28 μs	0.61	0.01	False
Utf8JsonReaderCommentParsing - Duration of single invocation	3.75 μs	2.29 μs	0.61	0.01	False
Utf8JsonReaderCommentParsing - Duration of single invocation	29.50 μs	19.68 μs	0.67	0.17	False
Utf8JsonReaderCommentParsing - Duration of single invocation	29.51 μs	19.63 μs	0.67	0.16	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

Repro Steps

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
AOT MONO build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false
Dotnet SDK installed for dotnet commands
Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create aot directory 
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Text.Json.Tests.Utf8JsonReaderCommentsTests*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Text.Json.Tests.Utf8JsonReaderCommentsTests* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Text.Json.Tests.Utf8JsonReaderCommentsTests*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Text.Json.Tests.Utf8JsonReaderCommentsTests* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200

Payloads

Baseline
Compare

Histogram

System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Skip, SegmentSize: 0, TestCase: LongSingleLine)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 2.2806045946571207 < 3.5573671774262645.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 1/5/2023 4:05:11 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 480.06739663178985 (T) = (0 -2283.987946687573) / Math.Sqrt((861.6534681443321 / (299)) + (23.06736157291137 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.3820038264345055 = (3695.7962595629547 - 2283.987946687573) / 3695.7962595629547 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Allow, SegmentSize: 0, TestCase: LongSingleLine)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 2.29026479186684 < 3.5627297423538997.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 1/5/2023 4:05:11 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 649.2664732709964 (T) = (0 -2293.301787808956) / Math.Sqrt((989.9493465508192 / (299)) + (5.625853803667572 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.38076956554155195 = (3703.470727847183 - 2293.301787808956) / 3703.470727847183 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Allow, SegmentSize: 0, TestCase: LongMultiLine)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 19.68209605841708 < 28.02333891342374.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/2/2023 11:02:42 PM, 4/11/2023 11:54:05 PM, 4/12/2023 8:39:35 PM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 74.41927775710057 (T) = (0 -19671.066934015424) / Math.Sqrt((9328238.103075325 / (299)) + (243.28317013437592 / (2))) is greater than 1.9679296690655894 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (2) - 2, .975) and 0.4010272191654652 = (32841.33697462543 - 19671.066934015424) / 32841.33697462543 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Skip, SegmentSize: 0, TestCase: LongMultiLine)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 19.62525595238095 < 28.025197024174528.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/5/2023 5:07:43 AM, 4/11/2023 11:54:05 PM, 4/12/2023 8:39:35 PM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 70.27221772143616 (T) = (0 -19614.24301378446) / Math.Sqrt((9477564.96859359 / (299)) + (242.569633060124 / (2))) is greater than 1.9679296690655894 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (2) - 2, .975) and 0.38990108291893255 = (32149.28344345545 - 19614.24301378446) / 32149.28344345545 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	7a57f6dd800e77508a4ba405e623059935c82f43
Compare	887c043eb94be364188e2b23a87efa214ea57f1e
Diff	Diff
Configs	CompilationMode:tiered, LLVM:false, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Improvements in System.Numerics.Tests.Perf_VectorOf<UInt64>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
EqualsBenchmark - Duration of single invocation	3.50 ns	0.00 ns	0.00	0.23	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

Repro Steps

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
AOT MONO build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false
Dotnet SDK installed for dotnet commands
Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create aot directory 
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_VectorOf<UInt64>.EqualsBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 4.396616197717673.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 62.21383824233481 (T) = (0 -0.000783266569323817) / Math.Sqrt((4.191547745737824 / (299)) + (3.0675325931015082E-06 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.9998936799185124 = (7.367061408951287 - 0.000783266569323817) / 7.367061408951287 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	7a57f6dd800e77508a4ba405e623059935c82f43
Compare	887c043eb94be364188e2b23a87efa214ea57f1e
Diff	Diff
Configs	CompilationMode:tiered, LLVM:false, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Improvements in System.Runtime.Intrinsics.Tests.Perf_Vector128

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
FloorDoubleBenchmark - Duration of single invocation	32.17 ns	1.20 ns	0.04	0.49	True
FloorFloatBenchmark - Duration of single invocation	60.26 ns	1.02 ns	0.02	0.45	True
CeilingDoubleBenchmark - Duration of single invocation	32.88 ns	1.08 ns	0.03	0.50	False
CeilingFloatBenchmark - Duration of single invocation	53.49 ns	1.08 ns	0.02	0.51	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

Repro Steps

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
AOT MONO build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false
Dotnet SDK installed for dotnet commands
Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create aot directory 
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Runtime.Intrinsics.Tests.Perf_Vector128* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Runtime.Intrinsics.Tests.Perf_Vector128* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200

Payloads

Baseline
Compare

Histogram

System.Runtime.Intrinsics.Tests.Perf_Vector128.FloorDoubleBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.2012584447524473 < 28.639343667198986.
IsChangePoint: Marked as a change because one of 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 232.96282959185697 (T) = (0 -1.2440875948759973) / Math.Sqrt((3.82059548383908 / (299)) + (0.007449070669262301 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.957729815661349 = (29.4317996086529 - 1.2440875948759973) / 29.4317996086529 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128.FloorFloatBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.022655267011856 < 56.61627810018035.
IsChangePoint: Marked as a change because one of 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 224.1067518260996 (T) = (0 -1.2544917596591265) / Math.Sqrt((14.979708463976158 / (299)) + (0.03703554217988591 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9775401450169919 = (55.8548468192786 - 1.2544917596591265) / 55.8548468192786 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128.CeilingDoubleBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.079745122401462 < 31.05410558687438.
IsChangePoint: Marked as a change because one of 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 226.5169352647464 (T) = (0 -1.1377968463625336) / Math.Sqrt((3.7515355167936195 / (299)) + (0.009204552072626604 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.9613565580657989 = (29.443465421633004 - 1.1377968463625336) / 29.443465421633004 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128.CeilingFloatBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0808289118680303 < 51.2233268679399.
IsChangePoint: Marked as a change because one of 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 280.6493341539086 (T) = (0 -1.186533123692554) / Math.Sqrt((10.440631017153445 / (299)) + (0.011758802439003117 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.9787332934953017 = (55.79298907569095 - 1.186533123692554) / 55.79298907569095 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	7a57f6dd800e77508a4ba405e623059935c82f43
Compare	887c043eb94be364188e2b23a87efa214ea57f1e
Diff	Diff
Configs	CompilationMode:tiered, LLVM:false, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Improvements in System.Text.Tests.Perf_Encoding

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
GetByteCount - Duration of single invocation	1.49 μs	158.32 ns	0.11	0.01	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

Repro Steps

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
AOT MONO build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false
Dotnet SDK installed for dotnet commands
Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create aot directory 
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Text.Tests.Perf_Encoding*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Text.Tests.Perf_Encoding* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Text.Tests.Perf_Encoding*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Text.Tests.Perf_Encoding* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200

Payloads

Baseline
Compare

Histogram

System.Text.Tests.Perf_Encoding.GetByteCount(size: 512, encName: "utf-8")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 158.3231122660187 < 1.411728151841757.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 4/12/2023 6:21:10 AM, 4/18/2023 2:18:01 AM falls between 4/9/2023 1:58:52 PM and 4/18/2023 2:18:01 AM.
IsImprovementStdDev: Marked as improvement because 374.28049103364407 (T) = (0 -158.52829246343367) / Math.Sqrt((2977.229275698812 / (299)) + (5.8654836163573245 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.8886391790045115 = (1423.5553495951326 - 158.52829246343367) / 1423.5553495951326 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

kotlarmilos · 2023-04-19T10:57:10Z

Could be dotnet/runtime#84289.

/cc: @fanyang-mono

performanceautofiler bot added ampere arch-arm64 branch-refs/heads/main kind-micro_mono mono-aot os-linux perf-improvement runtime-mono untriaged labels Apr 18, 2023

kotlarmilos removed the untriaged label Apr 19, 2023

kotlarmilos mentioned this issue Apr 20, 2023

.NET 8 Per-Preview Performance report on WASM, Mono AOT, and Interpreter dotnet/runtime#84302

Closed

kotlarmilos closed this as completed Apr 26, 2023

[Perf] Linux/arm64: 373 Improvements on 4/12/2023 2:59:13 PM #16460

[Perf] Linux/arm64: 373 Improvements on 4/12/2023 2:59:13 PM #16460

Comments

performanceautofiler bot commented Apr 18, 2023 • edited Loading

Run Information

Improvements in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>

Repro

Payloads

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

Payloads

Histogram

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>.GreaterThanOrEqualAllBenchmark

Description of detection logic

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>.LessThanOrEqualBenchmark

Description of detection logic

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>.MaxBenchmark

Description of detection logic

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>.GreaterThanBenchmark

Description of detection logic

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>.AndNotBenchmark

Description of detection logic

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>.GreaterThanAllBenchmark

Description of detection logic

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>.ConditionalSelectBenchmark

Description of detection logic

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>.GreaterThanOrEqualBenchmark

Description of detection logic

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>.GreaterThanOrEqualAnyBenchmark

Description of detection logic

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>.LessThanAllBenchmark

Description of detection logic

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>.GreaterThanAnyBenchmark

Description of detection logic

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>.MinBenchmark

Description of detection logic

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>.EqualsBenchmark

Description of detection logic

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>.EqualsAnyBenchmark

Description of detection logic

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>.LessThanOrEqualAnyBenchmark

Description of detection logic

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>.EqualsStaticBenchmark

Description of detection logic

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>.LessThanBenchmark

Description of detection logic

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>.LessThanOrEqualAllBenchmark

Description of detection logic

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>.LessThanAnyBenchmark

Description of detection logic

Docs

performanceautofiler bot commented Apr 18, 2023 • edited Loading

Run Information

Improvements in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Single>

Repro

Payloads

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

Payloads

Histogram

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Single>.GreaterThanBenchmark

Description of detection logic

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Single>.SquareRootBenchmark

Description of detection logic

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Single>.EqualsBenchmark

Description of detection logic

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Single>.AndNotBenchmark

Description of detection logic

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Single>.LessThanAnyBenchmark

Description of detection logic

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Single>.GreaterThanOrEqualBenchmark

Description of detection logic

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Single>.LessThanOrEqualBenchmark

Description of detection logic

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Single>.MaxBenchmark

Description of detection logic

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Single>.LessThanBenchmark

Description of detection logic

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Single>.SumBenchmark

Description of detection logic

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Single>.GreaterThanAllBenchmark

Description of detection logic

performanceautofiler bot commented Apr 18, 2023 •

edited

Loading

performanceautofiler bot commented Apr 18, 2023 •

edited

Loading

performanceautofiler bot commented Apr 18, 2023 •

edited

Loading