Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Perf] Linux/x64: 11 Regressions on 9/12/2023 4:19:08 AM #21829

Closed
performanceautofiler bot opened this issue Sep 19, 2023 · 1 comment
Closed

[Perf] Linux/x64: 11 Regressions on 9/12/2023 4:19:08 AM #21829

performanceautofiler bot opened this issue Sep 19, 2023 · 1 comment
Labels
arch-x64 branch-refs/heads/main kind-micro_mono mono-interpreter os-linux perf-regression PGO Applied if there were any profile guided optimization updates in the observed interval. runtime-mono

Comments

@performanceautofiler
Copy link

performanceautofiler bot commented Sep 19, 2023 •
edited
Loading

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline da4e544809b3b10b7db8dc170331988d817240d7
Compare f6c592995a4f8526508da33761230a3850d942ff
Diff Diff
Configs CompilationMode:tiered, LLVM:false, MonoAOT:false, MonoInterpreter:true, RunKind:micro_mono

Regressions in System.Tests.Perf_Uri

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
17.63 μs 70.61 μs 4.01 0.04 True
131.98 μs 141.27 μs 1.07 0.06 True
4.79 μs 5.11 μs 1.06 0.02 True
16.37 μs 22.21 μs 1.36 0.01 True
10.47 μs 11.12 μs 1.06 0.00 True
2.48 μs 5.28 μs 2.13 0.10 True
11.92 μs 12.66 μs 1.06 0.00 True
25.54 μs 30.88 μs 1.21 0.01 True
7.08 μs 7.58 μs 1.07 0.00 True
28.71 μs 34.33 μs 1.20 0.01 True
209.07 μs 316.59 μs 1.51 0.04 True

graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps

Prerequisites (Build files either built locally or downloaded from payload above)

  • Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
  • CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
  • Mono Runtime build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release
  • Dotnet SDK installed for dotnet commands
  • Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create mono dotnet
mkdir -p $RunDir/artifacts/dotnet-mono
$RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir/artifacts/bin/mono/$RunOS.$RunArch.Release /p:RuntimeFlavor=mono
cp $RunDir/artifacts/bin/runtime/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/shared/Microsoft.NETCore.App/8.0.0 -rf
cp $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/dotnet-mono -r
cp $RunDir/artifacts/bin/coreclr/$RunOS.$RunArch.Release/corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# Use Interpreter
export MONO_ENV_OPTIONS="--interpreter"

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Tests.Perf_Uri*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun" 

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Tests.Perf_Uri*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create mono dotnet
mkdir -p $RunDir/artifacts/dotnet-mono
$RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir\artifacts\bin\mono\$RunOS.$RunArch.Release /p:RuntimeFlavor=mono
xcopy $RunDir\artifacts\bin\runtime\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\shared\Microsoft.NETCore.App\8.0.0\ /e /y
xcopy $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\dotnet-mono\ /e /y
xcopy $RunDir\artifacts\bin\coreclr\$RunOS.$RunArch.Release\corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# Use Interpreter
export MONO_ENV_OPTIONS="--interpreter"

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Tests.Perf_Uri*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe" 

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Tests.Perf_Uri*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages

Payloads

Baseline
Compare

System.Tests.Perf_Uri.EscapeDataString(input: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 70.61386764649924 > 18.440939331585025.
IsChangePoint: Marked as a change because one of 9/11/2023 3:39:38 PM, 9/18/2023 8:11:22 PM falls between 9/9/2023 9:29:58 PM and 9/18/2023 8:11:22 PM.
IsRegressionStdDev: Marked as regression because -158.03014694841823 (T) = (0 -71352.70818352197) / Math.Sqrt((1561244.2139485595 / (17)) + (596157.6766419201 / (26))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (26) - 2, .025) and -3.0048922083945944 = (17816.38667676514 - 71352.70818352197) / 17816.38667676514 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Uri.EscapeDataString(input: "{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 141.2678237802934 > 132.03920562258833.
IsChangePoint: Marked as a change because one of 9/11/2023 10:38:32 AM, 9/18/2023 8:11:22 PM falls between 9/9/2023 9:29:58 PM and 9/18/2023 8:11:22 PM.
IsRegressionStdDev: Marked as regression because -14.789468205332215 (T) = (0 -141162.06022774064) / Math.Sqrt((16833372.284598317 / (16)) + (3772618.1986594475 / (26))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (26) - 2, .025) and -0.1294770784412192 = (124980.0132487479 - 141162.06022774064) / 124980.0132487479 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Uri.CombineAbsoluteRelative(input: "/new/path")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.105540256556645 > 4.95621730015081.
IsChangePoint: Marked as a change because one of 9/11/2023 3:39:38 PM, 9/18/2023 8:11:22 PM falls between 9/9/2023 9:29:58 PM and 9/18/2023 8:11:22 PM.
IsRegressionStdDev: Marked as regression because -36.49520252835809 (T) = (0 -5108.581261099339) / Math.Sqrt((1146.8306034488596 / (17)) + (1122.9198688803572 / (26))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (26) - 2, .025) and -0.08125275854132581 = (4724.6873783620385 - 5108.581261099339) / 4724.6873783620385 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Uri.CtorIdnHostPathAndQuery(input: "http://host/path with escapable values?key=va lue")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 22.21438292857143 > 17.202874159849475.
IsChangePoint: Marked as a change because one of 9/11/2023 8:56:48 PM, 9/18/2023 8:11:22 PM falls between 9/9/2023 9:29:58 PM and 9/18/2023 8:11:22 PM.
IsRegressionStdDev: Marked as regression because -183.36248533179247 (T) = (0 -22137.899304948663) / Math.Sqrt((3477.394688024145 / (18)) + (19672.443600074763 / (25))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (25) - 2, .025) and -0.35007831331960454 = (16397.492713230444 - 22137.899304948663) / 16397.492713230444 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Uri.CtorIdnHostPathAndQuery(input: "http://dot.net/path/with?key=value#fragment")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 11.119188135938135 > 10.949510755497615.
IsChangePoint: Marked as a change because one of 9/11/2023 8:56:48 PM, 9/18/2023 8:11:22 PM falls between 9/9/2023 9:29:58 PM and 9/18/2023 8:11:22 PM.
IsRegressionStdDev: Marked as regression because -29.878130791947438 (T) = (0 -11010.223948807747) / Math.Sqrt((2675.313095033867 / (18)) + (4178.452003689573 / (25))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (25) - 2, .025) and -0.050664580200091416 = (10479.294873261008 - 11010.223948807747) / 10479.294873261008 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Uri.BuilderToString

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.276887079166666 > 2.611027267487548.
IsChangePoint: Marked as a change because one of 9/11/2023 3:39:38 PM, 9/18/2023 8:11:22 PM falls between 9/9/2023 9:29:58 PM and 9/18/2023 8:11:22 PM.
IsRegressionStdDev: Marked as regression because -144.59711610209897 (T) = (0 -5221.614992111014) / Math.Sqrt((2243.9325150864133 / (17)) + (5644.260651686708 / (25))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (25) - 2, .025) and -1.099900031268685 = (2486.601702156411 - 5221.614992111014) / 2486.601702156411 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Uri.CtorIdnHostPathAndQuery(input: "https://CONTOSO.com/path/with?key=value#fragment")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.658240793392906 > 12.53741725629743.
IsChangePoint: Marked as a change because one of 9/11/2023 8:56:48 PM, 9/18/2023 8:11:22 PM falls between 9/9/2023 9:29:58 PM and 9/18/2023 8:11:22 PM.
IsRegressionStdDev: Marked as regression because -31.980050777980804 (T) = (0 -12676.890741774045) / Math.Sqrt((4158.469616663824 / (18)) + (5305.654719076681 / (25))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (25) - 2, .025) and -0.05609100667298037 = (12003.5969075338 - 12676.890741774045) / 12003.5969075338 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Uri.CtorIdnHostPathAndQuery(input: "http://host/path with escapable values?key=ünicode")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 30.883994059405943 > 26.83083251413423.
IsChangePoint: Marked as a change because one of 9/11/2023 3:39:38 PM, 9/18/2023 8:11:22 PM falls between 9/9/2023 9:29:58 PM and 9/18/2023 8:11:22 PM.
IsRegressionStdDev: Marked as regression because -93.38121916869697 (T) = (0 -30996.11475497115) / Math.Sqrt((30746.803472201613 / (17)) + (43479.473448862685 / (25))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (25) - 2, .025) and -0.2186881901893024 = (25433.999446697217 - 30996.11475497115) / 25433.999446697217 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Uri.CtorIdnHostPathAndQuery(input: "http://host/")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.579945436700664 > 7.423652393262791.
IsChangePoint: Marked as a change because one of 9/11/2023 3:39:38 PM, 9/18/2023 8:11:22 PM falls between 9/9/2023 9:29:58 PM and 9/18/2023 8:11:22 PM.
IsRegressionStdDev: Marked as regression because -24.64383248130947 (T) = (0 -7543.059833729839) / Math.Sqrt((2233.6044140940644 / (17)) + (4658.8928239382485 / (25))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (25) - 2, .025) and -0.06183835455530363 = (7103.774130374921 - 7543.059833729839) / 7103.774130374921 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Uri.CtorIdnHostPathAndQuery(input: "http://host/path%20with%20escapable%20values?key=%C3%BCnicode")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 34.325393433370046 > 30.323239088541666.
IsChangePoint: Marked as a change because one of 9/11/2023 8:56:48 PM, 9/18/2023 8:11:22 PM falls between 9/9/2023 9:29:58 PM and 9/18/2023 8:11:22 PM.
IsRegressionStdDev: Marked as regression because -105.97116433424765 (T) = (0 -34233.93773294595) / Math.Sqrt((26776.3150814091 / (18)) + (30434.245762714374 / (27))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (27) - 2, .025) and -0.18805450492638653 = (28815.123877727423 - 34233.93773294595) / 28815.123877727423 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Uri.EscapeDataString(input: "a{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{ü")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 316.5886020408163 > 221.12695625260417.
IsChangePoint: Marked as a change because one of 9/11/2023 3:39:38 PM, 9/18/2023 8:11:22 PM falls between 9/9/2023 9:29:58 PM and 9/18/2023 8:11:22 PM.
IsRegressionStdDev: Marked as regression because -99.14804994213226 (T) = (0 -321144.7778448902) / Math.Sqrt((12019863.282221733 / (17)) + (13480135.772048721 / (26))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (26) - 2, .025) and -0.5192424043789994 = (211384.81714256803 - 321144.7778448902) / 211384.81714256803 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
arch-x64 branch-refs/heads/main kind-micro_mono mono-interpreter os-linux perf-regression PGO Applied if there were any profile guided optimization updates in the observed interval. runtime-mono
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@kotlarmilos