Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Replay Binary Log Tool #72619

Merged
merged 10 commits into from
Mar 21, 2024
Merged

Replay Binary Log Tool #72619

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
d2448f6
Replay Binary Log Tool
jaredpar Mar 19, 2024
c1d2b2e
fixes
jaredpar Mar 20, 2024
76191df
Apply suggestions from code review
jaredpar Mar 20, 2024
15a8408
PR feedback
jaredpar Mar 20, 2024
5373ee9
PR feedback
jaredpar Mar 20, 2024
01ec280
Apply suggestions from code review
jaredpar Mar 20, 2024
93335cd
Merge remote-tracking branch 'origin/main' into replay
jaredpar Mar 21, 2024
546b39b
Merge remote-tracking branch 'jaredpar/replay' into replay
jaredpar Mar 21, 2024
711858b
Apply suggestions from code review
jaredpar Mar 21, 2024
dd6af01
PR feedback
jaredpar Mar 21, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions .vscode/launch.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,18 @@
"stopAtEntry": false,
"console": "internalConsole"
},
{
"name": "Launch Replay.dll",
"type": "coreclr",
"request": "launch",
"preLaunchTask": "build current project",
// If you have changed target frameworks, make sure to update the program path.
"program": "${workspaceFolder}/artifacts/bin/Replay/Debug/net8.0/Replay.dll",
"args": [],
"cwd": "${workspaceFolder}/artifacts/bin/Replay",
"stopAtEntry": false,
"console": "internalConsole"
},
{
"name": ".NET Core Launch (console)",
"type": "coreclr",
Expand Down
11 changes: 1 addition & 10 deletions .vscode/tasks.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -73,22 +73,13 @@
"command": "dotnet",
"args": [
"pwsh",
"-noprofile",
"${workspaceFolder}/scripts/vscode-build.ps1",
"-filePath",
"${file}",
"-msbuildEngine",
"dotnet"
],
"windows": {
"command": "powershell",
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is no need for this anymore as part of our repo setup includes installing pwsh as a local tool.

"args": [
"${workspaceFolder}/scripts/vscode-build.ps1",
"-filePath",
"${file}",
"-msbuildEngine",
"dotnet"
],
},
"problemMatcher": "$msCompile",
"group": "build"
},
Expand Down
1 change: 1 addition & 0 deletions Compilers.slnf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,7 @@
"src\\Scripting\\VisualBasic\\Microsoft.CodeAnalysis.VisualBasic.Scripting.vbproj",
"src\\Test\\PdbUtilities\\Roslyn.Test.PdbUtilities.csproj",
"src\\Tools\\BuildBoss\\BuildBoss.csproj",
"src\\Tools\\Replay\\Replay.csproj",
"src\\Tools\\BuildValidator\\BuildValidator.csproj",
"src\\Tools\\PrepareTests\\PrepareTests.csproj",
"src\\Tools\\Source\\CompilerGeneratorTools\\Source\\BoundTreeGenerator\\CompilersBoundTreeGenerator.csproj",
Expand Down
12 changes: 12 additions & 0 deletions Roslyn.sln
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -544,6 +544,12 @@ Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Microsoft.CodeAnalysis.Feat
EndProject
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Microsoft.CodeAnalysis.Features.Test.Utilities", "src\Features\TestUtilities\Microsoft.CodeAnalysis.Features.Test.Utilities.csproj", "{5762E483-75CE-4328-A410-511F30737712}"
EndProject
Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "src", "src", "{0F3118AE-8D36-4384-8E80-BD6566365305}"
EndProject
Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "Tools", "Tools", "{47D004BE-F797-430E-8A18-4B0CDFD56643}"
EndProject
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Replay", "src\Tools\Replay\Replay.csproj", "{DB96C25F-39A9-4A6A-92BC-D1E42717308F}"
EndProject
Project("{D954291E-2A0B-460D-934E-DC6B0785DB48}") = "Microsoft.CommonLanguageServerProtocol.Framework.Shared", "src\Features\LanguageServer\Microsoft.CommonLanguageServerProtocol.Framework\Microsoft.CommonLanguageServerProtocol.Framework.Shared.shproj", "{64EADED3-4B5D-4431-BBE5-A4ABA1C38C00}"
EndProject
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Microsoft.CommonLanguageServerProtocol.Framework.Binary", "src\Features\LanguageServer\Microsoft.CommonLanguageServerProtocol.Framework.Binary\Microsoft.CommonLanguageServerProtocol.Framework.Binary.csproj", "{730CADBA-701F-4722-9B6F-1FCC0DF2C95D}"
Expand Down Expand Up @@ -1350,6 +1356,10 @@ Global
{5762E483-75CE-4328-A410-511F30737712}.Debug|Any CPU.Build.0 = Debug|Any CPU
{5762E483-75CE-4328-A410-511F30737712}.Release|Any CPU.ActiveCfg = Release|Any CPU
{5762E483-75CE-4328-A410-511F30737712}.Release|Any CPU.Build.0 = Release|Any CPU
{DB96C25F-39A9-4A6A-92BC-D1E42717308F}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{DB96C25F-39A9-4A6A-92BC-D1E42717308F}.Debug|Any CPU.Build.0 = Debug|Any CPU
{DB96C25F-39A9-4A6A-92BC-D1E42717308F}.Release|Any CPU.ActiveCfg = Release|Any CPU
{DB96C25F-39A9-4A6A-92BC-D1E42717308F}.Release|Any CPU.Build.0 = Release|Any CPU
{730CADBA-701F-4722-9B6F-1FCC0DF2C95D}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{730CADBA-701F-4722-9B6F-1FCC0DF2C95D}.Debug|Any CPU.Build.0 = Debug|Any CPU
{730CADBA-701F-4722-9B6F-1FCC0DF2C95D}.Release|Any CPU.ActiveCfg = Release|Any CPU
Expand Down Expand Up @@ -1603,6 +1613,8 @@ Global
{4D9D7A28-BB44-4F3F-81DA-14F39B853718} = {CC126D03-7EAC-493F-B187-DCDEE1EF6A70}
{5BABC440-4F1B-46E8-9068-DD7F02ED25D3} = {3E5FE3DB-45F7-4D83-9097-8F05D3B3AEC6}
{5762E483-75CE-4328-A410-511F30737712} = {3E5FE3DB-45F7-4D83-9097-8F05D3B3AEC6}
{47D004BE-F797-430E-8A18-4B0CDFD56643} = {0F3118AE-8D36-4384-8E80-BD6566365305}
{DB96C25F-39A9-4A6A-92BC-D1E42717308F} = {47D004BE-F797-430E-8A18-4B0CDFD56643}
{64EADED3-4B5D-4431-BBE5-A4ABA1C38C00} = {D449D505-CC6A-4E0B-AF1B-976E2D0AE67A}
{730CADBA-701F-4722-9B6F-1FCC0DF2C95D} = {D449D505-CC6A-4E0B-AF1B-976E2D0AE67A}
EndGlobalSection
Expand Down
3 changes: 2 additions & 1 deletion eng/Directory.Packages.props
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -291,6 +291,7 @@
<PackageVersion Include="runtime.win-x64.Microsoft.NETCore.ILAsm" Version="$(ILAsmPackageVersion)" />
<PackageVersion Include="runtime.linux-x64.Microsoft.NETCore.ILAsm" Version="$(ILAsmPackageVersion)" />
<PackageVersion Include="runtime.osx-x64.Microsoft.NETCore.ILAsm" Version="$(ILAsmPackageVersion)" />
<PackageVersion Include="Basic.CompilerLog.Util" Version="0.6.1" />
<PackageVersion Include="Basic.Reference.Assemblies.NetStandard20" Version="1.2.4" />
<PackageVersion Include="Basic.Reference.Assemblies.Net50" Version="1.2.4" />
<PackageVersion Include="Basic.Reference.Assemblies.Net60" Version="1.2.4" />
Expand All @@ -310,7 +311,7 @@
Infra
-->
<PackageVersion Include="Microsoft.Win32.Registry" Version="5.0.0" />
<PackageVersion Include="MSBuild.StructuredLogger" Version="2.2.158" />
<PackageVersion Include="MSBuild.StructuredLogger" Version="2.2.169" />
<PackageVersion Include="Mono.Options" Version="6.6.0.161" />
<PackageVersion Include="RichCodeNav.EnvVarDump" Version="0.1.1643-alpha" />
<PackageVersion Include="Roslyn.Diagnostics.Analyzers" Version="$(RoslynDiagnosticsNugetPackageVersion)" />
Expand Down
15 changes: 9 additions & 6 deletions src/Compilers/Core/CommandLine/CompilerServerLogger.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -108,26 +108,29 @@ internal sealed class CompilerServerLogger : ICompilerServerLogger, IDisposable
/// <summary>
/// Static class initializer that initializes logging.
/// </summary>
public CompilerServerLogger(string identifier)
public CompilerServerLogger(string identifier, string? loggingFilePath = null)
{
_identifier = identifier;

try
{
// Check if the environment
if (Environment.GetEnvironmentVariable(EnvironmentVariableName) is string loggingFileName)
if (loggingFilePath is null)
{
loggingFilePath = Environment.GetEnvironmentVariable(EnvironmentVariableName);
// If the environment variable contains the path of a currently existing directory,
// then use a process-specific name for the log file and put it in that directory.
// Otherwise, assume that the environment variable specifies the name of the log file.
if (Directory.Exists(loggingFileName))
if (Directory.Exists(loggingFilePath))
{
var processId = Process.GetCurrentProcess().Id;
loggingFileName = Path.Combine(loggingFileName, $"server.{processId}.log");
loggingFilePath = Path.Combine(loggingFilePath, $"server.{processId}.log");
}
}

if (loggingFilePath is not null)
{
// Open allowing sharing. We allow multiple processes to log to the same file, so we use share mode to allow that.
_loggingStream = new FileStream(loggingFileName, FileMode.OpenOrCreate, FileAccess.Write, FileShare.ReadWrite);
_loggingStream = new FileStream(loggingFilePath, FileMode.OpenOrCreate, FileAccess.Write, FileShare.ReadWrite);
}
}
catch (Exception e)
Expand Down
2 changes: 1 addition & 1 deletion src/Compilers/Shared/BuildServerConnection.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -446,7 +446,7 @@ internal static (string processFilePath, string commandLineArguments, string too
/// compiler server process was successful, it does not state whether the server successfully
/// started or not (it could crash on startup).
/// </summary>
private static bool TryCreateServer(string clientDirectory, string pipeName, ICompilerServerLogger logger)
internal static bool TryCreateServer(string clientDirectory, string pipeName, ICompilerServerLogger logger)
{
var serverInfo = GetServerProcessInfo(clientDirectory, pipeName);

Expand Down
26 changes: 26 additions & 0 deletions src/Tools/Replay/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
# Replay

This is a tool for replaying the compilation events from a [binary log][binary-log] directly into the compiler server. This is very useful for profiling the compiler as it removes the overhead of msbuild from the logs. This gives a much simpler view of the compiler's performance.

The other advantage is the tool allows for easy testing of performance changes to the compiler. Developers can have two branches of the code, one with performance changes and one without, then run `replay` against the same binary log and analyze the performance differences.

Note: this is only supported for replaying binary logs on the same machine they were built.

## Usage

The first step is to run a build locally and collect a binary log. For example:

```cmd
e:\code\example> dotnet msbuild -bl -v:m -m Example.sln
```

Then run the replay tool against that log.

```cmd
e:\code\roslyn> cd src\Tools\Replay
e:\code\roslyn\src\Tools\Replay> dotnet run --framework net472 e:\code\example\msbuild.binlog -o e:\temp
```

This runs all of the compilation events in the binary log against the compiler and outputs the results to `e:\temp`.

[binary-log]: https://github.com/dotnet/msbuild/blob/main/documentation/wiki/Binary-Log.md
Loading
Loading