Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

jit-analyze issues on Unix #275

Open
am11 opened this issue Jul 23, 2020 · 1 comment
Open

jit-analyze issues on Unix #275

am11 opened this issue Jul 23, 2020 · 1 comment

Comments

@am11
Copy link
Member

am11 commented Jul 23, 2020

There seems to be few issues, which I think are worth the investigation for Unix.

Setup on macOS x64:

$ pushd runtime_base; ./build.sh -c Release; ./src/coreclr/build-test.sh -generatelayoutonly -release; popd
$ pushd runtime_pr;   ./build.sh -c Release; ./src/coreclr/build-test.sh -generatelayoutonly -release; popd

$ mkdir /tmp/diffs

$ jitutils/bin/jit-diff diff                                                         \
    --base runtime_base/artifacts/bin/coreclr/OSX.x64.Release                        \
    --base_root runtime_base                                                         \
    --core_root runtime_base/artifacts/tests/coreclr/OSX.x64.Release/Tests/Core_Root \
    --output /tmp/diffs --pmi -f

...
Diffs (if any) can be viewed by comparing: /tmp/diffs/dasmset_1/base /tmp/diffs/dasmset_1/diff
...

$ jitutils/bin/jit-diff diff                                                       \
    --diff runtime_pr/artifacts/bin/coreclr/OSX.x64.Release                        \
    --diff_root runtime_pr                                                         \
    --core_root runtime_pr/artifacts/tests/coreclr/OSX.x64.Release/Tests/Core_Root \
    --output /tmp/diffs --pmi -f

...
Diffs (if any) can be viewed by comparing: /tmp/diffs/dasmset_2/base /tmp/diffs/dasmset_2/diff
...
  1. allow symlinks.

    $ ln -s /tmp/diffs/dasmset_2/diff/ /tmp/diffs/diff
    $ ln -s /tmp/diffs/dasmset_1/base/ /tmp/diffs/base
    $ jitutils/bin/jit-analyze --base /tmp/diffs/base --diff /tmp/diffs/diff
    Couldn't parse --numstat output '1	1	/tmp/diffs/{diff => base}` : '/tmp/diffs/base' does not exist

    looks like jit-analyze is strict about directory type and does not accept link-to-directory.

  2. allow sub directories:

    $ jitutils/bin/jit-analyze --base /tmp/diffs/dasmset_1/base --diff /tmp/diffs/dasmset_2/diff

    currently it fails with these errors:

    Couldn't parse --numstat output '1	1	/tmp/diffs/{dasmset_2/diff => dasmset_1/base}/CommandLine.dasm` : '/tmp/diffs/{dasmset_2/diff' does not exist
    Couldn't parse --numstat output '1	1	/tmp/diffs/{dasmset_2/diff => dasmset_1/base}/Dia2Lib.dasm` : '/tmp/diffs/{dasmset_2/diff' does not exist
    Couldn't parse --numstat output '1	1	/tmp/diffs/{dasmset_2/diff => dasmset_1/base}/ILCompiler.Reflection.ReadyToRun.dasm` : '/tmp/diffs/{dasmset_2/diff' does not exist
    Couldn't parse --numstat output '1	1	/tmp/diffs/{dasmset_2/diff => dasmset_1/base}/Microsoft.CSharp.dasm` : '/tmp/diffs/{dasmset_2/diff' does not exist
    Couldn't parse --numstat output '1	1	/tmp/diffs/{dasmset_2/diff => dasmset_1/base}/Microsoft.CodeAnalysis.CSharp.dasm` : '/tmp/diffs/{dasmset_2/diff' does not exist
    Couldn't parse --numstat output '1	1	/tmp/diffs/{dasmset_2/diff => dasmset_1/base}/Microsoft.CodeAnalysis.VisualBasic.dasm` : '/tmp/diffs/{dasmset_2/diff' does not exist
    ...

    from @AndyAyersMS:

    jit-analyze uses git diff under the covers and sometimes git diff decides there are renames and messes up our parsing.

  3. crossgen2 does not produce executable on Unix:

    and crossgen2.dll fails:

    $ jitutils/bin/jit-diff diff --diff runtime_base/artifacts/bin/coreclr/OSX.x64.Release --diff_root runtime_base --core_root runtime_base/artifacts/tests/coreclr/OSX.x64.Release/Tests/Core_Root --output /tmp/diffs --crossgen runtime_base/artifacts/bin/coreclr/OSX.x64.Release/crossgen2/crossgen2.dll -f
    Beginning Crossgen CodeSize Diffs for System.Private.CoreLib.dll, framework assemblies
    - Finished 0/0 Base 0/265 Diff [0.1 sec]Unhandled exception. System.ComponentModel.Win32Exception (13): Permission denied
       at System.Diagnostics.Process.ForkAndExecProcess(String filename, String[] argv, String[] envp, String cwd, Boolean redirectStdin, Boolean redirectStdout, Boolean redirectStderr, Boolean setCredentials, UInt32 userId, UInt32 groupId, UInt32[] groups, Int32& stdinFd, Int32& stdoutFd, Int32& stderrFd, Boolean usesTerminal, Boolean throwOnNoExec)
       at System.Diagnostics.Process.StartCore(ProcessStartInfo startInfo)
       at System.Diagnostics.Process.Start()
       at Microsoft.DotNet.Cli.Utils.Command.Execute()

    i think we can either produce crossgen2 executable on Unix in dotnet/runtime, or accept the base path of crossgen2, then invoke with dotnet as a process name.

Originally posted by @am11 in dotnet/runtime#39730

@am11 am11 changed the title Does jit-diff work on macOS? I tried: jit-analyze issues on Unix Jul 23, 2020
@am11
Copy link
Member Author

am11 commented Jul 24, 2020

Some .log files under base and diff directories were showing:

DllNotFoundException Interop+HostPolicy::corehost_resolve_component_dependencies (Unable to load shared library 'libhostpolicy' or one of its dependencies. In order to help diagnose loading problems, consider setting the DYLD_PRINT_LIBRARIES environment variable: dlopen(liblibhostpolicy, 1): image not found)
DllNotFoundException Interop+HostPolicy::corehost_set_error_writer (Unable to load shared library 'libhostpolicy' or one of its dependencies. In order to help diagnose loading problems, consider setting the DYLD_PRINT_LIBRARIES environment variable: dlopen(liblibhostpolicy, 1): image not found)
Completed assembly System.Private.CoreLib - #types: 2643, #methods: 35706, skipped types: 25, skipped methods: 505

it required an additional copy step (in both runtime_base and runtime_pr) to fix:

cp artifacts//bin/osx-x64.Release/corehost/libhostpolicy.dylib artifacts/tests/coreclr/OSX.x64.Release/Tests/Core_Root/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant