Add glob filters support to disassembler to allow disassembling specific methods

[adamsitnik]

So far, the disassembler was always loading the type that was generated by BDN, searching for the benchmark method, disassembling it and when encountered direct method calls, disassembling the called methods as well (if their depth was <= configured depth).

This was working fine, but only for direct method calls. For indirect, the disassembly was incomplete.

When I was working on dotnet/runtime#73064 it was really a PITA to me, and I tried to come up with a workaround for it. So here it is: a possibility to specify a glob pattern(s) for methods that we want to be disassembled.

How it works:

• ClrMD is used to enumerate over all modules
• for each module, all types are enumerated
• for each type, every method that can be disassembled and its full signature matches one of the provided filters is getting disassembled
• recursive disassembling is disabled for filtered methods (if A is filtered and it calls B, but B does not satisfy the filters, B is not decompiled). I could change that easily, I just need a good use case example.

Sample results for the Perf_Encoding.GetBytes benchmark:

Before:

; System.Text.Tests.Perf_Encoding.GetBytes()
       mov       [rsp+8],rcx
       mov       rax,[rcx+10]
       mov       rcx,rax
       mov       rdx,[rsp+8]
       mov       rdx,[rdx+18]
       mov       rax,[rax]
       mov       rax,[rax+58]
       jmp       qword ptr [rax+10]
; Total bytes of code 32

After:

dotnet run -c Release -f net7.0 --filter "*Perf_Encoding.GetBytes" --disasm --disasmFilter *ASCIIUtility.NarrowUtf16ToAscii_Intrinsified*

; System.Text.ASCIIUtility.NarrowUtf16ToAscii_Intrinsified(Char*, Byte*, UIntPtr)
       sub       rsp,48
       movdqu    xmm0,[rcx]
       movups    xmm1,[System.Reflection.CustomAttributeExtensions.GetCustomAttribute[[System.__Canon, System.Private.CoreLib]](System.Reflection.Assembly)]
       ptest     xmm0,xmm1
       jne       near ptr M00_L04
       mov       r9,rdx
       movaps    xmm2,xmm0
       packuswb  xmm2,xmm0
       movaps    [rsp+20],xmm2
       mov       rax,[rsp+20]
       mov       [rsp+40],rax
       mov       rax,[rsp+40]
       mov       [r9],rax
       mov       eax,8
       test      dl,8
       jne       short M00_L00
       movdqu    xmm0,[rcx+10]
       ptest     xmm0,xmm1
       jne       short M00_L02
       movaps    xmm2,xmm0
       packuswb  xmm2,xmm0
       movaps    [rsp+10],xmm2
       mov       rax,[rsp+10]
       mov       [rsp+30],rax
       mov       rax,[rsp+30]
       lea       r10,[r9+8]
       mov       [r10],rax
M00_L00:
       and       rdx,0F
       mov       eax,10
       sub       rax,rdx
       sub       r8,10
M00_L01:
       movdqu    xmm0,[rcx+rax*2]
       movdqu    xmm2,[rcx+rax*2+10]
       movaps    xmm3,xmm0
       por       xmm3,xmm2
       ptest     xmm3,xmm1
       jne       short M00_L03
       packuswb  xmm0,xmm2
       movaps    xmm2,xmm0
       movdqu    [r9+rax],xmm2
       add       rax,10
       cmp       rax,r8
       jbe       short M00_L01
M00_L02:
       add       rsp,48
       ret
M00_L03:
       ptest     xmm0,xmm1
       jne       short M00_L02
       movaps    xmm2,xmm0
       packuswb  xmm2,xmm0
       movaps    [rsp],xmm2
       mov       rdx,[rsp]
       mov       [rsp+38],rdx
       mov       rdx,[rsp+38]
       add       r9,rax
       mov       [r9],rdx
       add       rax,8
       jmp       short M00_L02
M00_L04:
       xor       eax,eax
       add       rsp,48
       ret
       int       3
       int       3
       int       3
       int       3
       int       3
       int       3
       movdqu    xmm0,[rcx]
       pmovmskb  eax,xmm0
       test      eax,eax
       jne       short 00007FFAFCDBE1E4
       xorps     xmm1,xmm1
       punpcklbw xmm0,xmm1
       (bad)
; Total bytes of code 261

cc @kunalspathak @AndyAyersMS @EgorBo @janvorli @stephentoub

[janvorli]

Awesome!

[kunalspathak]

This indeed is valuable. Thanks for taking time to do it.

[kunalspathak]

By the way, I assume it will also work if I want to dump every single method's disassembly using --disasmFilter *.

[adamsitnik]

By the way, I assume it will also work if I want to dump every single method's disassembly using --disasmFilter *

Yes, exactly. But you are going to need a text editor that can open a HUGE file ;)