-
Notifications
You must be signed in to change notification settings - Fork 4.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Hide 'align' instruction behind jmp #60787
Merged
kunalspathak
merged 24 commits into
dotnet:main
from
kunalspathak:hide-align-behind-jmp
Nov 18, 2021
Merged
Changes from 1 commit
Commits
Show all changes
24 commits
Select commit
Hold shift + click to select a range
3244072
Hide align behind a jmp
kunalspathak e7c0710
Fix a problem where curIG==0 and loop might be emitted in curIG, adju…
kunalspathak bd922aa
Add stress mode to emit int3 for xarch
kunalspathak 4d0f912
Add stress mode to emit bkpt for arm64
kunalspathak 8d64351
Add a loop align instruction placement phase
kunalspathak 9b9b616
review comments
kunalspathak 6302975
Change from unsigned short to unsigned
kunalspathak d20da6d
review comments around cleanup
kunalspathak c6a2d70
emitForceNewIG
kunalspathak e9c5eec
Remove emitPrevIG
kunalspathak c1c5db3
Revert change to forceNewIG for align instruction
kunalspathak b8a9742
Use loopAlignCandidates
kunalspathak db98ec2
Use loopHeadIG reference
kunalspathak 5ab9edc
jit format
kunalspathak c8a9e01
Remove unneeded method
kunalspathak 5bb1563
Misc changes
kunalspathak 2c6e81d
Review feedback
kunalspathak bbc2ac5
Do not include align behind Jmp in PerfScore calculation
kunalspathak 64bba41
jit format and fix a bug
kunalspathak 1e24fcb
fix the loopCandidates == 0 scenario
kunalspathak b301fa5
Add unmarkLoopAlign(), add check for fgFirstBB
kunalspathak 57759d0
merge conflict fix
kunalspathak ef0e859
Add missing }
kunalspathak 976b253
Grammar nit
kunalspathak File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Fix a problem where curIG==0 and loop might be emitted in curIG, adju…
…st the targetIG to prevIG Add IGF_REMOVED_ALIGN flag for special scenarios
- Loading branch information
commit e7c07102f346714f4941ca4f48e2655aa1f42ba4
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -4932,9 +4932,24 @@ void emitter::emitLongLoopAlign(unsigned short alignmentBoundary) | |
// | ||
void emitter::emitConnectAlignInstrWithCurIG() | ||
{ | ||
JITDUMP("Mapping 'align' instruction in IG%02u to target IG%02u\n", emitRecentFirstAlign->idaIG->igNum, | ||
emitCurIG->igNum); | ||
// Since we never align overlapping instructions, it is always guaranteed that | ||
// the emitRecentFirstAlign points to the loop that is in process of getting aligned. | ||
emitRecentFirstAlign->idaTargetIG = emitCurIG; | ||
|
||
if (emitCurIGsize == 0) | ||
{ | ||
// However, if current IG is still empty (e.g. codegen didn't generate moves if they were redundant) | ||
// in that case, loop will start in emitCurIG itself. For such cases, make sure that targetIG is | ||
// pointing to previous IG. | ||
assert(emitPrevIG != emitCurIG); | ||
|
||
emitRecentFirstAlign->idaTargetIG = emitPrevIG; | ||
} | ||
else | ||
{ | ||
emitRecentFirstAlign->idaTargetIG = emitCurIG; | ||
} | ||
} | ||
|
||
//----------------------------------------------------------------------------- | ||
|
@@ -5024,9 +5039,14 @@ unsigned emitter::getLoopSize(insGroup* igLoopHeader, unsigned maxLoopSize DEBUG | |
for (insGroup* igInLoop = igLoopHeader; igInLoop != nullptr; igInLoop = igInLoop->igNext) | ||
{ | ||
loopSize += igInLoop->igSize; | ||
if (igInLoop->endsWithAlignInstr()) | ||
if (igInLoop->endsWithAlignInstr() || igInLoop->isAlignInstrRemoved()) | ||
{ | ||
// If igInLoop is marked as "IGF_HAS_ALIGN", the basic block flow detected a loop start. | ||
// If igInLoop can be in one of the following state: | ||
// IGF_HAS_ALIGN - igInLoop contains align instruction at the end, for next IG or some future IG. | ||
// IGF_REMOVED_ALIGN - igInLoop had align instruction at the end, but was removed. | ||
// | ||
// For both cases, remove the padding bytes from igInLoop's size so it is not included in loopSize. | ||
// | ||
// If the loop was formed because of forward jumps like the loop IG18 below, the backedge is not | ||
// set for them and such loops are not aligned. For such cases, the loop size threshold will never | ||
// be met and we would break as soon as loopSize > maxLoopSize. | ||
|
@@ -5039,9 +5059,9 @@ unsigned emitter::getLoopSize(insGroup* igLoopHeader, unsigned maxLoopSize DEBUG | |
// ... | ||
// jne IG05 | ||
// | ||
// If igInLoop is a legitimate loop, and igInLoop's next IG is also a loop that needs alignment, | ||
// then igInLoop should be the last IG of the current loop and should have backedge to current | ||
// loop header. | ||
// If igInLoop is a legitimate loop, and igInLoop's end with another 'align' instruction for different IG | ||
// representing a loop that needs alignment, then igInLoop should be the last IG of the current loop and | ||
// should have backedge to current loop header. | ||
// | ||
// Below, IG05 is the last IG of loop IG04-IG05 and its backedge points to IG04. | ||
// | ||
|
@@ -5198,7 +5218,7 @@ void emitter::emitSetLoopBackEdge(BasicBlock* loopTopBlock) | |
assert(!markedCurrLoop); | ||
|
||
// This IG should no longer contain alignment instruction | ||
alignInstr->idaIG->removeAlignInstr(); | ||
alignInstr->removeAlignFlags(); | ||
|
||
markedCurrLoop = true; | ||
JITDUMP("** Skip alignment for current loop IG%02u ~ IG%02u because it encloses an aligned loop " | ||
|
@@ -5215,7 +5235,7 @@ void emitter::emitSetLoopBackEdge(BasicBlock* loopTopBlock) | |
assert(alignInstr->idaIG->endsWithAlignInstr()); | ||
|
||
// This IG should no longer contain alignment instruction | ||
alignInstr->idaIG->removeAlignInstr(); | ||
alignInstr->removeAlignFlags(); | ||
|
||
markedLastLoop = true; | ||
JITDUMP("** Skip alignment for aligned loop IG%02u ~ IG%02u because it encloses the current loop " | ||
|
@@ -5275,7 +5295,7 @@ void emitter::emitLoopAlignAdjustments() | |
insGroup* containingIG = alignInstr->idaIG; | ||
|
||
JITDUMP(" Adjusting 'align' instruction in IG%02u that is targeted for IG%02u \n", containingIG->igNum, | ||
alignIG->igNum); | ||
alignIG->igNext->igNum); | ||
|
||
// Since we only adjust the padding up to the next align instruction which is behind the jump, we make sure | ||
// that we take into account all the alignBytes we removed until that point. Hence " - alignBytesRemoved" | ||
|
@@ -5307,7 +5327,7 @@ void emitter::emitLoopAlignAdjustments() | |
containingIG->igFlags |= IGF_UPD_ISZ; | ||
if (actualPaddingNeeded == 0) | ||
{ | ||
containingIG->removeAlignInstr(); | ||
alignInstr->removeAlignFlags(); | ||
} | ||
|
||
#ifdef TARGET_XARCH | ||
|
@@ -8415,6 +8435,12 @@ insGroup* emitter::emitAllocAndLinkIG() | |
|
||
ig->igFlags |= (emitCurIG->igFlags & IGF_PROPAGATE_MASK); | ||
|
||
#ifdef FEATURE_LOOP_ALIGN | ||
/* Save the prev IG */ | ||
|
||
emitPrevIG = emitCurIG; | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Is it possible to implement this goal without creating a |
||
#endif | ||
|
||
/* Set the new IG as the current IG */ | ||
|
||
emitCurIG = ig; | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Couldn't we always just force a new IG for an aligned loop head, and then not have to worry about
emitCurIGsize==0
oremitPrevIG
?