Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[JIT] X64/ARM64 - CAST removals for small types on ADD, SUB, MUL, AND, OR, XOR, NOT, NEG #77874

Merged
merged 71 commits into from
Jan 31, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
71 commits
Select commit Hold shift + click to select a range
f6bb7d4
Trying to optimize Add
TIHan Nov 4, 2022
81a1bcb
Trying to optimize Add
TIHan Nov 4, 2022
1b29b47
Trying to optimize Add
TIHan Nov 4, 2022
2599997
Merge remote-tracking branch 'upstream/main' into add-opt-x64
TIHan Nov 4, 2022
5e3741c
Added IntAdd disasm tests. Expanded cast removals to all arithmetic a…
TIHan Nov 5, 2022
371ca7d
Merge branch 'main' into add-opt-x64
TIHan Nov 5, 2022
2a43021
Checked unsigned
TIHan Nov 5, 2022
5428df9
Limit arithmetic opt
TIHan Nov 7, 2022
ac58639
Merge branch 'add-opt-x64' of https://github.com/TIHan/runtime into a…
TIHan Nov 7, 2022
7badcc0
Merge remote-tracking branch 'upstream/main' into add-opt-x64
TIHan Nov 8, 2022
6d85c1a
Moving optimization to Morph
TIHan Nov 8, 2022
02dbb22
Cleanup. Allow more opts.
TIHan Nov 8, 2022
779dd21
Fix build
TIHan Nov 8, 2022
a263faa
More opts
TIHan Nov 8, 2022
f7c8b68
Added regression test. Allowing all arithmetic ops. Only do transform…
TIHan Nov 9, 2022
3eba2fa
Do not do optimization for unsigned ops
TIHan Nov 9, 2022
ec3dcda
Added more division test cases
TIHan Nov 9, 2022
183d570
Fixed test. Checking overflows on cast. Comparing actual types proper…
TIHan Nov 9, 2022
4378c1b
Added flag
TIHan Nov 10, 2022
4a0b63d
Remove flag
TIHan Nov 10, 2022
73fbc61
Formatting
TIHan Nov 10, 2022
38ec57f
Revert minor change
TIHan Nov 10, 2022
343badf
Removed ClearRegOptionalAndContained
TIHan Nov 10, 2022
2e5c716
Merge remote-tracking branch 'upstream/main' into add-opt-x64
TIHan Nov 11, 2022
e417b63
Merge remote-tracking branch 'upstream/main' into add-opt-x64
TIHan Nov 13, 2022
95840b2
Trying out lowering
TIHan Nov 13, 2022
1c2e139
Expanded lowering cast removal to arm
TIHan Nov 13, 2022
49341e6
Added regression test
TIHan Nov 13, 2022
60f48ba
Do not use disasm checks for these tests
TIHan Nov 13, 2022
b267aa5
Update lower.cpp
TIHan Nov 13, 2022
caefb4a
Fixing
TIHan Nov 13, 2022
e226f30
For JCC, do not change castOp type if it is a bool
TIHan Nov 13, 2022
84ae5a7
Added more regression tests made from Fuzzlyn. Reverted minor cast ch…
TIHan Nov 13, 2022
937596b
Remove Morph impl as we are not favoring Lowering. Do not change var …
TIHan Nov 13, 2022
6cc95ca
Formatting
TIHan Nov 13, 2022
e5c9b93
Re-arranging regression tests. Some comments. Some And and Or disasm …
TIHan Nov 14, 2022
966df08
Fixed a few minor regressions
TIHan Nov 14, 2022
94411b5
minor change
TIHan Nov 14, 2022
4cb4d6d
Merge remote-tracking branch 'upstream/main' into add-opt-x64
TIHan Nov 14, 2022
3ec0f0d
Moving optimization to simple lowering
TIHan Nov 14, 2022
cb1983c
Moving back to morph
TIHan Nov 15, 2022
6f71cff
Add print for implicit cast
TIHan Nov 16, 2022
ca60055
Do it a little earlier
TIHan Nov 16, 2022
e445092
Mark as do-not-cse to prevent the CAST from being removed on assignment
TIHan Nov 16, 2022
4b9d344
Fix test
TIHan Nov 17, 2022
7022968
Remove do-not-set cse
TIHan Nov 17, 2022
ec5088f
Moving optimization to simple lowering again for correctness
TIHan Nov 18, 2022
62938ba
Disable some of the regression tests for mono
TIHan Nov 18, 2022
e90ce88
Fixing IntOr disasm test
TIHan Nov 18, 2022
769d1c1
Formatting, fixing test again
TIHan Nov 18, 2022
6903869
Smaller size operators can contain/reg-optional operands that are larger
TIHan Nov 18, 2022
de34d34
Formatting
TIHan Nov 18, 2022
f1ef6f3
Added subtract tests
TIHan Nov 28, 2022
38386d5
Update IntAnd.cs
TIHan Nov 29, 2022
f34f943
Update IntAnd.cs
TIHan Nov 29, 2022
232fe85
Update IntOr.cs
TIHan Nov 29, 2022
476143e
Update IntSubtract.cs
TIHan Nov 30, 2022
e0d49b3
Added IsContainableMemoryOpSize
TIHan Dec 2, 2022
21f3a8b
Merged
TIHan Dec 2, 2022
aad9ca3
Merge branch 'add-opt-x64' of https://github.com/TIHan/runtime into a…
TIHan Dec 2, 2022
37fc2fb
Merge remote-tracking branch 'upstream/main' into add-opt-x64
TIHan Jan 11, 2023
23427f6
Removed unnecessary code.
TIHan Jan 13, 2023
520be33
Update IntAnd.cs
TIHan Jan 13, 2023
7cac5e2
Update Regression1.cs
TIHan Jan 13, 2023
c6a31fe
Update Regression1.cs
TIHan Jan 13, 2023
2fbe829
Update Regression2.cs
TIHan Jan 13, 2023
3384759
Update Regression2.cs
TIHan Jan 14, 2023
2ea70d4
Merge remote-tracking branch 'upstream/main' into add-opt-x64
TIHan Jan 30, 2023
40b59c2
Disable if optimizations are disabled
TIHan Jan 30, 2023
051d513
Merge branch 'add-opt-x64' of https://github.com/TIHan/runtime into a…
TIHan Jan 30, 2023
25784fb
Merge
TIHan Jan 30, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions src/coreclr/jit/compiler.h
Original file line number Diff line number Diff line change
Expand Up @@ -4783,6 +4783,8 @@ class Compiler
// lowering that is distributed between fgMorph and the lowering phase of LSRA.
PhaseStatus fgSimpleLowering();

bool fgSimpleLowerCastOfSmpOp(LIR::Range& range, GenTreeCast* cast);

#if FEATURE_LOOP_ALIGN
PhaseStatus placeLoopAlignInstructions();
#endif
Expand Down
94 changes: 94 additions & 0 deletions src/coreclr/jit/flowgraph.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -2989,6 +2989,15 @@ PhaseStatus Compiler::fgSimpleLowering()
}
#endif // FEATURE_FIXED_OUT_ARGS

case GT_CAST:
{
if (tree->AsCast()->CastOp()->OperIsSimple() && fgSimpleLowerCastOfSmpOp(range, tree->AsCast()))
{
madeChanges = true;
}
break;
}

default:
{
// No other operators need processing.
Expand Down Expand Up @@ -3047,6 +3056,91 @@ PhaseStatus Compiler::fgSimpleLowering()
return madeChanges ? PhaseStatus::MODIFIED_EVERYTHING : PhaseStatus::MODIFIED_NOTHING;
}

//------------------------------------------------------------------------
// fgSimpleLowerCastOfSmpOp: Optimization to remove CAST nodes from operands of some simple ops that are safe to do so
// since the upper bits do not affect the lower bits, and result of the simple op is zero/sign-extended via a CAST.
// Example:
// CAST(ADD(CAST(x), CAST(y))) transforms to CAST(ADD(x, y))
//
// Returns:
// True or false, representing changes were made.
//
// Notes:
// This optimization could be done in morph, but it cannot because there are correctness
// problems with NOLs (normalized-on-load locals) and how they are handled in VN.
// Simple put, you cannot remove a CAST from CAST(LCL_VAR{nol}) in HIR.
//
// Because the optimization happens after rationalization, turning into LIR, it is safe to remove the CAST.
//
bool Compiler::fgSimpleLowerCastOfSmpOp(LIR::Range& range, GenTreeCast* cast)
{
GenTree* castOp = cast->CastOp();
var_types castToType = cast->CastToType();
var_types srcType = castOp->TypeGet();

assert(castOp->OperIsSimple());

if (opts.OptimizationDisabled())
return false;

if (cast->gtOverflow())
return false;

if (castOp->OperMayOverflow() && castOp->gtOverflow())
return false;

// Only optimize if the castToType is a small integer type.
// Only optimize if the srcType is an integer type.
if (!varTypeIsSmall(castToType) || !varTypeIsIntegral(srcType))
return false;

// These are the only safe ops where the CAST is not necessary for the inputs.
if (castOp->OperIs(GT_ADD, GT_SUB, GT_MUL, GT_AND, GT_XOR, GT_OR, GT_NOT, GT_NEG))
{
bool madeChanges = false;

if (castOp->gtGetOp1()->OperIs(GT_CAST))
{
GenTreeCast* op1 = castOp->gtGetOp1()->AsCast();

if (!op1->gtOverflow() && (genActualType(op1->CastOp()) == genActualType(srcType)) &&
(castToType == op1->CastToType()))
{
// Removes the cast.
castOp->AsOp()->gtOp1 = op1->CastOp();
range.Remove(op1);
madeChanges = true;
}
}

if (castOp->OperIsBinary() && castOp->gtGetOp2()->OperIs(GT_CAST))
{
GenTreeCast* op2 = castOp->gtGetOp2()->AsCast();

if (!op2->gtOverflow() && (genActualType(op2->CastOp()) == genActualType(srcType)) &&
(castToType == op2->CastToType()))
{
// Removes the cast.
castOp->AsOp()->gtOp2 = op2->CastOp();
range.Remove(op2);
madeChanges = true;
}
}

#ifdef DEBUG
if (madeChanges)
{
JITDUMP("Lower - Cast of Simple Op %s:\n", GenTree::OpName(cast->OperGet()));
DISPTREE(cast);
}
#endif // DEBUG

return madeChanges;
}

return false;
}

/*****************************************************************************************************
*
* Function to return the last basic block in the main part of the function. With funclets, it is
Expand Down
150 changes: 150 additions & 0 deletions src/tests/JIT/opt/Add/IntAdd.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,150 @@
// Licensed to the .NET Foundation under one or more agreements.
// The .NET Foundation licenses this file to you under the MIT license.

using System;
using System.Runtime.CompilerServices;

namespace CodeGenTests
{
static class IntAdd
{
[MethodImpl(MethodImplOptions.NoInlining)]
static sbyte Int8_Add(sbyte x, sbyte y)
{
// X64-NOT: movsx

// X64: add
// X64-NEXT: movsx

return (sbyte)(x + y);
}

[MethodImpl(MethodImplOptions.NoInlining)]
static byte UInt8_Add(byte x, byte y)
{
// X64-NOT: movzx

// X64: add
// X64-NEXT: movzx

return (byte)(x + y);
}

[MethodImpl(MethodImplOptions.NoInlining)]
static short Int16_Add(short x, short y)
{
// X64-NOT: movsx

// X64: add
// X64-NEXT: movsx

// X64-NOT: cwde

return (short)(x + y);
}

[MethodImpl(MethodImplOptions.NoInlining)]
static ushort UInt16_Add(ushort x, ushort y)
{
// X64-NOT: movzx

// X64: add
// X64-NEXT: movzx

return (ushort)(x + y);
}

[MethodImpl(MethodImplOptions.NoInlining)]
static int Int32_Add(int x, int y)
{
// X64: lea

return x + y;
}

[MethodImpl(MethodImplOptions.NoInlining)]
static uint UInt32_Add(uint x, uint y)
{
// X64: lea

return x + y;
}

[MethodImpl(MethodImplOptions.NoInlining)]
static long Int64_Add(long x, long y)
{
// X64: lea

return x + y;
}

[MethodImpl(MethodImplOptions.NoInlining)]
static ulong UInt64_Add(ulong x, ulong y)
{
// X64: lea

return x + y;
}

static int Main()
{
// Int8
if (Int8_Add(SByte.MaxValue, 15) != -114)
return 0;

if (Int8_Add(15, SByte.MaxValue) != -114)
return 0;

// UInt8
if (UInt8_Add(Byte.MaxValue, 15) != 14)
return 0;

if (UInt8_Add(15, Byte.MaxValue) != 14)
return 0;

// Int16
if (Int16_Add(Int16.MaxValue, 15) != -32754)
return 0;

if (Int16_Add(15, Int16.MaxValue) != -32754)
return 0;

// UInt16
if (UInt16_Add(UInt16.MaxValue, 15) != 14)
return 0;

if (UInt16_Add(15, UInt16.MaxValue) != 14)
return 0;

// Int32
if (Int32_Add(Int32.MaxValue, 15) != -2147483634)
return 0;

if (Int32_Add(15, Int32.MaxValue) != -2147483634)
return 0;

// UInt32
if (UInt32_Add(UInt32.MaxValue, 15) != 14)
return 0;

if (UInt32_Add(15, UInt32.MaxValue) != 14)
return 0;

// Int64
if (Int64_Add(Int64.MaxValue, 15) != -9223372036854775794)
return 0;

if (Int64_Add(15, Int64.MaxValue) != -9223372036854775794)
return 0;

// UInt64
if (UInt64_Add(UInt64.MaxValue, 15) != 14)
return 0;

if (UInt64_Add(15, UInt64.MaxValue) != 14)
return 0;

return 100;
}
}
}
17 changes: 17 additions & 0 deletions src/tests/JIT/opt/Add/IntAdd.csproj
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<OutputType>Exe</OutputType>
</PropertyGroup>
<PropertyGroup>
<DebugType>None</DebugType>
<Optimize>True</Optimize>
</PropertyGroup>
<ItemGroup>
<Compile Include="$(MSBuildProjectName).cs">
<HasDisasmCheck>true</HasDisasmCheck>
</Compile>

<CLRTestEnvironmentVariable Include="DOTNET_TieredCompilation" Value="0" />
<CLRTestEnvironmentVariable Include="DOTNET_JITMinOpts" Value="0" />
</ItemGroup>
</Project>
Loading