Unleash performance potential from compiler build process

[straight-shoota]

It has been discovered that different builds of the compiler shows huge differences in performance levels. At this point the influencing factors are not exactly clear however. This issue is supposed to coordinate investigations and propose guidelines for best performing builds.

The first mentions of the effect that a locally built compilers can perform much better (50% time) than the generic build was summarized in crystal-lang/distribution-scripts#147 (comment)

The compiler builds produced by 84codes at https://github.com/84codes/crystal-container-images/blob/b4bf7e48bc48ef9276959420fee7aefcf73a881b/alpine/Dockerfile have reportedly shown a 30% performance improvement over the builds from https://github.com/crystal-lang/distribution-scripts/blob/ce8ae2727f540b07d421b97fed71b0e5f262708c/linux/Dockerfile

Upgraded versions of the LLVM library may haven an effect, but they do not seem to explain everything.
MUSL libc vs GNU libc may also be a factor, so does static vs. dynamic linking (reports suggest dynamic GNU > dynamic MUSL > static MUSL).

[Blacksmoke16]

We should revisit this after 1.8 is released and see if LLVM15 resolves it.

EDIT: And/or if there is still a perf difference between a locally built compiler versus the pre-bilt one.

[kostya]

i think this is fixed:

rm -rf ~/.cache/crystal/
./bin/crystal -v
Crystal 1.8.2 [7aa5cdd86] (2023-05-09)

LLVM: 15.0.7
Default target: x86_64-unknown-linux-gnu

time make clean crystal release=1 stats=1                                                                                                   
rm -rf .build                                                                                                                                                                                
rm -rf ./docs                                                                                                                                                                                
rm -rf src/llvm/ext/llvm_ext.o                                                                                                                                                               
Using /usr/bin/llvm-config-15 [version= 15.0.7]                                                                                                                                              
g++ -c  -o src/llvm/ext/llvm_ext.o src/llvm/ext/llvm_ext.cc -I/usr/lib/llvm-15/include -std=c++14   -fno-exceptions -D_GNU_SOURCE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS                                                                                                                                                                                   
CRYSTAL_CONFIG_BUILD_COMMIT="7aa5cdd86" CRYSTAL_CONFIG_PATH='$ORIGIN/../share/crystal/src' SOURCE_DATE_EPOCH="1683967066"  CRYSTAL_CONFIG_LIBRARY_PATH='$ORIGIN/../lib/crystal' ./bin/crystal build -D strict_multi_assign -D preview_overload_order --release --stats -Dwithout_interpreter  -o .build/crystal src/compiler/crystal.cr -D without_openssl -D without_zlib -D use_pcre2
Parse:                             00:00:00.000334372 (   0.76MB)
Semantic (top level):              00:00:00.638895060 ( 154.87MB)
Semantic (new):                    00:00:00.002527632 ( 154.87MB)
Semantic (type declarations):      00:00:00.047661368 ( 170.87MB)
Semantic (abstract def check):     00:00:00.023129957 ( 186.87MB)
Semantic (restrictions augmenter): 00:00:00.015329667 ( 186.87MB)
Semantic (ivars initializers):     00:00:08.267588029 (1086.80MB)
Semantic (cvars initializers):     00:00:00.010548573 (1086.80MB)
Semantic (main):                   00:00:11.532512653 (1426.80MB)
Semantic (cleanup):                00:00:00.000720373 (1426.80MB)
Semantic (recursive struct check): 00:00:00.002305362 (1426.80MB)
Codegen (crystal):                 00:00:04.541256178 (1650.80MB)
Codegen (bc+obj):                  00:05:54.907858713 (1650.80MB)
Codegen (linking):                 00:00:01.565331562 (1650.80MB)

Macro runs:
 - /home/kostya/projects/crystal/src/ecr/process.cr: 00:00:06.820795264

Codegen (bc+obj):
 - no previous .o files were reused

real    6m24,647s
user    6m12,224s
sys     0m12,414s

rm -rf ~/.cache/crystal/
./bin/crystal -v
Using compiled compiler at .build/crystal
Crystal 1.8.2 [7aa5cdd86] (2023-05-09)

LLVM: 15.0.7
Default target: x86_64-pc-linux-gnu

time CRYSTAL_CONFIG_BUILD_COMMIT="7aa5cdd86" CRYSTAL_CONFIG_PATH='$ORIGIN/../share/crystal/src' SOURCE_DATE_EPOCH="1683622346"  CRYSTAL_CONFIG_LIBRARY_PATH='$ORIGIN/../lib/crystal' ./bin/crystal build -D strict_multi_assign -D preview_overload_order --release --stats -Dwithout_interpreter  -o .build/crystal src/compiler/crystal.cr -D without_openssl -D without_zlib -D use_pcre2
Using compiled compiler at .build/crystal
Parse:                             00:00:00.000054531 (   1.02MB)
Semantic (top level):              00:00:00.480335393 ( 155.10MB)
Semantic (new):                    00:00:00.002216272 ( 155.10MB)
Semantic (type declarations):      00:00:00.045637745 ( 171.10MB)
Semantic (abstract def check):     00:00:00.020463855 ( 187.10MB)
Semantic (restrictions augmenter): 00:00:00.015487850 ( 187.10MB)
Semantic (ivars initializers):     00:00:06.492652409 (1071.03MB)
Semantic (cvars initializers):     00:00:00.009869671 (1071.03MB)
Semantic (main):                   00:00:09.825243531 (1427.03MB)
Semantic (cleanup):                00:00:00.000700293 (1427.03MB)
Semantic (recursive struct check): 00:00:00.002143831 (1427.03MB)
Codegen (crystal):                 00:00:04.123753425 (1635.03MB)
Codegen (bc+obj):                  00:05:14.161241994 (1635.03MB)
Codegen (linking):                 00:00:01.859598811 (1635.03MB)

Macro runs:
 - /home/kostya/projects/crystal/src/ecr/process.cr: 00:00:05.918548524

Codegen (bc+obj):
 - no previous .o files were reused

real    5m37,262s
user    5m31,911s
sys     0m5,300s

local version still faster, but not 2 times now.