Commit graph

1159 commits

Author SHA1 Message Date
Amir Ayupov c31af7cfe3 [MC][BOLT] Add setter for AllowAtInName
Use the setter in BOLT to allow printing names with variant kind in the name
(e.g. "func@PLT").
Fixes BOLT buildbot tests that broke after D122516:
https://lab.llvm.org/buildbot/#/builders/215/builds/3595

Reviewed By: maksfb

Differential Revision: https://reviews.llvm.org/D122694
2022-03-30 13:04:28 -07:00
Vladislav Khmelevsky af9bdcfc46 [BOLT] Align constant islands to 8 bytes
AArch64 requires CI to be aligned to 8 bytes due to access instructions
restrictions. E.g. the ldr with imm, where imm must be aligned to 8 bytes.

Differential Revision: https://reviews.llvm.org/D122065
2022-03-27 22:30:42 +03:00
spupyrev 4609f60ebc [BOLT] Avoid pointless loop rotation
It seems the earlier implementation does not follow the description
in LoopRotationPass.h: It rotates loops even if they are already laid out
correctly. The diff adjusts the behaviour.

Given that the impact of LoopInversionPass is minor, this change won't
yield significant perf differences. Tested on clang-10: there seems to be a
0.1%-0.3% cpu win and a small reduction of branch misses.

**Before:**
BOLT-INFO: 120 Functions were reordered by LoopInversionPass

**After:**
BOLT-INFO: 79 Functions were reordered by LoopInversionPass

Reviewed By: yota9

Differential Revision: https://reviews.llvm.org/D121921
2022-03-22 12:42:42 -07:00
Vladislav Khmelevsky 5be5d0f56e [BOLT] LongJmp speedup refactoring
Run tentativeLayoutRelocMode twice only if UseOldText option was passed.
Refactor BF loop to break on condtition met.

Differential Revision: https://reviews.llvm.org/D121825
2022-03-18 16:16:47 +03:00
Amir Ayupov 42e8e00189 [BOLT][NFC] Use X86 mnemonic tables
Remove tables from X86MCPlusBuilder, make use of llvm::X86 mnemonic tables.

Reviewed By: rafauler

Differential Revision: https://reviews.llvm.org/D121573
2022-03-18 01:52:11 -07:00
Amir Ayupov dc1cf838a5 [BOLT] Strip redundant AdSize override prefix
Since LLVM MC now preserves redundant AdSize override prefix (0x67), remove it
in BOLT explicitly (-x86-strip-redundant-adsize, on by default).

Test Plan:
`bin/llvm-lit -a bolt/test/X86/addr32.s`

Reviewed By: rafauler

Differential Revision: https://reviews.llvm.org/D120975
2022-03-16 09:38:17 -07:00
Amir Ayupov 698127df51 [BOLT][NFC] Move isMOVSX64rm32 out of MCPlusBuilder
Reviewed By: rafauler

Differential Revision: https://reviews.llvm.org/D121669
2022-03-16 08:18:56 -07:00
Vladislav Khmelevsky 62a289d85c [BOLT] LongJmp: Fix hot text section alignment
The BinaryEmitter uses opts::AlignText value to align the hot text
section. Also check that the opts::AlignText is at least
equal opts::AlignFunctions for the same reason, as described in D121392.

Vladislav Khmelevsky,
Advanced Software Technology Lab, Huawei

Differential Revision: https://reviews.llvm.org/D121728
2022-03-16 15:57:46 +03:00
Sam McCall 75acad41bc Use lit_config.substitute instead of foo % lit_config.params everywhere
This mechanically applies the same changes from D121427 everywhere.

Differential Revision: https://reviews.llvm.org/D121746
2022-03-16 09:57:41 +01:00
Maksim Panchenko 57f03db195 [BOLT][NFC] Remove unused function
Reviewed By: yota9

Differential Revision: https://reviews.llvm.org/D121729
2022-03-15 12:39:14 -07:00
Vladislav Khmelevsky 8ab69baad5 [BOLT] Set cold sections alignment explicitly
The cold text section alignment is set using the maximum alignment value
passed to the emitCodeAlignment. In order to calculate tentetive layout
right we will set the minimum alignment of such sections to the maximum
possible function alignment explicitly.

Differential Revision: https://reviews.llvm.org/D121392
2022-03-15 22:12:17 +03:00
Amir Ayupov 5790441c45 [BOLT][NFC] Use getShortOpcodeArith in X86MCPlusBuilder
Unify `llvm::X86::getRelaxedOpcodeArith` and `getShortArithOpcode` in
X86MCPlusBuilder.cpp.

Addresses https://lists.llvm.org/pipermail/llvm-dev/2022-January/154526.html

Reviewed By: rafauler

Differential Revision: https://reviews.llvm.org/D121404
2022-03-12 09:07:28 -08:00
Petr Hosek 0c0f6cfb7b [CMake] Rename TARGET_TRIPLE to LLVM_TARGET_TRIPLE
This clarifies that this is an LLVM specific variable and avoids
potential conflicts with other projects.

Differential Revision: https://reviews.llvm.org/D119918
2022-03-11 15:43:01 -08:00
Elvina Yakubova db65429db5 [BOLT] Divide RegularPageSize for X86 and AArch64 cases
For AArch64 in some cases/some distributions ld uses 64K alignment of LOAD segments by default.

Reviewed By: yota9, maksfb

Differential Revision: https://reviews.llvm.org/D119267
2022-03-10 23:09:50 +03:00
Vladislav Khmelevsky 04b87cf0e7 [BOLT] LongJmp: Use per-function alignment values
The per-function alignment values must be used in order to create
tentative layout.

Differential Revision: https://reviews.llvm.org/D121298
2022-03-10 19:48:48 +03:00
Amir Ayupov d1638cb0b5 [BOLT][NFC] Fix print-cfg data race
Addresses ThreadSanitizer warning

Reviewed By: rafauler

Differential Revision: https://reviews.llvm.org/D121337
2022-03-09 20:28:06 -08:00
Amir Ayupov d16bbc5340 [BOLT][NFC] Check errors from Obj.dynamicEntries
Addresses fuzzer crash

Reviewed By: rafauler

Differential Revision: https://reviews.llvm.org/D121336
2022-03-09 20:24:44 -08:00
Vladislav Khmelevsky 506a91c089 [BOLT] Move some of the tests to common directory
Some of the tests are not x86-specific, move them to common directory.

Differential Revision: https://reviews.llvm.org/D121261
2022-03-09 14:51:17 +03:00
Vladislav Khmelevsky 8bdbcfe7d8 [BOLT] Handle ifuncs trampolines for aarch64
The aarch64 uses the trampolines located in .iplt section, which
contains plt-like trampolines on the value stored in .got. In this case
we don't have JUMP_SLOT relocation, but we have a symbol that belongs to
ifunc trampoline, so use it and set set plt symbol for such functions.

Vladislav Khmelevsky,
Advanced Software Technology Lab, Huawei

Differential Revision: https://reviews.llvm.org/D120850
2022-03-09 01:38:19 +03:00
Amir Ayupov 1e016c3bd5 [BOLT][NFC] Handle "dynamic section sizes should match"
Address fuzzer crash on malformed input

Reviewed By: rafauler

Differential Revision: https://reviews.llvm.org/D121070
2022-03-08 13:03:05 -08:00
Amir Ayupov 687e4af1c0 [BOLT] CMOVConversion pass
Convert simple hammocks into cmov based on misprediction rate.

Test Plan:
- Assembly test: `cmov-conversion.s`
- Testing on a binary:
  # Bootstrap clang with `-x86-cmov-converter-force-all` and `-Wl,--emit-relocs`
  (Release build)
  # Collect perf.data:

    - `clang++ <opts> bolt/lib/Core/BinaryFunction.cpp -E > bf.cpp`
    - `perf record -e cycles:u -j any,u -- clang-15 bf.cpp -O2 -std=c++14 -c -o bf.o`
  # Optimize clang-15 with and w/o -cmov-conversion:
    - `llvm-bolt clang-15 -p perf.data -o clang-15.bolt`
    - `llvm-bolt clang-15 -p perf.data -cmov-conversion -o clang-15.bolt.cmovconv`
  # Run perf experiment:
    - test: `clang-15.bolt.cmovconv`,
    - control: `clang-15.bolt`,
    - workload (clang options): `bf.cpp -O2 -std=c++14 -c -o bf.o`
Results:
```
  task-clock [delta: -360.21 ± 356.75, delta(%): -1.7760 ± 1.7589, p-value: 0.047951, balance: -6]
  instructions  [delta: 44061118 ± 13246382, delta(%): 0.0690 ± 0.0207, p-value: 0.000001, balance: 50]
  icache-misses [delta: -5534468 ± 2779620, delta(%): -0.4331 ± 0.2175, p-value: 0.028014, balance: -28]
  branch-misses [delta: -1624270 ± 1113244, delta(%): -0.3456 ± 0.2368, p-value: 0.030300, balance: -22]
```

Reviewed By: rafauler

Differential Revision: https://reviews.llvm.org/D120177
2022-03-08 10:44:31 -08:00
Amir Ayupov ced5472e09 [BOLT][NFC] Check section contents before registering it
Address fuzzer crash on malformed input:
```
BOLT-ERROR: cannot get section contents for .dynsym: The end of the file was unexpectedly encountered.
```

Reviewed By: rafauler

Differential Revision: https://reviews.llvm.org/D121068
2022-03-08 09:13:01 -08:00
Amir Ayupov 018ad03efa [BOLT][CMAKE] Remove CMake 3.13.4 incompatible parameter
Remove `TYPE BIN` parameter that is introduced in CMake 3.14 and revert back to
the equivalent compatible form `DESTINATION ${CMAKE_INSTALL_BINDIR}`.

Addresses https://github.com/llvm/llvm-project/issues/54099

Reviewed By: rafauler

Differential Revision: https://reviews.llvm.org/D121012
2022-03-07 18:42:00 -08:00
Yi Kong 8142ace0a7 Revert "Add CMake option not to build BOLT tests"
This reverts commit d8f4d54664.

Merged by accident.
2022-03-08 02:01:15 +08:00
Yi Kong d8f4d54664 Add CMake option not to build BOLT tests 2022-03-08 01:59:44 +08:00
Maksim Panchenko fada230920 [BOLT][NFC] Return MCRegister::NoRegister from MCPlusBuilder::getNoRegister()
Reviewed By: yota9

Differential Revision: https://reviews.llvm.org/D120863
2022-03-03 13:25:13 -08:00
Vladislav Khmelevsky 00b6efc830 [BOLT] Enable PLT analysis for aarch64
This patch enables PLT analysis for aarch64. It is used by the static
relocations in order to provide final symbol address of PLT entry for some
instructions like ADRP.

Vladislav Khmelevsky,
Advanced Software Technology Lab, Huawei

Differential Revision: https://reviews.llvm.org/D118088
2022-03-02 22:14:48 +03:00
Maksim Panchenko ae87445c25 [BOLT][test] Fix function size in test case 2022-03-01 17:53:41 -08:00
Amir Ayupov 08dcbed92f [BOLT] Fix X86MCPlusBuilder::replaceRegWithImm
Reassigning the operand didn't update the operand type which resulted in an
assertion (`Assertion `isReg() && "This is not a register operand!"' failed.`)
Reset the instruction instead.

Test Plan:
```
ninja check-bolt
...
PASS: BOLT-Unit :: Core/./CoreTests/X86/MCPlusBuilderTester.ReplaceRegWithImm/0 (90 of 136)
```

Reviewed By: rafauler

Differential Revision: https://reviews.llvm.org/D120263
2022-02-28 19:24:46 -08:00
Alexander Yermolovich a44fe31977 [BOLT][DWARF] Fix how DW_AT_high_pc [DW_FORM_udata] is handled
We were not handling correctly conversion from DW_AT_high_pc into DW_AT_ranges,
when size of DW_AT_high_pc is not 4/8 bytes.

Reviewed By: maksfb

Differential Revision: https://reviews.llvm.org/D120528
2022-02-25 10:32:05 -08:00
Maksim Panchenko 4101aa130a [BOLT] Support PC-relative relocations with addends
PC-relative memory operand could reference a different object from
the one located at the target address, e.g. when a negative offset
is used. Check relocations for the real referenced object.

Reviewed By: rafauler

Differential Revision: https://reviews.llvm.org/D120379
2022-02-23 22:54:42 -08:00
Amir Ayupov af6e66f44c [BOLT][NFC] Report errors from RewriteInstance discoverStorage and run
Further improve error handling in BOLT by reporting `RewriteInstance` errors in
a library and fuzzer-friendly way instead of exiting.

Follow-up to D119658

Reviewed By: rafauler

Differential Revision: https://reviews.llvm.org/D120224
2022-02-23 20:42:39 -08:00
Amir Ayupov 454c149898 [BOLT][NFC] Fix undefined behavior in encodeAnnotationImm
Fix UBSan-reported issue in MCPlusBuilder::encodeAnnotationImm (left shift of a
negative value).

Test Plan:
```
ninja check-bolt
...
PASS: BOLT-Unit :: Core/./CoreTests/AArch64/MCPlusBuilderTester.Annotation/0 (1 of 140)
PASS: BOLT-Unit :: Core/./CoreTests/X86/MCPlusBuilderTester.Annotation/0 (131 of 134)
```

Reviewed By: maksfb, yota9

Differential Revision: https://reviews.llvm.org/D120260
2022-02-23 16:02:49 -08:00
Alexander Yermolovich 210bb04e23 [BOLT][DWARF] Remove patchLowHigh unused function.
Cleanup after removing caching mechanims for ranges/abbrevs.

Reviewed By: rafauler, yota9

Differential Revision: https://reviews.llvm.org/D120174
2022-02-22 13:28:15 -08:00
Amir Ayupov d44f99c748 [BOLT] Added fuzzer target (llvm-bolt-fuzzer)
This adds a target that would consume random binary as an
input ELF file.
TBD: add structured input support (ELF).

Build:
```
cmake /path/to/llvm-project/llvm -GNinja \
-DLLVM_TARGETS_TO_BUILD="X86;AArch64" \
-DCMAKE_BUILD_TYPE=Release \
-DLLVM_ENABLE_ASSERTIONS=1 \
-DCMAKE_C_COMPILER=<sanitizer-capable clang> \
-DCMAKE_CXX_COMPILER=<sanitizer-capable clang++> \
-DLLVM_ENABLE_PROJECTS="bolt"  \
-DLLVM_USE_SANITIZER=Address \
-DLLVM_USE_SANITIZE_COVERAGE=On
ninja llvm-bolt-fuzzer
```

Test Plan: ninja llvm-bolt-fuzzer

Reviewed By: maksfb

Differential Revision: https://reviews.llvm.org/D120016
2022-02-20 17:24:16 -08:00
Amir Ayupov 36ada32727 [BOLT][NFC] Fix data race in ShrinkWrapping stats
Fix data race reported by ThreadSanitizer in clang.test:
```
ThreadSanitizer: data race /data/llvm-project/bolt/lib/Passes/ShrinkWrapping.cpp:1359:28
in llvm::bolt::ShrinkWrapping::moveSaveRestores()
```

The issue is with incrementing global counters from multiple threads.

Reviewed By: yota9

Differential Revision: https://reviews.llvm.org/D120218
2022-02-20 17:21:58 -08:00
Amir Ayupov 32d2473a5d [BOLT][NFC] Report errors from createBinaryContext and RewriteInstance ctor
Refactor createBinaryContext and RewriteInstance/MachORewriteInstance
constructors to report an error in a library and fuzzer-friendly way instead of
returning a nullptr or exiting.

Reviewed By: rafauler

Differential Revision: https://reviews.llvm.org/D119658
2022-02-17 00:50:52 -08:00
Vladislav Khmelevsky 729d29e167 [BOLT] Update dynamic relocations from section relocations
This patch changes patchELFAllocatableRelaSections from going through
old relocations sections and update the relocation offsets to emitting
the relocations stored in binary sections. This is needed in case we
would like to remove and add dynamic relocations during BOLT work and it
is used by golang support pass. Note: Currently we emit relocations in
the old sections, so the total number of them should be equal or less
of old number.

Testing: No special tests are neeeded, since this patch does not fix
anything or add new functionality (it only prepares to add). Every
PIC-compiled test binary will use this code and thus become a test.
But just in case the aarch64 dynamic relocations tests were added.

Vladislav Khmelevsky,
Advanced Software Technology Lab, Huawei

Reviewed By: maksfb

Differential Revision: https://reviews.llvm.org/D117612
2022-02-16 18:40:54 +03:00
Shao-Ce SUN 2aed07e96c [NFC][MC] remove unused argument MCRegisterInfo in MCCodeEmitter
Reviewed By: skan

Differential Revision: https://reviews.llvm.org/D119846
2022-02-16 13:10:09 +08:00
Shao-Ce SUN 9cc49c1951 Revert "[NFC][MC] remove unused argument MCRegisterInfo in MCCodeEmitter"
This reverts commit fe25c06cc5.
2022-02-16 11:57:49 +08:00
Shao-Ce SUN fe25c06cc5 [NFC][MC] remove unused argument MCRegisterInfo in MCCodeEmitter
For ten years, it seems that `MCRegisterInfo` is not used by any target.

Reviewed By: skan

Differential Revision: https://reviews.llvm.org/D119846
2022-02-16 11:47:17 +08:00
Alexander Yermolovich bd1ebe9d04 [BOLT][DWARF] Add ability to insert new entries in to DIE
Added ability to append new entries to DIE. This is useful to standadize DWARF4
Split Dwarf, and simplify implementation of DWARF5.
Multiple DIEs can share an abbrev. So currently limitation is that only unique
Attributes can be added.

Reviewed By: maksfb

Differential Revision: https://reviews.llvm.org/D119577
2022-02-15 18:07:19 -08:00
Vladislav Khmelevsky eccdf2d9b1 [BOLT] Fix aarch64 dwarf test
After "Remove caching of ranges/abbrevs" patch the dwarf offsets are a
bit changed and the subprograms high pc is replaced with AT_RANGES.

Reviewed By: rafauler

Differential Revision: https://reviews.llvm.org/D119733
2022-02-15 13:59:38 +03:00
serge-sans-paille 290e482342 Cleanup LLVMDWARFDebugInfo
As usual with that header cleanup series, some implicit dependencies now need to
be explicit:

llvm/DebugInfo/DWARF/DWARFContext.h no longer includes:
- "llvm/DebugInfo/DWARF/DWARFAcceleratorTable.h"
- "llvm/DebugInfo/DWARF/DWARFCompileUnit.h"
- "llvm/DebugInfo/DWARF/DWARFDebugAbbrev.h"
- "llvm/DebugInfo/DWARF/DWARFDebugAranges.h"
- "llvm/DebugInfo/DWARF/DWARFDebugFrame.h"
- "llvm/DebugInfo/DWARF/DWARFDebugLoc.h"
- "llvm/DebugInfo/DWARF/DWARFDebugMacro.h"
- "llvm/DebugInfo/DWARF/DWARFGdbIndex.h"
- "llvm/DebugInfo/DWARF/DWARFSection.h"
- "llvm/DebugInfo/DWARF/DWARFTypeUnit.h"
- "llvm/DebugInfo/DWARF/DWARFUnitIndex.h"

Plus llvm/Support/Errc.h not included by a bunch of llvm/DebugInfo/DWARF/DWARF*.h files

Preprocessed lines to build llvm on my setup:
after: 1065629059
before: 1066621848

Which is a great diff!

Discourse thread: https://discourse.llvm.org/t/include-what-you-use-include-cleanup
Differential Revision: https://reviews.llvm.org/D119723
2022-02-15 09:16:03 +01:00
Maksim Panchenko 5a343994c3 [BOLT] Make order of jump table successors deterministic
When a jump table is recovered in postProcessIndirectBranches(),
successors for the containing basic block are added in random order.
Make the order deterministic.

Reviewed By: yota9

Differential Revision: https://reviews.llvm.org/D119672
2022-02-14 10:37:20 -08:00
Maksim Panchenko 641e92d46b [BOLT] Skip warning message if no functions were ignored
Reviewed By: yota9, Amir

Differential Revision: https://reviews.llvm.org/D119673
2022-02-14 10:31:43 -08:00
serge-sans-paille e72c195fdc Cleanup LLVMObject headers
Most notably,

llvm/Object/Binary.h no longer includes llvm/Support/MemoryBuffer.h
llvm/Object/MachOUniversal*.h no longer include llvm/Object/Archive.h
llvm/Object/TapiUniversal.h no longer includes llvm/Object/TapiFile.h

llvm-project preprocessed size:
before: 1068185081
after:  1068324320

Discourse thread: https://discourse.llvm.org/t/include-what-you-use-include-cleanup
Differential Revision: https://reviews.llvm.org/D119457
2022-02-10 21:13:44 +01:00
Louis Dionne 4ae83bb2b1 Update all LLVM documentation mentioning runtimes in LLVM_ENABLE_PROJECTS
We are moving away from building the runtimes with LLVM_ENABLE_PROJECTS,
however the documentation was largely outdated. This commit updates all
the documentation I could find to use LLVM_ENABLE_RUNTIMES instead of
LLVM_ENABLE_PROJECTS for building runtimes.

Note that in the near future, libcxx, libcxxabi and libunwind will stop
supporting being built with LLVM_ENABLE_PROJECTS altogether. I don't know
what the plans are for other runtimes like libc, openmp and compiler-rt,
so I didn't make any changes to the documentation that would imply
something for those projects.

Once this lands, I will also cherry-pick this on the release/14.x branch
to make sure that LLVM's documentation is up-to-date and reflects what
we intend to support in the future.

Differential Revision: https://reviews.llvm.org/D119351
2022-02-10 15:05:23 -05:00
serge-sans-paille 57f7c7d90e Add missing MC includes in bolt/
Changes needed after ef736a1c39 that removes some implicit
dependencies from MrCV headers.
2022-02-09 08:28:34 -05:00
Alexander Yermolovich 0d9921daad [BOLT][DWARF] Remove caching of ranges/abbrevs
Removing caching of ranges/abbrevs to simplify the code.
Before we were doing it to get around a gdb limitation.
FBD34015613

Reviewed By: Amir, maksfb

Differential Revision: https://reviews.llvm.org/D119276
2022-02-08 16:37:40 -08:00