[Darwin] Limit parallelism for sanitizer tests that use shadow memory on AS

On Darwin, we want to limit the parallelism during test execution for
sanitizer tests that use shadow memory.  The reason is explained by this
existing comment:

> Only run up to 3 processes that require shadow memory simultaneously
> on 64-bit Darwin. Using more scales badly and hogs the system due to
> inefficient handling of large mmap'd regions (terabytes) by the
> kernel.

Previously we detected 3 cases:
* on-device: limit to 1 process
* 64-bit: macOS & simulators, limit to 3 processes
* others (32-bit): no limitation

We checked for the 64-bit case like this: `if arch in ['x86_64',
'x86_64h']` which misses macOS running on AS. Additionally, we don't
care about 32-bit anymore, so I've simplified this to 2 cases: on-device
and everything else.

Differential Revision: https://reviews.llvm.org/D122751
This commit is contained in:
Julian Lettner 2022-03-30 10:24:11 -07:00
parent 1f7b58f2a5
commit a5228bcaad
6 changed files with 9 additions and 22 deletions

View file

@ -62,5 +62,5 @@ if config.enable_per_target_runtime_dir and config.target_arch != config.host_ar
# Set LD_LIBRARY_PATH to pick dynamic runtime up properly.
push_ld_library_path(config, config.compiler_rt_libdir)
if config.host_os == 'Darwin':
config.parallelism_group = config.darwin_sanitizer_parallelism_group_func
if not config.parallelism_group:
config.parallelism_group = 'shadow-memory'

View file

@ -613,8 +613,8 @@ if platform.system() == 'Darwin':
# Only run up to 3 processes that require shadow memory simultaneously on
# 64-bit Darwin. Using more scales badly and hogs the system due to
# inefficient handling of large mmap'd regions (terabytes) by the kernel.
elif config.target_arch in ['x86_64', 'x86_64h']:
lit_config.warning('Throttling sanitizer tests that require shadow memory on Darwin 64bit')
else:
lit_config.warning('Throttling sanitizer tests that require shadow memory on Darwin')
lit_config.parallelism_groups['shadow-memory'] = 3
# Multiple substitutions are necessary to support multiple shared objects used

View file

@ -13,5 +13,5 @@ config.test_exec_root = os.path.join("@COMPILER_RT_BINARY_DIR@", "lib",
"sanitizer_common", "tests")
config.test_source_root = config.test_exec_root
if config.host_os == 'Darwin':
config.parallelism_group = config.darwin_sanitizer_parallelism_group_func
if not config.parallelism_group:
config.parallelism_group = 'shadow-memory'

View file

@ -12,9 +12,10 @@ config.name = 'ThreadSanitizer-Unit'
config.test_exec_root = "@COMPILER_RT_BINARY_DIR@/lib/tsan/tests"
config.test_source_root = config.test_exec_root
if config.host_os == 'Darwin':
config.parallelism_group = config.darwin_sanitizer_parallelism_group_func
if not config.parallelism_group:
config.parallelism_group = 'shadow-memory'
if config.host_os == 'Darwin':
# On Darwin, we default to ignore_noninstrumented_modules=1, which also
# suppresses some races the tests are supposed to find. See tsan/lit.cfg.py.
if 'TSAN_OPTIONS' in config.environment:

View file

@ -50,13 +50,3 @@ if config.host_os == 'Darwin':
#
# rdar://80086125
config.environment['MallocNanoZone'] = '0'
# The test config gets pickled and sent to multiprocessing workers, and that
# only works for code if it is stored at the top level of some module.
# Therefore, we have to put the code in a .py file, add it to path, and import
# it to store it in the config.
import site
site.addsitedir(os.path.dirname(__file__))
import lit_unittest_cfg_utils
config.darwin_sanitizer_parallelism_group_func = \
lit_unittest_cfg_utils.darwin_sanitizer_parallelism_group_func

View file

@ -1,4 +0,0 @@
# Put all 64-bit tests in the shadow-memory parallelism group. We throttle those
# in our common lit config (lit.common.unit.cfg.py).
def darwin_sanitizer_parallelism_group_func(test):
return "shadow-memory" if "x86_64" in test.file_path else None