[rocprof-sys] Fix segfault from thread ID array overflow (#2172)
**Thread limit configuration and enforcement: ** * Added a check in `CMakeLists.txt` to ensure `ROCPROFSYS_MAX_THREADS` is at least 128, automatically setting it to 128 with a warning if a lower value is provided. * Replaced hardcoded thread limit (`allowed_max_threads`) in `pthread_create_gotcha.cpp` with the configurable `ROCPROFSYS_MAX_THREADS` value, ensuring all runtime checks and warnings use the actual configured limit. **Documentation improvements: ** * Updated the development guide to explain the new thread limit behavior, including how exceeding the limit is handled gracefully, how to configure it, and the build-time validation rules. **Test updates: ** * Modified thread limit tests to use the configurable `ROCPROFSYS_MAX_THREADS` value instead of a hardcoded limit and expanded the range of tested thread values. * Increased test timeouts to accommodate larger thread counts and ensure reliability with higher limits.
This commit is contained in:
@@ -327,14 +327,46 @@ Thread-data class
|
||||
|
||||
Currently, most thread data is effectively stored in a static
|
||||
``std::array<std::unique_ptr<T>, ROCPROFSYS_MAX_THREADS>`` instance.
|
||||
``ROCPROFSYS_MAX_THREADS`` is a value defined a compile-time and set to ``2048``
|
||||
for release builds. During finalization,
|
||||
``ROCPROFSYS_MAX_THREADS`` is a value defined at compile-time for release builds. During finalization,
|
||||
ROCm Systems Profiler iterates through the thread-data and transforms that data
|
||||
into something that can be passed along to Perfetto and/or Timemory.
|
||||
The downside of the current model is that if the user exceeds ``ROCPROFSYS_MAX_THREADS``,
|
||||
a segmentation fault occurs. To fix this issue,
|
||||
a new model is being adopted which has all the benefits of this model
|
||||
but permits dynamic expansion.
|
||||
In the current model, if the user exceeds ``ROCPROFSYS_MAX_THREADS`` at runtime,
|
||||
thread creation fails gracefully with a warning message, excess threads operate with thread-local fallback,
|
||||
and profiling is skipped and not persisted to output files for threads beyond ``ROCPROFSYS_MAX_THREADS``.
|
||||
To support truly dynamic thread limits without compile-time constraints, a new model is being adopted which
|
||||
has all the benefits of static allocation but permits dynamic expansion beyond ``ROCPROFSYS_MAX_THREADS``.
|
||||
Currently, the thread limit can be increased at compile-time using the ``ROCPROFSYS_MAX_THREADS`` CMake configuration option.
|
||||
|
||||
Configuring thread limits
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
ROCm Systems Profiler uses a single CMake configuration option to control thread-related memory allocation:
|
||||
|
||||
* ``ROCPROFSYS_MAX_THREADS``: Maximum number of threads supported (default if not explicitly set: ``128`` if nproc < 8, otherwise ``pow2_ceil(16 * nproc)``; must be a power of 2)
|
||||
|
||||
This setting controls:
|
||||
|
||||
* Thread ID manager capacity (maximum thread IDs that can be tracked)
|
||||
* Storage array sizes for thread-local data across the codebase
|
||||
* Timemory's internal thread storage (``TIMEMORY_MAX_THREADS``)
|
||||
|
||||
**Build-time validation:**
|
||||
|
||||
CMake enforces that ``ROCPROFSYS_MAX_THREADS`` must be a power of 2:
|
||||
|
||||
.. code-block:: cmake
|
||||
|
||||
# Valid: 16, 32, 64, 128, 256, 512, 1024, 2048, 4096, 8192, 16384, ... any power of 2
|
||||
# Invalid: 100, 3000, 5000, 10000, ... (FATAL_ERROR)
|
||||
**Example: Building with custom thread limit**
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
# Build with support for 8192 threads
|
||||
cmake -B build \
|
||||
-DROCPROFSYS_MAX_THREADS=8192 \
|
||||
..
|
||||
cmake --build build
|
||||
|
||||
Sampling model
|
||||
========================================
|
||||
|
||||
Reference in New Issue
Block a user