From 6195f65f9e377b85c2d1dd44e75efe9b7ae3ebc8 Mon Sep 17 00:00:00 2001 From: Adel Johar Date: Thu, 13 Mar 2025 10:43:26 +0100 Subject: [PATCH] Docs: Add more variables to env_variables.rst [ROCm/ROCR-Runtime commit: d8d27d4fd66cdcbe52a9506f2992a05ac90b12ad] --- .../runtime/docs/data/env_variables.rst | 79 ++++++++++++++++++- 1 file changed, 76 insertions(+), 3 deletions(-) diff --git a/projects/rocr-runtime/runtime/docs/data/env_variables.rst b/projects/rocr-runtime/runtime/docs/data/env_variables.rst index c9da316822..4ae0c3e9a4 100644 --- a/projects/rocr-runtime/runtime/docs/data/env_variables.rst +++ b/projects/rocr-runtime/runtime/docs/data/env_variables.rst @@ -7,9 +7,9 @@ :header-rows: 1 :widths: 35,14,51 - * - **Environment variable** - - **Default value** - - **Value** + * - Environment variable + - Default value + - Value * - | ``ROCR_VISIBLE_DEVICES`` | Specifies a list of device indices or UUIDs to be exposed to the applications. @@ -30,6 +30,18 @@ - ``146800640`` - 0 to 4GB per XCC + * - | ``HSA_SCRATCH_SINGLE_LIMIT_ASYNC`` + | On GPUs that support asynchronous scratch reclaim, this variable is used instead of ``HSA_SCRATCH_SINGLE_LIMIT`` to specify the threshold for scratch memory allocation. + - ``3221225472`` (3GB) + - 0 to 4GB per XCC + + * - | ``HSA_ENABLE_SCRATCH_ASYNC_RECLAIM`` + | Controls asynchronous scratch memory reclamation on supported GPUs. + | When enabled, if a device memory allocation fails, ROCr will attempt to reclaim scratch memory assigned to all queues and retry the allocation. + - ``1`` + - | 0: Disable asynchronous scratch reclaim. + | 1: Enable asynchronous scratch reclaim on supported GPUs. + * - | ``HSA_XNACK`` | Enables XNACK. - None @@ -66,3 +78,64 @@ - ``0`` - | 0: Disable | 1: Enable + + * - | ``HSA_OVERRIDE_CPU_AFFINITY_DEBUG`` + | Controls whether ROCm helper threads inherit the parent process's CPU affinity mask. + - ``1`` + - | 0: Enable inheritance. Helper threads use the parent process's core affinity mask, which should be set with enough cores for all threads. + | 1: Disable inheritance. Helper threads spawn on all available cores, ignoring the parent's affinity settings, which may affect performance in certain environments. + + * - | ``HSA_ENABLE_DEBUG`` + | Enables additional debug information and validation in the runtime. + - ``0`` + - | 0: Disable debug mode. + | 1: Enable debug mode with additional validation and logging. + + +Hardware Debugging Environment Variables +---------------------------------------- + +The following environment variables are intended for experienced users who are debugging hardware-specific issues. +These settings may impact performance and stability and should only be used when troubleshooting specific hardware problems. + +.. _rocr-debug-env: +.. list-table:: + :header-rows: 1 + :widths: 35,14,51 + + * - Environment variable + - Default value + - Value + + * - | ``HSA_DISABLE_FRAGMENT_ALLOCATOR`` + | Disables internal memory fragment caching to help debug memory faults. + - ``0`` + - | 0: Fragment allocator enabled (normal operation). + | 1: Fragment allocator disabled. Helps debug tools identify memory faults at their origin by preventing cached memory blocks from masking out-of-bounds writes. + + * - | ``HSAKMT_DEBUG_LEVEL`` + | Controls the verbosity level of debug messages from the ``libhsakmt.so`` driver layer. + - ``3`` + - | 3: Only error messages (``pr_err``) are printed. + | 4: Error and warning messages (``pr_err``, ``pr_warn``) are printed. + | 5: Same as level 4 (notice level not implemented). + | 6: Error, warning, and info messages (``pr_err``, ``pr_warn``, ``pr_info``) are printed. + | 7: All debug messages including ``pr_debug`` are printed. + + * - | ``HSA_ENABLE_INTERRUPT`` + | Controls how completion signals are detected, useful for diagnosing interrupt storm issues. + - ``1`` + - | 0: Disable hardware interrupts. Uses memory-based polling for completion signals instead of interrupts. + | 1: Enable hardware interrupts (normal operation). + + * - | ``HSA_SVM_GUARD_PAGES`` + | Controls the use of guard pages in Shared Virtual Memory (SVM) allocations. + - ``1`` + - | 0: Disable SVM guard pages (for debugging memory access patterns). + | 1: Enable SVM guard pages (normal operation). + + * - | ``HSA_DISABLE_CACHE`` + | Controls GPU L2 cache utilization for all memory regions. + - ``0`` + - | 0: Normal caching behavior (L2 cache enabled). + | 1: Disables L2 cache entirely. Sets all memory regions as uncacheable (MTYPE=UC) in the GPU, bypassing the L2 cache. Useful for diagnosing cache-related performance or correctness issues.