diff --git a/README.md b/README.md index b37dbd872d..9ea43315c4 100644 --- a/README.md +++ b/README.md @@ -65,9 +65,9 @@ Please report in the Github Issues. - Individual XCC mode is not supported. - By default, PC sampling API is disabled. To use PC sampling. Setting the `ROCPROFILER_PC_SAMPLING_BETA_ENABLED` environment variable grants access to the PC Sampling experimental beta feature. This feature is still under development and may not be completely stable. - - **Risk Acknowledgment**: By activating this environment variable, you acknowledge and accept the following potential risks: - - **Hardware Freeze**: This beta feature could cause your hardware to freeze unexpectedly. - - **Need for Cold Restart**: In the event of a hardware freeze, you may need to perform a cold restart (turning the hardware off and on) to restore normal operations. + - **Risk Acknowledgment**: By activating this environment variable, you acknowledge and accept the following potential risks: + - **Hardware Freeze**: This beta feature could cause your hardware to freeze unexpectedly. + - **Need for Cold Restart**: In the event of a hardware freeze, you may need to perform a cold restart (turning the hardware off and on) to restore normal operations. Please use this beta feature cautiously. It may affect your system's stability and performance. Proceed at your own risk. - At this point, We do not recommend stress-testing the beta implementation. diff --git a/source/docs/api-reference/buffered_services.md b/source/docs/api-reference/buffered_services.md index f6a7eead6b..8f452853a3 100644 --- a/source/docs/api-reference/buffered_services.md +++ b/source/docs/api-reference/buffered_services.md @@ -4,7 +4,7 @@ For the buffered approach, supported buffer record categories are enumerated in ## Overview -In buffered approach, callbacks are receieved for batches of records from an internal (background) thread. +In buffered approach, callbacks are received for batches of records from an internal (background) thread. Supported buffered tracing services are enumerated in `rocprofiler_buffer_tracing_kind_t`. Configuring a buffer tracing service requires the creation of a buffer. When the buffer is "flushed", either implicitly or explicitly, a callback to the tool will be invoked which provides an array of one or more buffer records. diff --git a/source/docs/api-reference/callback_services.md b/source/docs/api-reference/callback_services.md index 1a4584901c..7bcb4d60ca 100644 --- a/source/docs/api-reference/callback_services.md +++ b/source/docs/api-reference/callback_services.md @@ -273,14 +273,14 @@ Sample Output: 2: rocprofiler_dim3_t dimBlocks = {z=1, y=32, x=32} 3: void** args = 0x7ffe6d8dd3c0 4: unsigned long sharedMemBytes = 0 - 5: ihipStream_t* stream = 0x17b40c0 + 5: hipStream_t* stream = 0x17b40c0 [HIP_RUNTIME_API] hipMemcpyAsync 0: void* dst = 0x7f06c7bbb010 1: void const* src = 0x7f0698800000 2: unsigned long sizeBytes = 393625600 3: hipMemcpyKind kind = DeviceToHost - 4: ihipStream_t* stream = 0x25dfcf0 + 4: hipStream_t* stream = 0x25dfcf0 ``` ## Code Object Tracing @@ -334,4 +334,4 @@ any/all buffers which might contain references to that code object or kernel sym deleting the associated data. For a sample of code object tracing, please see the `samples/code_object_tracing` example in the -[rocprofiler-sdk GitHub repository](https://github.com/ROCm/rocproifler-sdk). +[rocprofiler-sdk GitHub repository](https://github.com/ROCm/rocprofiler-sdk). diff --git a/source/docs/api-reference/counter_collection_services.md b/source/docs/api-reference/counter_collection_services.md index a7f58b5906..6d640aa053 100644 --- a/source/docs/api-reference/counter_collection_services.md +++ b/source/docs/api-reference/counter_collection_services.md @@ -128,7 +128,7 @@ rocprofiler_counter_id_t is a handle to a counter. The information about the cou for(auto& counter : gpu_counters) { // Contains name and other attributes about the counter. - // See API documenation for more info on the contents of this struct. + // See API documentation for more info on the contents of this struct. rocprofiler_counter_info_v0_t version; ROCPROFILER_CALL( rocprofiler_query_counter_info( @@ -150,6 +150,7 @@ After you have identified a set of counters you wish to collect, a profile can b The created profile can in turn be used for both dispatch and agent counter collection services. ##### Special Notes On Profile Behavior + - Profile created is *only valid* for the agent it was created for. - Profiles are immutable. If a new counter set is desired to be collected, construct a new profile. - A single profile can be used multiple times on the same agent. @@ -234,18 +235,18 @@ counter_name: # Counter name gfx90a: # Architecture name block: # Block information (SQ/etc) event: # Event ID (used by AQLProfile to identify counter register) - expression: # Formula for the counter (if derrived counter) + expression: # Formula for the counter (if derived counter) description: # Per-arch description (optional) gfx1010: ... description: # Description of the counter ``` -Architectures can be separately defined with their own definitions (i.e. gfx90a and gfx1010 in the above example). If two or more architectures share the same block/event/expression definition, they can be "/" delimited on a single line (i.e. "gfx90a/gfx1010:"). Hardware metrics have the elements block, event, and description defined. Derrived metrics have the element expression defined (and cannot have block or event defined). +Architectures can be separately defined with their own definitions (i.e. gfx90a and gfx1010 in the above example). If two or more architectures share the same block/event/expression definition, they can be "/" delimited on a single line (i.e. "gfx90a/gfx1010:"). Hardware metrics have the elements block, event, and description defined. Derived metrics have the element expression defined (and cannot have block or event defined). ## Derived Metrics -Derrived metrics allow for computations (via expressions) to be performed on collected hardware metrics with the result returned as it it were a real hardware counter. +Derived metrics allow for computations (via expressions) to be performed on collected hardware metrics with the result returned as it it were a real hardware counter. ```yaml GPU_UTIL: @@ -255,7 +256,7 @@ GPU_UTIL: description: Percentage of the time that GUI is active ``` -GPU_UTIL is an example of a derrived metric which takes the values of two GRBM hardware counters (GRBM_GUI_ACTIVE and GRBM_COUNT) and uses a mathematic expression to calculate the utilization rate of the GPU. Expressions support the standard set of math operators (/,*,-,+) along with a set of special functions (reduce and accumulate). +GPU_UTIL is an example of a derived metric which takes the values of two GRBM hardware counters (GRBM_GUI_ACTIVE and GRBM_COUNT) and uses a mathematic expression to calculate the utilization rate of the GPU. Expressions support the standard set of math operators (/,*,-,+) along with a set of special functions (reduce and accumulate). ### Reduce Function @@ -266,16 +267,20 @@ expression: 100*reduce(GL2C_HIT,sum)/(reduce(GL2C_HIT,sum)+reduce(GL2C_MISS,sum) Reduce() reduces counter values across all dimensions (shader engine, SIMD, etc) to produce a single output value. This is useful when you want to collect and compare values across the entire device. There are a number of reduction operations that can be perfomed: sum, average (avr), minimum value (selects minimum value across all dimensions, min), and max (selects the maximum value across all dimensions). For example reduce(GL2C_HIT,sum) sums all GL2C_HIT hardware register values together to return a single output value. ### Accumulate Function + ```yaml expression: accumulate(, ) ``` + #### Description + - The accumulate metric is used to sum the values of a basic level counter over a specified number of cycles. By setting the resolution parameter, you can control the frequency of the summing operation: - - HIGH_RES: Sums up the basic counter every clock cycle. Captures the value every single cycle for higher accuracy, suitable for fine-grained analysis. - - LOW_RES: Sums up the basic counter every four clock cycles. Reduces the data points and provides less detailed summing, useful for reducing data volume. - - NONE: Does nothing and is equivalent to collecting basic_level_counter. Outputs the value of the basic counter without any summing operation. + - HIGH_RES: Sums up the basic counter every clock cycle. Captures the value every single cycle for higher accuracy, suitable for fine-grained analysis. + - LOW_RES: Sums up the basic counter every four clock cycles. Reduces the data points and provides less detailed summing, useful for reducing data volume. + - NONE: Does nothing and is equivalent to collecting basic_level_counter. Outputs the value of the basic counter without any summing operation. #### Usage + ```yaml MeanOccupancyPerCU: architectures: @@ -283,5 +288,7 @@ MeanOccupancyPerCU: expression: accumulate(SQ_LEVEL_WAVES,HIGH_RES)/reduce(GRBM_GUI_ACTIVE,max)/CU_NUM description: Mean occupancy per compute unit. ``` - + + + - MeanOccupancyPerCU: This metric calculates the mean occupancy per compute unit. It uses the accumulate function with HIGH_RES to sum the SQ_LEVEL_WAVES counter at every clock cycle. This sum is then divided by GRBM_GUI_ACTIVE and the number of compute units (CU_NUM) to derive the mean occupancy. diff --git a/source/docs/api-reference/intercept_table.md b/source/docs/api-reference/intercept_table.md index 58cdc745cb..bce3dc04e0 100644 --- a/source/docs/api-reference/intercept_table.md +++ b/source/docs/api-reference/intercept_table.md @@ -65,7 +65,7 @@ get_dispatch_table() } // namespace impl ``` -### Implementaiton of public C API function +### Implementation of public C API function ```cpp extern "C" diff --git a/source/docs/api-reference/pc_sampling.md b/source/docs/api-reference/pc_sampling.md index a75cf03ee6..cadbbe0561 100644 --- a/source/docs/api-reference/pc_sampling.md +++ b/source/docs/api-reference/pc_sampling.md @@ -5,10 +5,10 @@ PC Sampling is a profiling method that uses statistical approximation of the ker **Note**: The PC sampling feature is still under development and may not be completely stable. **Risk Acknowledgment**: - - - By activating this feature through `ROCPROFILER_PC_SAMPLING_BETA_ENABLED` environment variable, you acknowledge and accept the following potential risks: - - - **Hardware Freeze**: This beta feature could cause your hardware to freeze unexpectedly. - - **Need for Cold Restart**: In the event of a hardware freeze, you may need to perform a cold restart (turning the hardware off and on) to restore normal operations. - + + By activating this feature through `ROCPROFILER_PC_SAMPLING_BETA_ENABLED` environment variable, you acknowledge and accept the following potential risks: + +- **Hardware Freeze**: This beta feature could cause your hardware to freeze unexpectedly. +- **Need for Cold Restart**: In the event of a hardware freeze, you may need to perform a cold restart (turning the hardware off and on) to restore normal operations. + Please use this beta feature cautiously. It may affect your system's stability and performance. Proceed at your own risk. diff --git a/source/docs/api-reference/tool_library.md b/source/docs/api-reference/tool_library.md index 0d63f5a547..063960e00c 100644 --- a/source/docs/api-reference/tool_library.md +++ b/source/docs/api-reference/tool_library.md @@ -9,6 +9,7 @@ The ROCm runtimes are designed to directly communicate with a helper library nam ## Tool library design When ROCprofiler-SDK detects `rocprofiler_configure` in a tool's symbol table, ROCprofiler-SDK invokes `rocprofiler-configure` with parameters such as ROCprofiler-SDK version that invokes the function, number of tools already invoked, and a unique identifier for the tool. The tool returns a pointer to a `rocprofiler_tool_configure_result_t` struct, which, if non-null, provides ROCprofiler-SDK with: + - Function to be called for tool initialization, which is also the opportunity for context creation. - Function to be called when ROCprofiler-SDK is finalized. - A pointer to data to be provided to the tool when ROCprofiler-SDK calls the initialization and finalization functions. diff --git a/source/docs/data/kernel_trace.csv b/source/docs/data/kernel_trace.csv index a39956ff37..b0725387aa 100644 --- a/source/docs/data/kernel_trace.csv +++ b/source/docs/data/kernel_trace.csv @@ -1,2 +1,9 @@ -"Kind","Agent_Id","Queue_Id","Kernel_Id","Kernel_Name","Correlation_Id","Start_Timestamp","End_Timestamp","Private_Segment_Size","Group_Segment_Size","Workgroup_Size_X","Workgroup_Size_Y","Workgroup_Size_Z","Grid_Size_X","Grid_Size_Y","Grid_Size_Z" -"KERNEL_DISPATCH",1,139690710949888,15,"matrixTranspose(float*, float*, int)",0,671599758568,671599825328,0,0,4,4,1,1024,1024,1 +"Kind","Agent_Id","Queue_Id","Thread_Id","Dispatch_Id","Kernel_Id","Kernel_Name","Correlation_Id","Start_Timestamp","End_Timestamp","Private_Segment_Size","Group_Segment_Size","Workgroup_Size_X","Workgroup_Size_Y","Workgroup_Size_Z","Grid_Size_X","Grid_Size_Y","Grid_Size_Z" +"KERNEL_DISPATCH",1,1,69,1,16,"void addition_kernel(float*, float const*, float const*, int, int)",1451,8819330200067564,8819330200116308,0,0,64,1,1,1024,1024,1 +"KERNEL_DISPATCH",1,2,69,5,16,"void addition_kernel(float*, float const*, float const*, int, int)",1484,8819330200118678,8819330200219573,0,0,64,1,1,1024,1024,1 +"KERNEL_DISPATCH",1,1,69,2,19,"subtract_kernel(float*, float const*, float const*, int, int)",1459,8819330200120456,8819330200223721,0,0,64,1,1,1024,1024,1 +"KERNEL_DISPATCH",1,3,69,9,16,"void addition_kernel(float*, float const*, float const*, int, int)",1517,8819330200152902,8819330200283428,0,0,64,1,1,1024,1024,1 +"KERNEL_DISPATCH",1,4,69,13,16,"void addition_kernel(float*, float const*, float const*, int, int)",1550,8819330200187127,8819330200320468,0,0,64,1,1,1024,1024,1 +"KERNEL_DISPATCH",1,2,69,6,19,"subtract_kernel(float*, float const*, float const*, int, int)",1492,8819330200225499,8819330200364618,0,0,64,1,1,1024,1024,1 +"KERNEL_DISPATCH",1,1,69,3,18,"multiply_kernel(float*, float const*, float const*, int, int)",1467,8819330200229796,8819330200369359,0,0,64,1,1,1024,1024,1 + diff --git a/source/docs/how-to/samples.md b/source/docs/how-to/samples.md index 3d6140cfbf..d8e2b0c850 100644 --- a/source/docs/how-to/samples.md +++ b/source/docs/how-to/samples.md @@ -11,6 +11,7 @@ The ROCm installation provides sample programs and `rocprofv3` tool. ```bash /opt/rocm/share/rocprofiler-sdk/samples ``` + - `rocprofv3` tool is installed here: ```bash diff --git a/source/docs/how-to/using-rocprofv3.rst b/source/docs/how-to/using-rocprofv3.rst index f8428c8723..2d7a2e1772 100644 --- a/source/docs/how-to/using-rocprofv3.rst +++ b/source/docs/how-to/using-rocprofv3.rst @@ -67,7 +67,7 @@ Here is the list of ``rocprofv3`` command-line options. Some options are used fo - Application tracing * - ``--hsa-image-trace`` - - Collects HSA API Ttaces (Image-extension API). + - Collects HSA API Traces (Image-extension API). - Application tracing * - ``--hsa-finalizer-trace`` @@ -303,7 +303,7 @@ Here are the contents of `kernel_trace.csv` file: .. csv-table:: Kernel trace :file: /data/kernel_trace.csv - :widths: 10,10,10,10,10,10,20,20,10,10,10,10,10,10,10,10 + :widths: 10,10,10,10,10,10,10,10,20,20,10,10,10,10,10,10,10,10 :header-rows: 1 For the description of the fields in the output file, see :ref:`output-file-fields`. @@ -346,7 +346,7 @@ Running the above command generates `hip_api_trace.csv`, `hsa_api_trace.csv`, `k Scratch memory trace ++++++++++++++++++++++ -This option collects scratch memory operation's traces. Scratch is an address space on AMDGPUs, which is roughly equivalent to the `local memory` in NVIDIA CUDA. The `local memory` in CUDA is a thread-local global memory with interleaved addressing, which is used for register spills or stack space. With this option, you can trace when the ``rocr`` runtime allocates, frees, and tries to reclaim scratch memory. +This option collects scratch memory operation's traces. Scratch is an address space on AMD GPUs, which is roughly equivalent to the `local memory` in NVIDIA CUDA. The `local memory` in CUDA is a thread-local global memory with interleaved addressing, which is used for register spills or stack space. With this option, you can trace when the ``rocr`` runtime allocates, frees, and tries to reclaim scratch memory. .. code-block:: shell @@ -433,7 +433,7 @@ Properties - **``hsa_finalize_trace``** *(boolean)*: For Collecting HSA API Traces (Finalizer-extension API). - **``hsa_image_trace``** *(boolean)*: For Collecting HSA API - Traces (Image-extenson API). + Traces (Image-extension API). - **``sys_trace``** *(boolean)*: For Collecting HIP, HSA, Marker (ROCTx), Memory copy, Scratch memory, and Kernel dispatch traces. diff --git a/source/docs/install/installation.md b/source/docs/install/installation.md index f4150ae38e..42e8ec2389 100644 --- a/source/docs/install/installation.md +++ b/source/docs/install/installation.md @@ -41,6 +41,7 @@ If the `CMake` installed on the system is too old, you can install a new version pip install --user 'cmake==3.22.0' export PATH=${HOME}/.local/bin:${PATH} ``` + ## Building ROCprofiler-SDK ```bash @@ -69,4 +70,4 @@ To run the built tests, `cd` into the `rocprofiler-sdk-build` directory and run: ```bash ctest --output-on-failure -O ctest.all.log -``` \ No newline at end of file +```