ROCR Runtime

This ROCm Runtime (ROCr) repo combines 2 previously separate repos into a single repo:

• The HSA Runtime (hsa-runtime) for AMD GPU application development and
• The ROCt Thunk Library (libhsakmt), a "thunk" interface to the ROCm kernel driver (ROCk), used by the runtime.

Infrastructure

The HSA runtime is a thin, user-mode API that exposes the necessary interfaces to access and interact with graphics hardware driven by the AMDGPU driver set and the ROCK kernel driver. Together they enable programmers to directly harness the power of AMD discrete graphics devices by allowing host applications to launch compute kernels directly to the graphics hardware.

The capabilities expressed by the HSA Runtime API are:

• Error handling
• Runtime initialization and shutdown
• System and agent information
• Signals and synchronization
• Architected dispatch
• Memory management
• HSA runtime fits into a typical software architecture stack.

The HSA runtime provides direct access to the graphics hardware to give the programmer more control of the execution. An example of low level hardware access is the support of one or more user mode queues provides programmers with a low-latency kernel dispatch interface, allowing them to develop customized dispatch algorithms specific to their application.

The HSA Architected Queuing Language is an open standard, defined by the HSA Foundation, specifying the packet syntax used to control supported AMD/ATI Radeon (c) graphics devices. The AQL language supports several packet types, including packets that can command the hardware to automatically resolve inter-packet dependencies (barrier AND & barrier OR packet), kernel dispatch packets and agent dispatch packets.

In addition to user mode queues and AQL, the HSA runtime exposes various virtual address ranges that can be accessed by one or more of the system's graphics devices, and possibly the host. The exposed virtual address ranges either support a fine grained or a coarse grained access. Updates to memory in a fine grained region are immediately visible to all devices that can access it, but only one device can have access to a coarse grained allocation at a time. Ownership of a coarse grained region can be changed using the HSA runtime memory APIs, but this transfer of ownership must be explicitly done by the host application.

Programmers should consult the HSA Runtime Programmer's Reference Manual for a full description of the HSA Runtime APIs, AQL and the HSA memory policy.

Known issues

• Each HSA process creates an internal DMA queue, but there is a system-wide limit of four DMA queues. When the limit is reached HSA processes will use internal kernels for copies.

Artifacts produced by the build

• libhsakmt (ROCt) - User-mode API interfaces for interacting with the ROCk driver
• Runtime (ROCr) - Core runtime supporting HSA standards
• rocrtst - Runtime test suites for HSA implementation validation and performance testing
• kfdtest - Validation tests for ROCt

Building the ROCR Runtime

Target platform requirements

Please see the ROCm System requirements (Linux).

Ensure you have the following installed:

• CMake 3.7 or higher
• libelf-dev
• g++
• libdrm-amdgpu-dev or libdrm-dev
• rocm-core

ROCr & ROCt Build Instructions

1. Clone this repository and cd into its root
2. Prepare the build directory

   mkdir build && cd build
   

3. Configure the build (example)

   cmake -DCMAKE_INSTALL_PREFIX=/opt/<rocm install dir>/llvm ..
   

4. Compile the project

   make
   

5. Install the runtime

   make install
   

6. (Optional) Build packages

   make package
   

Non-default CMake Build Options

• Produce a release build instead of debug

  -DCMAKE_BUILD_TYPE=Release
  

• Control whether libhsakmt and libhsa-runtime are shared or static The default (when BUILD_SHARED_LIBS is unset) is for libhsakmt to be static and libhsa-runtime to be shared. Setting BUILD_SHARED_LIBS to OFF will make both static and setting it to ON will make them both shared.

  -DBUILD_SHARED_LIBS=OFF   # or ON for both libs shared
  

Building the tests

rocrtst

1. Go to rocrtst root

   cd <rocr-runtime>/rocrtst/suites/test_common
   

2. Prepare the build directory

   mkdir build && cd build
   

3. Configure the build Example configuration:

   cmake \
       -DCMAKE_PREFIX_PATH="<rocm install root>;<llvm install root>" \
       -DROCM_DIR="$ROCM_INSTALL_PATH" \
       -DOPENCL_DIR="<rocm install root>" \
       ..
   

4. Compile the project

   make
   make rocrtst_kernels
   

5. ** Run the tests Make sure libhsa-runtime.so is in the library path; e.g.,

   $ LD_LIBRARY_PATH=<rocm install root> ./rocrtst -h    # See help options
   

kfdtest

1. Go to kfdtest root

   cd <rocr-runtime>/libhsakmt/tests/kfdtest
   

2. Prepare the build directory

   mkdir build && cd build
   

3. Configure the build Example configuration:

   cmake \
       -DCMAKE_PREFIX_PATH="<rocm install root>" \
       -DROCM_DIR="$ROCM_INSTALL_PATH" \
       ..
   

4. Compile the project

   make
   

Using the ROCR Runtime

After installation, you can link against the runtime by using the provided CMake package configurations. For example, to use the ROCR runtime in your project:

find_package(hsa-runtime64 1.0 REQUIRED)
add_executable(MyApp main.cpp)
target_link_libraries(MyApp PRIVATE hsa-runtime64::hsa-runtime64)

Disclaimer

The information contained herein is for informational purposes only, and is subject to change without notice. While every precaution has been taken in the preparation of this document, it may contain technical inaccuracies, omissions and typographical errors, and AMD is under no obligation to update or otherwise correct this information. Advanced Micro Devices, Inc. makes no representations or warranties with respect to the accuracy or completeness of the contents of this document, and assumes no liability of any kind, including the implied warranties of noninfringement, merchantability or fitness for particular purposes, with respect to the operation or use of AMD hardware, software or other products described herein. No license, including implied or arising by estoppel, to any intellectual property rights is granted by this document. Terms and limitations applicable to the purchase or use of AMD's products are as set forth in a signed agreement between the parties or in AMD's Standard Terms and Conditions of Sale.

AMD, the AMD Arrow logo, and combinations thereof are trademarks of Advanced Micro Devices, Inc. Other product names used in this publication are for identification purposes only and may be trademarks of their respective companies.

Copyright © 2014-2024 Advanced Micro Devices, Inc. All rights reserved.