* rocrtst: Reduce host memory limit to 70%
Reducing the upper bound for rocrtstFunc.Memory_Max_Mem to 70% from
90% to help reduce test execution time.
Signed-off-by: Sunday Clement <Sunday.Clement@amd.com>
* rocrtst: Add ROCRTST_LIMIT_POOL_SIZE env var
Add environment variable to override the memory pool sizes when running
tests.
Co-authored-by: David Yat Sin <David.YatSin@amd.com>
---------
Signed-off-by: Sunday Clement <Sunday.Clement@amd.com>
Co-authored-by: David Yat Sin <David.YatSin@amd.com>
* libhsakmt: Update hsakmt_fmm_get_handle to support address range
Currently, hsakmt_fmm_get_handle works only if the address is allocated
(staring) value. Update it so it can find the handle if address falls in
the valid allocated range. This is useful for AMD infinity storage
feature where data needs to be transferred to any memory within in the
allocated range
Signed-off-by: Harish Kasiviswanathan <Harish.Kasiviswanathan@amd.com>
* libhsakmt: Introduce AMD Infinity Storage (AIS) API
Add hsaKmtAisReadWriteFile() API to support AMD Infinity Storage. The
API moves data directly from GPU VRAM to a file.
v2: Add in/out ioctl arguments to provide more status information to
user space. Modify hsaKmt API also accordingly.
Signed-off-by: Harish Kasiviswanathan <Harish.Kasiviswanathan@amd.com>
* rocr: Initial implementation of AMD Infinity Storage (AIS)
Implement first two API: hsa_amd_ais_file_write and hsa_amd_ais_file_read
v2: Change API from hsa_amd_ to hsa_amd_ais_
Change API to take in handle instead of fd for compatibility accross
different platforms
Original Author: Chris Freehill <Chris.Freehill@amd.com>
Signed-off-by: Harish Kasiviswanathan <Harish.Kasiviswanathan@amd.com>
---------
Signed-off-by: Harish Kasiviswanathan <Harish.Kasiviswanathan@amd.com>
Because the base QueueWrapper class copies the wrapped queue's
amd_queue_v2_t queue descriptor struct the QueueProxy seems
superfluous as it will have the same effect as calling the
underlying methods on the wrapped queue itself.
Additionally, because the QueueProxy needs to access the wrapped
queue's queue descriptor it breaks the Queue API which is meant
to abstract the underlying agent's queue implementation.
This makes it easier to generalize the core::Queue as well as
the InterceptQueue.
Signed-off-by: Tony Gutierrez <anthony.gutierrez@amd.com>
Make sure ROCR can be compiled under windows. Extra setup for the windows build environment is required. The change should not have any functional changes under Linux.
* libhsakmt: fix UB due to signed integer literal in 1 << 31
Bit shift operations on signed numbers should not shift into or beyond
the signed bit as this results in Undefined Behaviour.
Signed-off-by: Sunday Clement <Sunday.Clement@amd.com>
* libhsakmt: Fix UB due to signed integer literal in 1 << x
Bit Shifting an unsigned integer is undefined behavior.
BUG: SWDEV-532853
Signed-off-by: Sunday Clement <Sunday.Clement@amd.com>
* rocr: Fix UB in various places due signed integer in bit shift
Bit shifting signed integers into or beyond the sign bit is undefined.
Signed-off-by: Sunday Clement <Sunday.Clement@amd.com>
* rocr: Change signed integer literals to unsigned
Changing the signed integers in the macro expressions throughout the file
to avoid overflow.
Signed-off-by: Sunday Clement <Sunday.Clement@amd.com>
---------
Signed-off-by: Sunday Clement <Sunday.Clement@amd.com>
Co-authored-by: Flora Cui <flora.cui@amd.com>
* rocr: Fix Incorrect Assertion Check
The wrong variable is used in the assertion statement, should be error
checking for the value of paramEndLoc after it is modified by the call
to find().
Signed-off-by: Sunday Clement <Sunday.Clement@amd.com>
* rocr: Fix Potential Undefined Behaviour
In the event that the SvmProfileControl destructor is called and
event == -1 is true then the call to close(event) is effectively
close(-1) which is undefined behaviour. This has been changed to only
call close() on valid file descriptors.
Signed-off-by: Sunday Clement <Sunday.Clement@amd.com>
* rocr: Add Error Check on Bytes Read
In the case that there is an incomplete read the call to copyTo() will
now return an error.
Signed-off-by: Sunday Clement <Sunday.Clement@amd.com>
* rocr: Fix Exception Error
Destructors are implicitly marked with noexcept being true by default
so if its not explicitly marked false in the destructor or the
functions it calls, any thrown exceptions will cause the program to
crash.
Signed-off-by: Sunday Clement <Sunday.Clement@amd.com>
---------
Signed-off-by: Sunday Clement <Sunday.Clement@amd.com>
Co-authored-by: Sunday Clement <Sunday.Clement@amd.com>
InterceptQueue::Submit had an "all-or-nothing" packet submission policy that
could cause infinite retry loops when the number of packets to submit exceeded
the available queue slots. When 504+ packets needed submission to a ~500-slot
queue, the system would:
1. Set submitted_count=0 (submit nothing)
2. Add retry barrier packet
3. Trigger async handler via StoreRelaxed
4. Attempt to submit overflow packets
5. Fail again due to same space constraints
6. Repeat
Solution:
Added partial packet submission capability during overflow processing while
preserving the original "all-or-nothing" behavior for normal operations.
When processing overflow packets and insufficient space exists for all packets,
the system now submits as many packets as possible rather than none.
The fix:
- Detects overflow processing via !overflow_.empty()
- Allows partial submission: submitted_count = free_slots - barrier_reservation
- Maintains atomicity guarantees for normal packet rewrites
- Prevents infinite retry loops by ensuring forward progress
This resolves deadlocks in high-throughput scenarios while maintaining
backward compatibility and the original design intent for packet rewrite
atomicity.
New test that does a memory_copy, and right after has the shader access
the data. This verifies that the memory is coherent and that all the
probes and flushes were done correctly by the memory_copy.
Signed-off-by: Sunday Clement <Sunday.Clement@amd.com>
* libhsakmt: Update ioctl version to 1.18
Sync with kernel ioctl version.
Also explicitly set the ioctl flag to KFD_PROC_FLAG_MFMA_HIGH_PRECISION
Signed-off-by: Harish Kasiviswanathan <Harish.Kasiviswanathan@amd.com>
* libhsakmt: Sync ioctl header by adding kfd_ioctl_profiler
Sync with kernel ioctl version. Add kfd_ioctl_profiler.
Signed-off-by: Harish Kasiviswanathan <Harish.Kasiviswanathan@amd.com>
---------
Signed-off-by: Harish Kasiviswanathan <Harish.Kasiviswanathan@amd.com>
When stochastic sampling is not active, the trap handler is incorrectly
branching to .check_exceptions, bypassing the software trap ID checks
and inturn not advancing the PC. Fixed the issue to always check software
traps regardless of PC sampling state.
Co-authored-by: Shweta Khatri <shweta.khatri@amd.com>
This commit reverts the following related commits which cause
test failures:
6d15779b3e rocr/driver: add PC sampling support to driver interface
56cb9390ff rocr/driver: add PC sampling support to driver interface
76bf829f09 rocr/driver: add ASAN header page management to Driver class
a47c060d6a rocr/driver: add ASAN header page management to Driver class
02d7eaf3b7 rocr: add memory sharing call to Driver interface
9312468655 rocr: add memory sharing call to Driver interface
Fix regression when hsa_amd_pointer_info is called on a pointer that was
allocated using non-VMM APIs. The helper function VMemoryPtrInfo should
return error when the address is not found so that PtrInfo does the
lookup via Thunk.
Also advice parameter of madvise() system call is not a bitmask. So fix
that also
v2: Use MAP_SHARED instead of MAP_PRIVATE. This avoids MMU notifiers and
evictions.
Signed-off-by: Harish Kasiviswanathan <Harish.Kasiviswanathan@amd.com>
Previously regardless of hsa_agent passed the prefetch is always driven
to node 0, now the agent of interest may be properly prefetched.
Signed-off-by: Sunday Clement <Sunday.Clement@amd.com>
Co-authored-by: Sunday Clement <Sunday.Clement@amd.com>
Co-authored-by: systems-assistant[bot] <systems-assistant[bot]@users.noreply.github.com>
* rocr: hsa_amd_pointer_info to support VMEM pointers
Extend hsa_amd_pointer_info to support virtual memory addresses.
If hsa_amd_pointer_info is called on an address that is reserved but not
mapped to memory, then the pointer type will be reported as
HSA_EXT_POINTER_TYPE_RESERVED_ADDR.
If hsa_amd_pointer_info is called on an address that is mapped, then the
pointer type will be reported as HSA_EXT_POINTER_TYPE_HSA_VMEM
* rocrtst: VirtMemory_Basic_Test test for pointer info
Extend rocrtstFunc.VirtMemory_Basic_Test to test for
hsa_amd_pointer_info
* rocrtst: Add SVM Memory Test
There is an issue with TheRock build currently. They have
a local source build of elfutils they want to use instead
of a system package. Currently, rocr uses it's own
FindLibElf.cmake module and this is inhibiting the build
from finding the libelf config built by TheRock.
Now we will first search in config mode and fallback to
module mode if nothing is found.
Authored-by: Ethan Stewart <ethan.stewart@amd.com>
Co-authored-by: systems-assistant[bot] <systems-assistant[bot]@users.noreply.github.com>
Format packet.workgroup_size_x correctly as a size_t.
Format packet.workgroup_size_y correctly as a size_t.
Format packet.workgroup_size_z correctly as a size_t.
Format packet.grid_size_x correctly as a size_t.
Format packet.grid_size_y correctly as a size_t.
Format packet.grid_size_z correctly as a size_t.
Format packet.group_segment_size correctly as a size_t.
Format packet.private_segment_size correctly as a size_t.
Format barrier_packet.completion_signal correctly as an address using %zx.
Format barrier_packet.dep_signal[0] correctly as an address using %zx.
Format barrier_packet.dep_signal[1] correctly as an address using %zx.
Format barrier_packet.dep_signal[2] correctly as an address using %zx.
Format barrier_packet.dep_signal[3] correctly as an address using %zx.
Format barrier_packet.dep_signal[4] correctly as an address using %zx.
Format packet.kernarg_address correctly as an address using %zx.
Format completion_signal correctly as an address using %zx.
Format this->queue_->public_handle()->id correctly as an unsigned long.
Format this->queue_->LoadReadIndexRelaxed() correctly as an unsigned long.
Format write_index correctly as an unsigned long.
Format index correctly as an unsigned long.
Signed-off-by: Alysa Liu <Alysa.Liu@amd.com>
[ROCm/ROCR-Runtime commit: 53873e32f3]
Add PC sampling functionality to the driver interface:
1. Add new PC sampling methods to Driver base class:
- PcSamplingQueryCapabilities
- PcSamplingCreate
- PcSamplingDestroy
- PcSamplingStart
- PcSamplingStop
2. Implement PC sampling methods in KfdDriver using HSAKMT APIs:
- Map HSAKMT status codes to HSA status codes
- Handle resource busy conditions
- Proper error handling for all operations
Signed-off-by: Honglei Huang <Honglei1.Huang@amd.com>
[ROCm/ROCR-Runtime commit: 56cb9390ff]
Add ASAN header page management to Driver
- Add ReplaceAsanHeaderPage and ReturnAsanHeaderPage to Driver interface
- Implement ASAN functions in KfdDriver using hsaKmt calls
Signed-off-by: Honglei Huang <Honglei1.Huang@amd.com>
[ROCm/ROCR-Runtime commit: a47c060d6a]
This change improves the abstraction of memory sharing operations by moving them
to the driver layer and adds safety checks for cross-driver operations.
- Add ShareMemory and RegisterSharedHandle methods to support memory sharing
between processes
- Add IsDifferentDriver utility methods to check driver compatibility across
agents/nodes
- Refactor IPC memory handling to use driver-based memory sharing instead of
direct HSAKMT calls
- Improve error handling for memory sharing operations across different drivers
Signed-off-by: Honglei Huang <Honglei1.Huang@amd.com>
[ROCm/ROCR-Runtime commit: 9312468655]
This patch uses udmabuf driver to allocate system memory instead of using amdgpu
driver for APU. With this function app can account its consumed system memory by
cgroup mechanism. This function is enabled by env variable HSA_USE_UDMABUF.
Signed-off-by: Xiaogang Chen<Xiaogang.Chen@amd.com>
[ROCm/ROCR-Runtime commit: 996e8bbfb7]
The ape1_size_ member was leftover after the removal
of KV and is no longer used.
Remove it to remove some compiler warnings.
Signed-off-by: Tony Gutierrez <anthony.gutierrez@amd.com>
[ROCm/ROCR-Runtime commit: 5285c24657]
Modify agent initialization to support different driver types,
to enable KFD_VIRTIO dirver for CPU and GPU agent here.
1. Add driver_type parameter to CpuAgent and GpuAgent constructors
2. Update topology discovery to handle multiple driver types
3. Fix MakeMemoryResident return value check in VirtioDriver
4. Add helper function IsGPUDriver to check driver types
5. Update agent discovery to iterate through all available drivers
This change makes the runtime more flexible by removing hardcoded KFD
driver assumptions and properly handling different driver backends.
Signed-off-by: Honglei Huang <Honglei1.Huang@amd.com>
[ROCm/ROCR-Runtime commit: 20806577ce]
This commit adds virtio driver support to the ROCm runtime by:
1. Implementing KfdVirtioDriver class that inherits from core::Driver
2. Adding KFD_VIRTIO to DriverType enum
3. Registering virtio driver discovery function in topology
4. Adding virtio driver source files to CMake build
The virtio driver implementation provides basic memory management and
queue operations for virtualized GPU environments. Some advanced features
like PC sampling and SMI are currently not supported.
Key changes:
- Add new files: amd_kfd_virtio_driver.h/cpp
- Update CMakeLists.txt to include virtio driver
- Add VIRTIO to DriverType enum in driver.h
- Register virtio driver in amd_topology.cpp
Signed-off-by: Honglei Huang <Honglei1.Huang@amd.com>
[ROCm/ROCR-Runtime commit: d36cb195da]
This patch adds VirtIO support to the libhsakmt library, enabling communication
with AMD GPUs via VirtIO.
Details
- CMakeLists.txt: Added a new CMakeLists.txt file for the VirtIO component
of libhsakmt.
- hsakmt_virtio.c/h: Implemented the core VirtIO functionality, including
VirtIO GPU device initialization, command execution, and memory management.
- virtio_gpu.c/h: Contains the implementation of the VirtIO GPU device,
including ioctl handling, shared memory management, and command execution.
- hsakmt_virtio_events.c: Implements event handling for VirtIO, such as event
creation, destruction, setting, resetting, and querying event states.
- hsakmt_virtio_memory.c: Manages memory operations for VirtIO, including memory
allocation, freeing, mapping, and unmapping.
- hsakmt_virtio_queues.c: Implements queue management for VirtIO, including
queue creation, destruction, and updating.
- hsakmt_virtio_topology.c: Handles system and node properties for VirtIO.
- hsakmt_virtio_vm.c: Manages VM-related operations for VirtIO, such as
reserving and dereserving VA space.
- include/linux/virtgpu_drm.h: Contains DRM definitions for VirtIO GPU.
Key Features
- VirtIO GPU Initialization: The library can now initialize a VirtIO GPU device
and communicate with it.
- Command Execution: Supports executing commands on the VirtIO GPU device.
- Memory Management: Provides functions for allocating, freeing, mapping, and
unmapping memory for VirtIO operations.
- Event Handling: Implements a comprehensive event system for VirtIO.
- Queue Management: Allows for creating, destroying, and updating queues
on the VirtIO GPU device.
- System and Node Properties: Retrieves and manages system and node
properties for VirtIO.
Signed-off-by: Honglei Huang <Honglei1.Huang@amd.com>
[ROCm/ROCR-Runtime commit: 48d3719dba]
Support was removed for these eng samples, so remove them from the
blacklist, and make sure that we're using 942 for the shader store
[ROCm/ROCR-Runtime commit: f755981f03]
- Refactored scratch memory handling by introducing fmm_is_scratch_aperture to
replace repeated for-loops.
- Simplified code paths in hsakmt_fmm_release, hsakmt_fmm_map_to_gpu, and
hsakmt_fmm_unmap_from_gpu by using the new helper.
Signed-off-by: Honglei Huang <Honglei1.Huang@amd.com>
[ROCm/ROCR-Runtime commit: 72061a9024]
so that aql-to-pm4 conversion could verify the validity of the kernel
object.
Signed-off-by: Flora Cui <flora.cui@amd.com>
[ROCm/ROCR-Runtime commit: a765dd7e94]
David Yat Sin
Sunday Clement
hkasivis
Tony G
German Andryeyev
systems-assistant[bot]
Alysa Liu
Benjamin Welton
Flora Cui
estewart08
SaleelK
jonatluu
shwetakhatri-amd
cfreeamd
jokim-amd
Yiannis Papadopoulos
Honglei Huang
Shweta Khatri
Xiaogang Chen
Yat Sin, David
Kent Russell