* SWDEV-558848 - vmm api support for rocr on windows
* Fixes to VMM handle Map/Unmap Set/Get Access
* Fix GetShareableHandle to use pointer for shareable handle
* Update os specific map/unmap memory calls
* clang format update
* Minor syntax fixes from code review
Co-authored-by: Yiannis Papadopoulos <102817138+ypapadop-amd@users.noreply.github.com>
---------
Co-authored-by: Rahul Manocha <rmanocha@amd.com>
Co-authored-by: Yiannis Papadopoulos <102817138+ypapadop-amd@users.noreply.github.com>
* Run pre-commit's whitespace related hooks on projects/rocr-runtime
In order for pre-commit to be useful, everything needs to meet a common
baseline.
Signed-off-by: Mario Limonciello (AMD) <superm1@kernel.org>
* Add missing semicolon which would block compilation on big endian CPUs
Signed-off-by: Mario Limonciello (AMD) <superm1@kernel.org>
---------
Signed-off-by: Mario Limonciello (AMD) <superm1@kernel.org>
* Fixing the copy back to the original buffer malformed packets
* Addressing Copilot Comments
* Addressing Review comments
* Adjust staging buffer size allocation
Change staging buffer size to match the number of packets.
This patch enhances compatibility for DXG environments by introducing conditional
checks for DRM operations, particularly around buffer object metadata handling
in IPC scenarios. These changes improve robustness in DXG IPC memory management
without impacting existing functionality in standard Linux environments.
Signed-off-by: Horatio Zhang <Hongkun.Zhang@amd.com>
WSL uses the call just for the thread wake-up, however under Windows
KMD needs the actual value (SWDEV-568592). The interface is changed
to avoid programming of a modified write_ptr value, which somewhat
changes the client's logic.
Changed ipc_sock_server_conns_ map's value type to size_t. Previous
type of int caused allocations of sizes greater than 2GB to overflow,
causing the message len to be stored as a negative value, preventing the
IPC server from exporting dmabuf file descriptors, which lead to hangs.
Signed-off-by: Sunday Clement <Sunday.Clement@amd.com>
Fix race condition when SetAsyncSignalHandler for the first time because
async_events_thread_ could be null and launched twice.
Refactored async-events to use lazy_pointer.
* Support Windows HANDLE in interop_map_buffer
* Refactored Windows HANDLE in interop_map_buffer
* ROCr System Dependent Handle Type
* Fix for ROCr Handle Conversion Bug
* Remove Windows Header
Workaround for rare issue on gfx90x asics when SDMA_OP_POLLREGMEM
returns before polled memory has value of 0.
Removing previous SW workaround to double-poll as it was not reliable.
* SWDEV-555347 - Remove lock contention in async events loop
* SWDEV-555347 - Introduce Pool of AsyncEventItems
* create generic mempool for AsyncEventItem
* Use BaseShared allocate and free for async event pool
---------
Co-authored-by: Rahul Manocha <rmanocha@amd.com>
* rocr: Add support for VMM and RDMA
Add extra CPU mapping so that kernel-mode drivers can look up the memory
mapping by virtual address.
* Update projects/rocr-runtime/runtime/hsa-runtime/core/runtime/runtime.cpp
Co-authored-by: Yiannis Papadopoulos <102817138+ypapadop-amd@users.noreply.github.com>
* Update projects/rocr-runtime/runtime/hsa-runtime/core/inc/runtime.h
Co-authored-by: Yiannis Papadopoulos <102817138+ypapadop-amd@users.noreply.github.com>
* rocr: Honor uncache flag in memory_lock_to_pool()
Also, combined several flag options used in apis into a
single integer.
Signed-off-by: Chris Freehill <cfreehil@amd.com>
* rocr: Fix hsa_amd_pointer_info on CPU agents
Fix hsa_amd_pointer_info query returning allowd on VMM pointers for CPU
agents when CPU mapping was mapped with PROT_NONE.
---------
Signed-off-by: Chris Freehill <cfreehil@amd.com>
Co-authored-by: Yiannis Papadopoulos <102817138+ypapadop-amd@users.noreply.github.com>
Co-authored-by: Chris Freehill <cfreehil@amd.com>
Co-authored-by: cfreeamd <166262151+cfreeamd@users.noreply.github.com>
Query IPC handles on shared memory export/import for any metadata as a
means to uniquely identify handles that happen to be backed by buffers
that point to the same memory.
* libhsakmt: Update hsakmt_fmm_get_handle to support address range
Currently, hsakmt_fmm_get_handle works only if the address is allocated
(staring) value. Update it so it can find the handle if address falls in
the valid allocated range. This is useful for AMD infinity storage
feature where data needs to be transferred to any memory within in the
allocated range
Signed-off-by: Harish Kasiviswanathan <Harish.Kasiviswanathan@amd.com>
* libhsakmt: Introduce AMD Infinity Storage (AIS) API
Add hsaKmtAisReadWriteFile() API to support AMD Infinity Storage. The
API moves data directly from GPU VRAM to a file.
v2: Add in/out ioctl arguments to provide more status information to
user space. Modify hsaKmt API also accordingly.
Signed-off-by: Harish Kasiviswanathan <Harish.Kasiviswanathan@amd.com>
* rocr: Initial implementation of AMD Infinity Storage (AIS)
Implement first two API: hsa_amd_ais_file_write and hsa_amd_ais_file_read
v2: Change API from hsa_amd_ to hsa_amd_ais_
Change API to take in handle instead of fd for compatibility accross
different platforms
Original Author: Chris Freehill <Chris.Freehill@amd.com>
Signed-off-by: Harish Kasiviswanathan <Harish.Kasiviswanathan@amd.com>
---------
Signed-off-by: Harish Kasiviswanathan <Harish.Kasiviswanathan@amd.com>
Because the base QueueWrapper class copies the wrapped queue's
amd_queue_v2_t queue descriptor struct the QueueProxy seems
superfluous as it will have the same effect as calling the
underlying methods on the wrapped queue itself.
Additionally, because the QueueProxy needs to access the wrapped
queue's queue descriptor it breaks the Queue API which is meant
to abstract the underlying agent's queue implementation.
This makes it easier to generalize the core::Queue as well as
the InterceptQueue.
Signed-off-by: Tony Gutierrez <anthony.gutierrez@amd.com>
Make sure ROCR can be compiled under windows. Extra setup for the windows build environment is required. The change should not have any functional changes under Linux.
* libhsakmt: fix UB due to signed integer literal in 1 << 31
Bit shift operations on signed numbers should not shift into or beyond
the signed bit as this results in Undefined Behaviour.
Signed-off-by: Sunday Clement <Sunday.Clement@amd.com>
* libhsakmt: Fix UB due to signed integer literal in 1 << x
Bit Shifting an unsigned integer is undefined behavior.
BUG: SWDEV-532853
Signed-off-by: Sunday Clement <Sunday.Clement@amd.com>
* rocr: Fix UB in various places due signed integer in bit shift
Bit shifting signed integers into or beyond the sign bit is undefined.
Signed-off-by: Sunday Clement <Sunday.Clement@amd.com>
* rocr: Change signed integer literals to unsigned
Changing the signed integers in the macro expressions throughout the file
to avoid overflow.
Signed-off-by: Sunday Clement <Sunday.Clement@amd.com>
---------
Signed-off-by: Sunday Clement <Sunday.Clement@amd.com>
Co-authored-by: Flora Cui <flora.cui@amd.com>
This commit reverts the following related commits which cause
test failures:
6d15779b3e rocr/driver: add PC sampling support to driver interface
56cb9390ff rocr/driver: add PC sampling support to driver interface
76bf829f09 rocr/driver: add ASAN header page management to Driver class
a47c060d6a rocr/driver: add ASAN header page management to Driver class
02d7eaf3b7 rocr: add memory sharing call to Driver interface
9312468655 rocr: add memory sharing call to Driver interface
* rocr: hsa_amd_pointer_info to support VMEM pointers
Extend hsa_amd_pointer_info to support virtual memory addresses.
If hsa_amd_pointer_info is called on an address that is reserved but not
mapped to memory, then the pointer type will be reported as
HSA_EXT_POINTER_TYPE_RESERVED_ADDR.
If hsa_amd_pointer_info is called on an address that is mapped, then the
pointer type will be reported as HSA_EXT_POINTER_TYPE_HSA_VMEM
* rocrtst: VirtMemory_Basic_Test test for pointer info
Extend rocrtstFunc.VirtMemory_Basic_Test to test for
hsa_amd_pointer_info
* rocrtst: Add SVM Memory Test
Add PC sampling functionality to the driver interface:
1. Add new PC sampling methods to Driver base class:
- PcSamplingQueryCapabilities
- PcSamplingCreate
- PcSamplingDestroy
- PcSamplingStart
- PcSamplingStop
2. Implement PC sampling methods in KfdDriver using HSAKMT APIs:
- Map HSAKMT status codes to HSA status codes
- Handle resource busy conditions
- Proper error handling for all operations
Signed-off-by: Honglei Huang <Honglei1.Huang@amd.com>
[ROCm/ROCR-Runtime commit: 56cb9390ff]
Add ASAN header page management to Driver
- Add ReplaceAsanHeaderPage and ReturnAsanHeaderPage to Driver interface
- Implement ASAN functions in KfdDriver using hsaKmt calls
Signed-off-by: Honglei Huang <Honglei1.Huang@amd.com>
[ROCm/ROCR-Runtime commit: a47c060d6a]
This change improves the abstraction of memory sharing operations by moving them
to the driver layer and adds safety checks for cross-driver operations.
- Add ShareMemory and RegisterSharedHandle methods to support memory sharing
between processes
- Add IsDifferentDriver utility methods to check driver compatibility across
agents/nodes
- Refactor IPC memory handling to use driver-based memory sharing instead of
direct HSAKMT calls
- Improve error handling for memory sharing operations across different drivers
Signed-off-by: Honglei Huang <Honglei1.Huang@amd.com>
[ROCm/ROCR-Runtime commit: 9312468655]
The ape1_size_ member was leftover after the removal
of KV and is no longer used.
Remove it to remove some compiler warnings.
Signed-off-by: Tony Gutierrez <anthony.gutierrez@amd.com>
[ROCm/ROCR-Runtime commit: 5285c24657]
Modify agent initialization to support different driver types,
to enable KFD_VIRTIO dirver for CPU and GPU agent here.
1. Add driver_type parameter to CpuAgent and GpuAgent constructors
2. Update topology discovery to handle multiple driver types
3. Fix MakeMemoryResident return value check in VirtioDriver
4. Add helper function IsGPUDriver to check driver types
5. Update agent discovery to iterate through all available drivers
This change makes the runtime more flexible by removing hardcoded KFD
driver assumptions and properly handling different driver backends.
Signed-off-by: Honglei Huang <Honglei1.Huang@amd.com>
[ROCm/ROCR-Runtime commit: 20806577ce]
This commit adds virtio driver support to the ROCm runtime by:
1. Implementing KfdVirtioDriver class that inherits from core::Driver
2. Adding KFD_VIRTIO to DriverType enum
3. Registering virtio driver discovery function in topology
4. Adding virtio driver source files to CMake build
The virtio driver implementation provides basic memory management and
queue operations for virtualized GPU environments. Some advanced features
like PC sampling and SMI are currently not supported.
Key changes:
- Add new files: amd_kfd_virtio_driver.h/cpp
- Update CMakeLists.txt to include virtio driver
- Add VIRTIO to DriverType enum in driver.h
- Register virtio driver in amd_topology.cpp
Signed-off-by: Honglei Huang <Honglei1.Huang@amd.com>
[ROCm/ROCR-Runtime commit: d36cb195da]
so that aql-to-pm4 conversion could verify the validity of the kernel
object.
Signed-off-by: Flora Cui <flora.cui@amd.com>
[ROCm/ROCR-Runtime commit: a765dd7e94]
This commit completes the memory register/deregister interface change.
Removed static RegisterMemory and DeregisterMemory from MemoryRegion class
- Added pure virtual methods to base Driver interface in driver class
- Added implementation in KFD driver
- Modified MemoryRegion Lock and Unlock to use driver interface
Signed-off-by: Honglei Huang <Honglei1.Huang@amd.com>
[ROCm/ROCR-Runtime commit: ab6bda7e96]
This commit introduces a new AvailableMemory API to the KfdDriver and
XdnaDriver classes.
- Implemented AvailableMemory in KfdDriver to return the available memory size
using hsaKmtAvailableMemory.
- Added a stub implementation of AvailableMemory in XdnaDriver that returns an error.
- Updated the GpuAgent class to use the new AvailableMemory API instead of
directly calling hsaKmtAvailableMemory.
Signed-off-by: Honglei Huang <Honglei1.Huang@amd.com>
[ROCm/ROCR-Runtime commit: 6c390e32cc]
This change adds a const-qualified version of the driver() method to the Agent
class, allowing const Agent objects to access their associated driver without
modifying the object's state.
Signed-off-by: Honglei Huang <Honglei1.Huang@amd.com>
[ROCm/ROCR-Runtime commit: 9c18618847]
Add AllocateScratchMemory interface to Driver base class and implement it
in both KFD and XDNA drivers. This change encapsulates the low-level
scratch memory allocation details within driver implementations, making
the code more maintainable and the interface cleaner.
The main changes include:
- Add AllocateScratchMemory virtual method to Driver interface
- Implement the interface in KfdDriver with existing allocation logic
- Add stub implementation in XdnaDriver
- Update GpuAgent to use the new interface instead of direct KMT calls
Signed-off-by: Honglei Huang <Honglei1.Huang@amd.com>
[ROCm/ROCR-Runtime commit: da8dd9e1e3]
Move the wallclock frequency query from GpuAgent to driver layer to improve
code organization and support multiple driver types. This change:
1. Add GetWallclockFrequency API to KFD/XDNA drivers
2. Move libdrm GPU info query from GpuAgent to driver implementation
3. Update GpuAgent to use the new driver API
Signed-off-by: Honglei Huang <Honglei1.Huang@amd.com>
[ROCm/ROCR-Runtime commit: 412e386b50]
- Implemented GetTileConfig in KfdDriver to retrieve tile configuration for
a specific node.
- Added a stub implementation of GetTileConfig in XdnaDriver.
- Updated driver.h to include a virtual GetTileConfig method.
- Extended hsa_internal.h with a new hsa_get_tile_config function.
- Integrated hsa_get_tile_config into hsa.cpp to call the driver-specific
implementation.
- Updated driver headers to declare the new GetTileConfig method.
Signed-off-by: Honglei Huang <Honglei1.Huang@amd.com>
[ROCm/ROCR-Runtime commit: 9bc38e2ee6]
This commit introduces a new GetClockCounters API to the driver interface.
- Implemented GetClockCounters in KfdDriver to fetch clock counters
using hsaKmtGetClockCounters.
- Added a stub implementation of GetClockCounters in XdnaDriver that
returns HSA_STATUS_ERROR.
- Modified GpuAgent to use driver().GetClockCounters instead of
directly calling hsaKmtGetClockCounters.
Signed-off-by: Honglei Huang <Honglei1.Huang@amd.com>
[ROCm/ROCR-Runtime commit: 8d077dba3b]
This commit introduces a new GetDeviceHandle API to the driver
interface, allowing retrieval of the device handle for a
specific node.
- Implemented GetDeviceHandle in KfdDriver to fetch the AMD GPU
device handle using hsaKmtGetAMDGPUDeviceHandle.
- Added a stub implementation of GetDeviceHandle in XdnaDriver
that returns HSA_STATUS_ERROR.
- Modified GpuAgent::InitLibDrm to use driver().GetDeviceHandle
instead of directly calling hsaKmtGetAMDGPUDeviceHandle.
Signed-off-by: Honglei Huang <Honglei1.Huang@amd.com>
[ROCm/ROCR-Runtime commit: 05b83e72d9]
This commit introduces a new SetTrapHandler API to the driver interface
- Implemented SetTrapHandler in KfdDriver to set trap handlers using
hsaKmtSetTrapHandler.
- Added a stub implementation of SetTrapHandler in XdnaDriver that returns
HSA_STATUS_ERROR.
- Updated the driver interface in driver.h to include the new SetTrapHandler
method.
- Modified GpuAgent to use driver().SetTrapHandler instead of directly calling
hsaKmtSetTrapHandler.
Signed-off-by: Honglei Huang <Honglei1.Huang@amd.com>
[ROCm/ROCR-Runtime commit: d874b8003a]
David Yat Sin
Rahul Manocha
Mario Limonciello
Ammar ELWazir
hongkzha-amd
German Andryeyev
Sunday Clement
Alysa Liu
Jin Jung
Yiannis Papadopoulos
cfreeamd
hkasivis
Tony G
Flora Cui
systems-assistant[bot]
Honglei Huang
Yiannis Papadopoulos
David Yat Sin
Shweta Khatri