* [rocprofiler-sdk] Fix fmt::join build errors
- remedy use of fmt::join without include <fmt/ranges.h>
* include memory header
* Disable FMT build for SDK CI
* Add -DROCPROFILER_BUILD_FMT=OFF to sanitizer steps
* Add temporary workaround for rccl.h issue
* Add ROCPROFILER_INTERNAL_RCCL_API_TRACE to SDK CI builds
* disable clang-tidy for vendored includes
---------
Co-authored-by: Jonathan R. Madsen <jonathanrmadsen@gmail.com>
Co-authored-by: jbonnell-amd <jason.bonnell@amd.com>
* Update what is hip
* Update HIP runtime page
* Update images
* Remove omnitrace
* Quick fix
* Feedback fixes
* Minor fixes
* Update SAXPY tutorial
Signed-off-by: Jan Stephan <jan.stephan@amd.com>
---------
Signed-off-by: Jan Stephan <jan.stephan@amd.com>
Co-authored-by: Adel Johar <adel.johar@amd.com>
Co-authored-by: Jan Stephan <jan.stephan@amd.com>
* libhsakmt/virtio: Add alloc memory align api
Signed-off-by: Honglei Huang <honghuan@amd.com>
* libhsakmt/virtio: Rename CLGL BO to AMDGPU BO
Rename VHSA_BO_CLGL to VHSA_BO_AMDGPU to support generic AMDGPU buffer objects, not just CL/GL interop.
* libhsakmt/virtio: Add atomic helpers and node lookup
Add vhsakmt_atomic_inc/dec macros and vhsakmt_get_node_by_id helper function.
* libhsakmt/virtio: Add AMDGPU device initialization support
Add vamdgpu_device_initialize and vamdgpu_device_deinitialize functions.
* libhsakmt/virtio: Add AMDGPU device handle and DRM command support
Add vamdgpu_device_get_fd, vdrmCommandWriteRead and update vhsaKmtGetAMDGPUDeviceHandle.
* libhsakmt/virtio: Add AMDGPU BO free and CPU map support
Add vamdgpu_bo_free and vamdgpu_bo_cpu_map functions.
* libhsakmt/virtio: Add AMDGPU BO import and export support
Add vamdgpu_bo_import, vamdgpu_bo_export and vhsakmt_bo_from_resid functions.
* libhsakmt/virtio: Add AMDGPU BO VA operation support
Add vamdgpu_bo_va_op function.
* libhsakmt/virtio: Add dma buf export support
Add vhsaKmtExportDMABufHandle API in virtio driver to support export
feature.
* libhsakmt/virtio: Fix potential deadlock in userptr deregistration
Refactor vhsakmt_deregister_userptr_non_svm to avoid calling
vhsakmt_destroy_userptr while holding the bo_handles_mutex lock.
Previously, destroying userptrs directly while iterating the tree
could cause deadlock issues due to nested locking.
- Move interval tree removal from vhsakmt_destroy_userptr to caller
- Collect BOs to free in a temporary array during tree traversal
- Destroy BOs after releasing the mutex to avoid lock contention
- Use dynamic array with realloc to handle arbitrary number of BOs
Signed-off-by: Honglei Huang <honghuan@amd.com>
* rocr: driver/virtio: Implement DMA-BUF import/export and memory mapping APIs
Implement the missing DMA-BUF handling and memory mapping functions
in the virtio KFD driver to enable cross-process memory sharing:
- ExportDMABuf: Export HSA memory as DMA-BUF file descriptor
- ImportDMABuf: Import DMA-BUF fd as shareable buffer object
- Map: Map imported buffer into virtual address space with permissions
- Unmap: Unmap buffer from virtual address space
- ReleaseShareableHandle: Free imported buffer object
Also add drm_perm() helper to convert HSA access permissions to
AMDGPU VM page flags (READABLE/WRITEABLE).
These APIs enable IPC memory sharing between HSA processes through
DMA-BUF mechanism in virtualized environments.
Signed-off-by: Honglei Huang <honghuan@amd.com>
* libhsakmt/virtio: Add register memory APIs
Add two new memory registration functions to the virtio HSA KMT library:
1. vhsaKmtRegisterMemory: A simplified wrapper for vhsaKmtRegisterMemoryWithFlags
that uses default CoarseGrain memory flags.
2. vhsaKmtRegisterMemoryToNodes: A stub implementation for registering memory
to specific nodes. Returns HSAKMT_STATUS_NOT_IMPLEMENTED as it's currently
not used in ROCR.
Changes:
- Added function declarations in hsakmt_virtio.h
- Implemented functions in hsakmt_virtio_memory.c
- Exported symbols in libhsakmt_virtio.ver
Signed-off-by: Honglei Huang <honghuan@amd.com>
* libhsakmt/virtio: Add graphics handle registration and mapping APIs
- Add vhsaKmtRegisterGraphicsHandleToNodesExt() with flags support
- Add vhsaKmtMapGraphicHandle() and vhsaKmtUnmapGraphicHandle() stubs
- Refactor existing registration API to use extended version
Signed-off-by: Honglei Huang <honghuan@amd.com>
* libhsakmt/virtio: Add virtio support for queue APIs
Implement vhsaKmtUpdateQueue, vhsaKmtSetQueueCUMask,
vhsaKmtAllocQueueGWS and vhsaKmtGetQueueInfo functions
with virtio protocol extensions and symbol exports.
Signed-off-by: Honglei Huang <honghuan@amd.com>
* libhsakmt/virtio: Add new virtio API support for model, SMI, and XNACK mode
Add three new API functions to the virtio backend:
- vhsaKmtModelEnabled: Check if pre-silicon model is enabled (returns false for virtio)
- vhsaKmtOpenSMI: Open SMI interface for a node (not yet supported in virtio)
- vhsaKmtSetXNACKMode: Set XNACK mode via virtio control command
Signed-off-by: Honglei Huang <honghuan@amd.com>
* libhsakmt/virtio: Add shared memory support for virtio backend
Implement shared memory APIs for the virtio backend to enable
memory sharing between processes:
- Add vhsaKmtShareMemory() to share memory regions and create
shared memory handles
- Add vhsaKmtRegisterSharedHandle() to register shared memory
handles in the current process
- Add vhsaKmtRegisterSharedHandleToNodes() for node-specific
shared memory registration
Signed-off-by: Honglei Huang <honghuan@amd.com>
* libhsakmt/virtio: Add memory management APIs for virtio
Add the following new memory management APIs to virtio implementation:
- vhsaKmtSetMemoryUserData: Set user data for memory pointer
- vhsaKmtSetMemoryPolicy: Configure memory policy for nodes
- vhsaKmtSVMGetAttr: Get SVM (Shared Virtual Memory) attributes
- vhsaKmtSVMSetAttr: Set SVM attributes
- vhsaKmtReplaceAsanHeaderPage: ASAN header page replacement (stub)
- vhsaKmtReturnAsanHeaderPage: ASAN header page return (stub)
Changes include:
- Added API declarations in hsakmt_virtio.h
- Implemented functions in hsakmt_virtio_memory.c
- Extended protocol definitions in hsakmt_virtio_proto.h
- Added user_data field to vhsakmt_bo structure
- Exported new symbols in libhsakmt_virtio.ver
Signed-off-by: Honglei Huang <honghuan@amd.com>
* libhsakmt/virtio: Add SPM APIs
Add three new SPM-related APIs to the virtio interface:
- vhsaKmtSPMAcquire: Acquire SPM resources on a preferred node
- vhsaKmtSPMRelease: Release SPM resources on a preferred node
- vhsaKmtSPMSetDestBuffer: Set destination buffer for SPM data with
optional userptr support and data loss detection
These APIs extend the virtio command protocol with new query types:
- VHSAKMT_CCMD_QUERY_SPM_ACQUIRE
- VHSAKMT_CCMD_QUERY_SPM_RELEASE
- VHSAKMT_CCMD_QUERY_SPM_SET_DST_BUFFER
The implementation includes proper buffer management for both
direct BO access and userptr fallback for smaller buffers.
Signed-off-by: Honglei Huang <honghuan@amd.com>
* libhsakmt/virtio: Add virtio stub for hsaKmtAisReadWriteFile API
Add vhsaKmtAisReadWriteFile stub implementation for the virtio backend
to support AIS (Accelerated I/O Service) file read/write operations.
This stub currently returns HSAKMT_STATUS_NOT_IMPLEMENTED.
Changes include:
- Add vhsaKmtAisReadWriteFile declaration in hsakmt_virtio.h
- Add stub implementation in hsakmt_virtio_memory.c
- Export the symbol in libhsakmt_virtio.ver
Signed-off-by: energystoryhhl <energystoryhhl@users.noreply.github.com>
* libhsakmt/virtio: Add vamdgpu_bo_query_info and vamdgpu_bo_set_metadata APIs
Implement two new virtio wrapper functions for AMDGPU buffer object operations:
1. vamdgpu_bo_query_info: Query buffer object information including
allocation parameters, memory usage, and metadata.
2. vamdgpu_bo_set_metadata: Set metadata for a buffer object, allowing
applications to attach custom data to GPU memory allocations.
Signed-off-by: Honglei Huang <honghuan@amd.com>
* libhsakmt/virtio: Add ProcessVMRead/Write stub implementations for virtio
Add vhsaKmtProcessVMRead and vhsaKmtProcessVMWrite stub functions
to the virtio interface. These APIs return HSAKMT_STATUS_NOT_IMPLEMENTED
since they are not supported in the baremetal implementation, matching
the behavior of the deprecated hsaKmtProcessVMRead/Write APIs.
Signed-off-by: energystoryhhl <energystoryhhl@users.noreply.github.com>
---------
Signed-off-by: Honglei Huang <honghuan@amd.com>
Signed-off-by: energystoryhhl <energystoryhhl@users.noreply.github.com>
Co-authored-by: energystoryhhl <energystoryhhl@users.noreply.github.com>
* Add cmake based instructions to create standalone binary
* Specify standalone binary extraction path in doc.
* Add documentation to explain how to specify self-extraction path
when building the standalone binary where contents of the binary
are extracted during execution
* Pin Nuitka to version 2.6 for consistency in building standalone binary
* Add verbose output for submods step
* Remove git config setting
* Determine git version
* Try different git install
* Update Dockerfile.ci
* Revert git location in Ubuntu jobs
* Update RHEL and SLES sections to use 2.52 as well
* Add git --version to each step, fix typo in SLES Docker
## Motivation
The `rocprof-sys-avail -H -c GPU` command is returning blank output which is expected to display a list of available GPU hardware counters instead.
The `rocprof-sys-sample` and `rocprof-sys-run` is missing the `--gpu-events` option for specifying GPU counter events during profiling.
## Technical Details
The initialize_event_info() function had a logic bug where it only called set_agents() if the agent_manager was empty, but the actual issue was that the gpu_agents and cpu_agents vectors were empty even when agents were discovered.
Fixed the conditional logic to properly call set_agents() when gpu_agents and cpu_agents are empty, regardless of the agent_manager state.
Added the `--gpu-events (-G)` option which sets the `ROCPROFSYS_ROCM_EVENTS` environment variable to the specified values.
Fixes an issue where unsupported GPU/APU arch is being skipped gracefully - more details about this issue in the below comment.
* Replace O(n^2²) nested loop with O(1) dictionary lookup when associating
metric values with metrics. Pre-group values by (metric_id, kernel_name)
to eliminate redundant iteration over entire values dataframe for each
metric-kernel combination.
* This optimization significantly improves database write performance for
workloads with large numbers of metrics and kernels.
* Standalone roofline should create HTML instead of PDF
* Eiminate the dependency on kaleido and plotly_get_chrome by moving
towards plotly native HTML image roofline chart generation
* Address review comments
SWDEV-539526 - Add support for Mipmapped Array in Rocr
Add support for Mipmapped Array functionality in Rocr Runtimeenabling GPU applications to work with multi-level texture mipmaps. The implementation introduces new public APIs for creating, querying, and managing mipmapped arrays across different GPU architectures.
Signed-off-by: Apurv Mishra <Apurv.Mishra@amd.com>
Co-authored-by: Shweta Khatri <shweta.khatri@amd.com>
Co-authored-by: taosang2 <tao.sang@amd.com>
config_hashes json had mismatched md5s for the delta_hash values, regenerated the file with the existing files in develop branch.
Signed-off-by: Carrie Fallows <Carrie.Fallows@amd.com>
* Data imputation strategy for iteration multiplexing
* Implement data imputation methodology to handle missing counter values
in case of iteration multiplexing
* Enable dispatch filtering with iteration multiplexing since we are no
longer merging dispatches
* Bugfix to prevent check for missing counter values when using csv
format when profiling with iteration multiplexing
* Move warning and info message in case of iteration multiplexing to
sanitize function which comes earlier in analyze mode
* Address review comments
* Fix typo in documentation
* Move profiling config init. after path check in sanitize()
* Graceful handling of dispatches with all counters empty within data
imputation logic
* Improve info message for iteration multiplexing based analysis
* Ensure proper error message when trying to run iteration multiplexing with attach/detach
* fix test case
Implements automatic device wake using getDRMDeviceId() DRM call when GPUs
are detected in low-power state. This ensures rocm-smi can access device
information on suspended GPUs.
Signed-off-by: Bindhiya Kanangot Balakrishnan <Bindhiya.KanangotBalakrishnan@amd.com>
Problem:
The existing SDMA engine selection logic had several issues:
1. Same VirtualGPU/stream could use different SDMA engines for consecutive
async copies since copy_engine_status may report engines as busy
2. Busy and Preferred engine check for every copy
3. No global tracking of which VirtualGPU uses which engine, leading to
suboptimal resource allocation
Solution:
Implemented a global SDMA engine allocator with per-stream affinity:
- Added Device::SdmaEngineAllocator to manage VirtualGPU → engine assignments
* Maintains global map of active assignments
* Enforces exclusivity: different streams use different engines (except
inter-GPU copies where preferred engines are prioritized for optimal
hardware paths like XGMI links)
* Thread-safe allocation/release with Monitor lock
- Modified VirtualGPU to cache assigned engine locally (assigned_sdma_engine_)
for fast lookup without map access on hot path
- Refactored rocrCopyBuffer() to:
1. Check local cached engine first → use if assigned
2. Call AllocateSdmaEngine() if not assigned → cache result
- Moved HSA API queries (memory_copy_engine_status, memory_get_preferred_copy_engine)
into AllocateEngine() for cleaner separation of concerns
- Engine release on HostQueue::finish() instead of only VirtualGPU destruction
* Improves engine utilization by releasing earlier
* Added virtual ReleaseSdmaEngines() method to device::VirtualDevice
- Added future path for simple round-robin allocation (kUseSimpleRR) for
next-gen GPUs with uniform SDMA bandwidth (disabled by default)
Cleanup:
- Removed selectSdmaEngine() helper (logic moved to allocator)
- Removed getSdmaRWMasks() (allocator accesses maxSdmaReadMask_/WriteMask_ directly)
- Removed unused sdmaEngineReadMask_/WriteMask_ member variables from DmaBlitManager
Benefits:
- Ensures consistent per-stream SDMA engine usage
- Prevents cross-stream contention and engine thrashing
- Prioritizes hardware-optimal paths for inter-GPU transfers
- Better resource utilization through earlier release
- Cleaner, more maintainable code structure
- Fixes SWDEV-559349
- Fix build failure caused by correct libunwind not being found in some environments.
- Updated the `timemory` submodule to commit `24407d37ab85c46ba6c18fba9498320f825ee4e4 `.
* Use static catch2.lib instead of catch2.dll
Using catch2.dll incraeses execution time by 12x
* handle debug option for static catch2
* SWDEV-573539 - skip atomics on windows since its taking a very long time to execute
mlsejenkins needs newer cmake but compiler breaks with newer versions
so skipping on windows can be a workaround for now
---------
Co-authored-by: Joseph Macaranas <145489236+jayhawk-commits@users.noreply.github.com>
**Thread limit configuration and enforcement: **
* Added a check in `CMakeLists.txt` to ensure `ROCPROFSYS_MAX_THREADS` is at least 128, automatically setting it to 128 with a warning if a lower value is provided.
* Replaced hardcoded thread limit (`allowed_max_threads`) in `pthread_create_gotcha.cpp` with the configurable `ROCPROFSYS_MAX_THREADS` value, ensuring all runtime checks and warnings use the actual configured limit.
**Documentation improvements: **
* Updated the development guide to explain the new thread limit behavior, including how exceeding the limit is handled gracefully, how to configure it, and the build-time validation rules.
**Test updates: **
* Modified thread limit tests to use the configurable `ROCPROFSYS_MAX_THREADS` value instead of a hardcoded limit and expanded the range of tested thread values.
* Increased test timeouts to accommodate larger thread counts and ensure reliability with higher limits.
This has started failing on various developer build systems. Looking at it, it is not precisely clear how this ever worked given that nothing appears to be adding the DRM include dirs.
I'd prefer that we remove this delay loading (at least for TheRock builds where it is never needed), but in the meantime, this does fix the issue and is verified on an affected system.
Fixes https://github.com/ROCm/TheRock/issues/2744
* SWDEV-508225 - do not assert() after calling digestFatBinary() if it fails. Otherwise this causes assertions to trigger easily in systems that have an APU and a discrete GPU and the code was compiled for the discrete one
* SWDEV-508225 - fix that when using a non-existent ordinal in HIP_VISIBLE_DEVICES, getCurrentArch() would crash
fix: Add gpu_metrics 1.0 support which is still used by some hardware
Code changes related to the following:
* APIs
* Unit tests
Signed-off-by: Oliveira, Daniel <daniel.oliveira@amd.com>
* Add support for get and set APIs for CPUISOFreqPolicy and DFCState Control
- Add support for get and set APIs for CPUISOFreqPolicy and DFCState Control
in AMD SMI and also in the CLI tool
* CHANGELOG.md file updated
* SWDEV-562837: Update amdsmi-py-api.md as per the new APIs
Updated amdsmi-py-api.md as per the new APIs added.
---------
Signed-off-by: Soumya <sranjanr@amd.com>
Signed-off-by: gabrpham <Gabriel.Pham@amd.com>
Co-authored-by: Saka Sitharammurthy <SitharamMurthy.Saka@amd.com>
* SWDEV-569319 Replace ScopedAcquire with stdcpp wrappers
* Remove KernelMutex and KernelSharedMutex abstractions with std::mutex and std::shared_mutex
* Replaced unique_locks with lock_guards
* More changes
* Replace new and deletes with smart pointers
* Replaced some more with shared ptrs
* Replacements with smart pointers - pt 2
* missed change
* Remove MFMA functionality in rocflop sample since its not supported in MI50
* Add gfx arc based support for MFMA and SMFMAC in rocflop.cpp
* Add --int32 usage doc
* Address review comments
* Fix buffer tracing synchronization lock
- PR #529 (in rocprofiler-sdk-internal) introduced waiting on the syncer flag when emplacing in a buffer to prevent the overwriting buffer records currently being processed in a buffer flush callback
- The above fix introduced a block on the both buffers when a buffer flush callback was being executed instead of a block on the buffer being flushed.
* Add rocpd tests for duplicate records
* Address code review comments
* Update rocprofiler workflows to use new runner naming for mi325
* Add input options to workflow_dispatch for rocprofiler-systems CI workflow
* Update runner name on therock-ci-linux.yml as well
SaleelK
Mythreya Kuricheti
Andrei Kochin
Istvan Kiss
Honglei Huang
vedithal-amd
Xie, AlexBin
Jason Bonnell
AidanBeltonS
Sajina PK
Apurv Mishra
Mario Limonciello
cfallows-amd
Yiannis Papadopoulos
Yazen AL Musaffar
systems-assistant[bot]
Bindhiya Kanangot Balakrishnan
koushikbillakanti-amd
Alexandra Sidorova
Flora Cui
David Galiffi
anujshuk-amd
Ajay GunaShekar
Alysa Liu
David Yat Sin
Aleksandar Djordjevic
Godavarthy Surya, Anusha
Stella Laurenzo
Gerardo Hernandez
Daniel Oliveira
SakaSitharammurthy
pghoshamd
Jonathan R. Madsen
habajpai-amd
Joseph Macaranas
Benjamin Welton
Julia Jiang