If some agents cannot access the memory buffer directly, this will cause
the hsa_amd_interop_map_buffer API call to fail
Change-Id: If2f0e1735c2926440d657831de50775d7f304c8e
[ROCm/ROCR-Runtime commit: 2360253b3b]
Enum type for compute AQL is defined as larger then targeted SDMAs
enum types. We should only deny legacy calls for SDMA queues that
require targeted engines.
Change-Id: I6386a8700b3b18af825b6f0d2be27052cc8de0f5
[ROCm/ROCR-Runtime commit: ae99effb29]
This change adds the initial classes for the AIE agent and AIE AQL
queue.
An AIE agent list is added to the core runtime object.
Change-Id: I84b02f52171b80726dfb2c8431582a3ea2986eb3
[ROCm/ROCR-Runtime commit: 8ea62f1cea]
Rewriting logic to fix issue where pthread_create would return errors
other than EINVAL, and these errors would be ignored.
Change-Id: I573958724dcf886c20e8c14e6a9182303b3ffa06
[ROCm/ROCR-Runtime commit: c8dd4d2b3b]
Core dump support relies on debugger related KFD ioctl which have been
introduced in version 1.13 of the interface. However, the code checks
for KFD_IOCTL_MINOR_VERSION (currently 17), making it impossible to
produce core dumps when using some drivers that should support it.
Update the CHECK_KFD_MINOR_VERSION calls in the debugger related ioctl
wrappers and look for KFD 1.13 or above.
Change-Id: I10a7fd03bf8f678b6318d7c25d6a7ded804dac67
[ROCm/ROCR-Runtime commit: d5acab2b39]
A recent patch introduced a build failure when building with Clang:
[ 65%] Building CXX object runtime/hsa-runtime/CMakeFiles/hsa-runtime64.dir/libamdhsacode/amd_core_dump.cpp.o
[…]/runtime/hsa-runtime/libamdhsacode/amd_core_dump.cpp:271:29: error: arithmetic on a pointer to void
271 | read = pread(fd_, buf + done, buf_size - done,
| ~~~ ^
1 error generated.
This patch fixes this by making sure the "void *" pointer is converting
to "char *" before doing arithmetic on it.
Change-Id: Ib1663ed30abce76e05f06d042975eccd7d729823
[ROCm/ROCR-Runtime commit: 3475a45137]
Recommended SDMA engines for DMA copies are now exposed for better
GPU-GPU performance. ROCr can now select those DMA engines.
Also lock-in host-device copies to SDMA0 and device-host copies to
SDMA1 for better stability and performance.
Change-Id: Ideff2e13daf537104efecb8b837bd49ee5096cb5
[ROCm/ROCR-Runtime commit: eb30a5bbc7]
Extend the current Thunk implementation of queue creation to target
specific SDMA engine IDs.
Also expose the new recommend SDMA engines per IO link from the KFD
sysfs.
Change-Id: I51f9a0d83c0f1fc4d5dc837f879a7ae332e7d7e9
[ROCm/ROCR-Runtime commit: 2f588a2406]
When HSA_OVERRIDE_GFX_VERSION is used, save the overrided GFX
version to OverrideEngineId instead of original EngineId. There
are places where real GFX properties still needed, e.g. CWSR size
calculation.
Change-Id: I9d9149bae465b7cfe55604fc19e7ca34e48b7b1c
Signed-off-by: Yifan Zhang <yifan1.zhang@amd.com>
[ROCm/ROCR-Runtime commit: 3f1f68c8cb]
The current trap handler has 2 limitations:
1) If it receives a HOST_TRAP, it clears the corresponding bit
and notifies the host, when it should not.
2) When it is entered because of a debug trap (s_trap 3) and the
debugger is not attached, it returns unconditionally. However,
if another exception is reported at the same time as the trap
handler is entered for the debug trap (a memory violation for
example), that other exception ends-up being ignored.
This patch addresses both of those issues. It makes it so host traps
and debug traps are ignored when necessary. If any other exception is
reported to the wave, we halt the wave and notify the host, and if no
other exception is reported (i.e. we entered the trap handler because of
host trap or debug trap), we return to shader code.
Other minor defects are also fixed during this refactor:
- Fixed SQ_WAVE_EXCP_FLAG_PRIV_XNACK_ERROR_SHIFT which had an incorrect
value
- Host traps can be sent at any time, including after we have halted a
wave. In such case, the old approach would have:
1) cleared the trap ID saved in ttmp6
2) clobbered ttmp10 where part of the actual wave's PC is saved.
Change-Id: I9ecd341f4967e686233dec182b3e5b0388ef19bd
[ROCm/ROCR-Runtime commit: 123b2c080a]
This fixes an issue for missing HW events when out of HW events.
We cannot determine whether a HW event has occurred unless we call the
underlying drivers with hsaKmtWaitOnMultipleEvents_Ext. Previous logic
in Signal::WaitAny would switch to ACTIVE_WAIT state if we run out of
hardware events (signal->EopEvent() == NULL) and this would cause the
hsaKmtWaitOnMultipleEvents_Ext call to be skipped. But also, when we
have some signals without hardware events, calling
hsaKmtWaitOnMultipleEvents_Ext with a timeout of 0 so that we can poll
for remaining signals adds overhead with an IOCTL call and may cause
extra delay. Separating AsyncEventLoop into two separate threads so
that:
1. We can have a new Signal::WaitAnyExceptions to wait for HW events
This function can be simpler as it does not have to perform all the
timer calculations because it is expected to be always waiting on
hsaKmtWaitOnMultipleEvents_Ext through the lifetime of a process.
2. Signal::WaitAny does not need to have extra code to check for HW
exceptions as it only needs to handle HSA_EVENTTYPE_SIGNAL events. It
can also skip the calls to hsaKmtWaitOnMultipleEvents_Ext if needed.
Change-Id: I52ba99fd6e483e0cb477b7931a0dcc03520aa523
Signed-off-by: David Yat Sin <David.YatSin@amd.com>
[ROCm/ROCR-Runtime commit: 88eaa834d0]
Delete queues used internally in agent destructor to make sure any
memory allocated by the queue objects are freed before the agent memory
regions are destroyed.
Change-Id: I4768c9cf66f77ac00a5a355f373f7f22dc266e47
[ROCm/ROCR-Runtime commit: 56ba584a22]
If user application tries to free memory that is currently being used by
the underlying HW device, the hsaKmtFreeMemory function call will fail.
This would be caused by an incorrect call by the user application. A
system memory error is raised and the user application is expected to
abort when this happens.
Note: This leaves the allocation_map_ table in an inconsistent state as
this address entry is removed from it while the pointer is not actually
free'd. But re-organising the FreeMemory() function would require the
memory_lock_ to be held for much longer and may affect performance.
Since this is a very unlikely and invalid use case, we prefer to leave
the FreeMemory() function as is.
Change-Id: I24279eb98620c32d34f4c5ad1b7a0a30cb65835d
Signed-off-by: David Yat Sin <David.YatSin@amd.com>
[ROCm/ROCR-Runtime commit: 921471bd94]
Skip coredump generation when receiving HSA_STATUS_ERROR_MEMORY_FAULT.
We also receive a system error of type HSA_EVENTTYPE_MEMORY and generate
the coredump there. Trying to generate coredump from 2 places sometimes
causes unnecessary error message because both places try to create a
coredump file with the same name.
Change-Id: If3f03bab2c24ad71dfeff39ab411bb9ac08b337e
Signed-off-by: David Yat Sin <David.YatSin@amd.com>
[ROCm/ROCR-Runtime commit: aae4dab88e]
This patch adds output (to stderr) to indicate step in the core dump
creation failed to improve debuggability.
Change-Id: I349692e278c2d744136d7fba7f7c2e5a7ada0c06
Signed-off-by: David Yat Sin <David.YatSin@amd.com>
[ROCm/ROCR-Runtime commit: 3646064a0e]
It is possible for the runtime to receive an interrupt while trying to
access VRAM data using /proc/self/mem. In such case, pread(2) would
return -1 and set errno to -EINTR. This is not an error case, the
pread(2) call just need to be restarted, however current implementation
would tread it as an error.
This patch changes the the implementation to correctly retry on EINTR.
While at it, this patch also handles cases where pread(2) reads less
data than originally requested.
Change-Id: I6a72fc5eda4afd90319f0d24b35c9eac6d1ff41c
Signed-off-by: David Yat Sin <David.YatSin@amd.com>
[ROCm/ROCR-Runtime commit: 3e0d3d6d61]
Force mem_flags to be explicit passed in then calling Queue constructor
to avoid ambiguity with calls to Queue constructor trying to only pass
the agent_node_id.
Change-Id: Ib6fedcb9e52d6c9f35f9051dfa989343456ca368
Signed-off-by: David Yat Sin <David.YatSin@amd.com>
[ROCm/ROCR-Runtime commit: 1d1d402dcc]
No functional change
Change-Id: Ibe97b03f62c4affcb60d3469312c8a0b6eb11391
Signed-off-by: David Yat Sin <David.YatSin@amd.com>
[ROCm/ROCR-Runtime commit: 8176a8830f]
In rocrtst helper_funcs.h, a function argument that gets
written to was previously incorrectly marked as const.
Change-Id: If8cc6555ebfa974b9665d9d5b93de01bb45fde2c
[ROCm/ROCR-Runtime commit: 1c6a4a55f1]
Current DMABUF implemenation is unstable. Switch back to legacy
support for now.
Change-Id: I3be871f38c6524b0bcc9225bab61de4e57771efb
[ROCm/ROCR-Runtime commit: ea646cf958]
Currently, the only error type is HSA_AMD_MEMORY_ERROR_MEMORY_IN_USE,
which happens when a user application incorrectly tries to free memory
that is currently being used by underlying device hardware.
Change-Id: I8ce352eb9719694135fba1fa56d62368036b2e5e
Signed-off-by: Chris Freehill <cfreehil@amd.com>
[ROCm/ROCR-Runtime commit: 2853bf03f0]
New API to accept a file stream for logging
Co-authored-by: David Yat Sin <David.YatSin@amd.com>
Change-Id: Ie09c35ae14ca86a97eb25f61251be287c55d7169
Signed-off-by: Chris Freehill <cfreehil@amd.com>
[ROCm/ROCR-Runtime commit: 26e105d9ab]
Fallback to old userptr registration in case SVM method fails.
Signed-off-by: Alex Sierra <alex.sierra@amd.com>
Change-Id: I70c3ec74a8b4f762713e6a0619453642f3fca8e5
Signed-off-by: Chris Freehill <cfreehil@amd.com>
[ROCm/ROCR-Runtime commit: 626eb4bfaf]
This lets you run two unsupported-but-really-supported cards of different architecture together in the same program. Works great w/ llama.cpp on my 7900XT + 6600.
Example usage (device 0 is RDNA3, device 1 is RDNA2):
HSA_OVERRIDE_GFX_VERSION_1="11.0.0" HSA_OVERRIDE_GFX_VERSION_2="10.3.0" ollama serve
Change-Id: Ic63ef462f698dee722d360f7fc3ef72789c277b7
Signed-off-by: AdamNiederer
Signed-off-by: Kent Russell <kent.russell@amd.com>
Signed-off-by: Chris Freehill <cfreehil@amd.com>
[ROCm/ROCR-Runtime commit: 84567b6416]
Fix register COMPUTE_PGM_RSRC2 in Dispatch code.
Bit 6 (called TRAP_PRESENT on pre-GFX12) should not be set on GFX12
as it has a different meaning (DYNAMIC_VGPR).
Minor instructions changes for CopyOnSignalIsa and WriteAndSignalIsa
shaders.
Change-Id: Ib4e75e3c92f220210bc45778738d81b91efb9d5e
Signed-off-by: David Belanger <david.belanger@amd.com>
Signed-off-by: Chris Freehill <cfreehil@amd.com>
[ROCm/ROCR-Runtime commit: 611911020c]
A function call was refactored out of CommandLine.h, so add the header
to include it
Change-Id: If5594e3abc2fdfdd59f108c4379802cedab127ee
Signed-off-by: Kent Russell <kent.russell@amd.com>
Signed-off-by: Chris Freehill <cfreehil@amd.com>
[ROCm/ROCR-Runtime commit: bd2d9770f7]
RDMATest.ContiguousVRAMAllocation test uses 4GB buffer, skip the test if
total VRAM size is less than 5GB, considering page table and other
reserved VRAM usage.
Change-Id: I0342417501cdd3477c2bf1b2f7d1e6bef61d1871
Signed-off-by: Philip Yang <Philip.Yang@amd.com>
Signed-off-by: Chris Freehill <cfreehil@amd.com>
[ROCm/ROCR-Runtime commit: 42df8b2b34]
This commit was ported from old repo.
Original author: Ranjith Ramakrishnan <Ranjith.Ramakrishnan@amd.com>
For static builds use drm and drm_amdgpu static libraries for linking.
Created a separate cmake target file and static library for static use case
The config file will include the respective target file based on
BUILD_SHARED_LIBS.
Default target file will be the one where drm and libdrm_amdgpu shaed libraries
are linked.
Applications using statically linked cmake targets of hsakmt should install the
required static libraries before building.
Change-Id: Idf4e1a2b5f18b344f5a9927803756d50c2b33702
[ROCm/ROCR-Runtime commit: 9e8477e1c9]
This will link static libraries of drm and libdrm_amdgpu libraries
This commit was ported from old repo and originally authored by:
Ranjith Ramakrishnan <Ranjith.Ramakrishnan@amd.com>
Date: Thu, 20 Jun 2024 08:29:03 -0700
Change-Id: I8b06811516335317d4fb3d7c98b001a12776a808
[ROCm/ROCR-Runtime commit: 2a5e433393]
Change max_slice type to uint64_t and calculation to 64-bit, otherwise
value overflows to 0.
Problem triggered only on GFX12 as field size was increased.
Change-Id: If26451224538743dabc41bdc1b327c6ef021bc24
Signed-off-by: David Belanger <david.belanger@amd.com>
Signed-off-by: Chris Freehill <cfreehil@amd.com>
[ROCm/ROCR-Runtime commit: 13c3f06dfe]
Since "libhsakmt: Prevent hsaKmtRegisterMemory* from registering non-userptr",
non-userptr is not allowed to be pinned any more.
Use hsa_amd_agents_allow_access to map host memory.
Change-Id: I898d2f83222907de58cafc1a2b18a636634d1b20
Signed-off-by: Lang Yu <lang.yu@amd.com>
Signed-off-by: Chris Freehill <cfreehil@amd.com>
[ROCm/ROCR-Runtime commit: 7e6c3d1bfa]
Fix encoding of pitch in SRD (1 bit missing).
Issue affects images with pitch > 8192.
Signed-off-by: David Belanger <david.belanger@amd.com>
Change-Id: Id0b431f51ab3984d1a47d3e8c13d35e28a6009cf
Signed-off-by: Chris Freehill <cfreehil@amd.com>
[ROCm/ROCR-Runtime commit: 4f453f3bd4]
This variable is now in a sub-project, but needs to be visible
in the super-project.
Change-Id: I14d307646253df8f0a8a50d01b8ca677b904234c
[ROCm/ROCR-Runtime commit: 5820fa37d7]
New API to support alignment parameter when reserving virtual addresses.
If the alignment is 0, then the default size is used. Otherwise the
alignment needs to be a power of 2 and greater than or equal to page
size.
Existing hsa_amd_vmem_address_reserve marked for future deprecation.
Change-Id: I17cee75420183dea5842fc1ecc2514cdcd760bac
Signed-off-by: Chris Freehill <cfreehil@amd.com>
[ROCm/ROCR-Runtime commit: 08c44fbda6]
Tony Gutierrez
David Yat Sin
Shweta Khatri
Jonathan Kim
Joseph Macaranas
Lancelot SIX
Yifan Zhang
Swati Rawat
James Xu
Chris Freehill
Saleel Kudchadker
Jonathan R. Madsen
Alex Sierra
Adam Niederer
Tim Huang
David Belanger
Kent Russell
Philip Yang
Lang Yu