Adds hsa_amd_portable_export_dmabuf and hsa_amd_portable_close_dmabuf
which allow obtaining dmabuf handles to rocr allocations. These handles
may be shared with other APIs to support cross vendor & cross device
memory sharing.
Adds query to return whether dmabuf export is supported
Signed-off-by: Jonathan Kim <Jonathan.Kim@amd.com>
Signed-off-by: David Yat Sin <David.YatSin@amd.com>
Change-Id: I7f98501087d9563d07fc2cb428cc886b1e518b1e
[ROCm/ROCR-Runtime commit: 42243c1e8f]
Forgot SDMA blit engine indices are offset by DevToDev 0-position in
a couple of places.
Change-Id: Ie811d8281bc812738ed0107694f3dffde5e93685
[ROCm/ROCR-Runtime commit: 7364a93b98]
Use mwaitx instructions when busy waiting for signals to reduce CPU
energy usage.
This can be disabled by setting HSA_ENABLE_MWAITX=0
Change-Id: Ic207895a491b2bf6dacba47ef0921df3faad5b5a
[ROCm/ROCR-Runtime commit: cc48dfdbff]
ROCr internally uses the same allocation_map_ list to track memory
allocations that are both for internal allocations and allocations by
users of ROCr library. In some edge cases, the library user would call
hsa_amd_pointer_info on an invalid pointer, but ROCR would return the
pointer as valid because this pointer belongs to a memory range that
was allocated internally within ROCr. Adding a flag to differentiate
between internal and external allocations.
Change-Id: I98c52bd85f3985d1ba1b0e3101d2254b003412cf
[ROCm/ROCR-Runtime commit: 59685f4492]
Track and report the size, in bytes, of pending unexecuted blit
commands. To be used in copy ganging.
Change-Id: Ia7453ff88571e927df771c6c819b73c17e67708e
[ROCm/ROCR-Runtime commit: 27596aef0c]
Fixes hang due to change in order of initialization of libraries
that have cyclical dependencies and they call hsa_init() during their
initialization phase.
This implementation looks for a symbol called "HSA_AMD_TOOL_PRIORITY"
across all loaded shared libraries using dynamic section entries of the
loaded lib instead of using dlopen and dlsym for the same purpose.
Change-Id: I4865f2fd18dd186ec311a432ec38fbb5583805d2
[ROCm/ROCR-Runtime commit: 8aac885318]
Reporting whether IOMMU V2 is supported.
IOMMU V1 support is not relevant to user, so not reporting it.
Change-Id: I77389484a87a352da9c2f7b2a5d9de264f90ee53
[ROCm/ROCR-Runtime commit: e30be76f37]
Currently, Wavefront::GetInfo(HSA_WAVEFRONT_INFO_SIZE.. always returns
64. Instead, return the proper wavefront size based on the ISA.
Temporarily, we only return 1 wavefront size for each ISA. As we do not
have mechanism from upper layers to determine correct wavefront when
there are multiple wavefronts supported. We are temporarily
returning 32 for all gfx1xxx cards even though they support 64 as the
kernels for gfx1xxx are compiled for wavefront-32 by default.
Change-Id: Ic6c2917b7e6d3704daf742d243f5ec7f49430de9
[ROCm/ROCR-Runtime commit: f7e3782b42]
Add two new agent info fields:
HSA_AMD_AGENT_INFO_UCODE_VERSION
HSA_AMD_AGENT_INFO_SDMA_UCODE_VERSION
Change-Id: I51cb853724b23a26e945e5c1ac32c16d0cb3bc31
[ROCm/ROCR-Runtime commit: ecdebef0b9]
Thunk keeps an internal cache of system topology that can be used to
speed up subsequent calls to hsaKmtAcquireSystemProperties(). This cache
is cleared by calling hsaKmtReleaseSystemProperties() at the beginning
of BuildTopology().
hsaKmtRuntimeEnable() also calls hsaKmtAcquireSystemProperties() inside
Thunk. Move call to hsaKmtRuntimeEnable() after BuildTopology() so that
we can re-use Thunks internal cache.
Parsing of of topology can take ~150 ms on systems for large number of
nodes.
Change-Id: I741709d49d67d244f5fbd707fe8f01ab923bb153
[ROCm/ROCR-Runtime commit: e39ad34d9c]
Simplified the callback method. Also fixed the way, loaded shared object were getting appended into a string vector,
which was not being passed to this callback method.
Change-Id: I68661dd73f61a11c42fa92f670e8e7b6ffcb5711
[ROCm/ROCR-Runtime commit: 8751e65b79]
Previous versions of HIP will call hsa_amd_ipc_memory_create with then
len aligned to granularity. Temporarily allow this so that we go not
break backward compability. Will remove this after 2 releaes
Change-Id: I6b5ac2cad5d32d62c803637cf1a2c6deebc03169
[ROCm/ROCR-Runtime commit: cb71e2d715]
MES devices need GART mappings and therefore need non-paged memory. But
using non-paged memory introduces performance regression where it can
take over 80 ms to see the signal changes if the memory is in the wrong
NUMA node. Currently, we cannot control NUMA affinity when allocating
non-paged memory. Using non-paged memory allocation only on devices that
have MES scheduler
Change-Id: Ib27fb01d75247aa4f2bb2aa4503c6af5a98afda0
[ROCm/ROCR-Runtime commit: c1e836b6ab]
Using previous method of std::thread for SVM profiler task was causing
segfaults on thread launch on RHEL 8 if libhsa-runtime library is loaded
using dlopen.
Change-Id: Ic010cd6ae9bc6e6ed0605de02b93f6aae8ed3e97
[ROCm/ROCR-Runtime commit: 0e4c7336ff]
Transient exec usage is not required for GFX11 and will result in a NULL
return of s_sendmsg_rtn if directly returned to exec_lo.
Directly fetch and mask the doorbell ID to ttmp3 for GFX11 instead.
Change-Id: Ie17ed69d68d84ab18869b1c7871a0ed0482cd661
[ROCm/ROCR-Runtime commit: f9edf73cd7]
If hsa_amd_agents_allow_access is called for an imported IPC handle,
ignore the request as this pointer will already have these pointers
mapped to other GPUs during IPCAttach()
Change-Id: I4bf33ed57e93b5a3ead749d4f87ab6f2750bed58
[ROCm/ROCR-Runtime commit: b4f26534eb]
If a user queries the pointer info on an invalid pointer,
hsaKmtQueryPointerInfo will return error or unknown pointer. The other
fields in HsaPointerInfo are invalid, so we do not return them to the
user.
Also removing the assert and returning unknown pointer instead. As the
assert will not trigger in release builds.
hsaKmtQueryPointerInfo may also return unknown pointer for userptrs as
they are not always tracked by thunk. Adjusting code to still treat
these pointers as valid in this case.
Change-Id: Idf5cd8b61cd532d31b072f449839d223369bb138
[ROCm/ROCR-Runtime commit: 18547173e9]
Amount of memory requested by user may be aligned-up internally to
the memory pool granularity. The extra padded memory should not be
considered when validating pointers from the user. Also return the
user requested size when user queries pointer information.
Change-Id: I28b25448ea03c836b44fafdb34b7330cf6887424
[ROCm/ROCR-Runtime commit: 39632a713e]
Add agent info query HSA_AMD_AGENT_INFO_ASIC_FAMILY_ID.
Then we can remove the codes to parse family id.
Signed-off-by: Lang Yu <Lang.Yu@amd.com>
Change-Id: I3ac4746d3015e89b32322ebc0f8a3084f98677a4
[ROCm/ROCR-Runtime commit: d0e7c617df]
The allocation logic of the SPI does not take into account compute
user thread management settings for masking CUs with the exception of
skipping fully disabled SEs. This means that occupancy limited
dispatches such as cooperative launch may over allocate onto hardware
resources that are not immediately available, resulting in a potential
barrier logic hang as occupying work groups are waiting on enqueued
work groups to reach the barrier.
Further work will have to be done to get the per-SA CU enablement count
from the KFD in order to correctly clip the cooperative CU limit based
on the CU mask, which will require breaking the current ABI.
For now, report that cooperative launch is not supported while a CU
mask has been applied to prevent potential shader hangs.
Change-Id: I8be4bb47d65ceb62d805f36ef6ef3996d756021f
[ROCm/ROCR-Runtime commit: 2b75a73ce7]
New environment variable HSA_OVERRIDE_CPU_AFFINITY_DEBUG to
enable/disable overriding CPU affinity.
Default value is enabled(1).
This is a temporary variable and may be removed in the future.
Change-Id: Id6a7c611730471ddc276ca333fde1e57046bf32a
[ROCm/ROCR-Runtime commit: df3fe8c2fb]
Add support to expose executable bit.
Change-Id: I054f5c3173822c369dd9908eec5c449459600ce1
Signed-off-by: David Yat Sin <David.YatSin@amd.com>
[ROCm/ROCR-Runtime commit: a7db31c5d1]
Non-paged allocation for queue memory necessary for binding wptr to
GART. Required to support usermode queue oversubscription with MES for
GFX11.
Adds AllocateNonPaged entry to MemoryRegion::AllocateEnum for clarity;
aliases AllocateIPC.
Signed-off-by: Graham Sider <Graham.Sider@amd.com>
Change-Id: I1a97a1820da26cf2433d9c237b2e6d2b0b8628b4
[ROCm/ROCR-Runtime commit: 061aa04147]
GPUs excluded by RVD are not expected to have scratch, memory, trap
handling nor memory regions set up. Now that these GPUs are added to
a new list, early return on agent destruction to prevent bad function
calls on destroy.
Also fix up broken memory releases between the gpu lists and ugly braces.
Change-Id: I52fc6e86ceba0a0383cedc63310eb409515eaf9f
[ROCm/ROCR-Runtime commit: 9d2fe1ac2a]
Atomic memory operations on these memory buffers are not guaranteed
to be visible at system scope
Change-Id: I4cccde114632071a000384502a83bc191e77e85b
[ROCm/ROCR-Runtime commit: 364715cbc6]
A work group processor (WGP) require both its CU to be enabled
in order to be enabled.
The KFD will round robin distribute by even-indexed pairs so
enforce this requirement for runtime set mask calls.
Change-Id: Ic46661b01f398aa1fe24d96b5c9c31f122f967a3
[ROCm/ROCR-Runtime commit: f600687537]
Discovered agent handles should only apply to copy routing, not to
copy device selection. The user may not have mapped all allocations
to all GPUs so we must ensure that the copying device is one passed
by the user.
Change-Id: I2532e66d30e6842624e594f235dd144a186220d4
[ROCm/ROCR-Runtime commit: a8603b9397]
Sean Keely
Jonathan Kim
David Yat Sin
Cordell Bloor
Shweta Khatri
Alexander Turek
David Belanger
Lang Yu
Yifan Zhang
Graham Sider
Jay Cornwall
skhatri