Unless SDMA blits have actually been used for copies, prevent the DMA
copy status from querying the blit's pending byte status to avoid
creating an unnecessary HW queue.
Change-Id: Ied1fbed73c08f0408f0e3583f9b56f2768c71708
Querying pending bytes on a blit kernel is unnecessary when runtime
runs out of SDMA resource since we are returning an SDMA availabilty
mask.
Change-Id: I347efba0c85b70ea3ba8749d76a499afc23909e8
Added HSA_AMD_MEMORY_POOL_GLOBAL_FLAG_EXT_SCOPE_FINE_GRAINED flag to enable extended scope memory region
where the device-scope atomics act as system-scope atomics
Change-Id: I79fc3207cb630dfc68bed2f8aabd75f35fe80b12
Enable sleep for all waiters with event age tracking support kernel.
Change-Id: Icd4e1e8d83b4a54e9f6aaa99691a6573211b3337
Signed-off-by: James Zhu <James.Zhu@amd.com>
KFD kernel version 1.13 starts to support event age
tracking which help elimating unncessary busy wait.
Change-Id: Ib447ed6e0350f3110a4d6b9b80a0388000dd0e72
Signed-off-by: James Zhu <James.Zhu@amd.com>
Earlier, hsa-runtime was unable to find symbols from a stripped ELF-image becasue
no support to find symbols from ".dynsym" section.
Looking for symbols in .dynsym is enabled by LOADER_USE_DYNSYM=1
environment variable
Change-Id: I4f0e8dd0eb053a6066d4d49b670c52e51149531a
GFX11 and up including some GFX9 devices will not support
old trap handling without the new exception handling.
Instead of a hard assert failure that runs into a core dump,
let ROCr initialization continue instead.
Change-Id: I309becdc72ef4fb2fafd118c1faf0801407e658e
status.priv may be read after returning from the trap handler, which
causes sq_interrupt_word_wave.priv to be 0 even though the s_sendmsg
instruction was initiated when status.priv was 1.
To work around this, added a s_waitcnt lgkmcnt(0) after s_sendmsg
to make sure the message is sent before continuing.
Signed-off-by: Jay Cornwall <Jay.Cornwall@amd.com>
Signed-off-by: Laurent Morichetti <Laurent.Morichetti@amd.com>
Change-Id: Ieb75005ca1559ef03d0efac80e966f521e41fcb7
Removing this definition as this should already be defined by compiler.
This is causing compile errors on newer versions of llvm because the
macro is being redefined.
Change-Id: Ica6a06f46a14e16d3f52e83b9b5ee8cfd7359510
Adding support for gfx941 and gfx942 ISAs.
gfx940 ISA will use sc0:1 sc1:1 on load/store operations
gfx942 ISA will use default load/store operations
Change-Id: If1efbef86f59e2cf2d48fe359cd4166405a0a579
When compiling in ASAN mode, remap the first page of device allocations
to system memory. ASAN's memory allocator uses a small amount of extra
memory to store data for housekeeping purpose. But because this memory
is from the GPU memory pool, it might have uncommon memory type for host
to access. Mapping this section of memory to the host makes this memory
accessible to ASAN.
Change-Id: I36f659d616a4d15558372592439a8723c5c84a69
Signed-off-by: Bing Ma <Bing.Ma@amd.com>
Add support for HSA_ENABLE_PEER_SDMA env variable that can be used to
disable use of SDMA engines for device-to-device transfers. Note that
setting HSA_ENABLE_SDMA=0 will disable all SDMA transfers and override
HSA_ENABLE_PEER_SDMA values.
Change-Id: I737b3c2b2efcf3ff237f98bc748f49b8252ed24a
Update documentation for hsa_amd_pointer_info to clarify which fields
are invalid when the allocation type is HSA_EXT_POINTER_TYPE_UNKNOWN.
Change-Id: Idaed985962c4a98d281ebe01bef8ec2459da3985
Some workloads running on multi-GPU create 1 process per GPU. So each
process creates a GPU agent on every GPU, but will only create queues on
one GPU. This would cause un-necessary scratch reservation.
Change-Id: I50a216f0bcc0b5f707f3943147390b0ecec1ac22
If the required scratch allocation is too large, ROCr will attempt to
reduce it by lowering the dispatch's targeted occupancy. The reduction
loop however was prone to overflow if waves_per_cu was not a multiple of
waves_per_group. Ensure no overflow by aligning waves_per_cu to
waves_per_group.
On GC 9.4.3 dGPU, dispatches with a large grid size and a
waves_per_group of e.g. 16 may require to reduce occupancy such that
waves_per_cu is less than waves_per_group to ensure the allocation size
is small enough. Allow this while also ensuring the tmpring scratch wave
count is kept divisible by the number of SEs per XCC.
Signed-off-by: Graham Sider <Graham.Sider@amd.com>
Change-Id: Ie4016dcd8166a9ae69e9decc26a3eec882b49480
Scratch cache reserved memory is only available for scratch memory use
so do not report this memory as available to the user via the
HSA_AMD_AGENT_INFO_MEMORY_AVAIL api.
Change-Id: I52f96e62536458bcaa52b9f4be5de856d5680dc4
Add HSA_ENABLE_SRAMECC environment variable that can be used to
override SRAM ECC mode reported by KFD
Change-Id: I2b95511820a2d3d146a76b03070659c0695b61fd
The gfx940 does not support IMAGE instructions. Any get_info with
IMAGE attributes should return failure.
Signed-off-by: Mike Li <Tianxinmike.Li@amd.com>
Change-Id: I12005628f92780f551ab6f8b41526c66b54c6a59
gfx940 uses ttmp11 to hold the queue packet index so the first level
trap handler uses ttmp13 instead to save ib_sts.
Repurpose ttmp11[31] to mean that the ttmps are initialized. The issue
was that the debugger could not tell whether ttmp6 was written by the
trap handler when determining the stop reason.
If ttmp11[31]=0, then the trap handler has not been executed and ttmp6
should be assumed to be 0. If ttmp11[31]=1, then ttmp6 holds the
trap_id, if an s_trap instruction caused the exception.
Signed-off-by: Laurent Morichetti <laurent.morichetti@amd.com>
Signed-off-by: Lancelot Six <lancelot.six@amd.com>
Change-Id: I9af903abae044b9ec530306229caf3b883f3ee46
On Linux, the os_thread abstraction is built on top of pthread. Many of
the pthread calls might fail and return error codes. The error
conditions are only checked via assertions (if ever checked) which means
that when doing a release build, no error condition is checked. The
same goes for dlsym/dlinfo and clock_gettime.
This commit improves the situation this by checking the error conditions
and acting accordingly. When the error condition is detected in a
function with a mean to indicate some error to its caller, then this
patch prints some error message and returns. If there is no way to
propagate the error up the call stack, print some error message and
abort the process.
For the os_info::os_info ctor, the only user is CreateThread, which
checks that the built thread is Valid(). If not, nullptr is returned to
the caller.
It could be possible to use exceptions when functions cannot pass
errors, but for now I only use abort as it is what abort would do with
debug build.
Change-Id: I815703c3b95777cc29bb89a7d654ac879c14a759
When building with g++-11.3.0, I have the following warning:
/home/.../core/runtime/runtime.cpp: In member function ‘hsa_status_t rocr::core::Runtime::GetSystemInfo(hsa_system_info_t, void*)’:
/home/.../core/runtime/runtime.cpp:693:56: warning: suggest parentheses around ‘&&’ within ‘||’ [-Wparentheses]
693 | kfd_version.KernelInterfaceMajorVersion == 1 &&
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~
694 | kfd_version.KernelInterfaceMinorVersion >= 12)
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
This patch adds the parenthesis as suggested. This silences the
compiler warning.
No functional change expected.
Change-Id: I69c1a73a432b0f2393dbaf36d4424cf0056c535f
We used to report HSA_STATUS_ERROR_INVALID_ISA when receiving error code
128, but there are several other reasons why we could be exceeding
number of VGPRs, so updating the error code.
Change-Id: I6a6980d5b07b09c93d00dee5207a0d52399bc77e
One some platforms, e.g Arch Linux, -D_GLIBCXX_ASSERTIONS compile flag
is enabled by default, causing a runtime assertion.
Avoid assertion by using std::vector accessor function data().
Change-Id: I118cdf102c3e353f32c618823e363ee1059f3453
Some applications will keep trying to allocate device memory until the
allocation fails. This causes all device memory to be used up and we are
then unable to allocate scratch memory for dispatches. Reserve enough
memory for 1 small scratch allocation.
Change-Id: I968400d41540ba1aca8f28581f229693eec02225
Wait on completion signal for amd_aql_pm4_ib processing
on ASICs with gfx version >= 9.
Signed-off-by: Lang Yu <Lang.Yu@amd.com>
Change-Id: Ia704d9cc5b2535dcf8564a30f694262b113f77a2
Engine offset that is the maximum number of engines is still valid
as offset enum 0 is occupied by blit copies so raise the limit by 1.
Change-Id: I6fcab106290e6647702efe297a4281861da4e0b8
Konstantin Zhuravlyov
Jonathan Kim
Shweta Khatri
James Zhu
raghavmedicherla
David Yat Sin
Laurent Morichetti
Ammar ELWazir
Xiaomeng Hou
Ranjith Ramakrishnan
Graham Sider
Saleel Kudchadker
Sam Wu
Mike Li
Alex Sierra
Lancelot SIX
Konstantin Zhuravlyov
Lang Yu