Make sure waiting_ count for queue signal is always > 0 so that we
always call hsaKmtWaitOnEvent to force hsaKmtWaitOnEvent to return.
Remove incorrect warning print when running in debug mode.
Call internal Signal::WaitAny instead of AMD::hsa_amd_signal_wait_any
to avoid extra function calls.
Change-Id: I9e41b704643e4e8ee7402b1379b1c30ff4c544ef
[ROCm/ROCR-Runtime commit: 5da1889fb7]
Check that generic ISAs are compatible with an agent separately
from where feature compatibility is checked.
Change-Id: I403012db5536ff1f2faf93cf013db03ef07ac1c8
[ROCm/ROCR-Runtime commit: e93efba9cc]
'const' member functions have syntax errors and struct
'data_rdy' have uninitialized members
v1: correct misplaced 'const' for member functions
v2: add initialization for 'data_rdy' in constructor
Change-Id: I29bada475217c9df81f0d0400e7a3f44aa8afe0c
Signed-off-by: Apurv Mishra <apurv.mishra@amd.com>
[ROCm/ROCR-Runtime commit: c48e8a918e]
When all 64-bits of the signal value are 0, we can skip polling for that
signal.
We need to keep signals as 64-bit numbers as part of the spec. But most
users of ROCr do will never set the signal value to more than 32-bits.
When the dependent-signals are less than 32-bits, avoid adding extra
SDMA poll packet as this adds latency to the SDMA copies.
Change-Id: I37dca65fe3f060dc7164f49b98cb1985023663c4
[ROCm/ROCR-Runtime commit: 0544c2336b]
This was missing from a previous commit regarding
dynamically allocated static data structures.
Change-Id: Iae1c674e762f85e3aebf338210ba96942ba80278
[ROCm/ROCR-Runtime commit: eec2130443]
Removed 'args' as a unique pointer and deletion in
'ThreadTrampoline', then declared as a class member.
Change-Id: Ia52058392d0170e8b5e57cfdd2c587f47a6f93f0
Signed-off-by: Apurv Mishra <apurv.mishra@amd.com>
[ROCm/ROCR-Runtime commit: 89115369cc]
Added initializations for 'ext_table' in 'hsa_system_get_major_extension_table()'
Change-Id: I5e46592192b7d7a294d30011481f16e93db11794
Signed-off-by: Apurv Mishra <apurv.mishra@amd.com>
Reviewed-by: David Yat Sin <david.yatsin@amd.com>
[ROCm/ROCR-Runtime commit: d91a14ae0c]
Some KFD versions can return from hsaKmtWaitOnMultipleEvents_Ext without
any wait and require the second call without age array init.
Change-Id: I8358c33080084d47c273c2a2827085d0570c8201
[ROCm/ROCR-Runtime commit: 816af44b05]
Initialized 'scratch_base' as a nullptr to avoid
uninitialized read in hsaKmtAllocMemory()
Change-Id: I3b0e67f3fd3b591e1d21d691f0777b1d1a059b73
Signed-off-by: Apurv Mishra <apurv.mishra@amd.com>
[ROCm/ROCR-Runtime commit: 6f6ee9679c]
Added check and initialized parameters for PtrInfo().
v1: Checking if PtrInfo() returns success.
v2: Initialization for variables being passed to PtrInfo().
Change-Id: If3ec4608c8e58be259b4fd51ad681b9bc34ddff6
Signed-off-by: Apurv Mishra <apurv.mishra@amd.com>
Reviewed-by: David Yat Sin <david.yatsin@amd.com>
[ROCm/ROCR-Runtime commit: 610f8a1e0f]
WaitSemaphore and PostSemaphore are used in the HybridMutex
implementation. If HybridMutex did not have to call WaitSemaphore when
acquired, then calling PostSemaphore would cause the internal count
inside sem_t to slowly grow to large values and eventually cause
overflow.
Change-Id: I173fc17c874b49926e56991405e9086ea8c138fc
[ROCm/ROCR-Runtime commit: f58aff630c]
Add support for abort timeout when hsa_signal_wait_relaxed is called and
signal does not clear within timeout.
timeout is in seconds
Change-Id: If1db5a8af33c82ddc4b48968c3d8eceb97d0ea6d
[ROCm/ROCR-Runtime commit: 4ec730f1dc]
To allow non-POD global variables to last until the last thread
has exited, use "new" to allocate the memory instead of static
allocation.
Change-Id: Ica571b61ff8068a52e472c49cb1c44917e60c8c8
[ROCm/ROCR-Runtime commit: 0878deda17]
Also restore dead signals cleanup for old path when HSA_WAIT_ANY_DEBUG
is used.
Change-Id: I51a7404991443c9f6cbf57b4b9e9faa694b9538c
[ROCm/ROCR-Runtime commit: 700f1d9abd]
An ASAN run of the release build revealed some elements of
the supported_isas static map were still using stack data. This
change makes it use heap data so it will persist.
Change-Id: Ie51887e88b9e2dec27acfc97ea45a6219fea971c
[ROCm/ROCR-Runtime commit: c7521a5f2a]
SDMA queue resources are limited when all SDMA copies are bottle necked
into 2 engines. Callers will not be able to make the best decisions
to allocate queue resources fairly so have ROCr fallback to old round
robin behaviour dictated by KFD.
Change-Id: I93d52297976d74e20129c5eb1dcfbfa5aa5067a7
[ROCm/ROCR-Runtime commit: 7f8676e177]
These are mostly AIE related, but there are a couple of others.
Change-Id: I549e004772160ca282d4c94dc9d94dd2ccae8b1c
[ROCm/ROCR-Runtime commit: 08699069d6]
- Add the new path to avoid WaitAny() calls in AsyncEventsLoopp() with
HSA_WAIT_ANY_DEBUG key. The new path is selected by default.
The optimizaiton combines all logic of WaitAny() in a single processing loop
and avoids extra memory allocations or ref counting. Also it won't spin
on the CPU if all events are busy.
Change-Id: I197ce60d0d023fbb672f700d6e87702686f1f55a
[ROCm/ROCR-Runtime commit: 0fc7369ba5]
On GPUs where EOP is handled in asic, the read_dispatch_id is not always
updated after each packet. Look for the first dispatch packet that needs
scratch memory before allocating scratch.
Change-Id: Ibf4b4b485f99bf2fabfe48e9609ca99111fdafbe
[ROCm/ROCR-Runtime commit: d90fbee9c4]
The supported_isas static unordered_map was adding stack
allocated Isa objects. Instead, make the objects statically
allocated, as supported_isas itself is.
Change-Id: I23405e218290d48deea6f984f76c57e7b43e314e
[ROCm/ROCR-Runtime commit: fd99b74287]
When ROCr is built as a static library, global variables
were often not initialized to valid values at their first
use. This change addresses that problem.
Change-Id: I550fa41feb3bc04b9cc686bcfb4acf2a7b651a88
[ROCm/ROCR-Runtime commit: 9b13bcd0ac]
Devices older than GFX90a hit a segfault on queue unmap when an
SDMA queue has been assigned a fixed engine. Bypass fixing the
engine for these devices for now.
Change-Id: I7d2f882d2377f004a7bb65f3b397396db07ce6d3
[ROCm/ROCR-Runtime commit: 1d6ff45673]
Fix memory leak for memory regions objects when GPU is masked using
ROCR_VISIBLE_DEVICES.
Change-Id: I610842a18adbc3cdc854b12650844e271bc00592
[ROCm/ROCR-Runtime commit: dbae8da515]
To correctly map to all GPUs after an import, use the new extended
registration call that can import a virtual address without having to
specify a target node.
Change-Id: Ifca8f6f6ee24fa99b2af357dcc3ea1de3ab234f7
[ROCm/ROCR-Runtime commit: 0ae064fe2d]
When hsa_amd_vmem_set_access is called, do not remove permissions for
unspecified agents. Also updating documentation in header to clarify
this.
Change-Id: I3bb4cf08ba399f85cc67b17fd13a4a40d862415f
[ROCm/ROCR-Runtime commit: 73f6bfa747]
Socket server accept calls do not guarantee synchronous actions
post-accept. This can result in a race condition.
To resolve this, first limit the socket server's listen backlog to a
single connection. This will force competing clients to busy-retry
until timeout.
Second, make the DMABUF IPC file descriptor send-receive and import
calls into an atomic routine per connection.
By doing these fixes, not only to we resolve potential races but
we guarantee that any exporter process will create at most one
file descriptor that will only last for the duration of the import
transaction. This alleviates any concern on running into system
limits for the number of open file descriptors per process.
Change-Id: I6d8b14795a680d89a2707e082fa027d525792e05
[ROCm/ROCR-Runtime commit: 909b82d463]
Discarding blocks for reallocation on IPC export for better memory
performance trigger memory violations with DMA BUF exports so bypass
this for now as application performance drops haven't been observed
with the bypass.
The raw fragment should be passed to the DMA Buf export call as well
since offsets will be implicitly applied in the Thunk/KFD for
export/import calls.
Also, use the agent information directly from the pointer
information so that the export call doesn't have to scan memory to find
this. Pass the node ID in the handle so that the import call doesn't
have to make two thunk imports to fetch the node ID for GPU memory
imports.
Finally, allow the user to use DMA Buf IPC via
HSA_ENABLE_IPC_MODE_LEGACY=0 for developer testing as legacy mode will
be applied by default.
Change-Id: Ie8fe267f8768fa5df37126078406f7065f69ff4e
[ROCm/ROCR-Runtime commit: 32bb0764b7]
When an entry is deleted from the array, it's set to nullptr
but not removed. Most other functions that
iterate over the array check if the entry is nullptr
but this loop in IterateExecutables did not.
Change-Id: I763b361eea59f6df201bb86ead0234e95f2cf79c
[ROCm/ROCR-Runtime commit: f3664fd124]
Return false if trying to free a NULL pointer (or invalid size)
internally in ROCr. This is to detect errors within ROCr when trying
to free NULL pointers. If a user of ROCr tries to free a NULL
pointer, this condition should be caught at the beginning of the
Runtime::FreeMemory(...) function and return HSA_STATUS_SUCCESS. This
matches the behavior of the free(...) or delete functions that
silently ignores calls when the passed a NULL pointer.
Change-Id: I84bc26928b35023e19cd9f214b42c6ee9508029c
[ROCm/ROCR-Runtime commit: 0af7a54ebe]
Refactor VMemorySetAccess so that it can be re-used in the following
patch.
Change-Id: I341241da7a59724bb3611172f0d26b0689d7bb46
[ROCm/ROCR-Runtime commit: 8f1b05660a]
David Yat Sin
Chris Freehill
Eddie Richter
Yiannis Papadopoulos
Apurv Mishra
taosang2
German Andryeyev
Konstantin Zhuravlyov
Jaydeep Patel
Jonathan Kim
Young Hui
James Xu