Added HSA_IMAGE_ENABLE_3D_SWIZZLE_DEBUG environment flag to
enable/disable this. Default value is false (view3dAs2dArray = 1)
Enabling this flag will enable support for swizzles that do 3D
interleaving. Note that all features of 3D images are supported
with 2D swizzles,it's just that the access patterns are different
and therefore cache hit-rates may be better or worse, depending
on how it's used. Volumetric algorithms do better with 3D and apps
that tend to access a single slice at a time do better with 2D.
Change-Id: Id8574a6710fe4333a1ee331e5ce9195a81434198
[ROCm/ROCR-Runtime commit: 6361466baa]
Replaces WaitAny with WaitMultiple to more closely align with the
underlying driver API for waiting on multiple events.
WaitMultiple adds a single parameter, wait_on_all, to the WaitAny
interface providing a single function for waiting on multiple
events when we only need AND and OR semantics for the signal
checking logic.
Change-Id: I68a4a45d48151d9d69aef02fd8f7263b9e6c0e75
[ROCm/ROCR-Runtime commit: 8a38f121ea]
Set priority to maximum for signal event handler and minimum for
exceptions event handler.
Change-Id: I1b982d3c2e4c880fafc073fe1a542d01692a6fdc
[ROCm/ROCR-Runtime commit: 7ea25ebb85]
Generalize the driver discovery and move driver-specific
functionality to the concrete driver implementations.
Currently, this process is tightly coupled to the hsakmt
which is GPU and OS specific.
Change-Id: Ie1c53fef407a71b5ec4c6eaf3a3ed00871184409
[ROCm/ROCR-Runtime commit: 15107afb11]
The recent static initialization changes cause this clean up to
happen when it previously never did. The result of ~RuntimeCleanup()
being executed is that the static global "loaded_" is set to false,
which in turn prevents hsa_init() from executing again. Clean up
already happens when hsa_shut_down() occurs.
Change-Id: Ib5cefb80d82880c1945e04eb6ec246bc2c7d2324
[ROCm/ROCR-Runtime commit: b1d6cacf79]
This add support for GC version 11.5.3
Change-Id: I1d55e33198620d3493967558c25c636d5f7ab347
Signed-off-by: Tim Huang <tim.huang@amd.com>
[ROCm/ROCR-Runtime commit: e515b0bca5]
This is to avoid use after free at the program's end, when statics
are destructed.
Change-Id: Id6bf26f25a58d13bdf1ee99c852adae8add76569
[ROCm/ROCR-Runtime commit: 67b0082443]
if .supports_exception_debugging is not enabled.
Change-Id: I944fe7aa4f3068964f47e23f5259c3802d1e9556
Signed-off-by: Flora Cui <flora.cui@amd.com>
[ROCm/ROCR-Runtime commit: ac64c54d74]
Minor modifications to multiple source and header
files based on Coverity report
Change-Id: I4a73d0f56640983c4d5124e13c8c280245cca672
Signed-off-by: Apurv Mishra <apurv.mishra@amd.com>
[ROCm/ROCR-Runtime commit: 699d0140be]
Add return checks, initialization and clean
redundant memory operations
fix 1: check return value of 'setsockopt' for error
fix 2: check return value of 'PtrInfo' for error
fix 3: move 'tool_names' instead of copying
fix 4: call 'munmap' for 'va' only once
fix 5: use 'ssize' for possible return values of -1 (err)
fix 6: add missing initialization in constructors
fix 7: add initialization for some scalars and pointers
Change-Id: I07d90e36d4e1fe48c4de4f44e18083e5ed4c5fbc
Signed-off-by: Apurv Mishra <apurv.mishra@amd.com>
[ROCm/ROCR-Runtime commit: 441bd9fe6c]
Make sure waiting_ count for queue signal is always > 0 so that we
always call hsaKmtWaitOnEvent to force hsaKmtWaitOnEvent to return.
Remove incorrect warning print when running in debug mode.
Call internal Signal::WaitAny instead of AMD::hsa_amd_signal_wait_any
to avoid extra function calls.
Change-Id: I9e41b704643e4e8ee7402b1379b1c30ff4c544ef
[ROCm/ROCR-Runtime commit: 5da1889fb7]
Check that generic ISAs are compatible with an agent separately
from where feature compatibility is checked.
Change-Id: I403012db5536ff1f2faf93cf013db03ef07ac1c8
[ROCm/ROCR-Runtime commit: e93efba9cc]
'const' member functions have syntax errors and struct
'data_rdy' have uninitialized members
v1: correct misplaced 'const' for member functions
v2: add initialization for 'data_rdy' in constructor
Change-Id: I29bada475217c9df81f0d0400e7a3f44aa8afe0c
Signed-off-by: Apurv Mishra <apurv.mishra@amd.com>
[ROCm/ROCR-Runtime commit: c48e8a918e]
When all 64-bits of the signal value are 0, we can skip polling for that
signal.
We need to keep signals as 64-bit numbers as part of the spec. But most
users of ROCr do will never set the signal value to more than 32-bits.
When the dependent-signals are less than 32-bits, avoid adding extra
SDMA poll packet as this adds latency to the SDMA copies.
Change-Id: I37dca65fe3f060dc7164f49b98cb1985023663c4
[ROCm/ROCR-Runtime commit: 0544c2336b]
This was missing from a previous commit regarding
dynamically allocated static data structures.
Change-Id: Iae1c674e762f85e3aebf338210ba96942ba80278
[ROCm/ROCR-Runtime commit: eec2130443]
Removed 'args' as a unique pointer and deletion in
'ThreadTrampoline', then declared as a class member.
Change-Id: Ia52058392d0170e8b5e57cfdd2c587f47a6f93f0
Signed-off-by: Apurv Mishra <apurv.mishra@amd.com>
[ROCm/ROCR-Runtime commit: 89115369cc]
Added initializations for 'ext_table' in 'hsa_system_get_major_extension_table()'
Change-Id: I5e46592192b7d7a294d30011481f16e93db11794
Signed-off-by: Apurv Mishra <apurv.mishra@amd.com>
Reviewed-by: David Yat Sin <david.yatsin@amd.com>
[ROCm/ROCR-Runtime commit: d91a14ae0c]
Some KFD versions can return from hsaKmtWaitOnMultipleEvents_Ext without
any wait and require the second call without age array init.
Change-Id: I8358c33080084d47c273c2a2827085d0570c8201
[ROCm/ROCR-Runtime commit: 816af44b05]
Initialized 'scratch_base' as a nullptr to avoid
uninitialized read in hsaKmtAllocMemory()
Change-Id: I3b0e67f3fd3b591e1d21d691f0777b1d1a059b73
Signed-off-by: Apurv Mishra <apurv.mishra@amd.com>
[ROCm/ROCR-Runtime commit: 6f6ee9679c]
Added check and initialized parameters for PtrInfo().
v1: Checking if PtrInfo() returns success.
v2: Initialization for variables being passed to PtrInfo().
Change-Id: If3ec4608c8e58be259b4fd51ad681b9bc34ddff6
Signed-off-by: Apurv Mishra <apurv.mishra@amd.com>
Reviewed-by: David Yat Sin <david.yatsin@amd.com>
[ROCm/ROCR-Runtime commit: 610f8a1e0f]
WaitSemaphore and PostSemaphore are used in the HybridMutex
implementation. If HybridMutex did not have to call WaitSemaphore when
acquired, then calling PostSemaphore would cause the internal count
inside sem_t to slowly grow to large values and eventually cause
overflow.
Change-Id: I173fc17c874b49926e56991405e9086ea8c138fc
[ROCm/ROCR-Runtime commit: f58aff630c]
Add support for abort timeout when hsa_signal_wait_relaxed is called and
signal does not clear within timeout.
timeout is in seconds
Change-Id: If1db5a8af33c82ddc4b48968c3d8eceb97d0ea6d
[ROCm/ROCR-Runtime commit: 4ec730f1dc]
To allow non-POD global variables to last until the last thread
has exited, use "new" to allocate the memory instead of static
allocation.
Change-Id: Ica571b61ff8068a52e472c49cb1c44917e60c8c8
[ROCm/ROCR-Runtime commit: 0878deda17]
Also restore dead signals cleanup for old path when HSA_WAIT_ANY_DEBUG
is used.
Change-Id: I51a7404991443c9f6cbf57b4b9e9faa694b9538c
[ROCm/ROCR-Runtime commit: 700f1d9abd]
An ASAN run of the release build revealed some elements of
the supported_isas static map were still using stack data. This
change makes it use heap data so it will persist.
Change-Id: Ie51887e88b9e2dec27acfc97ea45a6219fea971c
[ROCm/ROCR-Runtime commit: c7521a5f2a]
SDMA queue resources are limited when all SDMA copies are bottle necked
into 2 engines. Callers will not be able to make the best decisions
to allocate queue resources fairly so have ROCr fallback to old round
robin behaviour dictated by KFD.
Change-Id: I93d52297976d74e20129c5eb1dcfbfa5aa5067a7
[ROCm/ROCR-Runtime commit: 7f8676e177]
These are mostly AIE related, but there are a couple of others.
Change-Id: I549e004772160ca282d4c94dc9d94dd2ccae8b1c
[ROCm/ROCR-Runtime commit: 08699069d6]
- Add the new path to avoid WaitAny() calls in AsyncEventsLoopp() with
HSA_WAIT_ANY_DEBUG key. The new path is selected by default.
The optimizaiton combines all logic of WaitAny() in a single processing loop
and avoids extra memory allocations or ref counting. Also it won't spin
on the CPU if all events are busy.
Change-Id: I197ce60d0d023fbb672f700d6e87702686f1f55a
[ROCm/ROCR-Runtime commit: 0fc7369ba5]
Shweta Khatri
Tony Gutierrez
David Yat Sin
Ben Vanik
Longlong Yao
Swati Rawat
Chris Freehill
Flora Cui
Shane Xiao
Tim Huang
Apurv Mishra
Eddie Richter
Yiannis Papadopoulos
taosang2
German Andryeyev
Konstantin Zhuravlyov
Jaydeep Patel
Jonathan Kim