The new logic has a lock for PAL call and doesn't require the lock for queues.
Change-Id: I61b67c3c4abd2ede44809de1d6beed756766032e
[ROCm/clr commit: 2f3ad43c4a]
alwaysResident setting doesn't require per queue residency tracking.
Thus, the logic can be skipped to avoid the lock of queues.
Change-Id: Ib5cff5b79d3ecb8c2f2eb2565cf069f9a69438b0
[ROCm/clr commit: 95e3958748]
- Create a vector to allow multiple TS to be stored in Command.
- This would mean we dont wait for entire batch in Accumulate command
to finish when we exhaust signals.
- Reduce the number of signals created at init to 64. This min value
may still need to be tuned but the KFD allows max of 4094 interrupt
signals per device.
- Store kernel names whenever they are available and not just when
profiling. If we dynamically enable profiling like for Torch, a crash
can happen if hipGraphInstantiate wasnt included in Torch profile scope
beacuse we previously entered kernel names only when profiler is
attached.
Change-Id: I34e7881a25bbc763f82fdeb3408a8ea58e1ec006
[ROCm/clr commit: c157bfb202]
- With https://gerrit-git.amd.com/c/lightning/ec/llvm-project/+/1002628 applied, at -O0 Kernel::dynamicParallelism() returns true but virtual queue isn't created
- This causes segfault inside VirtualGPU::submitKernelInternal() when getVQVirtualAddress() is called
Change-Id: Ia7af042adad2329e870c142caaac3e8fa886f8b8
[ROCm/clr commit: d1fff7cea2]
Under ROCr physical allocations don't have initial VA and require extra
flag in ROCclr. Add an option to have a mempool of physical allocations.
Change-Id: I4d062fe0dd8113d4eaf6e8b51749ed56d8701d1e
[ROCm/clr commit: f296159f62]
PAL optimized the logic for the barriers, which caused failures with CP DMA on Navi4x.
Change barrier's code to match the most recent PAL optimizations.
Change-Id: I55eeab20f51eb8e920bcbb4b55fbe3c7f77fd3fa
[ROCm/clr commit: 1239309c90]
Previously, we used the following approach and Comgr actions
for device lib linking:
AMD_COMGR_COMPILE_SOURCE_TO_BC (compile with clang driver)
AMD_COMGR_ADD_DEVICE_LIBRARIES (link in device libs with
llvm-link API)
However, the clang driver can link in device libraries as part
of compilation, assuming a --rocm-path is set. In this context,
this is accomplished by using the following Comgr action instead:
AMD_COMGR_COMPILE_SOURCE_WITH_DEVICE_LIBS_TO_BC (compile and
link in device libs with clang driver)
Change-Id: Ie0bbee7d9a12672536b6d751056a941128ed58be
[ROCm/clr commit: 6311ed8a8e]
- Sometimes we want to mask out kernel names, use right level for kernel
logging
Change-Id: Ideae9647c57b86ae390ff2f4131f6d8c6df5c086
[ROCm/clr commit: f1adecd186]
The copies can get blocked if the last SDMA engine is used by another
copy and this can lead to perf drop in some of the tests like Gromacs.
Resetting the last engine by checking the engine status and fetching the
new mask after few copies can avoid this.
Change-Id: I8fe8ea678db508d291c6242f3741fa9215e99921
[ROCm/clr commit: 1b25484f0f]
Currently we force inlining everything for HIP. Now we'd like to enable function
supports. The first step is to remove uses of `-amdgpu-early-inline-all` in
various places. This patch is to remove all of them from clr.
Change-Id: Ib0cad1f586714c9989778b00746aa4c47a4eec95
[ROCm/clr commit: a09204388a]
OpenCL printf handling did not process vector of half precision floats properly
(mainly because compiler packs 2 halfs into a dword and runtime failed to extract the
individual parts).
This patch fixes the issue.
Change-Id: Ia1f15ccfb5db52b71c43cfd588dd38f551ee5277
[ROCm/clr commit: 6f390f5af9]
- Print SWq for AQL packets, this helps correlating a stream to the HWq
mapped
Change-Id: I610430c0872a1abc6636027c00163ec46983cd65
[ROCm/clr commit: 984c86f407]
- Enable Device kernel args for MI300* for now.
- Fix a perf issue which impacts graph instantiate when dev kernel args
are enabled.
Change-Id: I962e58fd9d8dd1a8db95e601cb03a8e9c7bac97f
[ROCm/clr commit: 68f40f78dd]
Node can be enabled/disabled only for kernel, memcpy and memset nodes.
If the node is disabled it becomes empty node.
To maintain ordering just enqueue marker with respective node dependencies.
Change-Id: I710f3e88ab4e76c81f6f86a40a7dc61fd4c7e440
[ROCm/clr commit: e0e63eb04d]
__amd_streamOpsWrite blitkernel in device-libs has only 3 args.
so getting rid of the 4th unused arg (sizeBytes)
Change-Id: I81cc1107f8b424bf58558c93a2495a1b878aef91
[ROCm/clr commit: e643406caa]
With multiple HIP streams it's possible to have a race condition when
one thread stops the traces, but another still performs submisisons.
That may cause a crash on the barrier callback.
Change-Id: Ic56f8277fcfd2c2142a4821d927b938b9f313add
[ROCm/clr commit: e2d2fad56c]
- Implement workaround to ensure HDP writes are done by writing and
reading the HDP MMIO register.
- Implement the same workaround for graphs, we no longer need sentinel
write/readback
Change-Id: I0d3027b46a1f61131ec62e3c8c669ff5184fa6b2
[ROCm/clr commit: f138e0d113]
Latest VidMM can provide free memory available on the system.
Use PAL interface to report free memory on the system instead
of per process.
Change-Id: I0e78b9d340299c16829177a8c5182d21cc353384
[ROCm/clr commit: bfa469ddc4]
Display queue base pointer in the log. This can be co-related with AQL
packets
Change-Id: I544f9b6db6ae01c85e57e4b3f0b3fffefcd7c2ed
[ROCm/clr commit: 0567c3b720]
- The correlation_id had random junk values which we were inserting in
the dispatch AQL packet even when no profiler was attached but if we had
a valid timestamp.
- Also make sure we dont even write the reserved2 field in the AQL
packet if no profiler attached.
Change-Id: Icdb7493198c1bb5e2d786a97e027288660854cd7
[ROCm/clr commit: 9a6ddae7b2]
hsa_amd_memory_lock_to_pool() and hsa_amd_memory_unlock()
should be called balanced.
Change-Id: I8b1549861bff752aabbb6399d717d1e346079a38
Signed-off-by: Lang Yu <Lang.Yu@amd.com>
[ROCm/clr commit: cc29df873a]
This reverts commit 1b05247a03.
Reason for revert: Waiting for staging results before finally merging it.
Change-Id: Iaabb510325f50147f368108e98531291217627c0
[ROCm/clr commit: 77be355fd9]
Extension allows to execute the kernels without a wait barrier and L1
invalidation.
Change-Id: I96c485204303f54a0240b93134f4560673e4bd17
[ROCm/clr commit: 13c6f56ca9]
Dont track the status of fence_dirty_ flag on the host, instead clear it
when we submit a barrier on the respective stream.
Change-Id: I4d98dbf20c81379c9c5da9f5b67629a8f9f6dfcd
[ROCm/clr commit: 0b0df605d4]
German Andryeyev
Jaydeep Patel
Saleel Kudchadker
Rakesh Roy
kjayapra-amd
Anusha GodavarthySurya
Ioannis Assiouras
Matt Arsenault
Jacob Lambert
pghafari
Satyanvesh Dittakavi
Shilei Tian
Vikram
cadolphe
Ajay
Alex Xie
Lang Yu