If the graph has kernels that does device side allocation, during packet capture, heap is
allocated because heap pointer has to be added to the AQL packet, and initialized during
graph launch.
Handle race with wait when 2 kernels with device heap are enqueued on multiple streams.
Change-Id: I45933b77fcaf7bc8fdf1bc906462e32b5d8d3688
[ROCm/clr commit: 57156c524d]
Adding a safety check prevents an invalid memory access
if timestamps and kernelNames vectors are of different size.
The patch also moves the addKernelNames for the accumulate command
into dispatchAqlPacket function.
Change-Id: Iea0927e1253800403a1ae3f3d72de1e7d96476c3
[ROCm/clr commit: d44f44a5b1]
This reverts commit 96e25898c8.
Reason for revert: SWDEV-455075, SWDEV-461507 - This change forces to
use ROCr's copy path. Reintroducing hostBlit copy path for
host-to-host copies.
Change-Id: Ic3c45b49e481c9dcdaa7611f61071778790b7e6c
[ROCm/clr commit: a9f05e22db]
- Print kernelname for graph launches, its hard to correlate packets
otherwise
- Print correlation_id if any
Change-Id: Ib8db7a00e4e7c98f570e71029e61d86f5dccc2ed
[ROCm/clr commit: 72d23a02c5]
- Remove Last graph node optimization and instead submit a barrier NOP
packet always. This simplifies the code.
Change-Id: Ied443173ba47a08b6df148ac7e3ead712acda11c
[ROCm/clr commit: badf2b0880]
The Readback and Avoid HDP Flush memory ordering workaround is
used as a fallback solution only when HDP flush register is invalid
Change-Id: Ic284eba1f95ed22b0270d3abeb904fb902015b1a
[ROCm/clr commit: 6cb7b6ec6b]
- Add LOG_TS mask for printing signal times
- Read raw ticks from signals
Change-Id: Ibdd0bf06c790729f6c65083a4784c97a3c3219e0
[ROCm/clr commit: 948ca5a931]
The "optimized" version of memcpy is outdated and
was used in win32 only.
Change-Id: I7f2e0e9051e37cec95438266824b5b0025c324c6
[ROCm/clr commit: 7448113cfc]
Dispatching multiple packets with ring the doorbell once is not supported by the lower layers
Change-Id: I7665a2dcdd4ef9e47dadfe410180fed64c5a4ee0
[ROCm/clr commit: d7f352dbed]
- Create a vector to allow multiple TS to be stored in Command.
- This would mean we dont wait for entire batch in Accumulate command
to finish when we exhaust signals.
- Reduce the number of signals created at init to 64. This min value
may still need to be tuned but the KFD allows max of 4094 interrupt
signals per device.
- Store kernel names whenever they are available and not just when
profiling. If we dynamically enable profiling like for Torch, a crash
can happen if hipGraphInstantiate wasnt included in Torch profile scope
beacuse we previously entered kernel names only when profiler is
attached.
Change-Id: I34e7881a25bbc763f82fdeb3408a8ea58e1ec006
[ROCm/clr commit: c157bfb202]
- With https://gerrit-git.amd.com/c/lightning/ec/llvm-project/+/1002628 applied, at -O0 Kernel::dynamicParallelism() returns true but virtual queue isn't created
- This causes segfault inside VirtualGPU::submitKernelInternal() when getVQVirtualAddress() is called
Change-Id: Ia7af042adad2329e870c142caaac3e8fa886f8b8
[ROCm/clr commit: d1fff7cea2]
Under ROCr physical allocations don't have initial VA and require extra
flag in ROCclr. Add an option to have a mempool of physical allocations.
Change-Id: I4d062fe0dd8113d4eaf6e8b51749ed56d8701d1e
[ROCm/clr commit: f296159f62]
- Sometimes we want to mask out kernel names, use right level for kernel
logging
Change-Id: Ideae9647c57b86ae390ff2f4131f6d8c6df5c086
[ROCm/clr commit: f1adecd186]
- Print SWq for AQL packets, this helps correlating a stream to the HWq
mapped
Change-Id: I610430c0872a1abc6636027c00163ec46983cd65
[ROCm/clr commit: 984c86f407]
- Enable Device kernel args for MI300* for now.
- Fix a perf issue which impacts graph instantiate when dev kernel args
are enabled.
Change-Id: I962e58fd9d8dd1a8db95e601cb03a8e9c7bac97f
[ROCm/clr commit: 68f40f78dd]
Node can be enabled/disabled only for kernel, memcpy and memset nodes.
If the node is disabled it becomes empty node.
To maintain ordering just enqueue marker with respective node dependencies.
Change-Id: I710f3e88ab4e76c81f6f86a40a7dc61fd4c7e440
[ROCm/clr commit: e0e63eb04d]
- Implement workaround to ensure HDP writes are done by writing and
reading the HDP MMIO register.
- Implement the same workaround for graphs, we no longer need sentinel
write/readback
Change-Id: I0d3027b46a1f61131ec62e3c8c669ff5184fa6b2
[ROCm/clr commit: f138e0d113]
Display queue base pointer in the log. This can be co-related with AQL
packets
Change-Id: I544f9b6db6ae01c85e57e4b3f0b3fffefcd7c2ed
[ROCm/clr commit: 0567c3b720]
- The correlation_id had random junk values which we were inserting in
the dispatch AQL packet even when no profiler was attached but if we had
a valid timestamp.
- Also make sure we dont even write the reserved2 field in the AQL
packet if no profiler attached.
Change-Id: Icdb7493198c1bb5e2d786a97e027288660854cd7
[ROCm/clr commit: 9a6ddae7b2]
This issue is due to the fact that we set fence_dirty_ flag for every
kernel launch. Whenever any next HIP API is caused, the stream logic
assumes fence is dirty and queues a marker, when its not needed.
Change-Id: I10a49ee951daa92221aa6be75e2334849579a45d
[ROCm/clr commit: d6b4892600]
- For new AccumulateCommand we enqueue nop barrier packet. So during stream sync we need system flush.
Change-Id: I0b97b626bcdae582ef95e7c95030d78df1fa5a54
[ROCm/clr commit: 748c6b3520]
It seems that due to removal of vdev()->isHandlerPending(),
Marker queued to ensure finish is not enqueued and that cause
hung at waiting event for kernel enqueue command.
Change-Id: I364abb2dcb4897b11a7eb61b5d85013b69292792
[ROCm/clr commit: eecbc2e436]
- Add the new fillBuffer kernel, which allows to launch a limited
number of workgroups for memory fill operation
- Switch fill memory to 16 bytes write by default
- Allow to limit the workgroups with DEBUG_CLR_LIMIT_BLIT_WG
Change-Id: Ibad1822f2d42b2fc71bcfc1917c31409c0623e8e
[ROCm/clr commit: f1dc81f427]
- Report kernel names for optimized graph path
- Refactor code so that we store profiling info in Accumulate command
Change-Id: Ib97735a0239aeb9fc3a50a4bb7126dd0bcadc8af
[ROCm/clr commit: b056686607]
- Do not use extra barrier to detect graph end. If its a kernel node we
can use a completion signal for the last packet. Saves roughly 6us for
Phantom testcase per graph launch.
Change-Id: I5e0c2479d9964fbeda86ed97533f6718f49a7f91
[ROCm/clr commit: c3bd229f4f]
- Track all captured commands under a new AccumulateCommand
- Add begin() and end() methods to capture commands
- Explicit TS object now passed to certain methods because
profilingBegin() and profilingEnd() now happen separately and thus can
run into threading issues
Change-Id: I171106bdcad72b057836cb2f3fc398db3533119f
[ROCm/clr commit: 40f41f4d0b]
- Refactor code and cleanup logic for callback saving for event records
Change-Id: I5c56aa8e9c968a5bca70fb07ad1796da318e9e89
[ROCm/clr commit: 1338ff37e8]
- Support graph with different types of nodes with single
branch when DEBUG_CLR_GRAPH_PACKET_CAPTURE flag is enabled
Change-Id: I149a8629769cd0d5849ffefb04f1352668a685b6
[ROCm/clr commit: 38d2c56784]
Anusha GodavarthySurya
Ioannis Assiouras
Jacob Lambert
Sourabh Betigeri
Saleel Kudchadker
Satyanvesh Dittakavi
German Andryeyev
kjayapra-amd
Rakesh Roy
cadolphe
Alex Xie
ksankisa
Jatin Chaudhary
Jaydeep Patel