Runtime can't assign internal HSA signals for HIP events, because
HIP application can destroy the HIP stream or signal reuse may
occur internally. Switch to global HSA signals for HIP events.
Change-Id: Ieaea2d6b039e492b2e7c5112782a8f4e601e50a1
Add lock protection for signal processing
If signal is reused, then disable reference to it from HIP
Increase the pool signal size to 32
Change-Id: I7d529b35910f83ce577c9eca6d3386759611ccc0
- Create an env var ROC_ACTIVE_WAIT_TIMEOUT to set active wait timeout
- Record profiling informaion if marker_ts_ property is valid.
Change-Id: If0d8aec8d9b0715027cf0f7c3dc8a4c722a6bae6
With HIP API callback runtime has to stall the queue until the
callback is done. Rocclr will introduce SW blocking HSA signal,
which will be released after the callback is done.
Change-Id: I6411f3efab31b468e3b87ebb5c8d155e116b613d
- Make sure barrier with dependent signals issues before queue
index reservation
- Don't issue extra barrier if it's already a barrier command
with dependent signals
Change-Id: I179a8b7adac79eed698f4a4d9eca2606d8e913aa
This addresses the rocprof hang seen with direct dispatch. The
workaround queues the handler back if any of the signal value in the batch
is not decremented. To rememmber the last position in the list, we save
the parsed command in the current timestamp struct.
Change-Id: I02959e463cfe3cee83c54808ffd6e6f48f43b4e8
MT doesn't use GPU waits, but CPU for sync between engines.
Change the threshold values for CPU waits for direct dispatch.
That will bring behavior closer to MT.
Change-Id: Ia41c3cb812614962aff2746b6cf858f1bf77dda2
The check has to be performed inside the signal loop, because
active signals need to be processed to avoid a stale timestamp
class.
Change-Id: I26af8287aae18eb19c096d9358cd0b86cfd1c561
- With direct disaptch profiling state is enabled to trigger the
callback on HSA signal. However ROCr has very low peformance on
the first call to get the profiling info. That impacts some tiny
performance tests.
Change-Id: Idacd1b10a473fcfb5feef3074b7191d35743f769
The copy image workaround could be recursively used by ROCclr blit kernel.
Avoid such situation by using stack variable.
Change-Id: Iadaa8cad9216220194760dd461a9533bb236aea0
HIP tests require HIP callbacks to be processed in another thread.
This change will use a thread from HSA signal callbacks to make
sure a HIP callback was done asynchronously.
Also process the callback before changing the status of command
Change-Id: Icef85d0e0f808663882cf6881ff1be3e5eca29ac
- Don't notify if the batch is empty, because that means
the current command was processed already.
- Disable pinning optimization to avoid a race condition on stall.
- TS marker submition requires extra AQL barrier
to track the status.
Change-Id: I17eff4ad12ac66cfe1bb44048bebb1891805279d
GPU waits have noticeable overheads on compute with extra
AQL barrier packet and on SDMA with power saving features. This
change introduces a wait on CPU for 30 us in case the app has tiny
operations.
Change-Id: I761ba3af595f3f48544980058a9077dda15aa5f9
This change unifies the hostcall implementation for all the backends,
by pushing the common logic to the device layer. This is done by
replacing the use of hsa_signal_t with device::Signal (a light wrapper
around it).
Change-Id: I7b6fca7930b5a0b199da5d85e2e048354cc04e7b
- Avoid GPU wait on the marker submission and update the command
batch after HSA signal callback upon HSA barrier completion.
Change-Id: I5c1c97212aefc2ae4b99aa9e2a81627ee9a38c1c
In pitch workaround, we need to copy the image to copy buffer
when application wants to read image to buffer. After this
patch, we unconditionally copy the image data to the copy buffer.
Change-Id: I71b0d19459542dfbb3ca51a2c8a3a81367fa2fb5
The existing workgroup calculation logic for GWS initialization is
incorrect. It tries to add together workgroups across dimensions,
leading to major under-count in 2D and 3D kernels. An (x,y,z) kernel
uses x * y * z blocks, not x + y + z.
In addition, the previous logic was incorrect for the case of launching
a single-threaded kernel. It calculated 0 workgroups, leading to
initializing GWS to -1.
Change-Id: I1bb20a0d5b6e0cc10ac55901c28d8f93aac61c09
- The logic will trace compute, sdma read/write operations and
apply signals when necessary
- ROC_CPU_WAIT_FOR_SIGNAL, ROC_SYSTEM_SCOPE_SIGNAL
and ROC_SKIP_COPY_SYNC were added to control the tracking
Change-Id: I9e8e6174c63bf7784f7ab00964e2918c8667d364
- ROCR fails the call for some reason, then the signal will
become invalid and can hang on a wait. The logic will reset the
active signal in such cases
Change-Id: Ia131420200f1bbd7c9a162b8f1b06db8cecf41c6
- There is a performance regression with a HW wait for HSA signal
on ROCr async operation. For now move the logic back to CPU wait.
- Fix profiling issue with multiple HSA signal per single timestamp
object. Some copies require multiple ROCR calls and if profiling is
required, then the execution time is derived from all used signals.
Change-Id: Id003e4abb8c2de378eedc152a7e389500fc6f4ce
Implement the global class for signals tracking per device queue.
Switch to the new tracking mechanism.
Change-Id: I3c4dda04b34e6d18d6a95510d84102909633b415
Use barrier packets for every profile marker that gets submitted
and use the completion signal to get GPU ts. This gives most accurate
dispatch time. Club cache flushes with profile marker if there is a
pending dispatch that needs cache flush. This optimization saves on
extra barrier and helps wall time
Change-Id: Ib62d6d7aabf4743827b561be6c9c5afa813203da
OCL can't distinguish different copy types, but ROC profiler
expects SDMA transfer visibility. Add extra code to detect
a transfer with the host memory and substitute OCL command
Change-Id: I5290acd0e10bc082e00c1d4ae1474a075de7f165
Enable this optimization when the barrier is disabled, since
reuse requires a signal wait.
Use the size of pending AQL signals as the size of signal pool.
Change-Id: I2754a0f8b67e19d2601c58945e10fdf0e8be1624
The change reuses HSA signals for dispatches as a wait signal.
Skipping the barrier requires to disable L2 cache for sysmem
allocations and extra tracking for HDP access with the large bar.
ROC_BARRIER_SYNC=0 activates the new logic. Barrier sync is
still used by default.
ROC_ACTIVE_WAIT=1 enables unconditional active wait in ROCr.
The change also consolidated ROCr wait logic under single function.
Change-Id: I6bd1be30aa88258da1b1f9de319ef5a45852afd8
SWDEV-249719 - root cause: queues with custom CU mask are not inserted
into queuePool_ (i.e., queue of reusable HSA queues) of ROC device class
causing a crash when creating hostcall buffers for printf
Change-Id: Ieee7005d9a5a30b3113394ce23ee65927126d0d6
root cause - cooperative queue is not inserted into queuePool_ (HSA queues) of ROC device calss causing a crash when creating hostcall buffers for printf
Change-Id: I3f9aceb4e5fe6a7c7a2a549a4bb0a3511fe02799
Fix a typo with the name define, when compilation wasn't enabled.
Force CPU prefetch if system was forced in runtime
Change-Id: Id4b578f9fa44a45426fdb5d8ecb1da803aa42313
agunashe
Saleel Kudchadker
German Andryeyev
Alex Xie
Vladislav Sytchenko
Ravi C Akkenapally
kjayapra-amd
Payam
Joseph Greathouse
Jason Tang
Sarbojit Sarkar
Aryan Salmanpour