Pass active queue for transfers in the cache coherency layer.
That will allow to use device transfer queue only for
cases when active queue isn't available, because using device
transfer queue from another active queue may cause a deadlock
Change-Id: Ifbe7e0303b77dbf6eeda3939ffbc25a3df7472de
If GlobalMemCacheLine reported is 0, runtime may run into an
infinite loop as the KernelSegmentAlignment is chosen as size of the
cache line.
Change-Id: Ide547940cc0407f16fab10ee210b4fd3ae4eaafc
Metadata in Codeobject version 5 is the extension of CO3 and CO4.
Add the detection of the new fields and program them in
the setup of the kernel arguments.
Change-Id: I27e58df77320ad00f4f16d35912668db803826af
Use HSA_AMD_AGENT_INFO_COOPERATIVE_COMPUTE_UNIT_COUNT to get compute
units. This is needed to work around assymentric CU harvesting bug on
gfx90a. Add a new device property to get the max available CUs on the
device.
Change-Id: I878f38f14f16c1af01fc0a77157aea1e816a63b8
Add a state indicator to retain ExternalSignals when needed.
Co-operative group launch uses external signals to indicate a dependency
to the next command.
Change-Id: I6d0daa006e2377c3bbf4aeca0fd5b63c7ac8fbbb
Crash was due to the fact that external signal structure was stale even
after destroyign the command. That is because we skipped wait due to a
missing check.
Detect external signals and dispatch a barrier in ReleaseGpuMemoryFence.
Also clear external_signals_ at ProfilingBegin.
Change-Id: I991387edcfe928b511bf5e780988ee131321ed5a
Add a threshold for ROCR/SDMA P2P transfers. ROCR copy path
requires extra barriers in compute for synchronization. That costs
extra performance with tiny transfers.
Reduce active wait time to 10us. Tensorflow uses extra thread
per GPU with constant hipEventQuery() calls. Longer active waits
in ROCr affect CPU performance.
Change-Id: I9020358438615fa2d4617f862f00a562f0a588e7
Stall in the host thread could occur earlier than the app expects.
Make sure rutnime can grow the signals to the queue size without
any stall. Also adding a new signal to the end of the pool could
break the dependency chain on signal reuse. The new logic will
insert the new signal after current to keep the chain intact.
Change-Id: I9c90b98515907db8b677528263c3e88cd9581a14
Set affinity to the closest node of the current GPU. This reduces
the latency to fetch kernel args since device would query the CPU cache
of core which did the dispatch. This behavior is controlled with
AMD_CPU_AFFINITY env var(disabled by default)
Change-Id: I65afba62cb818ea25a311b88d1c0dd5c51330292
Just signal check will still submit the marker and then later
runtime will have a timeout, but the barrier packet is still
generated. Hence early timeout will allow to skip the marker.
Change-Id: Ieb7d89becbcff43a4f4c46715354ca65ab4a80b9
The optimization is controlled with ROCR_SKIP_KERNEL_ARG_COPY.
This is initial check-in for experiments. Extra changes are
necessary for full support:
- handle graph capture with the original sysmem alloc
- avoid memobject references, otherwise there is a race condition with
reusage of the arg buffer
- Remove arg setup from hip
Change-Id: Ib0af710f93e79834711fa4049a7c66093711e68b
The kernel arg pool will be divided into 8 chunks to avoid long stalls,
when the pool will be reused.
Change-Id: I228e6ca1c09e428c1775f1e5b685220a9a5d71af
If we don't create the __amd_rocclr_gwsInit kernel, we still want
to create the rest of the image related blit kernels.
Change-Id: I8bc4645f9f9116eeecbb8b22e981ac4d520f3121
The queue can be destroyed at the time the app will request
the event status. Hence just get the active state from the device.
Change-Id: I887ecb0cfe414c2119247228b0d1255b8308da1e
When unsetting runtime should use HSA_AMD_SVM_ATTRIB_AGENT_ACCESSIBLE
for the agent and not HSA_AMD_SVM_ATTRIB_AGENT_ACCESSIBLE_IN_PLACE
Change-Id: I3814802d1fb3b72c54e7566defafafed6b0d5cee
The original logic left only one slot for HW processing in the queue.
For some reason there is a race condition on CPU overwrite of the slot
before the current active. The workaround is to avoid the previous to
the current active slot for possible unfinished HW processing.
Change-Id: I565495a8feeaedffc9fc8a505edbee5ff5816975
Saleel Kudchadker
German Andryeyev
Satyanvesh Dittakavi
Sarbojit Sarkar
Julia Jiang
kjayapra-amd
pghafari
Bing Ma
Jason Tang
Tao Sang
Sourabh