Add a state indicator to retain ExternalSignals when needed.
Co-operative group launch uses external signals to indicate a dependency
to the next command.
Change-Id: I6d0daa006e2377c3bbf4aeca0fd5b63c7ac8fbbb
[ROCm/clr commit: 1fbd75b825]
Crash was due to the fact that external signal structure was stale even
after destroyign the command. That is because we skipped wait due to a
missing check.
Detect external signals and dispatch a barrier in ReleaseGpuMemoryFence.
Also clear external_signals_ at ProfilingBegin.
Change-Id: I991387edcfe928b511bf5e780988ee131321ed5a
[ROCm/clr commit: 3239222516]
Add a threshold for ROCR/SDMA P2P transfers. ROCR copy path
requires extra barriers in compute for synchronization. That costs
extra performance with tiny transfers.
Reduce active wait time to 10us. Tensorflow uses extra thread
per GPU with constant hipEventQuery() calls. Longer active waits
in ROCr affect CPU performance.
Change-Id: I9020358438615fa2d4617f862f00a562f0a588e7
[ROCm/clr commit: 008133cf41]
This allows experimenting with env var GPU_PINNED_XFER_SIZE which is
still at a default of 32MB
Change-Id: I85ade700ed58d498eba29d1737601dc74d4c26a4
[ROCm/clr commit: 3f82b99f5d]
Stall in the host thread could occur earlier than the app expects.
Make sure rutnime can grow the signals to the queue size without
any stall. Also adding a new signal to the end of the pool could
break the dependency chain on signal reuse. The new logic will
insert the new signal after current to keep the chain intact.
Change-Id: I9c90b98515907db8b677528263c3e88cd9581a14
[ROCm/clr commit: 102c19adf3]
Reference for the first element can trigger an assert with
_GLIBCXX_ASSERTIONS build
Change-Id: I59c63c052831307edfe5dcc6384798a43e9596dd
[ROCm/clr commit: 6f2e7c3199]
Set affinity to the closest node of the current GPU. This reduces
the latency to fetch kernel args since device would query the CPU cache
of core which did the dispatch. This behavior is controlled with
AMD_CPU_AFFINITY env var(disabled by default)
Change-Id: I65afba62cb818ea25a311b88d1c0dd5c51330292
[ROCm/clr commit: b192beea52]
Just signal check will still submit the marker and then later
runtime will have a timeout, but the barrier packet is still
generated. Hence early timeout will allow to skip the marker.
Change-Id: Ieb7d89becbcff43a4f4c46715354ca65ab4a80b9
[ROCm/clr commit: bbb635bc32]
The optimization is controlled with ROCR_SKIP_KERNEL_ARG_COPY.
This is initial check-in for experiments. Extra changes are
necessary for full support:
- handle graph capture with the original sysmem alloc
- avoid memobject references, otherwise there is a race condition with
reusage of the arg buffer
- Remove arg setup from hip
Change-Id: Ib0af710f93e79834711fa4049a7c66093711e68b
[ROCm/clr commit: 7e12cf6318]
The kernel arg pool will be divided into 8 chunks to avoid long stalls,
when the pool will be reused.
Change-Id: I228e6ca1c09e428c1775f1e5b685220a9a5d71af
[ROCm/clr commit: f78b3a8919]
If we don't create the __amd_rocclr_gwsInit kernel, we still want
to create the rest of the image related blit kernels.
Change-Id: I8bc4645f9f9116eeecbb8b22e981ac4d520f3121
[ROCm/clr commit: 55a0cf0b0c]
ROC_AQL_QUEUE_SIZE will control the size of AQL queue.
The current sefault value is 4096.
Change-Id: Icd2a4ee3ba554c06aa05b08defd922d2c63e43fd
[ROCm/clr commit: 7fe696b6ef]
The original logic was left after initial testing when HMM
couldn't handle xnack properly
Change-Id: I0abf01805704171e931dfba8b6d95bfe87d5fab1
[ROCm/clr commit: d17108e8d0]
Implementation to use a blit kernel to perform
a hipStreamWait/write instead of an AQL packet.
Change-Id: I462671ed5cec37144dfe97ff66439249196117c1
[ROCm/clr commit: cbb8d82bdb]
info_.extensions_ and settings_ are deleted at amd::Device()::~Device().
Change-Id: I06f240a42e5c131dbd4e61a759f905bcdf84b45a
[ROCm/clr commit: f212fc91ca]
The queue can be destroyed at the time the app will request
the event status. Hence just get the active state from the device.
Change-Id: I887ecb0cfe414c2119247228b0d1255b8308da1e
[ROCm/clr commit: f116959b54]
When unsetting runtime should use HSA_AMD_SVM_ATTRIB_AGENT_ACCESSIBLE
for the agent and not HSA_AMD_SVM_ATTRIB_AGENT_ACCESSIBLE_IN_PLACE
Change-Id: I3814802d1fb3b72c54e7566defafafed6b0d5cee
[ROCm/clr commit: d8a86e4870]
The original logic left only one slot for HW processing in the queue.
For some reason there is a race condition on CPU overwrite of the slot
before the current active. The workaround is to avoid the previous to
the current active slot for possible unfinished HW processing.
Change-Id: I565495a8feeaedffc9fc8a505edbee5ff5816975
[ROCm/clr commit: 65ddfcc6a8]
For the fillBuffer shader, if there are two 32bit writes to a MMIO
register, it can get dropped. It has to be a single 64bit write.
Add optimization to fillBuffer to write 64bit and 16bit writes.
Change-Id: I3aa78e027898f8ae01e9c8f09004615673720c2b
[ROCm/clr commit: 21ba34d0fe]
Add a env var ROC_USE_FGS_KERNARG to toggle kernel arg placement
By default its in Fine Grain Kernel arg segment for supported asics.
Change-Id: I3d57ed69a1a4db2b392b0438ead499f3ddca4716
[ROCm/clr commit: e29b9c00ee]
Pass the device agent specified by the user to the ROCr api instead of passing the device agent attached to the specified stream
Change-Id: I86c98935b9dc404eaa6d47ccdd082a8c3678fb36
[ROCm/clr commit: 169cc857fd]
Fixes Seg fault caused when the attribute hipMemRangeAttributeAccessedBy
is queried using hipMemRangeGetAttribute
Change-Id: I2ceb2267d89bfc31a55d9eae2685610c7ad89b1f
[ROCm/clr commit: 48c1b895c0]
Satyanvesh Dittakavi
Saleel Kudchadker
Sarbojit Sarkar
German Andryeyev
Julia Jiang
kjayapra-amd
pghafari
Bing Ma
Jason Tang
Tao Sang
Sourabh