Checks for an IPC memory error and updates comments relevant
to rocr_visible_devices.
Change-Id: I9d2f2dd27f3fa04881d17387cce2692bc046edb2
[ROCm/ROCR-Runtime commit: a1c2439213]
HDP will now be used for coarse grain kernarg so needs to be
reported without consideration of fine grain vram over pcie.
Change-Id: I648167299faa583876a3d8685c3b3c4d8d31ebf9
[ROCm/ROCR-Runtime commit: 9c35780836]
Setting to 1 prevents the scratch handler from reducing peak occupancy.
Scratch allocations that would normally reduce peak occupancy will
instead fail.
Diagnostic for TF and PyTorch.
Change-Id: I2d7ea47077eb5cf708251c8aa3fd183ad4261be0
[ROCm/ROCR-Runtime commit: dc165c92bc]
scratch_used_large_ was uninitialized leading to the observed hang.
DynamicScratchHandler would wait for a large scratch release despite no
large scratch having yet been allocated. Fixes .
The patch also removes a potential race between AddScratchNotifier and
ReleaseQueueScratch. The race condition does not exist today since both
scratch alloc and release run on the same thread. The changes will
prevent this potential race from manifesting if the async event handler
is ever updated to use multiple threads.
Also enhances scratch occupancy reduction reporting. Reporting now
prints the initial request size as well as the allocated size and the
effect on occupancy this has. Occupancy is computed in terms of the
requesting dispatch grid size so may be >100%.
Change-Id: I0fc5ee01467ff4c29bdd25d545177c97862c3bd9
[ROCm/ROCR-Runtime commit: 6c556002d8]
Ensures that all CPU agents will have a pool handle to allocate
system memory. These pools will have no numa binding since the
node their owning Agent represents has no installed memory.
Change-Id: I9f72b455d633646839753c6719ff7f6a4c41f7c4
[ROCm/ROCR-Runtime commit: d53fe07687]
Lack of cache controls only allow operating SDMA at
agent scope. All copy APIs are defined at system scope so may
result in data errors.
Change-Id: I9cd10007defddcbf8feb14a2e3daa1ba17c0489f
[ROCm/ROCR-Runtime commit: 22a601292d]
Queues should transition to ref counting for all queues eventually.
That cleanup will be part of shared queue pooling support.
Change-Id: I217ff5d573156678b9559da6fb81baa8cd31c617
[ROCm/ROCR-Runtime commit: 0a43a107b1]
Temporary workaround for 2.10 release. RCCL, compiler, or firmware
must be corrected and this code reverted before another ASIC release.
Change-Id: I27851353289b93df9acb72d28b8c6ccb9f7f7d7a
[ROCm/ROCR-Runtime commit: 35c1ffa863]
Debugger path is taken for (trap_id >= 3) and single step exceptions.
Other traps/exceptions behave as before.
Change-Id: I276c0eb69953709968353a57717ee017d22348a2
[ROCm/ROCR-Runtime commit: 78e754935c]
- Use new buffer resource descriptor layout
- Handle wave32 scratch allocation error from CP
- Make wavefront size a property of scratch allocation requests
- Repurpose wave64-specific amd_queue_t.scratch_workitem_byte_size field
- Clear index_stride field in V# on gfx10, calculated per-dispatch by CP
Change-Id: If2acdf6430772abd4d6a8c792fc8c11260764dda
[ROCm/ROCR-Runtime commit: f8d0ccd159]
doorbell_queue_map should always be allocated or we will need to
add branches around all accesses.
Change-Id: I994c0eaf4be62c1a4a37bd06894272dba1fc1da6
[ROCm/ROCR-Runtime commit: f9d3796db8]
sdma end ts must be 256 bit aligned in oss 3.0 and prior. Using
the ts pool requires copying into the signal and is a significant
performance penalty for small copies.
SharedSignal is 128 bytes due to alignment so can host the end ts.
Move sdma end ts into SharedSignal and remove ts pool and ts copy.
Change-Id: I7899bda36ebc9adcaad1d3a3d2b7a489857cc9e8
[ROCm/ROCR-Runtime commit: ec5ac95dce]
Impacts GPU_ONLY signal type latency when waiting for small operations.
Using this type improves total SDMA small copy performance by ~40% if
the signal is allowed to spin freely.
Change-Id: I27aa128c63a1bacb3f51fb08f166e4e1d6fef651
[ROCm/ROCR-Runtime commit: 5adb73fffd]
Remove agent lookup in time stamp translation for IPC signals. The copy
agent handle is not shared so does not need to be checked for cross
process use. Cross process copy-timestamp read is illegal and continues
to deliver garbage.
Store the copy agent properly when doing CPU-CPU copies.
Change-Id: Ib4008f66ff866922047749dd556c84a32021c1fd
[ROCm/ROCR-Runtime commit: ea8c99f452]
ucode versions are per asic so not valid for feature enablement outside
of bringup/dev. Feature is older than the latest ioctl change that
the thunk depends on so use of this patch with kernel packages that
don't contain the feature is not possible in a supported environment.
Change-Id: I36b14176a7d642017ef1518aeade454b0f3dc749
[ROCm/ROCR-Runtime commit: 8133563a93]
Also removed an unnecessary cache flush in dependency barrier packet.
Change-Id: I573df3bdf0a10df0bcd78025672c44038f8091ff
[ROCm/ROCR-Runtime commit: 4647a5454d]
This is to allow allocations in system memory that exceed sizes
reported by a CPU device
Change-Id: I3d10d192aafcefbe4107f69b7c5e30bf7f836619
[ROCm/ROCR-Runtime commit: 3201f68f72]
If M0[23] is set then the driver will interpret the interrupt as a
debug event, rather than a signal event.
Clear M0 before sending the interrupt. All paths here are terminal so
it's not necessary to save/restore M0.
Change-Id: Ibd85b8cc6f8556941f2308a2c3fa3c68702cd606
[ROCm/ROCR-Runtime commit: ad717d2e98]
agentOwner from thunk reflects the GPU which holds the device alias.
We need to return a CPU to better reflect that the memory is system memory.
Change-Id: I9233f8779a4bfd471f68dbbbce07ae4528412e18
[ROCm/ROCR-Runtime commit: 6e07bc8dc4]
Report traps and fatal exceptions through a wavefront's
amd_queue_t.queue_inactive_signal. Previously, only traps were
reported and requireed the compiler to pass in the signal pointer
in s[0:1].
The signal is obtained through a mapping from doorbell index to
amd_queue_t*. The doorbell is fetched within a wavefront through
the gfx9+ S_SENDMSG(MSG_GET_DOORBELL) instruction.
Change-Id: I319b45f2e15dfcfe4db8f4065da1136e9539a42b
[ROCm/ROCR-Runtime commit: ff8f439112]
Assembler toolchains are moving from SP3 to LLVM. Replace trap handler
source code with LLVM equivalent.
Fix a trap issue with SQ_WAVE_IB_STS restore. Mostly harmless as all
traps are currently considered fatal to the wavefront.
Change-Id: Iacecd9dd31a1d96a083c8b8327f442f33c861f9f
[ROCm/ROCR-Runtime commit: 6ed686ee29]
Adds hsa_amd_register_deallocation_callback and hsa_amd_deregister_deallocation_callback
to notify when HSA memory has been released.
Change-Id: I1f33cee250ca890e5c2e7fddfa4479aa5874651d
[ROCm/ROCR-Runtime commit: 299874f17d]
CPUClockCounter is not NTP adjusted (CLOCK_MONOTONIC_RAW) so should be
better for measurements. However, it is implemented with syscall while
CLOCK_MONOTONIC is implemented via vDSO. The latency increase becomes
significant when language layers make corresponding clock measurements.
Reverting to CLOCK_MONOTONIC will reduce latency and allow small
duration events to be measured at the cost of incorporating NTP
frequency skew errors. NTP may adjust frequency by 500ppm so limits us
to ~3 decimals in elapsed time.
Change-Id: I920b9f707f47109d80d6c256c475638c03fb8d76
[ROCm/ROCR-Runtime commit: 4b22d24346]
API is a stateless lookup of RO data and needed to interpret
hsa_init error codes.
Change-Id: If80cba2f697843d08e529da0f790acf3c37127a7
[ROCm/ROCR-Runtime commit: 22de0e7fb9]
Joined threads can not be joined more than once nor can they be detached.
Thread library wait and close allows multiple waits and separate close so
this fixes the pthread implementation.
Change-Id: I0019271a438f11ed4c6c11854011f5c4f6e16b65
[ROCm/ROCR-Runtime commit: a913549190]
Ramesh Errabolu
Sean Keely
Laurent Morichetti
Qingchuan Shi
Jay Cornwall
Chris Freehill
Christian Sigg