Remove fence pool and use two signals. Two signals allows overlapped
submission and copy while reducing thread busy polling.
Change-Id: Idb5f8e4c7f482a596ffce9e7799191fdd785a216
[ROCm/ROCR-Runtime commit: 56ed5c8904]
Fix pitch overflow due to small element detection.
Add wide pitch 2D copy handling.
Cleanup code duplication.
Change-Id: I93b1584aba8e5964957eb7ab3544df806ca3e2f9
[ROCm/ROCR-Runtime commit: e0839ab27e]
llvm.debugtrap and other trap IDs are reserved and should not place
the queue into an error state.
Change-Id: I98193a35ac7da94c4a42ee75d87754ee552ebea0
[ROCm/ROCR-Runtime commit: 536823482b]
Ensure system release fence is set on GFX8 large scratch using packets.
Change-Id: I13cfdcd35969482ea6e95e0b352f5cb3a0454b86
[ROCm/ROCR-Runtime commit: 5f25619bb7]
Use async. signal handler to satisfy dependencies for SDMA blits.
Change-Id: Ifa8d3ee6810509f400a568ca2387ac6ab3ab7c36
[ROCm/ROCR-Runtime commit: 7cd6e366ed]
1/ Revised debug event handler to handle different events.
2/ Added queue error handler using the callback in queue create, which will print out wave info when queue in error state.
3/ Preempt queue instead of destory queue when queue error state.
Change-Id: Ib727d208de9caf1c72c76d42268483b24aaebde8
[ROCm/ROCR-Runtime commit: 49d2175c74]
Also improve small_heap used for scratch region allocation.
Change-Id: Ib7311b663b38968d88ebc355b81e12c0863dc541
[ROCm/ROCR-Runtime commit: 7caf9633f6]
Spec requires GPU release fences and CPU acquire fences at queue destroy.
Also update the recognized status codes.
Change-Id: If9166f5149f65417c7057ff7c0f69f6ac094d6ab
[ROCm/ROCR-Runtime commit: b6f0248f53]
Remove unused function (FenceRelease), add comments to barrier packet settings,
correct profiling controls to work with queue wrappers.
Change-Id: I45bb26227bcc2b78edb8ad5dc497603c33234e18
[ROCm/ROCR-Runtime commit: cd46954cc4]
This includes the changes provided by Konstantin, "Add xnack from elf header" (Change 136389).
Change-Id: I95e51141caa0d7c21903b09212c02e4906ec54a3
[ROCm/ROCR-Runtime commit: 8e3d26c617]
Create an interface for doorbell signals to reduce code duplication.
No functional changes.
Change-Id: I101a8997dd582ff99e1537758c804b21fe3bb6af
[ROCm/ROCR-Runtime commit: d2e70bb999]
Prefer using memfd_create() for the ring buffer.
We were using /dev/shm, but this won't work on systems that
either don't have /dev/shm or have mounted it with noexec, because
for everything other than gfx700 we map the ring buffer with PROT_EXEC.
memfd_create() is Linux specific and was added in Linux 3.17, so we
will fallback to using /dev/shm on systems where memfd_create() is
not available.
Change-Id: I58fb533eebc362f6d29dc3e316a80801014d50e8
[ROCm/ROCR-Runtime commit: b93ffafdc7]
Remove "zombie" queue state and report queue creation failure via
exceptions. Make Shared object a final container and support array
objects with Shared. Add message printing to hsa_exception in
debug builds.
Change-Id: I459f38c80846018acbf45538874e95f91dd6b195
[ROCm/ROCR-Runtime commit: f312a7386e]
Queue intercept is exposed as two tools-only APIs via the API
intercept table.
Change-Id: Iac9602ed3143974d85c3569e9092295ad18037f8
[ROCm/ROCR-Runtime commit: 0c7dde2d1f]
1. Add hsa ext api hsa_amd_register_vmfault_handler for debugger to register callback in case of VM fault.
2. Extend hsa_ven_amd_loader API to:
(1) iterate loaded code objects in executable:
hsa_ven_amd_loader_executable_iterate_loaded_code_objects
(2) get loaded code object info:
hsa_ven_amd_loader_loaded_code_object_get_info
3. Make the id of hsa_queue the same as the one used in communication with thunk (for amd_aql_queue)
Change-Id: I68910809e59e24297350d262606f00e96c14bcbd
[ROCm/ROCR-Runtime commit: ce6aee01ed]
Since access may only be manipulated on whole pages, suballocator fragments must cooperate to set the page's access.
Since the KFD does not migrate memory on access changes this implementation makes agent access sticky across the requests in a fragmented page.
Change-Id: I88479ed45fb40e9782b704526a7b8ffb22e7bd76
[ROCm/ROCR-Runtime commit: e9a6f2c3e6]
Track pointer info for sub 2MB fragment allocations in allocation_map_.
Add fragment support to IPC.
Change-Id: I00cfc2e2fa289aac90a4718c392f9bb056a61a87
[ROCm/ROCR-Runtime commit: 117be0b55a]
Added an API for creating signals with attributes.
Added two APIs for IPC operations on signals.
Initial use of exceptions for error handling.
Add ref counting to signals.
Removed spin loops from signal destructors.
Signals are no longer to be destroyed with delete, use DeleteSignal instead.
Added delete safety to doorbells.
Added secondary hsa_signal_t -> Signal* translation path for IPC enabled signals.
Change-Id: Id59065d002f0c2566b0a9425694da2ed27cb7d7f
[ROCm/ROCR-Runtime commit: c9642cf7af]
Also separate signal ABI block allocations from the runtime interface object.
Change-Id: If16763338db664f29163a1348f8f4c38cf0597b2
[ROCm/ROCR-Runtime commit: 2732b18092]
1. Correct amd::AqlQueue::ExecutePM4 to support interception.
2. Minor fixes to AqlPacket and SoftCP.
3. Minimal change to disable interception of runtime internal queues.
Change-Id: I103fece2ebf9a188d27f01e61221c737405d7253
[ROCm/ROCR-Runtime commit: bc0bd00746]
When a fatal memory fault occurs the scheduler context-saves all queues
in the process and notifies the runtime through the memory event. The
saved state contains all GPR/LDS data at the moment of the fault.
Retrieve this state and present it to the user if HSA_DEBUG_FAULT is set
to "analyze" and the wavefront caused the fault. If amdgcn-capable objdump
is in the PATH invoke this to disassemble code around the PC.
Queue lifetime is now managed by the runtime to allow querying the
context save state for all active queues.
Change-Id: I6fee662fad1c4f9aa125bf5c53d7d0ea1ab32f95
[ROCm/ROCR-Runtime commit: 75c9506f9d]
The trap protocol changed between gfx8 and gfx9. The return address
is in trap temporaries [0,1] on gfx9 rather than [4,5] on gfx8.
Unfortunately SP3 changes the meaning of the ttmp register aliases
in gfx9, further confusing the issue.
Clean up later when LLVM assembly build is introduced to the runtime.
Change-Id: I84ea9bf3736f060dd95d0361f9d5a0f9a3576178
[ROCm/ROCR-Runtime commit: f0a1c7c4c6]
When a new enough microcode build is running use a vendor AQL packet
to submit the PM4 IB.
Change-Id: Icd3e2b322c418477420ba4a29f4455ce340ef0d2
[ROCm/ROCR-Runtime commit: 4d62b9482a]
Gfx9 requires monotonic write pointer and doorbell.
Cound fields are 1-based compared with 0-based pre-Gfx9.
- Restructure implementation to use monotonic ring indices
- Remove redundant submission size checks (handled by AcquireWriteAddress)
- Unify copy/fill per-command limit (documentation is unclear)
Change-Id: I57c1675221d2e63aa319fee700d9951671e1bd65
[ROCm/ROCR-Runtime commit: 1cd46afe6d]
Currently HSA HW Profiler is failing to build due to this patch
This reverts commit 91866d4bbb.
Change-Id: Iabb2b958f33ba614a24b61bb370905b3b7362708
[ROCm/ROCR-Runtime commit: 5162a76616]
Initial work to import the latest (1.1) hsa_ext_image extension.
Change-Id: I4d55adb09ba4d4dbd43d47a4bc54077d4bc531d2
[ROCm/ROCR-Runtime commit: e0ce8855dd]
Also emit error messages to stderr if no async queue error callback was registered and queue fault messages are enabled (on by default).
Queue fault messages are controlled with env key HSA_ENABLE_QUEUE_FAULT_MESSAGE.
Change-Id: I496487b8d048b83aa95b9784e92928211f167b17
[ROCm/ROCR-Runtime commit: 0e17cc2887]
Sean Keely
Jay Cornwall
Qingchuan Shi
Konstantin Zhuravlyov
Ramesh Errabolu
Wilkin
Evgeny
Kenny Ho
Christophe Paquot
Konstantin Zhuravlyov
Kent Russell
James Edwards