When deferring a dmabuf export on an import call, there may be a
failure to export as the GEM object is not referenced by the kernel
mode driver. To get around this, do a non-deferred export and
immediately close the dmabuf FD to keep FD creation to a minimum.
This way, the GEM object will have a kernel mode driver reference
when a deferred export is done.
Also a bad dmabuf FD sent over a socket may not be received by an import
reader and this can cause a hang.
Set a 10 second timer so that importer is not blocking indefinitely.
Change-Id: I11a9b5ec64aa2e16fd6aecdf46c34e4eb56ccfd0
[ROCm/ROCR-Runtime commit: eb2100daad]
Extracts and creates a core dump ELF file from a fault event, using
core dump front end. GFX11 is not supported.
Signed-off-by: Alex Sierra <Alex.Sierra@amd.com>
Change-Id: I5ae154e886f39ab3ce7bbae5803efb27a96c7e2e
[ROCm/ROCR-Runtime commit: cbeddf9eb6]
When inspecting waves on architectures where SPI may not initialize TTMP
registers, the debugger cannot reliably know if the trap handler was
entered and if it saved valuable information in TTMP registers.
This patch uses the status.skip_export bit (unused by the compute
shaders) to indicate that it got executed before halting a wave.
This is done except for gfx940, where ttmp11[31] can be used (as long as
TTMP registers are always initialized by SPI for this architecture). It
could be possible to be more selective as architectures always
initializing TTMP registers do not require this step, but always doing
is makes maintenance simpler.
Change-Id: I5c4148c78062f7ffa049ac7856c2edc82dbc77d1
[ROCm/ROCR-Runtime commit: 5d3f6a63f1]
Work around SDMA hang in non-SPX modes for non-APU devices by disabling
ganging.
Root cause of hang not found.
non-APU xGMI modes have only 1 link between socket devices anyways so
there's likely no real system level gain in ganging intra-socket.
Change-Id: Ia4eda2f85cbf25151d3dbcf50cc45b8b775c60e2
[ROCm/ROCR-Runtime commit: ed462035fa]
Gang items have to wait on dependency signals as well as the leader.
Copies should not start if shaders are still operating on memory
to be copied.
Change-Id: I99703b420045ebcba2c9da39ec64678129dc140f
[ROCm/ROCR-Runtime commit: ed260ea970]
This allows the VA to be recorded in ROCr so that they are not
treated as an invalid pointer in future API calls.
Change-Id: I8d1d8ef9816a984c89d30a2179b0ce8940fef1da
[ROCm/ROCR-Runtime commit: f2006d6899]
- add rocprofiler-register to CPACK_DEBIAN_BINARY_PACKAGE_DEPENDS when found
- add rocprofiler-register to CPACK_RPM_BINARY_PACKAGE_REQUIRES when found
- remove report_tool_load_failures_explicit_
- add HSA_TOOLS_DISABLE_REGISTER flag
- add HSA_TOOLS_REPORT_REGISTER_FAILURE
- use HSA_TOOLS_REPORT_REGISTER_FAILURE instead of HSA_TOOLS_REPORT_LOAD_FAILURE
- changed rocprofiler-register message to not include the word "error"
Change-Id: Ib7fd7f14c42758a54c347874018281bb1b5477a6
[ROCm/ROCR-Runtime commit: 7ce263b0e4]
At hsa_shutdown(), scratch_lock_ may be gone. Blit queues don't need it.
Change-Id: Ic132ac8a6be31fb2f0623137115608b0b222f077
[ROCm/ROCR-Runtime commit: 24633c7a85]
If two attach requests to the same piece of shared memory occur,
a double export or premature dmabuf fd close can occur since the export
and close on demand calls are not atomic.
Use a reference counter on shared memory dmabuf FDs that have
already been opened to avoid this problem.
Change-Id: I14a59209c0385e32582af42a57b33b1c6838a9b1
[ROCm/ROCR-Runtime commit: 1f63ea3476]
Instead of caching shared memory fds for export on the exporter side,
only export the FD in the async handler when requested.
The importer should request export fd closure once import is done.
Change-Id: I469e0cd1749beeb9c506c8a6461745fb039d9c3b
[ROCm/ROCR-Runtime commit: e911335cee]
ToolsApiTable's version was incorrectly default initialized to 0.
Fixes error in commit fc889669
Change-Id: I41e9301a9c33b119ee50f6164d21ddf11dc188c4
[ROCm/ROCR-Runtime commit: 8e312471dc]
Adjust code to allow the use of non-contiguous chunks of memory to be
mapped within a single VA range.
Change-Id: Ida21ba202927229347b3a32d9b7106df10819cf5
[ROCm/ROCR-Runtime commit: f7de85082e]
Add new tools table and functions to notify in case of an event
Change-Id: I47f0c2f3c8e02d7bcb74d649903eb4f86721c154
[ROCm/ROCR-Runtime commit: a67af3807f]
Fix gang factor overwrite of 0 if there are no xGMI SDMAs
on the device and gang factor is 1.
Change-Id: I041d4b4ae87fb68f224ee4dedb758c6f06c022a9
[ROCm/ROCR-Runtime commit: 1dd4a7dc18]
Users can import device memory without specifying the target node.
DMA buf imports return a Thunk handle that's not useful for
gpu mapping calls.
Fix this by using the import node information to re-import and
map with the correct target GPU.
Also fix IPC detach calls by deregistering the Thunk handle
import immediately during attach instead of failing to do it later
on detach since Thunk handles aren't placed into ROCr allocation
map.
Finally refactor the IPC attach function for cleaner logic flow.
Change-Id: Ib2bf178110b2be98bd6917c765f724e4e613f5f2
[ROCm/ROCR-Runtime commit: a3efd13a2f]
We should also close the client side dmabuf fd after importing for target
nodes.
Change-Id: I74f61dd65bebb03dc002f5df7301efd1ef8d9603
[ROCm/ROCR-Runtime commit: 15691ae460]
Optimizations include:
- Greedy gang by placing gang leaders on first D2D sdma blit context
to avoid dead locking with other gang leaders and items. Note that
this is fine since we can't avoid an oversubscription problem when
there is only 1 xGMI link anyways, so treat all xGMI links as a single
pipe for ganging.
- Non-leader gang items don't have to poll on dependency signals so this
opens up more non-blocking SDMA channels.
- unlock gang lock when gangs are not needed.
- Change gang factor lookup from vector pair to map and register all
gpus in gang factor lookup regardless of link type so that we can take
advantage of the O(logN) direct key/value lookup time.
Fixes include:
- HSA_PAGE_SIZE_4KB was an incorrect macro to use for gang size limit.
As a result, small copies ended up ganging and hitting latency limit.
Use hardcoded 4096 bytes instead.
- Cap auxillary gang factor to the number of non-XGMI SDMA engines.
Change-Id: Ic23fde131502906a807134a04599aa6d012e8cbb
[ROCm/ROCR-Runtime commit: 62f3f250ce]
Use emplace to prevent copying the MappedHandle objects when inserting
entries into mapped_handle_map_.
Change-Id: Id3f40f1eb73ce30e62da53c5aea4dd715e83ac59
[ROCm/ROCR-Runtime commit: 32b3a3c299]
When allocating a memory handle, the NoAddress thunk flag should be set
so that this allocation does not have a virtual address range.
Also, skip mapping the memory when allocating a memory handle
Change-Id: I1c168bc00ddbc158d447197c4dc25f96bad02b19
[ROCm/ROCR-Runtime commit: 29efd8eccd]
After a memory handle is created. hsa_amd_vmem_get_access should return
HSA_ACCESS_PERMISSION_NONE insread of reporting the allocation as
invalid.
Change-Id: I1a09d15c220d48497d09c89059493e538f82aeb9
[ROCm/ROCR-Runtime commit: 2f97049da5]
When using multi-GPU for each BO, a new dmabuf_fd needs to be imported
into libdrm.
Change-Id: Iaa2415c8f655a1ce8e92b0878517a11ff014a1d5
[ROCm/ROCR-Runtime commit: 8b85f9e668]
Set HSA_ENABLE_IPC_MODE_LEGACY off (i.e. use DMA bufs implementation
by default).
Change-Id: I7b1c6cb7d19310adf6f0bfe060736f4adbf7adc2
[ROCm/ROCR-Runtime commit: e20f41df62]
As the KFD IPC IOCTLs will not be upstreamed, change runtime
implementation to use DMA bufs.
DMA buf fds will be passed over abstract unix domain sockets.
The exporter spins a thread that creates a socket server.
The importer connects to the server to fetch the fd.
libDRM will be required to do a manual import and GPU map for
memory that is not already imported and mapped.
For now, use the legacy IPC implementation by default as a
follow on patch will disable the HSA_ENABLE_IPC_MODE_LEGACY
environment variable.
Change-Id: Ifd8469e9adfc81f8a1ea78d6010fb10b515ba1b4
[ROCm/ROCR-Runtime commit: 5dfebdbca9]
Implement HybridMutex to improve latencies compared to KernelMutex when
there is contention between several threads calling hsa_signal_create
and hsa_amd_signal_async_handler.
Change-Id: If53377033e749b0050727964c9303f09b02527cc
[ROCm/ROCR-Runtime commit: 8d3fee5095]
Fixes issue where t1_ counters may not be updated when doing dispatch
profiling, causing a divide by 0.
Change-Id: I91060ac3f9fd2183d277e6e7cd810398a453a87f
[ROCm/ROCR-Runtime commit: 3d1563ee68]
KFD had some fixes for handling of virtual memory APIs. These fixes are
included in interface version 1.15.
Change-Id: Ie701eccf6e032f9ec0a1f4e8a43718964eebddc6
[ROCm/ROCR-Runtime commit: d16c6db2ee]
This reverts commit 4c8a849772. This
change is required for the runtime to generate reliable core dump files,
but this feature has been disabled for now by
816b46868a. Until it is needed, revert
the ABI change in the trap handler to maintain compatibility with older
debugger.
Change-Id: I77a1562dc7962befe2bf88442df858e2d2b1c5ab
[ROCm/ROCR-Runtime commit: 6f828d8609]
This reverts commit 9aa39b0979.
This commit disables core dump feature. Apparently, gfx1101 SA1 waves
can not enter the trap handler because they receive an invalid
address. However, core dump at the debugger has been moved to rocm
6.2.
Signed-off-by: Alex Sierra <alex.sierra@amd.com>
Change-Id: I7915caf58118658e5e7f435f91a0a6216d2fdb42
[ROCm/ROCR-Runtime commit: 5e3be9c28a]
On some systems, pthread_addr_setaffinity_np does not exist, so we need
to use pthread_setaffinity_np on thread after pthread_create
Provided by Julian Samaroo on github
https: //github.com/RadeonOpenCompute/ROCR-Runtime/pull/143
Change-Id: I4649f94333f2d7b0a5993b370a4bfc48d92acecb
[ROCm/ROCR-Runtime commit: 6333fdecf3]
Add query to return flags for GPU agent memory properties and AQL
extensions.
Implement flag to determine that GPU agent is an APU
Change-Id: Ic04c51290b2b9763e14989c117f35a2e22297453
[ROCm/ROCR-Runtime commit: c86837d8d6]
When inspecting waves on architectures where SPI may not initialize TTMP
registers, the debugger cannot reliably know if the trap handler was
entered and if it saved valuable information in TTMP registers.
This patch uses the status.skip_export bit (unused by the compute
shaders) to indicate that it got executed before halting a wave.
This is done except for gfx940, where ttmp11[31] can be used (as long as
TTMP registers are always initialized by SPI for this architecture). It
could be possible to be more selective as architectures always
initializing TTMP registers do not require this step, but always doing
is makes maintenance simpler.
Change-Id: I314db6b37772f7daa8bd405e6662a86658d3f5e0
[ROCm/ROCR-Runtime commit: c5db063b2f]
Extracts and creates a core dump ELF file from a fault event, using
core dump front end.
Signed-off-by: Alex Sierra <Alex.Sierra@amd.com>
Change-Id: Ibbbe41b3d13dd3fcb90161e927d48c329cf513a9
[ROCm/ROCR-Runtime commit: 803e37ded5]
Member added to KFDVersion to report if KFD supports core dump
mechanism. This is done through hsaKmtRuntimeEnable API call while
the topology is being built. It also dictates if core dump will be
generated by either KFD or hsa-runtime.
Signed-off-by: Alex Sierra <alex.sierra@amd.com>
Change-Id: I2e9d4166563402f78613d728446feb692c52d9d1
[ROCm/ROCR-Runtime commit: 54604654bd]
This API consists in one function to be called from a fault event at the
hsa-runtime to generate a core dump.
Signed-off-by: Alex Sierra <alex.sierra@amd.com>
Change-Id: Ib1b90d5beb13f93c4e8ebd21fd61705ebb12ca5d
[ROCm/ROCR-Runtime commit: 514b222368]
- fix logic for using HSA_TOOLS_LIB when rocprofiler-register support is enabled
- report tool load failure for rocprofiler-register
Change-Id: Ife23aa3e6ed19174376cd694764583b73f8976cd
[ROCm/ROCR-Runtime commit: 27eb0516bb]
The alternate scratch memory is used for dispatches that have a low
number of waves but relatively large wave size.
This allows us to keep the tmpring_size.bits.WAVES field of the main
scratch to full occupancy.
Change-Id: I32d240fac4b7d38200d1eebc1b0fdc8a823920d3
[ROCm/ROCR-Runtime commit: a7a3358067]
For devices where the CP FW supports asynchronous scratch reclaim, ROCr
is able to claw-back scratch memory that was assigned to an AQL queue.
With that ability, ROCr does not have to rely on using USO
(use-scratch-once) when assigning large amounts of memory to a queue.
If we reach a situation where we are running low on device memory, ROCr
will attempt to claw-back the scratch memory.
Change-Id: Iddf8ec84e37ab8b9fdc58bafbe2b61fe2acb6eb7
[ROCm/ROCR-Runtime commit: dca8f3a21d]
Separate the event handler and scratch handler portions of the code into
separate functions.
Change-Id: Ifdb7461e816b0f2d3c1c0a74d6f020b4d6fc736c
[ROCm/ROCR-Runtime commit: 64070a9acc]
For gfx11, the trap_handler fails to recognize a trap id 3 and report
the exception to the debugger if the debugger is attached.
This is because the 2nd level trap handler looks for the DEBUG_ENABLED
bit in ttmp13 instead of ttmp11. This bit is set by the 1st level trap
handler and is part of the 1st/2nd level trap handler ABI.
Change-Id: Ib36361f53d9bcbbed52320d8c3a9ab2c0b28c7cd
[ROCm/ROCR-Runtime commit: 6916ce358a]
This reverts commit a8e34eaec8.
gfx1150/1151 is merged into mainline now.
Change-Id: Id179949318a37888c74abb5a8610d95bc2f22906
[ROCm/ROCR-Runtime commit: 991bbdcf24]
David Yat Sin
pvanhout
Jonathan Kim
Alex Sierra
Lancelot SIX
Shweta Khatri
Jonathan R. Madsen
Mythreya
Ruili Ji
Lang Yu