We should also close the client side dmabuf fd after importing for target
nodes.
Change-Id: I74f61dd65bebb03dc002f5df7301efd1ef8d9603
[ROCm/ROCR-Runtime commit: 15691ae460]
Optimizations include:
- Greedy gang by placing gang leaders on first D2D sdma blit context
to avoid dead locking with other gang leaders and items. Note that
this is fine since we can't avoid an oversubscription problem when
there is only 1 xGMI link anyways, so treat all xGMI links as a single
pipe for ganging.
- Non-leader gang items don't have to poll on dependency signals so this
opens up more non-blocking SDMA channels.
- unlock gang lock when gangs are not needed.
- Change gang factor lookup from vector pair to map and register all
gpus in gang factor lookup regardless of link type so that we can take
advantage of the O(logN) direct key/value lookup time.
Fixes include:
- HSA_PAGE_SIZE_4KB was an incorrect macro to use for gang size limit.
As a result, small copies ended up ganging and hitting latency limit.
Use hardcoded 4096 bytes instead.
- Cap auxillary gang factor to the number of non-XGMI SDMA engines.
Change-Id: Ic23fde131502906a807134a04599aa6d012e8cbb
[ROCm/ROCR-Runtime commit: 62f3f250ce]
The old max memory search algorithm is using Binary Search
algorithm to find last successful memory allocation. But each
successful memory allocation takes times. Since the unsuccessful
memory allocation returns very quick. Changing the search algorithm
to find first successful memory allocation starting from MAX, each
testing step with granularity interval will speed up this test.
Change-Id: Idada3c6f750c94f3bb223f4f3bff4e4ebd3e98f7
Signed-off-by: James Zhu <James.Zhu@amd.com>
[ROCm/ROCR-Runtime commit: caedadcc6f]
Applies the following changes:
add version number to documentation left navigation bar and page title
add an "About" section with a license page
enable htmlzip, pdf, epub formats when publishing on Read the Docs
set pdf title, author, copyright, and version
rename .sphinx/.doxygen to sphinx/doxygen
remove docBin from URL
update rocm-docs-core dependency
Change-Id: I947cf32cd42d9f4e55b1ddd324ad4a7e4ba3f3e3
[ROCm/ROCR-Runtime commit: 1c6ad56dc6]
Use emplace to prevent copying the MappedHandle objects when inserting
entries into mapped_handle_map_.
Change-Id: Id3f40f1eb73ce30e62da53c5aea4dd715e83ac59
[ROCm/ROCR-Runtime commit: 32b3a3c299]
When allocating a memory handle, the NoAddress thunk flag should be set
so that this allocation does not have a virtual address range.
Also, skip mapping the memory when allocating a memory handle
Change-Id: I1c168bc00ddbc158d447197c4dc25f96bad02b19
[ROCm/ROCR-Runtime commit: 29efd8eccd]
After a memory handle is created. hsa_amd_vmem_get_access should return
HSA_ACCESS_PERMISSION_NONE insread of reporting the allocation as
invalid.
Change-Id: I1a09d15c220d48497d09c89059493e538f82aeb9
[ROCm/ROCR-Runtime commit: 2f97049da5]
When using multi-GPU for each BO, a new dmabuf_fd needs to be imported
into libdrm.
Change-Id: Iaa2415c8f655a1ce8e92b0878517a11ff014a1d5
[ROCm/ROCR-Runtime commit: 8b85f9e668]
Set HSA_ENABLE_IPC_MODE_LEGACY off (i.e. use DMA bufs implementation
by default).
Change-Id: I7b1c6cb7d19310adf6f0bfe060736f4adbf7adc2
[ROCm/ROCR-Runtime commit: e20f41df62]
As the KFD IPC IOCTLs will not be upstreamed, change runtime
implementation to use DMA bufs.
DMA buf fds will be passed over abstract unix domain sockets.
The exporter spins a thread that creates a socket server.
The importer connects to the server to fetch the fd.
libDRM will be required to do a manual import and GPU map for
memory that is not already imported and mapped.
For now, use the legacy IPC implementation by default as a
follow on patch will disable the HSA_ENABLE_IPC_MODE_LEGACY
environment variable.
Change-Id: Ifd8469e9adfc81f8a1ea78d6010fb10b515ba1b4
[ROCm/ROCR-Runtime commit: 5dfebdbca9]
Implement HybridMutex to improve latencies compared to KernelMutex when
there is contention between several threads calling hsa_signal_create
and hsa_amd_signal_async_handler.
Change-Id: If53377033e749b0050727964c9303f09b02527cc
[ROCm/ROCR-Runtime commit: 8d3fee5095]
Fixes issue where t1_ counters may not be updated when doing dispatch
profiling, causing a divide by 0.
Change-Id: I91060ac3f9fd2183d277e6e7cd810398a453a87f
[ROCm/ROCR-Runtime commit: 3d1563ee68]
KFD had some fixes for handling of virtual memory APIs. These fixes are
included in interface version 1.15.
Change-Id: Ie701eccf6e032f9ec0a1f4e8a43718964eebddc6
[ROCm/ROCR-Runtime commit: d16c6db2ee]
Update the `hsa.h` header to use the gcc / clang `__BYTE_ORDER__`
macros where available to more accurately autodetect endianness for
the target.
Change-Id: I7312f3badcba9287a30eb14882b91e2a247acc5f
[ROCm/ROCR-Runtime commit: 4971150576]
This reverts commit 4c8a849772. This
change is required for the runtime to generate reliable core dump files,
but this feature has been disabled for now by
816b46868a. Until it is needed, revert
the ABI change in the trap handler to maintain compatibility with older
debugger.
Change-Id: I77a1562dc7962befe2bf88442df858e2d2b1c5ab
[ROCm/ROCR-Runtime commit: 6f828d8609]
This reverts commit 9aa39b0979.
This commit disables core dump feature. Apparently, gfx1101 SA1 waves
can not enter the trap handler because they receive an invalid
address. However, core dump at the debugger has been moved to rocm
6.2.
Signed-off-by: Alex Sierra <alex.sierra@amd.com>
Change-Id: I7915caf58118658e5e7f435f91a0a6216d2fdb42
[ROCm/ROCR-Runtime commit: 5e3be9c28a]
On some systems, pthread_addr_setaffinity_np does not exist, so we need
to use pthread_setaffinity_np on thread after pthread_create
Provided by Julian Samaroo on github
https: //github.com/RadeonOpenCompute/ROCR-Runtime/pull/143
Change-Id: I4649f94333f2d7b0a5993b370a4bfc48d92acecb
[ROCm/ROCR-Runtime commit: 6333fdecf3]
Add query to return flags for GPU agent memory properties and AQL
extensions.
Implement flag to determine that GPU agent is an APU
Change-Id: Ic04c51290b2b9763e14989c117f35a2e22297453
[ROCm/ROCR-Runtime commit: c86837d8d6]
When inspecting waves on architectures where SPI may not initialize TTMP
registers, the debugger cannot reliably know if the trap handler was
entered and if it saved valuable information in TTMP registers.
This patch uses the status.skip_export bit (unused by the compute
shaders) to indicate that it got executed before halting a wave.
This is done except for gfx940, where ttmp11[31] can be used (as long as
TTMP registers are always initialized by SPI for this architecture). It
could be possible to be more selective as architectures always
initializing TTMP registers do not require this step, but always doing
is makes maintenance simpler.
Change-Id: I314db6b37772f7daa8bd405e6662a86658d3f5e0
[ROCm/ROCR-Runtime commit: c5db063b2f]
Extracts and creates a core dump ELF file from a fault event, using
core dump front end.
Signed-off-by: Alex Sierra <Alex.Sierra@amd.com>
Change-Id: Ibbbe41b3d13dd3fcb90161e927d48c329cf513a9
[ROCm/ROCR-Runtime commit: 803e37ded5]
Member added to KFDVersion to report if KFD supports core dump
mechanism. This is done through hsaKmtRuntimeEnable API call while
the topology is being built. It also dictates if core dump will be
generated by either KFD or hsa-runtime.
Signed-off-by: Alex Sierra <alex.sierra@amd.com>
Change-Id: I2e9d4166563402f78613d728446feb692c52d9d1
[ROCm/ROCR-Runtime commit: 54604654bd]
Core dump generation considers ulimit to generate the proper size
file.
Signed-off-by: Alex Sierra <Alex.Sierra@amd.com>
Change-Id: I61d991fc003b173f9075b66bff6a931447720695
[ROCm/ROCR-Runtime commit: 91f2a70817]
This API consists in one function to be called from a fault event at the
hsa-runtime to generate a core dump.
Signed-off-by: Alex Sierra <alex.sierra@amd.com>
Change-Id: Ib1b90d5beb13f93c4e8ebd21fd61705ebb12ca5d
[ROCm/ROCR-Runtime commit: 514b222368]
SegmentBuilder classes are used to get core dump data from the GPUs.
So far, it uses thunk API calls and smaps to collect all data from
the Hardware.
Signed-off-by: Alex Sierra <alex.sierra@amd.com>
Change-Id: I2ad70ca5a951885181d3142653b186b0f6be739e
[ROCm/ROCR-Runtime commit: 1083d5c35f]
- fix logic for using HSA_TOOLS_LIB when rocprofiler-register support is enabled
- report tool load failure for rocprofiler-register
Change-Id: Ife23aa3e6ed19174376cd694764583b73f8976cd
[ROCm/ROCR-Runtime commit: 27eb0516bb]
The alternate scratch memory is used for dispatches that have a low
number of waves but relatively large wave size.
This allows us to keep the tmpring_size.bits.WAVES field of the main
scratch to full occupancy.
Change-Id: I32d240fac4b7d38200d1eebc1b0fdc8a823920d3
[ROCm/ROCR-Runtime commit: a7a3358067]
For devices where the CP FW supports asynchronous scratch reclaim, ROCr
is able to claw-back scratch memory that was assigned to an AQL queue.
With that ability, ROCr does not have to rely on using USO
(use-scratch-once) when assigning large amounts of memory to a queue.
If we reach a situation where we are running low on device memory, ROCr
will attempt to claw-back the scratch memory.
Change-Id: Iddf8ec84e37ab8b9fdc58bafbe2b61fe2acb6eb7
[ROCm/ROCR-Runtime commit: dca8f3a21d]
Separate the event handler and scratch handler portions of the code into
separate functions.
Change-Id: Ifdb7461e816b0f2d3c1c0a74d6f020b4d6fc736c
[ROCm/ROCR-Runtime commit: 64070a9acc]
Update queue structure to add members required for asynchronous reclaim
mechanism and dual-scratch. CP will set the AMD_QUEUE_CAPS_ASYNC_RECLAIM
bit on queue-connect to indicate whether the new features are supported.
The new members are ignored by previous versions of CP FW
Change-Id: Ic8e9ef41c5b1d04f09b43bc9b44b31527863d10f
[ROCm/ROCR-Runtime commit: 0344c8c0b6]
For gfx11, the trap_handler fails to recognize a trap id 3 and report
the exception to the debugger if the debugger is attached.
This is because the 2nd level trap handler looks for the DEBUG_ENABLED
bit in ttmp13 instead of ttmp11. This bit is set by the 1st level trap
handler and is part of the 1st/2nd level trap handler ABI.
Change-Id: Ib36361f53d9bcbbed52320d8c3a9ab2c0b28c7cd
[ROCm/ROCR-Runtime commit: 6916ce358a]
This reverts commit a8e34eaec8.
gfx1150/1151 is merged into mainline now.
Change-Id: Id179949318a37888c74abb5a8610d95bc2f22906
[ROCm/ROCR-Runtime commit: 991bbdcf24]
Skip Extended-scope memory pool as allocation is very close to
fine-grain/coarse-grain but with just different PTE flags.
Only test coarse grain on CPU agent other than the first CPU agent.
Stop bisecting the max size once we are withing 5% to total size for
these pool to speed this test on large memory pools.
Change-Id: I77d1b45a1752ef092dda7c7f27723ea0a292a612
[ROCm/ROCR-Runtime commit: cb5a29955b]
SDMA4.4 and SDMA5.2+ has increased it's available copy size to 2^30 bytes
represented by exponent as bits set in the COUNT field of the
linear copy.
Also note that the full 2^22 byte limit is available from SDMA4 onwards
as it has corrected the 0x3fffe0 HW limitation from SDMA3.
As copy limit has increase, this can change system performance
so provide env var HSA_ENABLE_SDMA_COPY_SIZE_OVERRIDE=0 to fall
back to the original 0x3fffe0 limit for debugging purposes.
Change-Id: I0fb6e5378f68e5b8a00ff559271691a943ee06ee
[ROCm/ROCR-Runtime commit: 81c64228e0]
To be able to trace memcpy asynchronously, both dst and src agents need to have profiling enabled and the api for enabling profiling was only enabling for gpu agents. CPU agents didn't have profiling enabled so the signal owner could not be known. hsa_amd_profiling_get_async_copy_time will fail with an HSA status error because it can't read the agent for the given signal.
Change-Id: Ie165e0e39b8fcd6992a55695b9ffcead10a8e812
[ROCm/ROCR-Runtime commit: ae1da390bd]
Jonathan Kim
James Zhu
Sam Wu
David Yat Sin
Jonathan R. Madsen
Joseph Huber
Lancelot SIX
Ruili Ji
Alex Sierra
Giovanni LB
Shweta Khatri
Lang Yu
Youssef Aly