Implement three new APIs for IPC buffer sharing:
-hsaKmtShareMemory()
-hsaKmtRegisterSharedHandle()
-hsaKmtRegisterSharedHandleToNodes()
Add new ioclts necessary for the above APIs.
Change-Id: Ia2b4d0dc91ec64bff959395d11c0536467404792
Signed-off-by: Harish Kasiviswanathan <Harish.Kasiviswanathan@amd.com>
[ROCm/ROCR-Runtime commit: 559e31d6ff]
A memory region is allowed to be registered multiple times when the memory
is specified by a user pointer. If it's registered with the same user
pointer but with different sizes, it's treated as different instances and
multiple VM objects are created with different GPU address.
Change-Id: I49627111bb5db36d18f1133b252fb62a611f06a4
[ROCm/ROCR-Runtime commit: 2a50ebba98]
Currently, if a process' parent called hsaKmtOpen, the child will be
unable to open a connection to KFD, since kfd_open_count will be > 0.
When forking, the refcount should be reset, in order to allow the child
to re-open /dev/kfd.
Change-Id: Ia4b78f6bacc4f82e8ac724e5f488a3eff5084007
[ROCm/ROCR-Runtime commit: 0de39b6724]
Ensure that the write index and ring buffer contents are visible
to the HW before sending the doorbell. The latter is a write-combined
MMIO store and must be ordered with prior cacehable non-MMIO stores.
Also be more explicit about memory semantics for doorbell stores.
Change-Id: Ie4d96a7ee2a507237a8dbe7705fdf234d62ce9ba
[ROCm/ROCR-Runtime commit: d5b4078072]
gfx802 requires a workaround for a VM TLB bug in which lookups use
the ACTIVE bit of the 8th PTE within any aligned group of 8 PTEs.
Until this is fixed in amdgpu the GPUVM doorbell logic will fail.
Change-Id: I5ec7b1fcd8b7677011a141d27cfc486c45d9a415
Signed-off-by: Jay Cornwall <Jay.Cornwall@amd.com>
[ROCm/ROCR-Runtime commit: 5493ae420b]
If we issue too many copy commands without syncing and wrapping happens,
we need to wait for the blits to be done before moving forward otherwise
we will overwrite the kernel args of the blits in flight.
Change-Id: I9a21e31ce07f8e8157ca38e96dc264ff47fd3639
[ROCm/ROCR-Runtime commit: 5519c96b74]
Allocate doorbells for dGPUs in the SVM aperture and map them for
GPU access. This is necessary to allow GPU-initiated submissions to
user mode queues.
Depends on new doorbell BO allocation flag in KFD.
Change-Id: I0737bef4a4764bb4a66c43846707ead2108f6601
[ROCm/ROCR-Runtime commit: 2e0a6eb371]
CPU cache information reported by Thunk topology is obtained from cpuid
instruction. This instruction only applies to X86 systems. It can cause
compile errors on non-X86 platforms. This patch temporarily disables CPU
cache functions in topology for non-X86 platforms in order to compile.
Change-Id: If86671817b0d036cb324eebf3f354682bfb75856
[ROCm/ROCR-Runtime commit: 660a6ebbd4]
Add vm_find_object_by_userptr_range so QueryPointerInfo can find the
object as well when the pointer is not the starting address but it's
inside the memory range. Also rename vm_find_object_xxx functions to
_by_address and _by_address_range to be consistent.
Change-Id: I5c2b3a05b41493e32b7fd9154665bf078b043606
[ROCm/ROCR-Runtime commit: 4911c91389]
Introducing tiling format for images, still using LINEAR for now.
Using the new KFD/Thunk API hsaKmtGetTileConfig API for the address library.
Change-Id: Ic0677429dd320eef09ab62dddaf9b2dd94c4f904
[ROCm/ROCR-Runtime commit: 538736a660]
Add CPUVM aperture to keep track of memory allocation that is not known
to GPU driver. Together with GPUVM, this patch adds the pointer attributes
support to APU.
Change-Id: If13f9cf01ff8b9f709b99b66661e7505246adf4c
[ROCm/ROCR-Runtime commit: 19f2676ea7]
Add two pointer attributes APIs:
hsaKmtQueryPointerInfo - allow the user to query the memory information
using a pointer. This pointer can point to any address inside the
range known to HSA.
hsaKmtSetMemoryUserData - allow the user to attach data to a pointer to
add memory tracking information. This pointer must match the start
address of a memory allocation or registration.
TODO: This patch implements support on dGPU. Needs to add APU.
Change-Id: I4711809274248434901f0794f50ebfa13a7371a8
[ROCm/ROCR-Runtime commit: 51e4d27c37]
C11 atomics are not statically guaranteed to be lock free and so
may not be atomic with respect to atomic operations originating
outside the standard library, such as platform atomics.
C11 macros to statically discover always lock free operations
(ATOMIC_*_LOCK_FREE) do not cover uint64_t in GCC and
std::atomic<uint64_t> is not a type alias of any covered type.
All use of __atomic by atomic_helpers.h is statically checked to be
always lock free.
GCC builtin fencing does not appear to be strong enough for WC memory.
Added an option (enabled) to enforce consistency for WC memory on x64.
__sync builtin's were not used as they were declared legacy by GCC.
Added a strongly conservative option (ALWAYS_CONSERVATIVE) to enable
use of full memory fences in place of partial fences and compiler
driven processor specific optimization.
Change-Id: Id7aaaca626144070f58759f6a348cbee4612bbc0
[ROCm/ROCR-Runtime commit: 1bc15bbf79]
For APUs, use /proc/cpuinfo to get Marketing name.
Change-Id: I4a17516d26a092683f36631032be00ad44f7e7fe
Signed-off-by: Lan Xiao <Lan.Xiao@amd.com>
[ROCm/ROCR-Runtime commit: df593aa076]
Change hsa_code_object_serialize and hsa_code_object_deserialize to use memcpy instead of hsa_memory_copy since it is system->system copy
Change-Id: I329e270ae4e2fc25e177dc8080d93662ffb261ab
[ROCm/ROCR-Runtime commit: 73ed2116d5]
Compiling in 32bit mode is broken, and we don't have an intention on
restarting compatibility with 32bit apps.
Change-Id: I5524b5b63fe62e6026aa04d84c4510e290a86106
[ROCm/ROCR-Runtime commit: e0c77a38cb]
Konstantin Zhuravlyov
James Edwards
Harish Kasiviswanathan
Ramesh Errabolu
Amber Lin
Yong Zhao
Sean Keely
Andres Rodriguez
Chris Freehill
Jay Cornwall
Christophe Paquot
Felix Kuehling
Lan Xiao