Allocate doorbells for dGPUs in the SVM aperture and map them for
GPU access. This is necessary to allow GPU-initiated submissions to
user mode queues.
Depends on new doorbell BO allocation flag in KFD.
Change-Id: I0737bef4a4764bb4a66c43846707ead2108f6601
[ROCm/ROCR-Runtime commit: 2e0a6eb371]
Add two pointer attributes APIs:
hsaKmtQueryPointerInfo - allow the user to query the memory information
using a pointer. This pointer can point to any address inside the
range known to HSA.
hsaKmtSetMemoryUserData - allow the user to attach data to a pointer to
add memory tracking information. This pointer must match the start
address of a memory allocation or registration.
TODO: This patch implements support on dGPU. Needs to add APU.
Change-Id: I4711809274248434901f0794f50ebfa13a7371a8
[ROCm/ROCR-Runtime commit: 51e4d27c37]
The Runtime requested this information so they can tell easily
whether a pointer is part of HSA shared address space or not.
Change-Id: If2041ed34031636677d692bc2dc6625634027ed4
[ROCm/ROCR-Runtime commit: 0ed29f5191]
Use the object size when freeing address space, instead of the
parameter passed in by the caller. The parameter may be incorrect
due to app or runtime bugs, or when the buffers is an AQL ring
buffer with double mapping workaround.
Change-Id: I00bb31d4520ef969a49d6d5ea723e8a33418acc3
[ROCm/ROCR-Runtime commit: 006f3ee41b]
Allocate SVM address space for the registered memory and use new
userptr support in KFD to create a system memory BO associated with
the given user pointer. Map this BO at the SVM address for CPU
access.
MapMemoryToGPU can be used with the registered user pointer and
will return the SVM address as alternate GPUVA.
Change-Id: I4886e193c51fb6870a567878870c36bf8b5c3748
[ROCm/ROCR-Runtime commit: 85f9efb1a0]
Allocations from GPU nodes will return VRAM, not system memory.
Only non-paged allocation from GPU nodes is supported. System
memory can only be allocated from CPU nodes (usually node 0).
The HostAccess flag is no longer used to distinguish the memory
type. It only indicates, whether the memory is mapped for CPU
access.
Maintain compatibility with broken KfdTests by returning system
memory for paged-memory requested from GPU nodes.
Change-Id: I514defede735f55e6de436f41944125b6f2c4ccf
[ROCm/ROCR-Runtime commit: 887b32fe86]
Stop using NUM_OF_SUPPORTED_GPUS. For now the definitions itself cannot
be removed as ioctl code is in upstream Kernel.
Change-Id: If846625a8ad5062d5483e762850c793d3c00b9d0
[ROCm/ROCR-Runtime commit: ce83dc623f]
Fix hsaKmtRegisterMemory to be a no-op for now and move the multi-GPU
implementation to hsaKmtRegisterMemoryToNodes. Make GPU memory mappings
of host memory visible to all GPUs by default. Device memory is still
visible to the allocating GPU only by default (but can be overridden
with hsaKmtRegisterMemoryToNodes for experimenting with P2P).
Change-Id: I73408afbe3b10c8dad2ab3a780f58413249692e6
[ROCm/ROCR-Runtime commit: 063ad3ad9e]
Unify fmm_get_aperture_base and fmm_get_aperture_limit into one
function. Make the return value to HSAKMT_STATUS.
Change-Id: I0b3f563ffb268947ab891f4935f61788d0af0e01
[ROCm/ROCR-Runtime commit: cb53548c89]
Patch submitted by Besar Wicaksono
1. Bug on detecting local memory size interpreted as 32 bit value
instead of 64. The bug causes thunk to go into an infinite loop trying
to reserve virtual address range for dgpu system memory.
2. SIMD count in the node property is 0. Runtime use this attribute to
find a gpu device.
Regarding other attributes of intel+tonga topology, Harish started a
discussion on August iirc, could you please share an update ?
This would help me progress with more tests such as scratch memory,
which require the scratch aperture information in order to construct a
buffer srd in gpuvm space.
3. Bug on releasing memory via fmm_release, where no actual release is
being done. The vm_object can't be found because the memory size does
not match due to the allocation padded the size with 32KB.
4. Pointer arithmetic on vm_area allocation/release. The value of
vm_area_t::end seems to be interpreted inconsistently whether it is
(start + size -1) or (start + size).
One example of potential issue I see is the logic could report
larger size of the hole in the vm area list.
5. Resource cleanup on multiple library load/unload within a single
process.
- Any memory allocation on subsequent library load will result
an error "va above limit". To my understanding this is due to
the reserved memory for the system memory not being released on unload.
- The static variable events_page needs to be invalidated
appropriately on library unload so the next load could
reinitialize it.
6. Could you please update if AQL queue is ready to test with the stg
kfd/kmt ?
7. The system memory allocation with size larger than 32KB seems to be
padded by an extra 32KB. I was wondering if we could remove this
overhead.
Change-Id: I039988d36637525089c7569dc3b77e58750e2121
[ROCm/ROCR-Runtime commit: ee08f537a7]
Felix Kuehling
Amber Lin
Ben Goz
Harish Kasiviswanathan
Moses Reuben
Oded Gabbay
Alexey Skidanov