Because gfx700 has local memory but other APUs don't, we should reflect
that in the code. Meanwhile, fix a bug that on gfx902 svm aperture is not
added when calling hsaKmtGetNodeMemoryProperties().
Change-Id: Id840f2db0b14fda9ee713b219a9474c15f8a9771
Signed-off-by: Yong Zhao <Yong.Zhao@amd.com>
On some asics, like tonga, the memory alignment size is as big as 0x8000.
fmm_allocate* alloc vm area with size passed in which is not aligned mostly.
But __fmm_release free vm area with vm_object_t->size which is aligned.
That might cause aperture_release_area fail to free the vm area as the
size might be bigger than zone itself or it just free another vm area
nearby unexpected.
This patch somehow will alloc more space than it needed on tonga.
gfx900+ is not affected.
Change-Id: I5a88c92b08c4e6f6bc05881798f769b55d6debe9
Signed-off-by: xinhui pan <xinhui.pan@amd.com>
Previously we used the first dgpu mem, but after careful examination, we
found it only needs to be a GPU, so we modify the code to reflect that as
well.
Change-Id: I069d9b8e247aed55c1f885b79f743ea8e03ddf93
Signed-off-by: Yong Zhao <Yong.Zhao@amd.com>
The information can be obtained directly from node id. Also improve the
whole logic for future compatibility.
Change-Id: I130733be4e7930d5953d5e81409905e60c2ec35e
Signed-off-by: Yong Zhao <Yong.Zhao@amd.com>
Unset ret_addr when unmapping the address space reservation. Otherwise
it may try to unmap it again later.
Remember the actual map_size and use it instead of len outside the
reservation loops.
Change-Id: I1a6b3fecfb59e22a713e5ed49c3ed37914cb6fb5
Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com>
node is used repeatedly and excessively, which caused unnecessary confusion.
Change-Id: I4ae4171887df5e5b85209a5af8a636e6d72e5e82
Signed-off-by: Yong Zhao <Yong.Zhao@amd.com>
simple test of mapping many system memory to gpu.
before
[ RUN ] KFDMemoryTest.MMap
[ ] Using ISA for GFXIP 9.0
[ ] successfully register/map 32GB system memory to gpu
[ OK ] KFDMemoryTest.MMap (36932 ms)
after
[ RUN ] KFDMemoryTest.MMap
[ ] Using ISA for GFXIP 9.0
[ ] successfully register/map 32GB system memory to gpu
[ OK ] KFDMemoryTest.MMap (11441 ms)
So there is 11s VS 36s improvement.
Looks like we can do something similar with vm_area too.
Change-Id: I0349aacdeddec3534016d28176f0fabf632c61fc
Signed-off-by: xinhui pan <xinhui.pan@amd.com>
Compare with gpu_mem_count instead of deprecated NUM_OF_SUPPORTED_GPUS
to prevent overflows in case no dGPUs are present.
Change-Id: I71fcb7503ba4c20bffadbdb04cefc4e4027a7df7
Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com>
There were two doorbells, one embedded in another, which are very confusing.
Change the member variable name to mapping to differentiate them. Also,
rename doorbells_mutex to just mutext for brevity.
Change-Id: Iaa14a1a3ee09449a9089fc1fb39c916fdf32fb44
Signed-off-by: Yong Zhao <yong.zhao@amd.com>
If opening drm render device fails (usually when the user is not a member
of video group), fmm_init_process_apertures() still returns success,
resulting in weird segfault in a later stage.
Change-Id: Ifbde4481629988944ad7f384d59753c88e287fa9
Signed-off-by: Yong Zhao <yong.zhao@amd.com>
Avoid warnings of the type
error: 'strncpy' specified bound 64 equals destination size
With the destination being 0-initialized, subtracting 1 from the
destination buffer size will ensure that the destination will be a
0-terminated string, even when it's truncated.
Change-Id: I7c3a90482065ce4d020db215e3e41348de51a083
Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com>
Add back missing pthread_mutex_lock.
Handle all error cases in fmm_release.
Change-Id: I8efa561ddadfd769cede5bf86300215ba3fb3dd1
Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com>
__fmm_release actually fails to find the object if address is not
pagesize aligned. And the caller did not notice this as __fmm_release
has no err code return.
So to fix this, move the object lookup in caller, and use vm-object
instead. Also fmm_release will pass up the error code.
Change-Id: Ib8ea1ea5ae844844fd20e8e01f0fdb841d218f2c
Signed-off-by: xinhui pan <xinhui.pan@amd.com>
When KFD is already opened, Opening it again should return
HSAKMT_STATUS_KERNEL_ALREADY_OPENED to align with the specification.
Change-Id: Ib10a2d2c48781600bea7d072557d03ccb1a2bc19
Signed-off-by: Yong Zhao <yong.zhao@amd.com>
Upstream KFD doesn't support mapping doorbells to GPUVM yet. Fall
back to the old method.
Change-Id: I452a6fc59b88329b833844e3914c480c2f13c82d
Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com>
- Clean up and renumber scratch memory ioctl
- Renumber get_tile_config ioctl
- Renumber set_trap_handler ioctl
- Update KFD_IOC_ALLOC_MEM_FLAGS
- Renumber GPUVM memory management ioctls
- Remove unused SEP_PROCESS_DGPU_APERTURE ioctl
- Update memory management ioctls
Replace device_ids_array_size (in bytes) with n_devices. Fix error
handling and use n_success to update device_id arrays in objects.
This commit breaks the ABI and requires a corresponding KFD change.
Change-Id: Ibf0af5a5188e817c886eab388d1533130fc18293
Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com>
The main point is to move update_ctx_save_restore_size() out of if()
condition.
Change-Id: I58a1a4f3edca2d1c510fdd0e31e59b5c41e92a14
Signed-off-by: Yong Zhao <yong.zhao@amd.com>
Move opening of DRM render nodes from topology to FMM aperture
initialization. Keep the same FDs open for the life time of the
process to match how KFD uses the VMs in the FDs. Call acquire_vm
ioctl during aperture initialization to let KFD use the VMs from
the render nodes.
Change-Id: Ie07d57788cbe685b1841cccc00820c12894a0356
Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com>
Query GPUVM aperture limits of all dGPUs to determine SVM aperture
base and limit. This depends on a recent KFD change that reports
the GPUVM apurture limits for dGPUs in the
AMDKFD_IOC_GET_PROCESS_APERTURES_NEW ioctl (drm/amdkfd: Simplify
dGPU SVM aperture handling).
Only initialize SVM aperture once, instead of once per GPU.
Don't call AMDKFD_IOC_SET_PROCESS_DGPU_APERTURE. It's not needed any
more and will not be upstreamed.
Change-Id: Ib3389e8ba18505ba15fc33f45fe8a57e690a565d
Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com>
Define dgpu_mem_init before it's used and keep the code close to the
rest of the aperture initialization code.
Change-Id: I14ad11a364524a15affee9186b1298ba7d56d2c9
Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com>
kfdtest hsaKmtOpenKFD failed after 1019 loop if using --gtest_loop=-1,
because default max open file handle limit is 1024. Found shmem file handle
is not closed from lsof output.
Change-Id: I474de2bae6c03e879a219dedf5f18639118b73e5
Signed-off-by: Philip Yang <Philip.Yang@amd.com>
On discrete GPUs place the EOP queue in VRAM. The reader/writer of this
queue is the CP and the size is small. Dispatch latency improves
through lower read latency in AQL completion phase.
Change-Id: Id8351dcddbd21fd7c7d699803c96434c9132db71
Signed-off-by: Jay Cornwall <Jay.Cornwall@amd.com>
Invisible device memory is mmapped as PROT_NONE.
Normal CPU access to the memory is still not allowed but
struct vm_area_struct will be created for the memory address
so ptrace can access the memory via the vma.
Change-Id: I07c69208716c920ccce33e6b494b610b61a0a7c1
Signed-off-by: Oak Zeng <Oak.Zeng@amd.com>
UCX test cases are reporting uninitialized values when CMA fails. The
application should ideally ignore SizeCopied when the function fails but
it doesn't. This is leading to wrong diagnosis.
v2: Fill in partial SizeCopied in case of failure
Change-Id: I6b7e1c19a8b702ec91ca64201a3dda27bd897877
Signed-off-by: Harish Kasiviswanathan <Harish.Kasiviswanathan@amd.com>
This change is needed to match other higher level components.
Change-Id: I45114d23f2ed428dfbbb836061b3020c5ab166ec
Signed-off-by: Yong Zhao <yong.zhao@amd.com>
This reverts commit f7689d4fef,
Plus a bug fix to patch "Cleanup fmm.c":
Call id_in_array with correct parameter. The third parameter
of id_in_array is size in byte of the array, not the number
of array items. Call it correctly.
Change-Id: I72d8e2fcc0df32af76c72967386e92c1be18c159
Signed-off-by: Oak Zeng <Oak.Zeng@amd.com>
to fmm_allocate_memory_object. This function name was confusingly
similar to fmm_allocate_device and __fmm_allocate_device. The new name
reflects its function better: allocate the VM object and the kernel
mode buffer object.
Change-Id: I6604d228004b4d41e871d4de784786823608b5d6
Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com>
This reverts commit b4c89c1ea7.
This change caused a regression ()
Revert temporarily
Change-Id: Ic3829264151e37d1f8c6927c6f464006234ba17f
Signed-off-by: Oak Zeng <Oak.Zeng@amd.com>
This reverts commit 019f7cbd20.
This change caused a regression ()
Revert temporarily
Change-Id: I5af59d319afeb7f0b03e5a09e8397e3853b8b37b
Signed-off-by: Oak Zeng <Oak.Zeng@amd.com>
1. Renamed _fmm_map_to_gpu to _fmm_map_to_apu_local
to reflect the real semantics of this function
2. Renamed _fmm_map_to_gpu_gtt to _fmm_map_to_gpu
because this function is used to map both gtt
and local memory
3. Call _fmm_map_to_gpu in _fmm_map_to_apu_local
to get rid of duplicated codes
Change-Id: Id8e3ebfffe0a3c27ebdcac8a8f4dc3738d67d10a
Signed-off-by: Oak Zeng <Oak.Zeng@amd.com>
1. Initialize pointers to NULL in vm_create_and_init_object
2. Added helper function to add/remove device ids to/from mapped arrary
3. Only map nodes that were not mapped currently
4. Remove unnecessary condition check on object frees
Change-Id: I7aed6d40c7464be0d168d5796229af55451e0f34
Signed-off-by: Oak Zeng <Oak.Zeng@amd.com>
Print data in PMC trace when the debug level is set to 7(pr_debug).
Change-Id: I9abbb8f6c3f7962fb637528578c1a58b7784042d
Signed-off-by: Amber Lin <Amber.Lin@amd.com>
_fmm_unmap_from_gpu is called in fmm_map_to_gpu_nodes
to unmap buffer from nodes that is already mapped to
but not in the new map nodes list. Previously, the unmap
was called unconditionally even though the size of the
array to unmap is 0. This fixes the issue by calling
the unmap func only when the unmap array size is not 0.
Also releases the fmm_mutex on error returns
Change-Id: Iadd8383caf7ebb92f02618798c5efd138a352aaa
Signed-off-by: Oak Zeng <Oak.Zeng@amd.com>
Buffer mapping to devices and buffer registration to
devices can be changed b/t two pointer info queries.
Thus update buffer mapping info and registration info
only when mapping and registration changed. This is
done by free mapped_node_id_array on mapping to new
device and free registered_node_id_array on registration
and re-allocate them on next ptr info query.
Also uses fmm_mutex to avoid race conditions in case
of calling hsaKmtQueryPointerInfo concurrently with
calling of buffer mapping or registration
Change-Id: Ibc2e20be1fc0147066f873dfa44b21f5015104b7
Signed-off-by: Oak Zeng <Oak.Zeng@amd.com>
Also separated a function for removing CPU mapping
and reserving address, as a refactoring of codes
Change-Id: I1feb85b0b2ec942487f899ec3192c7c47dd7c7d5
Signed-off-by: Oak Zeng <Oak.Zeng@amd.com>
Previously kfd device is used to map memory for CPU access.
However this is not compatible with how TTM handles CPU mapping
on eviction - memory won't be unmapped and remapped on restore.
This fixes the issue by mmapping memory using DRM render device.
This patch requires a coordinated kernel driver change to work.
To make it compatible with old kernel driver, some temporary codes
are included. Once the coordinated kernel driver is checked in,
the temporary codes can be removed.
Change-Id: Ie7b304c4a82b7e8d5ab703acb81d66430af4f0bc
Signed-off-by: Oak Zeng <Oak.Zeng@amd.com>
Yong Zhao
xinhui pan
Felix Kuehling
Slava Grigorev
Shaoyun Liu
Oak Zeng
Philip Yang
Jay Cornwall
Harish Kasiviswanathan
Laurent Morichetti
Amber Lin