We should check compute core instead of cpu core,
in order to exclude the case of APU.
Signed-off-by: Jesse zhang <jesse.zhang@amd.com>
Change-Id: I2ec2a6807f51f49f80e0e500f5d9af81c2efae37
GC 9.4.3 to set gfx target version to 9.4.x dependent on revision and
capabilities. Due to this, where applicable, mask off the gfx target
stepping version and only check major/minor version (9.4). There are no
collisions due to this change since GC 9.4.3 is the only ASIC that uses
gfx target version 9.4.x.
Signed-off-by: Graham Sider <Graham.Sider@amd.com>
Change-Id: I72803e594c421f054d18ccfa7e92c507128fa5be
Use q->total_mem_alloc_size for munmap in SVM codepath of free_queue.
Signed-off-by: Graham Sider <Graham.Sider@amd.com>
Change-Id: I2fecaa1ddb337b1fe71f9cbba45a0c9467eff0c0
Currently, on queue destroy, context save restore memory is freed
only for a single XCC. Instead, we need to free the entire context
save restore memory, which was allocated for all XCCs.
Signed-off-by: Mukul Joshi <mukul.joshi@amd.com>
Change-Id: I51ebb12fa8d5ebed41979d68e74f7c5392dca062
Do not allocate the EOP buffer when not required.
Signed-off-by: David Belanger <david.belanger@amd.com>
Change-Id: I1664a3f0a882219a72278174006cdb8d46fd4f5e
In multi-partition modes, e.g. CPX, we want to create new file
descriptor despite using the same render node. Update
open_drm_render_device to use a gpu_id to fd map partitioned by render
node. Different gpu_id's requesting the same render node will be added
to that render node's map list for fetching its fd. Different gpu_id's
requesting different render nodes as well as the same gpu_id's
requesting the same render node will behave as they did previously.
Signed-off-by: Graham Sider <Graham.Sider@amd.com>
Change-Id: Ie153d42355d4d75b1c6ba6ff40fac3295bc87009
Allocate debug area big enough for all XCCs in the partition. Also, fix
the cu_num calculations as driver now reports cu_num as the total number
of CUs in the partition.
Signed-off-by: Mukul Joshi <mukul.joshi@amd.com>
Change-Id: I6e80d57196b770bb3c2506bc58cb366c0046084b
Add gfx version for VGPR size per CU calc, add FAMILY_AV to KfdFamilyId,
add blacklist filter to kfdtest.exclude.
Signed-off-by: Graham Sider <Graham.Sider@amd.com>
Change-Id: I9b8072e45f4d497e0a8fd3f8f97f1425238e8b42
Get more debug information about user pointers that were registered
through SVM API, and triggered by memory exception events.
A new kfdtest with this use case was also included inside
KFDExceptionTest.
Signed-off-by: Alex Sierra <alex.sierra@amd.com>
Change-Id: I0ef4929afe0625b9b5cbbbebef11ede66dda60ab
Register and map userptrs through Shared Virtual Memory(SVM) API at
the Kernel level when available. Using this approach, performance
will be improve as register/unregister memory will not trigger any
system call to KFD driver.
Signed-off-by: Alex Sierra <alex.sierra@amd.com>
Change-Id: I3726b4b5e1c6a52a83786fbe0af6322eb29ae7c9
Implement hsaKmtExportDMABufHandle, which can be used for a new
upstreamable RDMA solution. It exports a DMABuf handle for an arbitrary
virtual address along with the offset of the address within the
allocation. It also checks that the size of the intended export does
not exceed the allocation.
This uses the new AMDKFD_IOC_EXPORT_DMABUF, which requires KFD ioctl
API version 1.12.
Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com>
Change-Id: Ie5fdb1f73ab3c7fa36c315ce326b1fb89eacc8b6
The Shader Engines number should be shadder array_count divided by simd_arrays_per_engine
not array_count.
Signed-off-by: Xiaogang Chen<Xiaogang.Chen@amd.com>
Change-Id: I808d1fedd6b9843500719e902ecf759f5668a7d1
Fixed VGPR memory size, size was too small for some GPU, causing a memory overflow.
Refactored macro code into a function.
Thanks to Jay Cornwall for locating the problem and proposing the fix.
Change-Id: Iffedea1c4f341967f02c56d810ff048225b02c16
Signed-off-by: David Belanger <david.belanger@amd.com>
This is a temporary work around for GPU hang issues observed on GFX11.
Change-Id: I98fbedbbd1c51fe402c2116b35ca548931a390c9
Signed-off-by: David Belanger <david.belanger@amd.com>
This reverts commit 7787a039bd.
It causes a regression in pytorch benchmark.
Change-Id: I96173dbd061cf38d6f451c02cb181ae51b7f625e
Signed-off-by: Eric Huang <jinhuieric.huang@amd.com>
This reverts commit 178a619b80.
There are some openMP issues that were introduced after SVM userptr
feature was added.
Signed-off-by: Alex Sierra <Alex.Sierra@amd.com>
Change-Id: I7ef87c5232a3bcbe594c743fa4b4958601845ba5
This reverts commit 45fad29752.
There are some openMP issues that were introduced after SVM userptr
feature was added.
Signed-off-by: Alex Sierra <Alex.Sierra@amd.com>
Change-Id: I6566c9f0d39d05ecb92f38159880763f432939a5
This reverts commit 8a746bdaed.
There are some openMP issues that were introduced after SVM userptr
feature was added.
Signed-off-by: Alex Sierra <Alex.Sierra@amd.com>
Change-Id: Ib01046571d2c84fa0fd228ecba0dee0eae3f994d
This is handled by __fmm_release calling aperture_release_area.
Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com>
Change-Id: Ib8ed300e1734f03aeb9dfc8074897ece310b8af9
Use a common helper for CPU mappings to reduce duplicate code.
Consistently use MAP_SHARED for all render_fd mappings.
Remove double-mapping for AQL queue buffers on the CPU. This workaround
is only needed on the GPU.
Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com>
Change-Id: Iff86c8cc9f1e5c982614b3f11129bc2cf8cbba02
The NULL pointer check was the only way for that function to fail. And it
was done after the pointer was accessed. Simplify this by just returning
the result as a return value instead of using a pointer as output
parameter. This way the function can never fail and the caller doesn't
need to do any error handling.
Declare the function in libhsakmt.h instead of duplicating the
declaration in fmm.c.
Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com>
Change-Id: I91b90d66166fd3b5cdc47c73a9bbc369c45b51fe
Setting this variable to '0' will force to disable memory
registration/allocation through SVM API mechanism.
Not setting this or setting to '1', SVM API will be used only if all
GPUs support it.
Signed-off-by: Alex Sierra <Alex.Sierra@amd.com>
Change-Id: Icdf7656de09aa9988b567ec6c024953398e9bb48
When is hsaKmtCreateQueue called first time for node
doorbells[NodeId].size is initialized to zero in init_process_doorbells
but used to calculate the doorbell offset. It works just by accident
because doorbells[NodeId].size is uint32_t so -1 will be 0xFFFFFFFF which
is zero extended into 0x00000000FFFFFFFF and it will work as long as mmap
offset bits are not within lower 32 bits.
Bug: https://github.com/RadeonOpenCompute/ROCT-Thunk-Interface/issues/78
Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com>
Change-Id: Ia791adfc51363d4704cb50fa4f01137b7dd48a75
When hsa is closed, it would close open fds for /dev/kfd but
not for /dev/dri/renderD*. This caused issues with CRIU
checkpoint, which expects that /dev/kfd will be open if
/dev/dri/renderD* is.
As a workaround for the CRIU behaviour, leave /dev/kfd open
when closing hsa.
Signed-off-by: David Francis <David.Francis@amd.com>
Change-Id: Ie1b2d5b1d8986750b0e560ae2934b7c73cff942e
Get more debug information about user pointers that were registered
through SVM API, and triggered by memory exception events.
A new kfdtest with this use case was also included inside
KFDExceptionTest.
Signed-off-by: Alex Sierra <alex.sierra@amd.com>
Change-Id: I8e9df3c1c6c3f42d7b9235d12406d80d31746443
Register and map userptrs through Shared Virtual Memory(SVM) API at
the Kernel level when available. Using this approach, performance
will be improve as register/unregister memory will not trigger any
system call to KFD driver.
Signed-off-by: Alex Sierra <alex.sierra@amd.com>
Change-Id: I20723cbeb340bf48b95e1115f0102c031397bc14
Prior to launch some ASICs may re-use PCI DIDs from older generations.
This can cause issues during topology initialization as hsa_gfxip_table
lookups will override sysfs-provided gfx versions, causing incorrect
gfxip selection. Since no new entries will be added to hsa_gfxip_table,
limit its search only to pre-GFX11 ASICs.
Signed-off-by: Graham Sider <Graham.Sider@amd.com>
Change-Id: I53eaefac5db2650a36a6ce9f21daf750f50cfd26
fork process copy-on-write MMU nitifier on CWSR range will evict user
queues, and then update GPU mapping and resume queues, use MADV_DONTFORK
to avoid COW MMU notifier callback on CWSR SVM range.
Use mmap to alloc SVM range for CWSR because posix_memalign don't alloc
new range in child process, this fails to register svm range as range is
invalid address in forked child process.
Change-Id: Ibaea56a691dd6f577ed2e1f2d43f4a3500b8316f
Signed-off-by: Philip Yang <Philip.Yang@amd.com>
To remove duplicate mmap aligned allocation code.
Change-Id: Ibc05cc4aaf6d190bd2382e33bdeca1496960c5f2
Signed-off-by: Philip Yang <Philip.Yang@amd.com>
mmap alloc larger address range with align padding page plus guard
pages, then unmap the padding and guard pages at beginning and end
of the range, return aligned address range.
Change-Id: Iaf3c711a079c744289efbafee9b5e63aaf724765
Signed-off-by: Philip Yang <Philip.Yang@amd.com>
GFX1036(ISA version) is not included in the previous range.
This patch can really include all gfx10 series ASICs.
Signed-off-by: Aaron Liu <aaron.liu@amd.com>
Change-Id: I0e28dbfc031c216166b306b9fb39f644f75a330f
Avoiding the segfault, runtime debugger enable is not supported
if the firmware of gpu doesn't support debug exceptions.
Signed-off-by: jie1zhan <jesse.zhang@amd.com>
Change-Id: Ifad57a6e78cb1c92b1f8927355ece8c64e89c51b
Remove potential double free condition when free_queue() is called
after hsaKmtDestroyQueue() if mapping doorbell fails during queue
creation.
Signed-off-by: Mukul Joshi <mukul.joshi@amd.com>
Change-Id: If2aa19c455b30d2940b232dbafb9cc1eaad721a5
Queue ctx_save_restore memory is allocated with size
ctx_save_restore_size + debug_memory_size, use the same size
in free_queue to free ctx_save_restore memory.
Change-Id: I4902ff15fb82ddea64b8342b89776a1bf5c38d13
Signed-off-by: Philip Yang <Philip.Yang@amd.com>
Avoiding segfault when an invalid SharedMemoryHandle is passed in
when calling fmm_register_shared_memory.
Change-Id: I0e0bbed01487fc10afcbb170eb9330e70b209d14
Signed-off-by: David Yat Sin <David.YatSin@amd.com>
Now that HsaNodeProperties is passed in to
topology_get_node_props_from_drm, check that pointer instead of the
pointer for MarketingName (which throws a compiler warning)
Signed-off-by: kent.russell@amd.com <kent.russell@amd.com>
Change-Id: If76b24e1bab5a62e514ab440b6316c7b7cd264c1
Query family id info from drm render node, then
ROCr can query this info directly from Thunk
instead of parsing the info by itself.
Signed-off-by: Lang Yu <Lang.Yu@amd.com>
Change-Id: I030bd27ab2379fbf87f3d787302c3b8613456278
The debugger can now request snapshot copies with entry size and
set/clear watchpoints by device.
v3: drop min version check to v10.0
v2: check runtime allowance from v10.3 to 13.x
Signed-off-by: Jonathan Kim <jonathan.kim@amd.com>
Change-Id: I9befefb596201a11591de218db29a9317b41e69b
The debugger depends on the CWSR area being executable. Set the right
flag when registering SVM memory.
Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com>
Tested-by: Laurent Morichetti <Laurent.Morichetti@amd.com>
Change-Id: I7441e214d1a4da8324d775e777976fabd1c81a6f
The CMA feature is deprecated and about to be removed from the DKMS
branch. It was never supported upstream. Leave dummy functions in
place for now.
Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com>
Change-Id: I9e51403d753cb91630553aff4f19e931af509740
hsaKmtMapMemoryToGPU should not try to map VRAM on peer GPUs that don't
have an IO-Link to the memory. The new P2P mapping code in KFD will
fail otherwise.
Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com>
Change-Id: I6d59b55651b98756865a0f69eafef3e386372cf3
This allows init_process_apertures to use the whole consistent topoology
instead of taking its own partial snapshot.
Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com>
Change-Id: Ia13e7aa7fcd090ea8d6cacd4babb29a27c20207f
Kernel debug IOCTL got version bumped to v11.
Updated runtime enable but missed runtime disable check update.
Signed-off-by: Jonathan Kim <jonathan.kim@amd.com>
Change-Id: I71a8970ccfe7dc517abe7b4ad962369aea6a0496
To improve performance on queue preemption, allocate ctx s/r
area in VRAM instead of system memory, and migrate it back
to system memory when VRAM is full.
Signed-off-by: Eric Huang <jinhuieric.huang@amd.com>
Change-Id: If775782027188dbe84b6868260e429373675434c
The KFD no longer allow debug ops that modify HW state prior to
trap activation so permit bump in major version.
Signed-off-by: Jonathan Kim <jonathan.kim@amd.com>
Change-Id: I072d3998b7b043df9a67f0f6762b0afdfa9382c6
Non-paged allocation for queue memory necessary for binding wptr to
GART. Required to support usermode queue oversubscription with MES on
GFX11.
Change ensures queue memory does not specify ATS.
Signed-off-by: Graham Sider <Graham.Sider@amd.com>
Change-Id: I10b23b0205c90dad902c711a88cfb5e9b4979617
Jesse Zhang
Graham Sider
Mukul Joshi
David Belanger
Alex Sierra
Felix Kuehling
David Yat Sin
Xiaogang Chen
Eric Huang
David Francis
Philip Yang
Aaron Liu
kent.russell@amd.com
Lang Yu
Jonathan Kim