When is hsaKmtCreateQueue called first time for node
doorbells[NodeId].size is initialized to zero in init_process_doorbells
but used to calculate the doorbell offset. It works just by accident
because doorbells[NodeId].size is uint32_t so -1 will be 0xFFFFFFFF which
is zero extended into 0x00000000FFFFFFFF and it will work as long as mmap
offset bits are not within lower 32 bits.
Bug: https://github.com/RadeonOpenCompute/ROCT-Thunk-Interface/issues/78
Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com>
Change-Id: Ia791adfc51363d4704cb50fa4f01137b7dd48a75
[ROCm/ROCR-Runtime commit: 8e69b9c70e]
When hsa is closed, it would close open fds for /dev/kfd but
not for /dev/dri/renderD*. This caused issues with CRIU
checkpoint, which expects that /dev/kfd will be open if
/dev/dri/renderD* is.
As a workaround for the CRIU behaviour, leave /dev/kfd open
when closing hsa.
Signed-off-by: David Francis <David.Francis@amd.com>
Change-Id: Ie1b2d5b1d8986750b0e560ae2934b7c73cff942e
[ROCm/ROCR-Runtime commit: 88934cec2c]
Get more debug information about user pointers that were registered
through SVM API, and triggered by memory exception events.
A new kfdtest with this use case was also included inside
KFDExceptionTest.
Signed-off-by: Alex Sierra <alex.sierra@amd.com>
Change-Id: I8e9df3c1c6c3f42d7b9235d12406d80d31746443
[ROCm/ROCR-Runtime commit: 45fad29752]
Register and map userptrs through Shared Virtual Memory(SVM) API at
the Kernel level when available. Using this approach, performance
will be improve as register/unregister memory will not trigger any
system call to KFD driver.
Signed-off-by: Alex Sierra <alex.sierra@amd.com>
Change-Id: I20723cbeb340bf48b95e1115f0102c031397bc14
[ROCm/ROCR-Runtime commit: 178a619b80]
Prior to launch some ASICs may re-use PCI DIDs from older generations.
This can cause issues during topology initialization as hsa_gfxip_table
lookups will override sysfs-provided gfx versions, causing incorrect
gfxip selection. Since no new entries will be added to hsa_gfxip_table,
limit its search only to pre-GFX11 ASICs.
Signed-off-by: Graham Sider <Graham.Sider@amd.com>
Change-Id: I53eaefac5db2650a36a6ce9f21daf750f50cfd26
[ROCm/ROCR-Runtime commit: 79279e860f]
fork process copy-on-write MMU nitifier on CWSR range will evict user
queues, and then update GPU mapping and resume queues, use MADV_DONTFORK
to avoid COW MMU notifier callback on CWSR SVM range.
Use mmap to alloc SVM range for CWSR because posix_memalign don't alloc
new range in child process, this fails to register svm range as range is
invalid address in forked child process.
Change-Id: Ibaea56a691dd6f577ed2e1f2d43f4a3500b8316f
Signed-off-by: Philip Yang <Philip.Yang@amd.com>
[ROCm/ROCR-Runtime commit: 093cf898fb]
mmap alloc larger address range with align padding page plus guard
pages, then unmap the padding and guard pages at beginning and end
of the range, return aligned address range.
Change-Id: Iaf3c711a079c744289efbafee9b5e63aaf724765
Signed-off-by: Philip Yang <Philip.Yang@amd.com>
[ROCm/ROCR-Runtime commit: b7710a1dda]
GFX1036(ISA version) is not included in the previous range.
This patch can really include all gfx10 series ASICs.
Signed-off-by: Aaron Liu <aaron.liu@amd.com>
Change-Id: I0e28dbfc031c216166b306b9fb39f644f75a330f
[ROCm/ROCR-Runtime commit: 06a90612e9]
Avoiding the segfault, runtime debugger enable is not supported
if the firmware of gpu doesn't support debug exceptions.
Signed-off-by: jie1zhan <jesse.zhang@amd.com>
Change-Id: Ifad57a6e78cb1c92b1f8927355ece8c64e89c51b
[ROCm/ROCR-Runtime commit: d98c729ff9]
Remove potential double free condition when free_queue() is called
after hsaKmtDestroyQueue() if mapping doorbell fails during queue
creation.
Signed-off-by: Mukul Joshi <mukul.joshi@amd.com>
Change-Id: If2aa19c455b30d2940b232dbafb9cc1eaad721a5
[ROCm/ROCR-Runtime commit: 57a1c6f3ff]
Queue ctx_save_restore memory is allocated with size
ctx_save_restore_size + debug_memory_size, use the same size
in free_queue to free ctx_save_restore memory.
Change-Id: I4902ff15fb82ddea64b8342b89776a1bf5c38d13
Signed-off-by: Philip Yang <Philip.Yang@amd.com>
[ROCm/ROCR-Runtime commit: 3dbf5feffe]
Avoiding segfault when an invalid SharedMemoryHandle is passed in
when calling fmm_register_shared_memory.
Change-Id: I0e0bbed01487fc10afcbb170eb9330e70b209d14
Signed-off-by: David Yat Sin <David.YatSin@amd.com>
[ROCm/ROCR-Runtime commit: 1c385fb257]
Now that HsaNodeProperties is passed in to
topology_get_node_props_from_drm, check that pointer instead of the
pointer for MarketingName (which throws a compiler warning)
Signed-off-by: kent.russell@amd.com <kent.russell@amd.com>
Change-Id: If76b24e1bab5a62e514ab440b6316c7b7cd264c1
[ROCm/ROCR-Runtime commit: ea4d4917c1]
Query family id info from drm render node, then
ROCr can query this info directly from Thunk
instead of parsing the info by itself.
Signed-off-by: Lang Yu <Lang.Yu@amd.com>
Change-Id: I030bd27ab2379fbf87f3d787302c3b8613456278
[ROCm/ROCR-Runtime commit: 66e9e97e0d]
The debugger can now request snapshot copies with entry size and
set/clear watchpoints by device.
v3: drop min version check to v10.0
v2: check runtime allowance from v10.3 to 13.x
Signed-off-by: Jonathan Kim <jonathan.kim@amd.com>
Change-Id: I9befefb596201a11591de218db29a9317b41e69b
[ROCm/ROCR-Runtime commit: c1d8ac8437]
The debugger depends on the CWSR area being executable. Set the right
flag when registering SVM memory.
Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com>
Tested-by: Laurent Morichetti <Laurent.Morichetti@amd.com>
Change-Id: I7441e214d1a4da8324d775e777976fabd1c81a6f
[ROCm/ROCR-Runtime commit: deb7a20c92]
The CMA feature is deprecated and about to be removed from the DKMS
branch. It was never supported upstream. Leave dummy functions in
place for now.
Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com>
Change-Id: I9e51403d753cb91630553aff4f19e931af509740
[ROCm/ROCR-Runtime commit: 9b2b81e555]
hsaKmtMapMemoryToGPU should not try to map VRAM on peer GPUs that don't
have an IO-Link to the memory. The new P2P mapping code in KFD will
fail otherwise.
Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com>
Change-Id: I6d59b55651b98756865a0f69eafef3e386372cf3
[ROCm/ROCR-Runtime commit: 9ac2c75171]
This allows init_process_apertures to use the whole consistent topoology
instead of taking its own partial snapshot.
Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com>
Change-Id: Ia13e7aa7fcd090ea8d6cacd4babb29a27c20207f
[ROCm/ROCR-Runtime commit: 87aca673e8]
To improve performance on queue preemption, allocate ctx s/r
area in VRAM instead of system memory, and migrate it back
to system memory when VRAM is full.
Signed-off-by: Eric Huang <jinhuieric.huang@amd.com>
Change-Id: If775782027188dbe84b6868260e429373675434c
[ROCm/ROCR-Runtime commit: 37be876cad]
The KFD no longer allow debug ops that modify HW state prior to
trap activation so permit bump in major version.
Signed-off-by: Jonathan Kim <jonathan.kim@amd.com>
Change-Id: I072d3998b7b043df9a67f0f6762b0afdfa9382c6
[ROCm/ROCR-Runtime commit: 79cd63fab6]
Non-paged allocation for queue memory necessary for binding wptr to
GART. Required to support usermode queue oversubscription with MES on
GFX11.
Change ensures queue memory does not specify ATS.
Signed-off-by: Graham Sider <Graham.Sider@amd.com>
Change-Id: I10b23b0205c90dad902c711a88cfb5e9b4979617
[ROCm/ROCR-Runtime commit: e17b159230]
System Management Interface event is read from anonymous file handle,
this helper wrap the ioctl interface to get anonymous file handle for
GPU nodeid.
Define SMI event IDs, event triggers, copy the same value from
kfd_ioctl.h to avoid translation.
Change-Id: I5c8ba5301473bb3b80bb4e2aa33a9f675bedb001
Signed-off-by: Philip Yang <Philip.Yang@amd.com>
[ROCm/ROCR-Runtime commit: 405fbd6f93]
Add KFDGpuID to HsaNodeProperties to return gpu_id to upper layer,
gpu_id is hash ID generated by KFD to distinguish GPUs on the system.
ROCr and ROCProfiler will use gpu_id to analyze SMI event message.
Change-Id: I6eabe6849230e04120674f5bc55e6ea254a532d6
Signed-off-by: Philip Yang <Philip.Yang@amd.com>
[ROCm/ROCR-Runtime commit: 88778e13dc]
This env variable sets the max VA alignment order size as
"PAGE_SIZE * 2^alignment order" during mapping. By default the order
size is set to 9(2MB).
Signed-off-by: Alex Sierra <alex.sierra@amd.com>
Change-Id: I01ae4e0963f4d21c7c367464e60f865bc58d7fac
[ROCm/ROCR-Runtime commit: 0e908f05bb]
New API function to report available memory per GPU
Signed-off-by: Daniel Phillips <Daniel.Phillips@amd.com>
Change-Id: I63c1e4ca0020c657977ab3635947ab0ed0a81440
[ROCm/ROCR-Runtime commit: 6da6058d4a]
Tested on Talos II with Vega 64
POWER systems allocate NUMA nodes on multiples of 8 to allow CPU
onlining / offlining
Set the correct NUMA mask bits when requesting node-bound memory
allocations
This is a cleanup/squash/rebase of:
https://github.com/RadeonOpenCompute/ROCT-Thunk-Interface/pull/47
Change-Id: Id4af6dff7e66e9d464d6b17a1e99087eb3ac8e51
Signed-off-by: Jeremy Newton <Jeremy.Newton@amd.com>
[ROCm/ROCR-Runtime commit: 5fd3c868b2]
Import the latest version from the kernel tree.
Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com>
Change-Id: If5f998ad55085ebd5020adaa382181204d834e3e
[ROCm/ROCR-Runtime commit: f88aaa933b]
Currently, context save area size passed to KFD includes the
size of the debug area. Change this to report the actual size
of the context save area to KFD.
Signed-off-by: Mukul Joshi <mukul.joshi@amd.com>
Change-Id: I5d440ae802255a97ade046775f6a000bae79d5d5
[ROCm/ROCR-Runtime commit: b8dc875b3c]
Add PCI DID for cyan skillfish.
Signed-off-by: Lang Yu <Lang.Yu@amd.com>
Change-Id: I1d06936cccdf99af76fe5ca3ff323538fac76c9c
[ROCm/ROCR-Runtime commit: 052b7957ea]
The gfx version of gfx90c is 90C instead of 902.
Signed-off-by: Aaron Liu <aaron.liu@amd.com>
Change-Id: Id009c9357f816b8ccab605090df47626f1a579ef
[ROCm/ROCR-Runtime commit: 7cdf38f6c0]
Mapped memory areas become invalid after fork, and the child process is
required to remap the memory areas after a fork. So we mark these device
memory mappings with MADV_DONTFORK so that they are removed from the
child process after fork.
This was causing some issues when doing CRIU checkpoint/restore because
CRIU and amdgpu_plugin were not able to handle these mappings.
Change-Id: I50eb334aecea6dab7522d94da0273adcf4fb1ce0
Signed-off-by: David Yat Sin <david.yatsin@amd.com>
[ROCm/ROCR-Runtime commit: 4986f4a5c2]
The AMD compiler team has confirmed that they expect gfx90c
to be gfx90c, with a major/minor/stepping of 9, 0, and 12
respectively. It appears that there is a typo in the libhsakmt
topology information that lists this part as gfx902. This patch
fixes the issue.
Signed-off-by: Joseph Greathouse <Joseph.Greathouse@amd.com>
Change-Id: I6f907a7aa6f190b12aba8bb4210c7b341b3c720b
[ROCm/ROCR-Runtime commit: a06d1a3884]
The return code is just -1 if any error occurs. To detect debugger
unavailable we need to check the actual ioctl error code.
Change-Id: I8a294c754196aec916809497ec8e810da2f072b8
Signed-off-by: Sean Keely <Sean.Keely@amd.com>
[ROCm/ROCR-Runtime commit: b67bc6fcc9]
For userptr, after taking aperture lock, decrease registration_count and
ensure object registration_count equal to 0 to release KFD and thunk
object.
Move decrementing of registration count from fmm_deregister_memory into
__fmm_release to avoid a race condition when dropping the aperture lock
in fmm_deregister_memory.
Change-Id: I5381fa6b8a77a1516af2554e5174e91969c338c4
Signed-off-by: Philip Yang <Philip.Yang@amd.com>
[ROCm/ROCR-Runtime commit: f82927ad65]
to faciliate debugging, print errno when queue destroy fail
current log give very little information when fail:
[ RUN ] KFDQMTest.AllSdmaQueues
[ ] Regular SDMA engines number: 1 SDMA queues per engine: 2
[ OK ] KFDQMTest.AllSdmaQueues (11 ms)
[ RUN ] KFDQMTest.AllXgmiSdmaQueues
[ ] XGMI SDMA engines number: 0 SDMA queues per engine: 2
[ OK ] KFDQMTest.AllXgmiSdmaQueues (6 ms)
[ RUN ] KFDQMTest.AllQueues
/home/foreman/build/hsakmt-roct-amdgpu-1.0.9.40500/sources/libhsakmt/tests/kfdtest/src/KFDQMTest.cpp:381: Failure
Value of: (cpQueues[i].Destroy())
Actual: 1
Expected: HSAKMT_STATUS_SUCCESS
Which is: 0
Signed-off-by: Yifan Zhang <yifan1.zhang@amd.com>
Change-Id: I5b1b5b616a5fd7ff198360c893a7aeed685022bd
[ROCm/ROCR-Runtime commit: db1d6d92ab]
After calling ioctl to create userptr obj, take aperture lock, check if
there is same userptr obj created after finding object, to catch the
race that multiple threads register same userptr to multiple GPUs.
If same userptr obj exist, then increase userptr registeration_count,
and free the newly create obj.
Change-Id: I63ae3a4f54da8aedd11c124d8d53ebe727b8203a
Signed-off-by: Philip Yang <Philip.Yang@amd.com>
[ROCm/ROCR-Runtime commit: 8f7443494a]
This reverts commit f94be2078c.
Change-Id: I954326d9a291280de7c9b7ef49a8cce0cd9dd9ad
Signed-off-by: Philip Yang <Philip.Yang@amd.com>
[ROCm/ROCR-Runtime commit: a809a5bf32]
This reverts commit 559bb50c6a.
Change-Id: Ibd32ac36bcb9ee418957d25b240ef9dd9172fbd3
Signed-off-by: Philip Yang <Philip.Yang@amd.com>
[ROCm/ROCR-Runtime commit: 91b0fce535]
This reverts commit 14d821391e.
Change-Id: Idf636655ac01f569d8e92e9c0e100e5db7d1cb61
Signed-off-by: Philip Yang <Philip.Yang@amd.com>
[ROCm/ROCR-Runtime commit: 445fcd803b]
This reverts commit 8f735fb9d5.
Reason for revert: aqlprofile has merged
Change-Id: Iee0fbe8ac1e475123273b328949f68f10490d3ca
[ROCm/ROCR-Runtime commit: 268bb1ae84]
g_props is intialized in topology_take_snapshot which needs to call
hsaKmtAcquireSystemProperties. In hsaKmtOpenKFD, it doesn't call hsaKmtAcquireSystemProperties.
So it needs to change parameter of topology_is_svm_needed from node_id
to EngineId to avoid Segmentation fault on gfx902.
Signed-off-by: changzhu <Changfeng.Zhu@amd.com>
Change-Id: Iba8d20a9142510c70927454d26bdcaf579ad5574
[ROCm/ROCR-Runtime commit: 978028d3f6]
Use the database provided by libdrm-amdgpu,as this is maintained by AMD
Signed-off-by: Mike Li <Tianxinmike.Li@amd.com>
Change-Id: I319a46d833cb23173a77fdff1deae69ea58588b8
[ROCm/ROCR-Runtime commit: cb1c84d99b]
Felix Kuehling
David Yat Sin
David Francis
Alex Sierra
Graham Sider
Philip Yang
Aaron Liu
jie1zhan
Mukul Joshi
kent.russell@amd.com
Lang Yu
Jonathan Kim
Eric Huang
Daniel Phillips
Timothy Pearson
Joseph Greathouse
Sean Keely
Yifan Zhang
changzhu
Chengming Gui
Jinzhou Su
Mike Li