1. Create P2P links
2. Determine FRAMEBUFFER_PUBLIC/PRIVATE only based
host-accessibility, not peer-accesssibility
Signed-off-by: Gang Ba <gaba@amd.com>
Change-Id: I15fccdc60386b453e2a47849a16df15157324b21
This reverts commit 8f26c0c40c.
Reason for revert: This commit caused a regression rocrtst memory
subtest: Maximum Single Allocation in Memory Pools failed.
Change-Id: I15330625603f893200a08cd8b5b097f9bf95361f
While the ternary is nice to read, strlen in general is an expensive
call, so call it once and check if the value is greater than our maximum
allowable string length and adjust accordingly
Signed-off-by: Kent Russell <kent.russell@amd.com>
Change-Id: Id744f2ba0eb81bb2b3c52eb69f38a615398a655d
Don't update the vm_object if GPU mapping failed. Print an error message
to help diagnose underlying problems.
Change-Id: I801ab6fe6c155bd25e6c0358007c106a4a019480
Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com>
Use MAP_POPULATE when allocating anonymous system memory for later
GPU mapping as a userptr. This can speed up large allocations by
more than factor 2. I suspect populating pages in this way is more
efficient than the CPU page fault code path triggered by
get_user_pages in the kernel.
Change-Id: I188bbc1462ccb650d48cbfb1080dbb8eb7ada8b5
Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com>
On gfx9, the maximum number of wavefronts per queue is the minimum of
40 waves per compute units, or 512 waves per shader engine. On gfx10,
there can only be 32 waves per compute units.
Signed-off-by: Laurent Morichetti <laurent.morichetti@amd.com>
Change-Id: I148d1a4fe6c07cdbfaa1f77939eb29311c81c008
Reserve some space in the context save area for the debugger's
use. There should be 32 bytes per wave for a given queue.
Change-Id: I65ddb6123d0f6afd3149844617ad19023009101d
The queue control stack size cannot exceed 0x7000 on ASICs
gfx1010 through gfx1031. The lower limit is not achievable
with AQL so this should have no practical effect.
Fixes control stack size overflow on large ASICs.
Change-Id: Ib78cf6e4c5f096044bf8de24debe211689891caa
Signed-off-by: Jay Cornwall <jay.cornwall@amd.com>
This patch is the hot fix to fix the param number checking after remove
dgpu input.
Signed-off-by: Huang Rui <ray.huang@amd.com>
Change-Id: Ic980588f78616f99076de742af580afb4273fb2f
gfx90c should use GFX902 which is the same with gfx902.
Signed-off-by: Huang Rui <ray.huang@amd.com>
Change-Id: Id24dc2c85c9f49f36b00889c3b8b1b19cce34e09
Whether use dgpu path will check the props which exposed from kernel.
We won't need hard code in the ASIC table.
Signed-off-by: Huang Rui <ray.huang@amd.com>
Change-Id: I0c018a26b219914a41197ff36dbec7a75945d452
KFD already implemented the fallback path for APU. Thunk will use flag
which exposed by kfd to configure is_dgpu instead of hardcode before.
Signed-off-by: Huang Rui <ray.huang@amd.com>
Change-Id: I445f6cf668f9484dd06cd9ae1bb3cfe7428ec7eb
Gfx10 need 12bytes/wave control stack
Change-Id: I6c6f2819572e6b43aa3140d4dbe79d930e4c1c9c
Signed-off-by: Oak Zeng <Oak.Zeng@amd.com>
Signed-off-by: Jay Cornwall <Jay.Cornwall@amd.com>
This is needed to avoid additional references to mapped BOs in child
processes that can prevent freeing memory in the parent process and lead
to out-of-memory conditions.
Change-Id: I25c90510a14dde515cc23ea5dc1f68e8d7e37a66
Signed-off-by: Philip Cox <Philip.Cox@amd.com>
strlen(src) should not be used as the length in strncpy. Use memcpy
since we know the length of the string, and ensure that we
NULL-terminate regardless of length
Signed-off-by: Kent Russell <kent.russell@amd.com>
Change-Id: I21cc6d106510c69464e7ac9d3fc7da3a1e6d1a68
The option to use kfd_fd for cpu mapping is for very old broken KFD
version, it is not used in upstreaming process. This causes issue when
multiple process uses shared system memory because the GTT address is
over 40 bits.
Change to always use render node fd to create CPU mapping.
Change-Id: Id7e7b2a2e2f13c6e62c5de170589abfff4d456b0
Signed-off-by: Philip Yang <Philip.Yang@amd.com>
This reverts commit d675d1cce1.
Reason for revert: Change was submitted by accident
Change-Id: If05c705e22296fd3ca789f269737d379a933361d
Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com>
Using static_assert breaks in "Many Linux" build environment. It is not
supported by that libc version. _Static_assert is a compiler built-in
and does not depend on the libc version.
Change-Id: I37cf0ad10de94d8f6fc8cefc4fdda55c9520d599
Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com>
New queue suspend/resume update can now return the number of successful
queue requests so return success if IOCTL return is non-negative.
This should be backwards compatible since old queue suspend/resume returns
0 on success.
Signed-off-by: Jonathan Kim <Jonathan.Kim@amd.com>
Change-Id: I06b70d95d203b2bfc19a0cc1b88c5719c695159a
This is needed to allow gdb to access the memory.
Change-Id: I96c084b714e952d7b7000f0dd41e1c530fdd092f
Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com>
Don't set kfd_open_count=1 unless hsaKmtOpenKFD actually succeeds.
This prevents returning HSAKMT_STATUS_KERNEL_ALREADY_OPENED in
subsequent calls when KFD is actually closed.
Signed-off-by: Sean Keely <Sean.Keely@amd.com>
Change-Id: Ia870b5faa8626826a6c8795aa10784d376cf2e80
libpci was only used to find a marketing string for a device.
This patch looks for a pci.ids on disk and parses it to extract the
same string, using 'Device xxxx' as the fallback on file i/o error
or missing data from the text file. Tested by checking every vendor/
device pair against the values returned from libpci.
Change-Id: I21af3157472c1824d57fcee31393c6ee8ce07330
Signed-off-by: Jon Chesterfield <Jonathan.Chesterfield@amd.com>
Read device unique id from sysfs and expose it in HsaNodeProperties.
For devices not supported the value will be 0
Signed-off-by: Divya Shikre <DivyaUday.Shikre@amd.com>
Change-Id: I97b8689dfa090971c6876de6feaa97652e28c03d
If child process explicitly calls hsaKmtReleaseSystemProperties(), it
fails. Allow child process to release and acquire system properties.
Change-Id: I649a4600212711b2ad4474f605f3ca39a4003d03
Signed-off-by: Harish Kasiviswanathan <Harish.Kasiviswanathan@amd.com>
IPC memory was previously returned as HSA_POINTER_ALLOCATED and
had garbage in the node_id field. Due to ROCR_VISIBLE_DEVICES
we need to be able to distinguish between imported memory and
regular memory because imported memory may not be owned by an
agent that is visible in the process. Differentiating these flags
allows the users to expect null agent for the owning agent.
Fixes
Change-Id: Ide3489cec1ee2072dc9697fa5cb71ddb17999d14
From Thunk spec, flag NoSubstitute = 0, if specific memory type is not
available on node, allocation may fall back to other memory that can
replace it on that node. mbind return failure if no memory available on
the specific node, we should ignore the mbind failure for this case.
Change-Id: I651a1bedf1852330604e56965cc17862403ebf87
Signed-off-by: Philip Yang <Philip.Yang@amd.com>
Check for errno == EBADF in kmtIoctl to detect misuse of the kfd_fd
in a forked child process.
Detect being in a forked child process pro-actively by implementing
a pthread_atfork callback.
Make sure all mutexes get reinitialized in the child process to avoid
deadlocks.
Check for being in a forked child process in CHECK_KFD_OPENED so that
all hsaKmt functions will return the appropriate status
HSAKMT_STATUS_KERNEL_IO_CHANNEL_NOT_OPENED.
Update InvalidKFDHandleTest to expect that error code.
Change-Id: I0238e5fba344dcaa454e97a35db2e2dcc8d1f607
Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com>
NumCpQueues and NumSdmaQueuesPerEngine should be got by kfd driver not hardcode.
So add two data fields in HsaNodeProperties then thunk is able to get it from sysfs
that exposed by kfd.
v2: change NumCpQueues/NumSdmaQueuesPerEngine to one byte.
v3: merge two commits as one to avoid ABI update two times.
Change-Id: Ie386e4685f13493e22db6e207a399db6a4c5b9dc
Signed-off-by: Huang Rui <ray.huang@amd.com>
adds api and test to get newly create queue snapshot per ptraced process.
Change-Id: Ife97123a5b930e837ccaa386801145ef23c2cc2c
Signed-off-by: Jonathan Kim <Jonathan.Kim@amd.com>
This patch adds the support for gfx90c apu. So far we treat it as "dgpu" and
gfx900. Will update hsa gfxip table while the isa/llvm is implemented on gfx90c.
Change-Id: I6ef164bf3e751fe6dd6287cac212a500dce84b1a
Signed-off-by: Huang Rui <ray.huang@amd.com>
On NUMA system, allocate queue ctx save restore area on the closest NUMA
node to the GPU which the queue is going to run. This will improve
performance on NUMA system generally by reducing schedule latency and
fix the multi-node rccl-tests unstable performance issue.
If the closest NUMA node has no memory available, set flags NoNUMABind=1
to bypass mbind, to use default NUMA memory policy to allocate system
memory.
Change-Id: Ic62bfa5bb2efbf4f6ae79ff403e9610ddf18d45c
Signed-off-by: Philip Yang <Philip.Yang@amd.com>
This patch adds the non-privileged PMC blocks for GFX10/gfx1010.
Change-Id: I4b98cb2159d71113c12920ca7fd10e45096b4e2c
Signed-off-by: Ori Messinger <Ori.Messinger@amd.com>
FREE_MEMORY_OF_GPU ioctl could fail, e.g., if memory is still mapped
to GPU. Handle this failure by return error in fmm_release/HsaKmtFreeMemory
Change-Id: I5461db39964f733cf97376d50e44906a9b4c0f13
Signed-off-by: Oak Zeng <Oak.Zeng@amd.com>
Huang Rui
Gang Ba
Tianci.Yin
Felix Kuehling
Kent Russell
Laurent Morichetti
Chengming Gui
Jay Cornwall
Harish Kasiviswanathan
Oak Zeng
Philip Cox
Philip Yang
Yong Zhao
Ranieri Althoff
Jonathan Kim
Srinivasan Subramanian
Sean Keely
Jon Chesterfield
Divya Shikre
Ori Messinger