Before gfx90a, coherent memory is uncached. So it was reasonable
when environment variable HSA_DISABLE_CACHE is set, memory is mapped
as coherent. On gfx90a, coherent memory can be cached, so mapping
memory as coherent can't guarantee memory is uncached. When
HSA_DISABLE_CACHE is set, we have to map memory as uncached.
Change-Id: Ia5ed4cf0ad6aef5644dc8c9e6632b52d606f06f4
Signed-off-by: Oak Zeng <Oak.Zeng@amd.com>
[ROCm/ROCR-Runtime commit: f132fb2cd0]
This change might be redundant if ROCr takes care of it
Signed-off-by: Harish Kasiviswanathan <Harish.Kasiviswanathan@amd.com>
Change-Id: I7b67143a8ad21baa61b7eda7b8e5fe0ac1e33830
[ROCm/ROCR-Runtime commit: 10674916e4]
This change is for the A+A bring-up branch as it needs to made more
generic to handle all ASICs.
For A+A all the system buffers are mapped as NC (non coherent) unless
explicitly marked as UC (uncached). The coherency is then expected to be
handled by shader by explicitly using acquire/release instructions.
However, CP doesn't have same feature. The buffers used by CP thus have
to UC. For now queue buffer and Signal handler memory is marked as UC.
This change shouldn't affect other ASICs since Uncached flag is not used
in those. However, this change still need to be made more generic.
Signed-off-by: Harish Kasiviswanathan <Harish.Kasiviswanathan@amd.com>
Change-Id: I56c37a809913f7f08c94d01b0572d0f4864939aa
[ROCm/ROCR-Runtime commit: 7c05c5240f]
On gfx9, the maximum number of wavefronts per queue is the minimum of
40 waves per compute units, or 512 waves per shader engine. On gfx10,
there can only be 32 waves per compute units.
Signed-off-by: Laurent Morichetti <laurent.morichetti@amd.com>
Change-Id: I148d1a4fe6c07cdbfaa1f77939eb29311c81c008
[ROCm/ROCR-Runtime commit: 4cf11c3a7e]
Reserve some space in the context save area for the debugger's
use. There should be 32 bytes per wave for a given queue.
Change-Id: I65ddb6123d0f6afd3149844617ad19023009101d
[ROCm/ROCR-Runtime commit: a83f9b67ce]
XNACK API for GPUs that support this mode. This API
makes calls to amdgpu driver to configure xnack mode.
It supports set xnack mode and query the current mode used.
Change-Id: If865fd0e3f900f008243dc49504e1a0694e1791a
Signed-off-by: Alex Sierra <alex.sierra@amd.com>
[ROCm/ROCR-Runtime commit: 3f45f602d4]
strlen(src) should not be used as the length in strncpy. Use memcpy
since we know the length of the string, and ensure that we
NULL-terminate regardless of length
Signed-off-by: Kent Russell <kent.russell@amd.com>
Change-Id: I21cc6d106510c69464e7ac9d3fc7da3a1e6d1a68
[ROCm/ROCR-Runtime commit: 731a06c704]
It is to provide an option to map specific memory as
uncached on A+A HW platform.
Signed-off-by: Eric Huang <JinhuiEric.Huang@amd.com>
Change-Id: Ib665cb306a0e78aba3ea5ee2f0e46cb62ae139f8
[ROCm/ROCR-Runtime commit: 2464bfc714]
This reverts commit c0a0ada18b.
Reason for revert: Change was submitted by accident
Change-Id: If05c705e22296fd3ca789f269737d379a933361d
Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com>
[ROCm/ROCR-Runtime commit: fec3780c1a]
Signed-off-by: Gang Ba <gaba@amd.com>
Change-Id: I5c23a8dacf9bc50c740908aabe391432f2c7112e
Signed-off-by: Gang Ba <gaba@amd.com>
[ROCm/ROCR-Runtime commit: d675d1cce1]
Using static_assert breaks in "Many Linux" build environment. It is not
supported by that libc version. _Static_assert is a compiler built-in
and does not depend on the libc version.
Change-Id: I37cf0ad10de94d8f6fc8cefc4fdda55c9520d599
Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com>
[ROCm/ROCR-Runtime commit: df16950a0c]
New queue suspend/resume update can now return the number of successful
queue requests so return success if IOCTL return is non-negative.
This should be backwards compatible since old queue suspend/resume returns
0 on success.
Signed-off-by: Jonathan Kim <Jonathan.Kim@amd.com>
Change-Id: I06b70d95d203b2bfc19a0cc1b88c5719c695159a
[ROCm/ROCR-Runtime commit: 93c333711a]
This is needed to allow gdb to access the memory.
Change-Id: I96c084b714e952d7b7000f0dd41e1c530fdd092f
Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com>
[ROCm/ROCR-Runtime commit: e5062c4383]
Don't set kfd_open_count=1 unless hsaKmtOpenKFD actually succeeds.
This prevents returning HSAKMT_STATUS_KERNEL_ALREADY_OPENED in
subsequent calls when KFD is actually closed.
Signed-off-by: Sean Keely <Sean.Keely@amd.com>
Change-Id: Ia870b5faa8626826a6c8795aa10784d376cf2e80
[ROCm/ROCR-Runtime commit: 884fed4f04]
libpci was only used to find a marketing string for a device.
This patch looks for a pci.ids on disk and parses it to extract the
same string, using 'Device xxxx' as the fallback on file i/o error
or missing data from the text file. Tested by checking every vendor/
device pair against the values returned from libpci.
Change-Id: I21af3157472c1824d57fcee31393c6ee8ce07330
Signed-off-by: Jon Chesterfield <Jonathan.Chesterfield@amd.com>
[ROCm/ROCR-Runtime commit: 0a1718b753]
Read device unique id from sysfs and expose it in HsaNodeProperties.
For devices not supported the value will be 0
Signed-off-by: Divya Shikre <DivyaUday.Shikre@amd.com>
Change-Id: I97b8689dfa090971c6876de6feaa97652e28c03d
[ROCm/ROCR-Runtime commit: ebe7de1f99]
If child process explicitly calls hsaKmtReleaseSystemProperties(), it
fails. Allow child process to release and acquire system properties.
Change-Id: I649a4600212711b2ad4474f605f3ca39a4003d03
Signed-off-by: Harish Kasiviswanathan <Harish.Kasiviswanathan@amd.com>
[ROCm/ROCR-Runtime commit: 31530da7c6]
IPC memory was previously returned as HSA_POINTER_ALLOCATED and
had garbage in the node_id field. Due to ROCR_VISIBLE_DEVICES
we need to be able to distinguish between imported memory and
regular memory because imported memory may not be owned by an
agent that is visible in the process. Differentiating these flags
allows the users to expect null agent for the owning agent.
Fixes
Change-Id: Ide3489cec1ee2072dc9697fa5cb71ddb17999d14
[ROCm/ROCR-Runtime commit: 6957202df8]
From Thunk spec, flag NoSubstitute = 0, if specific memory type is not
available on node, allocation may fall back to other memory that can
replace it on that node. mbind return failure if no memory available on
the specific node, we should ignore the mbind failure for this case.
Change-Id: I651a1bedf1852330604e56965cc17862403ebf87
Signed-off-by: Philip Yang <Philip.Yang@amd.com>
[ROCm/ROCR-Runtime commit: 5858aa17a9]
Check for errno == EBADF in kmtIoctl to detect misuse of the kfd_fd
in a forked child process.
Detect being in a forked child process pro-actively by implementing
a pthread_atfork callback.
Make sure all mutexes get reinitialized in the child process to avoid
deadlocks.
Check for being in a forked child process in CHECK_KFD_OPENED so that
all hsaKmt functions will return the appropriate status
HSAKMT_STATUS_KERNEL_IO_CHANNEL_NOT_OPENED.
Update InvalidKFDHandleTest to expect that error code.
Change-Id: I0238e5fba344dcaa454e97a35db2e2dcc8d1f607
Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com>
[ROCm/ROCR-Runtime commit: 87e10cd0b4]
NumCpQueues and NumSdmaQueuesPerEngine should be got by kfd driver not hardcode.
So add two data fields in HsaNodeProperties then thunk is able to get it from sysfs
that exposed by kfd.
v2: change NumCpQueues/NumSdmaQueuesPerEngine to one byte.
v3: merge two commits as one to avoid ABI update two times.
Change-Id: Ie386e4685f13493e22db6e207a399db6a4c5b9dc
Signed-off-by: Huang Rui <ray.huang@amd.com>
[ROCm/ROCR-Runtime commit: 06464b917d]
adds api and test to get newly create queue snapshot per ptraced process.
Change-Id: Ife97123a5b930e837ccaa386801145ef23c2cc2c
Signed-off-by: Jonathan Kim <Jonathan.Kim@amd.com>
[ROCm/ROCR-Runtime commit: 8b01a1c4c5]
This patch adds the support for gfx90c apu. So far we treat it as "dgpu" and
gfx900. Will update hsa gfxip table while the isa/llvm is implemented on gfx90c.
Change-Id: I6ef164bf3e751fe6dd6287cac212a500dce84b1a
Signed-off-by: Huang Rui <ray.huang@amd.com>
[ROCm/ROCR-Runtime commit: fdba74c2fb]
On NUMA system, allocate queue ctx save restore area on the closest NUMA
node to the GPU which the queue is going to run. This will improve
performance on NUMA system generally by reducing schedule latency and
fix the multi-node rccl-tests unstable performance issue.
If the closest NUMA node has no memory available, set flags NoNUMABind=1
to bypass mbind, to use default NUMA memory policy to allocate system
memory.
Change-Id: Ic62bfa5bb2efbf4f6ae79ff403e9610ddf18d45c
Signed-off-by: Philip Yang <Philip.Yang@amd.com>
[ROCm/ROCR-Runtime commit: 59c857476f]
This patch adds the non-privileged PMC blocks for GFX10/gfx1010.
Change-Id: I4b98cb2159d71113c12920ca7fd10e45096b4e2c
Signed-off-by: Ori Messinger <Ori.Messinger@amd.com>
[ROCm/ROCR-Runtime commit: e7f45fae8a]
FREE_MEMORY_OF_GPU ioctl could fail, e.g., if memory is still mapped
to GPU. Handle this failure by return error in fmm_release/HsaKmtFreeMemory
Change-Id: I5461db39964f733cf97376d50e44906a9b4c0f13
Signed-off-by: Oak Zeng <Oak.Zeng@amd.com>
[ROCm/ROCR-Runtime commit: fa0cb9ebeb]
Now the gfx1012 device IDs are okay to reveal.
Change-Id: I9da2a036b74ec7b6b8b1fb7587597a5847f02205
Signed-off-by: Yong Zhao <Yong.Zhao@amd.com>
[ROCm/ROCR-Runtime commit: 1c7755d2da]
Without this change, the failure was hard to notice when it happened.
Change-Id: I99c3e8cea0d0cbd3bcfe79069410e6e870e225bf
Signed-off-by: Yong Zhao <Yong.Zhao@amd.com>
[ROCm/ROCR-Runtime commit: 16fa78b134]
When CONFIG_NUMA is not enabled in the kernel config, only one CPU node
presents on the system and /sys/devices/system/node/nodeX directories
don't exist. Read CPU cache information from /sys/devices/system/cpu in
this situation.
Change-Id: I017ff17dd72678a0551edcc77446664501aa42ca
Signed-off-by: Amber Lin <Amber.Lin@amd.com>
[ROCm/ROCR-Runtime commit: 23541e0289]
The debug trap accesses the data0/data1 registers, so we do not
want the userspace to write values to it. We remove the calls to
set the data0/data1 register values.
Change-Id: Iaba842a4c445f339f16a39fe1994526ff78a2f3c
Signed-off-by: Philip Cox <Philip.Cox@amd.com>
[ROCm/ROCR-Runtime commit: 6933540c81]
To support adding new features to the kfd debugger, and not break
functionality, we need to be able to check the kfd debugger support
version info from the kernel.
Change-Id: Icd88e4edab8430c35eaed588e62d892c1b5c62ec
Signed-off-by: Philip Cox <Philip.Cox@amd.com>
[ROCm/ROCR-Runtime commit: dbbd189b33]
When fail to get CPU dirs from //sys/devices/system/node/nodeX directory,
the error message should print node_dir, not path.
Change-Id: If76a51918c8dd55fa6605a62f3d29f9efc6fadb3
Signed-off-by: Amber Lin <Amber.Lin@amd.com>
[ROCm/ROCR-Runtime commit: 5a09880620]
numa_max_node() return the highest node number available on the current
system, number of NUMA nodes should be numa_max_node() + 1.
Change-Id: I20a6c17af071e73e853cb5ea6d0304c8aca52681
Signed-off-by: Philip Yang <Philip.Yang@amd.com>
[ROCm/ROCR-Runtime commit: 71cf3cf5d3]
Oak Zeng
Harish Kasiviswanathan
Laurent Morichetti
Alex Sierra
Kent Russell
Eric Huang
Jonathan Kim
Amber Lin
Yong Zhao
Gang Ba
Philip.Cox@amd.com
Felix Kuehling
Ranieri Althoff
Srinivasan Subramanian
Sean Keely
Jon Chesterfield
Divya Shikre
Philip Yang
Huang Rui
Ori Messinger
shaoyunl