When fail to get CPU dirs from //sys/devices/system/node/nodeX directory,
the error message should print node_dir, not path.
Change-Id: If76a51918c8dd55fa6605a62f3d29f9efc6fadb3
Signed-off-by: Amber Lin <Amber.Lin@amd.com>
[ROCm/ROCR-Runtime commit: 5a09880620]
numa_max_node() return the highest node number available on the current
system, number of NUMA nodes should be numa_max_node() + 1.
Change-Id: I20a6c17af071e73e853cb5ea6d0304c8aca52681
Signed-off-by: Philip Yang <Philip.Yang@amd.com>
[ROCm/ROCR-Runtime commit: 71cf3cf5d3]
on NUMA system, node 0 may have no memory, application pass node id
0 to hsaKmtAllocMemory will fail because mbind to specify the allocation
from node 0 return EINVAL.
Add new flag NoNUMABind for application to pass it to hsaKmtAllocMemory
to skip mbind.
hsaKmtCreateEvent and hsaKmtCreateQueue specify the new flag NoNUMABind
to allocate system memory for event page and CWSR area, don't bind the
system memory to a specific NUMA node.
Change-Id: I854e5a57502c7807c4c5ff2e441d499ae515c309
Signed-off-by: Philip Yang <Philip.Yang@amd.com>
[ROCm/ROCR-Runtime commit: 42392f093f]
Docker seccomp by default blocks mbind system call, so mbind return
failed on docker. thunk should not fail this otherwise application
cannot allocate system memory on docker.
Use pr_warn_once and pr_err_once to avoid duplicate same error messages
Change-Id: I61a7c0e4abaa3dcfe7abf2ea48db90f669f9638a
Signed-off-by: Philip Yang <Philip.Yang@amd.com>
[ROCm/ROCR-Runtime commit: 4da09813a3]
The gfx version item is yet to be added.
Change-Id: Ia6c487447e5a5df80c0c12fe150939175068024b
Signed-off-by: Yong Zhao <Yong.Zhao@amd.com>
[ROCm/ROCR-Runtime commit: 3ecd83e52d]
Force all the GPUs to a certain type, use the below command:
HSA_FORCE_ASIC_TYPE="10.1.0 1 gfx1010 14"
meaning major.minor.step dgpu asic_name asic_id
This will faciliate the cooperation across the teams for bringing up
ASICs which reuse existing device IDs.
Change-Id: I40fe4c9b46d3ccb3e38ea52250e80e82fb50fb0f
Signed-off-by: Yong Zhao <Yong.Zhao@amd.com>
[ROCm/ROCR-Runtime commit: d6539ddc24]
The memory size alignment workaround for a TLB bug on gfx802 was
breaking userptrs because it would attempt to get_user_pages beyond
the end of a VMA. Refine this workaround based on our understanding
of the HW bug. It only affects L2 cacheline allocation, which is
decided by the last page in the cache line (8 entries = 32KB of
address space). Thus aligning memory allocation so that the last
page falls on the end of a 8 entry TLB cache line allows caching
to work correctly.
Imported images require specific alignments. If their size is not
naturally aligned with 8 cache lines, it may have bad TLB cache
performance.
This patch will only have the desired effect if redundant size
padding in KFD is also removed.
Change-Id: I984cbe7fa61fec04d70fa387aaf9aab370eabeb9
Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com>
[ROCm/ROCR-Runtime commit: e320913e9e]
snprintf throws a warning from -Wformat-truncation where the string
could be truncated. We address this by referencing the maximum size that
can be returned from a file according to MAXNAMLEN . This should safely
guard us from truncating the path value.
Change-Id: If1d208990d8775e9494835b0deb890d2616fd15b
Signed-off-by: Kent Russell <kent.russell@amd.com>
[ROCm/ROCR-Runtime commit: ccac07cb14]
Replace cpuid call with sysfs data to get CPU cache information. With this
change, x86 check is also removed since sysfs applies to other platforms.
CPU cache information can be retrieved from
/sys/devices/system/node/nodeX/cpuY/cache where Y is processor number
represented in /proc/cpuinfo at "processor" entry.
Change-Id: Ic47df6d5dafaf1aae5b46b1fdee42691c697e49e
Signed-off-by: Amber Lin <Amber.Lin@amd.com>
[ROCm/ROCR-Runtime commit: 4fa930af5a]
PCI domain has moved to 32-bits to accommodate virtualization,
so a 32-bit integer is exposed for domain to reflect this change.
Change-Id: I0d767acadcdc8e4277db203b5865dd67dd001cef
Signed-off-by: Ori Messinger <ori.messinger@amd.com>
[ROCm/ROCR-Runtime commit: f2173254e4]
enable thunk query api to report if queue is newly created
test new queue bit test on clear events.
also fixup cleanup to disable debug trap.
Change-Id: I3ebe2d85da66f28b8c82f0e68461ee7d32ec0b0d
Signed-off-by: Jonathan Kim <Jonathan.Kim@amd.com>
Reviewed-by: Philip Cox <Philip.Cox@amd.com>
Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com>
[ROCm/ROCR-Runtime commit: 1ff5cb33b2]
Originally reserved 100 SGPRs per wave. Pre-gfx10 needs 102 SGPRs
and gfx10 needs 128 SGPRs. Reserve 128 SGPRs per wave for all ASICs
to simplify calculation.
Also double VGPR register size for gfx908 family
Change-Id: I98b741cbfa051f49ed37ff25d99f851f124be7b6
Signed-off-by: Jay Cornwall <Jay.Cornwall@amd.com>
Signed-off-by: Oak Zeng <Oak.Zeng@amd.com>
[ROCm/ROCR-Runtime commit: 814e0f0bdc]
By using user-allocated pages instead of kernel-allocated pages
from TTM, we're not subject to TTM's self imposed limits on kernel
memory usage. This also paves the way for for more wide-spread use
of HMM on upstream kernels.
Change-Id: Iac82964c98a441e29b7f1986d1be1bb5ccb1e569
Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com>
[ROCm/ROCR-Runtime commit: 626957a263]
child process clone vm objects from svm->apertures if parent process
doesn't free memory before fork. fmm_clear_all_mem suppose to clear the
apertures in forked child process but this only works if gpu_vm is not
NULL. parent process call hsaKmtCloseKFD reset gpu_vm to NULL and then
fork, then child process will not clear svm->apertures.
As a result, the child process will allocate vm object with same address
and add to aperture, there are duplicate vm objects with same address
in aperture. Then mapping to GPU will find the wrong vm object and
create incorrect GPU mapping cause rocrtst IPC test VM fault. The issue
happened with HSA_USERPTR_FOR_PAGED_MEM=1.
The fix is to clear vm objects in all apertures in clear_after_fork.
Change-Id: I92e42a967075a634a3f475b915c8242d82077ecb
Signed-off-by: Philip Yang <Philip.Yang@amd.com>
[ROCm/ROCR-Runtime commit: a11cb2a633]
This reverts commit bb4291dcac.
This change causes KFDTest failed on gfx803. The first hsaKmtCreateEvent
call allocate system memory for events_page because global variable
events_page is NULL. And this events page vm address should not be freed
until the process exit.
The change to destrory objects in hsaKmtCloseKFD removes events page. As
a result, KFDTest call hsaKmtOpenKFD again and then allocate memory will
get same events_page vm address on gfx803, and map this vm failed because
the vm conflict with events_page mapping.
KFDTest passed on VG10, gfx906 because allocate memory get different vm
address. hsaKmtCreateEvent still works fine as the driver keeps the
events page mapping of the process.
We should only destroy objects in fork cloned child process regardless
if gpu_vm is NULL or not.
Change-Id: I174ef65321cbd6074c855c2021318fe961c8c72c
Signed-off-by: Philip Yang <Philip.Yang@amd.com>
[ROCm/ROCR-Runtime commit: 7fc6a9f7c2]
/proc/cpuinfo are opened, read, and closed multiple times. Once for vendor
name and multiple times for model name -- each node opens once. For example
in a 2 CPUs + 4 GPUs system, it'll be opened 7 times. This patch reads it
one time and stores it in a cpuinfo buffer. This cpuinfo buffer is freed
when the snapshot is done.
Also replace returns with gotos inside the snapshot to avoid possible
memory leak.
Change-Id: Iaf26a6c7e7323a8651d137c3706179449b9e3c80
Signed-off-by: Amber Lin <Amber.Lin@amd.com>
[ROCm/ROCR-Runtime commit: 1fddfd316a]
Add data out for enable trap to return poll fd to user space.
Add query debug events interface.
Change-Id: Ia4afde1cf167e6aa61d502380a8b329ee89d5f44
Signed-off-by: Jonathan Kim <Jonathan.Kim@amd.com>
[ROCm/ROCR-Runtime commit: 836dfd0752]
Otherwise the parent call hsaKmtCloseKFD and then fork child process,
child process will duplicate the vm_objects from the parent.
Change-Id: Ia6ffc51cbae983b6a7cdc58ccf3b11ebe4087d97
Signed-off-by: Philip Yang <Philip.Yang@amd.com>
[ROCm/ROCR-Runtime commit: 632ad3a749]
The NodeId parameter is redundant and can be retrieved
from QueueId parameter.
Change-Id: I12853849b868b304bd27633fa7653ba644d69026
Signed-off-by: Oak Zeng <Oak.Zeng@amd.com>
[ROCm/ROCR-Runtime commit: f40f166e20]
Ignore requests to bind to invalid NUMA nodes. This affects only
legacy applications (such as KFDTest) that allocate system memory
as paged memory with a GPU node ID.
Change-Id: I81e514af6d0c1ab2ed5229adeeca1fa0ab2a0685
Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com>
[ROCm/ROCR-Runtime commit: 4d7b0990e4]
mbind walks through pages to setup vma memory policy. So we need do mmap
to create vma mappings first, then call mbind. mbind will do nothing if
vma does not exist.
And add numa available check before executing mbind, and return NULL to
hsaKmtAllocMemory if mbind failed.
Change-Id: I28ab661885d807ca51ef90e87230669dc80f10ec
Signed-off-by: Philip Yang <Philip.Yang@amd.com>
[ROCm/ROCR-Runtime commit: 67f366243d]
Remove the HSA_DEBUG environment variable that controlled the
creation of these mappings.
This should allow the debugger to attach to a running process and
access VRAM buffers through ptrace without having to do anything
special.
On processes that create many small VRAM mappings, this may cause
regressions due to the per-process mmap limit. However, the
sub-allocator in ROCr should consolidate most small allocations
into 2MB blocks nowadays, for good TLB efficiency. So this is
unlikely to cause problems.
Change-Id: I929da1be0f6cb51ec00a02f3f241d16083e4d95f
Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com>
[ROCm/ROCR-Runtime commit: 64b90261d9]
The queue IDs passed over to the kernel via kfd_ioctl_dbg_trap_args->ptr
should be a list of uint32_t's. Need to convert from the passed in
64 bit HSA_QUEUEID to 32 bit uint32_t's.
Change-Id: I8718566d9f9ffc90ce0b2ecc129b10c49d73186a
Signed-off-by: Philip Cox <Philip.Cox@amd.com>
[ROCm/ROCR-Runtime commit: 608bc7c3a0]
There was a mistake and RESUME was used when it should
have been suspend in two places in the suspend resume
code. This fixes that error.
Change-Id: I69be733d7ae7c14ce5ee8af57a307976e4212d62
[ROCm/ROCR-Runtime commit: b0d23aee16]
This is updating to the new suspend and resume API for the
KFD and the thunk. We now support passing in a list of queues
to suspend, and not just all of the queues for the process.
The kfdtest testcase was also updated so it still compiles.
Change-Id: I71d1b178476bd9df0c311bdedaa6a891528cebcf
Signed-off-by: Philip Cox <Philip.Cox@amd.com>
[ROCm/ROCR-Runtime commit: c2c1385e29]
hsaKmtGetQueueInfo needs to return the control stack size, and the
wave state size for the debugger. These changes are needed to support
returning the new values.
Change-Id: Ib4c60e0ea34446c06aef4a86996250989f348a69
Signed-off-by: Philip Cox <Philip.Cox@amd.com>
[ROCm/ROCR-Runtime commit: d21e9d5bbd]
Add a MMIO_REMAP heap type and expose mmio virtual address
through HsaKmtGetNodeMemoryProperties
Change-Id: I1e585e6dfbec8fa7c85f1dda7b89b763a8e2c439
Signed-off-by: Oak Zeng <Oak.Zeng@amd.com>
[ROCm/ROCR-Runtime commit: 804aa90a22]
HDP conherence registers are remapped at driver level
to an empty page in mmio space (the remapped mmio page).
This change allocate and map the remapped mmio page to
process space.
Change-Id: I89c405c41870a79c5b58eea0d8e564aa35f55182
Signed-off-by: Oak Zeng <Oak.Zeng@amd.com>
[ROCm/ROCR-Runtime commit: ae111689f0]
While this may not be supported in the runtime, the kernel/firmware
support it
Change-Id: I7fe4536a6b3055f39e25f453060e899938645d91
Signed-off-by: Kent Russell <kent.russell@amd.com>
[ROCm/ROCR-Runtime commit: 1304a92e17]
While adding x2APIC support, apicid for non-x2apic was missing out by
mistake.
Change-Id: I25eed362c035c0e9fb9ea948899c49f70311f269
Signed-off-by: Amber Lin <Amber.Lin@amd.com>
[ROCm/ROCR-Runtime commit: cfa47ac1f9]
Add SDMA engine info fields to node properties and
modify get node properties API to read SDMA engine
info from sysfs
Change-Id: Iea877b5bc008cc9df9405daf564a359535f1bc9f
Signed-off-by: Oak Zeng <Oak.Zeng@amd.com>
[ROCm/ROCR-Runtime commit: 414a3508d6]
Current processor/cache topology code implements xAPIC architecture, which
is 8 bits addressability. This is not enough for a system having more than
255 processors. x2APIC is the extension of xAPIC architecture to support
32 bit addressability of processors. This patch detects the x2APIC
enablement and uses the extension leaf to get apicid when detected.
Change-Id: I0826585d02f696a46cd5efb9a6630c60af01e2d8
Signed-off-by: Amber Lin <Amber.Lin@amd.com>
[ROCm/ROCR-Runtime commit: f8028a40fd]
The previous name doorbell_offset is used too extensively throughout
the code and did not reflect the true usage.
Change-Id: I50d33f5c00e82c46cdf4264a78b8f925705bed6a
Signed-off-by: Yong Zhao <Yong.Zhao@amd.com>
[ROCm/ROCR-Runtime commit: 776077fe65]
This variable is 0 by default. When set to 1, it means there is no frame
buffer, so all memory allocation is routed to system memory. This mode
is mainly used during emulation.
Use CoarseGrain for VRAM under ZFB mode
Change-Id: I29e8e98be56935e3ceb94782d70771cc45700749
Signed-off-by: Yong Zhao <Yong.Zhao@amd.com>
[ROCm/ROCR-Runtime commit: 51ee5c324a]
Amber Lin
shaoyunl
Philip Yang
Yong Zhao
Felix Kuehling
Jack Zhang
Kent Russell
Sean Keely
Ori Messinger
Jonathan Kim
Jay Cornwall
Oak Zeng
Philip Cox
Alex Voicu