While this may not be supported in the runtime, the kernel/firmware
support it
Change-Id: I7fe4536a6b3055f39e25f453060e899938645d91
Signed-off-by: Kent Russell <kent.russell@amd.com>
[ROCm/ROCR-Runtime commit: 1304a92e17]
While adding x2APIC support, apicid for non-x2apic was missing out by
mistake.
Change-Id: I25eed362c035c0e9fb9ea948899c49f70311f269
Signed-off-by: Amber Lin <Amber.Lin@amd.com>
[ROCm/ROCR-Runtime commit: cfa47ac1f9]
Add SDMA engine info fields to node properties and
modify get node properties API to read SDMA engine
info from sysfs
Change-Id: Iea877b5bc008cc9df9405daf564a359535f1bc9f
Signed-off-by: Oak Zeng <Oak.Zeng@amd.com>
[ROCm/ROCR-Runtime commit: 414a3508d6]
Current processor/cache topology code implements xAPIC architecture, which
is 8 bits addressability. This is not enough for a system having more than
255 processors. x2APIC is the extension of xAPIC architecture to support
32 bit addressability of processors. This patch detects the x2APIC
enablement and uses the extension leaf to get apicid when detected.
Change-Id: I0826585d02f696a46cd5efb9a6630c60af01e2d8
Signed-off-by: Amber Lin <Amber.Lin@amd.com>
[ROCm/ROCR-Runtime commit: f8028a40fd]
The previous name doorbell_offset is used too extensively throughout
the code and did not reflect the true usage.
Change-Id: I50d33f5c00e82c46cdf4264a78b8f925705bed6a
Signed-off-by: Yong Zhao <Yong.Zhao@amd.com>
[ROCm/ROCR-Runtime commit: 776077fe65]
This variable is 0 by default. When set to 1, it means there is no frame
buffer, so all memory allocation is routed to system memory. This mode
is mainly used during emulation.
Use CoarseGrain for VRAM under ZFB mode
Change-Id: I29e8e98be56935e3ceb94782d70771cc45700749
Signed-off-by: Yong Zhao <Yong.Zhao@amd.com>
[ROCm/ROCR-Runtime commit: 51ee5c324a]
Some test case alloc >4gb memory.
Use HSAuint64 in bytes and HSAuint32 in pages.
Change-Id: I0d5e6c299903b5898cfea024178a7a26b9ba3c90
Signed-off-by: xinhui pan <xinhui.pan@amd.com>
[ROCm/ROCR-Runtime commit: 41bf449e99]
RAS feature enabling bit and errors return are implemented in
existed topology and event mechanism.
v2: change library interface.
Change-Id: I75807c080b5b26e8115240b05b3d7016cb05a31a
Signed-off-by: Eric Huang <JinhuiEric.Huang@amd.com>
[ROCm/ROCR-Runtime commit: 8ee93b3187]
RAS feature enabling bit and errors return are implemented in
existed topology and event mechanism.
Change-Id: I9b018bba80cf4a6998e42a7bff64318c689b1d2a
Signed-off-by: Eric Huang <JinhuiEric.Huang@amd.com>
[ROCm/ROCR-Runtime commit: 1fbe010354]
Map scratch memory to the GPU that specified when allocate the memory
Change-Id: I788f9ef0dccb63b894a75e75cac5f94a60d7ec48
Signed-off-by: shaoyunl <shaoyun.liu@amd.com>
[ROCm/ROCR-Runtime commit: 29b45b8c0a]
EPERM means "operation not permitted" and is returned when CGroup
access checks fail. EACCES means "permission denied" and is returned
when the device file permission bits or access control list don't
allow access.
EPERM can fail silently, since we assume the administrator disabled
a device on purpose in the CGroup. EACCESS should produce an error
message and an info message to check the device file permissions.
Change-Id: Iee4c5584c5fdc4e113c3d760dede6661097b4341
Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com>
[ROCm/ROCR-Runtime commit: 5e4e19d47b]
This feature only support dgpu for now.
Change-Id: Ic766ec06892c955dd605ecc335a776335edc0df2
Signed-off-by: Gang Ba <gaba@amd.com>
[ROCm/ROCR-Runtime commit: c54c1dbdcb]
Device whiltelist controller cgroup allows to track and enforce open and
mknod restrictions on device files. Tasks should works with
/dev/dri/renderN devices that are whitelisted for its cgroup. If a
certain node is not whitelisted it is not an error condition.
Change-Id: I0b997423ccdc00aee98df5b6f04ed6794549604e
Signed-off-by: Harish Kasiviswanathan <Harish.Kasiviswanathan@amd.com>
[ROCm/ROCR-Runtime commit: c1994e28f0]
Move debug trap support capabilities to their own
structure to fix thunk spec vs header mismatch.
Change-Id: I6694601bfa36097502c8ab932e082d7a4645d5b2
Signed-off-by: Philip Cox <Philip.Cox@amd.com>
[ROCm/ROCR-Runtime commit: 105edd4bb4]
GPU Resource management can disable some of the GPU nodes.
The Kernel driver could be not aware of this.
Get from Kernel driver information of all the nodes and then filter it.
Change-Id: I4eeb126a5efce2192c35f5d2b72be1811e9ded32
Signed-off-by: Mike Li <Tianxinmike.Li@amd.com>
[ROCm/ROCR-Runtime commit: 3144a84b9a]
The existing call sysconf (_SC_NPROCESSORS_ONLN) provides the number of
processors available to the scheduler. When a KFD process is run under a
container environment, only a subset (cpuset) of processors are
available to the current process.
For getting CPU cache information use sched_getaffinity() to get the
number of processors available to the current process.
Change-Id: Ieac02f1f61c17e24ac34ba502968c69d3bc631cb
Signed-off-by: Harish Kasiviswanathan <Harish.Kasiviswanathan@amd.com>
[ROCm/ROCR-Runtime commit: fb79a0efe2]
Handle the case that svm.dgpu_aperture does not exist in vm_find_object.
Change-Id: Ic0983d4f321f1b6248514f2fa25162976e90bd75
Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com>
[ROCm/ROCR-Runtime commit: be574169c1]
Some nodes are unavailable based on the task's cgroup hierarchy. Handle
this situation by ignoring those nodes
Change-Id: I72f9e822d2ec8cf15732df95e427d5549a75b55d
Signed-off-by: Harish Kasiviswanathan <Harish.Kasiviswanathan@amd.com>
[ROCm/ROCR-Runtime commit: 7876bb70a9]
With GPU resource management, some nodes are unavailable based on the
cgroup hierarchy of the task. Kernel via sysfs specifies all the
iolinks. Skip the links which are not accessible.
Also iolinks specified by the kernel refer to sysfs Node IDs. Map it to
relevant user Node IDs
v2: NodeFrom mapped from sysfs Node to User Node
Change-Id: I95312ee6ca51b89fe9e6ca2a9185c2ea1e94afc4
Signed-off-by: Harish Kasiviswanathan <Harish.Kasiviswanathan@amd.com>
[ROCm/ROCR-Runtime commit: 866ef20054]
This reverts commit 84bb9072c0.
This fixes ambiguity when looking up GPU addresses with
hsaKmtQueryPointerInfo.
hsa_amd_agents_allow_access uses hsaKmtQueryPointerInfo, and
depends on finding the correct object from a GPU address. Finding
the wrong userptr object based on its CPU address leads to
incorrect GPU mappings and results in VM faults.
Change-Id: I7c5f571ee6e1f9d32687aa3eab6d96944ad032be
Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com>
[ROCm/ROCR-Runtime commit: a9bd6e6f8b]
Added a helper vm_find_object that can be used everywhere we need to
lookup objects by their address and optionally size. This unifies
all subtly different, partially incomplete, or broken ways of doing
this in various functions:
* map
* unmap
* register
* deregister
* free
* get_mem_info
* set_mem_user_data
At the same time fix some subtle problems for userptr lookup that
got a bit more complex when the userptr address can match the GPU
address.
Change-Id: I98572d1734fc7688a1d68f6a784e02c8dee90af5
Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com>
[ROCm/ROCR-Runtime commit: 855f1a32a9]
PCIe P2P (indirect) IOLinks should only be created if the remote GPU
is large-BAR
Change-Id: I55cbb5e37c5d41267583e07aca6bdcc708403029
Signed-off-by: shaoyunl <Shaoyun.Liu@amd.com>
[ROCm/ROCR-Runtime commit: 30a4ab39f3]
Currently, all HSA nodes are exposed to user. So the existing
implementation assumes a one to one mapping between user
NodeId and sysfs nodeId.
GPU Resource Management will provide control over the exposed
HSA nodes. This means not all HSA nodes will be exposed to the user.
Decouple it.
The mapping from user NodeId to sysfs NodeId will be local
to topology.c and topology helper functions. For others NodeId
should be sequential from 0 to Number of Nodes exposed to user.
v1: initial implementation
v2: map node id within the topology_* functions
v3: remove two static globals
v4: add bounds check got node id
Change-Id: Id12147ece41d682430f398944bbb339ca906eb1b
Signed-off-by: Mike Li <Tianxinmike.Li@amd.com>
[ROCm/ROCR-Runtime commit: 3437a356c7]
Add initial support for the kfd debugger trap support
for GFX9 chips.
- Adding support for Enable/Disable trap support
- Setting debug trap support data
- Setting wave launch trap override
- Setting wave launch mode
Change-Id: If39f2395c4b6cf56249cf76f1c44cfcbdcef891c
Signed-off-by: Philip Cox <Philip.Cox@amd.com>
[ROCm/ROCR-Runtime commit: db92d5af23]
AMDKFD_IOC_WAIT_EVENTS with multiple events and wait_for_all = 0
returns success after any of the events have signaled. So we can't
blindly assume that a memory fault event that was in the list has
actually signaled. Check the gpu_id as an indicator whether there
really was a memory fault before processing it further.
Change-Id: I6cc311bfc184c631beaf684027176a6ca42e05c1
Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com>
[ROCm/ROCR-Runtime commit: 9271e69ddf]
If the CPU addr of a userptr is accessible by the GPU, try to use it
instead of allocating a different GPU address. If something else is
already registered with an overlapping address range, we still need to
allocate a GPU address, because KFD does not support overlapping GPUVM
mappings.
Change-Id: I452963ee45a454f735755a0b43122b9aee5d55be
Signed-off-by: Felix Kuehling <felix.kuehling@gmail.com>
[ROCm/ROCR-Runtime commit: ab181c46c0]
If the GPU virtual address space is >= 47 bits, don't reserve virtual
address space at startup and use mmap to allocate virtual addresses.
Change-Id: Ic935b03c8e78271829fc8e6cfd0e543184aff818
Signed-off-by: Felix Kuehling <felix.kuehling@gmail.com>
[ROCm/ROCR-Runtime commit: 80f2cc644c]
So far we have assumed that userptrs are always memory outside
reserved SVM apertures that are mapped into the SVM aperture for
GPU access.
With an unreserved SVM aperture that covers the entire virtual
address range, this distinction will no longer be true. Userptrs
will generally be inside the unreserved SVM aperture. Take that
into consideration when registering, mapping and unmapping virtual
addresses.
We now need a retry logic when looking up buffers from addresses.
If it is not found by its GPU address, try it as a userptr.
We also need to consider the new possibility that a userptr is
registered at the same address for CPU and GPU access. So a buffer
found by its GPU address may also turn out to be a userptr. In
that case use a stricter lookup using the userptr and size (if
the size is known), to identify the correct one of multiple
overlapping objects.
Change-Id: Ia43633aaa40f9fd2a74918ae969a631d2ff68419
Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com>
[ROCm/ROCR-Runtime commit: 40c46cc6cb]
Make dgpu_aperture and dgpu_alt_aperture pointers that can point to
the same actual aperture. This will be useful on GFXv9 and later,
where the MType is not defined by the aperture and we want to have
a single aperture covering the entire virtual address space.
aperture->is_coherent can no longer be a reliable indicator of
coherency. Replace it with different conditions based on mem flags
and svm.disable_cache (from HSA_DISABLE_CACHE environment).
Change-Id: Iefc415b87b8abd96e3916586485a0a55d9b27c19
Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com>
[ROCm/ROCR-Runtime commit: d57026f447]
This prepares the code for an alternative aperture management method
that needs to unmap memory differently.
Change-Id: I5494aa5420f85edb8f7857f00c17e1d2e6479a51
Signed-off-by: Felix Kuehling <felix.kuehling@gmail.com>
[ROCm/ROCR-Runtime commit: 2d2181b478]
Only scratch_physical, for scratch-backing memory is managed by the Thunk.
Change-Id: I4716981aa908d9569584dc35f40ffd270a2f9014
Signed-off-by: Felix Kuehling <felix.kuehling@gmail.com>
[ROCm/ROCR-Runtime commit: 9d96af0150]
This parameter was used for non-canonical GPUVM allocations on GFX7/8 APUs
only, to prevent getting NULL pointers from valid allocation after
subtracting the aperture base. The same can be achieved less intrusively
by reserving address space at the start of the aperture during
initialization.
Change-Id: I0aae773f069c2b228824ba464b0612a4d8b489ce
Signed-off-by: Felix Kuehling <felix.kuehling@gmail.com>
[ROCm/ROCR-Runtime commit: 842359a826]
Hit queue create failure when do kfdtest with --gtest_repet=-1
fix: d89e9165("Remove the use of IS_DGPU()")
Change-Id: I04fa73f90cef13a5517dbaceb89c41dc0f821a79
Signed-off-by: xinhui pan <xinhui.pan@amd.com>
[ROCm/ROCR-Runtime commit: eb5539fb10]
Because gfx700 has local memory but other APUs don't, we should reflect
that in the code. Meanwhile, fix a bug that on gfx902 svm aperture is not
added when calling hsaKmtGetNodeMemoryProperties().
Change-Id: Id840f2db0b14fda9ee713b219a9474c15f8a9771
Signed-off-by: Yong Zhao <Yong.Zhao@amd.com>
[ROCm/ROCR-Runtime commit: 110e754f64]
On some asics, like tonga, the memory alignment size is as big as 0x8000.
fmm_allocate* alloc vm area with size passed in which is not aligned mostly.
But __fmm_release free vm area with vm_object_t->size which is aligned.
That might cause aperture_release_area fail to free the vm area as the
size might be bigger than zone itself or it just free another vm area
nearby unexpected.
This patch somehow will alloc more space than it needed on tonga.
gfx900+ is not affected.
Change-Id: I5a88c92b08c4e6f6bc05881798f769b55d6debe9
Signed-off-by: xinhui pan <xinhui.pan@amd.com>
[ROCm/ROCR-Runtime commit: 8fbf4a26ec]
Previously we used the first dgpu mem, but after careful examination, we
found it only needs to be a GPU, so we modify the code to reflect that as
well.
Change-Id: I069d9b8e247aed55c1f885b79f743ea8e03ddf93
Signed-off-by: Yong Zhao <Yong.Zhao@amd.com>
[ROCm/ROCR-Runtime commit: fe04dd6890]
Sean Keely
Kent Russell
Amber Lin
Oak Zeng
Yong Zhao
xinhui pan
Jay Cornwall
Philip Cox
Eric Huang
shaoyunl
Felix Kuehling
Gang Ba
Harish Kasiviswanathan
Mike Li
Felix Kuehling