This patch adds the support for gfx90c apu. So far we treat it as "dgpu" and
gfx900. Will update hsa gfxip table while the isa/llvm is implemented on gfx90c.
Change-Id: I6ef164bf3e751fe6dd6287cac212a500dce84b1a
Signed-off-by: Huang Rui <ray.huang@amd.com>
On NUMA system, allocate queue ctx save restore area on the closest NUMA
node to the GPU which the queue is going to run. This will improve
performance on NUMA system generally by reducing schedule latency and
fix the multi-node rccl-tests unstable performance issue.
If the closest NUMA node has no memory available, set flags NoNUMABind=1
to bypass mbind, to use default NUMA memory policy to allocate system
memory.
Change-Id: Ic62bfa5bb2efbf4f6ae79ff403e9610ddf18d45c
Signed-off-by: Philip Yang <Philip.Yang@amd.com>
Docker seccomp by default blocks mbind system call, so mbind return
failed on docker. thunk should not fail this otherwise application
cannot allocate system memory on docker.
Use pr_warn_once and pr_err_once to avoid duplicate same error messages
Change-Id: I61a7c0e4abaa3dcfe7abf2ea48db90f669f9638a
Signed-off-by: Philip Yang <Philip.Yang@amd.com>
Force all the GPUs to a certain type, use the below command:
HSA_FORCE_ASIC_TYPE="10.1.0 1 gfx1010 14"
meaning major.minor.step dgpu asic_name asic_id
This will faciliate the cooperation across the teams for bringing up
ASICs which reuse existing device IDs.
Change-Id: I40fe4c9b46d3ccb3e38ea52250e80e82fb50fb0f
Signed-off-by: Yong Zhao <Yong.Zhao@amd.com>
The queue IDs passed over to the kernel via kfd_ioctl_dbg_trap_args->ptr
should be a list of uint32_t's. Need to convert from the passed in
64 bit HSA_QUEUEID to 32 bit uint32_t's.
Change-Id: I8718566d9f9ffc90ce0b2ecc129b10c49d73186a
Signed-off-by: Philip Cox <Philip.Cox@amd.com>
While this may not be supported in the runtime, the kernel/firmware
support it
Change-Id: I7fe4536a6b3055f39e25f453060e899938645d91
Signed-off-by: Kent Russell <kent.russell@amd.com>
GPU Resource management can disable some of the GPU nodes.
The Kernel driver could be not aware of this.
Get from Kernel driver information of all the nodes and then filter it.
Change-Id: I4eeb126a5efce2192c35f5d2b72be1811e9ded32
Signed-off-by: Mike Li <Tianxinmike.Li@amd.com>
This reverts commit ab181c46c0.
This fixes ambiguity when looking up GPU addresses with
hsaKmtQueryPointerInfo.
hsa_amd_agents_allow_access uses hsaKmtQueryPointerInfo, and
depends on finding the correct object from a GPU address. Finding
the wrong userptr object based on its CPU address leads to
incorrect GPU mappings and results in VM faults.
Change-Id: I7c5f571ee6e1f9d32687aa3eab6d96944ad032be
Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com>
Currently, all HSA nodes are exposed to user. So the existing
implementation assumes a one to one mapping between user
NodeId and sysfs nodeId.
GPU Resource Management will provide control over the exposed
HSA nodes. This means not all HSA nodes will be exposed to the user.
Decouple it.
The mapping from user NodeId to sysfs NodeId will be local
to topology.c and topology helper functions. For others NodeId
should be sequential from 0 to Number of Nodes exposed to user.
v1: initial implementation
v2: map node id within the topology_* functions
v3: remove two static globals
v4: add bounds check got node id
Change-Id: Id12147ece41d682430f398944bbb339ca906eb1b
Signed-off-by: Mike Li <Tianxinmike.Li@amd.com>
If the CPU addr of a userptr is accessible by the GPU, try to use it
instead of allocating a different GPU address. If something else is
already registered with an overlapping address range, we still need to
allocate a GPU address, because KFD does not support overlapping GPUVM
mappings.
Change-Id: I452963ee45a454f735755a0b43122b9aee5d55be
Signed-off-by: Felix Kuehling <felix.kuehling@gmail.com>
Because gfx700 has local memory but other APUs don't, we should reflect
that in the code. Meanwhile, fix a bug that on gfx902 svm aperture is not
added when calling hsaKmtGetNodeMemoryProperties().
Change-Id: Id840f2db0b14fda9ee713b219a9474c15f8a9771
Signed-off-by: Yong Zhao <Yong.Zhao@amd.com>
The information can be obtained directly from node id. Also improve the
whole logic for future compatibility.
Change-Id: I130733be4e7930d5953d5e81409905e60c2ec35e
Signed-off-by: Yong Zhao <Yong.Zhao@amd.com>
On discrete GPUs place the EOP queue in VRAM. The reader/writer of this
queue is the CP and the size is small. Dispatch latency improves
through lower read latency in AQL completion phase.
Change-Id: Id8351dcddbd21fd7c7d699803c96434c9132db71
Signed-off-by: Jay Cornwall <Jay.Cornwall@amd.com>
Previously kfd device is used to map memory for CPU access.
However this is not compatible with how TTM handles CPU mapping
on eviction - memory won't be unmapped and remapped on restore.
This fixes the issue by mmapping memory using DRM render device.
This patch requires a coordinated kernel driver change to work.
To make it compatible with old kernel driver, some temporary codes
are included. Once the coordinated kernel driver is checked in,
the temporary codes can be removed.
Change-Id: Ie7b304c4a82b7e8d5ab703acb81d66430af4f0bc
Signed-off-by: Oak Zeng <Oak.Zeng@amd.com>
Because of HW design change, GPUVM aperture is no longer needed on GFX9
APUs. However, on APUs some functionalities still depend on GPUVM
aperture, so we choose to use SVM aperture instead to assume
the functionality of previous GPUVM aperture.
Change-Id: Ife7f0d598dd7989f2bcf7cdf3466d5a68703ca60
Signed-off-by: Yong Zhao <yong.zhao@amd.com>
This should allow us to take advantage of BigK fragments and huge pages
and improve TLB efficiency for VRAM allocations. Huge pages only work
with 4-level page tables (gfx900 and up). BigK fragments work on older
GPUs.
Change-Id: I02e1fbf74de554e16fdaf44e44d03b47df45c3b0
Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com>
Existing Thunk has printf/fprintf in the code while normally libraries
don't print any message. This patch introduces a print machenism similar to
how the Linux kernel prints to console based on the log level. The default
is not to print any message, but setting HSAKMT_DEBUG_LEVEL will enable the
prints.
Change-Id: Ic071e122d35a82260218e9914cde4815e69df742
Signed-off-by: Amber Lin <Amber.Lin@amd.com>
For items in HsaQueueInfo, control stack information comes from KFD, CU
mask information is maintained in Thunk, and others (queue detail error
and queue type extended) are ignored (value = 0) at this point.
Change-Id: Ib21370b0f52b2bb4ebe6a9b4b6ec6139cccb25ca
Signed-off-by: Amber Lin <Amber.Lin@amd.com>
Integrate the supported device ID list distributed in topology, queue, and
pmc into one place: topology.
Change-Id: If035cf8e4a6fc6caff6c94ec627647cfb11c3d79
If fork() is called, clear all duplicated data that is invalid in the
child process.
Change-Id: I4e27198060db593c630c6337b7071dfbd0d80b83
Signed-off-by: Harish Kasiviswanathan <Harish.Kasiviswanathan@amd.com>
Implement three new APIs for IPC buffer sharing:
-hsaKmtShareMemory()
-hsaKmtRegisterSharedHandle()
-hsaKmtRegisterSharedHandleToNodes()
Add new ioclts necessary for the above APIs.
Change-Id: Ia2b4d0dc91ec64bff959395d11c0536467404792
Signed-off-by: Harish Kasiviswanathan <Harish.Kasiviswanathan@amd.com>
HSA thunk API is currently reporting engineering name to MarketingName
and returning NULL when querying for AMDName.
-Change current name reporting from MarketingName to AMDName.
-Use libpci to get MarketingName
Change-Id: I819a6de7b067a2e724a6695e7d800274b83a71f8
Signed-off-by: Lan Xiao <Lan.Xiao@amd.com>
Allocations from GPU nodes will return VRAM, not system memory.
Only non-paged allocation from GPU nodes is supported. System
memory can only be allocated from CPU nodes (usually node 0).
The HostAccess flag is no longer used to distinguish the memory
type. It only indicates, whether the memory is mapped for CPU
access.
Maintain compatibility with broken KfdTests by returning system
memory for paged-memory requested from GPU nodes.
Change-Id: I514defede735f55e6de436f41944125b6f2c4ccf
Stop using NUM_OF_SUPPORTED_GPUS. For now the definitions itself cannot
be removed as ioctl code is in upstream Kernel.
Change-Id: If846625a8ad5062d5483e762850c793d3c00b9d0
Kernel ioctl AMDKFD_IOC_GET_PROCESS_APERTURES returns process apertures
only for GPU nodes. The current implementation assumed that this list of
GPU nodes returned by the ioctl has one to one correpondence to sysfs
topology nodes. This fails when non-GPU nodes exist in topology as in
case of Intel + gfx802
Fix this by using gpu_id (./sys/.../kfd/topology/nodes/1/gpu_id) to map
information obtained from kernel ioctl call.
Change-Id: I4ab8ae5354f12cf0b6609fc4b24182b82eb3677f
Fix TONGA_PAGE_SIZE value and move it to libhsakmt.h for usiing it
consistently in all places that require the same alignment for the
same reason. Create a generic alignment helper macro to replace some
incorrect hand-coded size alignments.
Move virtual address and size alignments down into aperture management
functions. Alignment is a per-aperture property that is set during
fmm_init_process_apertures. Doing the alignment there ensures that
all allocations in the same aperture are aligned the same way. Finding
objects by size and address can take the alignment into account.
Also align the size of physical allocations to back aligned virtual
address allocations. CPU mappings do not need to be aligned.
Map anonymous pages over released memory mappings to allow the
backing pages to be released, while keeping the address space
reserved.
Add alignment parameter to free_exec_aligned_memory_gpu to match the
interface of allocate_exec_aligned_memory_cpu. It doesn't make sense
to allow an alignment parameter in one but assume a specific
alignment in the other.
Change-Id: I74226ca6938f4948f643e5aee1d474720cd89e78
Huang Rui
Philip Yang
shaoyunl
Yong Zhao
Philip Cox
Alex Voicu
Kent Russell
Gang Ba
Mike Li
Felix Kuehling
Felix Kuehling
Jay Cornwall
Oak Zeng
Amber Lin
Harish Kasiviswanathan
Lan Xiao
Ben Goz
Alexey Skidanov
Oded Gabbay