If hsa_amd_agents_allow_access is called for an imported IPC handle,
ignore the request as this pointer will already have these pointers
mapped to other GPUs during IPCAttach()
Change-Id: I4bf33ed57e93b5a3ead749d4f87ab6f2750bed58
If a user queries the pointer info on an invalid pointer,
hsaKmtQueryPointerInfo will return error or unknown pointer. The other
fields in HsaPointerInfo are invalid, so we do not return them to the
user.
Also removing the assert and returning unknown pointer instead. As the
assert will not trigger in release builds.
hsaKmtQueryPointerInfo may also return unknown pointer for userptrs as
they are not always tracked by thunk. Adjusting code to still treat
these pointers as valid in this case.
Change-Id: Idf5cd8b61cd532d31b072f449839d223369bb138
Amount of memory requested by user may be aligned-up internally to
the memory pool granularity. The extra padded memory should not be
considered when validating pointers from the user. Also return the
user requested size when user queries pointer information.
Change-Id: I28b25448ea03c836b44fafdb34b7330cf6887424
Add agent info query HSA_AMD_AGENT_INFO_ASIC_FAMILY_ID.
Then we can remove the codes to parse family id.
Signed-off-by: Lang Yu <Lang.Yu@amd.com>
Change-Id: I3ac4746d3015e89b32322ebc0f8a3084f98677a4
The allocation logic of the SPI does not take into account compute
user thread management settings for masking CUs with the exception of
skipping fully disabled SEs. This means that occupancy limited
dispatches such as cooperative launch may over allocate onto hardware
resources that are not immediately available, resulting in a potential
barrier logic hang as occupying work groups are waiting on enqueued
work groups to reach the barrier.
Further work will have to be done to get the per-SA CU enablement count
from the KFD in order to correctly clip the cooperative CU limit based
on the CU mask, which will require breaking the current ABI.
For now, report that cooperative launch is not supported while a CU
mask has been applied to prevent potential shader hangs.
Change-Id: I8be4bb47d65ceb62d805f36ef6ef3996d756021f
New environment variable HSA_OVERRIDE_CPU_AFFINITY_DEBUG to
enable/disable overriding CPU affinity.
Default value is enabled(1).
This is a temporary variable and may be removed in the future.
Change-Id: Id6a7c611730471ddc276ca333fde1e57046bf32a
Non-paged allocation for queue memory necessary for binding wptr to
GART. Required to support usermode queue oversubscription with MES for
GFX11.
Adds AllocateNonPaged entry to MemoryRegion::AllocateEnum for clarity;
aliases AllocateIPC.
Signed-off-by: Graham Sider <Graham.Sider@amd.com>
Change-Id: I1a97a1820da26cf2433d9c237b2e6d2b0b8628b4
GPUs excluded by RVD are not expected to have scratch, memory, trap
handling nor memory regions set up. Now that these GPUs are added to
a new list, early return on agent destruction to prevent bad function
calls on destroy.
Also fix up broken memory releases between the gpu lists and ugly braces.
Change-Id: I52fc6e86ceba0a0383cedc63310eb409515eaf9f
A work group processor (WGP) require both its CU to be enabled
in order to be enabled.
The KFD will round robin distribute by even-indexed pairs so
enforce this requirement for runtime set mask calls.
Change-Id: Ic46661b01f398aa1fe24d96b5c9c31f122f967a3
Discovered agent handles should only apply to copy routing, not to
copy device selection. The user may not have mapped all allocations
to all GPUs so we must ensure that the copying device is one passed
by the user.
Change-Id: I2532e66d30e6842624e594f235dd144a186220d4
Mostly a demo at this point. Logs SVM (aka HMM) info to
HSA_SVM_PROFILE if set.
Example: HSA_SVM_PROFILE=log.txt SomeApp
Change-Id: Ib6fd688f661a21b2c695f586b833be93662a15f4
During hsa initializing stage, ROCr now searches all the loaded libraries
for a symbol "HSA_AMD_TOOL_PRIORITY" and adds all those libraries to
the tools library init list. Tools libraries listed in HSA_TOOLS_LIB
env variable are also loaded in the given order and take priority
over HSA_AMD_TOOL_PRIORITY.
Change-Id: I739af42bbd777c44a9152c11e17dd69979b65e82
Adds a script to run clang-format on the latest patch so we don't
need to remember the command line.
Also applies missing formatting to the prior commit,
"Add API for available GPU memory".
Change-Id: Ida51aedc38af229f6a26e275072654860748fa93
Add support for AMD Agent to return amount of memory available
Change-Id: I5c32e2cebbaa2993b044250aefe434e4cc02d8c2
Signed-off-by: David Yat Sin <david.yatsin@amd.com>
IPC use cases with RVD set can't convey proper agent handles.
Runtime discovery is required to properly route the copy in this
case.
Change-Id: I4c97e132fb4b6ac1040de1cb17fe5a3e36d6be48
This is consistent with KFD and has significantly better latency.
KFD is taking this as the definition of the SystemClockCounter.
Change-Id: I4c1b3bc58c738206265c55ebefd41356c013bfe5
Eliminates the need for manually assembling the source of the
second level trap handler to produce the shader binary. Also
separated blit shaders' binary source and version one second
level trap handler binary sources into different header files.
Change-Id: If29a18ee06dc083ec880ea962f234c6b5cac806a
Host to device SDMA copies do not require an HDP cache flush when
connected by xGMI since data copies over the data fabric and not HDP.
Signed-off-by: Jonathan Kim <jonathan.kim@amd.com>
Reviewed-by: Sean Keely <sean.keely@amd.com>
Change-Id: I78d73a47edcc1a9c0ba59f33cf91485f13f1c45b
Declare the type of HSA_AMD_AGENT_INFO_COOPERATIVE_COMPUTE_UNIT_COUNT
and add a missing break statement.
Change-Id: I86ce8a2e620438e046b60cee991ce1fbe07a3e88
On gfx10+ we need to issue a minimum count of active lanes or
groups before ADC moves on. Ensure that scratch allocations
attempt to reach this limit.
Occupancy throttling due to OOM condition may still drop below this
limit.
Change-Id: I0edf2e40fbe1a95e9a262564cebd2b6a82501a0b
__x86_64__ and __AMD64__ should be already defined by the compiler to
specify the compilation target and shouldn't be defined manually.
I fixed two x86_64 checks to include VS variables, as removing this
might cause it to fail to compile on that compiler.
Signed-off-by: Jeremy Newton <Jeremy.Newton@amd.com>
Change-Id: I600ff449af85bf7d83ecab167d97933922e2d917
Each time delay is grown we need to reset elapsed. We want to take
the most accurate sample from the set at fixed delay.
Without this we will hang if there is ever an insufficiently accurate,
high unit clock read.
Change-Id: Ic65f364067789ac85a6572d67af2d77528e265bb
The loader must use internal interfaces to access page allocation
flags. Code pages should also ensure use of cached memory.
Also relocate i-cache flush after code page copy.
Change-Id: I86d36243b6eebb1d46b991b372a5236baaf941ab
VM faults should not report via the queue error handler.
The system event contains much more useful information.
Change-Id: I744d9b97b23334d7ed2c0f450111c1b8032567e3
Hive ID is used during copy path selection to locate an optimal
pool of SDMA engines. However, for CPU-GPU connections we always
want to use the host port facing engines, known generally as the
PCIe optimzed engines. We want this selection even when the
connection is XGMI hence dropping the hive id for CPUs.
Change-Id: Iffe44174afecfc0bb3272b806fce549c930a49d9
David Yat Sin
David Belanger
Lang Yu
Jonathan Kim
Yifan Zhang
Graham Sider
Jay Cornwall
skhatri
Sean Keely
Shweta Khatri
Jeremy Newton