New environment variable HSA_CU_MASK allows users to
specify a cu mask to every queue allocated from any
GPU. hsa_amd_queue_cu_set_mask is restricted from
escaping this mask.
A new API hsa_amd_queue_cu_get_mask is added to query
the current cu mask.
Change-Id: I846c03a5faaca9b95067c31db84b59cc9fce2f03
[ROCm/ROCR-Runtime commit: 4455250be1]
Under xnack we can now identify the queue which generated a vm fault.
This allows users to identify which queue, and therefore which
dispatch, a vm fault came from.
Change-Id: If72ff3de05800f2b811aa7842a15eedff8b5e45a
[ROCm/ROCR-Runtime commit: 59ee761f81]
ttmp6.packet_index is reported as 0 for all waves, regardless of the
dispatch packet position in the queue, due to an issue in the clearing
of the previous trap_id and saved status.halt bit.
Fixed TTMP6_SAVED_STATUS_HALT_MASK to only be one bit, 1<<29.
Change-Id: Ia4934e51123a40d71de658efc387a1f3a6344f05
[ROCm/ROCR-Runtime commit: ef1955ad42]
If left non-zero the event loop will keep reinvoking the callback,
preventing AqlQueue::ExceptionHandler from running.
Change-Id: If85fbaf62f04ffd327ecf9d649aa23afad4442ce
[ROCm/ROCR-Runtime commit: 8d4608ed0e]
Also fix hsaKmtRuntimeEnable error handling. Continue if ioctl fails.
Change-Id: I754ccba5910ccfef6f1ada1415593ef89ce33aba
[ROCm/ROCR-Runtime commit: 7e4088309d]
Certain special signals do not carry their updates via their signal
value. These signals are wrappers around special KFD events, of
which the only current instance informs about VM faults. We either
need to check each signal for this special event type or rely on
the checking done in hsa_amd_signal_wait_any. Since there will always
be a small number of these signals it doesn't make much since to
penalize the performance path with this check. Additionally we know
that the signal indicated by hsa_amd_signal_wait_any is satisfied so
don't need to recheck it's conditions.
Change-Id: I9fc6298300ad543d823ecd28ca8fab4ad26c23ef
[ROCm/ROCR-Runtime commit: 3d6a18b67c]
Allows determining if the host can directly access HMM memory that
is physically resident in vram.
Change-Id: Ie452eedd0e27fe1b511afd416f5a1cd01b3d84e8
[ROCm/ROCR-Runtime commit: 9e53cab613]
Enables the fragment allocator to handle >2MB allocations, maintaining
good TLB alignment. Prior code contained a bug that caused the effective
API granule for vram allocations >2MB to be bumped to 2MB.
Also adjusts the block cache's block retention heuristic to not
count discarded blocks as in use. This will reduce block retention
when a significant amount of large blocks or IPC is in use.
Change-Id: I30bd85eb87951df822211f799d9cfe579ab109c6
[ROCm/ROCR-Runtime commit: 8adbda1c18]
Add macro debug_warning_n to stop printing a message after
N instances.
Change-Id: Id5f84b11eb63b3a20bd2bcb2ea8f10a066b457ef
[ROCm/ROCR-Runtime commit: ca8387768e]
Under high async handler load signal retention and event sorting
become bottlenecks. This change processes more handlers in a
single pass to amortize wait_any overheads.
Change-Id: I8b276e102db647e3858e120547aa0c6fca85ab4c
[ROCm/ROCR-Runtime commit: 6b398eb72c]
Old memory properties info name used after removing branches.
This caused the CPU coarse grain pool to initialize with random
bits.
Change-Id: I397bc5ecf09fab69bdf1d7fafadcf54d71b64070
[ROCm/ROCR-Runtime commit: 0439dc90cd]
Prevents poorly written tools which throw in tools interface
callbacks from causing ROCr to catch and return a generic error
code.
Change-Id: I2f5bf7104dc7d4ee688eb48423c7ffdb06bd7702
[ROCm/ROCR-Runtime commit: c9ce27a640]
Old logic did not consider memory held in the scratch cache to be
free when deciding whether or not to reclaim.
Change-Id: I7f7c7549c72d743edbf7c53489fe9a453dc4177a
[ROCm/ROCR-Runtime commit: 0b7d9db964]
Includes some workarounds and HMM.
Conflicts:
opensrc/hsa-runtime/core/runtime/amd_topology.cpp
opensrc/hsa-runtime/core/util/flag.h
Change-Id: I22976f07964a43dbb228a6231777dbd599112b8d
[ROCm/ROCR-Runtime commit: 7333c77e22]
When no isa's are available no callbacks should be invoked. This
is not an error and should return success.
Change-Id: Ie4048aa8cbe5c3fdf5431f6a865021549ecf8a13
[ROCm/ROCR-Runtime commit: 4197461b7f]
Sramecc is misreported in kfd 4.0 and prior. To prevent possible
corruption due to d16 instructions, deny use of gfx906 with older
kfds and correct misreport for gfx908. Denial of gfx906 may be
overridden by setting HSA_IGNORE_SRAMECC_MISREPORT=1.
Change-Id: I7d5c3a716fad01c348f8b88cd508cedbf914c989
[ROCm/ROCR-Runtime commit: 45fbe5b192]
Park the wave, if it is stopped, to avoid halting it at an s_endpgm
instruction if the architecture does not support it.
Free ttmp6 by converting the dispatch_ptr into a queue packet index
(25-bit) and storing it in ttmp7[24:0].
Save the exception PC in ttmp11[22:7] ttmp6[31:0].
Change-Id: Iaa3c5baf5b488c0b534044d338f12bffa63ddce2
[ROCm/ROCR-Runtime commit: ea6ee0aa81]
Scratch cache was not updated for IOMMUv2 systems previously.
This both negates the cache and causes segfault during scratch
release.
Change-Id: I71e81d6b642d65ca135868ff7225ea173529d458
[ROCm/ROCR-Runtime commit: 191664cd20]
Legacy p2p copy path incorrectly transfered in whole pages rather than
the requested size.
Change-Id: I9aa7337754f9e32f587a0cc5305f8ffeb6196f10
[ROCm/ROCR-Runtime commit: 34ac62274a]
Replace the stop reasons ttmp11.trap_raised and ttmp11.excp_raised
with ttmp11.wave_stopped which indicates that the trap handler has
halted the wave as the result of an event (trap, single-step or
exception).
If the wave is stopped because of a trap, also record the trap_id in
ttmp11.saved_trap_id[7:0].
Save status.halt in ttmp11.saved_status_halt, so that it can be
restored when resuming a wave (changing a wave's state from stopped to
running or single-stepping).
Change-Id: I7322f59b60e8cc1b92bf5f067dba606a3109ef49
[ROCm/ROCR-Runtime commit: 9ca79d072a]
This patch is to let ROCr recognize new gfx10.3.3 ISA.
Change-Id: Ied23eee2752e14c19c8c0a6d7789fded9940e31e
Signed-off-by: Huang Rui <ray.huang@amd.com>
[ROCm/ROCR-Runtime commit: feeb2f62e2]
To support single stepping the instruction preceding an s_endpgm,
unwind the PC by 8 bytes and set ttmp11[9] to notify the debugger
that the wave is halted with a modified PC.
Bump the debug r_version for this new trap handler ABI.
Change-Id: I55e4e0d65576f92da14a336266c31c513baab547
[ROCm/ROCR-Runtime commit: 8aec53969f]
Each SE must be assigned equal numbers of slots and slots
must be assigned in units of whole groups.
Change-Id: I8f3677237fa6f2e2d25e3e78210c5a7a0ad792f3
[ROCm/ROCR-Runtime commit: 7bc6aac5d2]
Sean Keely
Aaron Liu
Laurent Morichetti
Jay Cornwall
Chris Freehill
Mike (Tianxin) Li
Konstantin Zhuravlyov
Tony Tye
Ramesh Errabolu
Jason Tang
Huang Rui