For minimal latency we should place command queues and blit code
in the nearest numa node to each GPU. Add an allocator matching
the current runtime default allocator interface to each GpuAgent
that allocates on the closest numa node as represented by kfd
topology. Use this allocator for queue ring buffers and blit
objects.
Change-Id: I181127f9c27bafe68976312963146616e3f58369
Also make failure to handle queue errors fatal.
Motivation is to improve detection of queue error conditions
that currently appear as application hangs.
Change-Id: I655643616dc0bd303d7df3ce8aca2c099bec3d46
Take in const void* rather than void*. This does not break the
abi or existing code. Existing code would need to cast away any
const which is unnecessary and annoying.
Change-Id: I28787e8fab1b600bf6871ea82835e10a4f475c5b
New environment variable HSA_CU_MASK allows users to
specify a cu mask to every queue allocated from any
GPU. hsa_amd_queue_cu_set_mask is restricted from
escaping this mask.
A new API hsa_amd_queue_cu_get_mask is added to query
the current cu mask.
Change-Id: I846c03a5faaca9b95067c31db84b59cc9fce2f03
ttmp6.packet_index is reported as 0 for all waves, regardless of the
dispatch packet position in the queue, due to an issue in the clearing
of the previous trap_id and saved status.halt bit.
Fixed TTMP6_SAVED_STATUS_HALT_MASK to only be one bit, 1<<29.
Change-Id: Ia4934e51123a40d71de658efc387a1f3a6344f05
Old logic did not consider memory held in the scratch cache to be
free when deciding whether or not to reclaim.
Change-Id: I7f7c7549c72d743edbf7c53489fe9a453dc4177a
Includes some workarounds and HMM.
Conflicts:
opensrc/hsa-runtime/core/runtime/amd_topology.cpp
opensrc/hsa-runtime/core/util/flag.h
Change-Id: I22976f07964a43dbb228a6231777dbd599112b8d
Sramecc is misreported in kfd 4.0 and prior. To prevent possible
corruption due to d16 instructions, deny use of gfx906 with older
kfds and correct misreport for gfx908. Denial of gfx906 may be
overridden by setting HSA_IGNORE_SRAMECC_MISREPORT=1.
Change-Id: I7d5c3a716fad01c348f8b88cd508cedbf914c989
Park the wave, if it is stopped, to avoid halting it at an s_endpgm
instruction if the architecture does not support it.
Free ttmp6 by converting the dispatch_ptr into a queue packet index
(25-bit) and storing it in ttmp7[24:0].
Save the exception PC in ttmp11[22:7] ttmp6[31:0].
Change-Id: Iaa3c5baf5b488c0b534044d338f12bffa63ddce2
Replace the stop reasons ttmp11.trap_raised and ttmp11.excp_raised
with ttmp11.wave_stopped which indicates that the trap handler has
halted the wave as the result of an event (trap, single-step or
exception).
If the wave is stopped because of a trap, also record the trap_id in
ttmp11.saved_trap_id[7:0].
Save status.halt in ttmp11.saved_status_halt, so that it can be
restored when resuming a wave (changing a wave's state from stopped to
running or single-stepping).
Change-Id: I7322f59b60e8cc1b92bf5f067dba606a3109ef49
To support single stepping the instruction preceding an s_endpgm,
unwind the PC by 8 bytes and set ttmp11[9] to notify the debugger
that the wave is halted with a modified PC.
Bump the debug r_version for this new trap handler ABI.
Change-Id: I55e4e0d65576f92da14a336266c31c513baab547
Each SE must be assigned equal numbers of slots and slots
must be assigned in units of whole groups.
Change-Id: I8f3677237fa6f2e2d25e3e78210c5a7a0ad792f3
For small copies cache flush latency is larger than data transfer
latency in local VRAM. Select SDMA for small copies.
Environment key HSA_FORCE_SDMA_SIZE is added for easy adjustment
of the small copy size. This may be removed after tuning is done.
Change-Id: I733fa0ae01c616617c5de50e71226b51fd589ef2
Make explicit reference to hsa_api_trace.cpp from
initialization of hsa_table_interface.cpp. Breaks
the ability to use hsa_table_interface.cpp in plugins.
Change-Id: I22a42d3a132512b0d9ec7a1ca629b169e7f8eba7
The excess declarations mark implemenation functions as default
visibility. Normally this is not an issue since our linker script
will specify which visible symbols will be permitted into the dynamic
symbol table. However, for static linking methods which apply linker
directives during incremental linking symbol visibility must be correct
in the (non-dynamic) symbol table.
Change-Id: I13dc8dd1019368e8943920d36335a91f0c555a92
The size of the m0 payload for MSG_INTERRUPT has changed in gfx10. It is
now 23bit wide instead of 24bit wide in gfx9.
Since we are generating different binaries for gfx9 and gfx10, we can
conditionally set DEBUG_INTERRUPT_CONTEXT_ID_BIT to 23 for gfx9 and
22 for gfx10.
Change-Id: Ifc15a9fa4399d35328ab58b742f791f1660bcd9a
- Make code object reader use mmap when loading from a file on Linux.
- Support computing code object URI for memory either fro the loaded
host executables, or from all mmapped files. Define the environment
variable HSA_LOADER_ENABLE_MMAP_URI to non 0 to search the mmap
files, otherwise only the loaded executables will be seatched.
- For mmap search, determine file size and ommit offset and size URI
fragment when the code object is the whole file even when specifying
a file size explicitly or specifying memory that has been mmaped.
- Always return a non-empty code object URI.
- When a code object reader is created, complete all fields to ensure
it can be used in a multi-threaded manner using only const
operations.
- Add missing exception handlers in the AMD vendor extentions.
- More rigorous checking for errors.
Change-Id: I07797b1dc60c5c64245142d77becf9f7c9643395
For static linking all non-API functions must have internal linkage
or be in project namespace to avoid colliding with application symbols.
Change-Id: I3398e40a387cca36f3ddef2b978306c765161d72
Save and restore exec_lo/exec_hi around the mGetDoorbellId macro in
the signal_error case just like we do in the signal_debugger case.
Also reset the wave_id (ttmp4/ttmp5) to 0 since it isn't preserved.
0 will be detected as a new wave by the debugger api library.
Change-Id: I5123caa9431154ec1584bae85e42648c97c64c37
Set ttmp11[8] and send a signal for the debugger when the handler
is entered because of an address watch exception.
Change-Id: Icc83a79027bb7ca1e50e19e2f00464cb9ca862f3
Adds the following:
- New factory method to create a code object reader from
file with offset and size.
- A pair of queries on a loaded code object to get the URI name/length.
- A bump to the AMD vendor loader extension API and its associated table.
Change-Id: I17c83e9c2447d29a43c438459395365f786a3611
Memory from the suballocator may be exported via IPC. If this
happens then the allocating process should not reuse that memory
since it would still be connected to the remote process. IPC exported
memory must be released back to the driver.
Change-Id: I2ab0c814f63191f753fc3640cc4140ee144bf07f
The 1st level trap handler jumps to the 2nd level trap handler on
context save requests or regular traps if (mode.debug_en
&& !status.halt) is true.
If we return from the 2nd level trap handler without status.halt=1, then
we need to make sure mode.debug_en is cleared, or we will re-enter the
2nd level trap handler again and again when trapsts.savectx is set.
Change-Id: I4db6369de8c91a32842f488a4df5c9d94fa65aa9
Sean Keely
Laurent Morichetti
Jay Cornwall
Konstantin Zhuravlyov
Tony Tye
Ramesh Errabolu