Commit Graph

353 Commits

Author SHA1 Message Date
Laurent Morichetti ea6ee0aa81 New trap handler ABI (v5)
Park the wave, if it is stopped, to avoid halting it at an s_endpgm
instruction if the architecture does not support it.

Free ttmp6 by converting the dispatch_ptr into a queue packet index
(25-bit) and storing it in ttmp7[24:0].

Save the exception PC in ttmp11[22:7] ttmp6[31:0].

Change-Id: Iaa3c5baf5b488c0b534044d338f12bffa63ddce2
2021-03-04 21:44:14 -05:00
Laurent Morichetti 7e0f391a08 Correct the trap handler
ttmp11 no longer has an "excp_raised" field.

Change-Id: I8e673ca404c2b802470bbc9f76e7925782076c5a
2021-03-04 21:21:26 -05:00
Sean Keely 191664cd20 Insert scratch memory into scratch cache on full profile systems.
Scratch cache was not updated for IOMMUv2 systems previously.
This both negates the cache and causes segfault during scratch
release.

Change-Id: I71e81d6b642d65ca135868ff7225ea173529d458
2021-03-03 21:30:16 -05:00
Jason Tang ec22afb8a8 Correct GetIsa() typo
Change-Id: Ia6b5a86bd035fb077f0da9d52160ec8d12987b87
2021-02-17 11:57:58 -05:00
Sean Keely 34ac62274a Correct legacy copy path.
Legacy p2p copy path incorrectly transfered in whole pages rather than
the requested size.

Change-Id: I9aa7337754f9e32f587a0cc5305f8ffeb6196f10
2021-02-10 19:53:02 -05:00
Sean Keely 01f42dbe46 Add hsa_amd_signal_value_pointer.
Enables partial signal interop with non-HSA devices.

Change-Id: Ic39bca84ed1709cbd2cc24b1eb0f4fc6cccb39cf
2021-02-10 18:47:54 -05:00
Laurent Morichetti 9ca79d072a New trap handler ABI (v4)
Replace the stop reasons ttmp11.trap_raised and ttmp11.excp_raised
with ttmp11.wave_stopped which indicates that the trap handler has
halted the wave as the result of an event (trap, single-step or
exception).

If the wave is stopped because of a trap, also record the trap_id in
ttmp11.saved_trap_id[7:0].

Save status.halt in ttmp11.saved_status_halt, so that it can be
restored when resuming a wave (changing a wave's state from stopped to
running or single-stepping).

Change-Id: I7322f59b60e8cc1b92bf5f067dba606a3109ef49
2021-02-05 09:56:01 -08:00
Huang Rui feeb2f62e2 Add gfx10.3.3 ISA support for Van Gogh
This patch is to let ROCr recognize new gfx10.3.3 ISA.

Change-Id: Ied23eee2752e14c19c8c0a6d7789fded9940e31e
Signed-off-by: Huang Rui <ray.huang@amd.com>
2021-01-22 04:22:15 -05:00
Laurent Morichetti 8aec53969f Don't terminate waves halted at s_endpgm
To support single stepping the instruction preceding an s_endpgm,
unwind the PC by 8 bytes and set ttmp11[9] to notify the debugger
that the wave is halted with a modified PC.

Bump the debug r_version for this new trap handler ABI.

Change-Id: I55e4e0d65576f92da14a336266c31c513baab547
2021-01-21 20:51:38 -08:00
Laurent Morichetti 8808ed3177 Correct gfx10.3+ trap handler.
Change-Id: I77d2b41c8882014a430d741ecd777718a1f61561
2021-01-21 09:24:20 -08:00
Tony Tye 26fe26e415 Correct isa lookup for targets that do not support a target feature
Change-Id: I130070a53162e5d9fcc6a64a4bdda7869179be82
2021-01-18 15:47:19 +00:00
Chris Freehill 09bc75bf0d Correct some target ID strings for gfx908
Change-Id: I7833b561447b9928447cf49472cfe1ca1867e71d
2021-01-15 14:56:38 -06:00
Sean Keely 7bc6aac5d2 Correct computation of scratch slot requirements.
Each SE must be assigned equal numbers of slots and slots
must be assigned in units of whole groups.

Change-Id: I8f3677237fa6f2e2d25e3e78210c5a7a0ad792f3
2021-01-13 15:09:00 -05:00
Sean Keely 9fe8ccc3ee Revert "Revert "Cache scratch allocations.""
This reverts commit 7e2ba23566.

Change-Id: I3f3c257270016559f8b2e70151664f0931db28d2
2021-01-13 15:08:53 -05:00
Tony Tye 6bbf6b1c9c Improve Isa class
- Use consistent naming in Isa class.
- Remove unused Isa methods.
- Simplify Isa methods.

Change-Id: I7c4045d08fbfe0d94b3181db8ebc5e5ed8c8cc82
2021-01-10 18:23:54 +00:00
Tony 853ccc762e Store target ID in isa registry
Store target ID string in isa registry and use for returning agent and
isa name.

Change-Id: I72a20d8ff963c73d86392158aff3853e4c9bfdbd
2021-01-10 18:23:54 +00:00
Tony 12eb2764cd Correct code object V2 support
- Remove gfx800, gfx804 and gfx901 as they do not exist.
- Map the V2 note record of "AMD:AMDGPU:8:0:0" to gfx802 as they are
  the same target just connected to a differnt motherboard.
- Correct typo for supporting gfx902:xnack+.
- Support agent names with a minor or stepping version greater than 9.

Change-Id: Ife933449f60ab4687e2aaab9baf4c9fc5b86339d
2021-01-10 18:23:54 +00:00
Sean Keely 7e2ba23566 Revert "Cache scratch allocations."
This reverts commit 27e044ae4d.

Change-Id: I698b33bacb2be3de6c8185fe89597a60a79521c5
2021-01-08 11:57:40 -06:00
Sean Keely d39ae13420 Add support for gfx1032.
Change-Id: I36f93a6b61e74cf17aac1a05d7c1d4ba6369fcc9
2021-01-05 17:28:19 -06:00
Tony b443397bcc Make supported targets consistent
Add missing target names and make all parts consistent with which
targets are supported.

- Add gfx805 as a supported target.

- Add all ELF targets to genric code.

- Make offline loader match supported targets.

Change-Id: Idab4d69edc71645aecaa83aa55e29c1aeee4c1d6
2020-11-24 03:14:31 +00:00
Sean Keely 27e044ae4d Cache scratch allocations.
Avoids calling to KFD to map/unmap scratch allocations for
every large scratch using dispatch.

Change-Id: I9fab5705251ec82b03e4f2f2ca6da7cdccabefb9
2020-11-20 15:07:01 -05:00
Sean Keely 32d0fcafa9 Limit clock synchronization to 16Hz.
Improves HIP event performance in directed benchmarks where
clock sync latency is significant.

Change-Id: I78b724a14a8f5b6a9a2b9f4d85afe9d8b81808a6
2020-11-20 15:06:13 -05:00
Sean Keely b51f68b535 Style update for SDMA enable flag.
Updated to match xnack flag's style.

Change-Id: I6115c0b53660d789e698de1606a9388ae1789866
2020-11-20 15:06:02 -05:00
Konstantin Zhuravlyov 3a08d0964e Implement Target ID Proposal
Changes from Konstantin Zhuravlyov, Tony Tye

Change-Id: I532801193afa9d5b8ac2a877b5497eab661f0597
2020-11-10 13:42:35 -05:00
Sean Keely a09ba8bcc8 Diable sram ecc reporting.
Temporary workaround while language and compiler teams sort out
handling both modes.

Change-Id: I5d676cd546382dba05ec0b62bb885baa854614f6
2020-10-20 17:06:30 -05:00
Sean Keely 9192dfe1b0 Initialize intercept queue packets properly.
Change-Id: I0ff1540940665409a9ade3a517dd576a8f334c7b
2020-10-08 15:33:43 -05:00
Sean Keely a3c4aaf95a Correct return type error in hsa_amd_signal_wait_any.
The error checking macro IS_OPEN returns an hsa_signal_t.
This conflicts with the return type of uint32_t.

Add an assert and rely on spurious return rule to return zero
when rocr is not initialized.

Change-Id: Ifc9bb75e22ecdd675273de59b31e5026a69c62e0
2020-09-25 21:33:23 -04:00
Chris Freehill 4944c74189 Add gfx1031 support
Change-Id: I855f7fe8d096331d0c1da10b10adf6b1e75a527f
2020-09-10 11:06:58 -04:00
Sean Keely 2a0c6774fb Use SDMA for small copies in VRAM.
For small copies cache flush latency is larger than data transfer
latency in local VRAM.  Select SDMA for small copies.

Environment key HSA_FORCE_SDMA_SIZE is added for easy adjustment
of the small copy size.  This may be removed after tuning is done.

Change-Id: I733fa0ae01c616617c5de50e71226b51fd589ef2
2020-09-03 03:11:57 -05:00
Sean Keely 5f43778a51 Convert from double to uint64_t in two steps.
We want wraparound behavior here but we don't want to trigger sanitizer
warnings.  Converting to int64_t and then wraping around by cast to
uint64_t avoids the UB issue that triggers the sanitizer warning.

Change-Id: I9400b988dce7899e9ba42cab3e35c7ffedec8fe1
2020-08-25 20:12:52 -05:00
Sean Keely dc7e5e7e46 Add xnack isa recognition to gfx1030.
Change-Id: I99301a62f1952b6a3cc548272f4129ad8c0542da
2020-08-12 16:34:17 -05:00
Chris Freehill e702531b40 Add gfx1030 support
Change-Id: I4bccc731ba802480925f98c6c42593503bf9b98d
2020-08-12 16:34:10 -05:00
Sean Keely f4fe7ddf47 Make explicit reference between init modules.
Make explicit reference to hsa_api_trace.cpp from
initialization of hsa_table_interface.cpp.  Breaks
the ability to use hsa_table_interface.cpp in plugins.

Change-Id: I22a42d3a132512b0d9ec7a1ca629b169e7f8eba7
2020-07-15 16:02:15 -04:00
Sean Keely f6e6eae86d Remove unnecessary HSA_API declarations.
The excess declarations mark implemenation functions as default
visibility.  Normally this is not an issue since our linker script
will specify which visible symbols will be permitted into the dynamic
symbol table.  However, for static linking methods which apply linker
directives during incremental linking symbol visibility must be correct
in the (non-dynamic) symbol table.

Change-Id: I13dc8dd1019368e8943920d36335a91f0c555a92
2020-07-07 16:41:34 -04:00
Tony ac3244d431 Define URI for deprecated hsa_executable_load_code_object
Change-Id: Id9b6f8895bdafa3460a27984e0f773ec153378d7
2020-07-01 23:02:54 -04:00
Laurent Morichetti 23df617150 Set the correct bit in m0 for gfx10
The size of the m0 payload for MSG_INTERRUPT has changed in gfx10. It is
now 23bit wide instead of 24bit wide in gfx9.

Since we are generating different binaries for gfx9 and gfx10, we can
conditionally set DEBUG_INTERRUPT_CONTEXT_ID_BIT to 23 for gfx9 and
22 for gfx10.

Change-Id: Ifc15a9fa4399d35328ab58b742f791f1660bcd9a
2020-06-30 22:02:41 -04:00
Ramesh Errabolu f7350c6020 Update ROCr implementation of Queue ID
Change-Id: Iec48b1978e4d01563e71cfb58aed8f1bbc446443
2020-06-26 13:25:00 -05:00
Sean Keely cc7a197ff8 Use correct queue id for gws assignment.
Change-Id: I9157951abce65b60dfe0e1db5819510a7703547f
2020-06-26 14:24:06 -04:00
Tony 91cb98dab6 Code object reader improvements
- Make code object reader use mmap when loading from a file on Linux.
- Support computing code object URI for memory either fro the loaded
  host executables, or from all mmapped files. Define the environment
  variable HSA_LOADER_ENABLE_MMAP_URI to non 0 to search the mmap
  files, otherwise only the loaded executables will be seatched.
- For mmap search, determine file size and ommit offset and size URI
  fragment when the code object is the whole file even when specifying
  a file size explicitly or specifying memory that has been mmaped.
- Always return a non-empty code object URI.
- When a code object reader is created, complete all fields to ensure
  it can be used in a multi-threaded manner using only const
  operations.
- Add missing exception handlers in the AMD vendor extentions.
- More rigorous checking for errors.

Change-Id: I07797b1dc60c5c64245142d77becf9f7c9643395
2020-06-25 12:18:50 -04:00
Sean Keely 71d85855d7 Re-enable EDC reporting.
Change-Id: Iccd2532e22323ff5ba4c411b169fcda885968059
2020-06-23 18:23:50 -05:00
Sean Keely e9a4eff8a1 Update licensing and remove duplicate licenses.
Change-Id: I0aab6f310d96bf6c5a918e7a9c03713a00dc5c4a
2020-06-22 14:19:30 -04:00
Laurent Morichetti 4f1a9c8139 Don't reset queue_address/queue_size
We should preserve the queue address and queue size when suspending a
queue. The ROCdbgapi gets all its information from the KFD, and if we
reset the queue properties, a call to queue snapshot will not return
the correct information.

Change-Id: I646ed4a779cada55b0c1b4342ec140d3678beb67
2020-06-20 01:22:52 -04:00
Sean Keely bd51c61af8 Move tools only table interfaces into namespace rocr.
Change-Id: Ic0b8d958c2d27c921c6955a56110c6cdf5ba5e8e
2020-06-19 22:35:15 -04:00
Sean Keely 0be21c54a6 Move loader interface functions into rocr namespace.
For static linking all non-API functions must have internal linkage
or be in project namespace to avoid colliding with application symbols.

Change-Id: I3398e40a387cca36f3ddef2b978306c765161d72
2020-06-19 22:34:26 -04:00
Ramesh Errabolu fa13208698 Add rocr namespace to core header and impl files
Change-Id: I1e1b33f9bba1078d049bc19797889988c3e43360
2020-06-19 22:34:21 -04:00
Sean Keely 99e2942949 Fix improper Min/Max operands.
Operands were relying on implicit type conversion which may result
in range errors.

Change-Id: I4b35ef92dcaf8b547aed02fea36aed75d392c6af
2020-06-19 22:34:16 -04:00
Sean Keely ce19721c88 Update copyright date.
Change-Id: If4bf4c20cf051878bfe759080bb7345d884dd53d
2020-06-19 22:34:01 -04:00
Sean Keely 55a4f01b16 Update to modern cmake.
Adds support for find_package(), locates dependencies with
find_package(), swaps the roles of /hsa/include/hsa and /include/hsa
as well as /lib & /hsa/lib.

Kernel code objects no longer build at every make call but only
as needed.  Dependencies are tracked through to clang.

Device lib is still located with directory searches.  build_devicelibs.sh
does not yet install the cmake config files on the build systems.

Corrects DAZ mode mismatch in code object compilation.

Still needs updating to compiler properties rather than direct
manipulation of CMAKE_CXX_FLAGS.

Change-Id: I02d946c8a77d5cf753681f8e3d3153fca4aae86a
2020-06-19 22:33:36 -04:00
Ramesh Errabolu 0ca0691ca7 Build ROCr core and image libraries as one shared object
Change-Id: I3a16c1227e7db2e386ab33886965596fa0fb0c87
2020-06-19 22:33:36 -04:00
Laurent Morichetti db6a781f0c Preserve the exec mask for the queue error case
Save and restore exec_lo/exec_hi around the mGetDoorbellId macro in
the signal_error case just like we do in the signal_debugger case.

Also reset the wave_id (ttmp4/ttmp5) to 0 since it isn't preserved.
0 will be detected as a new wave by the debugger api library.

Change-Id: I5123caa9431154ec1584bae85e42648c97c64c37
2020-06-18 02:25:11 -04:00