Report traps and fatal exceptions through a wavefront's
amd_queue_t.queue_inactive_signal. Previously, only traps were
reported and requireed the compiler to pass in the signal pointer
in s[0:1].
The signal is obtained through a mapping from doorbell index to
amd_queue_t*. The doorbell is fetched within a wavefront through
the gfx9+ S_SENDMSG(MSG_GET_DOORBELL) instruction.
Change-Id: I319b45f2e15dfcfe4db8f4065da1136e9539a42b
[ROCm/ROCR-Runtime commit: ff8f439112]
Assembler toolchains are moving from SP3 to LLVM. Replace trap handler
source code with LLVM equivalent.
Fix a trap issue with SQ_WAVE_IB_STS restore. Mostly harmless as all
traps are currently considered fatal to the wavefront.
Change-Id: Iacecd9dd31a1d96a083c8b8327f442f33c861f9f
[ROCm/ROCR-Runtime commit: 6ed686ee29]
Adds hsa_amd_register_deallocation_callback and hsa_amd_deregister_deallocation_callback
to notify when HSA memory has been released.
Change-Id: I1f33cee250ca890e5c2e7fddfa4479aa5874651d
[ROCm/ROCR-Runtime commit: 299874f17d]
CPUClockCounter is not NTP adjusted (CLOCK_MONOTONIC_RAW) so should be
better for measurements. However, it is implemented with syscall while
CLOCK_MONOTONIC is implemented via vDSO. The latency increase becomes
significant when language layers make corresponding clock measurements.
Reverting to CLOCK_MONOTONIC will reduce latency and allow small
duration events to be measured at the cost of incorporating NTP
frequency skew errors. NTP may adjust frequency by 500ppm so limits us
to ~3 decimals in elapsed time.
Change-Id: I920b9f707f47109d80d6c256c475638c03fb8d76
[ROCm/ROCR-Runtime commit: 4b22d24346]
API is a stateless lookup of RO data and needed to interpret
hsa_init error codes.
Change-Id: If80cba2f697843d08e529da0f790acf3c37127a7
[ROCm/ROCR-Runtime commit: 22de0e7fb9]
Joined threads can not be joined more than once nor can they be detached.
Thread library wait and close allows multiple waits and separate close so
this fixes the pthread implementation.
Change-Id: I0019271a438f11ed4c6c11854011f5c4f6e16b65
[ROCm/ROCR-Runtime commit: a913549190]
Small times may be given to time conversion if GPU clocks are used to
accumulate elapsed time. Because HSA APIs deal in absolute time this
leads to large conversion offsets of order system uptime. Variation
in relative clock ratio estimation may be amplified in this case,
destroying elapsed time measurements.
This patch fixes the relative clock ratio used for times which predate
the call to hsa_init. This correlates errors in such times allowing
the elapsed time to be correctly computed.
The effective maximum system uptime before elapsed time conversion becomes
inaccurate is ~3.5 months. GPU event timestamps are good for process uptime
of ~3.5 months. These are limited by double's mantissa precision.
Change-Id: I48752ff354920439d91016d6f2b0c8ddfa60b445
[ROCm/ROCR-Runtime commit: 6e2a056e1b]
Exposed via agent info query. Only valid if fine grain PCIe memory is enabled.
Change-Id: Ib4770901592ec047276458926a947737f9b93bb5
[ROCm/ROCR-Runtime commit: 06376e726b]
At the moment it is not possible to build ROCr with Clang. This is
a spurious limitation. The present PR addresses it by guarding GCC
only flags and by fixing some additional warnings that Clang triggers;
one of said warnings did outline a rather interesting issue with math
being done on void*s. - AlexVlx
Void ptr arithmetic had already been fixed in amd-master branch.
Change-Id: I5ee97e20b5c40b10dd73facecabe75f02ba46462
[ROCm/ROCR-Runtime commit: e89f9807f1]
Non-paged memory can be IPC-shared even when HSA_USERPTR_FOR_PAGED_MEM
is enabled.
Change-Id: I8b1fa6d7a4a9327c78a77b3679697fbf55397093
Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com>
[ROCm/ROCR-Runtime commit: 0c6b9532d4]
KFD no longer reports MemoryAccessFault.Failure with retry fault
implementation. ROCr ignores the memory event when Failure = 0.
Use the Flags field instead, which will be non-zero when the
event is triggered.
Change-Id: Ie90799a303b0b2f1b476b20ffafdde79ae137182
[ROCm/ROCR-Runtime commit: 56f280c8a7]
Makes malloc memory accessible to GPUs so that the memory has the
capabilities of the pool it is locked to.
This admits fine grained locked memory and reserves API space for any future
special CPU pools.
Change-Id: If8c3dd8582a43f19d3d36b3763c1a688cc419ef0
[ROCm/ROCR-Runtime commit: a535e18cc1]
GCC allows arithmetic on void* treating void as char. Clang and
the language spec does not.
Change-Id: I939f2432f276979bb81881406e10528597ac6001
[ROCm/ROCR-Runtime commit: e5de33dd9a]
Modify the system event handler to support multiple users.
Name memory fault reason codes.
Change-Id: I1b5979b36ab15637eb2be59a61e2d57e76d0a70e
[ROCm/ROCR-Runtime commit: 67376e06ab]
Part 1 of 2.
Enables fine grain vram over PCIe based on env flag.
Part 2 will extend to XGMI.
Change-Id: I8ad506e004b398d56d462b0200274eae2293a461
[ROCm/ROCR-Runtime commit: c56d86100b]
hsa_exceptions with empty what() strings will not report in debug builds.
Change-Id: I0d424d3b1d3044808ece1720a460a57d68bf878e
[ROCm/ROCR-Runtime commit: 344d964f9f]
Both support dynamic scratch allocation so there is no reason
to preemptively allocate on APUs.
Change-Id: I22eaec01a83a091ee9dc1f594a1a9106e8dd81fc
[ROCm/ROCR-Runtime commit: 65d39cc476]
HW has limited bits for wave scratch base address stride. Enforcement
prevents programs with larger than supported scratch allocations from
running and clobbering neighboring scratch space.
Change-Id: I574da888e9d1d5e290a9c0025ba13b5ef9f1e5c0
[ROCm/ROCR-Runtime commit: 8e4177382a]
Also rename blit_agent to region_gpu and add comments to clarify
its role in deprecated region API support rather than to do blits.
Change-Id: I80b1043db2e1c5d40a58fc801eef70a688ea9169
[ROCm/ROCR-Runtime commit: 936ecd1885]
During registration we must not call any function that depends on registered
data as the lists are not yet complete. This includes signal allocation since
allocating shared GPU mapped memory depends on the list of GPUs.
Change-Id: I94d59e847802c546c2a5a0d9f55fe5ac3fd1d878
[ROCm/ROCR-Runtime commit: dda9c17b45]
Delete the runtime object when the last hsa_shut_down occurs.
Change-Id: I2005d52d06702eaef166714fd5e471cc277924db
[ROCm/ROCR-Runtime commit: 9ec37b5103]
Debug agent requires handles to internal queues for single step debugging.
Added tools only API hsa_amd_runtime_queue_create_register for reporting.
hsa_amd_runtime_queue_create_register sets a callback which is invoked
when internal queues are created.
Change-Id: Ia5190ae724fadba686c15f25b2cd085350eeff0e
[ROCm/ROCR-Runtime commit: 757502ccd6]
Required for debug agent requires copy API and trap handler to be initalized
prior to loading. Existing tools do not make use of internal queue or scratch
memory intercept which is what PostToolsInit allows.
PostToolsInit() will be removed in a following cleanup change.
Change-Id: If43377843808e3eff0defd9204910a67a852902f
[ROCm/ROCR-Runtime commit: 5975c465ad]
Apertures now overlap with the change to 48bit addressing which
precludes using aperture checks to discover buffer ownership.
Switches to ptrinfo to decide which device a buffer owned by.
This corrects faults in the legacy hsa_memory_copy api.
Change-Id: I5c7ce0216e1cdc96f836fc6fec9c3defdf4b9d90
[ROCm/ROCR-Runtime commit: 1e0d690948]
LinkInfo is already initialized to zero in its default constructor.
Change-Id: Ifa4fb886cce9b474c6879c9c82744044ab394082
[ROCm/ROCR-Runtime commit: 2843988dd7]
Remove fence pool and use two signals. Two signals allows overlapped
submission and copy while reducing thread busy polling.
Change-Id: Idb5f8e4c7f482a596ffce9e7799191fdd785a216
[ROCm/ROCR-Runtime commit: 56ed5c8904]
Fix pitch overflow due to small element detection.
Add wide pitch 2D copy handling.
Cleanup code duplication.
Change-Id: I93b1584aba8e5964957eb7ab3544df806ca3e2f9
[ROCm/ROCR-Runtime commit: e0839ab27e]
Can only check that the signal has some time stamp, can't check if
the translating agent matches the last used agent or not.
Change-Id: I62943a864318808059c617280bb65a269dfadd1b
[ROCm/ROCR-Runtime commit: aca00b7238]
Adds HSA_AMD_SYSTEM_INFO_BUILD_VERSION=0x200 to hsa_system_info_t.
This returns a const char* pointing at the build string (git describe).
Change-Id: I73e6612482bf6ffc4037fd365808eb9211a650ad
[ROCm/ROCR-Runtime commit: cd8e5c1da8]
Adds env flag HSA_REV_COPY_DIR. If set to 1 async copy will
copy from dst device to src device rather than from src to dst.
Change-Id: I3095642066fa026dc112c2eac06db9393341cd7e
[ROCm/ROCR-Runtime commit: 6c47780620]
Jay Cornwall
Sean Keely
Felix Kuehling
Ramesh Errabolu
Konstantin Zhuravlyov
Evgeny