Minimum queue size was not enforced at the Agent level. Minimum
size should be one page to give unifority across all asics.
Change-Id: I26394f79458d09fbceb79fc8aaf495e2c26a8ff3
[ROCm/ROCR-Runtime commit: a6742209f7]
Adds asserts for invalid dispatch dims and scratch requests that
don't actually use scratch.
Change-Id: I6e6eef3f17dc38adaf96550fa55bd8625868efa3
[ROCm/ROCR-Runtime commit: a65f3f5b71]
On gfx90a only a reduced number of CUs must be used for cooperative
dispatches due to CWSR and launcher interactions with asymetric
harvest. We must use one fewer CUs per SE than the lowest count of
CUs on any SE.
Also adds env var HSA_COOP_CU_COUNT which enables the cooperative
CU count computation. Set to 1 to enable the new computation.
This is an opt-in feature that will become enabled by default (opt-out)
in a future release.
Change-Id: Ifbb75ced3bbc15876eef44922c6a4f6fde8c4c28
[ROCm/ROCR-Runtime commit: 37942c982a]
Corrections have been made in libhsakmt, and corresponding changes are required here as well.
Signed-off-by: Chen Gong <curry.gong@amd.com>
Change-Id: Ib697ce25278c2c5ac6ef0206930ec285f46c60d1
[ROCm/ROCR-Runtime commit: dec63b4f15]
The start iterator becomes invalid after it is removed from
std::map prefetch_map_. This was causing a segfault when the iterator is
incremented afterwards.
Signed-off-by: David Yat Sin <david.yatsin@amd.com>
Change-Id: I4b0b763d2cb4ee99c0b8571c2c526b834e74077a
[ROCm/ROCR-Runtime commit: 86164fbfec]
ROCM_VISIBLE_DEVICES was used where ROCR_VISIBLE_DEVICES was
intended.
Change-Id: I644a546f3c9dd0b50898ef8a21dbb8f5c3a36926
[ROCm/ROCR-Runtime commit: fce6ba052e]
Prior solution used a single global lock to protect the memory tracking structures.
This change protects the memory tracking structure with a shared mutex (rw lock) in
shared (r) mode for memory allocations and frees so that long duration processes,
calling to kfd, can be done in parallel. Operations which must modify the memory map
take the mutex in exclusive mode (w) and must not call to the thunk while holding
the mutex.
The fragment allocator now requires separate protection and is protected with a
mutex at the device level. Protecting at the device level, rather than pool,
allows retention of the current recursive design and allows calling Trim from
withing Allocate. This could be made finer (pool level locks) but would
require backing out of Allocate entirely to call Trim. Trim and any retried
Allocation must be done in isolation (per device) or we may report OOM when
memory is actually available in some pool's fragment cache. So some device
level serialization is required in at least some paths.
Change-Id: I7c1e94d6965ffcc602b12fefdd3a6e97b84b5e00
[ROCm/ROCR-Runtime commit: df55cb0450]
Comments call out the specific operation being selected since the
ternary nest is a bit hard to read.
Change-Id: If033dbaa6cba132e96196ad3fc6d5572042041f4
[ROCm/ROCR-Runtime commit: fc75731034]
Agent temp array could have leaked if one of the given agent
handles was invalid.
Change-Id: I9e638b3a4f6bb917a4e3209ad81a1253bb603365
[ROCm/ROCR-Runtime commit: b198016949]
Argument must be checked for nullptr before being dereferenced and
filled with the default return value.
Change-Id: I9ff366f066a5e18c78129bf59cc3ba00fca3ef18
[ROCm/ROCR-Runtime commit: f48a786662]
This omission did not cause problems earlier due to having not been
instanced.
Change-Id: I7a54f82e06c299902f3bf6b4d3737cc5e30961ad
[ROCm/ROCR-Runtime commit: 322588a60e]
Size of the node map was used as the max node id previously. This
is wrong when RVD is used.
Change-Id: Ic632ec96891b92186e5b68cd53f81414db34f59f
[ROCm/ROCR-Runtime commit: 19454fcf26]
Was size of the map. Needs to be size of the node id range.
Change-Id: I92501ea7adca5c30dbb0fdabd2c421dea58f8d6f
[ROCm/ROCR-Runtime commit: c9eb85e205]
The event id assists KFD in locating the proper event associated
with the interrupt.
Change-Id: I75d58b6be74dd5b1edb0c5fe2b9d01538a649ba1
[ROCm/ROCR-Runtime commit: d65e00bcc5]
This really should be set to conform to distro standards.
Signed-off-by: Jeremy Newton <Jeremy.Newton@amd.com>
Change-Id: I8c3bdcc7eb103cec9db6aa9f9cfec25754784be8
[ROCm/ROCR-Runtime commit: 48e4e2c5ff]
Clang warns about bitwise operators on bools. Cast to int silences
the warning without introducing short circut logic.
Change-Id: I6e25138e1acf4a5562d3925ea5b2fcef3addb783
[ROCm/ROCR-Runtime commit: 4b0c94cfe8]
Would be nice to get warning count changes highlighted in CI though.
Clang's increasingly suspect diagnostics has caused multiple build
breaks without highlighting any actual issues.
Also: https://embeddedartistry.com/blog/2017/05/22/werror-is-not-your-friend/
Change-Id: I7dc82da58cd86f7b4f1a9fb511c4c039419271d4
[ROCm/ROCR-Runtime commit: efeee734db]
Limits CU masking application to cases where it is explicitly requested.
Change-Id: Ib65ad0ac98f86d840c0328fa15ce40c05cd4bfae
[ROCm/ROCR-Runtime commit: 5e8d261352]
Due to a CPACK bug the package needs to remove header file
symlinks. Cleanup is required for uninstall and upgrade
since each release installs to a different folder.
Change-Id: I5ec378b21e69235404781c7bce3c0203eb38eed1
[ROCm/ROCR-Runtime commit: ca899ea429]
KFD topology has been corrected and the defaults used by this
workaround are no longer true for all chips.
Change-Id: I0242d8077e9666ed1cf0dc3985244258ae5c0924
[ROCm/ROCR-Runtime commit: 19c1e92b4c]
Early exit if the range is found to be fine grain. Indeterminate
should only apply if the range is neither coarse nor fine.
Change-Id: I54133e14f4e8cfa53e2d612f6112cdcdb5a47dfa
[ROCm/ROCR-Runtime commit: a2fb1cbfbc]
Because of sharing ports with other engines, the
hardware design team has advised that SDMA0 on gfx90a
should only be used for host-to-device data transfers.
The recommendation is to use SDMA1 for any device-to-device
or device-to-host data transfers.
A driver change will ensure that, for each gfx90a
device, only the first PCIe SDMA queue a process
requests will possibly be from SDMA0. This patch ensures
that the first PCIe queue requested (which may be from
SDMA0) is always set up for host-to-device.
Change-Id: I6793ca95596dedaed9d5be1dbd9469ceef2a5c33
[ROCm/ROCR-Runtime commit: 280a458d0c]
Bumps cmake minimum version to 3.7 for version comparison operator.
Previously the Clang cmake project version strings were used. These
are not defined if the clang cmake project has not been loaded.
We should use CMAKE_CXX_COMPILER_VERSION to check the version when
only the compiler binary is redirected and the project files are
not available.
Also adjust device libs lookup logic to handle multiple paths in
CMAKE_PREFIX_PATH.
Change-Id: I67b6958d8241685cd6c3a0af68507c9fdc6331ef
[ROCm/ROCR-Runtime commit: 2e9a9f7c7a]
Was hard coded to false. Updated to reflect f16 availablity since
gfx8.
Change-Id: I7d5b9792c8e0163199c421a61b5d49b25cd98645
[ROCm/ROCR-Runtime commit: 5535b1f86f]
For minimal latency we should place command queues and blit code
in the nearest numa node to each GPU. Add an allocator matching
the current runtime default allocator interface to each GpuAgent
that allocates on the closest numa node as represented by kfd
topology. Use this allocator for queue ring buffers and blit
objects.
Change-Id: I181127f9c27bafe68976312963146616e3f58369
[ROCm/ROCR-Runtime commit: 5af558f739]
Also make failure to handle queue errors fatal.
Motivation is to improve detection of queue error conditions
that currently appear as application hangs.
Change-Id: I655643616dc0bd303d7df3ce8aca2c099bec3d46
[ROCm/ROCR-Runtime commit: 907679c989]
Sets package found and component lists. ROCr does not have components
so this is mostly cosmetic. It's part of maintaining a compliant
cmake project config file though.
Change-Id: Ida2ef746375143babd3a6f938727a47135606f01
[ROCm/ROCR-Runtime commit: 9e7d4629ca]
Set handler state to terminated before exiting.
Also simplify scratch handler exit loop.
Change-Id: I0a80c8a1899e8b60a6e7aa6989ba28de42ba31e7
[ROCm/ROCR-Runtime commit: e06dd39d89]
Per clang 13 option -Wno-error=unused-but-set-variable is not
recoginized nor is the diagnostic emitted. Set this option
conditional to the clang compiler version.
Change-Id: I3c0958dffa985d53b641f9eff4e702988dffd033
[ROCm/ROCR-Runtime commit: eec545fd9f]
Passing 0 into num_cu_mask_count used to be an implicit error.
This has been repurposed as a short hand for enabling all CUs.
Enabling all CUs when HSA_CU_MASK is set will cause the CU mask to
reset to whatever was set by HSA_CU_MASK which may then be queried.
Change-Id: I1d6bb2034595a78ee48fa72aa05563e8ea6c0fff
[ROCm/ROCR-Runtime commit: 02666ec0f1]
Delay parsing until after GPU discovery. Use the surfaced
GPU count and maximum phyiscal CU count to limit parsed bit masks.
This prevents pathological input such as
HSA_CU_MASK=0-8000000:0-8000000 from attempting to consume 7TiB.
Change-Id: I3773d2db3740c2023b0f6275d1818b69119b0495
[ROCm/ROCR-Runtime commit: 2aa0795b33]
Take in const void* rather than void*. This does not break the
abi or existing code. Existing code would need to cast away any
const which is unnecessary and annoying.
Change-Id: I28787e8fab1b600bf6871ea82835e10a4f475c5b
[ROCm/ROCR-Runtime commit: 270d042ef8]
APIs have been added and minimum cmake version number has been
changed.
Change-Id: Ic75849e6937c04faed2d206344df2cf9e9a78016
[ROCm/ROCR-Runtime commit: ed7eec14e1]
Branches are unused and emit noise to the console when running
commands for which we have no actions.
Change-Id: I1f8c49a20bd7f529172721f35d29665cfc8dc6a4
[ROCm/ROCR-Runtime commit: e8439cca08]
Some strings were missing the human readable form of the error code.
Also unifying source formatting via clang-format.
Change-Id: I0bcc2ab77dda476904c684cc2c584a5c7e8230d4
[ROCm/ROCR-Runtime commit: 081ab00f8e]
HSA_STATUS_ERROR_INVALID_MEMORY_POOL was missing from
hsa_status_string.
Change-Id: I9a9121d54a61f966d87081a55638397473bddbe4
[ROCm/ROCR-Runtime commit: 62b7c0ed3b]
Threads may race against xgmi_peer_list_ when dynamically assigning
peers to sdma engines.
Change-Id: I300c10f0cfa0ff7d6a5515364070a0895e2f4644
[ROCm/ROCR-Runtime commit: bb4dfbba1e]
global_flags reporting allows discovery of an allocation's memory
model (coarse, fine, kernarg). This is critical on gfx90a and
also allows discovery of the memory model of IPC imports.
Change-Id: Icbc3c243ca20e264af5e1931becd2419f762c7ad
[ROCm/ROCR-Runtime commit: e3a01690a5]
Previously ranges were reported as fine if and only if they were
entirely fine. Coarse and mixed ranges were reported as coarse.
For gfx90a it is critical to know if a range is coarse or fine as
fp atomics targeting fine do not function. Range queried reporting
coarse must be able to be trusted so must only report coarse if the
entire region is coarse.
Change-Id: I29c654a2afcd6943961eb2455e3654dfdb1283b5
[ROCm/ROCR-Runtime commit: e6e66e8a05]
Sean Keely
Chen Gong
Jeremy Newton
David Yat Sin
Freddy Paul
Oak Zeng