New environment variable HSA_CU_MASK allows users to
specify a cu mask to every queue allocated from any
GPU. hsa_amd_queue_cu_set_mask is restricted from
escaping this mask.
A new API hsa_amd_queue_cu_get_mask is added to query
the current cu mask.
Change-Id: I846c03a5faaca9b95067c31db84b59cc9fce2f03
[ROCm/ROCR-Runtime commit: 4455250be1]
Enables the fragment allocator to handle >2MB allocations, maintaining
good TLB alignment. Prior code contained a bug that caused the effective
API granule for vram allocations >2MB to be bumped to 2MB.
Also adjusts the block cache's block retention heuristic to not
count discarded blocks as in use. This will reduce block retention
when a significant amount of large blocks or IPC is in use.
Change-Id: I30bd85eb87951df822211f799d9cfe579ab109c6
[ROCm/ROCR-Runtime commit: 8adbda1c18]
Add macro debug_warning_n to stop printing a message after
N instances.
Change-Id: Id5f84b11eb63b3a20bd2bcb2ea8f10a066b457ef
[ROCm/ROCR-Runtime commit: ca8387768e]
Includes some workarounds and HMM.
Conflicts:
opensrc/hsa-runtime/core/runtime/amd_topology.cpp
opensrc/hsa-runtime/core/util/flag.h
Change-Id: I22976f07964a43dbb228a6231777dbd599112b8d
[ROCm/ROCR-Runtime commit: 7333c77e22]
Sramecc is misreported in kfd 4.0 and prior. To prevent possible
corruption due to d16 instructions, deny use of gfx906 with older
kfds and correct misreport for gfx908. Denial of gfx906 may be
overridden by setting HSA_IGNORE_SRAMECC_MISREPORT=1.
Change-Id: I7d5c3a716fad01c348f8b88cd508cedbf914c989
[ROCm/ROCR-Runtime commit: 45fbe5b192]
For small copies cache flush latency is larger than data transfer
latency in local VRAM. Select SDMA for small copies.
Environment key HSA_FORCE_SDMA_SIZE is added for easy adjustment
of the small copy size. This may be removed after tuning is done.
Change-Id: I733fa0ae01c616617c5de50e71226b51fd589ef2
[ROCm/ROCR-Runtime commit: 2a0c6774fb]
- Make code object reader use mmap when loading from a file on Linux.
- Support computing code object URI for memory either fro the loaded
host executables, or from all mmapped files. Define the environment
variable HSA_LOADER_ENABLE_MMAP_URI to non 0 to search the mmap
files, otherwise only the loaded executables will be seatched.
- For mmap search, determine file size and ommit offset and size URI
fragment when the code object is the whole file even when specifying
a file size explicitly or specifying memory that has been mmaped.
- Always return a non-empty code object URI.
- When a code object reader is created, complete all fields to ensure
it can be used in a multi-threaded manner using only const
operations.
- Add missing exception handlers in the AMD vendor extentions.
- More rigorous checking for errors.
Change-Id: I07797b1dc60c5c64245142d77becf9f7c9643395
[ROCm/ROCR-Runtime commit: 91cb98dab6]
Memory from the suballocator may be exported via IPC. If this
happens then the allocating process should not reuse that memory
since it would still be connected to the remote process. IPC exported
memory must be released back to the driver.
Change-Id: I2ab0c814f63191f753fc3640cc4140ee144bf07f
[ROCm/ROCR-Runtime commit: 29b660c91e]
Contructor function must not be attempted twice even if the construction
attempt returns nullptr.
Change-Id: I75353e5e511769a96e4332f7f60887f6559c1cd5
[ROCm/ROCR-Runtime commit: 2fbacccaed]
Setting to 1 prevents the scratch handler from reducing peak occupancy.
Scratch allocations that would normally reduce peak occupancy will
instead fail.
Diagnostic for TF and PyTorch.
Change-Id: I2d7ea47077eb5cf708251c8aa3fd183ad4261be0
[ROCm/ROCR-Runtime commit: dc165c92bc]
Joined threads can not be joined more than once nor can they be detached.
Thread library wait and close allows multiple waits and separate close so
this fixes the pthread implementation.
Change-Id: I0019271a438f11ed4c6c11854011f5c4f6e16b65
[ROCm/ROCR-Runtime commit: a913549190]
Part 1 of 2.
Enables fine grain vram over PCIe based on env flag.
Part 2 will extend to XGMI.
Change-Id: I8ad506e004b398d56d462b0200274eae2293a461
[ROCm/ROCR-Runtime commit: c56d86100b]
hsa_exceptions with empty what() strings will not report in debug builds.
Change-Id: I0d424d3b1d3044808ece1720a460a57d68bf878e
[ROCm/ROCR-Runtime commit: 344d964f9f]
Fix pitch overflow due to small element detection.
Add wide pitch 2D copy handling.
Cleanup code duplication.
Change-Id: I93b1584aba8e5964957eb7ab3544df806ca3e2f9
[ROCm/ROCR-Runtime commit: e0839ab27e]
Adds env flag HSA_REV_COPY_DIR. If set to 1 async copy will
copy from dst device to src device rather than from src to dst.
Change-Id: I3095642066fa026dc112c2eac06db9393341cd7e
[ROCm/ROCR-Runtime commit: 6c47780620]
Also improve small_heap used for scratch region allocation.
Change-Id: Ib7311b663b38968d88ebc355b81e12c0863dc541
[ROCm/ROCR-Runtime commit: 7caf9633f6]
Remove "zombie" queue state and report queue creation failure via
exceptions. Make Shared object a final container and support array
objects with Shared. Add message printing to hsa_exception in
debug builds.
Change-Id: I459f38c80846018acbf45538874e95f91dd6b195
[ROCm/ROCR-Runtime commit: f312a7386e]
Queue intercept is exposed as two tools-only APIs via the API
intercept table.
Change-Id: Iac9602ed3143974d85c3569e9092295ad18037f8
[ROCm/ROCR-Runtime commit: 0c7dde2d1f]
GCC can't reasonably be told that the lock ptr isn't null. Adding a private bool
allows the branch to be eliminated, along with the bool.
Change-Id: I0605d69474d6a6e6951be93c0af1d8caf3f77124
[ROCm/ROCR-Runtime commit: 9dfdce5b3c]
Track pointer info for sub 2MB fragment allocations in allocation_map_.
Add fragment support to IPC.
Change-Id: I00cfc2e2fa289aac90a4718c392f9bb056a61a87
[ROCm/ROCR-Runtime commit: 117be0b55a]
Sean Keely
Tony
Ramesh Errabolu
Christian Sigg
Evgeny
Hari Thangirala