Commit Graph

51 Commits

Author SHA1 Message Date
Ramesh Errabolu ccd4e85fc9 Extend Rocr Visible Devices functionality to include UUIDs
Change-Id: Ia2892e4033717556a422fe33dec0294fe2ca9e28


[ROCm/ROCR-Runtime commit: 89f7ef224c]
2020-04-09 00:42:53 -05:00
Ramesh Errabolu e8f4f2d9e2 Extend ROCr to surface UUID of GPU devices that suppport
Change-Id: I478db68d69a01578770403fa695f9e6391637573


[ROCm/ROCR-Runtime commit: 45958c727d]
2020-04-08 19:19:22 -05:00
Sean Keely 302c21ac31 Add env key HSA_NO_SCRATCH_THREAD_LIMITER.
Setting to 1 prevents the scratch handler from reducing peak occupancy.
Scratch allocations that would normally reduce peak occupancy will
instead fail.

Diagnostic for TF and PyTorch.

Change-Id: I2d7ea47077eb5cf708251c8aa3fd183ad4261be0


[ROCm/ROCR-Runtime commit: dc165c92bc]
2020-02-21 17:09:26 -05:00
Sean Keely 511b86a55e Allow disabling scratch reclaim.
Debug and RCCL NPI assist feature.

Change-Id: I2cb76f0a086fa341465df3ede26965ab713bc3b4


[ROCm/ROCR-Runtime commit: d2a50a0048]
2019-11-20 02:41:58 -05:00
Christian Sigg c28aadf5a8 Add missing include to lazy_ptr.h
Change-Id: I5b061692a4ec6def631d7c3182e5b644b6b9c519


[ROCm/ROCR-Runtime commit: 00b0ee15b3]
2019-09-05 02:44:27 -04:00
Ramesh Errabolu 08e994db50 Initial support for xgmi sdma queues
Change-Id: I1aee379c7b9eede5f4b913cf2f9af3abb32e5baa


[ROCm/ROCR-Runtime commit: 8864c188b4]
2019-08-24 02:03:37 -04:00
Sean Keely 5d5d40fcf9 PTHREAD_STACK_MIN may differ from system parameters.
Restrict stack adjustment to non-default stack requests and allow
stack growth within reason (20MB cutoff).

Change-Id: I320280c711402ac29683e94c7246b7c32c797611


[ROCm/ROCR-Runtime commit: 0c0e634458]
2019-06-17 21:04:17 -05:00
Sean Keely b9c2754101 Add exception and error safety for CreateThread.
Change-Id: I82aaf64e039ca9614b4948deec1f87147f56279a


[ROCm/ROCR-Runtime commit: 9f81bdfbe1]
2019-05-24 22:39:55 -04:00
Sean Keely f336a19a0f Correct pthread join/detach handling.
Joined threads can not be joined more than once nor can they be detached.
Thread library wait and close allows multiple waits and separate close so
this fixes the pthread implementation.

Change-Id: I0019271a438f11ed4c6c11854011f5c4f6e16b65


[ROCm/ROCR-Runtime commit: a913549190]
2019-05-16 12:14:06 -05:00
Sean Keely bdf4b84f82 Don't create blits when copy profiling is enabled.
Change-Id: I879827133957ee610c3381ea30c536ec7d10ffab


[ROCm/ROCR-Runtime commit: 1251842900]
2019-04-18 20:00:02 -05:00
Sean Keely f7dbaf103b Add fine grain vram pool.
Part 1 of 2.
Enables fine grain vram over PCIe based on env flag.
Part 2 will extend to XGMI.

Change-Id: I8ad506e004b398d56d462b0200274eae2293a461


[ROCm/ROCR-Runtime commit: c56d86100b]
2019-02-21 13:08:11 -05:00
Sean Keely c66675d192 Suppress exception reporting for well defined invalid signal handles.
hsa_exceptions with empty what() strings will not report in debug builds.

Change-Id: I0d424d3b1d3044808ece1720a460a57d68bf878e


[ROCm/ROCR-Runtime commit: 344d964f9f]
2019-02-15 19:35:57 -05:00
Ramesh Errabolu 30363b9dbb Allows users, via env ROCR_VISIBLE_DEVICES, to surface a subset of Gpu devices
Change-Id: I5662639d5d70f054831969669f9d30dec356dd5a

Update per review comments

Change-Id: I18c7d7cb00b261493b61c2cf5454d486166f40d8


[ROCm/ROCR-Runtime commit: 3fbf03af76]
2019-02-06 02:02:29 -06:00
Sean Keely ed18ee7f38 Add pooling for Signal ABI blocks (SharedSignal).
Makes better use of memory and greatly reduces mmap count.

Change-Id: Ib444cd1ccd144986adbcc7cec297a966e2c08bc7


[ROCm/ROCR-Runtime commit: 8323b2e1d7]
2018-11-12 22:37:28 -06:00
Sean Keely 61b53915d7 Implement SDMA copy rect for gfx9.
Fix pitch overflow due to small element detection.
Add wide pitch 2D copy handling.
Cleanup code duplication.

Change-Id: I93b1584aba8e5964957eb7ab3544df806ca3e2f9


[ROCm/ROCR-Runtime commit: e0839ab27e]
2018-08-29 19:13:07 -04:00
Sean Keely 896a035951 Experimental flag to swap copy agent for async copy APIs.
Adds env flag HSA_REV_COPY_DIR.  If set to 1 async copy will
copy from dst device to src device rather than from src to dst.

Change-Id: I3095642066fa026dc112c2eac06db9393341cd7e


[ROCm/ROCR-Runtime commit: 6c47780620]
2018-08-09 10:58:14 -04:00
Evgeny dd69a5edb6 Tool load failure report changing to unconditional print bcos it's already is controlled with the env var
Change-Id: I91b400ba94575a32005e825e6b41bda05c55b357


[ROCm/ROCR-Runtime commit: 0e0be791ec]
2018-05-03 22:31:17 -05:00
Hari Thangirala 28e79a42c2 Allow HSA_ENABLE_SDMA to override runtime defaults.
Change-Id: I2305304228010157bfb589c365f4a998577231cd


[ROCm/ROCR-Runtime commit: 3e0cd85d69]
2018-04-10 12:56:48 -04:00
Sean Keely 2b15fedb50 Support large scratch allocations and reclaim.
Also improve small_heap used for scratch region allocation.

Change-Id: Ib7311b663b38968d88ebc355b81e12c0863dc541


[ROCm/ROCR-Runtime commit: 7caf9633f6]
2018-04-05 21:51:56 -04:00
Ramesh Errabolu de9064fc44 Enable sDMA packet HDP Flush on Gfx9 and later devices
Change-Id: I85922e5266883ef7e9eed3565e2c3b209009d294


[ROCm/ROCR-Runtime commit: 987f3f97aa]
2018-04-02 11:47:59 -04:00
Sean Keely e844aa939d Defer creation of internal queues and blits until first needed.
Change-Id: I2e61d7e102f38389d806d9eb24beda910573157b


[ROCm/ROCR-Runtime commit: bd5dd47ca1]
2018-02-09 14:02:07 -05:00
Sean Keely e155bd0cb1 Report library load errors in debug builds.
Change-Id: I24e63b15ad74fb86ecfe839f543800c2140c09d9


[ROCm/ROCR-Runtime commit: ca4c884306]
2017-12-05 18:49:33 -05:00
Sean Keely e2efba0676 Exception support for Queue.
Remove "zombie" queue state and report queue creation failure via
exceptions.  Make Shared object a final container and support array
objects with Shared.  Add message printing to hsa_exception in
debug builds.

Change-Id: I459f38c80846018acbf45538874e95f91dd6b195


[ROCm/ROCR-Runtime commit: f312a7386e]
2017-11-08 15:50:02 -05:00
Sean Keely 2406218416 Add queue intercept support to the runtime.
Queue intercept is exposed as two tools-only APIs via the API
intercept table.

Change-Id: Iac9602ed3143974d85c3569e9092295ad18037f8


[ROCm/ROCR-Runtime commit: 0c7dde2d1f]
2017-11-08 15:50:01 -05:00
Sean Keely 36d1168183 Fix assert in simple_heap.
Also add comments to clarify pointer info constraints.

Change-Id: I8d07831a0e953d667c84c96fe53ed07c18ba115c


[ROCm/ROCR-Runtime commit: 476c8e36bf]
2017-09-21 00:47:18 -04:00
Sean Keely 595c7130d2 Enable use of CLOCK_MONOTONIC_RAW for post 4.4 kernels.
Change-Id: I3c1f27c7e639df5128c36d81f715fa16e6c1cf13


[ROCm/ROCR-Runtime commit: 30fce248c6]
2017-09-20 14:28:23 -04:00
Sean Keely c5f1027982 Improve branch elimination in ScopedAcquire.
GCC can't reasonably be told that the lock ptr isn't null.  Adding a private bool
allows the branch to be eliminated, along with the bool.

Change-Id: I0605d69474d6a6e6951be93c0af1d8caf3f77124


[ROCm/ROCR-Runtime commit: 9dfdce5b3c]
2017-09-19 06:08:36 -04:00
Sean Keely 3aeea948cb Add env key to disable 2MB suballocation.
Change-Id: Icca3041c3578aa180a656c01aae62f2ad6e8b583


[ROCm/ROCR-Runtime commit: 4275661682]
2017-09-19 06:08:36 -04:00
Sean Keely 9fd778eec1 Add suballocator for ordinary VRAM allocations smaller than 2MB.
Track pointer info for sub 2MB fragment allocations in allocation_map_.

Add fragment support to IPC.

Change-Id: I00cfc2e2fa289aac90a4718c392f9bb056a61a87


[ROCm/ROCR-Runtime commit: 117be0b55a]
2017-09-19 06:08:36 -04:00
hthangir 3dc1b18d54 Use non-RAW version in clock_getres to workaround bug in older kernels
Change-Id: Ice0606a42cd7054f0804baf4af3521ffae3b7d50


[ROCm/ROCR-Runtime commit: 87d2df3da3]
2017-09-14 13:56:15 -05:00
Sean Keely e150edc48c Report tools library load failures in debug builds.
Change-Id: Ie1ff313e929fc46134e58730a1d370c5d7ace8db


[ROCm/ROCR-Runtime commit: f1a661dedb]
2017-08-31 21:32:48 -04:00
Laurent Morichetti b2cb6bca96 Include <errno.h> for EBUSY
Change-Id: I9fa3417445866f3ce37af2169f623afa8e92e873


[ROCm/ROCR-Runtime commit: 0f05ef73ac]
2017-08-31 07:32:51 -07:00
hthangir ea88523fc7 Fix compilation issue reported with GLIBC 2.12 (RHEL 6.9)
Change-Id: I770b72ba1d61475a76aa72d0c52ebfb380db6019


[ROCm/ROCR-Runtime commit: 9ee0108e58]
2017-07-28 11:11:01 -04:00
Sean Keely 17d0e450cb Correct handling of slow clocks under linux.
Change-Id: I9a1b08d5457caa6739220603bbd37b00febc64d7


[ROCm/ROCR-Runtime commit: 29b5b5c029]
2017-07-12 12:49:49 -04:00
Ramesh Errabolu 1351f204d4 Support Perf Cntrs (PMC) and Thread Trace (SQTT) over AQL queues
Change-Id: I716b722895d90b46914c31377e791ad602acecc1


[ROCm/ROCR-Runtime commit: 08e0bca567]
2017-06-15 12:58:31 -04:00
Kenny Ho 415027b89f Revert "Implement memory fault analysis through context save area"
This reverts commit 498f3a7188.

Change-Id: Ibf11b764b383b9be291f3009a30550e1a1e2d115


[ROCm/ROCR-Runtime commit: 5b4df54b10]
2017-06-14 14:21:53 -04:00
Jay Cornwall 498f3a7188 Implement memory fault analysis through context save area
When a fatal memory fault occurs the scheduler context-saves all queues
in the process and notifies the runtime through the memory event. The
saved state contains all GPR/LDS data at the moment of the fault.

Retrieve this state and present it to the user if HSA_DEBUG_FAULT is set
to "analyze" and the wavefront caused the fault. If amdgcn-capable objdump
is in the PATH invoke this to disassemble code around the PC.

Queue lifetime is now managed by the runtime to allow querying the
context save state for all active queues.

Change-Id: I6fee662fad1c4f9aa125bf5c53d7d0ea1ab32f95


[ROCm/ROCR-Runtime commit: 75c9506f9d]
2017-06-13 23:12:28 -04:00
hthangir f5c06c5911 The fallback path covers not just ARM64, need this for Power as well.
Change-Id: I7bbf76f77bd3ac47a0a0987c1e880e23834588e2


[ROCm/ROCR-Runtime commit: a0957bc679]
2017-06-07 14:45:29 -05:00
Sean Keely c02e8c67df Adjust signal sleep to reflect null kernel latency. Performance tested on Gromacs.
Change-Id: I3851148ee8544b15d840f2c26ca73a83f8d0df2e


[ROCm/ROCR-Runtime commit: 426d41e27c]
2017-03-09 15:20:53 -05:00
Sean Keely de0fa51022 Fix Api table copy operation and tools version checking.
Change-Id: Ia76d16f3ea6d0abb931813f90bc3bc2119da5999


[ROCm/ROCR-Runtime commit: 505d722b7d]
2017-02-07 14:26:20 -05:00
Jay Cornwall 7b45ebfe7f Add support for ARM
- Build system fixes
- No user-mode high-precision timer by default, use clock_gettime
- Use C11 aligned_alloc pending C++17 std::aligned_alloc

Change-Id: I268365bdfd11d1e817a89584b9e086ee5b86e1dc


[ROCm/ROCR-Runtime commit: 9e575ea96a]
2017-01-31 16:43:49 -08:00
Sean Keely b75a8d9ed0 Allow reducing max occupancy (max scratch waves) when applications request large amounts of scratch.
Also emit error messages to stderr if no async queue error callback was registered and queue fault messages are enabled (on by default).
Queue fault messages are controlled with env key HSA_ENABLE_QUEUE_FAULT_MESSAGE.

Change-Id: I496487b8d048b83aa95b9784e92928211f167b17


[ROCm/ROCR-Runtime commit: 0e17cc2887]
2016-12-20 16:52:59 -06:00
Ramesh Errabolu e3d8eac639 Initial set of changes for ThreadTrace
Change-Id: I07ce31f9b4f508cef0fc9ca6dadcf26b6c90361e


[ROCm/ROCR-Runtime commit: eb2efb83d1]
2016-11-21 23:40:56 -06:00
Jay Cornwall c6bfeda697 Fix miscellaneous warnings flagged by Clang
Change-Id: I85a45cb3b44e4379b31bcc56af061fd1571f2af5


[ROCm/ROCR-Runtime commit: c30c25bd30]
2016-10-26 19:26:16 -05:00
Ramesh Errabolu 85aa61f011 Print Debug Mesg if private segment memory request is illegal
Change-Id: I46351651b6b2bf14e26645440a4321bc941900b2


[ROCm/ROCR-Runtime commit: c54304fe38]
2016-09-08 11:16:09 -04:00
Sean Keely 1306432ae1 Switch atomic_helpers.h from C11 atomics to GCC __atomic builtins.
C11 atomics are not statically guaranteed to be lock free and so
may not be atomic with respect to atomic operations originating
outside the standard library, such as platform atomics.

C11 macros to statically discover always lock free operations
(ATOMIC_*_LOCK_FREE) do not cover uint64_t in GCC and
std::atomic<uint64_t> is not a type alias of any covered type.

All use of __atomic by atomic_helpers.h is statically checked to be
always lock free.

GCC builtin fencing does not appear to be strong enough for WC memory.
Added an option (enabled) to enforce consistency for WC memory on x64.

__sync builtin's were not used as they were declared legacy by GCC.

Added a strongly conservative option (ALWAYS_CONSERVATIVE) to enable
use of full memory fences in place of partial fences and compiler
driven processor specific optimization.

Change-Id: Id7aaaca626144070f58759f6a348cbee4612bbc0


[ROCm/ROCR-Runtime commit: 1bc15bbf79]
2016-09-03 06:22:42 -04:00
Jay Cornwall 70e07dcdfd Enable VM fault message by default on Linux
This option was disabled by default to address issues writing to stderr
in Windows applications. The lack of an error message for memory access
faults is confusing to users, however.

Enable the error message by default on Linux only.

Change-Id: I1f44ba42362f8874abdc7c8e63ddd54a855b5394


[ROCm/ROCR-Runtime commit: acc5f15e4c]
2016-07-30 10:10:14 -04:00
Besar Wicaksono (xN/A) TX [TEXT] 2caa405817 Add environment flag to enable sdma workaround that will wait for the sdma queue to be idle before updating the write pointer. Add class to manage environment flags.
[git-p4: depot-paths = "//depot/stg/hsa/drivers/hsa/runtime/": change = 1254004]


[ROCm/ROCR-Runtime commit: c95f96a9e4]
2016-04-01 17:13:45 -05:00
James Edwards (xN/A) TX 8504fcb6ca Separate open source core runtime code from DK makefiles.
[git-p4: depot-paths = "//depot/stg/hsa/drivers/hsa/runtime/": change = 1250152]


[ROCm/ROCR-Runtime commit: 7d2bc9d113]
2016-03-22 18:10:13 -05:00
James Edwards (xN/A) TX 6da3325470 Remove opensrc test files.
[git-p4: depot-paths = "//depot/stg/hsa/drivers/hsa/runtime/": change = 1249961]


[ROCm/ROCR-Runtime commit: 7d1e6c3a57]
2016-03-22 13:39:51 -05:00