Commit Graph

1235 Commits

Author SHA1 Message Date
Sean Keely 3f4f6f778a Use atomic variable for Runtime ref_count_.
Change-Id: Ic4d0ad9ff93d0cc52cfe2df006ee3436d5960b07


[ROCm/ROCR-Runtime commit: ac5ccb45b7]
2018-03-06 03:45:14 -06:00
Sean Keely 282dba1036 Add exception safety to Runtime::Acquire.
Change-Id: Ia2a9baf08bb56971412f1ac3914592612de5f134


[ROCm/ROCR-Runtime commit: 31c05d2fc7]
2018-02-28 05:21:07 -06:00
Sean Keely f64d45a66d Improve fragment map reporting format.
Change-Id: I85d09d085b08de46271ec902c766a8609a4b921a


[ROCm/ROCR-Runtime commit: 95c926059d]
2018-02-09 14:03:03 -05:00
Sean Keely de3912dae4 Emit fragment map and thunk ptr info with VM faults.
Change-Id: If1302f674df7a636529c64bf66dfdda755a70c32


[ROCm/ROCR-Runtime commit: 9212e7a09f]
2018-02-09 14:02:26 -05:00
Sean Keely c1147397e0 Reduce to only one internal compute queue.
Change-Id: Ie42ecb3b242077624d74caeabfcd418dbbd9ff3e


[ROCm/ROCR-Runtime commit: 0eb534e3cf]
2018-02-09 14:02:15 -05:00
Sean Keely e844aa939d Defer creation of internal queues and blits until first needed.
Change-Id: I2e61d7e102f38389d806d9eb24beda910573157b


[ROCm/ROCR-Runtime commit: bd5dd47ca1]
2018-02-09 14:02:07 -05:00
James Edwards 27ce54a0aa : Fix compilation errors with gcc 7.2.1 for hsa runtime.
Change-Id: I3356388753ca78cc0f1e0c3188220d7f3f60283d


[ROCm/ROCR-Runtime commit: b913795c31]
2018-02-07 09:22:39 -06:00
Evgeny f22fab8263 ExecutePM4 queue full check fix
Change-Id: Id56ece6d3f5eab1ef3a2758922022f0996c1efe4


[ROCm/ROCR-Runtime commit: 8a8d7ad814]
2018-02-05 19:35:39 -06:00
Chris Freehill 67598cc036 Don't support platform atomics for gfx9XX
Change-Id: I302c862494e221ae2b6b3e1a843f06586b0b28ba


[ROCm/ROCR-Runtime commit: 3449f7dea6]
2018-02-02 18:21:16 -05:00
Sean Keely 00b5aec5d4 Guard against IPC signal use of when profiling copy APIs.
Also update IPC signal API text to allow single process profiling with IPC signals.

Change-Id: I90b246623129d57183acb4ba1789beec360547c3


[ROCm/ROCR-Runtime commit: f59b001c75]
2018-01-31 19:05:32 -05:00
Sean Keely 9c247a4c6b Revert CRAT table workaround.
Change-Id: Ic2bf9e1fb1d00c5a31d52560e0eb37e0ae1ab08a


[ROCm/ROCR-Runtime commit: 91f559802d]
2018-01-30 18:26:53 -06:00
Tony Tye ab3c53761b Add support for R_AMDGPU_RELATIVE64
- Add support for R_AMDGPU_RELATIVE64 relocation record.
- Return status error if any unsupported relocation record encountered.

Change-Id: Icbb5dcb81109a70c1f2195412a0df58a11be9da1


[ROCm/ROCR-Runtime commit: d472b24d05]
2018-01-30 18:20:26 -05:00
Chris Freehill f113a2dadc Report physical memory instead of virtual memory
Change-Id: I18105e3982a96aea40e05cd78521c0c3acf75de4


[ROCm/ROCR-Runtime commit: 563581223c]
2017-12-20 22:11:50 -04:00
Sean Keely 640ca3ba4f Merge system heap info.
Workaround pending thunk spec clarification.

Change-Id: I9d96227efde3a551157733cf4050d474d1e658f2


[ROCm/ROCR-Runtime commit: fe1763848a]
2017-12-19 18:57:29 -06:00
Sean Keely ecbd2eb820 Remove region/pool size limits for 902.
Temporary measure. Must be reverted once CRAT tables have been fixed.

Change-Id: Id2f2673edbf7b6fc5752f8d871042b4bf4de653c


[ROCm/ROCR-Runtime commit: b49e5b4917]
2017-12-14 16:02:05 -05:00
Sean Keely b8934623ee Don't use double mappings on GFX9 APUs.
Change-Id: I1225696211d4eac9ce982243ea0a1a9e8b2a318f


[ROCm/ROCR-Runtime commit: 1addb5e684]
2017-12-08 20:18:02 -05:00
Sean Keely e155bd0cb1 Report library load errors in debug builds.
Change-Id: I24e63b15ad74fb86ecfe839f543800c2140c09d9


[ROCm/ROCR-Runtime commit: ca4c884306]
2017-12-05 18:49:33 -05:00
Sean Keely a427fde26a Cleanup Signal interfaces for doorbells.
Create an interface for doorbell signals to reduce code duplication.
No functional changes.

Change-Id: I101a8997dd582ff99e1537758c804b21fe3bb6af


[ROCm/ROCR-Runtime commit: d2e70bb999]
2017-11-28 22:12:19 -06:00
Sean Keely 495c1da5f3 Pull from github (tstellar):
Prefer using memfd_create() for the ring buffer.

We were using /dev/shm, but this won't work on systems that
either don't have /dev/shm or have mounted it with noexec, because
for everything other than gfx700 we map the ring buffer with PROT_EXEC.

memfd_create() is Linux specific and was added in Linux 3.17, so we
will fallback to using /dev/shm on systems where memfd_create() is
not available.

Change-Id: I58fb533eebc362f6d29dc3e316a80801014d50e8


[ROCm/ROCR-Runtime commit: b93ffafdc7]
2017-11-28 20:47:12 -05:00
Sean Keely 37132e4a21 Improve loop variables.
Derived from github pull request by folklore1984.

Change-Id: I70cd3da131691543fed8bf913d6245d41c49280d


[ROCm/ROCR-Runtime commit: 4b603e803d]
2017-11-28 20:36:22 -05:00
Sean Keely 7ed62f815f Pull from github (pmargheritta):
Corrected semantics used in hsa_queue_load_write_index_relaxed.

The semantics that was used in hsa_queue_load_write_index_relaxed
didn't seem to match the name of the function.
I also removed a useless return keyword.

Change-Id: If3819d38fb367f122fc382edf8ee3771a23279ae


[ROCm/ROCR-Runtime commit: 5872b618de]
2017-11-28 20:35:50 -05:00
Evgeny acce35d21f _aqlprofile_start() API migration
Change-Id: I7c8c7a6fc6f9b20cc2e4074dde38fb19440927f1


[ROCm/ROCR-Runtime commit: 86939368d1]
2017-11-20 17:32:19 -05:00
Chris Freehill ed0537ed67 Device ID/family corrections for gfx9xx
Change-Id: Icb25fbbaeb99ce886a2852b48d02875ee0f197a2


[ROCm/ROCR-Runtime commit: 651ae1bf70]
2017-11-16 07:27:54 -05:00
Evgeny acaf0d0aac aqlprofil API: removing from HSA hsa_api_trace/hsa_ext_interface
Change-Id: I12fac55ea9ccfdb119899bf9d000e3c8b0bf4bbb


[ROCm/ROCR-Runtime commit: 6e1b9288f6]
2017-11-11 10:01:12 -06:00
Evgeny fd81986bb2 aqlprofile API: _aqlprofile_start() returns required profile buffer sizes if undersized
Change-Id: Ib14b2cb2e7e2026c3af0b7bd2f08f51e48e598b2


[ROCm/ROCR-Runtime commit: bb8eaf3ac8]
2017-11-09 20:03:55 -06:00
Sean Keely 76c28f38c4 Fix bad casts in tools.
Also virtualize queue profiling enable.

Change-Id: I761b41269be3df7eb64a5914ee9951ed6b51bb04


[ROCm/ROCR-Runtime commit: 6455a69b03]
2017-11-08 15:50:02 -05:00
Sean Keely 0fcdd63d88 Add callback exception forwarding.
Modified callbacks for intercept queue, queue error, iterate agent and
iterate region.

Change-Id: I8bdd67f2312510ea7eb9caec93babca244938b40


[ROCm/ROCR-Runtime commit: a6d8a48cbc]
2017-11-08 15:50:02 -05:00
Sean Keely e2efba0676 Exception support for Queue.
Remove "zombie" queue state and report queue creation failure via
exceptions.  Make Shared object a final container and support array
objects with Shared.  Add message printing to hsa_exception in
debug builds.

Change-Id: I459f38c80846018acbf45538874e95f91dd6b195


[ROCm/ROCR-Runtime commit: f312a7386e]
2017-11-08 15:50:02 -05:00
Sean Keely 2406218416 Add queue intercept support to the runtime.
Queue intercept is exposed as two tools-only APIs via the API
intercept table.

Change-Id: Iac9602ed3143974d85c3569e9092295ad18037f8


[ROCm/ROCR-Runtime commit: 0c7dde2d1f]
2017-11-08 15:50:01 -05:00
Kent Russell c52d3b6997 Revert "aqlprofile API: _start() sets buffers sizes with NULL ptr; block counters reg number / block name info"
This reverts commit 8518a48d4f.

Change-Id: Ie90b091df772bf9391494c773d63858aafbc1176


[ROCm/ROCR-Runtime commit: b29d3f63e2]
2017-11-08 06:59:33 -04:00
Evgeny 8518a48d4f aqlprofile API: _start() sets buffers sizes with NULL ptr; block counters reg number / block name info
Change-Id: I3cb93453b683c55bf5ec26271648232306a5d140


[ROCm/ROCR-Runtime commit: 3daa85fad8]
2017-11-07 15:05:47 -05:00
Sean Keely cb47089c17 Make HostQueue::queue_count_ a portable atomic type.
Also make lint happy.

Change-Id: I0f965df6a76fd959df9eb411d1f1b11847159790


[ROCm/ROCR-Runtime commit: d93f92f42d]
2017-10-31 02:38:25 -05:00
Qingchuan Shi 3e9a0561c0 Add APIs to support debugging vm fault
1. Add hsa ext api hsa_amd_register_vmfault_handler for debugger to register callback in case of VM fault.
2. Extend hsa_ven_amd_loader API to:
   (1) iterate loaded code objects in executable:
       hsa_ven_amd_loader_executable_iterate_loaded_code_objects
   (2) get loaded code object info:
       hsa_ven_amd_loader_loaded_code_object_get_info
3. Make the id of hsa_queue the same as the one used in communication with thunk (for amd_aql_queue)

Change-Id: I68910809e59e24297350d262606f00e96c14bcbd


[ROCm/ROCR-Runtime commit: ce6aee01ed]
2017-10-28 21:48:26 -04:00
Sean Keely 0a4a1a2433 Fix error message description.
Change-Id: I32efed68e970a4882aca9decbbcda3fcd5c5cb43


[ROCm/ROCR-Runtime commit: 6ee2ccb08b]
2017-10-24 21:52:21 -05:00
Sean Keely 8122fbebad Set doorbell kind code for gfx9+ device enqueue.
Change-Id: I93c4cea677ae51f97ac768614333743fb26b2f54


[ROCm/ROCR-Runtime commit: 5a4ab91be1]
2017-10-21 11:08:44 -04:00
Sean Keely 5dfef3ef77 Improve build system handling of non-default directory layouts.
Adds the thunk include and lib paths to the cache, removes paths
to indicator files from the cache, uses the cached path directory
(if any) as a search hint for indicator files.

Change-Id: I0859faa8d229a97abfaacb408d2c831e317aed5f


[ROCm/ROCR-Runtime commit: a8d818a6bc]
2017-10-21 11:08:15 -04:00
Sean Keely 41615ea7d5 Improve unhandled exception error reporting in debug builds.
Change-Id: Ia92d1a93163105d817a2147d96f2edd399e2b70d


[ROCm/ROCR-Runtime commit: 3cef9b1a04]
2017-10-21 11:08:01 -04:00
Sean Keely bdb5edad34 Fix memory leak in exception path.
Change-Id: Iad5f035cd1909be4a8f1a1f5dd7ca5abec0694b4


[ROCm/ROCR-Runtime commit: 737966eb25]
2017-10-21 11:08:00 -04:00
Ramesh Errabolu a4d753615b Changes to support surfacing of link weight as part of link info
Change-Id: I1c0705a9374af1245f0419c51beded0d7ee10639


[ROCm/ROCR-Runtime commit: dccbc9f2af]
2017-10-20 12:09:31 -04:00
Chris Freehill 26ebd1727c Use Major/Minor/Step device numbers to differentiate gfx devices
Change-Id: I0901871971a5b33018917ada6c0e69ac7aa91944


[ROCm/ROCR-Runtime commit: a7cbe78366]
2017-10-13 16:18:24 -04:00
Sean Keely 93e58b9100 Capture more memory allocation types with the 2MB allocator.
TensorFlow was running out of VRAM due to padding up allocations
from legacy memory APIs.  These allocations have been added to
the fragment allocator to improve VRAM utilization.

Change-Id: Ic680fff576a0434b3b17a4c91746da44e09957fa


[ROCm/ROCR-Runtime commit: 4f299a9909]
2017-10-12 23:22:10 -04:00
Evgeny b07cbdc10e aqlprofile API: enabling privilege memory related counters
Change-Id: I28a24ad1a3ce78c5d8a6319635ae1ffd392ab690


[ROCm/ROCR-Runtime commit: fd99e909ff]
2017-10-09 17:34:54 -05:00
Sean Keely 06c7997137 Release cached memory blocks when memory allocation fails and retry.
Change-Id: I6d96e136e147d8ffe9ff7baec26b4b5a678b739f


[ROCm/ROCR-Runtime commit: 9ba83d83f7]
2017-10-03 20:12:05 -04:00
Sean Keely 21a65a1233 Support hsa_amd_agents_allow_access on page fragments.
Since access may only be manipulated on whole pages, suballocator fragments must cooperate to set the page's access.
Since the KFD does not migrate memory on access changes this implementation makes agent access sticky across the requests in a fragmented page.

Change-Id: I88479ed45fb40e9782b704526a7b8ffb22e7bd76


[ROCm/ROCR-Runtime commit: e9a6f2c3e6]
2017-09-27 19:04:04 -05:00
Evgeny 64b516415c removing graphics specific block RMI, ennabling memory related blocks
Change-Id: I477adc49b9ee3c8593c193bdc69c0deb4a9726e1


[ROCm/ROCR-Runtime commit: 0e88414f5c]
2017-09-25 10:49:22 -05:00
Sean Keely 36d1168183 Fix assert in simple_heap.
Also add comments to clarify pointer info constraints.

Change-Id: I8d07831a0e953d667c84c96fe53ed07c18ba115c


[ROCm/ROCR-Runtime commit: 476c8e36bf]
2017-09-21 00:47:18 -04:00
Evgeny 4c5404b89d adding hsa_ven_amd_aqlprofile.h to the packaging
Change-Id: I3b69396e3cea129106d47be53218213e29de9843


[ROCm/ROCR-Runtime commit: fcaecfee80]
2017-09-20 14:40:49 -04:00
Sean Keely 595c7130d2 Enable use of CLOCK_MONOTONIC_RAW for post 4.4 kernels.
Change-Id: I3c1f27c7e639df5128c36d81f715fa16e6c1cf13


[ROCm/ROCR-Runtime commit: 30fce248c6]
2017-09-20 14:28:23 -04:00
Sean Keely c5f1027982 Improve branch elimination in ScopedAcquire.
GCC can't reasonably be told that the lock ptr isn't null.  Adding a private bool
allows the branch to be eliminated, along with the bool.

Change-Id: I0605d69474d6a6e6951be93c0af1d8caf3f77124


[ROCm/ROCR-Runtime commit: 9dfdce5b3c]
2017-09-19 06:08:36 -04:00
Sean Keely 3aeea948cb Add env key to disable 2MB suballocation.
Change-Id: Icca3041c3578aa180a656c01aae62f2ad6e8b583


[ROCm/ROCR-Runtime commit: 4275661682]
2017-09-19 06:08:36 -04:00