Commit Graph

39 Commits

Author SHA1 Message Date
Sean Keely 270d042ef8 Minor interface improvement to pointer info.
Take in const void* rather than void*.  This does not break the
abi or existing code.  Existing code would need to cast away any
const which is unnecessary and annoying.

Change-Id: I28787e8fab1b600bf6871ea82835e10a4f475c5b
2021-08-04 16:43:23 -04:00
Sean Keely 206e87d28b Support debugging hw exceptions.
Change-Id: I9780147294af2e9457fa54693580735452ee2ae6
2021-07-16 18:03:26 -05:00
Sean Keely 77046a1aaa Revert "Revert SVM and XNACK support."
This reverts commit 5bd153974d.

Conflicts:
	opensrc/hsa-runtime/core/util/flag.h

Change-Id: I16daf41588e6139126d66af54b0693de2e7e39f3
2021-04-21 14:49:43 -05:00
Sean Keely 5bd153974d Revert SVM and XNACK support.
KFD is not ready yet.

Change-Id: I61deb292ddb92185d33504c2115169888d56e211
2021-04-02 02:10:59 -04:00
Sean Keely 7333c77e22 Squash merge of cfreehil/amd-temp-gfx90a onto amd-staging.
Includes some workarounds and HMM.
Conflicts:
	opensrc/hsa-runtime/core/runtime/amd_topology.cpp
	opensrc/hsa-runtime/core/util/flag.h

Change-Id: I22976f07964a43dbb228a6231777dbd599112b8d
2021-04-02 02:10:15 -04:00
Sean Keely 45fbe5b192 Block ROCm 4.1+ running against 4.0 and prior kfd.
Sramecc is misreported in kfd 4.0 and prior.  To prevent possible
corruption due to d16 instructions, deny use of gfx906 with older
kfds and correct misreport for gfx908.  Denial of gfx906 may be
overridden by setting HSA_IGNORE_SRAMECC_MISREPORT=1.

Change-Id: I7d5c3a716fad01c348f8b88cd508cedbf914c989
2021-04-01 00:03:32 -04:00
Sean Keely 34ac62274a Correct legacy copy path.
Legacy p2p copy path incorrectly transfered in whole pages rather than
the requested size.

Change-Id: I9aa7337754f9e32f587a0cc5305f8ffeb6196f10
2021-02-10 19:53:02 -05:00
Sean Keely 32d0fcafa9 Limit clock synchronization to 16Hz.
Improves HIP event performance in directed benchmarks where
clock sync latency is significant.

Change-Id: I78b724a14a8f5b6a9a2b9f4d85afe9d8b81808a6
2020-11-20 15:06:13 -05:00
Ramesh Errabolu f7350c6020 Update ROCr implementation of Queue ID
Change-Id: Iec48b1978e4d01563e71cfb58aed8f1bbc446443
2020-06-26 13:25:00 -05:00
Ramesh Errabolu fa13208698 Add rocr namespace to core header and impl files
Change-Id: I1e1b33f9bba1078d049bc19797889988c3e43360
2020-06-19 22:34:21 -04:00
Sean Keely ce19721c88 Update copyright date.
Change-Id: If4bf4c20cf051878bfe759080bb7345d884dd53d
2020-06-19 22:34:01 -04:00
Sean Keely 4e0bf29704 Separate memory map info printing from vmfault handler.
Allows the print utility to be called from GDB.

Change-Id: Ia27ff9ba23ad6c0117ba1278693d4c8b0473bdc1
2020-05-28 23:34:49 -04:00
Ramesh Errabolu 627991b1c1 Update how code references publicly available ROCr headers
Change-Id: I357c51eb713a23704d4fee71081be46a73a71806
2020-02-21 20:01:11 -05:00
Sean Keely 299874f17d Initial support for deallocation callbacks.
Adds hsa_amd_register_deallocation_callback and hsa_amd_deregister_deallocation_callback
to notify when HSA memory has been released.

Change-Id: I1f33cee250ca890e5c2e7fddfa4479aa5874651d
2019-06-26 04:12:17 -05:00
Sean Keely 67376e06ab Report SRAM ECC errors through the system event handler.
Modify the system event handler to support multiple users.
Name memory fault reason codes.

Change-Id: I1b5979b36ab15637eb2be59a61e2d57e76d0a70e
2019-02-27 18:08:07 -05:00
Sean Keely 4e8597681b Cache KFD Events used by user allocated InterruptSignals.
Change-Id: I7f102f880fea9c78febe28cd262f93ee77f03184
2018-11-12 22:37:42 -06:00
Sean Keely 8323b2e1d7 Add pooling for Signal ABI blocks (SharedSignal).
Makes better use of memory and greatly reduces mmap count.

Change-Id: Ib444cd1ccd144986adbcc7cec297a966e2c08bc7
2018-11-12 22:37:28 -06:00
Sean Keely 936ecd1885 Remove legacy SVM region concept.
Also rename blit_agent to region_gpu and add comments to clarify
its role in deprecated region API support rather than to do blits.

Change-Id: I80b1043db2e1c5d40a58fc801eef70a688ea9169
2018-11-09 06:27:53 -06:00
Sean Keely 9ec37b5103 Ensure runtime cleanup when hsa_init ref count reaches 0.
Delete the runtime object when the last hsa_shut_down occurs.

Change-Id: I2005d52d06702eaef166714fd5e471cc277924db
2018-10-22 19:32:00 -05:00
Sean Keely 757502ccd6 Report internal queue creation to tools.
Debug agent requires handles to internal queues for single step debugging.
Added tools only API hsa_amd_runtime_queue_create_register for reporting.

hsa_amd_runtime_queue_create_register sets a callback which is invoked
when internal queues are created.

Change-Id: Ia5190ae724fadba686c15f25b2cd085350eeff0e
2018-10-20 23:12:27 -04:00
Sean Keely 1e0d690948 Use ptrinfo rather than apertures in hsa_memory_copy
Apertures now overlap with the change to 48bit addressing which
precludes using aperture checks to discover buffer ownership.
Switches to ptrinfo to decide which device a buffer owned by.

This corrects faults in the legacy hsa_memory_copy api.

Change-Id: I5c7ce0216e1cdc96f836fc6fec9c3defdf4b9d90
2018-10-11 13:34:53 -04:00
Ramesh Errabolu 01eea21d6c Capture number of Numa Nodes present on system
Change-Id: Ic789a6b9da8e316cb483e50b0fe9faa03798f97c
2018-09-18 16:27:30 -05:00
Qingchuan Shi 49d2175c74 debug suport for queue error.
1/ Revised debug event handler to handle different events.
2/ Added queue error handler using the callback in queue create, which will print out wave info when queue in error state.
3/ Preempt queue instead of destory queue when queue error state.

Change-Id: Ib727d208de9caf1c72c76d42268483b24aaebde8
2018-04-20 14:25:16 -04:00
Sean Keely ac5ccb45b7 Use atomic variable for Runtime ref_count_.
Change-Id: Ic4d0ad9ff93d0cc52cfe2df006ee3436d5960b07
2018-03-06 03:45:14 -06:00
Sean Keely 0c7dde2d1f Add queue intercept support to the runtime.
Queue intercept is exposed as two tools-only APIs via the API
intercept table.

Change-Id: Iac9602ed3143974d85c3569e9092295ad18037f8
2017-11-08 15:50:01 -05:00
Sean Keely d93f92f42d Make HostQueue::queue_count_ a portable atomic type.
Also make lint happy.

Change-Id: I0f965df6a76fd959df9eb411d1f1b11847159790
2017-10-31 02:38:25 -05:00
Qingchuan Shi ce6aee01ed Add APIs to support debugging vm fault
1. Add hsa ext api hsa_amd_register_vmfault_handler for debugger to register callback in case of VM fault.
2. Extend hsa_ven_amd_loader API to:
   (1) iterate loaded code objects in executable:
       hsa_ven_amd_loader_executable_iterate_loaded_code_objects
   (2) get loaded code object info:
       hsa_ven_amd_loader_loaded_code_object_get_info
3. Make the id of hsa_queue the same as the one used in communication with thunk (for amd_aql_queue)

Change-Id: I68910809e59e24297350d262606f00e96c14bcbd
2017-10-28 21:48:26 -04:00
Sean Keely 117be0b55a Add suballocator for ordinary VRAM allocations smaller than 2MB.
Track pointer info for sub 2MB fragment allocations in allocation_map_.

Add fragment support to IPC.

Change-Id: I00cfc2e2fa289aac90a4718c392f9bb056a61a87
2017-09-19 06:08:36 -04:00
Sean Keely e38ff18990 Unmap GPUs when allow_access removes them from system pools.
Change-Id: Ib9eb88622fded43ebd9eddbf78ad6771a5b91e77
2017-05-17 20:58:05 -04:00
Sean Keely 8081758a55 Add InterProcess memory sharing support.
Support is disabled pending KFD / Thunk readiness.

Change-Id: I55def748e3d56cbfcfa6e24983a0ab78567aa81d
2016-11-15 18:58:29 -06:00
Sean Keely 9dd76dbeda Add pointer info support.
Change-Id: I3edcc0bfddbf12465065c9bc3b6565288faff1b8
2016-11-11 18:40:16 -06:00
Jay Cornwall 74f5aca93d Refactor: Consolidate calls to hsaKmtAllocMemory
Route all device-visible system memory allocations through system_allocator.

Change-Id: I5e90a1bf491e432678a6d8ab1f9f3770734cbda1
2016-08-24 23:57:19 -04:00
Jay Cornwall 0c9f96cfa4 Propagate errors from hsaKmtOpenKFD back to hsa_init
Errors are otherwise silently ignored and hsa_init completes successfully.

Change-Id: Ib1b7dbd7a65d40f869cdbb2792fa97132873d3d7
2016-08-22 17:28:48 -05:00
Jay Cornwall b44417043b Recognize all CPU nodes in hsa_signal_create consumer list
On multi-node systems only the first CPU node was recognized in the
signal consumer list, causing fallback to non-interrupt signals.

Change-Id: I9bd0706bafbe046be9d7f210d05fa4cf1fcd16fa
2016-07-09 18:40:39 -05:00
Besar Wicaksono (xN/A) TX [TEXT] 5a584fa1ab Fix query HSA_AMD_AGENT_MEMORY_POOL_INFO_LINK_INFO
Querying HSA_AMD_AGENT_MEMORY_POOL_INFO_LINK_INFO between a gpu agent
and its own local memory pool returns a wrong information.
Fix: return link with 0 hop count.

[git-p4: depot-paths = "//depot/stg/hsa/drivers/hsa/runtime/": change = 1257544]
2016-04-13 12:39:25 -05:00
Besar Wicaksono (xN/A) TX [TEXT] c95f96a9e4 Add environment flag to enable sdma workaround that will wait for the sdma queue to be idle before updating the write pointer. Add class to manage environment flags.
[git-p4: depot-paths = "//depot/stg/hsa/drivers/hsa/runtime/": change = 1254004]
2016-04-01 17:13:45 -05:00
James Edwards (xN/A) TX 7d2bc9d113 Separate open source core runtime code from DK makefiles.
[git-p4: depot-paths = "//depot/stg/hsa/drivers/hsa/runtime/": change = 1250152]
2016-03-22 18:10:13 -05:00
James Edwards (xN/A) TX 7d1e6c3a57 Remove opensrc test files.
[git-p4: depot-paths = "//depot/stg/hsa/drivers/hsa/runtime/": change = 1249961]
2016-03-22 13:39:51 -05:00
James Edwards (xN/A) TX c9ffe0004e Check open source core runtime code into perforce. This includes license and README files.
[git-p4: depot-paths = "//depot/stg/hsa/drivers/hsa/runtime/": change = 1249136]
2016-03-20 15:39:40 -05:00