Commit Graph

248 Commits

Author SHA1 Message Date
Joseph Huber 298e6cc495 Improve endianness check
Update the `hsa.h` header to use the gcc / clang `__BYTE_ORDER__`
macros where available to more accurately autodetect endianness for
the target.

Change-Id: I7312f3badcba9287a30eb14882b91e2a247acc5f


[ROCm/ROCR-Runtime commit: 4971150576]
2024-01-16 21:29:39 +00:00
David Yat Sin ff3818c725 Add query for agent memory and aql ext properties
Add query to return flags for GPU agent memory properties and AQL
extensions.

Implement flag to determine that GPU agent is an APU

Change-Id: Ic04c51290b2b9763e14989c117f35a2e22297453


[ROCm/ROCR-Runtime commit: c86837d8d6]
2023-12-07 14:41:37 -05:00
David Yat Sin ff72129092 Add queries for HSA Ext interface version
Change-Id: I26860fb1364cd3a33cdc9b284ac807b2702bb241


[ROCm/ROCR-Runtime commit: ed1b0b9b1a]
2023-12-06 13:58:52 -05:00
Alex Sierra 663e42663b core dump: SegmentBuilder classes added
SegmentBuilder classes are used to get core dump data from the GPUs.
So far, it uses thunk API calls and smaps to collect all data from
the Hardware.

Signed-off-by: Alex Sierra <alex.sierra@amd.com>
Change-Id: I2ad70ca5a951885181d3142653b186b0f6be739e


[ROCm/ROCR-Runtime commit: 1083d5c35f]
2023-12-05 23:19:14 -05:00
Giovanni LB 952f750a77 Adding coordinate query to aqlprofile
Change-Id: I9f2fee62a24cf2a4784ba9e8c813b7b7296d034b


[ROCm/ROCR-Runtime commit: 71bc875ccd]
2023-12-05 13:25:30 -05:00
Giovanni LB a633cba5ea : Adding ATT API extension to aqlprofile
Change-Id: Ic511cf871d5d98638d7041ca277f945ae8ced3a5


[ROCm/ROCR-Runtime commit: e8920cacc8]
2023-12-05 13:25:10 -05:00
David Yat Sin 38ce852b12 Add RISC-V support
Patch provided by user Xeonacid via github:
https://github.com/RadeonOpenCompute/ROCR-Runtime/pull/172/

Change-Id: I5f9086b536383093e7995b9cfdc19dab213f0265


[ROCm/ROCR-Runtime commit: 251601b20b]
2023-12-04 15:05:22 +00:00
Giovanni LB ecd768797e Extending AQLprofile API to include counter dimensions
Change-Id: If59489a085959f3f765a30e3e445df5151e30350


[ROCm/ROCR-Runtime commit: e0c6c5e5bf]
2023-12-04 15:05:22 +00:00
David Yat Sin 66b9fdc2d6 Implement async scratch reclaim
For devices where the CP FW supports asynchronous scratch reclaim, ROCr
is able to claw-back scratch memory that was assigned to an AQL queue.
With that ability, ROCr does not have to rely on using USO
(use-scratch-once) when assigning large amounts of memory to a queue.
If we reach a situation where we are running low on device memory, ROCr
will attempt to claw-back the scratch memory.

Change-Id: Iddf8ec84e37ab8b9fdc58bafbe2b61fe2acb6eb7


[ROCm/ROCR-Runtime commit: dca8f3a21d]
2023-12-04 15:05:22 +00:00
David Yat Sin 03e87e3d66 Update queue structure to support async reclaim
Update queue structure to add members required for asynchronous reclaim
mechanism and dual-scratch. CP will set the AMD_QUEUE_CAPS_ASYNC_RECLAIM
bit on queue-connect to indicate whether the new features are supported.

The new members are ignored by previous versions of CP FW

Change-Id: Ic8e9ef41c5b1d04f09b43bc9b44b31527863d10f


[ROCm/ROCR-Runtime commit: 0344c8c0b6]
2023-12-04 15:05:22 +00:00
Lang Yu 43ae931ad5 Revert "Revert "Add support for GC 11.5.0 and 11.5.1""
This reverts commit a8e34eaec8.

gfx1150/1151 is merged into mainline now.

Change-Id: Id179949318a37888c74abb5a8610d95bc2f22906


[ROCm/ROCR-Runtime commit: 991bbdcf24]
2023-12-04 15:03:31 +00:00
David Yat Sin b177c0e9ca Handle HW_EXCEPTION events
Add handler to handle HW exception events reported by underlying
drivers. These events are generally caused by GPU resets and need the
application to abort.
As an improvement, in the future, we can provide additional information
about the exception (e.g mode-reset level)

Change-Id: If3fb5f19f9fce181a9d3b5e34a5506725856e7b0


[ROCm/ROCR-Runtime commit: 750212e50e]
2023-11-20 14:49:26 +00:00
David Yat Sin eb664927dd Add LoongArch64 Support
Patch submitted by user Xinmudotmoe on github

Change-Id: I58fd035b4ec4856f20d63747ababd49fa9764348


[ROCm/ROCR-Runtime commit: 1a7de9588e]
2023-10-26 11:36:16 -04:00
Tony Tye fd757292fb Add AMD_AQL_FORMAT_INTERCEPT_MARKER vendor packet
Define AMD_AQL_FORMAT_INTERCEPT_MARKER AMD vendor AQL packet. Add
support to intercept queue to invoke a callback for these packets.

Change-Id: Ia58d5fe2171f563632b4edd6343e02585f49d149


[ROCm/ROCR-Runtime commit: 23b4ce501d]
2023-10-18 12:54:36 -04:00
David Yat Sin 48cb2f5a9e Add query for Xnack enabled
Add system query for whether Xnack is enabled on a system.

Change-Id: I2832110e4f33f6a951d13acd06636442debf27ae


[ROCm/ROCR-Runtime commit: 22becfb1e8]
2023-09-19 00:25:30 +00:00
David Yat Sin 5b9dcfd0d8 Update interface version for virtual memory APIs
Change-Id: Ifbf1af08ee7aa4d55387ff9786f6a61b89b56f88


[ROCm/ROCR-Runtime commit: 1e7b078628]
2023-08-30 17:01:13 -04:00
David Yat Sin 4e46eded66 Increment HSA API table stepping on new APIs
Add compile time asserts to force incrementing API table STEP versions
each time a new function is added to each table. This is required for
profiler team to be able to add preprocessor macros to determine which
versions contain the new APIs.

Also incrementing the major versions to 2 to indicate new numbering
scheme.

Change-Id: I148a436a5ceab6be3906f8263b40ea9b07841577


[ROCm/ROCR-Runtime commit: 03f2f69d16]
2023-08-29 21:59:36 +00:00
David Yat Sin 0637810752 Fix memory pool ALLOC_REC_GRANULE query
Also changed enum value to leave gap between enums that only exist in
hsa_region_info_t and enums that exist in both hsa_amd_memory_pool_info_t

Change-Id: I8f9f31200de66648e9328e4203ab283068c993f0


[ROCm/ROCR-Runtime commit: 4317f8dece]
2023-08-22 17:46:48 -04:00
Jonathan Kim 2994cfa875 Bump the number of SDMA engines for gfx940
GFX940 can support up to 16 SDMA engines so bump it.

Change-Id: I41a95e66383036735712e317a57b239d84fcb78d


[ROCm/ROCR-Runtime commit: 30982ff6aa]
2023-08-17 07:25:08 -04:00
David Yat Sin a8e34eaec8 Revert "Add support for GC 11.5.0 and 11.5.1"
Reverting this as current mainline compiler branch does not support
gfx1150/gfx1151 yet. Will bring back later.

This reverts commit 75ce1848cf.

Change-Id: I31ff4fb2d5817538094a7ffaeba96dd6a7d660c7


[ROCm/ROCR-Runtime commit: ebc51dd0eb]
2023-07-26 15:03:54 +00:00
David Yat Sin 2cfb82a1a6 Add agent query for nearest CPU agent
Add agent info query to return nearest CPU agent. This can be used to
determine which CPU agent is in the same NUMA region as the GPU agent.

Change-Id: I5400b4347ffbf4d2a836df31c4de443a38b0ecd1


[ROCm/ROCR-Runtime commit: 469defa78a]
2023-07-24 13:59:13 -04:00
Lang Yu 75ce1848cf Add support for GC 11.5.0 and 11.5.1
Signed-off-by: Lang Yu <Lang.Yu@amd.com>
Change-Id: I3c4116e78a5c1ddac2389f5fece57485bdb17f68


[ROCm/ROCR-Runtime commit: e877840197]
2023-07-22 16:06:22 +08:00
David Yat Sin bf41567189 Add retain handle and get allocation properties
Support function to retain allocation handle for memory mappings.
The get allocation properties function will return the current
allocation properties for existing memory mappings.

This is part of patch series for Virtual Memory API.

Change-Id: I0a53a11b6efc2b5bf9d463512a489a2abd812551


[ROCm/ROCR-Runtime commit: 687eb043d4]
2023-07-21 15:17:01 -04:00
David Yat Sin 0bcc573ed7 Support exporting and importing memory mappings
Support exporting  and importing dmabuf file descriptors for memory
mappings. The exported dmabuf file descriptors are shareable posix
file descriptors that can be used for cross-vendor, cross-device
and cross-process memory sharing.

This is part of patch series for Virtual Memory API.

Change-Id: I3673fc009f7e73bc26be8349e19f66e20d0607c5


[ROCm/ROCR-Runtime commit: b03c96c264]
2023-07-21 15:17:01 -04:00
David Yat Sin 933aac4cda Support Get and Set access for memory mappings
Mapping memory handles to virtual memory addresses do not make them
accessible. The set access function is needed to make the memory
mappings accessible to specific agents. The get access function
returns current access properties for individual agents.

This is part of patch series for Virtual Memory API.

Change-Id: I152ba0557fd2a802eb9d840568b68cdd1911b72c


[ROCm/ROCR-Runtime commit: 13fbd8a232]
2023-07-21 15:17:01 -04:00
David Yat Sin 203934445a Support mapping and unmapping memory handles
Add support for mapping and unmapping memory handles to virtual
address ranges.

This is part of patch series for Virtual Memory API.

Change-Id: If512d49ff4211e68f2064249add607a3200e458a


[ROCm/ROCR-Runtime commit: 179dcf1c77]
2023-07-21 15:17:01 -04:00
David Yat Sin 15aa42edb5 Support memory handles
Add support for creating and releasing memory handles. Memory
handles are memory allocations on device memory without a virtual
address.

This is part of patch series for Virtual Memory API.

Change-Id: I5dfb162eb1661621cce171b2870a3c93b24d840e


[ROCm/ROCR-Runtime commit: e4a84c4a9c]
2023-07-21 15:17:01 -04:00
David Yat Sin b219d0224d Support Virtual Address reservations
Add support for reserving virtual address ranges. Virtual address
ranges are addresses without any memory backing. These address ranges
need to be mapped to memory handles later.

This is part of patch series for Virtual Memory API.

Change-Id: I5d066e7421d6896f933f524312afc230a13d594e


[ROCm/ROCR-Runtime commit: 1085311f1a]
2023-07-21 15:17:01 -04:00
David Yat Sin 2a5f4263a8 Add check/query for virtual memory API support
Checks whether version of libdrm library installed on current
system supports the amdgpu_device_get_fd API. This API is
required to support the virtual memory API functions. The
amdgpu_device_get_fd function was introduced in libdrm-2.4.109.
Using a runtime check test instead of static dependency to be
able to support previous APIs on older versions of libdrm.
Add query for virtual memory API support.

This is part of patch series for Virtual Memory API.

Change-Id: Iec831eb24b5d1689c392e50ae86f4d52d4870ac4


[ROCm/ROCR-Runtime commit: e65edb35fc]
2023-07-21 15:17:01 -04:00
David Yat Sin 2be68e2502 Add query for recommended granularity size
Add new query for recommended granularity size. This is the
internal blocksize used. While the existing query for granularity
size returns the minimum size possible, it is recommended that
allocations and mappings are multiple of the recommended granularity
size to minimise internal memory fragmentation.

This is part of patch series for Virtual Memory API.

Change-Id: Ia82c8f073b2a2c47ecd26fbb0aba27b8b7cd965f


[ROCm/ROCR-Runtime commit: 3ebe1fdff9]
2023-07-21 15:17:01 -04:00
David Yat Sin 175265aef4 Add query for driver gpu_id
Add query OS driver node ID (gpu_id)

Change-Id: I72ebc54d8ae5dbcd1346535912160a642b1065ae


[ROCm/ROCR-Runtime commit: 60a0fd64c4]
2023-06-23 15:02:48 +00:00
Shweta Khatri 76cc9034ff Defined a new extended scope memory region
Added HSA_AMD_MEMORY_POOL_GLOBAL_FLAG_EXT_SCOPE_FINE_GRAINED flag to enable extended scope memory region
where the device-scope atomics act as system-scope atomics

Change-Id: I79fc3207cb630dfc68bed2f8aabd75f35fe80b12


[ROCm/ROCR-Runtime commit: 77bf357647]
2023-06-20 11:00:05 -04:00
David Yat Sin 8c3acb3974 Update documentation for IPC handles
Explicitly mention that IPC handles can only be created on GPU agents.

Change-Id: I19bc3578d6e5243c795bf6fbf981ea4bd3bfc2e8


[ROCm/ROCR-Runtime commit: 5e4490f180]
2023-06-14 16:21:26 -04:00
Ammar ELWazir 5675ed837a : Adding support to UMC & MMEA System Blocks
Change-Id: I92601f37757e0cff3f1fdc10f2e5e0db51c1ee2d


[ROCm/ROCR-Runtime commit: fc603d58d2]
2023-06-08 21:22:19 +00:00
David Yat Sin 3661d76c74 Bump interface versions due to hsa_amd_memory_async_copy_on_engine added
Change-Id: Iff36719e800280d58217647bb70d3b5d5fcc91fe


[ROCm/ROCR-Runtime commit: b290d65ec9]
2023-05-26 12:04:06 +00:00
David Yat Sin 3345ada378 Adding gfx941 and gfx942
Adding support for gfx941 and gfx942 ISAs.
gfx940 ISA will use sc0:1 sc1:1 on load/store operations
gfx942 ISA will use default load/store operations

Change-Id: If1efbef86f59e2cf2d48fe359cd4166405a0a579


[ROCm/ROCR-Runtime commit: 41f6d0426d]
2023-05-23 11:13:16 -04:00
David Yat Sin b8e97a8d1b Update documentation for hsa_amd_pointer_info
Update documentation for hsa_amd_pointer_info to clarify which fields
are invalid when the allocation type is HSA_EXT_POINTER_TYPE_UNKNOWN.

Change-Id: Idaed985962c4a98d281ebe01bef8ec2459da3985


[ROCm/ROCR-Runtime commit: 39feb83b88]
2023-05-16 18:36:54 -04:00
David Yat Sin 101755c207 Add query for number of XCCs per agent
Change-Id: I4b694b4904ba0326c998356388a62c19a972a7ff


[ROCm/ROCR-Runtime commit: f024d21e3d]
2023-04-27 16:15:59 -04:00
Mike Li fe9b01e916 Added gfx940 ISA
Signed-off-by: Mike Li <Tianxinmike.Li@amd.com>
Change-Id: Icb1830fe186abc69fe7ee709b7f12b882cab9e87


[ROCm/ROCR-Runtime commit: bd98a1e5bf]
2023-04-27 16:08:58 -04:00
David Yat Sin 6812573d06 Change error reported when receiving code 128
We used to report HSA_STATUS_ERROR_INVALID_ISA when receiving error code
128, but there are several other reasons why we could be exceeding
number of VGPRs, so updating the error code.

Change-Id: I6a6980d5b07b09c93d00dee5207a0d52399bc77e


[ROCm/ROCR-Runtime commit: f43a284b8e]
2023-04-14 09:12:07 -04:00
Sean Keely deee152909 Add support for exporting portable handles to GPU allocations.
Adds hsa_amd_portable_export_dmabuf and hsa_amd_portable_close_dmabuf
which allow obtaining dmabuf handles to rocr allocations.  These handles
may be shared with other APIs to support cross vendor & cross device
memory sharing.
Adds query to return whether dmabuf export is supported

Signed-off-by: Jonathan Kim <Jonathan.Kim@amd.com>
Signed-off-by: David Yat Sin <David.YatSin@amd.com>

Change-Id: I7f98501087d9563d07fc2cb428cc886b1e518b1e


[ROCm/ROCR-Runtime commit: 42243c1e8f]
2023-03-06 12:39:01 -05:00
David Yat Sin d022746cb7 Use mwaitx when busy-waiting signals
Use mwaitx instructions when busy waiting for signals to reduce CPU
energy usage.
This can be disabled by setting HSA_ENABLE_MWAITX=0

Change-Id: Ic207895a491b2bf6dacba47ef0921df3faad5b5a


[ROCm/ROCR-Runtime commit: cc48dfdbff]
2023-02-22 16:55:43 +00:00
Jonathan Kim ff620e9fdc Add interface to DMA copy directly to a target engine.
Change-Id: Ic87cfeabb11c1a465f98f3f444d39955f5300525


[ROCm/ROCR-Runtime commit: 30920fc94d]
2023-02-13 13:50:49 -05:00
Jonathan Kim f161963c09 Make SDMA engine availability status queryable.
Report the availability of SDMA engines for memory copies.

Change-Id: Ie31b02d6b65355122bb8c98bc73700a59bee166e


[ROCm/ROCR-Runtime commit: 8f27f495c6]
2023-02-13 13:50:49 -05:00
Jonathan Kim 9021f5970d Make the number of per agent SDMA engines queryable.
Change-Id: Iae1cc9b7ec783fdda05f9384f0ad0327ea1a8cc3


[ROCm/ROCR-Runtime commit: 4f283d9bb3]
2023-02-13 13:50:49 -05:00
David Yat Sin 8a86cddd3a Add query for IOMMU support
Reporting whether IOMMU V2 is supported.
IOMMU V1 support is not relevant to user, so not reporting it.

Change-Id: I77389484a87a352da9c2f7b2a5d9de264f90ee53


[ROCm/ROCR-Runtime commit: e30be76f37]
2023-01-19 11:33:21 -05:00
David Yat Sin 580ce4fd25 Add memory pool query to return location
Change-Id: I240b77119d7b8ccfc5ff6a3190d6669d69f243e8


[ROCm/ROCR-Runtime commit: 722794e258]
2023-01-19 08:45:05 -05:00
David Yat Sin 93c4ffe473 Add Stream Performance Monitor(SPM) APIs
Change-Id: I0d48782887814ef245b7e0182e2d5570aa8c3f50


[ROCm/ROCR-Runtime commit: 6bfe57aeb2]
2022-12-08 13:56:29 -05:00
David Yat Sin 652a617846 Add agent info for fw and sdma ucode
Add two new agent info fields:
HSA_AMD_AGENT_INFO_UCODE_VERSION
HSA_AMD_AGENT_INFO_SDMA_UCODE_VERSION

Change-Id: I51cb853724b23a26e945e5c1ac32c16d0cb3bc31


[ROCm/ROCR-Runtime commit: ecdebef0b9]
2022-12-07 19:07:31 -05:00
David Belanger 9f2c88c46a Initial changes for gfx1101, based on gfx1100/gfx1102 implementation.
Change-Id: I949c1027ccabf38b4f924590e42e7327dc550f73
Signed-off-by: David Belanger <david.belanger@amd.com>
Reviewed-by: Jonathan Kim <jonathan.kim@amd.com>


[ROCm/ROCR-Runtime commit: a0d3db6e8d]
2022-10-13 09:28:39 -04:00