Commit gráf

368 Commit-ok

Szerző SHA1 Üzenet Dátum
Chris Freehill 287986ab65 rocr: Add hsa_amd_portable_export_dmabuf_v2
The original version of hsa_amd_portable_export_dmabuf() did not
consider the conditions under which a dmabuf could be shared.
In the new version (hsa_amd_portable_export_dmabuf_v2()), the caller
can specify the flag HSA_AMD_DMABUF_MAPPING_TYPE_PCIE, which means they
want to share the dmabuf over PCIe. In that case, the new code will check
that if it is a PCIe GPU and it is not in a XGMI Hive then if
large-BAR is not supported, we will return an error.


[ROCm/ROCR-Runtime commit: 3a9d14bb66]
2025-06-09 15:42:58 -05:00
Alysa Liu 03430838af rocr: Remove structurally dead code
Remove unreachable return statement.

Signed-off-by: Alysa Liu <Alysa.Liu@amd.com>


[ROCm/ROCR-Runtime commit: 9b3d15e68d]
2025-06-09 14:01:39 -04:00
Alysa Liu 6de1c81b71 rocr: Fix inefficient copy operations
Refactor variable assignments to use std::move() where appropriate.
Updat function headers to accept parameters by const& where appropriate.

Signed-off-by: Alysa Liu <Alysa.Liu@amd.com>


[ROCm/ROCR-Runtime commit: f6c8cbd293]
2025-06-02 11:18:36 -04:00
Alysa Liu 65f5ce6f0a rocr: Fixed inefficient copy operations
Changed variable assignments to use std::move() where appropriate.
Changed function headers to pass string arguments by reference where appropriate.

Signed-off-by: Alysa Liu <Alysa.Liu@amd.com>


[ROCm/ROCR-Runtime commit: ae6851dbb4]
2025-06-02 11:18:36 -04:00
Alysa Liu 88dd451c64 rocr: Fixed inefficient copy operations
Changed variable assignments to use std::move() where appropriate

Signed-off-by: Alysa Liu <Alysa.Liu@amd.com>


[ROCm/ROCR-Runtime commit: 369d89ade3]
2025-06-02 11:18:36 -04:00
David Yat Sin 1b1d4e017a rocr:Fix compile warnings
[ROCm/ROCR-Runtime commit: 11da1293de]
2025-05-28 16:12:02 -04:00
David Yat Sin 39ecc88315 rocr: Remove deprecated doorbell type 1 support
[ROCm/ROCR-Runtime commit: 0d70045817]
2025-05-28 16:12:02 -04:00
David Yat Sin b83b8b4535 rocr: Remove deprecated queue doubleMap code
[ROCm/ROCR-Runtime commit: 4bae509296]
2025-05-28 16:12:02 -04:00
David Yat Sin e84a855c98 rocr: Remove queue_full_workaround code
Remove deprecated queue_full_workaround code as gfx7 and gfx8 GPUs are
EoL.


[ROCm/ROCR-Runtime commit: b8434529a5]
2025-05-28 16:12:02 -04:00
David Yat Sin 4ecd0382b7 rocr: update required CP FW version
Update required CP FW version required for async-scratch memory support
on gfx950.


[ROCm/ROCR-Runtime commit: 04dbf769f6]
2025-05-28 13:03:58 -04:00
David Yat Sin 5e7bd6145d rocr: Fix compile error when using clang
[ROCm/ROCR-Runtime commit: 9d38ca0d22]
2025-05-27 23:56:28 -04:00
David Yat Sin 342e478e7d rocr: Perform memcpy for small code-object loads
On large BAR systems, for small-sized code-objects, we get performance
using direct memcpy due to latencies when doing the blit-copy.


[ROCm/ROCR-Runtime commit: da2607024b]
2025-05-22 18:39:19 -04:00
David Yat Sin 9c5bb61708 rocr: Perform range based cache invalidates
Invalidate only the address range that covers the newly copied
code-object. This avoids invalidating I$ for old code objects and thus
might increase I$ hit rate.


[ROCm/ROCR-Runtime commit: e969e01f54]
2025-05-22 18:39:19 -04:00
Jiadong Zhu b99015f30a rocr/dtif: use default signal for intercept queue for DTIF
Signed-off-by: Jiadong Zhu <Jiadong.Zhu@amd.com>
Reviewed-by: David Yat Sin <David.YatSin@amd.com>


[ROCm/ROCR-Runtime commit: 0f9d2b836c]
2025-05-13 16:44:31 -04:00
Jiadong Zhu a0dc167541 rocr/dtif: add hsaKmtQueueRingDoorbell in thunk loader
hsaKmtQueueRingDoorbell is specfic to DTIF backend

Signed-off-by: Flora Cui <flora.cui@amd.com>
Signed-off-by: Jiadong Zhu <Jiadong.Zhu@amd.com>
Signed-off-by: Aaron Liu <aaron.liu@amd.com>
Reviewed-by: Shane Xiao <shane.xiao@amd.com>
Reviewed-by: David Yat Sin <David.YatSin@amd.com>


[ROCm/ROCR-Runtime commit: e2d767879d]
2025-05-13 16:44:31 -04:00
Aaron Liu 008bbd94d5 rocr/dtif: add CreateThunkInstance/DestroyThunkInstance interfaces
Signed-off-by: Aaron Liu <aaron.liu@amd.com>
Reviewed-by: David Yat Sin <David.YatSin@amd.com>


[ROCm/ROCR-Runtime commit: e9088d6e47]
2025-05-13 16:44:31 -04:00
Aaron Liu c6ffc85a47 rocr/dtif: add DRM APIs wrapper in thunk loader
Signed-off-by: Aaron Liu <aaron.liu@amd.com>
Reviewed-by: David Yat Sin <David.YatSin@amd.com>


[ROCm/ROCR-Runtime commit: 0cd4ddd62b]
2025-05-13 16:44:31 -04:00
Aaron Liu 6cf184a0d4 rocr/dtif: replace hsakmt interfaces with HSAKMT_CALL(...)
Signed-off-by: Aaron Liu <aaron.liu@amd.com>
Reviewed-by: David Yat Sin <David.YatSin@amd.com>


[ROCm/ROCR-Runtime commit: 1b79caa214]
2025-05-13 16:44:31 -04:00
Aaron Liu 87dcbf1255 rocr/dtif: add thunk loader to wrap hsaKmt APIs
For native and DTIF backends, unify to use HSAKMT_CALL(...) to call
hsaKmt APIs.

Signed-off-by: Aaron Liu <aaron.liu@amd.com>
Reviewed-by: David Yat Sin <David.YatSin@amd.com>


[ROCm/ROCR-Runtime commit: 7ba77fb193]
2025-05-13 16:44:31 -04:00
christian-heusel 6c8a2da29a rocr:Add missing cstdint include
[ROCm/ROCR-Runtime commit: 5cc61b714d]
2025-05-06 20:52:48 -04:00
Tony Gutierrez ce61e3301b rocr: Add large_bar_enabled var to the GPU agent
Adds a bool to the GPU agent and a public member method to
check if the GPU supports large BAR. This is needed so we can
check if large BAR is supported when a user tries to allocate
an AQL queue in device memory on a given GPU agent.

Also adds an exception to the AQL queue if device-side AQL queues
are requested and the GPU owner of the AQL doesn't support large
BAR. Otherwise, ROCr will currently allow device-side queues
that can cause faults when the user tries to touch their ring
buffers and the user will not know why the faults are occuring.

This relies on the fact that the KFD does not exposed any links
from the CPU to the GPU if large BAR is not enabled (though
links from the GPU to the CPU may still be exposed by the KFD).


[ROCm/ROCR-Runtime commit: f2c482d923]
2025-04-23 15:53:29 -04:00
Tony Gutierrez 6f37386eb2 rocr: Flags to alloc queue buf/struct in dev mem
This builds on a prior change that allowed for allocating
a user-mode queue's packet buffer in device memory to also
allocate the queue struct in device memory. This provides
additional latency benefits particularly for cases where
dispatches are performed from the GPU itself. Flags are
added to support the various use cases.


[ROCm/ROCR-Runtime commit: 6e3c375bf1]
2025-04-23 15:53:29 -04:00
Tony Gutierrez 3ebcf3020f rocr/libhsakmt: Add coarse-grain allocator to GPU
[ROCm/ROCR-Runtime commit: adbc0495e2]
2025-04-23 15:53:29 -04:00
Saleel Kudchadker 945d6da90b rocr: return preferred SDMA engine mask
- Add a new AMD extension API to return preferred SDMA engine mask.
This can use used in conjunction with copy_on_engine API to get
optimal bandwidth.


[ROCm/ROCR-Runtime commit: 57c0c643ce]
2025-04-22 13:28:38 -07:00
Shane Xiao 8d34f4e12d rocr: Add rec sdma engines with limited XGMI SDMA engine
This patch will adds recommended sdma supports with
limited XGMI SDMA engine. It will use one PCIe SDMA
to do gpu <-> gpu copies which will help improve all
to all copy performance.

Signed-off-by: Shane Xiao <shane.xiao@amd.com>


[ROCm/ROCR-Runtime commit: 6a63170b38]
2025-04-11 23:54:15 +08:00
Yiannis Papadopoulos 96b7e42776 rocr/aie: Increment write pointer upon packet submission
[ROCm/ROCR-Runtime commit: 2d2c47bdef]
2025-04-08 15:36:40 -05:00
Yiannis Papadopoulos f53a9c72c4 rocr/aie: Using PDI address instead of cu_mask for dispatch. Automatic hw ctx reconfiguration upon new PDI addition.
[ROCm/ROCR-Runtime commit: c63e01724c]
2025-04-03 15:13:20 -05:00
Yiannis Papadopoulos c142b04fc1 rocr/aie: Bundling XDNA BOs and addresses, adding cleanup guard in case of error
[ROCm/ROCR-Runtime commit: e55503e7f8]
2025-03-27 13:15:13 -04:00
Yiannis Papadopoulos 4c0e8b5f70 rocr/aie: Avoiding XdnaDriver class in queue API
[ROCm/ROCR-Runtime commit: f4e1c9b0ba]
2025-03-27 13:15:13 -04:00
David Yat Sin edcc3a1ed5 rocr: Release agent resources before pools
Adding a general stage for agents to release their resources on
shutdown. This avoids a circular dependency during shutdown because
we have to delete allocated resources before deleting memory pools, but
we also have to delete memory pools before destroying agents.


[ROCm/ROCR-Runtime commit: 947391deac]
2025-03-25 14:25:04 -04:00
jordans 938b34da24 hsakmt: Initial Commit for the HSA KMT Model
The over arching goal it so provide an API that pre-silicon models can latch into for software bring up.# Please enter the commit message for your changes. Lines starting


[ROCm/ROCR-Runtime commit: d4b85b6bf5]
2025-03-18 16:22:17 -04:00
Tony Gutierrez 3b30b8a975 rocr: Remove KMT usage from AMD ext
Use the core Driver in AMD's HSA extension API to make it
agnostic to the underlying OS and kernel-mode driver.


[ROCm/ROCR-Runtime commit: d3a4dc9687]
2025-02-25 21:51:52 -05:00
Khatri, Shweta e00c926d27 rocr: Adding support for Stochastic PC Sampling for gfx94x (#47)
Change-Id: Ide4c2e25b88f1f25ea4ce35a619b93963c0355ee

[ROCm/ROCR-Runtime commit: 322a794cf6]
2025-02-22 00:13:08 -05:00
Tony Gutierrez 727159b4db rocr: Remove KMT usage from CPU agent
Use the core Driver object in the CPU agent to make it OS/driver
agnostic.

Implement the GetMemoryProperties() and GetCacheProperties methods
for the KFD driver.


[ROCm/ROCR-Runtime commit: a9f6bc8d0e]
2025-02-21 10:00:38 -05:00
David Yat Sin 5905b82579 rocr: Update for new async scratch reclaim
Updating ROCr code to match new handshake protocol with CP FW for
asynchronous scratch reclaim.
Increase previous limits when scratch reclaim feature is available.


[ROCm/ROCR-Runtime commit: aa2f98e6f9]
2025-02-19 21:02:00 -05:00
Saleel Kudchadker d5f08e6fa8 rocr: Skip uSleep for non-interrupt signals
- When waiting on non-interrupt signals, do not uSleep. This causes
  regressions compared to interrupt signal usage.
- Cleanup code.

Change-Id: I706bda0b13e64ffec0b607c1915d8380a2ce0dea


[ROCm/ROCR-Runtime commit: 890399a7cf]
2025-02-06 23:48:35 -05:00
Yiannis Papadopoulos 71165b9460 rocr: Changing to a device SVM flag
Change-Id: Ib085801d23604eeef0a17a05cf2b298170fb3d24


[ROCm/ROCR-Runtime commit: 03bd4c9508]
2025-01-28 17:06:16 +00:00
Yiannis Papadopoulos f107054dbe rocr: Use SVM information to separate dev heap
Use SVM information from user accessible memory

Change-Id: I8fad37eb1a90dc1f5827a096552130a3fd6187f4


[ROCm/ROCR-Runtime commit: 144e7674d1]
2025-01-28 17:05:52 +00:00
Yiannis Papadopoulos 17807d78bd rocr/aie: AIE agent memory pools correct size and user data pool
Change-Id: I831711a7d1cdc36cbc9ed30bd74d0dc984228ce7


[ROCm/ROCR-Runtime commit: 1d8a77db34]
2025-01-28 10:48:16 -05:00
Yiannis Papadopoulos 428cc5b47c rocr/aie: Add dma-buf import support for AIEAgents via the Driver interface
Change-Id: I70f8d8772dda7c06944d75042cb3034ddd89aff4


[ROCm/ROCR-Runtime commit: 26bfa0b8f6]
2025-01-27 15:22:46 -05:00
Tony Gutierrez ff52d6fc13 rocr: Add WaitMultiple to core Signal
Replaces WaitAny with WaitMultiple to more closely align with the
underlying driver API for waiting on multiple events.

WaitMultiple adds a single parameter, wait_on_all, to the WaitAny
interface providing a single function for waiting on multiple
events when we only need AND and OR semantics for the signal
checking logic.

Change-Id: I68a4a45d48151d9d69aef02fd8f7263b9e6c0e75


[ROCm/ROCR-Runtime commit: 8a38f121ea]
2025-01-27 09:21:43 -05:00
Ben Vanik bda034bb82 rocr: Fix HostQueue to obey the alignment requirement
Change-Id: I06542e9ff94e826ca0abba0328b301fec50a95ea


[ROCm/ROCR-Runtime commit: 7d64fe49fa]
2025-01-24 12:08:11 -05:00
Tony Gutierrez e82871f20b rocr: Generalize driver discovery
Generalize the driver discovery and move driver-specific
functionality to the concrete driver implementations.
Currently, this process is tightly coupled to the hsakmt
which is GPU and OS specific.

Change-Id: Ie1c53fef407a71b5ec4c6eaf3a3ed00871184409


[ROCm/ROCR-Runtime commit: 15107afb11]
2025-01-23 15:09:14 -05:00
Tony Gutierrez c325fac9ba rocr: Make Open() and Close() virtual in Driver
Change-Id: Iac054c08383b080ca2b2ec6d65019bf2f083b763


[ROCm/ROCR-Runtime commit: 77fa5af618]
2025-01-23 15:09:06 -05:00
Tony Gutierrez d6504e1f2d rocr: Forward declare Driver in the Agent class
Change-Id: Ib27081bf31446af92602f723f352fb75ec3f378e


[ROCm/ROCR-Runtime commit: 8bbc44d51b]
2025-01-23 15:08:59 -05:00
Apurv Mishra 23ab95b5f2 rocr: multiple uninitialized and unused variables
Minor modifications to multiple source and header
files based on Coverity report

Change-Id: I4a73d0f56640983c4d5124e13c8c280245cca672
Signed-off-by: Apurv Mishra <apurv.mishra@amd.com>


[ROCm/ROCR-Runtime commit: 699d0140be]
2024-12-18 10:11:13 -05:00
Apurv Mishra 2f2cb50cf2 rocr: refactor of runtime.cpp based on Coverity
Add return checks, initialization and clean
redundant memory operations

fix 1: check return value of 'setsockopt' for error
fix 2: check return value of 'PtrInfo' for error
fix 3: move 'tool_names' instead of copying
fix 4: call 'munmap' for 'va' only once
fix 5: use 'ssize' for possible return values of -1 (err)
fix 6: add missing initialization in constructors
fix 7: add initialization for some scalars and pointers

Change-Id: I07d90e36d4e1fe48c4de4f44e18083e5ed4c5fbc
Signed-off-by: Apurv Mishra <apurv.mishra@amd.com>


[ROCm/ROCR-Runtime commit: 441bd9fe6c]
2024-12-18 10:06:55 -05:00
Eddie Richter 8ea388af92 rocr/aie: AIE Queue Processing
Change-Id: I681c971ba7229037ca85d5529838aa7bbe5820e2


[ROCm/ROCR-Runtime commit: e9cc839b2b]
2024-12-10 10:50:02 -05:00
Chris Freehill 9c7e73ff98 rocr: Dynamically allocate static global memory
To allow non-POD global variables to last until the last thread
has exited, use "new" to allocate the memory instead of static
allocation.

Change-Id: Ica571b61ff8068a52e472c49cb1c44917e60c8c8


[ROCm/ROCR-Runtime commit: 0878deda17]
2024-11-07 09:53:31 -05:00
Yiannis Papadopoulos 83513d4daf rocr: Adding pointer to the owner driver in Agent class
Change-Id: If913d7c7e4caf6d6e6eee3a858a27c6027c2923f


[ROCm/ROCR-Runtime commit: 2837825b14]
2024-10-31 12:29:10 -04:00