AI/rocm-systems
2
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

rocr: Update for new async scratch reclaim

Browse Source 
Updating ROCr code to match new handshake protocol with CP FW for
asynchronous scratch reclaim.
Increase previous limits when scratch reclaim feature is available.
This commit is contained in:
David Yat Sin
2025-02-06 16:21:07 +00:00
committed by Yat Sin, David
parent 2f8a9b28d0
commit aa2f98e6f9
9 changed files with 232 additions and 88 deletions
Download Patch File Download Diff File Expand all files Collapse all files
runtime/hsa-runtime/core/util/flag.h
+6 -2
View File
@@ -66,8 +66,9 @@ class Flag {
// Lift limit for 2.10 release RCCL workaround. This limit is not used when asynchronous scratch
// reclaim is supported
const size_t DEFAULT_SCRATCH_SINGLE_LIMIT = 146800640; // small_limit >> 2;
const size_t DEFAULT_PCS_MAX_DEVICE_BUFFER_SIZE = 256 * 1024 * 1024;
const size_t DEFAULT_SCRATCH_SINGLE_LIMIT = (140 * (1UL<<20)); // small_limit >> 2;
const size_t DEFAULT_SCRATCH_SINGLE_LIMIT_ASYNC_PER_XCC = (3 * (1UL<<30)); // 3 GB
const size_t DEFAULT_PCS_MAX_DEVICE_BUFFER_SIZE = (256 * (1UL<<20)); //256 MB
explicit Flag() { Refresh(); }
@@ -135,6 +136,9 @@ class Flag {
// On GPUs that support asynchronous scratch reclaim
// Scratch memory sizes > HSA_SCRATCH_SINGLE_LIMIT_ASYNC will trigger a use-once scheme
// Note: This only sets the initial value for the threshold. If
// hsa_amd_agent_set_async_scratch_limit is called after initialization, the threshold
// will be updated.
if (os::IsEnvVarSet("HSA_SCRATCH_SINGLE_LIMIT_ASYNC")) {
var = os::GetEnvVar("HSA_SCRATCH_SINGLE_LIMIT_ASYNC");
char* end;