The alternate scratch memory is used for dispatches that have a low
number of waves but relatively large wave size.
This allows us to keep the tmpring_size.bits.WAVES field of the main
scratch to full occupancy.
Change-Id: I32d240fac4b7d38200d1eebc1b0fdc8a823920d3
The alternate scratch memory is used for dispatches that have a low
number of waves but relatively large wave size.
This allows us to keep the tmpring_size.bits.WAVES field of the main
scratch to full occupancy.
Change-Id: I32d240fac4b7d38200d1eebc1b0fdc8a823920d3
[ROCm/ROCR-Runtime commit: a7a3358067]
For devices where the CP FW supports asynchronous scratch reclaim, ROCr
is able to claw-back scratch memory that was assigned to an AQL queue.
With that ability, ROCr does not have to rely on using USO
(use-scratch-once) when assigning large amounts of memory to a queue.
If we reach a situation where we are running low on device memory, ROCr
will attempt to claw-back the scratch memory.
Change-Id: Iddf8ec84e37ab8b9fdc58bafbe2b61fe2acb6eb7
For devices where the CP FW supports asynchronous scratch reclaim, ROCr
is able to claw-back scratch memory that was assigned to an AQL queue.
With that ability, ROCr does not have to rely on using USO
(use-scratch-once) when assigning large amounts of memory to a queue.
If we reach a situation where we are running low on device memory, ROCr
will attempt to claw-back the scratch memory.
Change-Id: Iddf8ec84e37ab8b9fdc58bafbe2b61fe2acb6eb7
[ROCm/ROCR-Runtime commit: dca8f3a21d]
Separate the event handler and scratch handler portions of the code into
separate functions.
Change-Id: Ifdb7461e816b0f2d3c1c0a74d6f020b4d6fc736c
[ROCm/ROCR-Runtime commit: 64070a9acc]
Update queue structure to add members required for asynchronous reclaim
mechanism and dual-scratch. CP will set the AMD_QUEUE_CAPS_ASYNC_RECLAIM
bit on queue-connect to indicate whether the new features are supported.
The new members are ignored by previous versions of CP FW
Change-Id: Ic8e9ef41c5b1d04f09b43bc9b44b31527863d10f
Update queue structure to add members required for asynchronous reclaim
mechanism and dual-scratch. CP will set the AMD_QUEUE_CAPS_ASYNC_RECLAIM
bit on queue-connect to indicate whether the new features are supported.
The new members are ignored by previous versions of CP FW
Change-Id: Ic8e9ef41c5b1d04f09b43bc9b44b31527863d10f
[ROCm/ROCR-Runtime commit: 0344c8c0b6]
For gfx11, the trap_handler fails to recognize a trap id 3 and report
the exception to the debugger if the debugger is attached.
This is because the 2nd level trap handler looks for the DEBUG_ENABLED
bit in ttmp13 instead of ttmp11. This bit is set by the 1st level trap
handler and is part of the 1st/2nd level trap handler ABI.
Change-Id: Ib36361f53d9bcbbed52320d8c3a9ab2c0b28c7cd
For gfx11, the trap_handler fails to recognize a trap id 3 and report
the exception to the debugger if the debugger is attached.
This is because the 2nd level trap handler looks for the DEBUG_ENABLED
bit in ttmp13 instead of ttmp11. This bit is set by the 1st level trap
handler and is part of the 1st/2nd level trap handler ABI.
Change-Id: Ib36361f53d9bcbbed52320d8c3a9ab2c0b28c7cd
[ROCm/ROCR-Runtime commit: 6916ce358a]
This reverts commit a8e34eaec8.
gfx1150/1151 is merged into mainline now.
Change-Id: Id179949318a37888c74abb5a8610d95bc2f22906
[ROCm/ROCR-Runtime commit: 991bbdcf24]
Skip Extended-scope memory pool as allocation is very close to
fine-grain/coarse-grain but with just different PTE flags.
Only test coarse grain on CPU agent other than the first CPU agent.
Stop bisecting the max size once we are withing 5% to total size for
these pool to speed this test on large memory pools.
Change-Id: I77d1b45a1752ef092dda7c7f27723ea0a292a612
Skip Extended-scope memory pool as allocation is very close to
fine-grain/coarse-grain but with just different PTE flags.
Only test coarse grain on CPU agent other than the first CPU agent.
Stop bisecting the max size once we are withing 5% to total size for
these pool to speed this test on large memory pools.
Change-Id: I77d1b45a1752ef092dda7c7f27723ea0a292a612
[ROCm/ROCR-Runtime commit: cb5a29955b]
SDMA4.4 and SDMA5.2+ has increased it's available copy size to 2^30 bytes
represented by exponent as bits set in the COUNT field of the
linear copy.
Also note that the full 2^22 byte limit is available from SDMA4 onwards
as it has corrected the 0x3fffe0 HW limitation from SDMA3.
As copy limit has increase, this can change system performance
so provide env var HSA_ENABLE_SDMA_COPY_SIZE_OVERRIDE=0 to fall
back to the original 0x3fffe0 limit for debugging purposes.
Change-Id: I0fb6e5378f68e5b8a00ff559271691a943ee06ee
SDMA4.4 and SDMA5.2+ has increased it's available copy size to 2^30 bytes
represented by exponent as bits set in the COUNT field of the
linear copy.
Also note that the full 2^22 byte limit is available from SDMA4 onwards
as it has corrected the 0x3fffe0 HW limitation from SDMA3.
As copy limit has increase, this can change system performance
so provide env var HSA_ENABLE_SDMA_COPY_SIZE_OVERRIDE=0 to fall
back to the original 0x3fffe0 limit for debugging purposes.
Change-Id: I0fb6e5378f68e5b8a00ff559271691a943ee06ee
[ROCm/ROCR-Runtime commit: 81c64228e0]
To be able to trace memcpy asynchronously, both dst and src agents need to have profiling enabled and the api for enabling profiling was only enabling for gpu agents. CPU agents didn't have profiling enabled so the signal owner could not be known. hsa_amd_profiling_get_async_copy_time will fail with an HSA status error because it can't read the agent for the given signal.
Change-Id: Ie165e0e39b8fcd6992a55695b9ffcead10a8e812
To be able to trace memcpy asynchronously, both dst and src agents need to have profiling enabled and the api for enabling profiling was only enabling for gpu agents. CPU agents didn't have profiling enabled so the signal owner could not be known. hsa_amd_profiling_get_async_copy_time will fail with an HSA status error because it can't read the agent for the given signal.
Change-Id: Ie165e0e39b8fcd6992a55695b9ffcead10a8e812
[ROCm/ROCR-Runtime commit: ae1da390bd]
- Update CMakeLists.txt
- find_package for rocprofiler-register
- this is an optional package until rocprofiler-register is added to the CI
- define HSA_VERSION_{MAJOR,MINOR,PATCH} ppdefs
- Update runtime.cpp
- include <rocprofiler-register/rocprofiler-register.h>
- if rocprofiler-register succeeds, do not support v1 unless explicitly requested
Change-Id: I8f48bbf3f6b52fb91ddade2f198491a1256035fe
- Update CMakeLists.txt
- find_package for rocprofiler-register
- this is an optional package until rocprofiler-register is added to the CI
- define HSA_VERSION_{MAJOR,MINOR,PATCH} ppdefs
- Update runtime.cpp
- include <rocprofiler-register/rocprofiler-register.h>
- if rocprofiler-register succeeds, do not support v1 unless explicitly requested
Change-Id: I8f48bbf3f6b52fb91ddade2f198491a1256035fe
[ROCm/ROCR-Runtime commit: f9cf1852e5]
Remove override that forces ROCr image blit source and ROCr test to use
code object version 4 now that mainline has been updated to version 5.
Change-Id: I94681e86835c0e382475306ead4cd4132a2ee78f
Remove override that forces ROCr image blit source and ROCr test to use
code object version 4 now that mainline has been updated to version 5.
Change-Id: I94681e86835c0e382475306ead4cd4132a2ee78f
[ROCm/ROCR-Runtime commit: 2f847cf05f]
C++ does not allow const qualifier on function type, even if we add it it will get ignored and clang will fail with failed cast from const void* to func*. const_cast here is necessary to make it work.
Change-Id: I72cec8d9e715bdf9e163cb9b08393dd733dafaf2
[ROCm/clr commit: 5b5213adf5]
C++ does not allow const qualifier on function type, even if we add it it will get ignored and clang will fail with failed cast from const void* to func*. const_cast here is necessary to make it work.
Change-Id: I72cec8d9e715bdf9e163cb9b08393dd733dafaf2
Giovanni LB
David Yat Sin
Shweta Khatri
Lancelot SIX
Lang Yu
Jonathan Kim
Youssef Aly
Jonathan R. Madsen
Jatin Chaudhary
Ioannis Assiouras
Bertan Dogancay
Wenkai Du
Karl W Schulz
Aryan Salmanpour
colramos-amd