When oversubscribing SDMA gangs, a circular deadlock can occur since
gang enqueue is staggered with respect to SDMA engine leader based
on source to destination.
As a result, an enqueued leader may be waiting on a gang item that is
waiting on another enqueued leader or gang item and so on.
To prevent this, first lock the submission to ensure dma status query
and submissions are atomic. Once this is in place, be more stringent
with ganging in that all SDMA engines must be available in order to gang.
Finally, re-enable SDMA ganging by default.
Change-Id: I4511e3487db9d26475b5aece4897f10168cc5322
When oversubscribing SDMA gangs, a circular deadlock can occur since
gang enqueue is staggered with respect to SDMA engine leader based
on source to destination.
As a result, an enqueued leader may be waiting on a gang item that is
waiting on another enqueued leader or gang item and so on.
To prevent this, first lock the submission to ensure dma status query
and submissions are atomic. Once this is in place, be more stringent
with ganging in that all SDMA engines must be available in order to gang.
Finally, re-enable SDMA ganging by default.
Change-Id: I4511e3487db9d26475b5aece4897f10168cc5322
[ROCm/ROCR-Runtime commit: 8f21793a3e]
xGMI for compute partitioning in non-SPX modes does not have
a reported bandwith.
Fix it to at most 2 since each partition is either bounded
by the number of xGMI links or the number of available
SDMA contexts.
Change-Id: I09094bd7548d9eee6f039b0efe849838e5de166e
xGMI for compute partitioning in non-SPX modes does not have
a reported bandwith.
Fix it to at most 2 since each partition is either bounded
by the number of xGMI links or the number of available
SDMA contexts.
Change-Id: I09094bd7548d9eee6f039b0efe849838e5de166e
[ROCm/ROCR-Runtime commit: 4c74e47e91]
There's no need to keep looking in the list once we find a ganged agent.
Change-Id: Ia0b9b484c88221a7966a814456942c19b1741978
[ROCm/ROCR-Runtime commit: f8664e88e0]
__pycache__ directory wasn't getting removed.
Turns out we missed some rocm-smi renames when merging changes from it.
Culprit: I695bd085d4a43b678b563b4c35f6d2e8ddfa7d7c
Change-Id: Ieb0db41163af0337f1a3c06eb63a6960e6c52ff6
Signed-off-by: Galantsev, Dmitrii <dmitrii.galantsev@amd.com>
__pycache__ directory wasn't getting removed.
Turns out we missed some rocm-smi renames when merging changes from it.
Culprit: I695bd085d4a43b678b563b4c35f6d2e8ddfa7d7c
Change-Id: Ieb0db41163af0337f1a3c06eb63a6960e6c52ff6
Signed-off-by: Galantsev, Dmitrii <dmitrii.galantsev@amd.com>
[ROCm/amdsmi commit: 7cd72a583d]
SDMA ganging is causing some regressions with some applications hanging.
Temporarily disabling SDMA ganging by default until issue is fixed.
Change-Id: I65e172923a53a967df27b30d969ad5d215c4fa09
SDMA ganging is causing some regressions with some applications hanging.
Temporarily disabling SDMA ganging by default until issue is fixed.
Change-Id: I65e172923a53a967df27b30d969ad5d215c4fa09
[ROCm/ROCR-Runtime commit: a20a0a5bac]
- Added arbitrary env-vars
- Fixed some UBs with atomic variables
- Fixed loading of file plugin
- ATT vs. kernel-trace off by one
Change-Id: I69c75f66f722e4085b5279f41afd05813628846d
Cole Ramos
Ajay
Wenkai Du
Ammar ELWazir
Jonathan Kim
Galantsev, Dmitrii
Hao Zhou
Jaydeep Patel
Juan Manuel MARTINEZ CAAMAÑO
David Yat Sin
Jenkins
Giovanni LB