AI/rocm-systems
2
0
Bifurcation 0
Code Tickets Demandes d'ajout Actions Paquets Projets Publications Wiki Activité
1 547 Révisions 3 Branches 0 Étiquette
595cda2ab935f44749a7bbc8533a643238df7518
Graphe des révisions

1430 Révisions

Auteur SHA1 Message Date
Sylvain Jeaugey b16882a024 2.19.4-1 
Split transport connect phase into multiple steps to avoid port
exhaustion when connecting alltoall at large scale. Defaults to 128
peers per round.
Fix memory leaks on CUDA graph capture.
Fix alltoallv crash on self-sendrecv.
Make topology detection more deterministic when PCI speeds are not
available (fix issue #1020).
Properly close shared memory in NVLS resources.
Revert proxy detach after 5 seconds.
Add option to print progress during transport connect.
Add option to set NCCL_DEBUG to INFO on first WARN.


[ROCm/rccl commit: 88d44d777f]
 2023-11-13 10:36:12 -08:00
Wenkai Du 534af85d0f msccl: enable basic collective trace (#959) 
To avoid increasing number of kernels, colltrace is only enabled with
RCCL_MSCCL_FORCE_FULLOPS=1

[ROCm/rccl commit: 5a800e00cd]
 2023-11-08 20:14:28 -08:00
Bertan Dogancay 1eef637d8c Revert "Remove hip::device (#954)" (#956) 
This reverts commit 2d04f5390d.

[ROCm/rccl commit: 8e0258a73d]
 2023-11-07 13:40:41 -07:00
Wen-Heng (Jack) Chung 270aa41f6b Use send instead of sendWithBarrier. (#727) 
[ROCm/rccl commit: efc42d9045]
 2023-11-07 13:47:24 -06:00
dependabot[bot] cf60052394 Bump rocm-docs-core from 0.26.0 to 0.27.0 in /docs/sphinx (#951) 
Bumps [rocm-docs-core](https://github.com/RadeonOpenCompute/rocm-docs-core) from 0.26.0 to 0.27.0.
- [Release notes](https://github.com/RadeonOpenCompute/rocm-docs-core/releases)
- [Changelog](https://github.com/RadeonOpenCompute/rocm-docs-core/blob/develop/CHANGELOG.md)
- [Commits](https://github.com/RadeonOpenCompute/rocm-docs-core/compare/v0.26.0...v0.27.0)

---
updated-dependencies:
- dependency-name: rocm-docs-core
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

[ROCm/rccl commit: 7291144c94]
 2023-11-07 10:01:16 -07:00
Nusrat Islam 83a36c65c1 Merge pull request #950 from nusislam/msccl-red2 
msccl: remove cases from numReduction switch statement

[ROCm/rccl commit: 022735d208]
 2023-11-04 02:48:03 -05:00
Bertan Dogancay 2d04f5390d Remove hip::device (#954) 
[ROCm/rccl commit: 7edb486154]
 2023-11-03 19:31:00 -06:00
Wenkai Du aa02d2b675 Use parallel init of LDS and adjust P2P channels for gfx94x (#943) 
* Use parallel init of LDS and adjust P2P channels for gfx94x

* Move another init to parallel

* Fix NCCL_NCHANNELS_PER_PEER setting

[ROCm/rccl commit: dbcba2923b]
 2023-11-03 16:06:49 -07:00
Nusrat Islam b6f47bad7c msccl: remove cases from numReduction switch statement 
[ROCm/rccl commit: f545b94d4b]
 2023-11-03 16:56:51 -05:00
gilbertlee-amd d8471eaddf Adding LaunchBench tool (#952) 
[ROCm/rccl commit: d50bab28bf]
 2023-11-03 12:04:52 -06:00
Wenkai Du 1557a1f258 msccl: use 32-bit LDS access and add RCCL_MSCCL_FORCE_FULLOPS (#953) 
[ROCm/rccl commit: bb84345943]
 2023-11-03 10:38:02 -07:00
akolliasAMD 4cd86b185c MSCCL stream fix (#948) 
[ROCm/rccl commit: 988efe605a]
 2023-11-03 09:10:52 -06:00
Wenkai Du 297023a7a6 msccl: add templated kernel (#945) 
* msccl: add templated kernel

* Use defines to improve code readability

* Fix kernel indexing and review feedback

[ROCm/rccl commit: f484ff17b9]
 2023-11-02 17:21:53 -07:00
Nusrat Islam 88c8bb1495 msccl: remove dereference of reduce args 
It can be removed because the msccl kernel will never execute this code
according to the current msccl setup.


[ROCm/rccl commit: 6b80a0d0d4]
 2023-11-02 13:20:00 -05:00
Wenkai Du 3eeaea3f00 msccl: use atomic to set dependency flags (#941) 
[ROCm/rccl commit: a7400218a2]
 2023-10-31 14:46:57 -07:00
akolliasAMD 691df735a3 Revert "Introduce allgather for MSCCL on 8 sockets up to 320KB. (#931)" (#939) 
This reverts commit 769f00db5c.

[ROCm/rccl commit: 9f02ee8dea]
 2023-10-30 23:52:58 -06:00
Wenkai Du b736b506c0 NPkit: misc fixes for MSCCL (#936) 
* msccl: add xcc_id to timestamp sync

* NPKit: add timestamp for rrc operator

* NPKit: add timestamp for MSCCL init

[ROCm/rccl commit: a497722894]
 2023-10-30 10:00:12 -07:00
Nilesh M Negi 8b1254a4f1 Fix gcnArchName bug in topology dump (#937) 
Signed-off-by: nileshnegi <Nilesh.Negi@amd.com>

[ROCm/rccl commit: 1e5ca6820b]
 2023-10-28 12:30:36 -05:00
Ziyue Yang e1dfb82023 Fix MSCCL work FIFO out-of-bound issue (#935) 
[ROCm/rccl commit: 4c117e5335]
 2023-10-27 11:24:52 -07:00
Nilesh M Negi b4ba7cc79d SRC/INIT: fix typo for ENABLE_PROFILING (#934) 
Signed-off-by: nileshnegi <Nilesh.Negi@amd.com>

[ROCm/rccl commit: 96ec3ffe2e]
 2023-10-26 23:52:46 -05:00
Nilesh M Negi 706750597c remove gcnArch support (#920) 
Signed-off-by: nileshnegi <Nilesh.Negi@amd.com>

[ROCm/rccl commit: f22df90e5c]
 2023-10-26 12:09:15 -05:00
Wenkai Du 446c8cbf66 msccl: reduce debug output when using NCCL_DEBUG=INFO (#932) 
[ROCm/rccl commit: fb0eccb57b]
 2023-10-25 08:05:19 -07:00
Wen-Heng (Jack) Chung 769f00db5c Introduce allgather for MSCCL on 8 sockets up to 320KB. (#931) 
[ROCm/rccl commit: bfb8642450]
 2023-10-24 18:41:12 -05:00
Wen-Heng (Jack) Chung 89a8493ef8 Introduce allgather MSCCL XML specification for MI250X up to 320KB. (#930) 
[ROCm/rccl commit: 3f9ffe4788]
 2023-10-24 18:35:55 -05:00
Wen-Heng (Jack) Chung fc2a13c077 Introduce 1-shot allreduce for MI250X Hayabusa. (#929) 
[ROCm/rccl commit: 72d5fbddfd]
 2023-10-24 16:31:18 -05:00
Wenkai Du cc4de02a86 Add missing gfx942 support (#927) 
[ROCm/rccl commit: c4e65fd382]
 2023-10-23 12:04:37 -07:00
akolliasAMD bc7df769a2 AllReduceTests,fixed the number of roots (#925) 
[ROCm/rccl commit: d8dc282eeb]
 2023-10-20 10:25:11 -06:00
dependabot[bot] 187e9c1958 Bump urllib3 from 1.26.17 to 1.26.18 in /docs/sphinx (#921) 
Bumps [urllib3](https://github.com/urllib3/urllib3) from 1.26.17 to 1.26.18.
- [Release notes](https://github.com/urllib3/urllib3/releases)
- [Changelog](https://github.com/urllib3/urllib3/blob/main/CHANGES.rst)
- [Commits](https://github.com/urllib3/urllib3/compare/1.26.17...1.26.18)

---
updated-dependencies:
- dependency-name: urllib3
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

[ROCm/rccl commit: b173c13831]
 2023-10-20 10:15:42 -06:00
searlmc1 212453b2fb Remove quotes causing asan build breakage 
The quotes around "-fsanitize=address -shared-libasan" cause the BUILD_ADDRESS_SANITIZER build to fail; remove the quotes

[ROCm/rccl commit: f59de10524]
 2023-10-19 16:13:39 -07:00
Bertan Dogancay 1a538d0218 Update install.sh --fast and README (#924) 
[ROCm/rccl commit: 3807c203fc]
 2023-10-19 16:35:10 -06:00
Wenkai Du 6f0f614d0b Remove LDS based software barriers from MSCCL (#923) 
[ROCm/rccl commit: dbb5611a3a]
 2023-10-19 16:39:41 -05:00
Wenkai Du edeea499b5 Update rome models (#922) 
[ROCm/rccl commit: 4278a9918b]
 2023-10-18 17:28:01 -07:00
Wen-Heng (Jack) Chung 49e52e7269 Introduce 1pass allreduce. Tailor it for very small message sizes <= 20KB. (#919) 
[ROCm/rccl commit: 341926c60a]
 2023-10-16 16:31:08 -05:00
Wenkai Du e0cc7de446 NPKit: add xcc_id field (#918) 
[ROCm/rccl commit: 39812ce757]
 2023-10-13 15:24:59 -07:00
Wenkai Du c0bd012e6c Fix incorrect arch name parsing (#916) 
[ROCm/rccl commit: 1b80d041cb]
 2023-10-13 10:01:11 -07:00
Wenkai Du 102f0165d6 Port init_once fix from NCCL (#915) 
[ROCm/rccl commit: 6d0b5c1e89]
 2023-10-13 08:01:12 -07:00
dependabot[bot] 376de87fa9 Bump rocm-docs-core from 0.25.0 to 0.26.0 in /docs/sphinx (#917) 
Bumps [rocm-docs-core](https://github.com/RadeonOpenCompute/rocm-docs-core) from 0.25.0 to 0.26.0.
- [Release notes](https://github.com/RadeonOpenCompute/rocm-docs-core/releases)
- [Changelog](https://github.com/RadeonOpenCompute/rocm-docs-core/blob/develop/CHANGELOG.md)
- [Commits](https://github.com/RadeonOpenCompute/rocm-docs-core/compare/v0.25.0...v0.26.0)

---
updated-dependencies:
- dependency-name: rocm-docs-core
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

[ROCm/rccl commit: f7e530259d]
 2023-10-13 08:27:04 -06:00
Wen-Heng (Jack) Chung dfa0d98f9e Change MSCCL kernel signature to allow kernel arguments be preloaded via SGPR (#911) 
* Adding a script that will download/compile/run TransferBench/RCCL/UCX/RCCL-tests/RCCL-Unittests/hip-mpi-testsuite (#895)

Co-authored-by: Pedram Alizadeh <pmohamma@banff-pla-r27-05.pla.dcgpu>

* Only build gfx941

* demo

* fine tune malloc

* Fix merge errors

* Fix merge errors

* Disable parallel build

* Adopt --amdgpu-kernarg-preload-count

* Revert "Adding a script that will download/compile/run TransferBench/RCCL/UCX/RCCL-tests/RCCL-Unittests/hip-mpi-testsuite (#895)"

This reverts commit f5e252dddf02a41b4d1bc512f306f45f97166304.

* Revert CMake changes.

* NPKIT changes.

* Remove some license declarations.

* Address code review feedbacks on msccl_kernel_impl.h

* Update CMakeLists.txt

* Add CMake logic to check the existence of --amdgpu-kernarg-preload-count

* Fix NPKIT trace logic.

---------

Co-authored-by: Pedram Alizadeh <pmohamma@amd.com>
Co-authored-by: Pedram Alizadeh <pmohamma@banff-pla-r27-05.pla.dcgpu>
Co-authored-by: Ziyue Yang <ziyyang@microsoft.com>

[ROCm/rccl commit: 7ee5c1c28b]
 2023-10-12 20:17:08 -05:00
mberenjk 9a0c9ba3e9 adding cuda support for EmptyKernelTest (#913) 
[ROCm/rccl commit: 7e2d905376]
 2023-10-11 14:11:12 -05:00
dependabot[bot] 5096358a70 Bump gitpython from 3.1.35 to 3.1.37 in /docs/sphinx (#912) 
Bumps [gitpython](https://github.com/gitpython-developers/GitPython) from 3.1.35 to 3.1.37.
- [Release notes](https://github.com/gitpython-developers/GitPython/releases)
- [Changelog](https://github.com/gitpython-developers/GitPython/blob/main/CHANGES)
- [Commits](https://github.com/gitpython-developers/GitPython/compare/3.1.35...3.1.37)

---
updated-dependencies:
- dependency-name: gitpython
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

[ROCm/rccl commit: 01d9da8046]
 2023-10-10 15:43:49 -06:00
Sylvain Jeaugey 90d7d1b07b 2.18.6-1 
[ROCm/rccl commit: 0b083e5209]
 2023-10-10 00:34:18 -07:00
gilbertlee-amd c1a7b56b9b Adding a simple EmptyKernelTest to measure launch latency (#910) 
[ROCm/rccl commit: 7dbf47e07b]
 2023-10-04 17:22:48 -06:00
Bertan Dogancay 6f7965796f Revert "Remove 2H4P condition from P2P channels adjustment (#890)" (#904) 
This reverts commit 057e30e705.

[ROCm/rccl commit: a6ff4618c7]
 2023-10-04 09:46:11 -06:00
dependabot[bot] c0a707ea50 Bump rocm-docs-core from 0.24.2 to 0.25.0 in /docs/sphinx (#909) 
Bumps [rocm-docs-core](https://github.com/RadeonOpenCompute/rocm-docs-core) from 0.24.2 to 0.25.0.
- [Release notes](https://github.com/RadeonOpenCompute/rocm-docs-core/releases)
- [Changelog](https://github.com/RadeonOpenCompute/rocm-docs-core/blob/develop/CHANGELOG.md)
- [Commits](https://github.com/RadeonOpenCompute/rocm-docs-core/compare/v0.24.2...v0.25.0)

---
updated-dependencies:
- dependency-name: rocm-docs-core
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

[ROCm/rccl commit: f2600af812]
 2023-10-04 09:14:59 -06:00
dependabot[bot] 928cf93c4b Bump urllib3 from 1.26.15 to 1.26.17 in /docs/sphinx (#906) 
Bumps [urllib3](https://github.com/urllib3/urllib3) from 1.26.15 to 1.26.17.
- [Release notes](https://github.com/urllib3/urllib3/releases)
- [Changelog](https://github.com/urllib3/urllib3/blob/main/CHANGES.rst)
- [Commits](https://github.com/urllib3/urllib3/compare/1.26.15...1.26.17)

---
updated-dependencies:
- dependency-name: urllib3
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

[ROCm/rccl commit: 4b4e7ecdf9]
 2023-10-03 15:54:37 -06:00
akolliasAMD 1ffd3eff31 Dma buf support optin (#905) 
* dmaBufSupport Optin added on every part of the code that should invoke it

[ROCm/rccl commit: 28d7fe5629]
 2023-10-03 03:17:48 -06:00
Edgar Gabriel e6c3e9fd8e turn bfd compilation off by default 
revert the logic to ensure that we are not accidentally creating
a dependency on the bfd libraries when deploying rccl binaries.


[ROCm/rccl commit: 88a55cef83]
 2023-09-29 20:25:33 +00:00
akolliasAMD 12b2fc9774 install.sh fix (#903) 
[ROCm/rccl commit: a773def279]
 2023-09-29 07:42:17 -06:00
Cen Zhao d3c20a1210 Update install.sh to take "--static" option (#894) 
* Update install.sh to take "--static" option

* Fix static build errors

---------

Co-authored-by: BertanDogancay <bertan.dogancay@gmail.com>

[ROCm/rccl commit: fb57a438d7]
 2023-09-27 12:45:21 -04:00
Bertan Dogancay b35ea4bd78 Modify All-To-All doc (#896) 
* Modify All-To-All doc

* Update nccl.h.in

* update unit-tests

---------

Co-authored-by: gilbertlee-amd <44450918+gilbertlee-amd@users.noreply.github.com>

[ROCm/rccl commit: c1f57a7041]
 2023-09-27 12:45:21 -04:00