aa8a5f3b2a
device: fine tune MI200/MI250 simple protocol performance
...
With Simple protocol, unroll factor of 4 offers better
performance for most of the collectives (on MI200. MI250, and
MI300) except large message allreduce with Ring algorithm
on MI250 and MI200). This PR changes the default unroll factor
to 4 while adding fine tuning for reduction operations.
[ROCm/rccl commit: 58e53dfd37
2023-07-08 20:21:18 -05:00
42cb0a7b10
Bump rocm-docs-core from 0.18.2 to 0.18.4 in /docs/sphinx ( #796 )
...
Bumps [rocm-docs-core](https://github.com/RadeonOpenCompute/rocm-docs-core ) from 0.18.2 to 0.18.4.
- [Release notes](https://github.com/RadeonOpenCompute/rocm-docs-core/releases )
- [Changelog](https://github.com/RadeonOpenCompute/rocm-docs-core/blob/develop/CHANGELOG.md )
- [Commits](https://github.com/RadeonOpenCompute/rocm-docs-core/compare/v0.18.2...v0.18.4 )
---
updated-dependencies:
- dependency-name: rocm-docs-core
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com >
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
[ROCm/rccl commit: 6ef70811d2
2023-07-06 08:41:28 -06:00
c749c08c18
Merge pull request #795 from wenkaidu/nccl_sync
...
Sync latest update from NCCL
[ROCm/rccl commit: 397716c04d
2023-07-05 09:51:48 -07:00
97d1856104
Merge remote-tracking branch 'nccl/master' into nccl_sync
...
[ROCm/rccl commit: e0c70af46b
2023-07-05 07:53:53 -07:00
9b2475ef06
Merge pull request #782 from ROCmSoftwarePlatform/2.18.3
...
Sync up with NCCL 2.18.3
[ROCm/rccl commit: ce6a2ffac8
2023-06-29 15:04:16 -07:00
e1ac484d4e
added npkit support into the all_gather run ring algorithm ( #790 )
...
[ROCm/rccl commit: 9bba4a2f2a
2023-06-29 13:59:54 -06:00
3d153f7583
Bump rocm-docs-core from 0.18.1 to 0.18.2 in /docs/sphinx ( #791 )
...
Bumps [rocm-docs-core](https://github.com/RadeonOpenCompute/rocm-docs-core ) from 0.18.1 to 0.18.2.
- [Release notes](https://github.com/RadeonOpenCompute/rocm-docs-core/releases )
- [Changelog](https://github.com/RadeonOpenCompute/rocm-docs-core/blob/develop/CHANGELOG.md )
- [Commits](https://github.com/RadeonOpenCompute/rocm-docs-core/compare/v0.18.1...v0.18.2 )
---
updated-dependencies:
- dependency-name: rocm-docs-core
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com >
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
[ROCm/rccl commit: 2e00fa4f09
2023-06-29 08:31:25 -06:00
59e62c807b
Re-enabled graph tests ( #736 )
...
* enabled graph tests
* joined multi and single process CI testing
[ROCm/rccl commit: cf8cfa88a8
2023-06-29 08:08:17 -06:00
894b2d75e7
Report unit test environment variable values as part of output ( #789 )
...
[ROCm/rccl commit: f7c553edad
2023-06-29 07:13:05 -06:00
b25a188806
Bump rocm-docs-core from 0.17.2 to 0.18.1 in /docs/sphinx ( #788 )
...
Bumps [rocm-docs-core](https://github.com/RadeonOpenCompute/rocm-docs-core ) from 0.17.2 to 0.18.1.
- [Release notes](https://github.com/RadeonOpenCompute/rocm-docs-core/releases )
- [Changelog](https://github.com/RadeonOpenCompute/rocm-docs-core/blob/develop/CHANGELOG.md )
- [Commits](https://github.com/RadeonOpenCompute/rocm-docs-core/compare/v0.17.2...v0.18.1 )
---
updated-dependencies:
- dependency-name: rocm-docs-core
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com >
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
[ROCm/rccl commit: 8de3714152
2023-06-28 08:37:12 -06:00
b1b47d2e18
Prevent WR index truncation in the InfiniBand transport plugin
...
[ROCm/rccl commit: 6e24ef4e1f
2023-06-28 11:39:19 +02:00
a2305f11e1
Bump rocm-docs-core from 0.16.0 to 0.17.2 in /docs/sphinx ( #787 )
...
* Bump rocm-docs-core from 0.16.0 to 0.17.2 in /docs/sphinx
Bumps [rocm-docs-core](https://github.com/RadeonOpenCompute/rocm-docs-core ) from 0.16.0 to 0.17.2.
- [Release notes](https://github.com/RadeonOpenCompute/rocm-docs-core/releases )
- [Changelog](https://github.com/RadeonOpenCompute/rocm-docs-core/blob/develop/CHANGELOG.md )
- [Commits](https://github.com/RadeonOpenCompute/rocm-docs-core/compare/v0.16.0...v0.17.2 )
---
updated-dependencies:
- dependency-name: rocm-docs-core
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com >
* set external_projects_current_project in conf.py
---------
Signed-off-by: dependabot[bot] <support@github.com >
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Sam Wu <sam.wu2@amd.com >
[ROCm/rccl commit: e2097e088e
2023-06-27 09:24:44 -06:00
f98715baea
Merge remote-tracking branch 'nccl/master' into develop
...
[ROCm/rccl commit: abd0615351
2023-06-26 22:51:56 +00:00
1607567e42
ASAN build excluding additional files, Algodir support for share folder
...
* ASAN build excluding additional files, Algodir support for share folder (#786 )
* Algodir support for share folder
[ROCm/rccl commit: bd14ac8b59
2023-06-23 10:57:20 -04:00
2bda28cf7e
Limiting # parallel jobs in install script to 16 by default, and new -j/--jobs flag ( #785 )
...
[ROCm/rccl commit: bb55848450
2023-06-22 14:30:44 -06:00
d49cbb939d
Bump rocm-docs-core from 0.15.0 to 0.16.0 in /docs/sphinx ( #784 )
...
Bumps [rocm-docs-core](https://github.com/RadeonOpenCompute/rocm-docs-core ) from 0.15.0 to 0.16.0.
- [Release notes](https://github.com/RadeonOpenCompute/rocm-docs-core/releases )
- [Changelog](https://github.com/RadeonOpenCompute/rocm-docs-core/blob/develop/CHANGELOG.md )
- [Commits](https://github.com/RadeonOpenCompute/rocm-docs-core/compare/v0.15.0...v0.16.0 )
---
updated-dependencies:
- dependency-name: rocm-docs-core
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com >
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
[ROCm/rccl commit: 4ebf17b15e
2023-06-22 08:49:19 -06:00
d411d52b19
Disable Colltrace for --fast option ( #778 )
...
* Disable Colltrace for --fast option
* Limit nprocs for CI
[ROCm/rccl commit: 0c77c66221
2023-06-21 14:16:09 -06:00
547c300b48
Bump rocm-docs-core from 0.14.0 to 0.15.0 in /docs/sphinx ( #781 )
...
Bumps [rocm-docs-core](https://github.com/RadeonOpenCompute/rocm-docs-core ) from 0.14.0 to 0.15.0.
- [Release notes](https://github.com/RadeonOpenCompute/rocm-docs-core/releases )
- [Changelog](https://github.com/RadeonOpenCompute/rocm-docs-core/blob/develop/CHANGELOG.md )
- [Commits](https://github.com/RadeonOpenCompute/rocm-docs-core/compare/v0.14.0...v0.15.0 )
---
updated-dependencies:
- dependency-name: rocm-docs-core
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com >
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
[ROCm/rccl commit: 399d31ed40
2023-06-21 12:39:02 -06:00
225e871de2
fixed npkit size to never be a negative number ( #779 )
...
[ROCm/rccl commit: 9bdf6797a5
2023-06-21 08:26:40 -06:00
f6785b57c8
Switching to using atomicAdd_system within kernel for collective trace ( #780 )
...
[ROCm/rccl commit: 52a28ff2fc
2023-06-20 17:49:52 -06:00
ec6cc9c295
Bump rocm-docs-core from 0.13.4 to 0.14.0 in /docs/sphinx ( #777 )
...
Bumps [rocm-docs-core](https://github.com/RadeonOpenCompute/rocm-docs-core ) from 0.13.4 to 0.14.0.
- [Release notes](https://github.com/RadeonOpenCompute/rocm-docs-core/releases )
- [Changelog](https://github.com/RadeonOpenCompute/rocm-docs-core/blob/develop/CHANGELOG.md )
- [Commits](https://github.com/RadeonOpenCompute/rocm-docs-core/compare/v0.13.4...v0.14.0 )
---
updated-dependencies:
- dependency-name: rocm-docs-core
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com >
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
[ROCm/rccl commit: 8802de7761
2023-06-16 08:48:41 -06:00
834634194e
Merge pull request #775 from nusislam/unroll-1
...
device: use unroll factor based on platforms
[ROCm/rccl commit: fd6553bcf8
2023-06-14 17:42:38 -05:00
7ea099612f
device: use unroll factor based on platforms
...
[ROCm/rccl commit: 3a741787bf
2023-06-14 13:36:15 -05:00
1ae071944e
improve compilation time and create timetrace plot ( #773 )
...
* improve compilation time and create time-trace plot
* set default value for nproc
[ROCm/rccl commit: f35777e9b0
2023-06-14 09:17:51 -06:00
2dc2c86ec1
2.18.3-1
...
Fix data corruption with Tree/LL128 on systems with 1GPU:1NIC.
Fix hang with Collnet on bfloat16 on systems with less than one NIC
per GPU.
Fix long initialization time.
Fix data corruption with Collnet when mixing multi-process and
multi-GPU per process.
Fix crash when shared memory creation fails.
Fix Avg operation with Collnet/Chain.
Fix performance of alltoall at scale with more than one NIC per GPU.
Fix performance for DGX H800.
Fix race condition in connection progress causing a crash.
Fix network flush with Collnet.
Fix performance of aggregated allGather/reduceScatter operations.
Fix PXN operation when CUDA_VISIBLE_DEVICES is set.
Fix NVTX3 compilation issues on Debian 10.
[ROCm/rccl commit: ea38312273
2023-06-14 01:29:17 -07:00
3041d86342
Enable --fast ( #774 )
...
[ROCm/rccl commit: b89c5e0632
2023-06-13 13:40:40 -06:00
8438fd9e42
Wall clock update and npkit trace script Update ( #771 )
...
* changed builtin clock to wall_clock64
* updated npkit_Trace_generator to the new version of npkit
[ROCm/rccl commit: 9cdac774ea
2023-06-07 17:47:10 -06:00
5168be1867
Update Read the Docs, documentation, and dependabot ( #772 )
...
* update documentation
add version number to documentation
rename .sphinx/.doxygen to sphinx/doxygen
enable htmlzip, pdf, epub formats when publishing on Read the Docs
* add noCI label for dependabot PRs
since RTD CI is separate from math lib CI
* update rocm-docs-core to v0.13.4
* update README with link to rocm.docs.amd.com
[ROCm/rccl commit: c3f47853bd
2023-06-07 15:31:58 -06:00
78817b499f
Bump cryptography from 40.0.2 to 41.0.0 in /docs/.sphinx ( #766 )
...
Bumps [cryptography](https://github.com/pyca/cryptography ) from 40.0.2 to 41.0.0.
- [Changelog](https://github.com/pyca/cryptography/blob/main/CHANGELOG.rst )
- [Commits](https://github.com/pyca/cryptography/compare/40.0.2...41.0.0 )
---
updated-dependencies:
- dependency-name: cryptography
dependency-type: indirect
...
Signed-off-by: dependabot[bot] <support@github.com >
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
[ROCm/rccl commit: 5f1f0142ac
2023-06-07 10:27:56 -06:00
9a7fc6002f
Bump requests from 2.28.2 to 2.31.0 in /docs/.sphinx ( #747 )
...
Bumps [requests](https://github.com/psf/requests ) from 2.28.2 to 2.31.0.
- [Release notes](https://github.com/psf/requests/releases )
- [Changelog](https://github.com/psf/requests/blob/main/HISTORY.md )
- [Commits](https://github.com/psf/requests/compare/v2.28.2...v2.31.0 )
---
updated-dependencies:
- dependency-name: requests
dependency-type: indirect
...
Signed-off-by: dependabot[bot] <support@github.com >
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
[ROCm/rccl commit: 5e8654a0b0
2023-06-07 10:27:36 -06:00
05b8523a7b
Updating NOTICES.txt and LICENSE.txt ( #770 )
...
[ROCm/rccl commit: 20b567caac
2023-06-07 09:45:03 -06:00
8df2b752cd
Fix build on additional architectures ( #740 )
...
* Fix build on additional architectures
Instead of directly wrapping a platform-specific operation with a
preprocessor check against a gfx macro, it can be more flexible to
check a macro that can be overriden by the user. The gfx macro can then
just provide the default value for the macro, resulting in the same
default behaviour as if the gfx macro was checked directly but with
more control at build-time.
For example, to build rccl without using buffer_wbinvl1_vol on
gfx902, but still use the default on other archs, a user could
export CXXFLAGS='-Xarch_gfx902 -DRCCL_USE_WBINVL1_VOL=1' before
configuring the build. This flexibility isn't always necessary, but
it's nicer to have it and not need it than to need it and not have it.
* Define WARP_SIZE using warpSize builtin
[ROCm/rccl commit: b1a65afd58
2023-06-06 16:45:50 -06:00
184a6365a2
Bump rocm-docs-core from 0.11.0 to 0.13.3 in /docs/.sphinx ( #765 )
...
Bumps [rocm-docs-core](https://github.com/RadeonOpenCompute/rocm-docs-core ) from 0.11.0 to 0.13.3.
- [Release notes](https://github.com/RadeonOpenCompute/rocm-docs-core/releases )
- [Changelog](https://github.com/RadeonOpenCompute/rocm-docs-core/blob/develop/CHANGELOG.md )
- [Commits](https://github.com/RadeonOpenCompute/rocm-docs-core/compare/v0.11.0...v0.13.3 )
---
updated-dependencies:
- dependency-name: rocm-docs-core
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com >
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
[ROCm/rccl commit: a029d34628
2023-06-06 16:45:15 -06:00
a34679b2e6
resolving the pthread-gtest linking issue for rccl-UnitTests ( #768 )
...
[ROCm/rccl commit: 520f15e61b
2023-06-06 14:21:40 -04:00
90cbef7042
Add NCCL_NCHANNELS_PER_PEER override ( #767 )
...
Also fix topol_expl build issue
[ROCm/rccl commit: 3af90902c8
2023-06-06 08:41:38 -07:00
a32eeae7b5
add DMA_BUF support ( #763 )
...
* add DMA_BUF support
* remove unused libraries in src/init.cc
* change NCCL_ALL to NCCL_INIT
* remove extra pointer functions in transport/net.cc
[ROCm/rccl commit: d52b6c0d24
2023-06-01 12:46:42 -06:00
05dac01df5
Removing init_nvtx.cc from source list ( #762 )
...
[ROCm/rccl commit: c62aebe882
2023-05-31 14:44:55 -06:00
61e30182b2
Rework barrier and event code ( #761 )
...
* Rework barrier and event code
* Switch to inline asm
[ROCm/rccl commit: 5a38ff192b
2023-05-31 13:36:51 -07:00
f4d63d21ca
Merge pull request #757 from nusislam/unroll
...
device: change unroll factor
[ROCm/rccl commit: 7519ecb476
2023-05-30 14:19:32 -05:00
fd64cb242d
added time results on npkit generator ( #749 )
...
[ROCm/rccl commit: 2b1efa9e9a
2023-05-30 12:57:25 -06:00
d2c1295f79
Refactoring CMakeFiles ( #755 )
...
[ROCm/rccl commit: 777d8747a5
2023-05-25 16:08:54 -06:00
325ff1dc11
device: change unroll factor
...
The default value of unroll factor is 2. Changing the unroll
factor to 4 provides better performance for most of the collectives.
[ROCm/rccl commit: 4d1cfb17c8
2023-05-25 15:42:35 -05:00
f371afe6fd
updated install script to enable all of npkit ( #754 )
...
[ROCm/rccl commit: 58db1cb96d
2023-05-24 14:44:01 -06:00
a7557cf7b0
revert npkit ( #748 )
...
[ROCm/rccl commit: 7d6e7bcd7d
2023-05-24 07:41:05 -07:00
4430e4448f
Limit MSCCL reduce unrolling to pow-2 cases to shrink kernel size ( #746 )
...
[ROCm/rccl commit: ed252c30f4
2023-05-19 11:46:36 -07:00
73c6d51454
fix min, max and avg ( #745 )
...
[ROCm/rccl commit: 11676267b5
2023-05-18 11:02:59 -07:00
2707b6c6c1
Update documentation requirements ( #743 )
...
Co-authored-by: samjwu <samjwu@users.noreply.github.com >
[ROCm/rccl commit: edb2ee1a44
2023-05-18 10:52:35 -06:00
1bafe9c240
Merge pull request #742 from whchung/skip_done_event_msccl
...
Allow skipping doneEvent inside MSCCL.
[ROCm/rccl commit: eba4e9e100
2023-05-18 10:17:20 -05:00
f119977daa
Fix merge error ( #744 )
...
[ROCm/rccl commit: 403cda6322
2023-05-18 08:09:27 -07:00
932822fc46
Address review feedbacks and make the flag be disabled by default.
...
[ROCm/rccl commit: ca4a1dfd67
2023-05-17 17:50:25 +00:00
Nusrat Islam
dependabot[bot]
Wenkai Du
Wenkai Du
akolliasAMD
gilbertlee-amd
Dmitrii Gabor
arvindcheru
Bertan Dogancay
Sylvain Jeaugey
Sam Wu
Cory Bloor
Pedram Alizadeh
Ziyue Yang
Wen-Heng (Jack) Chung
Wen-Heng (Jack) Chung