Wenkai Du
366cd12bed
topo-expl: fix broken build ( #1048 )
...
[ROCm/rccl commit: 600b44fee5 ]
2024-01-17 08:59:03 -08:00
Wenkai Du
cd7a346297
Doubling buffer size to fix NCCL INFO corruption with increased channels ( #1035 )
...
[ROCm/rccl commit: f7e39fced2 ]
2024-01-08 08:14:33 -08:00
Wenkai Du
dcf623f2ec
Add special handling of gfx940 ( #976 )
...
* Add special handling of gfx940
* Update ring base
[ROCm/rccl commit: 50b2dd9fd7 ]
2023-11-22 15:07:36 -08:00
akolliasAMD
8685535346
Fixed topo_expl ( #891 )
...
[ROCm/rccl commit: 762a42859e ]
2023-09-13 12:05:35 -06:00
Audrey MP
2e3d45a53a
Gcn arch name ( #886 )
...
We use CMake to determine if we're compiling against a version of ROCm that supports gcnArchName and handles architecture checking appropriately. It includes a few helper functions as drop ins for the functionality we used gcnArch for before; sometimes to enable flags, and sometimes to set frequencies.
[ROCm/rccl commit: e58ec78d35 ]
2023-09-12 15:34:40 -04:00
akolliasAMD
56129830a6
NCCL_TREES variable and rome model fixes ( #856 )
...
[ROCm/rccl commit: d33cd5a233 ]
2023-08-21 10:35:37 -06:00
Wenkai Du
0c31452135
Add new model support ( #847 )
...
* Add new model support
* Update new rings
[ROCm/rccl commit: 7044599575 ]
2023-08-10 17:14:51 -07:00
Wenkai Du
dfda1d6fab
Enable gfx94x ( #808 ) ( #816 )
...
(cherry picked from commit 94da229a7788d74685d1591a4e75a8341de64f41)
[ROCm/rccl commit: a7fcd58a97 ]
2023-07-21 07:31:27 -07:00
Wenkai Du
f98715baea
Merge remote-tracking branch 'nccl/master' into develop
...
[ROCm/rccl commit: abd0615351 ]
2023-06-26 22:51:56 +00:00
Wenkai Du
90cbef7042
Add NCCL_NCHANNELS_PER_PEER override ( #767 )
...
Also fix topol_expl build issue
[ROCm/rccl commit: 3af90902c8 ]
2023-06-06 08:41:38 -07:00
Ziyue Yang
f7f669e7f0
MSCCL: Improve executor and integrate scheduler ( #694 )
...
* MSCCL: improve executor and add scheduler for testing
* Use external scheduler
* Fix cmake error
* Address comments
* Fix thread safe issue
* Make MSCCL lifecycle APIs thread safe
* Make MSCCL internal scheduler aware of topology hint
* Revise error message
[ROCm/rccl commit: e3b2342f39 ]
2023-03-14 14:34:25 -07:00
Wenkai Du
c76bc214c8
Merge remote-tracking branch 'nccl/master' into HEAD
...
[ROCm/rccl commit: e1cb45ff22 ]
2023-02-04 01:44:43 +00:00
Wenkai Du
7be2c55b32
topo_expl: fix broken build by adding hipify steps ( #670 )
...
[ROCm/rccl commit: a0dd8e0b84 ]
2023-01-06 07:29:40 -08:00
Wenkai Du
ffecb74b1e
Update tuning table and fix topo_expl
...
[ROCm/rccl commit: 94ad7f6f51 ]
2022-11-07 18:24:24 +00:00
Wenkai Du
36e5e02e46
Merge remote-tracking branch 'nccl/master' into develop
...
[ROCm/rccl commit: 4f0e223db4 ]
2022-10-20 15:41:29 +00:00
Wenkai Du
7fe0b0161f
topo_expl: fix compilation error ( #639 )
...
[ROCm/rccl commit: fc554a2428 ]
2022-10-19 14:19:50 -07:00
gilbertlee-amd
0ca30fb88a
Updating files for missing licenses ( #637 )
...
[ROCm/rccl commit: ebb8b5bf63 ]
2022-10-14 13:49:16 -06:00
Wenkai Du
7874a99c75
Merge remote-tracking branch 'nccl/master' into develop
...
[ROCm/rccl commit: a79d9e3586 ]
2022-09-09 16:05:38 +00:00
arvindcheru
a44be6655d
HIP Path default updated to ROCM_PATH (reorg path) ( #592 )
...
Updated default path for hip to ROCM_PATH (/opt/rocm instead of /opt/rocm/hip) as per new/current structure.
[ROCm/rccl commit: 2cb2f9493a ]
2022-08-04 13:38:41 -04:00
Edgar
8953f5b5ca
Introduce multi-rank support per device.
...
This is a single commit of the source code changes required to
introduce support for multiple ranks per device.
A new interface (ncclCommRankInitMulti) has to be used to make use of
this new feature.
[ROCm/rccl commit: 0336ffdf70 ]
2022-06-10 14:23:12 +00:00
Wenkai Du
5becf1669f
Add another Rome model ( #553 )
...
* Add another Rome model
* Add option to force enable intranet on single node
* Limit p2p channels to number of ranks
* Refine p2p channels handling
[ROCm/rccl commit: ef499c4810 ]
2022-05-31 11:31:30 -07:00
Wenkai Du
2c125ce6ed
Update Rome model ( #552 )
...
[ROCm/rccl commit: c5b77121f0 ]
2022-05-26 09:59:23 -07:00
akolliasAMD
22dc8bd246
Added creation of new tree and added switch for using treesplit for specific cases ( #551 )
...
[ROCm/rccl commit: 98f0809a39 ]
2022-05-25 18:55:14 -04:00
Wenkai Du
b30b8becea
Refine and add new Rome models ( #548 )
...
[ROCm/rccl commit: 283dc86a73 ]
2022-05-17 08:23:59 -07:00
Wenkai Du
95b30d9762
topo_expl: fix build and add tuning support ( #539 )
...
[ROCm/rccl commit: 063da25563 ]
2022-04-26 15:40:07 -07:00
Wenkai Du
67e7e6507e
Merge remote-tracking branch 'nccl/master' into develop
...
[ROCm/rccl commit: d28e1cb44f ]
2022-04-18 11:15:25 -07:00
Wenkai Du
011447e4dc
Add new Rome model ( #536 )
...
[ROCm/rccl commit: 2151c79d14 ]
2022-04-13 11:45:40 -07:00
Wenkai Du
f8023f2e07
Add new Rome model ( #535 )
...
[ROCm/rccl commit: ba4c165bf3 ]
2022-04-12 13:27:32 -07:00
Wenkai Du
3332cdff07
Support multiple tuning tables ( #522 )
...
* Support multiple tuning tables
* [UnitTests] Skip managed memory testing
[ROCm/rccl commit: bbe780ca6c ]
2022-03-31 17:09:21 -07:00
Wenkai Du
db1e628ba3
Update Rome model matching and add new models ( #516 )
...
* Update Rome model matching and add new models
* Add missing file
* Models update
[ROCm/rccl commit: cd17cf6dce ]
2022-03-21 10:54:40 -07:00
Ziyue Yang
dfa9b9e958
Add Pivot AllToAll algorithm for Rome model ( #503 )
...
* add a2a pivot interface
* remove debug info
* address comments
* fix bug
* remove custom script
* address comments
* fix bug
[ROCm/rccl commit: b569c0a1db ]
2022-02-20 21:09:47 -08:00
Wenkai Du
4d43d9ce22
Update Rome models ( #491 )
...
[ROCm/rccl commit: 598c6fdded ]
2022-01-14 10:03:30 -08:00
Wenkai Du
02a94fc552
topo_expl: update for 2.11.4 ( #490 )
...
* topo_expl: update for 2.11.4
* topo_expl: revert a few logging changes
[ROCm/rccl commit: 369c021992 ]
2022-01-13 13:33:07 -08:00
Wenkai Du
93c7526fdc
Add another Rome model ( #483 )
...
[ROCm/rccl commit: f8d0775a6f ]
2022-01-05 09:26:31 -08:00
Wenkai Du
fd98ee84b4
Update Rome model matching ( #461 )
...
* Update Rome model matching
* Add another Rome model
* Automatically setup NET GDR level from model
[ROCm/rccl commit: 0331e39f81 ]
2021-11-05 08:53:47 -07:00
Wenkai Du
15143b1cfb
Query XGMI link count through rocm_smi_lib API ( #442 )
...
[ROCm/rccl commit: 14a184eb67 ]
2021-10-26 10:30:20 -07:00
Wenkai Du
b587b55c2e
Add more Rome models ( #434 )
...
* Add more Rome models
* Update models and tuning
* Update tuning
[ROCm/rccl commit: 2249a1d9d3 ]
2021-10-12 08:23:20 -07:00
Wenkai Du
d377c4dcc6
Add another Rome model ( #431 )
...
[ROCm/rccl commit: e0053311c0 ]
2021-10-06 08:17:12 -07:00
Wenkai Du
91eca0d7d2
Trim NICs when all GPUs are connected by XGMI ( #430 )
...
* Trim NICs when all GPUs are connected by XGMI
* Only enable clique with maximum of 2 hops
[ROCm/rccl commit: 29c729d8b6 ]
2021-10-05 18:27:43 -07:00
Wenkai Du
4fd7a14087
Merge remote-tracking branch 'origin/develop' into 2.10.3
...
[ROCm/rccl commit: d5f93649ff ]
2021-08-24 09:49:47 -07:00
Wenkai Du
b9508a6aba
Implement NIC identification and remapping ( #420 )
...
* Add 1H16P GPU model
* Implement NIC identification and remapping
* Revert "Sort IB devices based on device name (#413 )"
This reverts commit de0c586bad .
* Fix permute and check order
* Correction on IB speed reporting
* Revert "Allow user to link layer with RCCL_IB_HCA_SKIP_LINK_LAYER (#361 )"
This reverts commit fa690c47a0 .
[ROCm/rccl commit: 5c8380ff5b ]
2021-08-24 09:42:04 -07:00
Wenkai Du
57518da006
Add gfx908 VM model ( #418 )
...
[ROCm/rccl commit: 5f15ed6e3e ]
2021-08-10 08:55:11 -07:00
Wenkai Du
4b89e98675
Merge remote-tracking branch 'nccl/master' into 2.10.3
...
[ROCm/rccl commit: bf2339f93e ]
2021-07-30 16:23:14 -07:00
Wenkai Du
faea6ead5c
Query XGMI links from xml and adjust gfx906 channel usage ( #410 )
...
[ROCm/rccl commit: 818cdb16a8 ]
2021-07-27 17:32:41 -07:00
Wenkai Du
8fbeb14175
topo_expl: fix build after switching to rocm-smi-lib ( #405 )
...
* topo_expl: fix build after switching to rocm-smi-lib
* Use minimal of 4 channels for gfx908
[ROCm/rccl commit: 135d47d125 ]
2021-07-27 08:30:08 -07:00
Wenkai Du
90ae176437
Fixes for NCCL_MAX_NCHANNELS and topo_expl ( #398 )
...
[ROCm/rccl commit: fa6d7e9a63 ]
2021-06-22 08:41:49 -07:00
Wenkai Du
5bebcb0015
Setup collectives threshold for enabling intranet ( #387 )
...
* Setup collectives threshold for enabling intranet
* Use separate operation counters for coll and p2p
[ROCm/rccl commit: b815a2800f ]
2021-06-09 13:24:26 -07:00
Wenkai Du
c8a432dc25
Allow intranode use of network connection ( #383 )
...
* Allow intranode use of network connection
* Checking for graph for null pointer
[ROCm/rccl commit: a3a8c2d56b ]
2021-06-08 07:37:59 -07:00
Wenkai Du
4b31e521e9
Add option to enable multiple SAT in SHARP ( #380 )
...
* Add option to enable multiple SAT in SHARP
* Extend number of NICs to 16
[ROCm/rccl commit: 961922ea02 ]
2021-06-03 19:45:18 -07:00
Wenkai Du
cdf2780687
topo_expl: update to 2.9.9
...
[ROCm/rccl commit: 13dc80ee14 ]
2021-05-26 09:24:34 -07:00