AI/rocm-systems
2
0 フォーク 0
コード イシュー プルリクエスト Actions パッケージ プロジェクト リリース Wiki アクティビティ
763 コミット 3 ブランチ 0 タグ
3493750b6bd8fc7ffe169ee2d5ec9891bb7de868
コミットグラフ
346 コミット
作成者 SHA1 メッセージ 日付
Wenkai Du 133aed2dfb Force ring algorithm on single node (#509) 
[ROCm/rccl commit: d6d6af710e]
 2022-03-04 10:29:02 -08:00
gilbertlee-amd a182076a0e Unit test refactor (#500) 
Refactoring and consolidating single-process / multi-process unit testing

[ROCm/rccl commit: 29ad0f5fbe]
 2022-02-25 08:59:07 -07:00
Ziyue Yang dfa9b9e958 Add Pivot AllToAll algorithm for Rome model (#503) 
* add a2a pivot interface

* remove debug info

* address comments

* fix bug

* remove custom script

* address comments

* fix bug

[ROCm/rccl commit: b569c0a1db]
 2022-02-20 21:09:47 -08:00
Wenkai Du 0f0388ba0b Allow additional options to be passed in through model's definition (#501) 
[ROCm/rccl commit: 94e0dc8bfd]
 2022-02-17 08:28:58 -08:00
Wenkai Du 5b697e40db Add another Rome model (#497) 
[ROCm/rccl commit: 02096c9936]
 2022-02-12 10:30:16 -08:00
Wenkai Du 635c0bcc01 Generate proper b-tree with non-repeating channels (#493) 
[ROCm/rccl commit: 400df49dbe]
 2022-01-19 15:09:17 -08:00
Wenkai Du 663c1db898 Merge remote-tracking branch 'nccl/master' into develop 
[ROCm/rccl commit: 973d0111db]
 2022-01-17 13:34:36 -08:00
Wenkai Du 4d43d9ce22 Update Rome models (#491) 
[ROCm/rccl commit: 598c6fdded]
 2022-01-14 10:03:30 -08:00
Wenkai Du 93ab252d52 Use noinline and a few other fixes (#489) 
* Use noinline and a few other fixes

* Tune collectives

[ROCm/rccl commit: 6ff7690cb5]
 2022-01-11 16:51:06 -08:00
Wenkai Du 123de1621f Use hipGraph instead of cudaGraph (#487) 
[ROCm/rccl commit: 3669e12432]
 2022-01-10 08:26:01 -08:00
Wenkai Du 3f16835565 Tune collectives for 2.11.4 (#486) 
[ROCm/rccl commit: 565fbeb5e9]
 2022-01-10 08:25:47 -08:00
Wenkai Du e5ff692673 Merge pull request #482 from ROCmSoftwarePlatform/2.11.4 
Sync up with 2.11.4

[ROCm/rccl commit: 4234a638b5]
 2022-01-05 09:31:51 -08:00
Wenkai Du 93c7526fdc Add another Rome model (#483) 
[ROCm/rccl commit: f8d0775a6f]
 2022-01-05 09:26:31 -08:00
Wenkai Du 618fbfc644 Merge remote-tracking branch 'origin/develop' into 2.11.4 
[ROCm/rccl commit: 434ecb0e1f]
 2022-01-03 09:54:16 -08:00
Ke Wen 07d9648aec Add env NCCL_NET_DISABLE_INTRA 
Disable NET transport for intra-node communication by setting the env to 1
It provides an option to error out instead of falling back to NET when superior intra-node transports (P2P and SHM) are unavailable


[ROCm/rccl commit: c88c9f873f]
 2021-12-08 16:28:19 +01:00
Wenkai Du 7406aed9c9 Update Rome model (#479) 
[ROCm/rccl commit: a94b953bcc]
 2021-12-03 08:24:51 -08:00
Wenkai Du dd4a53db37 Update tuning parameters (#478) 
[ROCm/rccl commit: 8a08a2f579]
 2021-11-30 08:51:11 -08:00
gilbertlee-amd 2da8358a74 Minor cppcheck fixes, adding suppression file (#475) 
* Minor cppcheck fixes, adding suppression file

[ROCm/rccl commit: 539de1216f]
 2021-11-24 10:23:59 -07:00
Wenkai Du 864c6a645e Determine fine grained memory availability at RCCL bootstrapping (#471) 
[ROCm/rccl commit: e9bf01fb7e]
 2021-11-19 08:12:53 -08:00
Chris Jones a52ca36a9e Perform busIdToInt64 on the stack. 
I noticed when I enabled `NCCL_DEBUG_SUBSYS=ALLOC` that this function is
called thousands of times, making the log output unintelligible.
Fortunately, this function can be implemented without heap allocations.


[ROCm/rccl commit: 8cf7325d69]
 2021-11-19 09:35:55 +01:00
Wenkai Du f8bd2d0cfa Merge remote-tracking branch 'nccl/master' into develop 
[ROCm/rccl commit: 3a919c1f49]
 2021-11-11 14:22:12 -08:00
Wenkai Du fc5f692f71 Remove extra work element copy (#465) 
[ROCm/rccl commit: e05de8fd26]
 2021-11-09 13:52:03 -08:00
Wenkai Du 680660fcf7 Do not use async stream for memory allocation and transport setup without graph (#460) 
[ROCm/rccl commit: 33bdd557c8]
 2021-11-05 13:39:14 -07:00
Wenkai Du fd98ee84b4 Update Rome model matching (#461) 
* Update Rome model matching

* Add another Rome model

* Automatically setup NET GDR level from model

[ROCm/rccl commit: 0331e39f81]
 2021-11-05 08:53:47 -07:00
Wenkai Du befaa35e12 profiling: fix incorrect print out in timing profile (#457) 
[ROCm/rccl commit: 26fc6b0919]
 2021-11-03 16:22:21 -07:00
Wenkai Du df59f64e3f Support different protocols and algorithms in all reduce only build (#455) 
* Support different protocols and algorithms in all reduce only build

* Restore deleted line in error

[ROCm/rccl commit: 29170a8b5f]
 2021-11-02 08:39:08 -07:00
Wenkai Du a11b55a37f Check rocm_smi64Config.h on older ROCm build (#452) 
[ROCm/rccl commit: 4643a17f83]
 2021-10-28 07:26:28 -07:00
Wenkai Du b0b2e27df3 Rework kernel launch code (#449) 
[ROCm/rccl commit: d221fb672a]
 2021-10-28 07:26:11 -07:00
Wenkai Du 747216e2b2 Enable timing profiling mode (#447) 
[ROCm/rccl commit: ec36c4c326]
 2021-10-27 08:21:48 -07:00
Stanley Tsang a6feafd5dc Build AllReduce only mode (#443) 
* Initial commit of all_reduce_only support

* Working AllReduce only build

* Removing printfs and restoring release build

* Restore P2P index

* Updates to build_allreduce_only mode.

* cleaning up macro ifdefs

[ROCm/rccl commit: 7e55b211c5]
 2021-10-26 17:36:46 -06:00
Wenkai Du 15143b1cfb Query XGMI link count through rocm_smi_lib API (#442) 
[ROCm/rccl commit: 14a184eb67]
 2021-10-26 10:30:20 -07:00
Wenkai Du aac9d57633 Fix collnet tuning parameters (#441) 
[ROCm/rccl commit: b4cefc05ed]
 2021-10-20 20:45:36 -07:00
Wenkai Du 4d979ce13d Fix PCIe gen detection (#437) 
* Fix PCIe gen detection

* Update profiling support

[ROCm/rccl commit: 2508507d0a]
 2021-10-15 08:23:50 -07:00
Wenkai Du b587b55c2e Add more Rome models (#434) 
* Add more Rome models

* Update models and tuning

* Update tuning

[ROCm/rccl commit: 2249a1d9d3]
 2021-10-12 08:23:20 -07:00
Wenkai Du d377c4dcc6 Add another Rome model (#431) 
[ROCm/rccl commit: e0053311c0]
 2021-10-06 08:17:12 -07:00
Wenkai Du 91eca0d7d2 Trim NICs when all GPUs are connected by XGMI (#430) 
* Trim NICs when all GPUs are connected by XGMI

* Only enable clique with maximum of 2 hops

[ROCm/rccl commit: 29c729d8b6]
 2021-10-05 18:27:43 -07:00
John Bachan 389213c7e8 Fix compilation failure in "src/enqueue.cc" on older GCC because of 
missing `#include <cstring>`.


[ROCm/rccl commit: 30ca3fcacf]
 2021-09-23 09:55:16 -07:00
Sylvain Jeaugey 761812841b Fix Collnet when GDR is disabled 
[ROCm/rccl commit: 4ec992fab7]
 2021-09-22 05:19:16 -07:00
Ke Wen e51f2a83da 2.11.4-1 
Add new API for creating a reduction operation which multiplies the input by a rank-specific scalar before doing an inter-rank summation (see: ncclRedOpCreatePreMulSum).
Improve CollNet (SHARP) performance of ncclAllReduce when captured in a CUDA Graph via user buffer registration.
Add environment variable NCCL_NET_PLUGIN="<suffix>" to allow user to choose among multiple NCCL net plugins by substituting into "libnccl-net-<suffix>.so".
Fix memory leak of NVB connections.
Fix topology detection of IB Virtual Functions (SR-IOV).


[ROCm/rccl commit: e11238b302]
 2021-09-08 16:06:23 -07:00
John Bachan 04553b802a Fix to https://github.com/NVIDIA/nccl/issues/560 
ncclGroup's containing operations of mixed datatype, element, or collective
would induce crash.


[ROCm/rccl commit: 5f2f2f670f]
 2021-08-31 15:50:05 -07:00
Wenkai Du a44ed86b46 Merge remote-tracking branch 'origin/develop' into 2.10.3 
[ROCm/rccl commit: 5b72727670]
 2021-09-15 10:33:25 -07:00
Wenkai Du 1eaf495391 Remove extra L1 cache invalidate and restore __ATOMIC_SEQ_CST atomics (#426) 
[ROCm/rccl commit: 5ae3f3f954]
 2021-09-14 18:30:16 -07:00
Wenkai Du d6064367f0 Merge remote-tracking branch 'origin/develop' into 2.10.3 
[ROCm/rccl commit: 3667d308ab]
 2021-09-13 17:32:48 -07:00
Wenkai Du f4387b2954 Use relaxed atomics and add sleep and wakeup in barrier loop (#425) 
* Use relaxed atomics and add sleep and wakeup in barrier loop

* atomicAdd in ROCm 4.3 only support unsigned long long

* Switch to atomicAdd and atomicExch in more places

* Restore LOAD/STORE define to __ATOMIC_SEQ_CST

* Restore atomic for sizes FIFO

[ROCm/rccl commit: 020484bf40]
 2021-09-13 17:03:49 -07:00
Wenkai Du 8acdb77cc0 Merge remote-tracking branch 'origin/develop' into 2.10.3 
[ROCm/rccl commit: 8ee2b7932a]
 2021-09-13 15:51:53 -07:00
Wenkai Du 9ffeb41fe1 Update tuning table (#424) 
[ROCm/rccl commit: ef432e48e1]
 2021-09-13 08:39:01 -07:00
Wenkai Du d2580c8cf5 Improve barrier implementation 
[ROCm/rccl commit: adb8d63352]
 2021-09-08 16:14:32 -05:00
Wenkai Du d75504e9dc Remove atomic from profiling 
[ROCm/rccl commit: 31bd4236f1]
 2021-09-08 14:20:32 -05:00
Wenkai Du 95b37d97cf Merge remote-tracking branch 'nccl/master' into 2.10.3 
[ROCm/rccl commit: 21969d7f89]
 2021-09-01 16:34:44 -07:00
Wenkai Du b9508a6aba Implement NIC identification and remapping (#420) 
* Add 1H16P GPU model

* Implement NIC identification and remapping

* Revert "Sort IB devices based on device name (#413)"

This reverts commit de0c586bad.

* Fix permute and check order

* Correction on IB speed reporting

* Revert "Allow user to link layer with RCCL_IB_HCA_SKIP_LINK_LAYER (#361)"

This reverts commit fa690c47a0.

[ROCm/rccl commit: 5c8380ff5b]
 2021-08-24 09:42:04 -07:00