58b2c1ec9c
Fix random segfault ( #537 )
...
[ROCm/rccl commit: fd2f1b3b88
2022-04-15 14:32:11 -07:00
011447e4dc
Add new Rome model ( #536 )
...
[ROCm/rccl commit: 2151c79d14
2022-04-13 11:45:40 -07:00
f8023f2e07
Add new Rome model ( #535 )
...
[ROCm/rccl commit: ba4c165bf3
2022-04-12 13:27:32 -07:00
5ccdd9f5e1
Increase chunk steps of broadcast and reduce ( #528 )
...
[ROCm/rccl commit: 15b572751e
2022-04-07 13:34:04 -07:00
3830310156
Doxygen fix for ncclRecv ( #527 )
...
Changed the Doxygen command for ncclRecv and pncclRecv, to be consistent with other APIs.
[ROCm/rccl commit: b2ffcf6d89
2022-04-05 14:07:56 -07:00
9884e61367
Add tuning model ( #523 )
...
[ROCm/rccl commit: 5cc0a405c0
2022-04-04 10:19:57 -07:00
3332cdff07
Support multiple tuning tables ( #522 )
...
* Support multiple tuning tables
* [UnitTests] Skip managed memory testing
[ROCm/rccl commit: bbe780ca6c
2022-03-31 17:09:21 -07:00
828f3d11a0
Update tuning parameters ( #518 )
...
* Update tuning parameters
* Respect user algo and topo selections
[ROCm/rccl commit: 7cbbca4da1
2022-03-29 08:15:37 -07:00
db1e628ba3
Update Rome model matching and add new models ( #516 )
...
* Update Rome model matching and add new models
* Add missing file
* Models update
[ROCm/rccl commit: cd17cf6dce
2022-03-21 10:54:40 -07:00
133aed2dfb
Force ring algorithm on single node ( #509 )
...
[ROCm/rccl commit: d6d6af710e
2022-03-04 10:29:02 -08:00
a182076a0e
Unit test refactor ( #500 )
...
Refactoring and consolidating single-process / multi-process unit testing
[ROCm/rccl commit: 29ad0f5fbe
2022-02-25 08:59:07 -07:00
dfa9b9e958
Add Pivot AllToAll algorithm for Rome model ( #503 )
...
* add a2a pivot interface
* remove debug info
* address comments
* fix bug
* remove custom script
* address comments
* fix bug
[ROCm/rccl commit: b569c0a1db
2022-02-20 21:09:47 -08:00
0f0388ba0b
Allow additional options to be passed in through model's definition ( #501 )
...
[ROCm/rccl commit: 94e0dc8bfd
2022-02-17 08:28:58 -08:00
5b697e40db
Add another Rome model ( #497 )
...
[ROCm/rccl commit: 02096c9936
2022-02-12 10:30:16 -08:00
635c0bcc01
Generate proper b-tree with non-repeating channels ( #493 )
...
[ROCm/rccl commit: 400df49dbe
2022-01-19 15:09:17 -08:00
663c1db898
Merge remote-tracking branch 'nccl/master' into develop
...
[ROCm/rccl commit: 973d0111db
2022-01-17 13:34:36 -08:00
4d43d9ce22
Update Rome models ( #491 )
...
[ROCm/rccl commit: 598c6fdded
2022-01-14 10:03:30 -08:00
93ab252d52
Use noinline and a few other fixes ( #489 )
...
* Use noinline and a few other fixes
* Tune collectives
[ROCm/rccl commit: 6ff7690cb5
2022-01-11 16:51:06 -08:00
123de1621f
Use hipGraph instead of cudaGraph ( #487 )
...
[ROCm/rccl commit: 3669e12432
2022-01-10 08:26:01 -08:00
3f16835565
Tune collectives for 2.11.4 ( #486 )
...
[ROCm/rccl commit: 565fbeb5e9
2022-01-10 08:25:47 -08:00
e5ff692673
Merge pull request #482 from ROCmSoftwarePlatform/2.11.4
...
Sync up with 2.11.4
[ROCm/rccl commit: 4234a638b5
2022-01-05 09:31:51 -08:00
93c7526fdc
Add another Rome model ( #483 )
...
[ROCm/rccl commit: f8d0775a6f
2022-01-05 09:26:31 -08:00
618fbfc644
Merge remote-tracking branch 'origin/develop' into 2.11.4
...
[ROCm/rccl commit: 434ecb0e1f
2022-01-03 09:54:16 -08:00
07d9648aec
Add env NCCL_NET_DISABLE_INTRA
...
Disable NET transport for intra-node communication by setting the env to 1
It provides an option to error out instead of falling back to NET when superior intra-node transports (P2P and SHM) are unavailable
[ROCm/rccl commit: c88c9f873f
2021-12-08 16:28:19 +01:00
7406aed9c9
Update Rome model ( #479 )
...
[ROCm/rccl commit: a94b953bcc
2021-12-03 08:24:51 -08:00
dd4a53db37
Update tuning parameters ( #478 )
...
[ROCm/rccl commit: 8a08a2f579
2021-11-30 08:51:11 -08:00
2da8358a74
Minor cppcheck fixes, adding suppression file ( #475 )
...
* Minor cppcheck fixes, adding suppression file
[ROCm/rccl commit: 539de1216f
2021-11-24 10:23:59 -07:00
864c6a645e
Determine fine grained memory availability at RCCL bootstrapping ( #471 )
...
[ROCm/rccl commit: e9bf01fb7e
2021-11-19 08:12:53 -08:00
a52ca36a9e
Perform busIdToInt64 on the stack.
...
I noticed when I enabled `NCCL_DEBUG_SUBSYS=ALLOC` that this function is
called thousands of times, making the log output unintelligible.
Fortunately, this function can be implemented without heap allocations.
[ROCm/rccl commit: 8cf7325d69
2021-11-19 09:35:55 +01:00
f8bd2d0cfa
Merge remote-tracking branch 'nccl/master' into develop
...
[ROCm/rccl commit: 3a919c1f49
2021-11-11 14:22:12 -08:00
fc5f692f71
Remove extra work element copy ( #465 )
...
[ROCm/rccl commit: e05de8fd26
2021-11-09 13:52:03 -08:00
680660fcf7
Do not use async stream for memory allocation and transport setup without graph ( #460 )
...
[ROCm/rccl commit: 33bdd557c8
2021-11-05 13:39:14 -07:00
fd98ee84b4
Update Rome model matching ( #461 )
...
* Update Rome model matching
* Add another Rome model
* Automatically setup NET GDR level from model
[ROCm/rccl commit: 0331e39f81
2021-11-05 08:53:47 -07:00
befaa35e12
profiling: fix incorrect print out in timing profile ( #457 )
...
[ROCm/rccl commit: 26fc6b0919
2021-11-03 16:22:21 -07:00
df59f64e3f
Support different protocols and algorithms in all reduce only build ( #455 )
...
* Support different protocols and algorithms in all reduce only build
* Restore deleted line in error
[ROCm/rccl commit: 29170a8b5f
2021-11-02 08:39:08 -07:00
a11b55a37f
Check rocm_smi64Config.h on older ROCm build ( #452 )
...
[ROCm/rccl commit: 4643a17f83
2021-10-28 07:26:28 -07:00
b0b2e27df3
Rework kernel launch code ( #449 )
...
[ROCm/rccl commit: d221fb672a
2021-10-28 07:26:11 -07:00
747216e2b2
Enable timing profiling mode ( #447 )
...
[ROCm/rccl commit: ec36c4c326
2021-10-27 08:21:48 -07:00
a6feafd5dc
Build AllReduce only mode ( #443 )
...
* Initial commit of all_reduce_only support
* Working AllReduce only build
* Removing printfs and restoring release build
* Restore P2P index
* Updates to build_allreduce_only mode.
* cleaning up macro ifdefs
[ROCm/rccl commit: 7e55b211c5
2021-10-26 17:36:46 -06:00
15143b1cfb
Query XGMI link count through rocm_smi_lib API ( #442 )
...
[ROCm/rccl commit: 14a184eb67
2021-10-26 10:30:20 -07:00
aac9d57633
Fix collnet tuning parameters ( #441 )
...
[ROCm/rccl commit: b4cefc05ed
2021-10-20 20:45:36 -07:00
4d979ce13d
Fix PCIe gen detection ( #437 )
...
* Fix PCIe gen detection
* Update profiling support
[ROCm/rccl commit: 2508507d0a
2021-10-15 08:23:50 -07:00
b587b55c2e
Add more Rome models ( #434 )
...
* Add more Rome models
* Update models and tuning
* Update tuning
[ROCm/rccl commit: 2249a1d9d3
2021-10-12 08:23:20 -07:00
d377c4dcc6
Add another Rome model ( #431 )
...
[ROCm/rccl commit: e0053311c0
2021-10-06 08:17:12 -07:00
91eca0d7d2
Trim NICs when all GPUs are connected by XGMI ( #430 )
...
* Trim NICs when all GPUs are connected by XGMI
* Only enable clique with maximum of 2 hops
[ROCm/rccl commit: 29c729d8b6
2021-10-05 18:27:43 -07:00
389213c7e8
Fix compilation failure in "src/enqueue.cc" on older GCC because of
...
missing `#include <cstring>`.
[ROCm/rccl commit: 30ca3fcacf
2021-09-23 09:55:16 -07:00
761812841b
Fix Collnet when GDR is disabled
...
[ROCm/rccl commit: 4ec992fab7
2021-09-22 05:19:16 -07:00
e51f2a83da
2.11.4-1
...
Add new API for creating a reduction operation which multiplies the input by a rank-specific scalar before doing an inter-rank summation (see: ncclRedOpCreatePreMulSum).
Improve CollNet (SHARP) performance of ncclAllReduce when captured in a CUDA Graph via user buffer registration.
Add environment variable NCCL_NET_PLUGIN="<suffix>" to allow user to choose among multiple NCCL net plugins by substituting into "libnccl-net-<suffix>.so".
Fix memory leak of NVB connections.
Fix topology detection of IB Virtual Functions (SR-IOV).
[ROCm/rccl commit: e11238b302
2021-09-08 16:06:23 -07:00
04553b802a
Fix to https://github.com/NVIDIA/nccl/issues/560
...
ncclGroup's containing operations of mixed datatype, element, or collective
would induce crash.
[ROCm/rccl commit: 5f2f2f670f
2021-08-31 15:50:05 -07:00
a44ed86b46
Merge remote-tracking branch 'origin/develop' into 2.10.3
...
[ROCm/rccl commit: 5b72727670
2021-09-15 10:33:25 -07:00
Wenkai Du
Colin Smith
gilbertlee-amd
Ziyue Yang
Wenkai Du
Ke Wen
Chris Jones
Stanley Tsang
John Bachan
Sylvain Jeaugey
Ke Wen