0fb000932f
Change default nchannels per peer ( #563 )
...
[ROCm/rccl commit: f274c865c1
2022-06-13 06:39:05 -07:00
8953f5b5ca
Introduce multi-rank support per device.
...
This is a single commit of the source code changes required to
introduce support for multiple ranks per device.
A new interface (ncclCommRankInitMulti) has to be used to make use of
this new feature.
[ROCm/rccl commit: 0336ffdf70
2022-06-10 14:23:12 +00:00
11a6cdd52f
Fix P2P scheduling ( #560 )
...
[ROCm/rccl commit: 5cb2aca3d9
2022-06-06 13:32:28 -07:00
f2dbc77afe
Enable timing profile option ( #558 )
...
[ROCm/rccl commit: 7a6c6927ae
2022-06-03 07:05:13 -07:00
0b55e01ef3
Merge remote-tracking branch 'ncclRepo/master' into develop
...
[ROCm/rccl commit: e0864e7093
2022-06-02 15:27:24 +00:00
1e36b432f1
Revert chunksteps changes ( #555 )
...
[ROCm/rccl commit: eef812bed7
2022-05-31 14:45:51 -07:00
5becf1669f
Add another Rome model ( #553 )
...
* Add another Rome model
* Add option to force enable intranet on single node
* Limit p2p channels to number of ranks
* Refine p2p channels handling
[ROCm/rccl commit: ef499c4810
2022-05-31 11:31:30 -07:00
a03ab8e752
code cleanup ( #554 )
...
[ROCm/rccl commit: a0a686e74c
2022-05-31 09:59:36 -04:00
2c125ce6ed
Update Rome model ( #552 )
...
[ROCm/rccl commit: c5b77121f0
2022-05-26 09:59:23 -07:00
22dc8bd246
Added creation of new tree and added switch for using treesplit for specific cases ( #551 )
...
[ROCm/rccl commit: 98f0809a39
2022-05-25 18:55:14 -04:00
a2a4888497
Moving opt-in custom signal handler from UnitTests into RCCL ( #550 )
...
* Enable via RCCL_ENABLE_SIGNALHANDLER=1
[ROCm/rccl commit: 700b473211
2022-05-20 09:56:38 -06:00
86e8797602
Add switch for pivot alltoall kernel ( #549 )
...
[ROCm/rccl commit: 6707a270b1
2022-05-17 18:14:04 -07:00
b30b8becea
Refine and add new Rome models ( #548 )
...
[ROCm/rccl commit: 283dc86a73
2022-05-17 08:23:59 -07:00
1c5734046d
2.12.12-1
...
Improve allreduce performance when we have more than one network interface per
GPU and we need to use PXN to close rings.
Add support for PCI Gen5 on 5.4 kernels.
Fix crash when setting NCCL_SET_THREAD_NAME.
Fix random crash in init due to uninitialized struct.
Fix hang on cubemesh topologies.
Add P2P_DIRECT_DISABLE parameter to disable direct access to pointers within a
process.
[ROCm/rccl commit: 7aa1c46fd5
2022-05-13 00:26:57 -07:00
b37180b7ed
Improve LL performance ( #546 )
...
* Improve LL performance
* Add split barriers for LL
[ROCm/rccl commit: c9919e0e35
2022-05-10 13:32:10 -07:00
347ea354c2
Update tuning parameters
...
[ROCm/rccl commit: 83fd4f70e7
2022-04-18 16:04:04 -07:00
67e7e6507e
Merge remote-tracking branch 'nccl/master' into develop
...
[ROCm/rccl commit: d28e1cb44f
2022-04-18 11:15:25 -07:00
58b2c1ec9c
Fix random segfault ( #537 )
...
[ROCm/rccl commit: fd2f1b3b88
2022-04-15 14:32:11 -07:00
011447e4dc
Add new Rome model ( #536 )
...
[ROCm/rccl commit: 2151c79d14
2022-04-13 11:45:40 -07:00
f8023f2e07
Add new Rome model ( #535 )
...
[ROCm/rccl commit: ba4c165bf3
2022-04-12 13:27:32 -07:00
e89ff21d35
Update Makefile to install static library.
...
Make sure make install also installs the static library.
Fixes #662
[ROCm/rccl commit: 9bfc1c6e35
2022-04-08 14:00:43 +02:00
5ccdd9f5e1
Increase chunk steps of broadcast and reduce ( #528 )
...
[ROCm/rccl commit: 15b572751e
2022-04-07 13:34:04 -07:00
3830310156
Doxygen fix for ncclRecv ( #527 )
...
Changed the Doxygen command for ncclRecv and pncclRecv, to be consistent with other APIs.
[ROCm/rccl commit: b2ffcf6d89
2022-04-05 14:07:56 -07:00
9884e61367
Add tuning model ( #523 )
...
[ROCm/rccl commit: 5cc0a405c0
2022-04-04 10:19:57 -07:00
3332cdff07
Support multiple tuning tables ( #522 )
...
* Support multiple tuning tables
* [UnitTests] Skip managed memory testing
[ROCm/rccl commit: bbe780ca6c
2022-03-31 17:09:21 -07:00
74f8baa0f3
Merge remote-tracking branch 'origin/master'
...
[ROCm/rccl commit: 8133784b32
2022-03-30 02:29:05 -07:00
27130280b2
2.12.10-1
...
Fix bug with CollNet
Fix bug with zero-bytes send/recv operations
Fix NCCL_PARAM implementation to avoid taking a lock on every call
Fix bug when setting NCCL_IB_QPS_PER_CONNECTION to more than one.
Improve error reporting for network errors.
[ROCm/rccl commit: 353e8ba446
2022-03-30 02:27:01 -07:00
a52e328ba4
Fix merging error
...
[ROCm/rccl commit: 2247152a8e
2022-03-30 02:14:32 -07:00
3bc2e34df2
Merge branch 'master' into truncated_msg_warning
...
[ROCm/rccl commit: 2dfd83752c
2022-03-30 10:58:05 +02:00
92bdde35eb
Display host name instead of numeric IP when referring to a peer
...
For easier interpretation of debug messages like "connection closed by
peer", "peer message truncated" and "peer collective mismatch"
[ROCm/rccl commit: 1382a87306
2022-03-30 10:47:10 +02:00
f6d1c7261f
Fix typo in net_ib.cc
...
[ROCm/rccl commit: b895abcdb8
2022-03-30 10:45:01 +02:00
54590464ca
Remove unnecessary newline in plugin logging
...
Signed-off-by: Felix Abecassis <fabecassis@nvidia.com >
[ROCm/rccl commit: 1c7c014ceb
2022-03-30 10:44:49 +02:00
828f3d11a0
Update tuning parameters ( #518 )
...
* Update tuning parameters
* Respect user algo and topo selections
[ROCm/rccl commit: 7cbbca4da1
2022-03-29 08:15:37 -07:00
db1e628ba3
Update Rome model matching and add new models ( #516 )
...
* Update Rome model matching and add new models
* Add missing file
* Models update
[ROCm/rccl commit: cd17cf6dce
2022-03-21 10:54:40 -07:00
7707479804
Add pthread_detach()'s for threads we never pthread_join(). Helps
...
reduce diagnostic noise for ThreadSanitizer.
Fixes https://github.com/NVIDIA/nccl/issues/649
[ROCm/rccl commit: 44eb40da0e
2022-03-15 10:27:59 -07:00
133aed2dfb
Force ring algorithm on single node ( #509 )
...
[ROCm/rccl commit: d6d6af710e
2022-03-04 10:29:02 -08:00
f8886d8687
2.12.7-1
...
Add network communication through another GPU connected with NVLink
(PXN).
Add aggregation of messages coming from different local GPUs through
PXN and going to the same destination.
Add new v5 plugin API with grouped receives and tags.
Add compat for v4 plugins.
Add naming of NCCL threads to help debugging.
Fix NVLink detection and avoid data corruption when some NVLinks are
down.
Add support for Relaxed Ordering for IB.
Add profiling and timing infrastructure.
[ROCm/rccl commit: 3c223c105a
2022-03-02 20:48:56 +01:00
a182076a0e
Unit test refactor ( #500 )
...
Refactoring and consolidating single-process / multi-process unit testing
[ROCm/rccl commit: 29ad0f5fbe
2022-02-25 08:59:07 -07:00
dfa9b9e958
Add Pivot AllToAll algorithm for Rome model ( #503 )
...
* add a2a pivot interface
* remove debug info
* address comments
* fix bug
* remove custom script
* address comments
* fix bug
[ROCm/rccl commit: b569c0a1db
2022-02-20 21:09:47 -08:00
0f0388ba0b
Allow additional options to be passed in through model's definition ( #501 )
...
[ROCm/rccl commit: 94e0dc8bfd
2022-02-17 08:28:58 -08:00
5b697e40db
Add another Rome model ( #497 )
...
[ROCm/rccl commit: 02096c9936
2022-02-12 10:30:16 -08:00
92d6888bdc
Split IB parameter sanity check into two parts
...
First part on collective mismatch, second part on internal errors
[ROCm/rccl commit: fbfb6ac5d7
2022-02-08 15:21:22 -08:00
51df47f9b8
Revert "remove unused basePath"
...
This reverts commit d973ddac8bcc78e9fab8
2022-01-21 12:30:34 +01:00
d973ddac8b
remove unused basePath
...
[ROCm/rccl commit: 445bc19657
2022-01-21 12:12:26 +01:00
635c0bcc01
Generate proper b-tree with non-repeating channels ( #493 )
...
[ROCm/rccl commit: 400df49dbe
2022-01-19 15:09:17 -08:00
663c1db898
Merge remote-tracking branch 'nccl/master' into develop
...
[ROCm/rccl commit: 973d0111db
2022-01-17 13:34:36 -08:00
4d43d9ce22
Update Rome models ( #491 )
...
[ROCm/rccl commit: 598c6fdded
2022-01-14 10:03:30 -08:00
93ab252d52
Use noinline and a few other fixes ( #489 )
...
* Use noinline and a few other fixes
* Tune collectives
[ROCm/rccl commit: 6ff7690cb5
2022-01-11 16:51:06 -08:00
123de1621f
Use hipGraph instead of cudaGraph ( #487 )
...
[ROCm/rccl commit: 3669e12432
2022-01-10 08:26:01 -08:00
3f16835565
Tune collectives for 2.11.4 ( #486 )
...
[ROCm/rccl commit: 565fbeb5e9
2022-01-10 08:25:47 -08:00
Wenkai Du
Edgar
Aristotelis
akolliasAMD
gilbertlee-amd
Sylvain Jeaugey
Wenkai Du
Colin Smith
Ke Wen
Christopher Hesse
Felix Abecassis
John Bachan
Ziyue Yang
void-main