2d0ed8dff6
Sort IB devices based on device name ( #413 )
2021-08-03 15:32:41 -07:00
3e27227562
XGMI connection is always prioritized over NET regardless of hops ( #412 )
2021-07-29 11:12:42 -07:00
229ca88ee6
Add basic rtest.xml ( #411 )
2021-07-28 11:53:03 -06:00
818cdb16a8
Query XGMI links from xml and adjust gfx906 channel usage ( #410 )
2021-07-27 17:32:41 -07:00
e579de7ec2
Update changelog with packaging information ( #409 )
2021-07-27 10:38:36 -06:00
135d47d125
topo_expl: fix build after switching to rocm-smi-lib ( #405 )
...
* topo_expl: fix build after switching to rocm-smi-lib
* Use minimal of 4 channels for gfx908
2021-07-27 08:30:08 -07:00
56801656f3
Extend test stage timeout to 24 hours ( #408 )
2021-07-26 15:29:21 -06:00
97d9cf40e7
Setup runtime and development packages ( #407 )
...
* changes to enable devel package
* Update rocm-cmake version & build
2021-07-26 15:06:17 -06:00
dfc62d5fbb
Skipping unnecessary functions in Doxygen by marking as internal ( #353 ) ( #406 )
...
(cherry picked from commit 1c982d819d9c7fe0310b80f9a25808e54c71137e)
Co-authored-by: saadrahim <44449863+saadrahim@users.noreply.github.com >
2021-07-24 11:04:27 -07:00
c3e2e7cb5d
Merge pull request #404 from hubertlu-tw/coll_trace
...
Enhancement of RCCL logging information for topology-aware optimization
2021-07-22 13:50:28 -07:00
bd6dbca8fb
Add more info to RCCL logging for topo-aware optim.
2021-07-22 09:52:39 -07:00
215904ee8e
Fix unit tests static build ( #403 )
2021-07-09 09:35:32 -07:00
56155ff5b6
Use rocm_smi_lib for getting topology information ( #402 )
...
* Use rocm_smi_lib for getting topology information
* Add rocm-smi-lib dependency to RCCL package
2021-07-08 13:23:11 -07:00
5c3e7d8b67
Fix static builds ( #393 )
2021-06-23 09:19:48 -06:00
2b0b608270
[TransferBench] Fixing a typo in TransferBench usage example ( #401 )
2021-06-22 17:08:57 -06:00
e75bc53e06
Deduct ROCM_PATH from CXX unless specified ( #400 )
2021-06-22 13:29:08 -07:00
fa6d7e9a63
Fixes for NCCL_MAX_NCHANNELS and topo_expl ( #398 )
2021-06-22 08:41:49 -07:00
720374a767
[TransferBench] Switching from little-endian fill pattern to big-endian ( #399 )
2021-06-21 14:28:51 -06:00
59d2867b01
Remove hard coded /opt/rocm from cmake ( #396 )
2021-06-21 08:29:23 -07:00
ff413be933
[TransferBench] Adding ability to specify source data pattern ( #394 )
...
* [TransferBench] Adding ability to specify source data pattern
2021-06-15 08:41:57 -06:00
fb267ea333
Move address-sanitizer build above addition of rccl library in CMakeLists ( #392 )
2021-06-11 14:43:54 -06:00
6dcae8a459
Select sendrecv path based on collective data size ( #391 )
...
* Select sendrecv path based on collective data size
* Add comments on packing and unpacking group field
* Toggling RCCL_P2P_NET_DISABLE in combined calls unit tests
2021-06-10 17:51:04 -07:00
f6f5e16fe6
Fixing bug with ExtractSubDataset function not fully initializing subdataset ( #390 )
2021-06-10 14:35:39 -06:00
eea7b24058
Add address sanitizer build option ( #389 )
2021-06-10 09:14:54 -06:00
b815a2800f
Setup collectives threshold for enabling intranet ( #387 )
...
* Setup collectives threshold for enabling intranet
* Use separate operation counters for coll and p2p
2021-06-09 13:24:26 -07:00
c2064adcc7
Add support for another Rome model ( #385 )
2021-06-08 13:58:20 -07:00
6842429a14
Updating changelog to show install script fix ( #384 )
2021-06-08 13:00:40 -06:00
a3a8c2d56b
Allow intranode use of network connection ( #383 )
...
* Allow intranode use of network connection
* Checking for graph for null pointer
2021-06-08 07:37:59 -07:00
820a53287f
Fixing install script so that invoking -r alone does not trigger rebuild ( #382 )
2021-06-04 09:46:04 -06:00
961922ea02
Add option to enable multiple SAT in SHARP ( #380 )
...
* Add option to enable multiple SAT in SHARP
* Extend number of NICs to 16
2021-06-03 19:45:18 -07:00
903c84050d
ROCm 4.3 changelog update ( #379 )
...
* Update CHANGELOG.md (#378 )
* Updating CHANGELOG.md for ROCm 4.3
2021-06-03 10:56:02 -06:00
03ac898825
Merge pull request #377 from wenkaidu/2.9.9
...
Sync up with NCCL 2.9.9
2021-05-26 11:38:19 -07:00
13dc80ee14
topo_expl: update to 2.9.9
2021-05-26 09:24:34 -07:00
e3abf1c2ec
Merge remote-tracking branch 'nccl/master' into develop
2021-05-25 20:52:15 -07:00
256403d4f0
Adding support for hipMallocManaged() in unit tests ( #375 )
...
* Adding HMM support for unit tests
* Fixing HMM opt-in check
2021-05-25 17:07:12 -06:00
4c83adb75c
Update Rome models matching ( #376 )
2021-05-25 10:12:40 -07:00
8e817ecd6d
Tweak clique channel usage for gfx908 ( #374 )
2021-05-21 15:36:21 -06:00
50da1b48af
Correction on max number of groups ( #373 )
2021-05-20 08:58:45 -07:00
8cde34be51
Use fixed segment size for sendrecv ( #369 )
2021-05-19 08:25:26 -07:00
42b080867e
Running only sum for CI quick test ( #370 )
2021-05-19 08:25:13 -07:00
ddceadc313
Tune clique-based AllReduce for device type 908 ( #372 )
...
* Changing switch-over point for clique-based AllReduce
2021-05-18 15:36:07 -06:00
2daadcc834
Disabling env var caching for all unit tests ( #371 )
...
* Disabling env var caching for all unit tests
2021-05-18 12:56:30 -06:00
2f31289fe6
Merge pull request #368 from ROCmSoftwarePlatform/2.9.8
...
Merge NCCL 2.9.8
2021-05-18 08:38:59 -07:00
87727383fe
Merge remote-tracking branch 'nccl/master' into 2.9.8
2021-05-17 10:15:16 -07:00
0b2bfdd6d8
Multiprocess unit test various fixes ( #367 )
...
* Re-enabling mp unit tests
* Fixing shared memory leak and other bugs related to shared mem for MP unit tests
* Revert 43bfbfc97bf9edbae1f386d461439091618ff8ed
* Further tightening up unlinks
* Moving test check macros to separate header file
* Tightening up shared memory unlinking for clique kernels, add munmap for host barrier for MP unit tests
* Updating new MP unit test
* Fixing mqueue bug
* Fixing memory leak in MP unit tests
2021-05-14 09:38:49 -06:00
caf5c9992a
Allow user to link layer with RCCL_IB_HCA_SKIP_LINK_LAYER ( #361 )
...
To skip Infiniband, set RCCL_IB_HCA_SKIP_LINK_LAYER=1.
To skip Ethernet, RCCL_IB_HCA_SKIP_LINK_LAYER=2.
2021-05-12 14:14:53 -07:00
3fec2fa5ee
2.9.9-1
...
Fix crash when setting NCCL_MAX_P2P_NCHANNELS below nchannels.
Fix hang during sendrecv dynamic NVB connection establishment on
cubemesh topologies.
Add environment variable to only use SHARP on communicators beyond
a given number of ranks.
Add debug subsystem to trace memory allocations.
Fix compilation with TRACE=1. (Issue #505 )
2021-05-12 11:09:31 -07:00
abde40197a
Merge pull request #366 from ROCmSoftwarePlatform/2.9.6
...
Sync up to NCCL 2.9.6
2021-05-11 20:20:42 -07:00
330b82df3b
Revert "Sync up to NCCL 2.9.6 ( #363 )" ( #365 )
...
This reverts commit 6021329af0
2021-05-11 20:18:17 -07:00
6021329af0
Sync up to NCCL 2.9.6 ( #363 )
...
* 2.9.6-1
Add support for CUDA graphs.
Fuse BCM Gen4 switches to avoid suboptimal performance on some platforms. Issue #439 .
Fix bootstrap issue caused by connection reordering.
Fix CPU locking block.
Improve CollNet algorithm.
Improve performance on DGX A100 for communicators with only one GPU per node.
* Clique tuning upgrade (#352 ) (#19 )
* Enabling clique for any XGMI-connected topology, adding tuning
* Updating CHANGELOG for clique tuning
* Re-working clique barrier system to work on multi-process / multi-gpu
Co-authored-by: Sylvain Jeaugey <sjeaugey@nvidia.com >
Co-authored-by: gilbertlee-amd <44450918+gilbertlee-amd@users.noreply.github.com >
2021-05-11 19:40:34 -07:00
Wenkai Du
Eiden Yoshida
Liam Wrubleski
Hubert Lu
Lu
gilbertlee-amd
Stanley Tsang
Wenkai Du
Sylvain Jeaugey