Commit Graph

407 Commits

Author SHA1 Message Date
Cosmic Fusion 1dcf1da5ca fix error: use of undeclared identifier 'free'
include stdlib.h to fix compilation error in rccl :

[39/58] Building CXX object CMakeFiles/rccl.dir/src/misc/signals.cc.o
FAILED: CMakeFiles/rccl.dir/src/misc/signals.cc.o 
/opt/rocm/bin/hipcc -DENABLE_COLLTRACE -DHAVE_BFD -DHAVE_CPLUS_DEMANGLE -DUSE_ROCM_SMI64CONFIG -D__HIP_PLATFORM_AMD__=1 -D__HIP_PLATFORM_HCC__=1 -Drccl_EXPORTS -I/home/cosmo/build/flgrwqa/build/include -I/home/cosmo/build/flgrwqa/build/include/rccl -I/home/cosmo/build/flgrwqa/rccl/src -I/home/cosmo/build/flgrwqa/rccl/src/include -I/home/cosmo/build/flgrwqa/rccl/src/collectives -I/home/cosmo/build/flgrwqa/rccl/src/collectives/device -I/opt/hsa/include -fPIC -fvisibility=hidden -fgpu-rdc -parallel-jobs=8 -Wno-format-nonliteral -x hip --offload-arch=gfx803 --offload-arch=gfx900:xnack- --offload-arch=gfx906:xnack- --offload-arch=gfx908:xnack- --offload-arch=gfx90a:xnack- --offload-arch=gfx90a:xnack+ --offload-arch=gfx1030 -std=c++14 -MD -MT CMakeFiles/rccl.dir/src/misc/signals.cc.o -MF CMakeFiles/rccl.dir/src/misc/signals.cc.o.d -o CMakeFiles/rccl.dir/src/misc/signals.cc.o -c /home/cosmo/build/flgrwqa/rccl/src/misc/signals.cc
In file included from /home/cosmo/build/flgrwqa/rccl/src/misc/signals.cc:8:
/home/cosmo/build/flgrwqa/rccl/src/include/BfdBacktrace.hpp:138:9: error: use of undeclared identifier 'free'
        free(file->syms);
        ^
/home/cosmo/build/flgrwqa/rccl/src/include/BfdBacktrace.hpp:155:5: error: use of undeclared identifier 'free'
    free(file->syms);
    ^

[ROCm/rccl commit: 080fc2d9d6]
2022-08-19 20:25:06 +03:00
Wenkai Du c2e9ada40b Repurpose profiling implementation to simple timestamps tracing (#600)
[ROCm/rccl commit: 14b8ff153f]
2022-08-18 15:34:46 -07:00
Wenkai Du 6c3f1366e8 Add XGMI sys type and clean up detection code (#597)
[ROCm/rccl commit: f5c0b243a8]
2022-08-12 09:52:29 -07:00
Ziyue Yang 478d8312b8 Improve alignment and tuning for Pivot A2A algorithm (#593)
* Improve alignment and tuning for Pivot A2A algorithm

* enable pivot a2a by default

[ROCm/rccl commit: f6b9686482]
2022-08-05 19:40:19 -07:00
akolliasAMD 6fb5c5d5e3 minor latency tuning (#591)
* minor tuning for tree ll

[ROCm/rccl commit: 4cecdc9be5]
2022-08-03 15:07:44 -06:00
Wenkai Du f70830d629 Revert "Use nontemporal in slow path and add XGMI sys type (#575)" (#590)
This reverts commit e04bba619a.

[ROCm/rccl commit: 9089e68a99]
2022-08-02 09:31:53 -07:00
akolliasAMD d5ca0be51f Fixed issue with attomicEXCH creating errors on multi node runs (#587)
[ROCm/rccl commit: 254208e7dd]
2022-07-22 11:32:49 -06:00
akolliasAMD 18d9fd1b8f Removing redundant LOAD and STORE on primitives plus adding some atomics (#585)
[ROCm/rccl commit: 451c287aa6]
2022-07-21 13:04:57 -06:00
Hubert Lu 088d62ff58 Merge pull request #580 from hubertlu-tw/develop
Enhancement of RCCL logging information for topology-aware utilities

[ROCm/rccl commit: 6dd090917a]
2022-07-15 15:16:37 -07:00
Edgar Gabriel a9e0333dba intercept SIGUSR2 in RCCL
add support for intercepting SIGUSR2 in RCCL. This signal will
not terminate the execution of the application, but print the stacktrace
of the process that the signal was sent to instead.


[ROCm/rccl commit: 2b1d5d3bc1]
2022-07-15 16:28:46 +00:00
Ziyue Yang e1aae026bf Remove redundant LOAD/STORE usage in LL initialization
[ROCm/rccl commit: 77c2bef952]
2022-07-14 00:40:36 +00:00
hubertlu-tw e13eb2eab9 Enhancement of RCCL logging information for topology-aware utilities
[ROCm/rccl commit: a1842df858]
2022-07-11 19:01:10 +00:00
Wenkai Du c129677fe0 Skip HDP cache flush for gfx90a (#578)
* Skip HDP cache flush for gfx90a

* Remove extra debug print

[ROCm/rccl commit: 8c3c8b78c0]
2022-07-08 10:13:32 -07:00
Wenkai Du 659cd52d5c Add more constraints to enable GDR (#579)
* Add more constraints to enable GDR

* Revert deleted line

[ROCm/rccl commit: aa0d7ca882]
2022-07-08 09:52:27 -07:00
Yifan Xiong bf15ad1d72 Reduce AlltoAll port usage in send/recv proxy (#577)
* Reduce AlltoAll port usage when connecting proxy

Reuse socket ports when connecting proxies in AlltoAll.

Existing port usage in AlltoAll is O(n) for recv and O(n) for send,
reusing socket ports in server or client side will make one of them
O(1), reusing both will reduce the total port usage to O(1) and enables
AlltoAll in >64 MI200 nodes.

* Update changelog accordingly

Update changelog accordingly.

[ROCm/rccl commit: 80f53cc171]
2022-07-07 16:15:52 -07:00
Wenkai Du e04bba619a Use nontemporal in slow path and add XGMI sys type (#575)
* Use nontemporal in slow path and add XGMI sys type

* Clean up XGMI detection

[ROCm/rccl commit: b250c01cbe]
2022-07-06 07:58:41 -07:00
Wenkai Du 2f4aea93e0 Fix GPU to NIC mapping in tree (#573)
* Fix GPU to NIC mapping in tree

* Update tuning table

[ROCm/rccl commit: 00af1f64e9]
2022-07-03 20:52:52 -07:00
gilbertlee-amd cb5ae7224e Adding git hash info to version output line (#572)
[ROCm/rccl commit: a89a9966aa]
2022-06-28 16:42:51 -06:00
Wenkai Du 915a9d3934 Do not set NET GDR level automatically (#571)
[ROCm/rccl commit: 9a285b5e1d]
2022-06-23 16:28:28 -07:00
Wenkai Du 784b12bf75 Use different atomics to check flags in kernel (#568)
[ROCm/rccl commit: c3bb9e70d0]
2022-06-23 09:16:41 -07:00
Ziyue Yang 2b418b5dee Add Feature - Add NPKit Support in RCCL (#564)
* apply npkit

* fix bug

* add npkit in readme

[ROCm/rccl commit: 6e93fafdc3]
2022-06-20 14:30:19 -07:00
Wenkai Du 0fb000932f Change default nchannels per peer (#563)
[ROCm/rccl commit: f274c865c1]
2022-06-13 06:39:05 -07:00
Edgar 8953f5b5ca Introduce multi-rank support per device.
This is a single commit of the source code changes required to
introduce support for multiple ranks per device.
A new interface (ncclCommRankInitMulti) has to be used to make use of
this new feature.


[ROCm/rccl commit: 0336ffdf70]
2022-06-10 14:23:12 +00:00
Wenkai Du 11a6cdd52f Fix P2P scheduling (#560)
[ROCm/rccl commit: 5cb2aca3d9]
2022-06-06 13:32:28 -07:00
Wenkai Du f2dbc77afe Enable timing profile option (#558)
[ROCm/rccl commit: 7a6c6927ae]
2022-06-03 07:05:13 -07:00
Aristotelis 0b55e01ef3 Merge remote-tracking branch 'ncclRepo/master' into develop
[ROCm/rccl commit: e0864e7093]
2022-06-02 15:27:24 +00:00
Wenkai Du 1e36b432f1 Revert chunksteps changes (#555)
[ROCm/rccl commit: eef812bed7]
2022-05-31 14:45:51 -07:00
Wenkai Du 5becf1669f Add another Rome model (#553)
* Add another Rome model

* Add option to force enable intranet on single node

* Limit p2p channels to number of ranks

* Refine p2p channels handling

[ROCm/rccl commit: ef499c4810]
2022-05-31 11:31:30 -07:00
akolliasAMD a03ab8e752 code cleanup (#554)
[ROCm/rccl commit: a0a686e74c]
2022-05-31 09:59:36 -04:00
Wenkai Du 2c125ce6ed Update Rome model (#552)
[ROCm/rccl commit: c5b77121f0]
2022-05-26 09:59:23 -07:00
akolliasAMD 22dc8bd246 Added creation of new tree and added switch for using treesplit for specific cases (#551)
[ROCm/rccl commit: 98f0809a39]
2022-05-25 18:55:14 -04:00
gilbertlee-amd a2a4888497 Moving opt-in custom signal handler from UnitTests into RCCL (#550)
* Enable via RCCL_ENABLE_SIGNALHANDLER=1

[ROCm/rccl commit: 700b473211]
2022-05-20 09:56:38 -06:00
Wenkai Du 86e8797602 Add switch for pivot alltoall kernel (#549)
[ROCm/rccl commit: 6707a270b1]
2022-05-17 18:14:04 -07:00
Wenkai Du b30b8becea Refine and add new Rome models (#548)
[ROCm/rccl commit: 283dc86a73]
2022-05-17 08:23:59 -07:00
Sylvain Jeaugey 1c5734046d 2.12.12-1
Improve allreduce performance when we have more than one network interface per
GPU and we need to use PXN to close rings.
Add support for PCI Gen5 on 5.4 kernels.
Fix crash when setting NCCL_SET_THREAD_NAME.
Fix random crash in init due to uninitialized struct.
Fix hang on cubemesh topologies.
Add P2P_DIRECT_DISABLE parameter to disable direct access to pointers within a
process.


[ROCm/rccl commit: 7aa1c46fd5]
2022-05-13 00:26:57 -07:00
Wenkai Du b37180b7ed Improve LL performance (#546)
* Improve LL performance

* Add split barriers for LL

[ROCm/rccl commit: c9919e0e35]
2022-05-10 13:32:10 -07:00
Wenkai Du 347ea354c2 Update tuning parameters
[ROCm/rccl commit: 83fd4f70e7]
2022-04-18 16:04:04 -07:00
Wenkai Du 67e7e6507e Merge remote-tracking branch 'nccl/master' into develop
[ROCm/rccl commit: d28e1cb44f]
2022-04-18 11:15:25 -07:00
Wenkai Du 58b2c1ec9c Fix random segfault (#537)
[ROCm/rccl commit: fd2f1b3b88]
2022-04-15 14:32:11 -07:00
Wenkai Du 011447e4dc Add new Rome model (#536)
[ROCm/rccl commit: 2151c79d14]
2022-04-13 11:45:40 -07:00
Wenkai Du f8023f2e07 Add new Rome model (#535)
[ROCm/rccl commit: ba4c165bf3]
2022-04-12 13:27:32 -07:00
Sylvain Jeaugey e89ff21d35 Update Makefile to install static library.
Make sure make install also installs the static library. 
Fixes #662

[ROCm/rccl commit: 9bfc1c6e35]
2022-04-08 14:00:43 +02:00
Wenkai Du 5ccdd9f5e1 Increase chunk steps of broadcast and reduce (#528)
[ROCm/rccl commit: 15b572751e]
2022-04-07 13:34:04 -07:00
Colin Smith 3830310156 Doxygen fix for ncclRecv (#527)
Changed the Doxygen command for ncclRecv and pncclRecv, to be consistent with other APIs.

[ROCm/rccl commit: b2ffcf6d89]
2022-04-05 14:07:56 -07:00
Wenkai Du 9884e61367 Add tuning model (#523)
[ROCm/rccl commit: 5cc0a405c0]
2022-04-04 10:19:57 -07:00
Wenkai Du 3332cdff07 Support multiple tuning tables (#522)
* Support multiple tuning tables

* [UnitTests] Skip managed memory testing

[ROCm/rccl commit: bbe780ca6c]
2022-03-31 17:09:21 -07:00
Sylvain Jeaugey 74f8baa0f3 Merge remote-tracking branch 'origin/master'
[ROCm/rccl commit: 8133784b32]
2022-03-30 02:29:05 -07:00
Sylvain Jeaugey 27130280b2 2.12.10-1
Fix bug with CollNet
Fix bug with zero-bytes send/recv operations
Fix NCCL_PARAM implementation to avoid taking a lock on every call
Fix bug when setting NCCL_IB_QPS_PER_CONNECTION to more than one.
Improve error reporting for network errors.


[ROCm/rccl commit: 353e8ba446]
2022-03-30 02:27:01 -07:00
Sylvain Jeaugey a52e328ba4 Fix merging error
[ROCm/rccl commit: 2247152a8e]
2022-03-30 02:14:32 -07:00
Sylvain Jeaugey 3bc2e34df2 Merge branch 'master' into truncated_msg_warning
[ROCm/rccl commit: 2dfd83752c]
2022-03-30 10:58:05 +02:00