3389838def
Rework network proxy profiling
...
[ROCm/rccl commit: 7484e53ff7
2020-06-13 03:13:58 +00:00
a8952356d1
Reduce RCCL kernel count as we don't pass first coll in argument
...
[ROCm/rccl commit: b257676f30
2020-06-12 21:30:04 +00:00
a90fac8579
Sender rank's opCount maybe ahead by one if it finishes earlier
...
[ROCm/rccl commit: a6d621176c
2020-06-12 03:39:45 +00:00
d1f8fdc3a8
gtest: add scatter, gather and all to all unit tests
...
[ROCm/rccl commit: fee1a20b74
2020-06-09 17:44:15 -07:00
e7dd92dc87
Log NUMA node of RDMA host buffer allocation
...
[ROCm/rccl commit: e98891d039
2020-06-09 17:44:15 -07:00
846e1a4427
Add network proxy profiling support
...
[ROCm/rccl commit: 812543104d
2020-06-09 17:44:15 -07:00
bc6840f937
Calculate and use total wait cycles for RCCL profiling
...
[ROCm/rccl commit: c9aa11928a
2020-06-09 17:44:15 -07:00
15cd2aff3c
Add gather, scatter and alltoall collectives
...
Introducing 3 new APIs:
ncclResult_t ncclGather(const void* sendbuff, void* recvbuff, size_t sendcount,
ncclDataType_t datatype, int root, ncclComm_t comm, hipStream_t stream);
ncclResult_t ncclScatter(const void* sendbuff, void* recvbuff,
size_t recvcount, ncclDataType_t datatype, int root, ncclComm_t comm,
hipStream_t stream);
ncclResult_t ncclAllToAll(const void* sendbuff, void* recvbuff, size_t count,
ncclDataType_t datatype, ncclComm_t comm, hipStream_t stream);
Only out of place operation is supported.
Preprocessor symbol RCCL_GATHER_SCATTER=1 indicates API availibility.
By default the APIs launche RCCL kernel implementation, which can be disabled by
RCCL_ALLTOALL_KERNEL_DISABLE=1. Then the APIs use wrapper around ncclSend and ncclRecv.
[ROCm/rccl commit: e80e29573c
2020-06-09 17:44:08 -07:00
69eb70ce43
tpol_expl: update to 2.7
...
[ROCm/rccl commit: 71ec3e09df
2020-06-09 17:40:24 -07:00
67fb4f6cdb
Merge remote-tracking branch 'nccl/master' into develop
...
[ROCm/rccl commit: 26a0fd2517
2020-06-09 17:40:11 -07:00
89072c82e5
2.7.3-1
...
Add support for A100 GPU and related platforms.
Add support for CUDA 11.
Add support for send/receive operations (beta).
[ROCm/rccl commit: 5949d96f36
2020-06-08 09:31:44 -07:00
3077902de6
Fixing CI as install.sh script should not install dependencies without user request ( #217 )
...
[ROCm/rccl commit: 87db65f22d
2020-06-05 11:04:03 -06:00
8e43b854f1
Making hip-clang the default compiler; documentation update ( #216 )
...
* Making hip-clang the default compiler; documentation update
* Adding back --hip-clang to install.sh as a silent option for CI
[ROCm/rccl commit: dc403e0ca2
2020-06-04 11:58:27 -06:00
0108a1219d
Use cached value for detecting GDR support only once
...
[ROCm/rccl commit: 67c8e72ce3
2020-05-22 17:19:10 +00:00
b449a3b36d
Report HIP version in logs
...
[ROCm/rccl commit: e41ab173cf
2020-05-20 18:15:32 +00:00
e7eff47be4
Revert "Tuning the inline and unroll to reduce the scratch usage"
...
This reverts commit d8a06589c9ca493a6b51
2020-05-15 14:15:40 -07:00
5780637fff
Merge pull request #209 from wenkaidu/hip-clang
...
Rename files which only diffs in extension
[ROCm/rccl commit: c245f1507e
2020-05-15 13:51:12 -07:00
61a82ff572
Merge pull request #208 from wenkaidu/perf_xgmi
...
Give preference to path with more XGMI connections
[ROCm/rccl commit: 706de76046
2020-05-15 10:07:22 -07:00
c2afb1f4ca
Rename files which only diffs in extension
...
[ROCm/rccl commit: e7b36304c8
2020-05-15 09:16:32 -07:00
27519fd019
Give preference to path with more XGMI connections
...
[ROCm/rccl commit: b3c9852634
2020-05-14 15:33:16 -07:00
ced9958094
rccl-prim-test: add flags when calling hipExtLaunchMultiKernelMultiDevice in hip-clang
...
[ROCm/rccl commit: f1058b6353
2020-05-12 23:54:07 +00:00
abba9d5504
Restoring doxygen documentation to nccl.h.in.
...
[ROCm/rccl commit: 787ac13486
2020-05-12 22:03:31 +00:00
e35e4d3401
Updating README and readthedocs documentation.
...
[ROCm/rccl commit: b59b9d328b
2020-05-12 20:11:49 +00:00
5ba51914f1
Update rccl_bfloat16.h to match rocBLAS
...
[ROCm/rccl commit: d5a07a7b5c
2020-05-08 22:48:07 +00:00
069322d05a
Set flags when calling hipExtLaunchMultiKernelMultiDevice in hip-clang
...
[ROCm/rccl commit: 24ea2ef6dd
2020-05-08 15:57:14 +00:00
65ad48404b
Merge remote-tracking branch 'upstream/master' into develop
...
[ROCm/rccl commit: 33c23fdcda
2020-04-29 16:12:37 -07:00
75863c97a4
Refactoring packaging ( #193 )
...
[ROCm/rccl commit: 308e96877e
2020-04-29 16:24:21 -06:00
2badecfa20
Merge pull request #199 from wenkaidu/para_jobs
...
Enable parallel jobs for hip-clang build
[ROCm/rccl commit: 914b6ca27c
2020-04-29 13:49:48 -07:00
fa25190ddb
Junit test storage call corrected ( #197 )
...
* Focus testing on Centos for now
* storing junit
* Reducing test suite to Ubuntu
[ROCm/rccl commit: 65390f9872
2020-04-29 14:22:03 -06:00
52096f2cb6
Enable parallel jobs for hip-clang build
...
[ROCm/rccl commit: 3f471ab5b1
2020-04-29 17:58:16 +00:00
c0c0e92ef5
Adding NCCL_DEBUG=INFO Logging to CI ( #196 )
...
[ROCm/rccl commit: 6b1d70b03b
2020-04-27 15:12:15 -06:00
779ee97ada
topo_expl: fix build error
...
[ROCm/rccl commit: 5743c6b7d2
2020-04-27 17:17:05 +00:00
9813d67cd1
Merge remote-tracking branch 'nccl/master' into HEAD
...
[ROCm/rccl commit: c4edc257b0
2020-04-27 17:16:54 +00:00
5f57e6b466
Merge pull request #194 from wenkaidu/search
...
Fix incorrect next device ID in PCI ordered search
[ROCm/rccl commit: cf5070f6c0
2020-04-27 09:54:09 -07:00
7b7f781658
Fix incorrect next device ID in PCI ordered search
...
[ROCm/rccl commit: edb49ed2d5
2020-04-25 01:01:13 +00:00
b9acac2db6
Enabling CI Testing Again ( #192 )
...
Adding CI support based on AMD internal CI refactor.
[ROCm/rccl commit: cc66dd46e9
2020-04-24 10:36:57 -06:00
eebc6f2844
Adding option to re-use streams instead of re-creating per topology
...
[ROCm/rccl commit: 339bf9ff19
2020-04-23 15:53:40 +00:00
c43022b9d8
Fix crash when only a subset of GPUs are visible within a container.
...
Fixes #326 .
[ROCm/rccl commit: f36540f55a
2020-04-17 10:03:14 -07:00
5df2502deb
Improve robustness of PCI detection
...
Fallback to default values when class/speed is unknown.
[ROCm/rccl commit: 23a9fbb788
2020-04-16 14:27:50 -07:00
acd868dfcb
Fix wrong variable name "slice" to "chunk"
...
https://github.com/NVIDIA/nccl/issues/287
[ROCm/rccl commit: a783484ab5
2020-04-14 19:00:51 -07:00
728cf9ee10
Revert "Temporary disable 0x803 target due to build error"
...
This reverts commit 8b1ce44c2a5170bd1c02
2020-04-14 16:58:41 +00:00
2de0b24c30
rccl-prim-test: auto-detect rings in 4P and 8P configurations
...
[ROCm/rccl commit: ef7064ba9b
2020-04-10 18:17:21 +00:00
627e1a06d0
Fix bug #307 : wrong NIC selection on the reduction tree.
...
The reduction tree (tree up) was inverting the NICs to use,
causing performance issue in cases where we are using different
NICs on a given channel.
[ROCm/rccl commit: b5b6c6acdd
2020-04-09 17:14:07 -07:00
bfbfe370c3
Fix HIP-Clang build with HSA headers
...
HIP-Clang does not include these HSA headers, and they need to be explicitly added in RCCL.
[ROCm/rccl commit: a95090d981
2020-04-03 17:58:23 -04:00
8852e54181
topo_expl: update to 2.6
...
[ROCm/rccl commit: 6f54b23503
2020-04-01 13:37:08 -07:00
4aeb7f041e
Merge remote-tracking branch 'nccl/master' into v2.6.4_merge
...
[ROCm/rccl commit: fa36fd9ef9
2020-04-01 13:35:12 -07:00
b996c2ca00
Merge pull request #314 from NVIDIA/v2.6
...
2.6.4-1
[ROCm/rccl commit: 533e3702cf
2020-03-26 17:31:24 -07:00
40adc74496
2.6.4-1
...
Add support for network collectives.
Add support for XML topology dump/injection.
Add text values for GDR and P2P Levels, including "NVL".
Add speed detection for PCI, Infiniband and Ethernet cards.
Add CPU detection for ARM and AMD CPUs.
Add support for adaptive routing on Infiniband.
Change NET plugin API to v3 : merge PCI path and GPU pointer
capability into a single structure and add other properties.
[ROCm/rccl commit: b221128eca
2020-03-20 14:58:36 -07:00
38b445c94f
Check return code for Flush operation
...
Current NCCL code does not abort for failed Flush operations by
underlying network. This may compromise data integrity.
Signed-off-by: Rashika Kheria <rashika@amazon.com >
[ROCm/rccl commit: 6c61492eba
2020-03-16 20:40:59 -07:00
e3e1c6b29c
rccl-prim-test: add all-to-all benchmark ( #185 )
...
For gfx908, support simple detection of ring topology.
Call ReduceOrCopyMulti directly from kernel.
Also simplify code by removing kernel start synchronization option
which has no effect on throughput measurements.
[ROCm/rccl commit: ebc823e603
2020-03-16 10:00:54 -07:00
Wenkai Du
Sylvain Jeaugey
saadrahim
Stanley Tsang
Wenkai Du
Saad Rahim
Gilbert Lee
aokomoriuta
Aaron Enye Shi
Rashika Kheria