saadrahim
0dc019e35f
Download GTest if not found in system ( #237 )
...
Co-authored-by: Stanley Tsang <stanley.tsang@amd.com >
2020-08-06 09:36:58 -06:00
Wenkai Du
d1e20b4c5e
Improve 4P2H topology on Rome ( #243 )
...
1. Use bi-directional rings
2. GPU search is sorted by PCI device ID to get consistent results
2020-07-28 14:21:44 -07:00
Wenkai Du
e7a10aa0e4
Topology tuning for 4P2H on Rome ( #242 )
...
* Topology tuning for 4P2H on Rome
* Use ncclTopoIdToIndex
2020-07-27 11:53:57 -07:00
Wenkai Du
8d5fb920b6
ib-test: support multiple channels ( #241 )
2020-07-27 11:03:12 -07:00
Sourav Chakraborty
fe3d520601
Merge pull request #240 from ROCmSoftwarePlatform/sourav/topo-expl-1
...
simplify model definitions in topo expl
2020-07-22 12:35:17 -05:00
Sourav Chakraborty
2475daafee
add 4 node 8P6L 1 NIC 2nd Hive model
2020-07-22 16:27:15 +00:00
Sourav Chakraborty
db55afb014
simplify model definitions in topo expl
2020-07-22 16:05:53 +00:00
Wenkai Du
d5f90e19b5
Add 8P6L multi-node models ( #239 )
2020-07-21 14:10:36 -07:00
Stanley Tsang
684f3e6af4
Adding better naming to unit tests for filtering; adding short and full unit test suites ( #235 )
2020-07-21 12:19:47 -06:00
Wenkai Du
35c5a7fe45
Fix RCCL build package name ( #236 )
2020-07-20 14:43:00 -07:00
saadrahim
99a491273f
Changing GTest inclusion in cmake to use find_package ( #234 )
...
* GTest is used via find_package. No longer downloaded in cmake.
* Adding error handling
2020-07-15 20:51:48 -06:00
saadrahim
7f93aa7e53
Changing dependency to hip-rocclr ( #228 )
2020-07-14 17:49:56 -06:00
Wenkai Du
ab787c767e
Change default channels duplication for chordal ring ( #233 )
2020-07-14 15:16:50 -07:00
gilbertlee-amd
f87ba17737
Removing UnitTest as install, removing unused env var ( #231 )
2020-07-10 09:30:28 -06:00
Wenkai Du
5215130168
Revert "Split primitive class to smaller structures" ( #230 )
...
This reverts commit 486fd436af .
2020-07-08 11:06:50 -07:00
Wenkai Du
1addf4f196
Match RCCL package name to API version ( #229 )
2020-07-07 13:30:39 -07:00
Stanley Tsang
9bd4c14603
Adding appropriate references in rccl-prim-test ( #227 )
...
Adding appropriate references to rccl-prim-test.
2020-07-06 10:15:03 -06:00
Wenkai Du
ecae1cd76a
Merge pull request #226 from wenkaidu/develop
...
Sync up to NCCL 2.7.6
2020-07-06 09:10:09 -07:00
Wenkai Du
da3b197d6c
Merge remote-tracking branch 'nccl/master' into develop
2020-07-01 16:51:25 -07:00
Wenkai Du
d3548cc474
topo_expl: each rank needs to have its own memory for graphs ( #225 )
2020-07-01 15:11:02 -07:00
Wenkai Du
a6be82f5ab
topo_expl: fix broken build ( #224 )
2020-06-30 11:11:23 -07:00
Wenkai Du
a144a85465
Merge pull request #223 from wenkaidu/sendrecv
...
Use separate threads for send and receive
2020-06-30 10:50:06 -07:00
Wenkai Du
8db0aa8f4c
gtest: extend testing up to 8 GPUs
2020-06-29 09:32:31 -07:00
Wenkai Du
964c4c2061
Merge sendrecv kernel from NCCL 2.7.3
...
This commit was cherry-picked and modified from
https://github.com/NVIDIA/nccl/commit/5949d96f36d050e59d05872f8bbffd2549318e95
2020-06-29 08:47:46 -07:00
Wenkai Du
b90735c935
Use separate threads for send and receive
2020-06-29 08:47:15 -07:00
Sylvain Jeaugey
1952325569
2.7.6-1
...
Fix crash when NVswitch is not visible inside a VM.
2020-06-26 16:35:54 -07:00
Sylvain Jeaugey
01afd20a77
2.7.5-1
...
Minor fixes for A100 platforms.
Add a WARN for invalid GroupEnd call.
2020-06-26 14:39:49 -07:00
Wenkai Du
84f8ba3bb0
Revert use posix_memalign for network buffer allocation on host memory ( #222 )
2020-06-24 11:25:55 -07:00
Wenkai Du
0eb19a563a
Use posix_memalign for network buffer allocation on host memory ( #221 )
...
* Use posix_memalign for network buffer allocation on host memory
* ib-test: add ability to specify run iterations
* ib-test: define iterations as multiple of default cycles
* Add checking to posix_memalign return value
2020-06-22 13:06:25 -07:00
Wenkai Du
dc739c4e70
ib-test: support host memory allocation through posix_memalign ( #220 )
...
* ib-test: support host memory allocation through posix_memalign
* ib-test: add missing CUDACHECK to hip calls
2020-06-17 16:16:54 -07:00
Stanley Tsang
8d21adb5e3
Documentation updates for NCCL 2.7.0 ( #219 )
...
* Making hip-clang the default compiler; documentation update
* Adding back --hip-clang to install.sh as a silent option for CI
* Documentation updates for NCCL 2.7
* Restoring deleted line in install script
2020-06-16 16:48:11 -06:00
Wenkai Du
0023b9b081
Merge pull request #218 from wenkaidu/develop
...
Develop
2020-06-16 13:30:50 -07:00
Wenkai Du
cfa97eccd3
Add IB/RDMA unit test
2020-06-16 18:29:17 +00:00
Wenkai Du
95b8f70d15
Limit network profiling support to simple protocol and avoid overflow
2020-06-15 20:51:36 +00:00
Wenkai Du
7484e53ff7
Rework network proxy profiling
2020-06-13 03:13:58 +00:00
Wenkai Du
b257676f30
Reduce RCCL kernel count as we don't pass first coll in argument
2020-06-12 21:30:04 +00:00
Wenkai Du
a6d621176c
Sender rank's opCount maybe ahead by one if it finishes earlier
2020-06-12 03:39:45 +00:00
Wenkai Du
fee1a20b74
gtest: add scatter, gather and all to all unit tests
2020-06-09 17:44:15 -07:00
Wenkai Du
e98891d039
Log NUMA node of RDMA host buffer allocation
2020-06-09 17:44:15 -07:00
Wenkai Du
812543104d
Add network proxy profiling support
2020-06-09 17:44:15 -07:00
Wenkai Du
c9aa11928a
Calculate and use total wait cycles for RCCL profiling
2020-06-09 17:44:15 -07:00
Wenkai Du
e80e29573c
Add gather, scatter and alltoall collectives
...
Introducing 3 new APIs:
ncclResult_t ncclGather(const void* sendbuff, void* recvbuff, size_t sendcount,
ncclDataType_t datatype, int root, ncclComm_t comm, hipStream_t stream);
ncclResult_t ncclScatter(const void* sendbuff, void* recvbuff,
size_t recvcount, ncclDataType_t datatype, int root, ncclComm_t comm,
hipStream_t stream);
ncclResult_t ncclAllToAll(const void* sendbuff, void* recvbuff, size_t count,
ncclDataType_t datatype, ncclComm_t comm, hipStream_t stream);
Only out of place operation is supported.
Preprocessor symbol RCCL_GATHER_SCATTER=1 indicates API availibility.
By default the APIs launche RCCL kernel implementation, which can be disabled by
RCCL_ALLTOALL_KERNEL_DISABLE=1. Then the APIs use wrapper around ncclSend and ncclRecv.
2020-06-09 17:44:08 -07:00
Wenkai Du
71ec3e09df
tpol_expl: update to 2.7
2020-06-09 17:40:24 -07:00
Wenkai Du
26a0fd2517
Merge remote-tracking branch 'nccl/master' into develop
2020-06-09 17:40:11 -07:00
Sylvain Jeaugey
5949d96f36
2.7.3-1
...
Add support for A100 GPU and related platforms.
Add support for CUDA 11.
Add support for send/receive operations (beta).
2020-06-08 09:31:44 -07:00
saadrahim
87db65f22d
Fixing CI as install.sh script should not install dependencies without user request ( #217 )
2020-06-05 11:04:03 -06:00
Stanley Tsang
dc403e0ca2
Making hip-clang the default compiler; documentation update ( #216 )
...
* Making hip-clang the default compiler; documentation update
* Adding back --hip-clang to install.sh as a silent option for CI
2020-06-04 11:58:27 -06:00
Wenkai Du
2a4514772c
Merge pull request #214 from wenkaidu/gdr
...
Use cached value for detecting GDR support only once
2020-05-22 13:36:23 -07:00
Wenkai Du
67c8e72ce3
Use cached value for detecting GDR support only once
2020-05-22 17:19:10 +00:00
Wenkai Du
957be85944
Merge pull request #212 from wenkaidu/version
...
Report HIP version in logs
2020-05-20 16:25:54 -07:00