6dfdfef98f
Add gfx908 Rome 4 NICs model
2021-02-06 00:19:47 +00:00
f372c53d52
[TransferBench] Fixing some merge issues
2021-02-05 16:46:20 +00:00
ab1e7a0318
Merge remote-tracking branch 'origin/develop' into 2.8.3
2021-02-04 20:02:34 -05:00
2f541508c5
[topo_expl] Updating for 2.8.3
2021-02-04 19:08:42 +00:00
9aac1ed38f
[ib-test] Update for 2.8.3]
2021-02-04 19:05:03 +00:00
9ce203dd0a
[TransferBench] Updating for 2.8.3
2021-02-04 18:58:25 +00:00
1990ffd76a
Tuning some clique-based kernel parameters ( #315 )
2021-02-03 20:00:08 -07:00
5f97122442
Enable GPU direct RDMA read from GPU
2021-02-03 02:48:30 +00:00
62e0447e9a
[TransferBench] Restore some previous fixes - memory leak, PCIe address ( #314 )
2021-02-01 09:48:09 -07:00
01a998b17c
Removing in-place tests from Combined calls (no support for send/recv)
2021-01-28 20:09:03 +00:00
3e62ceddc5
Clique kernel support ( #295 ) ( #15 )
...
* Adding experimental clique-based kernels (opt-in only)
Co-authored-by: Stanley Tsang <stanley.tsang@amd.com >
Co-authored-by: Gilbert Lee <gilbert.lee@amd.com >
Co-authored-by: Wenkai Du <43822138+wenkaidu@users.noreply.github.com >
Co-authored-by: Stanley Tsang <stanley.tsang@amd.com >
Co-authored-by: Wenkai Du <43822138+wenkaidu@users.noreply.github.com >
2021-01-28 09:45:01 -07:00
41e47a36e7
Use less unroll for clique kernels ( #313 )
2021-01-15 17:48:10 -08:00
d3fa257682
Adding multiprocess unit tests ( #312 )
...
Adding multiprocess unit tests for collectives.
To run, NCCL_COMM_ID=$HOSTNAME:12345 build/release/test/UnitTestsMultiProcess
2021-01-15 16:34:36 -07:00
2ddbe6646b
Improve collective trace
2021-01-14 19:28:01 -05:00
b33a2cac8b
gtest: add scatter to combined calls and use loops ( #303 )
...
* gtest: add scatter to combined calls and use loops
* gtest: run validation inside loop
* gtest: revert small element count to 2520
* gtest: fix memory leak in validation
(cherry picked from commit b0853ccd51
2021-01-14 19:28:01 -05:00
f4d5d3d620
Port alltoall[v]
2021-01-14 19:28:01 -05:00
105db19a11
Do not allow GPU as intermediate
2021-01-14 19:28:01 -05:00
e055229e56
Revert "Changes to topology based on XGMI ( #272 )"
...
This reverts commit 01bd2573db
2021-01-14 19:28:01 -05:00
d469947641
Merge remote-tracking branch 'nccl/master' into no-target-id
2021-01-14 19:27:53 -05:00
373a108516
Fix Rome PCIe 2 node topology generation ( #310 )
2020-12-15 17:16:17 -08:00
41c35dad48
[TransferBench] Fixing bug with fine-grained memory allocation ( #311 )
...
* Fixing bug with fine-grained memory
2020-12-15 17:37:31 -07:00
ae0c4092c7
[TransferBench] Adding ability to perform CPU-executed copies, various upgrades ( #309 )
...
* Adding CPU based execution, fixing typos, adding Fine-grained mem
* Exposing sampling factor when generating range of data sizes
* Refactoring how Links are launched, now once per thread
* Documentation updates
2020-12-11 10:21:14 -07:00
b80ae551b1
[TransferBench] Support multiple of 4 byte sizes, changing default GPU timing mechanism ( #307 )
...
* Changing default timing mechanism, adjusting CPU bandwidth calc, adding flag to use combined timing
* Adding support for smaller transfers (byte size must be multiple of 4 instead of 128)
2020-12-04 14:57:13 -07:00
882d52ad7e
Adding backward compatibility for target-id syntax for AMDGPU_TARGETS ( #306 )
2020-12-04 13:55:56 -08:00
975b14dffa
Add Rome model and improve search ( #305 )
2020-11-17 14:55:06 -08:00
920dbe5b35
2.8.3-1
...
Optimization for Tree allreduce on A100.
Improve aggregation performance.
Use shared buffers for inter-node send/recv.
Add NVTX profiling hooks.
Accelerate alltoall connections by merging communication for all
channels.
Add support for one hop communication through NVLink, for faster
send/recv communication on cubemesh topologies like DGX-1.
Improve alltoall scheduling to better balance intra/inter node
communication.
Increase send/recv parallelism by 8x, each warp sending or
receiving to a different peer.
Net: move to v4.
Net: make flush operation asynchronous to accelerate alltoall.
Net: define maximum number of requests.
Fix hang when using LL128 protocol after 2^31 steps.
Fix #379 : topology injection failing when using less GPUs than
described in the XML.
Fix #394 : protocol mismatch causing hangs or crashes when using
one GPU per node.
2020-11-17 11:08:52 -08:00
1943bac646
Merge remote-tracking branch 'origin/master' into develop
2020-11-16 12:16:53 -05:00
554729079d
Use device's link width and speed if port doesn't report ( #304 )
2020-11-13 17:58:04 -08:00
b0853ccd51
gtest: add scatter to combined calls and use loops ( #303 )
...
* gtest: add scatter to combined calls and use loops
* gtest: run validation inside loop
* gtest: revert small element count to 2520
* gtest: fix memory leak in validation
2020-11-13 17:57:44 -08:00
2958f7eace
Fixing IPC handle leak ( #302 )
2020-11-13 10:32:42 -07:00
c8d08a7c2f
Adding RCCL_CLIQUE_DEBUG to help debug experimental clique feature ( #300 )
2020-11-13 09:07:11 -07:00
4e68229c8b
Skip unused peer connection in scatter and gather ( #301 )
2020-11-12 15:47:34 -08:00
377b43470b
Merge pull request #299 from ROCmSoftwarePlatform/develop
...
Enable target id build
2020-11-10 15:47:42 -07:00
41bcfb8878
Clique kernel support ( #295 )
...
* Adding experimental clique-based kernels (opt-in only)
Co-authored-by: Stanley Tsang <stanley.tsang@amd.com >
Co-authored-by: Gilbert Lee <gilbert.lee@amd.com >
Co-authored-by: Wenkai Du <43822138+wenkaidu@users.noreply.github.com >
2020-11-10 15:44:10 -07:00
1fdb216f87
Use target id of xnack off ( #298 )
2020-11-10 11:10:48 -08:00
2e8b3a0857
Use ncclSend/ncclRecv for alltoall type of collectives as default ( #297 )
2020-11-09 11:23:17 -08:00
bdd8adf1ca
Adding a CHANGELOG ( #296 )
2020-11-05 13:38:30 -07:00
709b7e4880
Improve GPU direct RDMA handling on Rome ( #294 )
2020-11-03 14:29:08 -08:00
dfa3c41ede
Add more Rome models ( #292 )
2020-10-30 21:26:04 -07:00
bfab1d3592
Adding output to CSV, removing OpenMP, decreasing default numBytes to 64MB, adding aggregate stats ( #290 )
2020-10-27 09:00:33 -06:00
2ecfc62ec8
Fix lintian errors ( #287 )
2020-10-21 16:20:53 -07:00
61e1a71d14
[TransferBench] Displaying PCIe Bus ID ( #288 )
...
* Adding PCIe BusID per GPU in topology display
2020-10-21 16:13:36 -06:00
769418c5c7
TransferBench Typo. Pinned host memory uses C not P ( #286 )
2020-10-21 12:05:38 -06:00
084207e685
fix proxyArgs for trace log
2020-10-21 09:18:40 -07:00
e8177c9ee7
Adding sles15, centos7 and centos8 testing ( #283 )
2020-10-20 09:39:03 -06:00
dcad0ef7cb
Fix incorrect pointer checking for scatter and gather ( #285 )
2020-10-19 13:27:09 -07:00
9b3f762b68
Removing unnecessary flags from CI ( #278 )
...
* Removing unnecessary flags from CI
* Re-adding HSA_FORCE_FINE_GRAIN_PCIE in CI
2020-10-19 13:08:24 -06:00
49aa6d7afe
Updating copyright for documentation ( #282 )
2020-10-19 13:07:15 -06:00
a7deecb104
Merge pull request #279 from wenkaidu/nccl_sync
...
Sync up with latest NCCL master branch
2020-10-16 11:21:35 -07:00
205b5507b4
Update sramecc and xnack to ANY ( #284 )
...
Co-authored-by: Tony <Tony.Tye@amd.com >
Co-authored-by: Wenkai Du<Wenkai.Du@amd.com >
2020-10-16 00:25:18 -06:00
Wenkai Du
Gilbert Lee
gilbertlee-amd
Wenkai Du
Stanley Tsang
Sylvain Jeaugey
Colin Smith
xietingwew
saadrahim
Eiden Yoshida