373a108516
Fix Rome PCIe 2 node topology generation ( #310 )
2020-12-15 17:16:17 -08:00
975b14dffa
Add Rome model and improve search ( #305 )
2020-11-17 14:55:06 -08:00
554729079d
Use device's link width and speed if port doesn't report ( #304 )
2020-11-13 17:58:04 -08:00
709b7e4880
Improve GPU direct RDMA handling on Rome ( #294 )
2020-11-03 14:29:08 -08:00
dfa3c41ede
Add more Rome models ( #292 )
2020-10-30 21:26:04 -07:00
33babcb5e2
Update Rome single node models ( #277 )
2020-10-13 13:33:09 -07:00
ae008fd2db
Rework Rome detection and add multiple network ports models ( #274 )
...
* Rework Rome detection and add multiple network ports models
* Remove unused opCount in p2p transport
2020-10-07 13:37:36 -07:00
b871ea3c0c
Add Alltoallv RCCL kernel implementation ( #269 )
...
* Add alltoallv API and implementation
* Extend Rome P2P channel limit to multinode and alltoall kernels
* topo_expl: fix compilation and sync up with main
* gtest: use RCCL alltoallv API
* Code review changes
2020-09-30 16:25:36 -07:00
01bd2573db
Changes to topology based on XGMI ( #272 )
...
* Alterations to topology search to improve XGMI-enabled nodes
2020-09-25 12:20:09 -06:00
42955f5f4f
Limit P2P channels on Rome
2020-09-17 17:20:32 -07:00
c5cbece6d0
Increase minimal channels for gfx908 ( #259 )
2020-08-26 11:40:11 -07:00
391bbf3f1e
Add NPS4 support on some models ( #256 )
...
* Add NPS4 support on some models
* Add XML models
2020-08-19 11:03:20 -07:00
a51e4071e3
Add another Rome model ( #249 )
...
* Add another Rome model
* Add gfx908 4P3L models and support
* Revert "Use cached value for detecting GDR support only once"
This reverts commit 67c8e72ce3
2020-08-17 10:51:02 -07:00
7e3d8a31cc
Collect gcnArch and hipDeviceArch_t in XML ( #252 )
2020-08-12 15:48:38 -07:00
066223333d
Merge pull request #248 from wenkaidu/2.7.8
...
2.7.8
2020-08-11 08:20:37 -07:00
7e3f841fab
Merge remote-tracking branch 'nccl/master' into 2.7.8
2020-08-10 16:11:00 +00:00
09ef75656a
Add more Rome 4P2H models
2020-08-06 18:20:02 +00:00
de49a77074
Setting type when gpu sub node is discovered
2020-08-05 13:39:23 -07:00
700c0e0f24
Don't require NIC devices to have specific PCI class
...
If a PCI node is the parent of a NIC, treat it as such, regardless of
the PCI class code for the device. This allows non-traditional devices
to act as NICs via the net plugin mechanism.
For consistency, treat GPUs similarly.
2020-08-05 12:46:29 -07:00
5b03132ace
Allow setup ring through NCCL_RINGS to facilitate testing
2020-08-04 21:07:00 +00:00
d1e20b4c5e
Improve 4P2H topology on Rome ( #243 )
...
1. Use bi-directional rings
2. GPU search is sorted by PCI device ID to get consistent results
2020-07-28 14:21:44 -07:00
e7a10aa0e4
Topology tuning for 4P2H on Rome ( #242 )
...
* Topology tuning for 4P2H on Rome
* Use ncclTopoIdToIndex
2020-07-27 11:53:57 -07:00
d5f90e19b5
Add 8P6L multi-node models ( #239 )
2020-07-21 14:10:36 -07:00
ab787c767e
Change default channels duplication for chordal ring ( #233 )
2020-07-14 15:16:50 -07:00
da3b197d6c
Merge remote-tracking branch 'nccl/master' into develop
2020-07-01 16:51:25 -07:00
1952325569
2.7.6-1
...
Fix crash when NVswitch is not visible inside a VM.
2020-06-26 16:35:54 -07:00
01afd20a77
2.7.5-1
...
Minor fixes for A100 platforms.
Add a WARN for invalid GroupEnd call.
2020-06-26 14:39:49 -07:00
0eb19a563a
Use posix_memalign for network buffer allocation on host memory ( #221 )
...
* Use posix_memalign for network buffer allocation on host memory
* ib-test: add ability to specify run iterations
* ib-test: define iterations as multiple of default cycles
* Add checking to posix_memalign return value
2020-06-22 13:06:25 -07:00
26a0fd2517
Merge remote-tracking branch 'nccl/master' into develop
2020-06-09 17:40:11 -07:00
5949d96f36
2.7.3-1
...
Add support for A100 GPU and related platforms.
Add support for CUDA 11.
Add support for send/receive operations (beta).
2020-06-08 09:31:44 -07:00
b3c9852634
Give preference to path with more XGMI connections
2020-05-14 15:33:16 -07:00
c4edc257b0
Merge remote-tracking branch 'nccl/master' into HEAD
2020-04-27 17:16:54 +00:00
edb49ed2d5
Fix incorrect next device ID in PCI ordered search
2020-04-25 01:01:13 +00:00
f36540f55a
Fix crash when only a subset of GPUs are visible within a container.
...
Fixes #326 .
2020-04-17 10:03:14 -07:00
23a9fbb788
Improve robustness of PCI detection
...
Fallback to default values when class/speed is unknown.
2020-04-16 14:27:50 -07:00
b5b6c6acdd
Fix bug #307 : wrong NIC selection on the reduction tree.
...
The reduction tree (tree up) was inverting the NICs to use,
causing performance issue in cases where we are using different
NICs on a given channel.
2020-04-09 17:14:07 -07:00
a95090d981
Fix HIP-Clang build with HSA headers
...
HIP-Clang does not include these HSA headers, and they need to be explicitly added in RCCL.
2020-04-03 17:58:23 -04:00
6f54b23503
topo_expl: update to 2.6
2020-04-01 13:37:08 -07:00
fa36fd9ef9
Merge remote-tracking branch 'nccl/master' into v2.6.4_merge
2020-04-01 13:35:12 -07:00
b221128eca
2.6.4-1
...
Add support for network collectives.
Add support for XML topology dump/injection.
Add text values for GDR and P2P Levels, including "NVL".
Add speed detection for PCI, Infiniband and Ethernet cards.
Add CPU detection for ARM and AMD CPUs.
Add support for adaptive routing on Infiniband.
Change NET plugin API to v3 : merge PCI path and GPU pointer
capability into a single structure and add other properties.
2020-03-20 14:58:36 -07:00
62dc28bd2e
Remove condition for ring duplication
...
Fix insufficent number of rings on single node after pull #179
2020-03-02 12:55:06 -08:00
8b5bc8bca2
Merge pull request #179 from wenkaidu/search
...
Use fraction of system maxWidth as steps for searching
2020-02-28 11:05:46 -08:00
8e73a2ad60
Merge remote-tracking branch 'remotes/nccl/master'
2020-02-27 12:53:03 -08:00
d2adc61bf6
Revise PCI BW numbers on Rome
2020-02-26 13:17:49 -08:00
8391637613
Use fraction of system maxWidth as steps for searching
...
This reverts previous workaround of deducting only half of width
from paths.
2020-02-26 09:14:35 -08:00
9b80b3633f
Fix system maxSpeed and maxWidth calculation
2020-02-24 15:18:57 -08:00
f54dc58113
Fix incorrect CR8 detection
...
Also change level of ring graph print to help debugging
2020-02-21 10:09:49 -08:00
5b3856f2ed
Merge pull request #172 from wenkaidu/topo_expl
...
Add topology explorer
2020-02-20 15:16:55 -08:00
55f8e2dec7
Add topology explorer
2020-02-19 14:42:06 -08:00
abcfbf1231
Generate 8G6L chordal ring from reference
2020-02-11 22:01:12 +00:00
Wenkai Du
gilbertlee-amd
Wenkai Du
Jack Snyder
Eric Badger
Sylvain Jeaugey
Aaron Enye Shi