AI/rocm-systems
2
0
Fork 0
Koodi Ongelmat Pull-pyynnöt Actions Paketit Projektit Julkaisut Wiki Toiminta
501 Commitit 3 Branchit 0 Tagit
54396499366bbd6fbc798ba8f41c2a8e59b49e1e
Commit Graph

201 Commitit

Tekijä SHA1 Viesti Päivämäärä
Wenkai Du 1aae6b1344 Fix incorrect pointer checking for scatter and gather (#285) 
[ROCm/rccl commit: dcad0ef7cb]
 2020-10-19 13:27:09 -07:00
Wenkai Du 194135a40c Merge remote-tracking branch 'nccl/master' into nccl_sync 
[ROCm/rccl commit: c835d8263a]
 2020-10-15 18:42:38 -04:00
gilbertlee-amd 94437eef28 Revert "Initial support for clique-based kernels (#276)" (#280) 
This reverts commit d68a532bc6.

[ROCm/rccl commit: 84a2541e01]
 2020-10-15 11:30:18 -07:00
Sylvain Jeaugey 591ffd32fe Fix affinity move 
[ROCm/rccl commit: 0e14394c5f]
 2020-10-13 16:58:05 -07:00
Sylvain Jeaugey 5de6b6681d Make sure proxy threads inherit the CPU affinity. 
[ROCm/rccl commit: c6dbdb0084]
 2020-10-13 16:37:52 -07:00
Wenkai Du 8b120c0508 Update Rome single node models (#277) 
[ROCm/rccl commit: 33babcb5e2]
 2020-10-13 13:33:09 -07:00
gilbertlee-amd d68a532bc6 Initial support for clique-based kernels (#276) 
* Initial support for clique-based kernels

[ROCm/rccl commit: 2b8184808d]
 2020-10-13 11:22:04 -06:00
Wenkai Du 41260bb948 Rework Rome detection and add multiple network ports models (#274) 
* Rework Rome detection and add multiple network ports models

* Remove unused opCount in p2p transport

[ROCm/rccl commit: ae008fd2db]
 2020-10-07 13:37:36 -07:00
Wenkai Du dbde26e681 Add Alltoallv RCCL kernel implementation (#269) 
* Add alltoallv API and implementation

* Extend Rome P2P channel limit to multinode and alltoall kernels

* topo_expl: fix compilation and sync up with main

* gtest: use RCCL alltoallv API

* Code review changes

[ROCm/rccl commit: b871ea3c0c]
 2020-09-30 16:25:36 -07:00
Stanley Tsang 67a8d86d78 Updating inline asm to not require explicit L1 cache invalidation (#270) 
[ROCm/rccl commit: acca2ae20a]
 2020-09-25 13:46:26 -06:00
gilbertlee-amd 0a9adc16f4 Changes to topology based on XGMI (#272) 
* Alterations to topology search to improve XGMI-enabled nodes

[ROCm/rccl commit: 01bd2573db]
 2020-09-25 12:20:09 -06:00
Wenkai Du 7ba087e069 Ensure all ranks on same send/receive or alltoall kernel path (#271) 
[ROCm/rccl commit: 44fcde7835]
 2020-09-24 08:25:04 -07:00
Wenkai Du 37f7eec6b7 Change network plugin name to librccl-net.so (#266) 
[ROCm/rccl commit: d871fceb54]
 2020-09-18 13:23:30 -07:00
Wenkai Du f0a303664e Limit P2P channels on Rome 
[ROCm/rccl commit: 42955f5f4f]
 2020-09-17 17:20:32 -07:00
Wenkai Du a3402d6aeb Merge pull request #262 from wenkaidu/alignment 
Make data alignment requirements matching ISA manual

[ROCm/rccl commit: 60819dcf8d]
 2020-09-08 10:40:42 -07:00
Wenkai Du 09639a5d54 Fix broken profiling build (#263) 
[ROCm/rccl commit: e2042ccf8a]
 2020-09-02 15:39:52 -07:00
Wenkai Du cfa1228504 Make data alignment requirements matching ISA manual 
From https://developer.amd.com/wp-content/resources/Vega_Shader_ISA.pdf

8.1.7. Alignment
For Dword or larger reads or writes, the two LSBs of the byte-address
are ignored, thus forcing Dword alignment.


[ROCm/rccl commit: 4751992231]
 2020-09-01 21:21:58 +00:00
Wenkai Du 778ab61097 Fix incorrect threads split in sendrecv (#261) 
[ROCm/rccl commit: 4180e6409e]
 2020-08-31 17:33:22 -07:00
Wenkai Du 03bb6bcb54 Increase minimal channels for gfx908 (#259) 
[ROCm/rccl commit: c5cbece6d0]
 2020-08-26 11:40:11 -07:00
Wenkai Du 0898fea746 Only use software barrier for synchronization (#258) 
[ROCm/rccl commit: b0919dc46c]
 2020-08-25 13:16:34 -07:00
Wenkai Du 5f49a0e088 Add NPS4 support on some models (#256) 
* Add NPS4 support on some models

* Add XML models

[ROCm/rccl commit: 391bbf3f1e]
 2020-08-19 11:03:20 -07:00
Wenkai Du 3d5fb8142e Add another Rome model (#249) 
* Add another Rome model

* Add gfx908 4P3L models and support

* Revert "Use cached value for detecting GDR support only once"

This reverts commit 0108a1219d.

* Skip using ibverb for GPU direct RDMA detection

* Fine tune one Rome model

[ROCm/rccl commit: a51e4071e3]
 2020-08-17 10:51:02 -07:00
Wenkai Du f242a2f0b0 Collect gcnArch and hipDeviceArch_t in XML (#252) 
[ROCm/rccl commit: 7e3d8a31cc]
 2020-08-12 15:48:38 -07:00
Wenkai Du e5ec2d94d5 Merge pull request #248 from wenkaidu/2.7.8 
2.7.8

[ROCm/rccl commit: 066223333d]
 2020-08-11 08:20:37 -07:00
Wenkai Du 14ad6ff3b4 Merge remote-tracking branch 'nccl/master' into 2.7.8 
[ROCm/rccl commit: 7e3f841fab]
 2020-08-10 16:11:00 +00:00
Wenkai Du c9815aaa36 Add more Rome 4P2H models 
[ROCm/rccl commit: 09ef75656a]
 2020-08-06 18:20:02 +00:00
Jack Snyder dca19952fd Setting type when gpu sub node is discovered 
[ROCm/rccl commit: de49a77074]
 2020-08-05 13:39:23 -07:00
Eric Badger d6a78cb1c7 Don't require NIC devices to have specific PCI class 
If a PCI node is the parent of a NIC, treat it as such, regardless of
the PCI class code for the device. This allows non-traditional devices
to act as NICs via the net plugin mechanism.

For consistency, treat GPUs similarly.


[ROCm/rccl commit: 700c0e0f24]
 2020-08-05 12:46:29 -07:00
Wenkai Du 5f96f13e75 Allow setup ring through NCCL_RINGS to facilitate testing 
[ROCm/rccl commit: 5b03132ace]
 2020-08-04 21:07:00 +00:00
Wenkai Du 22a6211eaf Improve 4P2H topology on Rome (#243) 
1. Use bi-directional rings
2. GPU search is sorted by PCI device ID to get consistent results

[ROCm/rccl commit: d1e20b4c5e]
 2020-07-28 14:21:44 -07:00
David Addison 4acaffebb3 2.7.8-1 
Fix collective mismatch error when using ncclSend/ncclRecv


[ROCm/rccl commit: 033d799524]
 2020-07-27 16:34:09 -07:00
Wenkai Du 487f93b83f Topology tuning for 4P2H on Rome (#242) 
* Topology tuning for 4P2H on Rome

* Use ncclTopoIdToIndex

[ROCm/rccl commit: e7a10aa0e4]
 2020-07-27 11:53:57 -07:00
Wenkai Du 49c667ac8a ib-test: support multiple channels (#241) 
[ROCm/rccl commit: 8d5fb920b6]
 2020-07-27 11:03:12 -07:00
Wenkai Du f604fc774e Add 8P6L multi-node models (#239) 
[ROCm/rccl commit: d5f90e19b5]
 2020-07-21 14:10:36 -07:00
Wenkai Du 3e2c9054cd Change default channels duplication for chordal ring (#233) 
[ROCm/rccl commit: ab787c767e]
 2020-07-14 15:16:50 -07:00
Wenkai Du 47dfed5f01 Revert "Split primitive class to smaller structures" (#230) 
This reverts commit 622b49e80a.

[ROCm/rccl commit: 5215130168]
 2020-07-08 11:06:50 -07:00
Riatre Foo e791f8c8c7 Fix build action order 
Add $(INCTARGETS) to build dependencies of %.o and $(DEVICELIB).
As there were no dep files during the first build, Make may kick off source
compilation before nccl.h got generated, which leads to occasional build
failures on systems with high core count. The build failure could be
reproduced reliably with a `sleep 5` in $(INCDIR)/nccl.h rule.


[ROCm/rccl commit: 2d8601701d]
 2020-07-07 10:20:51 -07:00
Wenkai Du c7805f224e Merge remote-tracking branch 'nccl/master' into develop 
[ROCm/rccl commit: da3b197d6c]
 2020-07-01 16:51:25 -07:00
Wenkai Du 77d94493c3 Merge sendrecv kernel from NCCL 2.7.3 
This commit was cherry-picked and modified from
https://github.com/NVIDIA/nccl/commit/89072c82e551d7c8bf7bd5ca77780ef540b7611e


[ROCm/rccl commit: 964c4c2061]
 2020-06-29 08:47:46 -07:00
Wenkai Du 3c674c9c1f Use separate threads for send and receive 
[ROCm/rccl commit: b90735c935]
 2020-06-29 08:47:15 -07:00
Sylvain Jeaugey 5432eb8e9d 2.7.6-1 
Fix crash when NVswitch is not visible inside a VM.


[ROCm/rccl commit: 1952325569]
 2020-06-26 16:35:54 -07:00
Sylvain Jeaugey 1f6b75ca8b 2.7.5-1 
Minor fixes for A100 platforms.
Add a WARN for invalid GroupEnd call.


[ROCm/rccl commit: 01afd20a77]
 2020-06-26 14:39:49 -07:00
Wenkai Du 117f0f6e6e Revert use posix_memalign for network buffer allocation on host memory (#222) 
[ROCm/rccl commit: 84f8ba3bb0]
 2020-06-24 11:25:55 -07:00
Wenkai Du e7d664200f Use posix_memalign for network buffer allocation on host memory (#221) 
* Use posix_memalign for network buffer allocation on host memory

* ib-test: add ability to specify run iterations

* ib-test: define iterations as multiple of default cycles

* Add checking to posix_memalign return value

[ROCm/rccl commit: 0eb19a563a]
 2020-06-22 13:06:25 -07:00
Stanley Tsang dafe176570 Documentation updates for NCCL 2.7.0 (#219) 
* Making hip-clang the default compiler; documentation update

* Adding back --hip-clang to install.sh as a silent option for CI

* Documentation updates for NCCL 2.7

* Restoring deleted line in install script

[ROCm/rccl commit: 8d21adb5e3]
 2020-06-16 16:48:11 -06:00
Wenkai Du 9ca3176346 Add IB/RDMA unit test 
[ROCm/rccl commit: cfa97eccd3]
 2020-06-16 18:29:17 +00:00
Wenkai Du 5d93f83ae3 Limit network profiling support to simple protocol and avoid overflow 
[ROCm/rccl commit: 95b8f70d15]
 2020-06-15 20:51:36 +00:00
Wenkai Du 3389838def Rework network proxy profiling 
[ROCm/rccl commit: 7484e53ff7]
 2020-06-13 03:13:58 +00:00
Wenkai Du a8952356d1 Reduce RCCL kernel count as we don't pass first coll in argument 
[ROCm/rccl commit: b257676f30]
 2020-06-12 21:30:04 +00:00
Wenkai Du a90fac8579 Sender rank's opCount maybe ahead by one if it finishes earlier 
[ROCm/rccl commit: a6d621176c]
 2020-06-12 03:39:45 +00:00