AI/rocm-systems
2
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
810 Commits 3 Branches 0 Tags
eef812bed7fbd3bd4f31de48bde6a41e1dfc9242
Commit Graph

810 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Wenkai Du eef812bed7 Revert chunksteps changes (#555) 2022-05-31 14:45:51 -07:00
Wenkai Du ef499c4810 Add another Rome model (#553) 
* Add another Rome model

* Add option to force enable intranet on single node

* Limit p2p channels to number of ranks

* Refine p2p channels handling
 2022-05-31 11:31:30 -07:00
akolliasAMD a0a686e74c code cleanup (#554) 2022-05-31 09:59:36 -04:00
Wenkai Du c5b77121f0 Update Rome model (#552) 2022-05-26 09:59:23 -07:00
akolliasAMD 98f0809a39 Added creation of new tree and added switch for using treesplit for specific cases (#551) 2022-05-25 18:55:14 -04:00
gilbertlee-amd 700b473211 Moving opt-in custom signal handler from UnitTests into RCCL (#550) 
* Enable via RCCL_ENABLE_SIGNALHANDLER=1
 2022-05-20 09:56:38 -06:00
Wenkai Du 6707a270b1 Add switch for pivot alltoall kernel (#549) 2022-05-17 18:14:04 -07:00
Wenkai Du 283dc86a73 Refine and add new Rome models (#548) 2022-05-17 08:23:59 -07:00
Wenkai Du c9919e0e35 Improve LL performance (#546) 
* Improve LL performance

* Add split barriers for LL
 2022-05-10 13:32:10 -07:00
Edgar Gabriel 46b30c5f9b Merge pull request #544 from edgargabriel/topic/header-file-include 
fix cmake logic to handle old and new include dirs
 2022-04-28 16:29:08 -05:00
Edgar 4c4a7cb696 fix cmake logic to handle old and new include dirs 
Starting from rocm 5.2 there is a reorganization of the
include directories. This pr allows to compile
rccl on both the old and the new directory layout.
This solution is using find_package() for identifying correct
settings for rocm_smi starting from rocm-5.2, and the original (manual)
settings for all previous releases.

Tested with rocm-5.2, 5.1.1, 5.0.2, and 4.5.2.
 2022-04-28 14:33:46 -04:00
gilbertlee-amd 685bcea127 [TransferBench] Syncing with TransferBench v1.02 (#541) 2022-04-27 20:43:24 -06:00
Wenkai Du 063da25563 topo_expl: fix build and add tuning support (#539) 2022-04-26 15:40:07 -07:00
Wenkai Du 379940dfac Merge pull request #533 from ROCmSoftwarePlatform/2.12.10 
Sync up with NCCL 2.12.10
 2022-04-26 10:09:37 -07:00
Edgar Gabriel 39e3002e19 Merge pull request #530 from edgargabriel/topic/signal-intercept 
Topic/signal intercept
 2022-04-25 10:44:26 -05:00
Edgar 2bf6d254b6 add a signal handler and backtrace 
Tweak the signal handler and force non-release build
Increase ulimit locked memory value
Update the singal handler to use bfd symbol resolution.
Include configure logic to find bfd functions.
Add optionally c++ function name demangling
 2022-04-25 10:48:17 -04:00
Wenkai Du 83fd4f70e7 Update tuning parameters 2022-04-18 16:04:04 -07:00
Wenkai Du d28e1cb44f Merge remote-tracking branch 'nccl/master' into develop 2022-04-18 11:15:25 -07:00
Wenkai Du fd2f1b3b88 Fix random segfault (#537) 2022-04-15 14:32:11 -07:00
Wenkai Du 2151c79d14 Add new Rome model (#536) 2022-04-13 11:45:40 -07:00
Wenkai Du ba4c165bf3 Add new Rome model (#535) 2022-04-12 13:27:32 -07:00
nunnikri b83efe9c5c Installing rccl.h wrapper to /opt/rocm-xxx/include path (#532) 
* Fixing the broken library soft link

* Installing rccl.h wrapper to /opt/rocm-xxx/include path.

This missing wrapper was causing compilation errors with pytorch. Fixing it
 2022-04-09 07:55:39 -07:00
gilbertlee-amd def6832287 Transfer bench single stream mode (#531) 
- Adding single stream mode
- Removing some unused env vars
- Adding output to CSV mode for p2p benchmark, topology listing modes
 2022-04-08 15:20:55 -06:00
Sylvain Jeaugey 9bfc1c6e35 Update Makefile to install static library. 
Make sure make install also installs the static library. 
Fixes #662
 2022-04-08 14:00:43 +02:00
nunnikri acfb0210ea Fixing the broken library soft link (#529) 2022-04-07 15:19:33 -07:00
Wenkai Du 15b572751e Increase chunk steps of broadcast and reduce (#528) 2022-04-07 13:34:04 -07:00
Colin Smith b2ffcf6d89 Doxygen fix for ncclRecv (#527) 
Changed the Doxygen command for ncclRecv and pncclRecv, to be consistent with other APIs.
 2022-04-05 14:07:56 -07:00
Wenkai Du 5cc0a405c0 Add tuning model (#523) 2022-04-04 10:19:57 -07:00
Wenkai Du bbe780ca6c Support multiple tuning tables (#522) 
* Support multiple tuning tables

* [UnitTests] Skip managed memory testing
 2022-03-31 17:09:21 -07:00
Sylvain Jeaugey 8133784b32 Merge remote-tracking branch 'origin/master' 2022-03-30 02:29:05 -07:00
Sylvain Jeaugey 353e8ba446 2.12.10-1 
Fix bug with CollNet
Fix bug with zero-bytes send/recv operations
Fix NCCL_PARAM implementation to avoid taking a lock on every call
Fix bug when setting NCCL_IB_QPS_PER_CONNECTION to more than one.
Improve error reporting for network errors.
 2022-03-30 02:27:01 -07:00
Sylvain Jeaugey 2247152a8e Fix merging error 2022-03-30 02:14:32 -07:00
Sylvain Jeaugey 2dfd83752c Merge branch 'master' into truncated_msg_warning 2022-03-30 10:58:05 +02:00
Ke Wen 1382a87306 Display host name instead of numeric IP when referring to a peer 
For easier interpretation of debug messages like "connection closed by
peer", "peer message truncated" and "peer collective mismatch"
 2022-03-30 10:47:10 +02:00
Christopher Hesse b895abcdb8 Fix typo in net_ib.cc 2022-03-30 10:45:01 +02:00
Felix Abecassis 1c7c014ceb Remove unnecessary newline in plugin logging 
Signed-off-by: Felix Abecassis <fabecassis@nvidia.com>
 2022-03-30 10:44:49 +02:00
Wenkai Du 7cbbca4da1 Update tuning parameters (#518) 
* Update tuning parameters

* Respect user algo and topo selections
 2022-03-29 08:15:37 -07:00
gilbertlee-amd 2d558c9abc Adding explicit request for coarse-grained host memory due to changes in HipHostMalloc (#517) 2022-03-25 13:05:07 -06:00
Liam Wrubleski a8f1e61f48 Packages for test and benchmark executables on all supported OSes using CPack. (#512) 2022-03-21 15:04:14 -06:00
Wenkai Du cd17cf6dce Update Rome model matching and add new models (#516) 
* Update Rome model matching and add new models

* Add missing file

* Models update
 2022-03-21 10:54:40 -07:00
akolliasAMD 65ea3d80db Added alltoallv test and optional args variable on collective args (#514) 
* Added alltoallv test and optional args variable on collective args
 2022-03-18 13:55:11 -04:00
John Bachan 44eb40da0e Add pthread_detach()'s for threads we never pthread_join(). Helps 
reduce diagnostic noise for ThreadSanitizer.

Fixes https://github.com/NVIDIA/nccl/issues/649
 2022-03-15 10:27:59 -07:00
nunnikri a04da71647 Merge pull request #511 from nunnikri/develop 
File reorganization as per the new defined standard
 2022-03-10 08:39:29 -08:00
Nirmal Unnikrishnan 115461cc04 File reorganization with backward compatibility 
Updated the header file location and export path
 2022-03-10 01:28:41 +00:00
Nirmal Unnikrishnan 676a4737c1 File reorganization as per the new defined standard 
The header files will in /opt/rocm-xxx/include/rccl
Libraries and cmake will be in /opt/rocm-xxx/lib folder.
Added wrappers for header files using rocm-cmake functions for backward compatibility.
 2022-03-08 17:32:02 +00:00
gilbertlee-amd 0687940b84 Changing initialization method for UnitTests (#510) 2022-03-07 09:22:55 -07:00
Wenkai Du d6d6af710e Force ring algorithm on single node (#509) 2022-03-04 10:29:02 -08:00
gilbertlee-amd b634b2f1c2 Adding NCCL_DEBUG=INFO for CI runs (#508) 2022-03-03 18:04:28 -07:00
gilbertlee-amd 699dc30f05 [UnitTests] Check process mask for custom tests (#507) 2022-03-02 17:24:14 -07:00
akolliasAMD ff54e79799 Added Unit test for nccl send recv (#506) 
Added Send Receive test that tests through all pairs
 2022-03-02 15:50:16 -05:00