نمودار کامیت

598 کامیت‌ها

مولف SHA1 پیام تاریخ
Wenkai Du a4ea1fed5b Merge remote-tracking branch 'nccl/master' into develop 2021-05-05 16:01:01 -07:00
gilbertlee-amd 4f8e788a61 Fixing potential race-condition in env var parameter macro (#359) 2021-04-28 12:04:41 -06:00
saadrahim 96782191cf Expanding CI coverage for 8GPU configurations plus extended tests (#350) 2021-04-27 09:57:00 -06:00
Wenkai Du ad54a14a5c Add libdl linking option (#358) 2021-04-26 15:24:58 -07:00
Wenkai Du ed237dcaa7 Use better name for kernel collective trace enable (#357)
"NCCL_DEBUG=INFO NCCL_DEBUG_SUBSYS=INIT,COLL" enables collectives API
trace. Adding "RCCL_KERNEL_COLL_TRACE_ENABLE=1" enables kernel traces.
2021-04-26 08:35:53 -07:00
Wenkai Du 9cc9c3360b Control collective trace from kernel separately (#356) 2021-04-23 16:36:19 -07:00
Stanley Tsang 70597789d0 Message queue refactor to POSIX implementation and leak fix (#355)
* Fixing message queue leak.

* Using POSIX implementation of Message Queues

* Adding unlink to msgqueue

* MsgQueue update

* Adding timeout check to msgqueue broadcast; tightening up system checks

* Removing unnecessary code

* Removing extra argument from print

* Adding explicit msg queue close call to all other ranks
2021-04-23 11:33:20 -06:00
Wenkai Du 415c7cd3d1 Tune number of channels for gfx90a (#349) 2021-04-19 15:27:01 -07:00
Wenkai Du 9c718ce6d6 Use correct WARP_SIZE for gfx1030 (#348) 2021-04-14 14:09:52 -07:00
Wenkai Du a79f74082e Limit max channels for ring graph on single node Rome (#347)
* Limit max channels for ring graph on single node Rome
* Partially revert "Use non-temporal access for streaming data (#341)"
2021-04-14 10:14:54 -07:00
Wenkai Du 1fe031402a Add gfx90a target (#344)
* Add gfx90a target

* Support gfx90a topology

Co-authored-by: Eiden Yoshida <eiden.yoshida@amd.com>
2021-04-14 09:29:00 -06:00
Sylvain Jeaugey a46ea10583 2.9.6-1
Add support for CUDA graphs.
Fuse BCM Gen4 switches to avoid suboptimal performance on some platforms. Issue #439.
Fix bootstrap issue caused by connection reordering.
Fix CPU locking block.
Improve CollNet algorithm.
Improve performance on DGX A100 for communicators with only one GPU per node.
2021-04-12 16:00:46 -07:00
Wenkai Du 3f18540f50 Remove link to NUMA lib as it is no longer needed (#346) 2021-04-12 09:53:17 -07:00
TomSang 87f12cbb86 Add detection of cooperative multi device launch attribute (#345) 2021-04-11 13:29:24 -07:00
Wenkai Du def8b4ca0d Move RCCL changelog and Copyright out of /usr/share (#343) 2021-04-09 14:08:40 -07:00
Wenkai Du 9dfc2c183e Use non-temporal access for streaming data (#341)
* Use non-temporal access for streaming data

* Revert to ulong2 after fixing compiling issue
2021-04-07 17:34:35 -07:00
gilbertlee-amd caba0a63d2 Fixing clique-topology detection (#342)
* Fixing clique-topology detection
* Fix to enable multi-process clique-based kernels
2021-04-07 11:29:44 -06:00
Wenkai Du e26ad2995e Cleanup number of channels calculation (#340) 2021-04-05 17:51:56 -07:00
Wenkai Du 17491c918e Fix incorrect net counting (#339)
* Fix incorrect net counting

* Add comments
2021-04-05 12:21:57 -07:00
Wenkai Du 1d2946ee4b Rework network port trimming code (#338)
* Rework network port trimming code

* Move Rome related changes to separate source files
2021-03-31 10:25:59 -07:00
Wenkai Du 0c78553ee0 Check fine grained memory on peer GPU before enabling P2P (#337) 2021-03-30 09:06:39 -07:00
Wenkai Du d87dc7c2e8 collnet: support multiple NICs (#335) 2021-03-25 20:59:32 -07:00
gilbertlee-amd 69c0b44553 Updating CHANGELOG.md for ROCm 4.1.0 (#336) 2021-03-25 21:44:24 -06:00
Stanley Tsang 289db2a636 Fixing message queue leak. (#331) 2021-03-25 19:11:43 -06:00
Wenkai Du 0fbb9510a5 Remove HDP workaround for ROCm 4.2 HIP (#334) 2021-03-23 20:11:37 -07:00
Wenkai Du 1d6244b18d Enable collnet in RCCL (#333)
* Enable CollNet and use different number of channels

* topo_expl: enable collnet
2021-03-19 12:58:13 -07:00
Wenkai Du 244e25d980 Hide non-public symbols from library (#332)
* Hide non-public symbols from library

* Move flag outside of parallel-jobs check
2021-03-18 18:08:08 -07:00
Wenkai Du b46260260a Sort GPUs by HIP device ID (#329)
* Sort GPUs by HIP device ID

* Remove extra space
2021-03-16 16:51:32 -07:00
Wenkai Du f60b76c67a Add GPU memory usage tracker (#326) 2021-03-06 20:32:30 -08:00
Wenkai Du 8e180cf087 Revert "Port alltoall[v]" (#325)
This reverts commit f4d5d3d620.
2021-03-06 13:59:31 -08:00
Wenkai Du c018edf0f2 Enable local sendrecv over network if GDR is available on all GPUs (#324) 2021-03-05 19:59:41 -08:00
gilbertlee-amd f4a9b9acba Adding pthread_join / pthread_detach to clean up pthreads to avoid leaks (#322) 2021-02-26 16:29:55 -07:00
Wenkai Du 7191d0959b Merge pull request #320 from wenkaidu/nbio
Match NBIO only when GPUs and NICs are directly connected to CPU
2021-02-25 17:49:09 -08:00
Stanley Tsang 5d3b1fa4d0 Temporarily disabling multiprocess unit tests 2021-02-25 23:59:44 +00:00
Eiden Yoshida 25a7dad765 Add ubuntu gfx908 target to CI 2021-02-24 19:01:56 -07:00
Wenkai Du e820a943e9 Update tuning parameters for XGMI and NET 2021-02-23 21:41:26 +00:00
Wenkai Du ec8d89b1dd Match NBIO only when GPUs and NICs are directly connected to CPU 2021-02-22 18:52:29 -05:00
Stanley Tsang 61d66c8e18 Do not search for GTest if we're force installing it into build directory (#318) 2021-02-22 15:22:01 -07:00
Wenkai Du a058d5639e Merge pull request #319 from wenkaidu/2.8.3
Update to 2.8.4
2021-02-22 10:17:38 -08:00
Stanley Tsang 45f5255f7c Fixing cache deletion for CliqueManager; updating copyright 2021-02-19 22:22:46 +00:00
Stanley Tsang 6a568a8652 Make CI only run subset of unit tests. 2021-02-19 21:37:51 +00:00
Cory Bloor 8aea5edb29 Add Jenkins docs build (#18)
* Fix typo in copyright

* Minor README improvements

- Prevent underscores from being interpreted as italics in test name format.
- Switch URL to HTTPS.

* Update docs scripts config

- Allow run_doc.sh and run_doxygen.sh to be called from any directory.

* Add docs build to Jenkins
2021-02-18 16:37:37 -07:00
Wenkai Du 95f178324c Add support to another Rome model 2021-02-18 02:00:31 +00:00
Wenkai Du c985358e11 Merge remote-tracking branch 'nccl/master' into 2.8.3 2021-02-15 18:44:47 -05:00
Wenkai Du 3a1aebd742 Merge remote-tracking branch 'rccl/develop' into 2.8.3 2021-02-15 13:17:38 -05:00
Wenkai Du bf8eb40705 Move HDP flush to CPU 2021-02-12 18:06:19 +00:00
pramenku e9f7908592 Update install.sh (#317)
* Update install.sh

Install.sh having hard code like /opt/rocm/bin/hipcc for rocm_path and default_path=/opt/rocm
This will work only when we have standalone rocm installed. If anyone has installed, side-by-side, they will face below error.

Can we keep like ROCM_PATH=$ROCM_PATH  instead of “default_path” as variable name and 
ROCM_BIN_PATH=$ROCM_PATH/bin ,rocm_path can be replaced with ROCM_BIN_PATH.

This way, we will have option to export ROCM_PATH as env variable as per need and use the script. 
I have also tried locally, it’s working.  ROCM_PATH is common variable name, we are having.

If you are ok, I can also submit the PR for the same.


Error when side-by-side install is done for driver.
# ./install.sh -dtr 2>&1 | tee /dockerx/6519_rccl-test.log
CMake Error at /usr/share/cmake/Modules/CMakeDetermineCXXCompiler.cmake:48 (message):
Could not find compiler set in environment variable CXX:
/opt/rocm/bin/hipcc.
Call Stack (most recent call first):
CMakeLists.txt:12 (project)

CMake Error: CMAKE_CXX_COMPILER not set, after EnableLanguage
-- Configuring incomplete, errors occurred!
See also "/root/driver/rccl/build/release/CMakeFiles/CMakeOutput.log".

* Update install.sh

Removed ROCM_PATH=$ROCM_PATH

* Update install.sh

Set default value if external value is not supplied.
2021-02-12 08:44:30 -08:00
Stanley Tsang 6b7b312fb9 Fixed temp file creation/deletion with clique mode (#316) 2021-02-12 08:44:10 -08:00
Sylvain Jeaugey 911d61f214 2.8.4-1
Fix hang in corner cases of alltoallv using point to point send/recv.
Harmonize error messages.
Fix missing NVTX section in the license.
Update README.
2021-02-09 15:36:48 -08:00
Gilbert Lee f1a9ce3fa5 Using GTEST_SKIP() to skip unit tests that have insufficient devices. Skipping out earlier 2021-02-09 03:54:04 +00:00