f4bf47f325
NPKit: improve clock calibration and fix GPU clock API ( #683 )
...
* Improve clock calibration in NPKit
* Improve gfx macro
* Fix macro
2023-02-17 12:26:57 -07:00
f7a456122c
Remove workaround and use indirect function call ( #684 )
2023-02-14 13:59:48 -08:00
39534e8724
Add HIP event optimization and remove special code for gfx90a
2023-02-10 16:46:01 +00:00
e1cb45ff22
Merge remote-tracking branch 'nccl/master' into HEAD
2023-02-04 01:44:43 +00:00
2288e9ae80
Switch to hipLaunchHostFunc for HIP graph ( #667 )
2022-12-15 10:16:46 -08:00
adafc0f759
Add MSCCL Support ( #658 )
...
* Add MSCCL support
* Add alignment and message size checking
* Fix nRanks checking, in-place and out-of-place tests and group call handling
* Fix hipGraph unit test
* Change MSCCL init warning to INFO
* Revise license info
2022-12-12 15:51:04 -08:00
b953544a59
Fix typo in detecting Intel platforms ( #661 )
2022-12-07 13:36:11 -08:00
eca623df07
decreased warp size for gfx110x ( #655 )
2022-12-01 12:19:21 -07:00
fb9938cffa
Query DMABuf support through HSA runtime API ( #654 )
2022-11-30 08:53:03 -08:00
28189e2df8
2.16.2-1
...
Add support for CUDA 12.0, drop Kepler (sm_35).
Support for H100 features.
Make socket code more robust and protected. Solves #555 .
Improve performance on large CUDA graphs, reducing dependencies.
Reduce inter-socket bandwidth on AMD CPUs to favor better paths.
Various fixes to ncclCommAbort.
Make service thread polling resistant to EINTR.
Compile with profiling API by default.
Extend NVTX instrumentation with call arguments.
2022-11-30 02:31:59 -08:00
9594bbee3b
Adjust P2P channels on Intel platform ( #653 )
2022-11-29 13:57:10 -08:00
57764f8152
Fix incorrect rocm-smi ID conversion ( #648 )
2022-11-21 19:44:39 -08:00
9cb72a3d0f
Fix collective trace timestamp format ( #647 )
2022-11-21 08:11:12 -08:00
cf3c32a626
Fix typo in previous hipify change ( #645 )
2022-11-15 11:51:47 -08:00
562dd87036
Move hipify to cmake stage
...
Add minimal ROCm/HIP version requirements for Graph support
2022-11-14 18:10:45 +00:00
94ad7f6f51
Update tuning table and fix topo_expl
2022-11-07 18:24:24 +00:00
9a077e6947
Merge remote-tracking branch 'nccl/master' into develop
2022-11-03 21:17:42 +00:00
72ef100050
Fix P2P scheduling
2022-10-31 08:54:34 -07:00
2f4cb874ba
Merge tag 'v2.15.5-1'
2022-10-25 01:15:22 -07:00
cb111f764a
2.15.5-1
...
Fix crash with CollnetChain on some node topologies
Fix hang when interleaving the capture of different graphs
Fix hang during init in multi-threaded mode
Fix potential data corruption with LL128 protocol on unaligned buffers.
Fix CPU usage during preconnect
Fixes double-free in the error path for ncclCommInitAll
Workaround hang on H100 with Ring/LL128 on 2 GPUs.
2022-10-25 00:55:55 -07:00
4f0e223db4
Merge remote-tracking branch 'nccl/master' into develop
2022-10-20 15:41:29 +00:00
bc8ef779df
Fix missing initialization due to merge error ( #640 )
2022-10-19 21:20:11 -07:00
9ddf0e0649
Support P2P with invisible devices ( #636 )
...
* Support P2P with invisible devices
* Update copyright year
2022-10-17 10:24:59 -07:00
9916a09818
Merge pull request #634 from yzygitzh/ziyyang/npkit-fix
...
Apply several fixes to NPKit
2022-10-17 08:01:24 -07:00
ebb8b5bf63
Updating files for missing licenses ( #637 )
2022-10-14 13:49:16 -06:00
7d6bbc19d4
apply npkit
2022-10-14 01:28:17 +00:00
d128d62238
Merge tag 'v2.15.1-1'
2022-10-07 11:00:26 -07:00
2401f4a918
Fixes a double-free in the error path of ncclCommInitAll.
...
Fixes https://github.com/NVIDIA/nccl/issues/726
2022-10-03 17:12:32 -07:00
e645b02cd8
introduce a hw topology aware bintree
...
for hayabusa architecture.
2022-10-03 15:26:21 +00:00
ef71550738
Added new gpu targets ( #631 )
2022-09-29 14:53:55 -06:00
a523b37ac7
Another threadfence and flags rework ( #629 )
2022-09-28 16:49:29 -07:00
021932b3c8
Add LL128 tuning ( #630 )
2022-09-27 09:39:09 -07:00
da8152e57a
2.15.1-1
...
Add support for H100 (sm90).
Make sure NCCL kernel honor user stream priorities.
2022-09-27 02:31:13 -07:00
99c28f2e75
Merge remote-tracking branch 'origin/master'
2022-09-27 02:24:41 -07:00
78313a6d21
Use compatibility shim only with static cudart
...
Closes issue 658
2022-09-27 02:22:48 -07:00
e8af0716c4
Merge pull request #619 from whchung/exp_reduce_code_size
...
Only use split tree algorithm to reduce kernel code size.
2022-09-26 10:06:27 -05:00
ecab28a7c9
Fix potential deadlock during init in multi-thread mode.
...
Make sure all calls calling cudaMalloc (including devCommSetup) are
called before the last bootstrapBarrier. That way, we avoid calls to
cudaMalloc be blocked by a NCCL kernel launched on another GPU by
another thread which completed init faster.
Resolve #623 .
2022-09-26 02:13:10 -07:00
35f1fe3434
Merge pull request #621 from whchung/exp_reduce_sleep_cycles
...
Reduce s_sleep cycles
2022-09-23 15:31:16 -05:00
a80cc7e6e1
Merge pull request #624 from whchung/exp_tweak_unroll_factors
...
Tweak unroll factors.
2022-09-23 11:30:05 -05:00
84054c3b30
Tweak unroll factors.
2022-09-22 13:03:04 -05:00
02929cffb6
Only use split tree algorithm to reduce kernel code size.
2022-09-22 12:01:53 -05:00
a3c8ef8c03
Reduce s_sleep cycles
2022-09-22 12:01:12 -05:00
b9ae02d4ad
Abolish syncthreads and only wait on LDS traffic.
2022-09-22 12:00:37 -05:00
49c811ecf9
Rework threadfence and flag setting ( #627 )
2022-09-22 08:35:42 -07:00
d9216af48b
Revert changes to gfx1030 ( #622 )
2022-09-20 20:06:17 -07:00
9e6c87a2bf
Define ncclShmem as global shared ( #618 )
...
* Use global defined shared memory
* Add --hipcc-func-supp to compile option
* Force inline some device functions
* Add back threadfence
2022-09-20 09:00:20 -07:00
f89fd4777d
address review comments
2022-09-20 11:58:33 +02:00
79fb0326ac
Fix intermittent 11.6 builds: generate unique .cu file for each object file
2022-09-20 11:58:33 +02:00
8f3219dbd4
make binary tree work on 2.13.4
2022-09-15 00:01:54 +00:00
a06e14e39b
Misc fixes and disable binTree
2022-09-14 00:26:19 +00:00
Ziyue Yang
Wenkai Du
Wenkai Du
akolliasAMD
Sylvain Jeaugey
gilbertlee-amd
John Bachan
Edgar Gabriel
Cliff Woolley
Wen-Heng (Jack) Chung
Wen-Heng (Jack) Chung
Jane Xu