d2fbcfea02
Misc fixes and improvements for 2.5.6
...
1. Fix RCCL unit test
2. Add ROME detection and tuning
3. Change default P2P level
4. Fix search algorithm for XGMI
5. Remove explicit channel duplication with implicit by using half of link speed
6. Add collective trace support
7. Correct Intel Skylake CPU detection and bandwidth
8. Fix topo connect function
9. Disable GDR read and remove unreachable code
10. Disable LL128 kernels
11. Add tuning parameters
12. Use original clock64() implementation which returns RTC counter value
13. Print out timestamp of collective trace
14. Do not use struct ncclColl in kernel launch parameter
15. Fix abort handling and add tracing
17. Add __launch_bounds__ to kernel functions
18. Remove unused abortCount
19. Unset default MIN_NRINGS and MIN_NCHANNELS
20. Do not allocate shared memory when not using LL128 kernels
21. Correct time print out in tuning log
[ROCm/rccl commit: 1e55645d97
2020-01-29 15:27:05 -08:00
d7d4175df0
Merge remote-tracking branch 'remotes/nccl/master' into rccl_2.5.6
...
[ROCm/rccl commit: 6648c81dc6
2019-12-03 15:42:04 -08:00
4cb52294d5
Disable direct buffers to reduce scratch memory size
...
[ROCm/rccl commit: a0be2b8812
2019-11-20 13:03:16 -08:00
71560fd67b
2.5.6-1 ( #255 )
...
Add LL128 Protocol.
Rewrite the topology detection and tree/ring creation (#179 ). Improve
tree performance by sending/receiving from different GPUs. Add
model-based tuning to switch between the different algorithms and
protocols.
Rework P2P/SHM detection in containers (#155 , #248 ).
Detect duplicated devices and return an error (#231 ).
Add tuning for GCP
[ROCm/rccl commit: 299c554dcc
2019-11-19 14:57:39 -08:00
55ad3c801e
Support bfloat16 on rest of the unit tests
...
[ROCm/rccl commit: 4ca05c1297
2019-11-18 14:18:34 -08:00
7dc39b8928
Add bfloat16 all reduce unit test
...
[ROCm/rccl commit: bdac0256a5
2019-11-18 13:50:29 -08:00
1e182391ad
Add bfloat16 support in RCCL
...
Preprocessor symbol RCCL_BFLOAT16 is used as feature indicator
[ROCm/rccl commit: 5e109ed400
2019-11-18 13:45:53 -08:00
8b1ce44c2a
Temporary disable 0x803 target due to build error
...
[ROCm/rccl commit: cd7ab1425b
2019-11-14 11:17:41 -08:00
8e60479385
Merge pull request #151 from wenkaidu/prim_test
...
rccl_prim_test: Generalize ring topology and duplications
[ROCm/rccl commit: 55c07e4fb7
2019-11-13 08:17:55 -08:00
9b19999918
Bump up HCC version for -hc-function-calls switch
...
[ROCm/rccl commit: 08ba92f1b0
2019-11-12 14:16:35 -05:00
25b3175e82
rccl_prim_test: Generalize ring topology and duplications
...
Allow user specified ring topology from command line and duplicated
to requested number of workgroups:
./rccl_prim_test -w 12 -p copy -r "0 1 2 3|3 2 1 0|0 2 1 3|3 1 2 0|0 2 3 1|1 3 2 0"
[ROCm/rccl commit: 07bb6fce8f
2019-11-11 15:42:24 -08:00
66e4337c6e
Merge pull request #149 from wenkaidu/rtc
...
Correct RTC frequencies for profiling purpose
[ROCm/rccl commit: 277c72a638
2019-11-06 08:02:58 -08:00
22cbbb9004
Adding interactive mode for profiling purposes ( #150 )
...
[ROCm/rccl commit: fd94f4fa25
2019-11-05 17:10:16 -07:00
62042e47bc
Correct RTC frequencies for profiling purpose
...
[ROCm/rccl commit: 8995047830
2019-11-05 11:36:45 -08:00
41f6319b33
Check for fine grain support using memory allocation
...
[ROCm/rccl commit: 669f1951a4
2019-11-01 15:58:49 -07:00
0d6b476b08
Merge pull request #145 from wenkaidu/prim_test
...
rccl-prim-test: use hipExtLaunchMultiKernelMultiDevice and minor cleanup
[ROCm/rccl commit: 90b2921207
2019-11-01 13:30:01 -07:00
f9ef1553aa
Single Sync Timing mode ( #144 )
...
* Adding single sync timing mode to emulate timing reported by rccl-prim-test / rccl-tests
* Adding duration / overhead info
[ROCm/rccl commit: 2f9edd2432
2019-11-01 10:18:25 -06:00
e43e1f1b3d
additional check for fine grain support in p2pCanConnect ( #146 )
...
[ROCm/rccl commit: 5a502955c9
2019-10-31 08:58:38 -07:00
91b906cf88
rccl-prim-test: use hipExtLaunchMultiKernelMultiDevice and minor cleanup
...
[ROCm/rccl commit: ab91cdd5c9
2019-10-30 13:15:02 -07:00
a99accb2cb
Adding ability to switch between fine/coarse grain destination GPU memory
...
Adding ability to switch between memset/memcpy
[ROCm/rccl commit: 648c1ee7cc
2019-10-29 12:00:32 -06:00
b4ab922f94
Merge pull request #140 from scchan/rocm210_hc_function_calls
...
add -hc-function-calls switch back for HCC ROCm 2.10
[ROCm/rccl commit: 9be7ae8f0d
2019-10-28 09:56:47 -07:00
4b94f25d08
[cmake] Allow GPU targets to be parameterized with AMDGPU_TARGETS.
...
[ROCm/rccl commit: ec10a5cf14
2019-10-25 13:55:27 -04:00
d3f399f619
Disable HDP flush for RDMA
...
[ROCm/rccl commit: 296176a4fd
2019-10-23 14:40:17 -07:00
8d2018d372
add -hc-function-calls switch back for HCC ROCm 2.10
...
[ROCm/rccl commit: d779eae1d0
2019-10-21 18:00:02 -04:00
21bc1ef493
Revert collective chunk and slice steps to avoid drop in throughput
...
[ROCm/rccl commit: df74d12946
2019-10-18 12:54:00 -07:00
0f4d4d63ec
CI Re-enabled for Ubuntu ( #135 )
...
[ROCm/rccl commit: a95529a6e2
2019-10-18 11:38:51 -06:00
7560929bd7
Reverting GenericOp bug workaround modifications to slice/chunk steps
...
[ROCm/rccl commit: 37603ae6cb
2019-10-11 09:20:10 -07:00
cf597ff257
Performing __threadfence_system() with only first thread
...
[ROCm/rccl commit: 1392dd2997
2019-10-11 09:16:19 -07:00
d257970ad1
Fix for GenericOp device primitive bug
...
[ROCm/rccl commit: 8ae1bce3bb
2019-10-10 22:39:45 -07:00
fbcdfd8348
Merge pull request #136 from wenkaidu/tree
...
Enable tree kernels in build
[ROCm/rccl commit: 062c798c86
2019-10-09 10:58:52 -07:00
f86ee41415
Enable tree kernels in build
...
Need to tune and specify NCCL_TREE_THRESHOLD to allow usage
[ROCm/rccl commit: 76976c9e2e
2019-10-08 23:20:11 +00:00
d8a06589c9
Tuning the inline and unroll to reduce the scratch usage
...
Summary:
1. remove the noinline attribute for AllReduceThreeKernel;
2. change AUTPUNROLL for tree functions to 1 or 2;
Combining 1 and 2 will reduce the scratch usage from 1256 to 952
[ROCm/rccl commit: eec319038e
2019-10-08 14:02:25 -07:00
0af7f5268f
detect the hcc version and conditionally add the -hc-function-calls switch
...
[ROCm/rccl commit: b87ef4f152
2019-10-03 13:25:25 -04:00
57dcac6afe
Only generate kernels for sum and copy
...
[ROCm/rccl commit: 61ef1dcad5
2019-09-24 17:01:12 -07:00
a401a91fd7
Re-adding gfx908 target
...
[ROCm/rccl commit: 6232985e34
2019-09-13 16:57:34 +00:00
dbb7500fd1
RDMA HDP flush fix
...
[ROCm/rccl commit: 86ce0a93b5
2019-09-06 16:35:55 +00:00
2847ad9576
Revert "Set RDMA default to off state"
...
This reverts commit 4afd6818ba3e6b326a19
2019-09-05 18:16:53 +00:00
31670133a1
Read operation throughput
...
[ROCm/rccl commit: a270ee080e
2019-09-03 14:58:40 +05:30
bc51b5bc28
display each workgroup ,links and directions with throughputs
...
[ROCm/rccl commit: e5b13d69e5
2019-08-30 13:28:23 +05:30
daf2c4b200
Allocate opCount in pinned host memory for P2P transport
...
To avoid remote P2P read access when checking remote GPU's opCount
[ROCm/rccl commit: 8c975353ed
2019-08-29 10:22:09 -07:00
ea0ce5c064
Merge pull request #128 from amdkila/hip-clang
...
Added hip-clang options to install script, and openmp/pthread flags
[ROCm/rccl commit: 259583cde6
2019-08-27 16:23:40 -06:00
4afd6818ba
Set RDMA default to off state
...
[ROCm/rccl commit: 0f16ad966a
2019-08-26 10:59:33 -07:00
e433b21b23
Updating versioning to follow rocm-cmake standard ( #126 )
...
[ROCm/rccl commit: 544d4fb704
2019-08-23 16:33:38 -06:00
94b33a7550
Added hip-clang options to install script, and openmp/pthread options to CMakeLists.txt
...
[ROCm/rccl commit: f48ae5c98d
2019-08-23 22:02:42 +00:00
4df1defc3b
Merge pull request #125 from wenkaidu/fix_nvml_id
...
Assign unused nmvlDev to avoid random number
[ROCm/rccl commit: 6759660529
2019-08-19 09:08:13 -07:00
54608abf5c
Merge pull request #117 from rpathani/xgmi_bench
...
Modified the code to use RTC clock frequency based on gpu gcn id
[ROCm/rccl commit: ee5dec4467
2019-08-19 08:59:34 -07:00
c441f2ff9b
Update rccl_prim_test.cpp
...
[ROCm/rccl commit: 40e30b5168
2019-08-19 12:44:11 +05:30
04cd446d89
Assign unused nmvlDev to avoid random number
...
[ROCm/rccl commit: 86efdfc3b5
2019-08-16 16:34:14 -07:00
60989a3fc9
Merge remote-tracking branch 'remotes/nccl/master' into HEAD
...
[ROCm/rccl commit: 7c38da0939
2019-08-16 16:13:34 -07:00
7396d5c3ba
Tune AUTOUNROLL for better performance
...
Also remove all unused UNROLL defines
[ROCm/rccl commit: 1faededc03
2019-08-16 10:34:53 -07:00
Wenkai Du
Sylvain Jeaugey
Wenkai Du
Siu Chi Chan
gilbertlee-amd
Jeff Daily
Gilbert Lee
Michael LIAO
saadrahim
Changpeng Fang
rohit pathania
amdkila
Akila Premachandra