69d976532b
GPU allocation for CPX Unit Tests using PCI bus id ( #1403 )
...
* mapping devices wrt pci
* Gpu allocation by using pci mapping
* Passing gpuPriorityOrder in as an argument rather than making the functions non-static.
* Removing redundant testBed instance calling
[ROCm/rccl commit: 69b2b712ab
2024-11-04 10:51:00 -06:00
ad1384bea1
Hide or fix all build warnings ( #1331 )
...
* Changing C-strings to be const.
* Changed variable-length arrays to std::vector to avoid warnings. VLA is a compiler extension.
* Changed `#define` inside functions into `constexpr int` to preserve scoping and avoid macro redefinition warnings.
* Disabled warnings for modifying `CMAKE_CXX_FLAGS` caused by `check_symbol_exists`, which temporarily modifies the flag to do a compile check.
* Fixed VLA in rccl UT.
[ROCm/rccl commit: 1c45962273
2024-11-04 09:46:42 -07:00
251df02d42
Increase MAX_STACK_SIZE for UT ( #1398 )
...
[ROCm/rccl commit: 984f1e4343
2024-11-01 13:07:45 -04:00
e346e19065
Adjustment for UT Sendrecv ( #1400 )
...
Enabled UT sendrecv to same rank and refactor UBR call
[ROCm/rccl commit: fd9924cfe7
2024-10-30 15:13:53 -04:00
3eda60a031
static: Enable true rccl static library build ( #1379 )
...
* static: Enable true rccl static library build
Rccl uses `-fgpu-rdc` to compile, which requires a specialized link command in order to produce a true static library.
When "linking" with `amdclang++`, you need to use `--emit-static-lib` and `--hip-link` to get a static library with all gpu code generated. Subsequent links with binaries do not need any special flags to generate gpu code.`
Building a static library:
```
$ cmake -DROCM_PATH=$ROCM_PATH -DCMAKE_PREFIX_PATH=$ROCM_PATH -DCMAKE_BUILD_TYPE=Release -DBUILD_SHARED_LIBS=off -DCMAKE_POSITION_INDEPENDENT_CODE=on -DAMDGPU_TARGETS=gfx942 -DCMAKE_CXX_COMPILER=$ROCM_PATH/lib/llvm/bin/amdclang++ -DCMAKE_C_COMPILER=$ROCM_PATH/lib/llvm/bin/amdclang .. 2>&1 | tee -a /tmp/build.txt
-- Could NOT find GTest (missing: GTEST_LIBRARY GTEST_INCLUDE_DIR GTEST_MAIN_LIBRARY) (Required is at least version "1.11")
-- Checking for ROCm support for GPU targets: gfx942
-- Compiling for gfx942
-- Could NOT find GTest (missing: GTEST_LIBRARY GTEST_INCLUDE_DIR GTEST_MAIN_LIBRARY) (Required is at least version "1.11")
-- ROCM_PATH found: /opt/rocm
-- Compiling with amdclang++
-- HIP compiler: clang
-- HIP runtime: rocclr
-- amdclang++ executable: /opt/rocm/llvm/bin/amdclang++
-- amdclang++ version: 18.0.0git
-- hipconfig executable: /opt/rocm/bin/hipconfig
-- amdclang++ HIP version: 6.2.41133
-- ROCm version: 6.2.0
...
$ make -j 32
[ 0%] Updating git_version.cpp if necessary
-- Updating git_version.cpp
[ 0%] Built target git_version_check
[ 0%] Hipifying src/transport/shm.cc -> /home/skarlage/local/rccl/build/hipify/src/transport/shm.cc
[ 0%] Hipifying src/bootstrap.cc -> /home/skarlage/local/rccl/build/hipify/src/bootstrap.cc
[ 0%] Hipifying src/channel.cc -> /home/skarlage/local/rccl/build/hipify/src/channel.cc
[ 1%] Hipifying src/device/all_reduce.h -> /home/skarlage/local/rccl/build/hipify/src/device/all_reduce.h
[ 1%] Hipifying src/device/broadcast.h -> /home/skarlage/local/rccl/build/hipify/src/device/broadcast.h
[ 1%] Hipifying src/device/all_gather.h -> /home/skarlage/local/rccl/build/hipify/src/device/all_gather.h
[ 1%] Hipifying src/device/common.cu -> /home/skarlage/local/rccl/build/hipify/src/device/common.cu.cpp
[ 1%] Hipifying src/debug.cc -> /home/skarlage/local/rccl/build/hipify/src/debug.cc
[ 1%] Hipifying src/device/alltoall_pivot.h -> /home/skarlage/local/rccl/build/hipify/src/device/alltoall_pivot.h
[ 1%] Hipifying src/device/network/unpack/unpack.h -> /home/skarlage/local/rccl/build/hipify/src/device/network/unpack/unpack.h
[ 4%] Hipifying src/collectives.cc -> /home/skarlage/local/rccl/build/hipify/src/collectives.cc
[ 4%] Hipifying src/device/msccl_kernel_impl.h -> /home/skarlage/local/rccl/build/hipify/src/device/msccl_kernel_impl.h
[ 4%] Hipifying src/device/network/unpack/unpack_defs.h -> /home/skarlage/local/rccl/build/hipify/src/device/network/unpack/unpack_defs.h
[ 4%] Hipifying src/device/op128.h -> /home/skarlage/local/rccl/build/hipify/src/device/op128.h
[ 4%] Hipifying src/device/onerank.cu -> /home/skarlage/local/rccl/build/hipify/src/device/onerank.cu.cpp
[ 4%] Hipifying src/device/common.h -> /home/skarlage/local/rccl/build/hipify/src/device/common.h
[ 6%] Hipifying src/device/prims_ll.h -> /home/skarlage/local/rccl/build/hipify/src/device/prims_ll.h
[ 6%] Hipifying src/device/primitives.h -> /home/skarlage/local/rccl/build/hipify/src/device/primitives.h
[ 6%] Hipifying src/device/prims_ll128.h -> /home/skarlage/local/rccl/build/hipify/src/device/prims_ll128.h
[ 6%] Hipifying src/device/reduce.h -> /home/skarlage/local/rccl/build/hipify/src/device/reduce.h
[ 7%] Hipifying src/device/common_kernel.h -> /home/skarlage/local/rccl/build/hipify/src/device/common_kernel.h
[ 7%] Hipifying src/device/reduce_scatter.h -> /home/skarlage/local/rccl/build/hipify/src/device/reduce_scatter.h
[ 7%] Hipifying src/device/sendrecv.h -> /home/skarlage/local/rccl/build/hipify/src/device/sendrecv.h
[ 7%] Hipifying src/device/prims_simple.h -> /home/skarlage/local/rccl/build/hipify/src/device/prims_simple.h
[ 7%] Hipifying src/enqueue.cc -> /home/skarlage/local/rccl/build/hipify/src/enqueue.cc
[ 7%] Hipifying src/device/reduce_kernel.h -> /home/skarlage/local/rccl/build/hipify/src/device/reduce_kernel.h
[ 7%] Hipifying src/graph/connect.cc -> /home/skarlage/local/rccl/build/hipify/src/graph/connect.cc
[ 7%] Hipifying src/graph/rings.h -> /home/skarlage/local/rccl/build/hipify/src/graph/rings.h
[ 8%] Hipifying src/graph/rings.cc -> /home/skarlage/local/rccl/build/hipify/src/graph/rings.cc
[ 8%] Hipifying src/graph/rome_models.cc -> /home/skarlage/local/rccl/build/hipify/src/graph/rome_models.cc
[ 8%] Hipifying src/graph/rome_models.h -> /home/skarlage/local/rccl/build/hipify/src/graph/rome_models.h
[ 8%] Hipifying src/graph/paths.cc -> /home/skarlage/local/rccl/build/hipify/src/graph/paths.cc
[ 9%] Hipifying src/graph/search.cc -> /home/skarlage/local/rccl/build/hipify/src/graph/search.cc
[ 9%] Hipifying src/graph/topo.cc -> /home/skarlage/local/rccl/build/hipify/src/graph/topo.cc
...
[100%] Linking CXX static library librccl.a
Elapsed time: 270 s. (time), 0.00046 s. (clock)
Elapsed time: 0 s. (time), 0.000342 s. (clock)
[100%] Built target rccl
```
Static rccl exists:
```
$ file librccl.a
librccl.a: current ar archive
```
* Fix up tests Cmake for static builds
We also need to fix up the tests CMakeLists.txt to:
* Remove the unused `BUILD_STATIC` option
* Use `SHARED_LIBS` as a definition of whether we're building static or
not.
[ROCm/rccl commit: bdf9544c81
2024-10-16 06:58:50 -07:00
9c4ac4cae5
Regression timing fix ( #1361 )
...
* Removed testbed initialization on standalone tests
* .jenkins renabled all tests
[ROCm/rccl commit: 7fb9189760
2024-10-03 10:41:26 -06:00
94ac752578
Remove 0 size UBR ( #1346 )
...
ncclCommRegister, required for UBR, will call IB dmabuf regMr directly which forbids 0 size message
[ROCm/rccl commit: 40e93ebc29
2024-09-24 18:16:51 -04:00
e3b39ab309
Making variable names consistent in EnvVars.cpp ( #1327 )
...
* Making variable names consistent in EnvVars.cpp
[ROCm/rccl commit: 4856309413
2024-09-11 09:23:31 -05:00
fdaef9dd82
Enabling Unit Tests for CPX mode ( #1324 )
...
* Unit Tests for RCCL in CPX mode
* override pow2gpus set by cpx mode by user argument
* Adding comment for UT_POW2_GPUS
* Additional comment on why using pow2gpus for cpx mode.
[ROCm/rccl commit: 289a80c4e9
2024-09-09 10:12:33 -05:00
1bd3db8fc7
Update EnvVars.cpp
...
[ROCm/rccl commit: 757d1891e9
2024-09-04 16:55:36 -04:00
886b576722
adding all nccl apis to api_support to enable rccl tracing by rocprofv3 ( #1297 )
...
* adding all nccl apis to api_support to enable rccl tracing by rocprofv3
Co-authored-by: Marzieh Berenjkoub <mberenjk@amd.com >
Co-authored-by: Jonathan R. Madsen <jonathanrmadsen@gmail.com >
[ROCm/rccl commit: db840f024e
2024-08-22 12:36:07 -05:00
3261e2a5fd
Adding User Buffer Registration support for Unit test ( #1199 )
...
* Adding UBR support for UT SendRecv
Signed-off-by: Tim Hu <timhu102@amd.com >
* Update test/common/TestBedChild.cpp
Co-authored-by: corey-derochie-amd <161367113+corey-derochie-amd@users.noreply.github.com >
---------
Signed-off-by: Tim Hu <timhu102@amd.com >
Co-authored-by: corey-derochie-amd <161367113+corey-derochie-amd@users.noreply.github.com >
[ROCm/rccl commit: a4793286c7
2024-07-30 13:39:25 -04:00
37c44d531b
gfx12 Disable ll protocol ( #1268 )
...
[ROCm/rccl commit: c246e25f8e
2024-07-26 08:59:55 -06:00
54e4899607
Template unroll for RCCL kernels ( #1250 )
...
* Template unroll for RCCL kernels
* Adding unroll template arg during CMake hipification
* Reduce linking parallel jobs to avoid OOM in CI
* Workaround issues with UT tests
SWDEV-469533: register spill fix is needed for mainline build
LWPCOMMLIBS-369: cannot enable 112 channels with 80 CUs
Use -parallel-jobs=8 for linking
* CI: do not use -j 16 when building
* CI: use -j 8 when building
* Only reduce parallel linking job for CI extended
* Restore original jenkins command. Change parallel linking jobs in cmake
* Disable MSCCLPP
---------
Co-authored-by: gilbertlee-amd <gilbert.lee@amd.com >
[ROCm/rccl commit: 89349f2ce4
2024-07-19 08:15:59 -07:00
37bf54b8f8
Enable multi-threading for MSCCL ( #1203 )
...
MSCCL can now run in a multi-threaded configuration. To test in the unit tests, added the ENABLE_OPENMP compile definition flag and the --openmp-test-enable flag to the unit test build script. To activate, set the environment variables UT_MULTITHREADED=1 and UT_PROCESS_MASK=1. Set Jenkins to use this mode.
[ROCm/rccl commit: 0c36d571ea
2024-07-04 09:34:38 -06:00
de7ea612d7
Unit Tests for testing channels ( #1222 )
...
[ROCm/rccl commit: e170f41ddd
2024-06-25 10:10:10 -05:00
44064a612c
enable UT to test with channels greater than 64
...
[ROCm/rccl commit: 392a73fdef
2024-06-13 13:54:08 -05:00
dea5e83940
[UT] Start supporting multiple group calls and graphs ( #1151 )
...
* Start supporting multiple group calls UT
[ROCm/rccl commit: 0ec41f1386
2024-04-25 11:11:16 -06:00
36f9492cda
Merge remote-tracking branch 'nccl/master' into develop
...
[ROCm/rccl commit: e1a835910e
2024-04-23 13:34:00 -07:00
da835cff9c
replacing rccl_bfloat16 with hip_bfloat16 ( #1126 )
...
Co-authored-by: mberenjk <mberenjk@amd.com >
[ROCm/rccl commit: 428837ffe4
2024-04-11 11:30:37 -05:00
a285fda3a1
Static Build update - Moved all cmake install() to rocm-cmake APIs, static build update ( #1123 )
...
[ROCm/rccl commit: c0a51dc84b
2024-03-26 11:11:09 -04:00
e373bd44bf
Enable fp8 support ( #1101 )
...
* initial checkin
* resolve cr comments
* resolve the build issue
* fix the data correctless issue
* update fp8 header file and update the unit test for fp8 support
* remove fp16 from fp8 headers
* fix ut issue and catch up the latest code from develop
* udate according to cr comments
* update ut according to cr comments
* update num floats for each SumPostDiv from 4 to 6
* update fp8 header file name
* fix the typo
[ROCm/rccl commit: 6777e65c1d
2024-03-08 15:17:53 -08:00
826d20495f
Adding FP16 cases to unit tests( #1093 )
...
Signed-off-by: Tim Hu <timhu102@amd.com >
[ROCm/rccl commit: 0d06b0f1de
2024-02-26 12:08:04 -05:00
24d9e1c36b
Increase max stack size when ll128 enabled
...
[ROCm/rccl commit: b098120c40
2024-02-15 15:56:59 -08:00
7842411fb3
Merge remote-tracking branch 'rccl/develop' into 2.19.4
...
[ROCm/rccl commit: 76f83f95ab
2024-02-15 13:37:14 -08:00
bee47d9e91
Add stack size UT ( #1081 )
...
* Add stack size UT
[ROCm/rccl commit: dc2d486ba0
2024-02-12 17:56:15 -07:00
49f0e82815
Add a constructor for PtrUnion in case it is not initialized explicitly ( #1064 )
...
[ROCm/rccl commit: ba9f7917ba
2024-01-26 08:00:27 -08:00
11674674fc
[DEV] Configure functions in RCCL ( #986 )
...
* configure functions in rccl
[ROCm/rccl commit: 28d9b170c9
2024-01-18 15:07:16 -07:00
0343d9ccac
Relaxing default timeout limit, add error log ( #1052 )
...
Signed-off-by: Tim Hu <timhu102@amd.com >
[ROCm/rccl commit: 05850e89f2
2024-01-18 15:09:08 -05:00
5f7ef6b671
Adding regression test ( #1045 )
...
* adding regression test
Signed-off-by: Tim Hu <timhu102@amd.com >
* modifying regression test
Signed-off-by: Tim Hu <timhu102@amd.com >
* Update StandaloneTests.cpp
---------
Signed-off-by: Tim Hu <timhu102@amd.com >
[ROCm/rccl commit: c2a073a97d
2024-01-18 10:46:16 -05:00
245e757b26
Adding timeout functionality/EnvVar to TestBed ( #1044 )
...
* Adding timeout functionality/EnvVar to TestBed
* updating timeout unit to microseconds
Signed-off-by: Tim Hu <timhu102@amd.com >
[ROCm/rccl commit: 9c0ef11ac7
2024-01-17 11:33:01 -05:00
0c1f773021
rearranged how the min and max functions are part of msccl ( #1025 )
...
* rearranged how the min and max functions are part of msccl
* added more coverage on in place graph tests
[ROCm/rccl commit: f4858e14b2
2023-12-21 08:58:33 -07:00
bc7df769a2
AllReduceTests,fixed the number of roots ( #925 )
...
[ROCm/rccl commit: d8dc282eeb
2023-10-20 10:25:11 -06:00
b35ea4bd78
Modify All-To-All doc ( #896 )
...
* Modify All-To-All doc
* Update nccl.h.in
* update unit-tests
---------
Co-authored-by: gilbertlee-amd <44450918+gilbertlee-amd@users.noreply.github.com >
[ROCm/rccl commit: c1f57a7041
2023-09-27 12:45:21 -04:00
bcb8075e38
Add 0-byte test for send/recv ( #865 )
...
[ROCm/rccl commit: 0a01dc2f19
2023-08-29 09:14:18 -06:00
487391e8bb
Add ncclCommSplit test ( #852 )
...
Add ncclSplitCommTest
[ROCm/rccl commit: 9d11cd092f
2023-08-25 16:26:45 -06:00
71ab758d36
Removing unnecessary chrpath check for unit tests ( #811 )
...
[ROCm/rccl commit: a5a25bdff7
2023-07-20 10:28:04 -06:00
9b2475ef06
Merge pull request #782 from ROCmSoftwarePlatform/2.18.3
...
Sync up with NCCL 2.18.3
[ROCm/rccl commit: ce6a2ffac8
2023-06-29 15:04:16 -07:00
59e62c807b
Re-enabled graph tests ( #736 )
...
* enabled graph tests
* joined multi and single process CI testing
[ROCm/rccl commit: cf8cfa88a8
2023-06-29 08:08:17 -06:00
894b2d75e7
Report unit test environment variable values as part of output ( #789 )
...
[ROCm/rccl commit: f7c553edad
2023-06-29 07:13:05 -06:00
f98715baea
Merge remote-tracking branch 'nccl/master' into develop
...
[ROCm/rccl commit: abd0615351
2023-06-26 22:51:56 +00:00
a34679b2e6
resolving the pthread-gtest linking issue for rccl-UnitTests ( #768 )
...
[ROCm/rccl commit: 520f15e61b
2023-06-06 14:21:40 -04:00
d2c1295f79
Refactoring CMakeFiles ( #755 )
...
[ROCm/rccl commit: 777d8747a5
2023-05-25 16:08:54 -06:00
aeffdf872b
Disabled hipgraph tests! ( #725 )
...
[ROCm/rccl commit: 53c1c38f0e
2023-04-13 17:42:05 -04:00
af8c2194a7
lessened the amount of child processes to active ones ( #720 )
...
[ROCm/rccl commit: 2ce7d971e5
2023-04-11 08:59:56 -06:00
ff2c1c5d0f
Unit test performance refactor ( #700 )
...
* Refactoring unit tests to improve performance
* Spawning child processes during InitComms instead of on TestBed construction
* Temporarily disabling graph unit tests
[ROCm/rccl commit: 27e0cb43c2
2023-04-06 12:28:53 -06:00
b859549866
Adding interactive mode for unit tests (UT_INTERACTIVE) ( #715 )
...
[ROCm/rccl commit: 00c3d8d850
2023-03-21 10:58:24 -06:00
9daf0bc3d1
Test Fixes ( #710 )
...
* splitting CI tests in running SP first and MP second
* set device before hipStreamSynchronize on tests
[ROCm/rccl commit: 9a0d4a07a6
2023-03-21 08:48:39 -06:00
f7f669e7f0
MSCCL: Improve executor and integrate scheduler ( #694 )
...
* MSCCL: improve executor and add scheduler for testing
* Use external scheduler
* Fix cmake error
* Address comments
* Fix thread safe issue
* Make MSCCL lifecycle APIs thread safe
* Make MSCCL internal scheduler aware of topology hint
* Revise error message
[ROCm/rccl commit: e3b2342f39
2023-03-14 14:34:25 -07:00
0da1d6a6cd
Multi stream unit test ( #693 )
...
* Adding multi-stream support to unit tests
[ROCm/rccl commit: 80ed608a9d
2023-02-23 13:28:50 -07:00
saurabhAMD
corey-derochie-amd
Bertan Dogancay
Tim
Sean Karlage
akolliasAMD
mberenjk
Wenkai Du
saurabhAMD
BertanDogancay
arvindcheru
Andy li
Shilei Tian
gilbertlee-amd
Wenkai Du
Pedram Alizadeh
Ziyue Yang