AI/rocm-systems
2
0 Форкнуть 0
Код Задачи Запросы на слияние Действия Пакеты Проекты Релизы Вики Активность
1 423 коммитов 3 Ветки 0 Теги
5d7078e383d6c40a08d1a9f430a19c88baf7149c
Граф коммитов
23 Коммитов
Автор SHA1 Сообщение Дата
mberenjkиmberenjk 428837ffe4 replacing rccl_bfloat16 with hip_bfloat16 (#1126) 
Co-authored-by: mberenjk <mberenjk@amd.com>
 2024-04-11 11:30:37 -05:00
Andy li 6777e65c1d Enable fp8 support (#1101) 
* initial checkin

* resolve cr comments

* resolve the build issue

* fix the data correctless issue

* update fp8 header file and update the unit test for fp8 support

* remove fp16 from fp8 headers

* fix ut issue and catch up the latest code from develop

* udate according to cr comments

* update ut according to cr comments

* update num floats for each SumPostDiv from 4 to 6

* update fp8 header file name

* fix the typo
 2024-03-08 15:17:53 -08:00
Wenkai Du 1a134b283b Merge remote-tracking branch 'rccl/develop' into 2.19.4 2024-01-31 11:53:10 -06:00
BertanDogancay 81ddf9de89 Merge remote-tracking branch 'nccl/v2.19' into develop 2024-01-24 15:25:33 -08:00
Wenkai Du 7987015a19 Revert "msccl: build same number of kernels as in ROCm 5.7" (#1058) 
This reverts commit f960174d03be7e5174baa83b256526d388a38842.
 2024-01-24 08:43:50 -08:00
Nilesh M NegiиJenkins 249e9f7f65 Un-escaped character causes error with address sanitizer builds (#992) 
Signed-off-by: Nilesh M Negi <Nilesh.Negi@amd.com>
Co-authored-by: Jenkins <jenkins-compute@amd.com>
 2024-01-09 13:28:32 -06:00
akolliasAMD f4858e14b2 rearranged how the min and max functions are part of msccl (#1025) 
* rearranged how the min and max functions are part of msccl

* added more coverage on in place graph tests
 2023-12-21 08:58:33 -07:00
Ziyue Yang 655742a3a6 Fully disable MSCCL when machine is not matched (#1017) 
* Disable MSCCL algorithm meta loading when machine is not matched

* fully disable init

* fix potential segfault
 2023-12-13 08:36:21 -08:00
Wenkai Du 12c08fc52a msccl: build same number of kernels as in ROCm 5.7 (#1005) 
Removed fullOps kernels from build
 2023-12-07 13:36:04 -06:00
Wen-Heng (Jack) Chung 293f0fb752 Use a map to host scratch buffers (#1004) 
* Use a map to host scratch buffers

* Address review feedbacks. Deliberately keep mscclSetupScratch function.
 2023-12-05 13:15:28 -06:00
Ziyue Yang 4bb0b4a380 Move MSCCL algorithm loading to initialization to workaround HIP graph conflict (#982) 
* MSCCL: pre-specify channels and pre-load algorithms

* add mutex

* fix bug

* clean include

* disable all-gathers temporarily
 2023-11-30 09:47:20 -08:00
akolliasAMD 56ce9ef05f recreated pr 914 to work with current develop branch (#979) 2023-11-28 16:33:47 -07:00
Ziyue Yang 7fc891bc8d Fix MSCCL work FIFO allocation with HIP graph enabled (#967) 2023-11-15 16:43:28 -08:00
Wenkai Du 5a800e00cd msccl: enable basic collective trace (#959) 
To avoid increasing number of kernels, colltrace is only enabled with
RCCL_MSCCL_FORCE_FULLOPS=1
 2023-11-08 20:14:28 -08:00
Wenkai Du f484ff17b9 msccl: add templated kernel (#945) 
* msccl: add templated kernel

* Use defines to improve code readability

* Fix kernel indexing and review feedback
 2023-11-02 17:21:53 -07:00
Ziyue Yang 4c117e5335 Fix MSCCL work FIFO out-of-bound issue (#935) 2023-10-27 11:24:52 -07:00
Wen-Heng (Jack) ChungPedram AlizadehPedram AlizadehZiyue Yang
 		7ee5c1c28b Change MSCCL kernel signature to allow kernel arguments be preloaded via SGPR (#911) 
* Adding a script that will download/compile/run TransferBench/RCCL/UCX/RCCL-tests/RCCL-Unittests/hip-mpi-testsuite (#895)

Co-authored-by: Pedram Alizadeh <pmohamma@banff-pla-r27-05.pla.dcgpu>

* Only build gfx941

* demo

* fine tune malloc

* Fix merge errors

* Fix merge errors

* Disable parallel build

* Adopt --amdgpu-kernarg-preload-count

* Revert "Adding a script that will download/compile/run TransferBench/RCCL/UCX/RCCL-tests/RCCL-Unittests/hip-mpi-testsuite (#895)"

This reverts commit f5e252dddf02a41b4d1bc512f306f45f97166304.

* Revert CMake changes.

* NPKIT changes.

* Remove some license declarations.

* Address code review feedbacks on msccl_kernel_impl.h

* Update CMakeLists.txt

* Add CMake logic to check the existence of --amdgpu-kernarg-preload-count

* Fix NPKIT trace logic.

---------

Co-authored-by: Pedram Alizadeh <pmohamma@amd.com>
Co-authored-by: Pedram Alizadeh <pmohamma@banff-pla-r27-05.pla.dcgpu>
Co-authored-by: Ziyue Yang <ziyyang@microsoft.com>
 2023-10-12 20:17:08 -05:00
Andy li e1dc4d5e42 enable hip graph on multi-node (#884) 
* initial checkin

* enable msccl when hip graph is on

* remove the commented out code of msccl enable check

* clean up the code

* remove the msccl HighestTransportType check logic
 2023-09-11 15:30:04 -07:00
David Pagan 2ec2648247 Fix static_assert string literal that contains a "\%". This is no longer (#860) 
valid. They can only be simple escape sequences. Removing '\' fixes
issue. Assert message now compiles and emits the '%' as expected.
 2023-08-21 16:19:59 -07:00
Ziyue Yang 11676267b5 fix min, max and avg (#745) 2023-05-18 11:02:59 -07:00
Wenkai Du 4b09ffba43 msccl: print stack and memory usage (#723) 
* msccl: print stack and memory usage

* Update number of kernels calculation
 2023-04-14 14:59:03 -07:00
Ziyue Yang e3b2342f39 MSCCL: Improve executor and integrate scheduler (#694) 
* MSCCL: improve executor and add scheduler for testing

* Use external scheduler

* Fix cmake error

* Address comments

* Fix thread safe issue

* Make MSCCL lifecycle APIs thread safe

* Make MSCCL internal scheduler aware of topology hint

* Revise error message
 2023-03-14 14:34:25 -07:00
Ziyue Yang adafc0f759 Add MSCCL Support (#658) 
* Add MSCCL support

* Add alignment and message size checking

* Fix nRanks checking, in-place and out-of-place tests and group call handling

* Fix hipGraph unit test

* Change MSCCL init warning to INFO

* Revise license info
 2022-12-12 15:51:04 -08:00