AI/rocm-systems
2
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Enable fp8 support (#1101)

Browse Source 
* initial checkin

* resolve cr comments

* resolve the build issue

* fix the data correctless issue

* update fp8 header file and update the unit test for fp8 support

* remove fp16 from fp8 headers

* fix ut issue and catch up the latest code from develop

* udate according to cr comments

* update ut according to cr comments

* update num floats for each SumPostDiv from 4 to 6

* update fp8 header file name

* fix the typo

[ROCm/rccl commit: 6777e65c1d]
This commit is contained in:
Andy li
2024-03-09 07:17:53 +08:00
committed by GitHub
parent 2354601589
commit e373bd44bf
29 changed files with 1243 additions and 48 deletions
Download Patch File Download Diff File Expand all files Collapse all files
projects/rccl/src/include/nccl_common.h
+3 -3
View File
@@ -15,9 +15,9 @@ typedef void (*ncclDebugLogger_t)(ncclDebugLogLevel level, unsigned long flags,
#define NCCL_NUM_FUNCTIONS 5 // Send/Recv and AllToAllPivot not included for now
typedef enum { ncclFuncBroadcast, ncclFuncReduce, ncclFuncAllGather, ncclFuncReduceScatter, ncclFuncAllReduce, ncclFuncSendRecv, ncclFuncSend, ncclFuncRecv, ncclFuncAllToAllPivot, ncclNumFuncs} ncclFunc_t;
#define FUNC_INDEX_P2P 835
#define FUNC_INDEX_ALLTOALL_PIVOT 555
#define FUNC_INDEX_TOTAL 846
#define FUNC_INDEX_P2P 979
#define FUNC_INDEX_ALLTOALL_PIVOT 651
#define FUNC_INDEX_TOTAL 992
#define NCCL_NUM_ALGORITHMS 6 // Tree/Ring/CollNet*
#define NCCL_ALGO_UNDEF -1