Enable fp8 support (#1101)

* initial checkin

* resolve cr comments

* resolve the build issue

* fix the data correctless issue

* update fp8 header file and update the unit test for fp8 support

* remove fp16 from fp8 headers

* fix ut issue and catch up the latest code from develop

* udate according to cr comments

* update ut according to cr comments

* update num floats for each SumPostDiv from 4 to 6

* update fp8 header file name

* fix the typo
This commit is contained in:
Andy li
2024-03-09 07:17:53 +08:00
committed by GitHub
parent ff951e607d
commit 6777e65c1d
29 changed files with 1243 additions and 48 deletions
+2
View File
@@ -114,6 +114,8 @@ namespace RcclUnitTesting
dataTypes.push_back(ncclFloat32);
dataTypes.push_back(ncclFloat64);
dataTypes.push_back(ncclBfloat16);
dataTypes.push_back(ncclFp8E4M3);
dataTypes.push_back(ncclFp8E5M2);
}
// Build list of possible # GPU ranks based on env vars