MSCCL: Improve executor and integrate scheduler (#694)

* MSCCL: improve executor and add scheduler for testing

* Use external scheduler

* Fix cmake error

* Address comments

* Fix thread safe issue

* Make MSCCL lifecycle APIs thread safe

* Make MSCCL internal scheduler aware of topology hint

* Revise error message
This commit is contained in:
Ziyue Yang
2023-03-15 05:34:25 +08:00
committed by GitHub
parent 6e48e518d9
commit e3b2342f39
25 changed files with 36884 additions and 231 deletions
+3
View File
@@ -319,6 +319,9 @@ struct ncclComm {
bool finalizeCalled;
// shared structures for finalization
int finalizeRankCnt;
// Whether this comm is compatible with MSCCL
bool mscclCompatible;
};
enum ncclLaunchMode {