AI/rocm-systems
2
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Enable multi-threading for MSCCL (#1203)

Browse Source 
MSCCL can now run in a multi-threaded configuration. To test in the unit tests, added the ENABLE_OPENMP compile definition flag and the --openmp-test-enable flag to the unit test build script. To activate, set the environment variables UT_MULTITHREADED=1 and UT_PROCESS_MASK=1. Set Jenkins to use this mode.
This commit is contained in:
corey-derochie-amd
2024-07-04 09:34:38 -06:00
committed by GitHub
parent 45f3fbc52f
commit 0c36d571ea
19 changed files with 279 additions and 148 deletions
Download Patch File Download Diff File Expand all files Collapse all files
test/common/EnvVars.cpp
+3 -1
View File
@@ -108,6 +108,7 @@ namespace RcclUnitTesting
showTiming = GetEnvVar("UT_SHOW_TIMING", 1);
useInteractive = GetEnvVar("UT_INTERACTIVE", 0);
timeoutUs = GetEnvVar("UT_TIMEOUT_US" , 5000000);
useMultithreading = GetEnvVar("UT_MULTITHREAD", false);
// Total number of reduction ops
int numOps = ncclNumOps;
@@ -232,7 +233,8 @@ namespace RcclUnitTesting
std::make_tuple("UT_PRINT_VALUES" , printValues , "Print array values (-1 for all)"),
std::make_tuple("UT_SHOW_TIMING" , showTiming , "Show timing table"),
std::make_tuple("UT_INTERACTIVE" , useInteractive, "Run in interactive mode"),
std::make_tuple("UT_TIMEOUT_US" , timeoutUs , "Timeout limit for collective calls in us")
std::make_tuple("UT_TIMEOUT_US" , timeoutUs , "Timeout limit for collective calls in us"),
std::make_tuple("UT_MULTITHREAD" , useMultithreading, "Multi-thread single-process ranks"),
};
printf("================================================================================\n");