AI/rocm-systems
2
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add MSCCLPP user buffer registration APIs and integrate with RCCL (#1477)

Browse Source 
* ext-src: add MSCCLPP memory registration APIs

* update mem-reg patch with mscclpp helper routine to check if buffer is registered

* RCCL integration of MSCCL++ user-buffer registration APIs

* only include mscclpp_nccl header if ENABLE_MSCCLPP is defined

* ext-src: update mscclpp mem-reg patch

* add helper routine to patch

* check handle before MSCCL++ deregister

* fix typo to replace send buff with recv buff

* in case of no mscclpp registration, dduring deRegister call, ont fall back to rccl deRegister which will return an error

* Apply suggestions from code review

Whitespace suggestions and reducing diffs to avoid future merge conflicts

Co-authored-by: corey-derochie-amd <161367113+corey-derochie-amd@users.noreply.github.com>

* rename helper functions and change their return type

* set RCCL user-buffer registration to occur if attempting MSCCL++ registration with a buffer in managed memory

---------

Co-authored-by: isaki001 <Ioannis.Sakiotis@amd.com>
Co-authored-by: isaki001 <36317038+isaki001@users.noreply.github.com>
Co-authored-by: corey-derochie-amd <161367113+corey-derochie-amd@users.noreply.github.com>
This commit is contained in:
Nusrat Islam
2025-01-14 08:20:24 -06:00
committed by GitHub
parent 3fee623d5a
commit e9b6bbca8a
6 changed files with 215 additions and 9 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/include/mscclpp/mscclpp_nccl.h
+8
View File
@@ -38,6 +38,14 @@ extern "C" {
/* See ncclAllGather. */
ncclResult_t mscclpp_ncclAllGather(const void* sendbuff, void* recvbuff, size_t sendcount,
ncclDataType_t datatype, mscclppComm_t comm, hipStream_t stream);
ncclResult_t mscclpp_ncclCommRegister(mscclppComm_t comm, void* buff, size_t size, void** handle);
ncclResult_t mscclpp_ncclCommDeregister(mscclppComm_t comm, void* handle);
bool mscclpp_BuffIsRegistered(mscclppComm_t comm, const void* buff, size_t count);
size_t mscclpp_BufferSize(mscclppComm_t comm, void* handle);
}
namespace std {