Enable GDA+IPC (#249)

* Enable GDA+IPC
Fix ROCSHMEM_DISABLE_IPC for both RO and GDA

* add more functionality to bootstrap class

we need a few more functions in the boostrap class to be able to fully
handle the rocshmem requirements:
 - add a function to return the list of local ranks
 - provide a groupAllgather operation which takes a vector of ranks
   participating
 - provide a groupAlltoall operation which takes a vector of ranks
   participating

Also, update the functionality of the gda-Alltoall and gda-Allreduce
operations to take advantage of these functions.

* ipc_policy adapted to use bootstrap groupallgather

* bugfix: there was a mistake in computing sendto in groupallgather

* bugfix: shm_size and shm_rank were set in a local variable rather than
the class member

* mpi-bootstrap: remove an unecessary allgather

---------

Co-authored-by: Edgar Gabriel <Edgar.Gabriel@amd.com>
This commit is contained in:
Aurelien Bouteiller
2025-09-16 11:54:53 -04:00
zatwierdzone przez GitHub
rodzic 3100d71f4b
commit 801d2c5012
20 zmienionych plików z 378 dodań i 152 usunięć
+5 -6
Wyświetl plik
@@ -44,17 +44,16 @@ __host__ ROHostContext::ROHostContext(Backend *backend, long options)
int *pes_with_ipc_avail = new int[backend->ipcImpl.shm_size];
char** ipc_bases = new char*[b->ipcImpl.shm_size];
CHECK_HIP(hipMemcpy(pes_with_ipc_avail,
if (backend->ipcImpl.pes_with_ipc_avail != nullptr) {
CHECK_HIP(hipMemcpy(pes_with_ipc_avail,
backend->ipcImpl.pes_with_ipc_avail,
backend->ipcImpl.shm_size * sizeof(int),
hipMemcpyDeviceToHost));
CHECK_HIP(hipMemcpy(ipc_bases,
CHECK_HIP(hipMemcpy(ipc_bases,
backend->ipcImpl.ipc_bases,
backend->ipcImpl.shm_size * sizeof(char *),
hipMemcpyDeviceToHost));
}
ipcImpl_.pes_with_ipc_avail = pes_with_ipc_avail;
ipcImpl_.ipc_bases = ipc_bases;
ipcImpl_.shm_size = backend->ipcImpl.shm_size;
@@ -62,9 +61,9 @@ __host__ ROHostContext::ROHostContext(Backend *backend, long options)
}
__host__ ROHostContext::~ROHostContext() {
// host_interface->release_window_context(context_window_info);
delete[] ipcImpl_.pes_with_ipc_avail;
delete[] ipcImpl_.ipc_bases;
// host_interface->release_window_context(context_window_info);
}
__host__ void ROHostContext::putmem_nbi(void *dest, const void *source,