if the number of elements to be used in the allreduce operation is not
exact multiple of the work-array buffer size and number of pe's, we need
to adjust the algorithm to:
- initially perform a ring_allreduce on n_segments * chunk_size (which
is the integer division of the number of elements and the work-buffer
size, i.e. will not cover the entire buffer)
- perform another ring_allreduce where chunk_size is reduced to match
the remaining elements
- if the remaining elements from the previous step cannot evenly be
divded by the number of pe's, we need to perform a direct_allreduce on
the outstanding number of elements.
[ROCm/rocshmem commit: a4b4281f50]
* Add interanl put_mem/get_mem{_wave, _wg} functions to read/write to work/sync arrays
* Add condition check to ensure all MPI processes are on the same compute node for IPC conduit
[ROCm/rocshmem commit: 6685d0ab60]
* Update roc_shmem_ctx_fence API to use team-relative PE numbering
* Update backend to populate team_opaque member of ROC_SHMEM_CTX_DEFAULT (used to store information about the team wrt TEAM_WORLD)
[ROCm/rocshmem commit: 92fb1abaf2]
code is based on the GPUIB implementations of the routines, which seem
however generic enough to work also for the IPC conduit.
Some code is in for broadcast, fcollect, and alltoall.
[ROCm/rocshmem commit: 0de3b5e6fc]
adding the device versions of the wait_until* and test functions in the
ipc folder leads to linking errors of the functional tests. Remove them
and use for now the upper level versions of the functions, similarly to
the RO conduit. Might have to revisit this later again.
[ROCm/rocshmem commit: e2e30b5339]
* add backend_ipc.{cpp & hpp}
* rename context_ipc.{cpp & hpp} to context_ipc_device.{cpp & hpp}
* add host interface to IPC backend
* add context_ipc_host.{cpp & hpp} to support host interface
* add USE_RO compile flag to enable support for single backend interface at a time
* add ipc_single script to build rocSHMEM with IPC backend
[ROCm/rocshmem commit: 49779863c2]
mv ipc_policy.{hpp,cpp} into a separate folder as a start for the
standalone IPC conduit.
Unit tests and functional tests pass on the developmpent system.
[ROCm/rocshmem commit: 1183006e20]
Edgar Gabriel
Avinash Kethineedi
Yiltan Hassan Temucin
Brandon Potter