AI/rocm-systems
2
0
Форкнуть 0
Код Задачи Запросы на слияние Действия Пакеты Проекты Релизы Вики Активность

Add 'projects/rocshmem/' from commit '0496586829058af5cfd7f23acda2a6d0040da584'

Просмотр исходного кода 
git-subtree-dir: projects/rocshmem
git-subtree-mainline: 5fd976da70
git-subtree-split: 0496586829
Этот коммит содержится в:
Ameya Keshava Mallya
2026-01-21 20:25:37 +00:00
родитель 5fd976da70 0496586829
Коммит 12ab8df3bc
367 изменённых файлов: 81890 добавлений и 0 удалений
Скачать Patch файл Скачать Diff файл Показать все файлы Свернуть все файлы
projects/rocshmem/docs/api/amo.rst
+418
Просмотреть файл
@@ -0,0 +1,418 @@
.. meta::
:description: rocSHMEM intra-kernel networking runtime for AMD dGPUs on the ROCm platform.
:keywords: rocSHMEM, API, ROCm, documentation, HIP, Networking, Communication
.. _rocshmem-api-amo:
---------------------------
Atomic memory operations
---------------------------
You can call these functions from divergent control paths at the per-thread level.
ROSHMEM_ATOMIC_FETCH
--------------------
.. cpp:function:: __device__ TYPE rocshmem_TYPENAME_atomic_fetch(TYPE *source, int pe)
.. cpp:function:: __device__ TYPE rocshmem_ctx_TYPENAME_atomic_fetch(rocshmem_ctx_t ctx, TYPE *source, int pe)
:param ctx: Context with which to perform this operation.
:param dest: Destination address. Must be an address on the symmetric heap.
:param pe: PE of the remote process.
:returns: The value of ``dest``.
**Description:**
This function atomically returns the value of ``dest`` to the calling PE.
Valid ``TYPENAME`` and ``TYPE`` values are listed in EXTENDED_AMO_TYPES_.
SHMEM_ATOMIC_SET
----------------
.. cpp:function:: __device__ void rocshmem_TYPENAME_atomic_set(TYPE *dest, TYPE value, int pe);
.. cpp:function:: __device__ void rocshmem_ctx_TYPENAME_atomic_set(rocshmem_ctx_t ctx, TYPE *dest, TYPE value, int pe);
:param ctx: Context with which to perform this operation.
:param dest: Destination address. Must be an address on the symmetric heap.
:param value: The value to be atomically set.
:param pe: PE of the remote process.
:returns: None.
**Description:**
This function atomically sets the value ``value`` to ``dest`` on ``pe``.
Valid ``TYPENAME`` and ``TYPE`` values are listed in EXTENDED_AMO_TYPES_.
SHMEM_ATOMIC_COMPARE_SWAP
-------------------------
.. cpp:function:: __device__ TYPE rocshmem_TYPENAME_atomic_compare_swap(TYPE *dest, TYPE cond, TYPE value, TYPE pe);
.. cpp:function:: __device__ TYPE rocshmem_ctx_TYPENAME_atomic_compare_swap(rocshmem_ctx_t ctx, TYPE *dest, TYPE cond, TYPE value, TYPE pe);
:param ctx: Context with which to perform this operation.
:param dest: Destination address. Must be an address on the symmetric heap.
:param cond: The value to be compare with.
:param value: The value to be atomically swapped.
:param pe: PE of the remote process.
:return: The old value of ``dest``.
**Description:**
This function atomically compares the value in ``dest`` with ``cond``. If they are equal, it stores ``value`` in ``dest``.
The operation returns the older value of ``dest`` to the calling PE.
The operation is blocking.
Valid ``TYPENAME`` and ``TYPE`` values are listed in STANDARD_AMO_TYPES_.
SHMEM_ATOMIC_SWAP
-----------------
.. cpp:function:: __device__ TYPE rocshmem_TYPENAME_atomic_swap(TYPE *dest, TYPE value, TYPE pe);
.. cpp:function:: __device__ TYPE rocshmem_ctx_TYPENAME_atomic_swap(rocshmem_ctx_t ctx, TYPE *dest, TYPE value, TYPE pe);
:param ctx: Context with which to perform this operation.
:param dest: Destination address. Must be an address on the symmetric heap.
:param value: The value to be atomically swapped.
:param pe: PE of the remote process.
:return: The old value of ``dest``.
**Description:**
This function atomically swaps the value ``val`` with ``dest`` on ``pe``.
The operation is blocking.
Valid ``TYPENAME`` and ``TYPE`` values are listed in EXTENDED_AMO_TYPES_.
SHMEM_ATOMIC_FETCH_INC
----------------------
.. cpp:function:: __device__ TYPE rocshmem_TYPENAME_atomic_fetch_inc(TYPE *dest, TYPE pe);
.. cpp:function:: __device__ TYPE rocshmem_ctx_TYPENAME_atomic_fetch_inc(rocshmem_ctx_t ctx, TYPE *dest, TYPE pe);
:param ctx: Context with which to perform this operation.
:param dest: Destination address. Must be an address on the symmetric heap.
:param pe: PE of the remote process.
:return: The old value of ``dest``.
**Description:**
This function atomically adds ``1`` to ``dest`` on ``pe``.
The operation is blocking.
Valid ``TYPENAME`` and ``TYPE`` values are listed in STANDARD_AMO_TYPES_.
SHMEM_ATOMIC_INC
----------------
.. cpp:function:: __device__ void rocshmem_TYPENAME_atomic_inc(TYPE *dest, TYPE pe);
.. cpp:function:: __device__ void rocshmem_ctx_TYPENAME_atomic_inc(rocshmem_ctx_t ctx, TYPE *dest, TYPE pe);
:param ctx: Context with which to perform this operation.
:param dest: Destination address. Must be an address on the symmetric heap.
:param pe: PE of the remote process.
:return: None.
**Description:**
This function atomically adds ``1`` to ``dest`` on ``pe``.
The operation is blocking.
Valid ``TYPENAME`` and ``TYPE`` values are listed in STANDARD_AMO_TYPES_.
SHMEM_ATOMIC_FETCH_ADD
----------------------
.. cpp:function:: __device__ TYPE rocshmem_TYPENAME_atomic_fetch_add(TYPE *dest, TYPE value, TYPE pe);
.. cpp:function:: __device__ TYPE rocshmem_ctx_TYPENAME_atomic_fetch_add(rocshmem_ctx_t ctx, TYPE *dest, TYPE value, TYPE pe);
:param ctx: Context with which to perform this operation.
:param dest: Destination address. Must be an address on the symmetric heap.
:param value: The value to be atomically added.
:param pe: PE of the remote process.
:return: The old value of ``dest``.
**Description:**
This function atomically adds ``value`` to ``dest`` on ``pe``.
The operation is blocking.
Valid ``TYPENAME`` and ``TYPE`` values are listed in STANDARD_AMO_TYPES_.
SHMEM_ATOMIC_ADD
----------------
.. cpp:function:: __device__ void rocshmem_TYPENAME_atomic_add(TYPE *dest, TYPE value, TYPE pe);
.. cpp:function:: __device__ void rocshmem_ctx_TYPENAME_atomic_add(rocshmem_ctx_t ctx, TYPE *dest, TYPE value, TYPE pe);
:param ctx: Context with which to perform this operation.
:param dest: Destination address. Must be an address on the symmetric heap.
:param value: The value to be atomically added.
:param pe: PE of the remote process.
:return: None.
**Description:**
This function atomically adds ``value`` to ``dest`` on ``pe``.
The operation is blocking.
Valid ``TYPENAME`` and ``TYPE`` values can be seen in STANDARD_AMO_TYPES_.
SHMEM_ATOMIC_FETCH_AND
----------------------
.. cpp:function:: __device__ TYPE rocshmem_TYPENAME_atomic_fetch_and(TYPE *dest, TYPE value, TYPE pe);
.. cpp:function:: __device__ TYPE rocshmem_ctx_TYPENAME_atomic_fetch_and(rocshmem_ctx_t ctx, TYPE *dest, TYPE value, TYPE pe);
:param ctx: Context with which to perform this operation.
:param dest: Destination address. Must be an address on the symmetric heap.
:param value: The value to be atomically ``AND``.
:param pe: PE of the remote process.
:return: The old value of ``dest``.
**Description:**
This function atomically bitwise-and ``value`` to the value at ``dest`` on ``pe``.
The operation is blocking.
Valid ``TYPENAME`` and ``TYPE`` values are listed in BITWISE_AMO_TYPES_.
SHMEM_ATOMIC_AND
----------------
.. cpp:function:: __device__ TYPE rocshmem_TYPENAME_atomic_and(TYPE *dest, TYPE value, TYPE pe);
.. cpp:function:: __device__ TYPE rocshmem_ctx_TYPENAME_atomic_and(rocshmem_ctx_t ctx, TYPE *dest, TYPE value, TYPE pe);
:param ctx: Context with which to perform this operation.
:param dest: Destination address. Must be an address on the symmetric heap.
:param value: The value to be atomically ``AND``.
:param pe: PE of the remote process.
:return: None
**Description:**
This function atomically bitwise-and ``value`` to the value at ``dest`` on ``pe``.
The operation is blocking.
Valid ``TYPENAME`` and ``TYPE`` values are listed in BITWISE_AMO_TYPES_.
SHMEM_ATOMIC_FETCH_OR
----------------------
.. cpp:function:: __device__ TYPE rocshmem_TYPENAME_atomic_fetch_or(TYPE *dest, TYPE value, TYPE pe)
.. cpp:function:: __device__ TYPE rocshmem_ctx_TYPENAME_atomic_fetch_or(rocshmem_ctx_t ctx, TYPE *dest, TYPE value, TYPE pe)
:param ctx: Context with which to perform this operation.
:param dest: Destination address. Must be an address on the symmetric heap.
:param value: The value to be atomically ``OR``.
:param pe: PE of the remote process.
:return: The old value of ``dest``.
**Description:**
This function atomically bitwise-or ``value`` to the value at ``dest`` on ``pe``.
The operation is blocking.
Valid ``TYPENAME`` and ``TYPE`` values are listed in BITWISE_AMO_TYPES_.
SHMEM_ATOMIC_OR
---------------
.. cpp:function:: __device__ TYPE rocshmem_TYPENAME_atomic_or(TYPE *dest, TYPE value, TYPE pe)
.. cpp:function:: __device__ TYPE rocshmem_ctx_TYPENAME_atomic_or(rocshmem_ctx_t ctx, TYPE *dest, TYPE value, TYPE pe)
:param ctx: Context with which to perform this operation.
:param dest: Destination address. Must be an address on the symmetric heap.
:param value: The value to be atomically ``OR``.
:param pe: PE of the remote process.
:return: None.
**Description:**
This function atomically bitwise-or ``value`` to the value at ``dest`` on ``pe``.
The operation is blocking.
Valid ``TYPENAME`` and ``TYPE`` values are listed in BITWISE_AMO_TYPES_.
SHMEM_ATOMIC_FETCH_XOR
----------------------
.. cpp:function:: __device__ TYPE rocshmem_TYPENAME_atomic_fetch_xor(TYPE *dest, TYPE value, TYPE pe);
.. cpp:function:: __device__ TYPE rocshmem_ctx_TYPENAME_atomic_fetch_xor(rocshmem_ctx_t ctx, TYPE *dest, TYPE value, TYPE pe);
:param ctx: Context with which to perform this operation.
:param dest: Destination address. Must be an address on the symmetric heap.
:param value: The value to be atomically ``XOR``.
:param pe: PE of the remote process.
:return: The old value of ``dest``.
**Description:**
This function atomically bitwise-xor ``value`` to the value at ``dest`` on ``pe``.
The operation is blocking.
Valid ``TYPENAME`` and ``TYPE`` values are listed in BITWISE_AMO_TYPES_.
SHMEM_ATOMIC_XOR
----------------
.. cpp:function:: __device__ TYPE rocshmem_TYPENAME_atomic_xor(TYPE *dest, TYPE value, TYPE pe)
.. cpp:function:: __device__ TYPE rocshmem_ctx_TYPENAME_atomic_xor(rocshmem_ctx_t ctx, TYPE *dest, TYPE value, TYPE pe)
:param ctx: Context with which to perform this operation.
:param dest: Destination address. Must be an address on the symmetric heap.
:param value: The value to be atomically ``XOR``.
:param pe: PE of the remote process.
:return: None.
**Description:**
This function atomically bitwise-xor ``value`` to the value at ``dest`` on ``pe``.
The operation is blocking.
Valid ``TYPENAME`` and ``TYPE`` values are listed in BITWISE_AMO_TYPES_.
Supported AMO data types
------------------------
.. _STANDARD_AMO_TYPES:
.. list-table:: Standard AMO Data Types
:widths: 10 20 20
:header-rows: 1
* - TYPE
- TYPENAME
- Supported
* - int
- int
- Yes
* - long
- long
- Yes
* - long long
- longlong
- Yes
* - unsigned int
- uint
- Yes
* - unsigned long
- ulong
- Yes
* - unsigned long long
- ulonglong
- Yes
* - int32_t
- int32
- Yes
* - int64_t
- int64
- Yes
* - uint32_t
- uint32
- Yes
* - uint64_t
- uint64
- Yes
* - size_t
- size
- Yes
* - ptrdiff_t
- ptrdiff
- Yes
.. _EXTENDED_AMO_TYPES:
.. list-table:: Extended AMO Data Types
:widths: 10 20 20
:header-rows: 1
* - TYPE
- TYPENAME
- Supported
* - float
- float
- Yes
* - double
- double
- Yes
* - int
- int
- Yes
* - long
- long
- Yes
* - long long
- longlong
- Yes
* - unsigned int
- uint
- Yes
* - unsigned long
- ulong
- Yes
* - unsigned long long
- ulonglong
- Yes
* - int32_t
- int32
- Yes
* - int64_t
- int64
- Yes
* - uint32_t
- uint32
- Yes
* - uint64_t
- uint64
- Yes
* - size_t
- size
- Yes
* - ptrdiff_t
- ptrdiff
- Yes
.. _BITWISE_AMO_TYPES:
.. list-table:: Bitwise AMO Data Types
:widths: 10 20 20
:header-rows: 1
* - TYPE
- TYPENAME
- Supported
* - unsigned int
- uint
- Yes
* - unsigned long
- ulong
- Yes
* - unsigned long long
- ulonglong
- Yes
* - int32_t
- int32
- Yes
* - int64_t
- int64
- Yes
* - uint32_t
- uint32
- Yes
* - uint64_t
- uint64
- Yes
projects/rocshmem/docs/api/coll.rst
+323
Просмотреть файл
@@ -0,0 +1,323 @@
.. meta::
:description: rocSHMEM intra-kernel networking runtime for AMD dGPUs on the ROCm platform.
:keywords: rocSHMEM, API, ROCm, documentation, HIP, Networking, Communication
.. _rocshmem-api-coll:
---------------------------
Collective routines
---------------------------
ROCSHMEM_BARRIER_ALL
--------------------
.. cpp:function:: __device__ void rocshmem_barrier_all()
.. cpp:function:: __device__ void rocshmem_barrier_all_wave()
.. cpp:function:: __device__ void rocshmem_barrier_all_wg()
:returns: None.
**Description:**
This routine performs a collective barrier across all PEs in the system.
The caller is blocked until the barrier is resolved and all updates local and remote are completed.
These APIs should be called from only one thread/wavefront/workgroup within the grid to avoid undefined behavior.
ROCSHMEM_BARRIER_ALL_ON_STREAM
-------------------------------
.. cpp:function:: __host__ void rocshmem_barrier_all_on_stream(hipStream_t stream)
:param stream: HIP stream on which to enqueue the operation.
:returns: None.
**Description:**
This routine enqueues a collective barrier operation on a HIP stream. The barrier is performed
across all PEs in the system. The operation is enqueued on the specified stream and will execute
asynchronously. The caller must synchronize the stream (e.g., using ``hipStreamSynchronize``)
to ensure completion.
ROCSHMEM_BARRIER
----------------
.. cpp:function:: __device__ void rocshmem_ctx_barrier(rocshmem_ctx_t ctx, rocshmem_team_t team)
.. cpp:function:: __device__ void rocshmem_ctx_barrier_wave(rocshmem_ctx_t ctx, rocshmem_team_t team)
.. cpp:function:: __device__ void rocshmem_ctx_barrier_wg(rocshmem_ctx_t ctx, rocshmem_team_t team)
:param ctx: Context with which to perform this operation.
:returns: None.
**Description:**
This routine performs a collective barrier between all PEs in the system.
The caller is blocked until the barrier is resolved.
ROCSHMEM_TEAM_SYNC
------------------
.. cpp:function:: __device__ void rocshmem_ctx_sync(rocshmem_ctx_t ctx, rocshmem_team_t team)
.. cpp:function:: __device__ void rocshmem_ctx_sync_wave(rocshmem_ctx_t ctx, rocshmem_team_t team)
.. cpp:function:: __device__ void rocshmem_ctx_sync_wg(rocshmem_ctx_t ctx, rocshmem_team_t team)
:param ctx: Context with which to perform this operation.
:param team: Team with which to perform this operation.
:returns: None.
**Description:**
This routine registers the arrival of a PE at a barrier.
The caller is blocked until the synchronization is resolved.
Unlike the ``shmem_barrier_all`` routine, ``shmem_team_sync`` only ensures the
completion and visibility of previously issued memory stores, but does not
ensure the completion of remote memory updates issued via OpenSHMEM routines.
ROCSHMEM_SYNC_ALL
-----------------
.. cpp:function:: __device__ void rocshmem_sync_all()
.. cpp:function:: __device__ void rocshmem_sync_all_wave()
.. cpp:function:: __device__ void rocshmem_sync_all_wg()
:returns: None.
**Description:**
These routines behaves the same way as ``rocshmem_team_sync_*`` when called on the world team.
These APIs should be called from only one thread/wavefront/workgroup within the grid to avoid undefined behavior.
ROSHMEM_ALLTOALL
----------------
.. cpp:function:: __device__ void rocshmem_TYPENAME_alltoall_wg(rocshmem_team_t team, TYPE *dest, const TYPE *source, int nelems)
.. cpp:function:: __device__ void rocshmem_ctx_TYPENAME_alltoall_wg(rocshmem_ctx_t ctx, rocshmem_team_t team, TYPE *dest, const TYPE *source, int nelems)
:param team: The team participating in the collective.
:param dest: Destination address. Must be an address on the
symmetric heap.
:param source: Source address. Must be an address on the symmetric
heap.
:param nelems: Number of data blocks transferred per pair of PEs.
:returns: None.
**Description:**
This routine exchanges a fixed amount of contiguous data blocks between all pairs
of PEs participating in the collective routine.
This function must be called as a work-group collective.
Valid ``TYPENAME`` and ``TYPE`` values are listed in :ref:`RMA_TYPES`.
ROCSHMEM_ALLTOALLMEM_ON_STREAM
-------------------------------
.. cpp:function:: __host__ void rocshmem_alltoallmem_on_stream(rocshmem_team_t team, void *dest, const void *source, size_t size, hipStream_t stream)
:param team: The team participating in the collective.
:param dest: Destination address. Must be an address on the symmetric heap.
:param source: Source address. Must be an address on the symmetric heap.
:param size: Number of bytes to transfer per pair of PEs.
:param stream: HIP stream on which to enqueue the operation.
:returns: None.
**Description:**
This routine enqueues an alltoall collective operation on a HIP stream. The function
exchanges a fixed amount of contiguous data blocks between all pairs of PEs participating
in the collective routine. The operation is enqueued on the specified stream and will
execute asynchronously. The caller must synchronize the stream (e.g., using
``hipStreamSynchronize``) to ensure completion.
This function creates a separate context for each workgroup to avoid contention on the
default context, allowing parallel execution across multiple streams.
ROCSHMEM_BROADCAST
------------------
.. cpp:function:: __device__ void rocshmem_ctx_TYPENAME_broadcast_wg(rocshmem_ctx_t ctx, rocshmem_team_t team, TYPE *dest, const TYPE *source, int nelems, int pe_root)
:param ctx: Context with which to perform this collective.
:param team: The team participating in the collective.
:param dest: Destination address. Must be an address on the
symmetric heap.
:param source: Source address. Must be an address on the symmetric
heap.
:param nelems: Number of data blocks transferred per pair of PEs.
:returns: None.
**Description:**
This routine performs a broadcast across PEs in the team.
The caller is blocked until the broadcast completes.
Valid ``TYPENAME`` and ``TYPE`` values are listed in :ref:`RMA_TYPES`.
ROCSHMEM_BROADCASTMEM_ON_STREAM
--------------------------------
.. cpp:function:: __host__ void rocshmem_broadcastmem_on_stream(rocshmem_team_t team, void *dest, const void *source, size_t nelems, int pe_root, hipStream_t stream)
:param team: The team participating in the collective.
:param dest: Destination address. Must be an address on the symmetric heap.
:param source: Source address. Must be an address on the symmetric heap.
:param nelems: Number of bytes to broadcast.
:param pe_root: Root PE (relative to team) from which to broadcast.
:param stream: HIP stream on which to enqueue the operation.
:returns: None.
**Description:**
This routine enqueues a broadcast collective operation on a HIP stream. The function broadcasts
data from the root PE to all other PEs participating in the collective routine. The operation
is enqueued on the specified stream and will execute asynchronously. The caller must synchronize
the stream (e.g., using ``hipStreamSynchronize``) to ensure completion.
This function creates a separate context for each workgroup to avoid contention on the
default context, allowing parallel execution across multiple streams.
ROCSHMEM_FCOLLECT
-----------------
.. cpp:function:: __device__ void rocshmem_ctx_TYPENAME_fcollect_wg(rocshmem_ctx_t ctx, rocshmem_team_t team, TYPE *dest, const TYPE *source, int nelems)
:param ctx: Context with which to perform this collective.
:param team: The team participating in the collective.
:param dest: Destination address. Must be an address on the
symmetric heap.
:param source: Source address. Must be an address on the symmetric
heap.
:param nelems: Number of data blocks transferred per pair of PEs.
:returns: None.
**Description:**
This routine concatenates blocks of data from multiple PEs to an array in every
PE participating in the collective routine.
ROCSHMEM_REDUCTION
------------------
.. cpp:function:: __device__ int rocshmem_ctx_TYPENAME_OPNAME_reduce_wg(rocshmem_ctx_t ctx, rocshmem_team_t team, TYPE *dest, const TYPE *source, int nreduce)
:param ctx: Context with which to perform this collective.
:param team: The team participating in the collective.
:param dest: Destination address. Must be an address on the
symmetric heap.
:param source: Source address. Must be an address on the symmetric
heap.
:param nreduce: Number of data blocks transferred per pair of PEs.
:returns: Zero on successful local completion. Nonzero otherwise.
**Description:**
This routine performs an allreduce operation across PEs in the team.
Valid ``TYPENAME``, ``TYPE``, and ``OPNAME`` values are listed in :ref:`REDUCE_TYPES`.
Supported reduction types and operations
----------------------------------------
.. _REDUCE_TYPES:
.. list-table:: Reduction Types, Names and Operations
:widths: 20 20 20 20
:header-rows: 1
* - TYPE
- TYPENAME
- OPNAME
- Supported
* - char
- char
- max, min, sum, prod
- No
* - signed char
- schar
- max, min, sum, prod
- No
* - short
- short
- max, min, sum, prod
- Yes
* - int
- int
- max, min, sum, prod
- Yes
* - long
- long
- max, min, sum, prod
- Yes
* - long long
- longlong
- max, min, sum, prod
- Yes
* - ptrdiff_t
- ptrdiff
- max, min, sum, prod
- No
* - unsigned char
- uchar
- and, or, xor, max, min, sum, prod
- No
* - unsigned short
- ushort
- and, or, xor, max, min, sum, prod
- No
* - unsigned int
- uint
- and, or, xor, max, min, sum, prod
- No
* - unsigned long
- ulong
- and, or, xor, max, min, sum, prod
- No
* - unsigned long long
- ulonglong
- and, or, xor, max, min, sum, prod
- No
* - int8_t
- int8
- and, or, xor, max, min, sum, prod
- No
* - int16_t
- int16
- and, or, xor, max, min, sum, prod
- No
* - int32_t
- int32
- and, or, xor, max, min, sum, prod
- No
* - int64_t
- int64
- and, or, xor, max, min, sum, prod
- No
* - uint8_t
- uint8
- and, or, xor, max, min, sum, prod
- No
* - uint16_t
- uint16
- and, or, xor, max, min, sum, prod
- No
* - uint32_t
- uint32
- and, or, xor, max, min, sum, prod
- No
* - uint64_t
- uint64
- and, or, xor, max, min, sum, prod
- No
* - size_t
- size
- and, or, xor, max, min, sum, prod
- No
* - float
- float
- max, min, sum, prod
- Yes
* - double
- double
- max, min, sum, prod
- Yes
* - long double
- longdouble
- max, min, sum, prod
- No
* - double _Complex
- complexd
- sum, prod
- No
* - float _Complex
- complexf
- sum, prod
- No
projects/rocshmem/docs/api/ctx.rst
+59
Просмотреть файл
@@ -0,0 +1,59 @@
.. meta::
:description: rocSHMEM intra-kernel networking runtime for AMD dGPUs on the ROCm platform.
:keywords: rocSHMEM, API, ROCm, documentation, HIP, Networking, Communication
.. _rocshmem-api-ctx:
-----------------------------------
Context management routines
-----------------------------------
ROCSHMEM_CTX_CREATE
-------------------
.. cpp:function:: __device__ int rocshmem_wg_ctx_create(int64_t options, rocshmem_ctx_t *ctx)
.. cpp:function:: __device__ int rocshmem_wg_team_create_ctx(rocshmem_team_t team, long options, rocshmem_ctx_t *ctx)
:param team: Team handle to derive the context from.
:param options: Options for context creation. Ignored in current design; use the value ``0``.
:param ctx: A handle to the newly created context.
:returns: All threads returns ``0`` if the context was created successfully.
If any thread returns non-zero value, the operation fails, ctx is set to ``ROCSHMEM_CTX_INVALID`` and a
higher number of ``ROCSHMEM_MAX_NUM_CONTEXTS`` is required.
**Description:**
This routine creates an rocSHMEM context. By design, the context is private to the calling work-group.
It must be called collectively by all threads in the work-group. If the context was created successfully, a value
of zero is returned and the context handle pointed to by ctx specifies a valid context; otherwise, a nonzero value
is returned and ctx is set to ``ROCSHMEM_CTX_INVALID``. An unsuccessful context creation call is not treated as an
error and the rocSHMEM library remains in a correct state. The creation call can be reattempted after additional
resources become available.
ROCSHMEM_CTX_DESTROY
--------------------
.. cpp:function:: __device__ void rocshmem_wg_ctx_destroy(rocshmem_ctx_t *ctx)
:param ctx: Context handle.
:returns: None.
**Description:**
This routine destroys an rocSHMEM context. It must be called collectively by all threads in the work-group.
If ctx has the value ``ROCSHMEM_CTX_INVALID``, no operation is performed.
ROCSHMEM_GET_DEVICE_CTX
-----------------------
.. cpp:function:: __host__ void * rocshmem_get_device_ctx()
:param: None.
:returns: Returns ``ROCSHMEM_CTX_DEFAULT`` device pointer that users.
can query from one instance of rocSHMEM host library and
use later for dynamic module initialization in
kernel bitcode device library in the same application.
**Description:**
This routine queries rocSHMEM default device context from host API.
projects/rocshmem/docs/api/env_variables.rst
+96
Просмотреть файл
@@ -0,0 +1,96 @@
.. meta::
:description: rocSHMEM environment variables reference
:keywords: rocSHMEM, ROCm, API, environment variables, environment, reference
.. _rocshmem-api-env-variables:
********************************************************************
rocSHMEM environment variables
********************************************************************
This section describes the important environment variables used to
control the behavior of rocSHMEM.
.. list-table::
:header-rows: 1
:widths: 35,14,51
* - **Environment variable**
- **Default value**
- **Value**
* - | ``ROCSHMEM_HEAP_SIZE``
| Defines the size of the rocSHMEM symmetric heap in bytes (per PE).
- ``1073741824`` (1 GB)
- | Size in bytes (per PE).
| Note: the heap is on GPU memory.
* - | ``ROCSHMEM_MAX_NUM_CONTEXTS``
| Defines the number of contexts an application can use.
- ``32``
- Maximum number of contexts.
* - | ``ROCSHMEM_MAX_NUM_TEAMS``
| Defines the number of teams an application can use.
- ``40``
- Maximum number of teams.
* - | ``ROCSHMEM_BACKEND``
| When rocSHMEM is compiled for all backends, this enviroment variable
| selects which backend to execute. The default value is an empty string and rocSHMEM auto-selects the most appropriate backend.
- `` ``
- | ``ipc``: IPC Backend
| ``ro``: Reverse Offload Backend
| ``gda``: GPU Direct Async Backend
* - | ``ROCSHMEM_UNIQUEID_WITH_MPI``
| Defines whether rocSHMEM is expected to use MPI when using the uniqueId based initialization.
- ``0``
- | ``0``: Do not use MPI.
| ``1``: Use MPI.
* - | ``ROCSHMEM_DISABLE_MIXED_IPC``
| Defines whether to force using the network conduit even when IPC is available.
- ``0``
- | ``0``: Use IPC when available.
| ``1``: Force network conduit.
* - | ``ROCSHMEM_USE_IB_HCA``
| Defines which NIC that this PE should be bound to. The default value is an empty string and rocSHMEM auto-detects the most appropriate NIC.
- `` ``
- | Example value: ``bnxt_re0``
* - | ``ROCSHMEM_BOOTSTRAP_SOCKET_IFNAME``
| Chooses the interface to bootstrap rocSHMEM with.
| Only valid when not using MPI.
| The default value is an empty string and rocSHMEM auto-detects the most appropriate interface.
- `` ``
- | Example value: ``eno8303``
* - | ``ROCSHMEM_GDA_PROVIDER``
| When rocSHMEM is compiled with support for multiple NIC vendors,
| the enviroment variable selects the desired provider.
| The default value is an empty string and rocSHMEM auto-detects the most appropriate NIC.
- `` ``
- | ``bnxt``: Broadcom Thor 2
| ``pensando``: AMD Pensando Pollara
| ``ionic``: AMD Pensando Pollara (alias)
| ``mlx5``: Mellanox ConnectX-7
* - | ``ROCSHMEM_GDA_ALTERNATE_QP_PORTS``
| Enables or disables alternating QP mappings across rocSHMEM contexts.
- ``1``
- | ``0``: Disabled.
| ``1``: Enabled. This helps saturate bandwidth on multiport bonded interfaces.
* - | ``ROCSHMEM_GDA_TRAFFIC_CLASS``
| When using an NIC with an Ethernet link layer, this sets the traffic class for the QPs.
- ``0``
- The traffic class number.
* - | ``ROCSHMEM_GDA_PCIE_RELAXED_ORDERING``
| Enables PCIe Relaxed Ordering when registering the symmetric heap with the RDMA NICs.
- ``0``
- | ``0``: Disabled.
| ``1``: Enabled.
projects/rocshmem/docs/api/init.rst
+161
Просмотреть файл
@@ -0,0 +1,161 @@
.. meta::
:description: rocSHMEM intra-kernel networking runtime for AMD dGPUs on the ROCm platform.
:keywords: rocSHMEM, API, ROCm, documentation, HIP, Networking, Communication
.. _rocshmem-api-init:
---------------------------------------
Library setup, exit, and query routines
---------------------------------------
ROCSHMEM_INIT
-------------
.. cpp:function:: __host__ void rocshmem_init(void)
:Parameters: None.
:returns: None.
**Description:**
This routine initializes the rocSHMEM library and underlying transport layer.
Before ``rocshmem_init`` is called,
you must select the device that this PE is associated to by calling
`hipSetDevice
<https://rocm.docs.amd.com/projects/HIP/en/docs-6.0.0/doxygen/html/group___device.html#ga43c1e7f15925eeb762195ccb5e063eae>`_.
.. WARNING::
Routine `rocshmem_wg_init` has been deprecated.
.. cpp:function:: [[deprecated]] __device__ void rocshmem_wg_init(void)
:Parameters: None.
:returns: None.
**Description:**
This routine has been deprecated, please do not use.
This routine initializes device-side rocSHMEM resources.
It must be called before any threads in this work-group invoke other rocSHMEM functions.
It must be called collectively by all threads in the work-group.
ROCSHMEM_FINALIZE
-----------------
.. cpp:function:: __host__ void rocshmem_finalize(void)
:Parameters: None.
:returns: None.
**Description:**
This routine finalizes the rocSHMEM library.
.. WARNING::
Routine `rocshmem_wg_finalize` has been deprecated.
.. cpp:function:: [[deprecated]] __device__ void rocshmem_wg_finalize(void)
:Parameters: None.
:returns: None.
**Description:**
This routine has been deprecated, please do not use.
This routine finalizes device-side rocSHMEM resources.
It must be called before work-group completion if the work-group also called ``rocshmem_wg_init``.
It must be called collectively by all threads in the work-group.
ROCSHMEM_INIT_ATTR
------------------
.. cpp:function:: __host__ int rocshmem_init_attr(unsigned int flags, rocshmem_init_attr_t *attr)
:param flags: The initialization method to be used.
:param attr: Attribute structure specifying input characteristics.
:returns int: Returns ``0`` on success; otherwise, returns a nonzero value.
**Description:**
This routine initializes the rocSHMEM runtime and underlying transport layer using
the provided mode and attributes.
The parameter ``flags`` can be either
``ROCSHMEM_INIT_WITH_UNIQUEID`` or ``ROCSHMEM_INIT_WITH_MPI_COMM``.
ROCSHMEM_GET_UNIQUEID
---------------------
.. cpp:function:: __host__ int rocshmem_get_uniqueid(rocshmem_uniqueid_t *uid)
:param uid: Pointer to a unique ID handle.
:returns: Returns ``0`` on success; otherwise, returns a nonzero value.
**Description:**
This routine returns a unique ID.
ROCSHMEM_SET_ATTR_UNIQUEID_ARGS
-------------------------------
.. cpp:function:: __host__ int rocshmem_set_attr_uniqueid_args(int rank, int nranks, rocshmem_uniqueid_t *uid, rocshmem_init_attr_t *attr)
:param rank: Rank of the calling process.
:param nranks: Number of PEs.
:param uid: Unique ID used to identify the group processes.
:param attr: Attribute structure to be passed to ``rocshmem_init_attr_t``.
:returns: Returns ``0`` on success; otherwise, returns a nonzero value.
**Description:**
This routine initializes the ``rocshmem_init_attr_t`` struct.
ROCSHMEM_N_PES
--------------
.. cpp:function:: __host__ int rocshmem_n_pes(void)
:Parameters: None.
:returns: Total number of PEs.
**Description:**
This routine queries the total number of PEs.
It can be called before ``rocshmem_init``.
.. cpp:function:: __device__ int rocshmem_n_pes(void)
.. cpp:function:: __device__ int rocshmem_ctx_n_pes(rocshmem_ctx_t ctx)
:param ctx: GPU side context handle.
:returns: Total number of PEs.
**Description:**
This routine queries the total number of PEs for a given context.
It can be called per thread with no performance penalty.
ROCSHMEM_MY_PE
--------------
.. cpp:function:: __host__ int rocshmem_my_pe(void)
:Parameters: None.
:returns: PE ID of the caller.
**Description:**
This routine queries the PE ID of the caller.
It can be called before ``rocshmem_init``.
.. cpp:function:: __device__ int rocshmem_my_pe(void)
.. cpp:function:: __device__ int rocshmem_ctx_my_pe(rocshmem_ctx_t ctx)
:param ctx: GPU side context handle.
:returns: PE ID of the caller.
**Description:**
This routine queries the PE ID of the caller.
It can be called per thread with no performance penalty.
ROCSHMEM_PTR
--------------
.. cpp:function:: __host__ void* rocshmem_ptr(const void *dest, int pe);
.. cpp:function:: __device__ void* rocshmem_ptr(const void *dest, int pe);
:param dest: Local symmetric heap allocation pointer for current PE.
:param pe: Remote PE.
:returns: Returns remote symmetric heap device pointer from host-side API.
``NULL`` is returned if a valid device pointer cannot be provided.
This pointer can be used to issue load/store from custom kernels
instead of using rocshmem device side get/put APIs for RMA operations.
**Description:**
This routine queries rocSHMEM remote symmetric heap pointer.
projects/rocshmem/docs/api/memory_management.rst
+35
Просмотреть файл
@@ -0,0 +1,35 @@
.. meta::
:description: rocSHMEM intra-kernel networking runtime for AMD dGPUs on the ROCm platform.
:keywords: rocSHMEM, API, ROCm, documentation, HIP, Networking, Communication
.. _rocshmem-api-memory-management:
---------------------------
Memory management routines
---------------------------
ROCSHMEM_MALLOC
---------------
.. cpp:function:: __host__ void *rocshmem_malloc(size_t size)
:param size: Memory allocation size in bytes.
:returns: A pointer to the allocated memory on the symmetric heap.
If a valid allocation cannot be made, it returns ``NULL``.
**Description:**
This routine allocates memory of ``size`` bytes from the symmetric heap.
This is a collective operation and must be called by all PEs.
ROCSHMEM_FREE
-------------
.. cpp:function:: __host__ void rocshmem_free(void *ptr)
:param ptr: A pointer to previously allocated memory on the symmetric heap.
:returns: None.
**Description:**
This routine frees a memory allocation from the symmetric heap.
It is a collective operation and must be called by all PEs.
projects/rocshmem/docs/api/memory_ordering.rst
+51
Просмотреть файл
@@ -0,0 +1,51 @@
.. meta::
:description: rocSHMEM intra-kernel networking runtime for AMD dGPUs on the ROCm platform.
:keywords: rocSHMEM, API, ROCm, documentation, HIP, Networking, Communication
.. _rocshmem-api-memory-ordering:
---------------------------
Memory ordering routines
---------------------------
ROCSHMEM_FENCE
--------------
.. cpp:function:: __device__ void rocshmem_fence()
.. cpp:function:: __device__ void rocshmem_fence(int pe)
.. cpp:function:: __device__ void rocshmem_ctx_fence(rocshmem_ctx_t ctx)
.. cpp:function:: __device__ void rocshmem_ctx_fence(rocshmem_ctx_t ctx, int pe)
:param ctx: Context with which to perform this operation.
:param pe: Destination ``pe``.
:returns: None.
**Description:**
This routine ensures order between messages in this context to follow OpenSHMEM semantics.
ROCSHMEM_QUIET
--------------
.. cpp:function:: __device__ void rocshmem_ctx_quiet(rocshmem_ctx_t ctx)
.. cpp:function:: __device__ void rocshmem_quiet()
:param ctx: Context with which to perform this operation.
:returns: None.
**Description:**
This routine completes all previous operations posted to this context.
ROCSHMEM_PE_QUIET
-----------------
.. cpp:function:: __device__ void rocshmem_ctx_pe_quiet(shmem_ctx_t ctx, const int *target_pes, size_t npes)
.. cpp:function:: __device__ void rocshmem_pe_quiet(const int *target_pes, size_t npes)
:param ctx: Context with which to perform this operation.
:param target_pes: Address of target PE array where the operations need to be completed
:param npes: The number of PEs in the target PE array
:returns: None.
**Description:**
This routine completes all previous operations posted to this context
for the PEs in the `target_pes` array.
projects/rocshmem/docs/api/pt2pt_sync.rst
+142
Просмотреть файл
@@ -0,0 +1,142 @@
.. meta::
:description: rocSHMEM intra-kernel networking runtime for AMD dGPUs on the ROCm platform.
:keywords: rocSHMEM, API, ROCm, documentation, HIP, Networking, Communication
.. _rocshmem-api-pt2pt-sync:
-----------------------------------------
Point-to-point synchronization routines
-----------------------------------------
ROCSHMEM_WAIT_UNTIL
-------------------
.. cpp:function:: __device__ void rocshmem_TYPENAME_wait_until(TYPE *ivars, int cmp, TYPE val)
:param ivars: Pointer to memory on the symmetric heap to wait for.
:param cmp: Operation for the comparison.
:param val: Value to compare the memory at ``ivars`` to.
:returns: None.
**Description:**
This routine blocks the caller until the condition ``(*ivars cmp val)`` is true.
Valid ``cmp`` values are listed in :ref:`CMP_VALUES`.
Valid ``TYPENAME`` and ``TYPE`` values are listed in :ref:`STANDARD_AMO_TYPES`.
ROCSHMEM_WAIT_UNTIL_ALL
-----------------------
.. cpp:function:: __device__ void rocshmem_TYPENAME_wait_until_all(TYPE *ivars, size_t nelems, const int* status, int cmp, TYPE val)
:param ivars: Pointer to memory on the symmetric heap to wait for.
:param nelems: Number of elements in the ``ivars`` array.
:param status: Array of length ``nelems`` to exclude elements from the wait.
:param cmp: Operation for the comparison.
:param val: Value to compare.
:returns: None.
**Description:**
This routine blocks the caller until the condition ``(ivars[i] cmp val)`` is true for all ``ivars``.
Valid ``cmp`` values are listed in :ref:`CMP_VALUES`.
Valid ``TYPENAME`` and ``TYPE`` values are listed in :ref:`STANDARD_AMO_TYPES`.
ROCSHMEM_WAIT_UNTIL_ANY
-----------------------
.. cpp:function:: __device__ size_t rocshmem_TYPENAME_wait_until_any(TYPE *ivars, size_t nelems, const int* status, int cmp, TYPE val)
:param ivars: Pointer to memory on the symmetric heap to wait for.
:param nelems: Number of elements in the ``ivars`` array.
:param status: Array of length ``nelems`` to exclude elements from the wait.
:param cmp: Operation for the comparison.
:param val: Value to compare.
:returns: The index of an element in the ``ivars`` array that satisfies the wait condition. If the wait set is empty, this routine returns ``SIZE_MAX``.
**Description:**
This routine blocks the caller until any of the condition ``(ivars[i] cmp val)`` is true.
Valid ``cmp`` values are listed in :ref:`CMP_VALUES`.
Valid ``TYPENAME`` and ``TYPE`` values are listed in :ref:`STANDARD_AMO_TYPES`.
ROCSHMEM_WAIT_UNTIL_SOME
------------------------
.. cpp:function:: __device__ size_t rocshmem_TYPENAME_wait_until_some(TYPE *ivars, size_t nelems, size_t* indices, const int* status, int cmp, TYPE val)
:param ivars: Pointer to memory on the symmetric heap to wait for.
:param nelems: Number of elements in the ``ivars`` array.
:param indices: List of indices with a length of at least ``nelems``.
:param status: Array of length ``nelems`` to exclude elements from the wait.
:param cmp: Operation for the comparison.
:param val: Value to compare.
:returns: The number of indices returned in the indices array. If the wait set is empty, this routine returns ``0``.
**Description:**
This routine blocks the caller until any of the conditions ``(ivars[i] cmp val)`` is true.
Valid ``cmp`` values are listed in :ref:`CMP_VALUES`.
Valid ``TYPENAME`` and ``TYPE`` values are listed in :ref:`STANDARD_AMO_TYPES`.
ROCSHMEM_TEST
-------------
.. cpp:function:: __device__ int rocshmem_TYPENAME_test(TYPE *ivars, int cmp, TYPE val)
:param ivars: Pointer to memory on the symmetric heap to wait for.
:param cmp: Operation for the comparison.
:param val: Value to compare the memory at ``ivars`` to.
:returns: ``1`` if the evaluation is true. ``0`` otherwise.
**Description:**
This routine tests if the condition ``(*ivars cmp val)`` is true.
ROCSHMEM_SIGNAL_WAIT_UNTIL_ON_STREAM
-------------------------------------
.. cpp:function:: __host__ void rocshmem_signal_wait_until_on_stream(uint64_t *sig_addr, int cmp, uint64_t cmp_value, hipStream_t stream)
:param sig_addr: Address of the signal variable on the symmetric heap.
:param cmp: Comparison operator (e.g., ROCSHMEM_CMP_EQ, ROCSHMEM_CMP_GE, etc.).
:param cmp_value: Value to compare against.
:param stream: HIP stream on which to enqueue the operation.
:returns: None.
**Description:**
This routine enqueues a wait operation on a HIP stream. The function blocks the calling thread
until the signal variable at ``sig_addr`` satisfies the comparison condition ``(*sig_addr cmp cmp_value)``.
The wait operation is executed asynchronously on the specified stream. The caller must synchronize
the stream (e.g., using ``hipStreamSynchronize``) to ensure the wait condition has been satisfied.
Valid ``cmp`` values are listed in :ref:`CMP_VALUES`.
.. _CMP_VALUES:
Supported comparisons
---------------------
The following table lists the point-to-point comparison constants:
.. list-table:: Point-to-Point Comparison Constants
:widths: 20 20
:header-rows: 1
* - Constant
- Description
* - ROCSHMEM_CMP_EQ
- Equal
* - ROCSHMEM_CMP_NE
- Not equal
* - ROCSHMEM_CMP_GT
- Greater than
* - ROCSHMEM_CMP_GE
- Greater than or equal to
* - ROCSHMEM_CMP_LT
- Less than
* - ROCSHMEM_CMP_LE
- Less than or equal to
projects/rocshmem/docs/api/rma.rst
+278
Просмотреть файл
@@ -0,0 +1,278 @@
.. meta::
:description: rocSHMEM intra-kernel networking runtime for AMD dGPUs on the ROCm platform.
:keywords: rocSHMEM, API, ROCm, documentation, HIP, Networking, Communication
.. _rocshmem-api-rma:
-----------------------------------------
Remote memory access routines
-----------------------------------------
- Routines with the ``_wave`` and ``_wg`` suffixes require all threads in a wavefront and workgroup, respectively,
to call the routine with the same parameters.
- Routines with the ``_nbi`` substring will return as soon as the request is posted.
- Routines without the ``_nbi`` substring will block until the operation completes locally.
- Valid ``TYPENAME`` and ``TYPE`` values can be found in RMA_TYPES_.
ROCSHMEM_PUT
------------
.. cpp:function:: __device__ void rocshmem_TYPENAME_put(TYPE *dest, const TYPE *source, size_t nelems, int pe)
.. cpp:function:: __device__ void rocshmem_TYPENAME_put_wave(TYPE *dest, const TYPE *source, size_t nelems, int pe)
.. cpp:function:: __device__ void rocshmem_TYPENAME_put_wg(TYPE *dest, const TYPE *source, size_t nelems, int pe)
.. cpp:function:: __device__ void rocshmem_TYPENAME_put_nbi(TYPE *dest, const TYPE *source, size_t nelems, int pe)
.. cpp:function:: __device__ void rocshmem_TYPENAME_put_nbi_wave(TYPE *dest, const TYPE *source, size_t nelems, int pe)
.. cpp:function:: __device__ void rocshmem_TYPENAME_put_nbi_wg(TYPE *dest, const TYPE *source, size_t nelems, int pe)
.. cpp:function:: __device__ void rocshmem_ctx_TYPENAME_put(rocshmem_ctx_t ctx, TYPE *dest, const TYPE *source, size_t nelems, int pe)
.. cpp:function:: __device__ void rocshmem_ctx_TYPENAME_put_wave(rocshmem_ctx_t ctx, TYPE *dest, const TYPE *source, size_t nelems, int pe)
.. cpp:function:: __device__ void rocshmem_ctx_TYPENAME_put_wg(rocshmem_ctx_t ctx, TYPE *dest, const TYPE *source, size_t nelems, int pe)
.. cpp:function:: __device__ void rocshmem_ctx_TYPENAME_put_nbi(rocshmem_ctx_t ctx, TYPE *dest, const TYPE *source, size_t nelems, int pe)
.. cpp:function:: __device__ void rocshmem_ctx_TYPENAME_put_nbi_wave(rocshmem_ctx_t ctx, TYPE *dest, const TYPE *source, size_t nelems, int pe)
.. cpp:function:: __device__ void rocshmem_ctx_TYPENAME_put_nbi_wg(rocshmem_ctx_t ctx, TYPE *dest, const TYPE *source, size_t nelems, int pe)
:param ctx: Context with which to perform this operation.
:param dest: Destination address. Must be an address on the symmetric heap.
:param source: Source address. Must be an address on the symmetric heap.
:param nelems: The number of elements to transfer.
:param pe: PE of the remote process.
:returns: None.
**Description:**
This routine writes contiguous data of ``nelems`` elements from source on the calling PE to ``dest`` at ``pe``.
ROCSHMEM_PUTMEM
---------------
.. cpp:function:: __device__ void rocshmem_putmem(void *dest, const void *source, size_t nelems, int pe)
.. cpp:function:: __device__ void rocshmem_putmem_wave(void *dest, const void *source, size_t nelems, int pe)
.. cpp:function:: __device__ void rocshmem_putmem_wg(void *dest, const void *source, size_t nelems, int pe)
.. cpp:function:: __device__ void rocshmem_putmem_nbi(void *dest, const void *source, size_t nelems, int pe)
.. cpp:function:: __device__ void rocshmem_putmem_nbi_wave(void *dest, const void *source, size_t nelems, int pe)
.. cpp:function:: __device__ void rocshmem_putmem_nbi_wg(void *dest, const void *source, size_t nelems, int pe)
.. cpp:function:: __device__ void rocshmem_ctx_putmem(rocshmem_ctx_t ctx, void *dest, const void *source, size_t nelems, int pe)
.. cpp:function:: __device__ void rocshmem_ctx_putmem_wave(rocshmem_ctx_t ctx, void *dest, const void *source, size_t nelems, int pe)
.. cpp:function:: __device__ void rocshmem_ctx_putmem_wg(rocshmem_ctx_t ctx, void *dest, const void *source, size_t nelems, int pe)
.. cpp:function:: __device__ void rocshmem_ctx_putmem_nbi(rocshmem_ctx_t ctx, void *dest, const void *source, size_t nelems, int pe)
.. cpp:function:: __device__ void rocshmem_ctx_putmem_nbi_wave(rocshmem_ctx_t ctx, void *dest, const void *source, size_t nelems, int pe)
.. cpp:function:: __device__ void rocshmem_ctx_putmem_nbi_wg(rocshmem_ctx_t ctx, void *dest, const void *source, size_t nelems, int pe)
:param ctx: Context with which to perform this operation.
:param dest: Destination address. Must be an address on the symmetric heap.
:param source: Source address. Must be an address on the symmetric heap.
:param nelems: Size of the transfer in bytes.
:param pe: PE of the remote process.
:returns: None.
**Description:**
This routine writes contiguous data of ``nelems`` bytes from source on the calling PE to ``dest`` at ``pe``.
ROCSHMEM_PUTMEM_ON_STREAM
--------------------------
.. cpp:function:: __host__ void rocshmem_putmem_on_stream(void *dest, const void *source, size_t nelems, int pe, hipStream_t stream)
:param dest: Destination address. Must be an address on the symmetric heap.
:param source: Source address. Must be an address on the symmetric heap.
:param nelems: Size of the transfer in bytes.
:param pe: PE of the remote process.
:param stream: HIP stream on which to enqueue the operation.
:returns: None.
**Description:**
This routine enqueues a putmem RMA operation on a HIP stream. The function writes contiguous
data of ``nelems`` bytes from source on the calling PE to ``dest`` at ``pe``. The operation
is enqueued on the specified stream and will execute asynchronously. The caller must
synchronize the stream (e.g., using ``hipStreamSynchronize``) to ensure completion.
ROCSHMEM_P
----------
.. cpp:function:: __device__ void rocshmem_TYPENAME_p(TYPE *dest, TYPE value, int pe)
.. cpp:function:: __device__ void rocshmem_ctx_TYPENAME_p(rocshmem_ctx_t ctx, TYPE *dest, TYPE value, int pe)
:param ctx: Context with which to perform this operation.
:param dest: Destination address. Must be an address on the symmetric heap.
:param value: Value to write to ``dest`` at ``pe``.
:param pe: PE of the remote process.
:returns: None.
**Description:**
This routine writes a single value to to ``dest`` at ``pe``.
ROCSHMEM_GET
------------
.. cpp:function:: __device__ void rocshmem_TYPENAME_get(TYPE *dest, const TYPE *source, size_t nelems, int pe)
.. cpp:function:: __device__ void rocshmem_TYPENAME_get_wave(TYPE *dest, const TYPE *source, size_t nelems, int pe)
.. cpp:function:: __device__ void rocshmem_TYPENAME_get_wg(TYPE *dest, const TYPE *source, size_t nelems, int pe)
.. cpp:function:: __device__ void rocshmem_TYPENAME_get_nbi(TYPE *dest, const TYPE *source, size_t nelems, int pe)
.. cpp:function:: __device__ void rocshmem_TYPENAME_get_nbi_wave(TYPE *dest, const TYPE *source, size_t nelems, int pe)
.. cpp:function:: __device__ void rocshmem_TYPENAME_get_nbi_wg(TYPE *dest, const TYPE *source, size_t nelems, int pe)
.. cpp:function:: __device__ void rocshmem_ctx_TYPENAME_get(rocshmem_ctx_t ctx, TYPE *dest, const TYPE *source, size_t nelems, int pe)
.. cpp:function:: __device__ void rocshmem_ctx_TYPENAME_get_wave(rocshmem_ctx_t ctx, TYPE *dest, const TYPE *source, size_t nelems, int pe)
.. cpp:function:: __device__ void rocshmem_ctx_TYPENAME_get_wg(rocshmem_ctx_t ctx, TYPE *dest, const TYPE *source, size_t nelems, int pe)
.. cpp:function:: __device__ void rocshmem_ctx_TYPENAME_get_nbi(rocshmem_ctx_t ctx, TYPE *dest, const TYPE *source, size_t nelems, int pe)
.. cpp:function:: __device__ void rocshmem_ctx_TYPENAME_get_nbi_wave(rocshmem_ctx_t ctx, TYPE *dest, const TYPE *source, size_t nelems, int pe)
.. cpp:function:: __device__ void rocshmem_ctx_TYPENAME_get_nbi_wg(rocshmem_ctx_t ctx, TYPE *dest, const TYPE *source, size_t nelems, int pe)
:param ctx: Context with which to perform this operation.
:param dest: Destination address; Must be an address on the symmetric heap.
:param source: Source address. Must be an address on the symmetric heap.
:param nelems: The number of elements to transfer.
:param pe: PE of the remote process.
:returns: None.
**Description:**
This routine reads contiguous data of ``nelems`` elements from source on ``pe`` to ``dest`` on the calling PE.
ROCSHMEM_GETMEM
---------------
.. cpp:function:: __device__ void rocshmem_getmem(void *dest, const void *source, size_t nelems, int pe)
.. cpp:function:: __device__ void rocshmem_getmem_wave(void *dest, const void *source, size_t nelems, int pe)
.. cpp:function:: __device__ void rocshmem_getmem_wg(void *dest, const void *source, size_t nelems, int pe)
.. cpp:function:: __device__ void rocshmem_getmem_nbi(void *dest, const void *source, size_t nelems, int pe)
.. cpp:function:: __device__ void rocshmem_getmem_nbi_wave(void *dest, const void *source, size_t nelems, int pe)
.. cpp:function:: __device__ void rocshmem_getmem_nbi_wg(void *dest, const void *source, size_t nelems, int pe)
.. cpp:function:: __device__ void rocshmem_ctx_getmem(rocshmem_ctx_t ctx, void *dest, const void *source, size_t nelems, int pe)
.. cpp:function:: __device__ void rocshmem_ctx_getmem_wave(rocshmem_ctx_t ctx, void *dest, const void *source, size_t nelems, int pe)
.. cpp:function:: __device__ void rocshmem_ctx_getmem_wg(rocshmem_ctx_t ctx, void *dest, const void *source, size_t nelems, int pe)
.. cpp:function:: __device__ void rocshmem_ctx_getmem_nbi(rocshmem_ctx_t ctx, void *dest, const void *source, size_t nelems, int pe)
.. cpp:function:: __device__ void rocshmem_ctx_getmem_nbi_wave(rocshmem_ctx_t ctx, void *dest, const void *source, size_t nelems, int pe)
.. cpp:function:: __device__ void rocshmem_ctx_getmem_nbi_wg(rocshmem_ctx_t ctx, void *dest, const void *source, size_t nelems, int pe)
:param ctx: Context with which to perform this operation.
:param dest: Destination address. Must be an address on the symmetric heap.
:param source: Source address. Must be an address on the symmetric heap.
:param nelems: Size of the transfer in bytes.
:param pe: PE of the remote process.
:returns: None.
**Description:**
This routine reads contiguous data of ``nelems`` bytes from source on ``pe`` to ``dest`` on the calling PE.
ROCSHMEM_GETMEM_ON_STREAM
--------------------------
.. cpp:function:: __host__ void rocshmem_getmem_on_stream(void *dest, const void *source, size_t nelems, int pe, hipStream_t stream)
:param dest: Destination address. Must be an address on the symmetric heap.
:param source: Source address. Must be an address on the symmetric heap.
:param nelems: Size of the transfer in bytes.
:param pe: PE of the remote process.
:param stream: HIP stream on which to enqueue the operation.
:returns: None.
**Description:**
This routine enqueues a getmem RMA operation on a HIP stream. The function reads contiguous
data of ``nelems`` bytes from source on ``pe`` to ``dest`` on the calling PE. The operation
is enqueued on the specified stream and will execute asynchronously. The caller must
synchronize the stream (e.g., using ``hipStreamSynchronize``) to ensure completion.
ROCSHMEM_G
----------
.. cpp:function:: __device__ float rocshmem_ctx_float_g(rocshmem_ctx_t ctx, const float *source, int pe)
.. cpp:function:: __device__ float rocshmem_float_g(const float *source, int pe)
:param ctx: Context with which to perform this operation.
:param source: Source address. Must be an address on the symmetric heap.
:param pe: PE of the remote process.
:returns: The value read from source at ``pe``.
**Description:**
This routine reads and returns single value from source at ``pe``.
Supported RMA data types
------------------------
The following table lists the supported RMA data types:
.. _RMA_TYPES:
.. list-table:: RMA Data Types
:widths: 10 20 20
:header-rows: 1
* - TYPE
- TYPENAME
- Supported
* - float
- float
- Yes
* - double
- double
- Yes
* - long double
- longdouble
- No
* - char
- char
- Yes
* - signed char
- schar
- Yes
* - short
- short
- Yes
* - int
- int
- Yes
* - long
- long
- Yes
* - long long
- longlong
- Yes
* - unsigned char
- uchar
- Yes
* - unsigned short
- ushort
- Yes
* - unsigned int
- uint
- Yes
* - unsigned long
- ulong
- Yes
* - unsigned long long
- ulonglong
- Yes
* - int8_t
- int8
- No
* - int16_t
- int16
- No
* - int32_t
- int32
- No
* - int64_t
- int64
- Yes
* - uint8_t
- uint8
- No
* - uint16_t
- uint16
- No
* - uint32_t
- uint32
- No
* - uint64_t
- uint64
- No
* - size_t
- size
- No
* - ptrdiff_t
- ptrdiff
- No
projects/rocshmem/docs/api/sigops.rst
+125
Просмотреть файл
@@ -0,0 +1,125 @@
.. meta::
:description: rocSHMEM intra-kernel networking runtime for AMD dGPUs on the ROCm platform.
:keywords: rocSHMEM, API, ROCm, documentation, HIP, Networking, Communication
.. _rocshmem-api-sigops:
---------------------
Signaling operations
---------------------
ROCSHMEM_PUTMEM_SIGNAL
----------------------
.. cpp:function:: __device__ void rocshmem_putmem_signal(void *dest, const void *source, size_t nelems, uint64_t *sig_addr, uint64_t signal, int sig_op, int pe)
.. cpp:function:: __device__ void rocshmem_putmem_signal_wave(void *dest, const void *source, size_t nelems, uint64_t *sig_addr, uint64_t signal, int sig_op, int pe)
.. cpp:function:: __device__ void rocshmem_putmem_signal_wg(void *dest, const void *source, size_t nelems, uint64_t *sig_addr, uint64_t signal, int sig_op, int pe)
.. cpp:function:: __device__ void rocshmem_putmem_signal_nbi(void *dest, const void *source, size_t nelems, uint64_t *sig_addr, uint64_t signal, int sig_op, int pe)
.. cpp:function:: __device__ void rocshmem_putmem_signal_nbi_wave(void *dest, const void *source, size_t nelems, uint64_t *sig_addr, uint64_t signal, int sig_op, int pe)
.. cpp:function:: __device__ void rocshmem_putmem_signal_nbi_wg(void *dest, const void *source, size_t nelems, uint64_t *sig_addr, uint64_t signal, int sig_op, int pe)
.. cpp:function:: __device__ void rocshmem_ctx_putmem_signal(rocshmem_ctx_t ctx, void *dest, const void *source, size_t nelems, uint64_t *sig_addr, uint64_t signal, int sig_op, int pe)
.. cpp:function:: __device__ void rocshmem_ctx_putmem_signal_wave(rocshmem_ctx_t ctx, void *dest, const void *source, size_t nelems, uint64_t *sig_addr, uint64_t signal, int sig_op, int pe)
.. cpp:function:: __device__ void rocshmem_ctx_putmem_signal_wg(rocshmem_ctx_t ctx, void *dest, const void *source, size_t nelems, uint64_t *sig_addr, uint64_t signal, int sig_op, int pe)
.. cpp:function:: __device__ void rocshmem_ctx_putmem_signal_nbi(rocshmem_ctx_t ctx, void *dest, const void *source, size_t nelems, uint64_t *sig_addr, uint64_t signal, int sig_op, int pe)
.. cpp:function:: __device__ void rocshmem_ctx_putmem_signal_nbi_wave(rocshmem_ctx_t ctx, void *dest, const void *source, size_t nelems, uint64_t *sig_addr, uint64_t signal, int sig_op, int pe)
.. cpp:function:: __device__ void rocshmem_ctx_putmem_signal_nbi_wg(rocshmem_ctx_t ctx, void *dest, const void *source, size_t nelems, uint64_t *sig_addr, uint64_t signal, int sig_op, int pe)
:param ctx: Context with which to perform this operation.
:param dest: Destination address. Must be an address on the symmetric heap.
:param source: Source address. Must be an address on the symmetric heap.
:param nelems: The number of bytes to transfer.
:param sig_addr: Signal address. Must be an address on the symmetric heap.
:param signal: Signal value.
:param sig_op: Atomic operation to apply the signal value.
:param pe: PE of the remote process.
:returns: None.
**Description:**
This function writes contiguous data of ``nelems`` bytes from source on the calling PE to ``dest`` at ``pe``,
then applies ``sig_op`` at ``sig_addr`` with the signal value.
Valid ``sig_op values`` are listed in SIGNAL_OPERATORS_.
ROCSHMEM_PUT_SIGNAL
-------------------
.. cpp:function:: __device__ void rocshmem_TYPENAME_put_signal(TYPE *dest, const TYPE *source, size_t nelems, uint64_t *sig_addr, uint64_t signal, int sig_op, int pe)
.. cpp:function:: __device__ void rocshmem_TYPENAME_put_signal_wave(TYPE *dest, const TYPE *source, size_t nelems, uint64_t *sig_addr, uint64_t signal, int sig_op, int pe)
.. cpp:function:: __device__ void rocshmem_TYPENAME_put_signal_wg(TYPE *dest, const TYPE *source, size_t nelems, uint64_t *sig_addr, uint64_t signal, int sig_op, int pe)
.. cpp:function:: __device__ void rocshmem_TYPENAME_put_signal_nbi(TYPE *dest, const TYPE *source, size_t nelems, uint64_t *sig_addr, uint64_t signal, int sig_op, int pe)
.. cpp:function:: __device__ void rocshmem_TYPENAME_put_signal_nbi_wave(TYPE *dest, const TYPE *source, size_t nelems, uint64_t *sig_addr, uint64_t signal, int sig_op, int pe)
.. cpp:function:: __device__ void rocshmem_TYPENAME_put_signal_nbi_wg(TYPE *dest, const TYPE *source, size_t nelems, uint64_t *sig_addr, uint64_t signal, int sig_op, int pe)
.. cpp:function:: __device__ void rocshmem_ctx_TYPENAME_put_signal(rocshmem_ctx_t ctx, TYPE *dest, const TYPE *source, size_t nelems, uint64_t *sig_addr, uint64_t signal, int sig_op, int pe)
.. cpp:function:: __device__ void rocshmem_ctx_TYPENAME_put_signal_wave(rocshmem_ctx_t ctx, TYPE *dest, const TYPE *source, size_t nelems, uint64_t *sig_addr, uint64_t signal, int sig_op, int pe)
.. cpp:function:: __device__ void rocshmem_ctx_TYPENAME_put_signal_wg(rocshmem_ctx_t ctx, TYPE *dest, const TYPE *source, size_t nelems, uint64_t *sig_addr, uint64_t signal, int sig_op, int pe)
.. cpp:function:: __device__ void rocshmem_ctx_TYPENAME_put_signal_nbi(rocshmem_ctx_t ctx, TYPE *dest, const TYPE *source, size_t nelems, uint64_t *sig_addr, uint64_t signal, int sig_op, int pe)
.. cpp:function:: __device__ void rocshmem_ctx_TYPENAME_put_signal_nbi_wave(rocshmem_ctx_t ctx, TYPE *dest, const TYPE *source, size_t nelems, uint64_t *sig_addr, uint64_t signal, int sig_op, int pe)
.. cpp:function:: __device__ void rocshmem_ctx_TYPENAME_put_signal_nbi_wg(rocshmem_ctx_t ctx, TYPE *dest, const TYPE *source, size_t nelems, uint64_t *sig_addr, uint64_t signal, int sig_op, int pe)
:param ctx: Context with which to perform this operation.
:param dest: Destination address. Must be an address on the symmetric heap.
:param source: Source address. Must be an address on the symmetric heap.
:param nelems: The number of elements of size ``TYPE`` to transfer.
:param sig_addr: Signal address. Must be an address on the symmetric heap.
:param signal: Signal value.
:param sig_op: Atomic operation to apply the signal value.
:param pe: PE of the remote process.
:returns: None.
**Description:**
This function writes contiguous data of ``nelems`` elements of ``TYPE`` from source on the calling PE to ``dest`` at ``pe``,
then applies ``sig_op`` at ``sig_addr`` with the signal value.
Valid ``sig_op values`` are listed in SIGNAL_OPERATORS_.
Valid ``TYPENAME`` and ``TYPE`` values are listed in :ref:`RMA_TYPES`.
ROCSHMEM_PUTMEM_SIGNAL_ON_STREAM
---------------------------------
.. cpp:function:: __host__ void rocshmem_putmem_signal_on_stream(void *dest, const void *source, size_t nelems, uint64_t *sig_addr, uint64_t signal, int sig_op, int pe, hipStream_t stream)
:param dest: Destination address on the remote PE. Must be an address on the symmetric heap.
:param source: Source address on the local PE. Must be an address on the symmetric heap.
:param nelems: Size of the transfer in bytes.
:param sig_addr: Address of signal variable on the remote PE. Must be an address on the symmetric heap.
:param signal: Signal value to be written.
:param sig_op: Signal operation (ROCSHMEM_SIGNAL_SET or ROCSHMEM_SIGNAL_ADD).
:param pe: PE number of the remote PE.
:param stream: HIP stream on which to enqueue the operation.
:returns: None.
**Description:**
This routine enqueues a put-with-signal operation on a HIP stream. The function writes contiguous
data of ``nelems`` bytes from source on the calling PE to ``dest`` at ``pe``, then applies ``sig_op``
at ``sig_addr`` with the signal value. The operation is enqueued on the specified stream and will
execute asynchronously. The caller must synchronize the stream (e.g., using ``hipStreamSynchronize``)
to ensure completion.
Valid ``sig_op`` values are listed in SIGNAL_OPERATORS_.
ROCSHMEM_SIGNAL_FETCH
---------------------
.. cpp:function:: __device__ uint64_t rocshmem_signal_fetch(const uint64_t *sig_addr)
.. cpp:function:: __device__ uint64_t rocshmem_signal_fetch_wg(const uint64_t *sig_addr)
.. cpp:function:: __device__ uint64_t rocshmem_signal_fetch_wave(const uint64_t *sig_addr)
:param sig_addr: Signal address. Must be an address on the symmetric heap.
:returns: Value at ``sig_addr``.
**Description:**
This function atomically fetches the value stored at ``sig_addr``.
Signal operators
----------------
.. _SIGNAL_OPERATORS:
.. list-table:: Signal Operators
:widths: 20 40
:header-rows: 1
* - Value
- Description
* - ROCSHMEM_SIGNAL_SET
- The signaling operation routines will atomically set the signal value at ``sig_addr``.
* - ROCSHMEM_SIGNAL_ADD
- The signaling operation routines will atomically add the signal value at ``sig_addr``.
projects/rocshmem/docs/api/teams.rst
+90
Просмотреть файл
@@ -0,0 +1,90 @@
.. meta::
:description: rocSHMEM intra-kernel networking runtime for AMD dGPUs on the ROCm platform.
:keywords: rocSHMEM, API, ROCm, documentation, HIP, Networking, Communication
.. _rocshmem-api-teams:
-------------------------
Team management routines
-------------------------
ROCSHMEM_TEAM_MY_PE
-------------------
.. cpp:function:: __host__ int rocshmem_team_my_pe(rocshmem_team_t team)
:param team: The team to query.
:returns: PE ID of the caller in the provided team.
**Description:**
This routine queries the PE ID of the caller in a team.
ROCSHMEM_TEAM_N_PES
-------------------
.. cpp:function:: __host__ int rocshmem_team_n_pes(rocshmem_team_t team)
:param team: The team to query.
:returns: Number of PEs in the provided team.
**Description:**
This routine queries the number of PEs in a team.
ROCSHMEM_TEAM_TRANSLATE_PE
--------------------------
.. cpp:function:: __host__ int rocshmem_team_translate_pe(rocshmem_team_t src_team, int src_pe, rocshmem_team_t dest_team)
:param src_team: Handle of the team from which to translate.
:param src_pe: PE-of-interest's index in ``src_team``.
:param dest_team: Handle of the team to which to translate.
:returns: PE of ``src_pe`` in ``dest_team``.
If any input is invalid or if ``src_pe`` is
not in both source and destination teams, a value of ``-1`` is returned.
**Description:**
This routine translates the PE in ``src_team`` to that in ``dest_team``.
ROCSHMEM_TEAM_SPLIT_STRIDED
---------------------------
.. cpp:function:: __host__ int rocshmem_team_split_strided(rocshmem_team_t parent_team, int start, int stride, int size, const rocshmem_team_config_t *config, long config_mask, rocshmem_team_t *new_team)
:param parent_team: The team to split from.
:param start: The lowest PE number of the subset of the PEs
from the parent team that will form the new
team.
:param stride: The stride between team PE members in the
parent team that comprise the subset of PEs
that will form the new team.
:param size: The number of PEs in the new team.
:param config: Pointer to the config parameters for the new team.
:param config_mask: Bitwise mask representing parameters to use from config.
:param new_team: Pointer to the newly created team.
If an error occurs during team creation, or if the PE in
the parent team is not in the new team, the value will be
``ROCSHMEM_TEAM_INVALID``.
:returns: Zero upon successful team creation; non-zero if erroneous.
**Description:**
This routine creates a new a team of PEs. It must be called by all PEs in the parent team.
ROCSHMEM_TEAM_DESTROY
---------------------
.. cpp:function:: __host__ void rocshmem_team_destroy(rocshmem_team_t team)
:param team: The team to destroy. The behavior is undefined if
the input team is ``ROCSHMEM_TEAM_WORLD`` or any other
invalid team. If the input is ``ROCSHMEM_TEAM_INVALID``,
this function will not perform any operation.
:returns: None
**Description:**
This routine destroys a team. It must be called by all PEs in the team.
You must destroy all private contexts created in the
team before destroying this team. Otherwise, the behavior
is undefined. This call will destroy only the shareable contexts
created from the referenced team.