GDA docs style edits (#362)
* Update docs/install.rst
Co-authored-by: yugang-amd <yugang.wang@amd.com>
* Update docs/install.rst
Co-authored-by: yugang-amd <yugang.wang@amd.com>
* Update docs/install.rst
Co-authored-by: yugang-amd <yugang.wang@amd.com>
* Update docs/install.rst
Co-authored-by: yugang-amd <yugang.wang@amd.com>
* Update docs/install.rst
Co-authored-by: yugang-amd <yugang.wang@amd.com>
* Update docs/install.rst
Co-authored-by: yugang-amd <yugang.wang@amd.com>
* Update docs/install.rst
Co-authored-by: yugang-amd <yugang.wang@amd.com>
* Update docs/install.rst
Co-authored-by: yugang-amd <yugang.wang@amd.com>
* Update docs/install.rst
Co-authored-by: yugang-amd <yugang.wang@amd.com>
* Update docs/sphinx/_toc.yml.in
Co-authored-by: yugang-amd <yugang.wang@amd.com>
* Apply suggestions from code review
Co-authored-by: yugang-amd <yugang.wang@amd.com>
---------
Co-authored-by: Yiltan <ytemucin@amd.com>
[ROCm/rocshmem commit: bbad1d8539]
Αυτή η υποβολή περιλαμβάνεται σε:
yugang-amd
GitHub
@@ -37,60 +37,60 @@ control the behavior of rocSHMEM.
|
||||
|
||||
* - | ``ROCSHMEM_BACKEND``
|
||||
| When rocSHMEM is compiled for all backends, this enviroment variable
|
||||
| selects which backend to execute. Default is auto-selection
|
||||
| selects which backend to execute. The default value is an empty string and rocSHMEM auto-selects the most appropriate backend.
|
||||
- `` ``
|
||||
- | ipc: IPC Backend
|
||||
| ro: Reverse Offload Backend
|
||||
| gda: GPU Direct Async Backend
|
||||
- | ``ipc``: IPC Backend
|
||||
| ``ro``: Reverse Offload Backend
|
||||
| ``gda``: GPU Direct Async Backend
|
||||
|
||||
* - | ``ROCSHMEM_UNIQUEID_WITH_MPI``
|
||||
| Defines whether rocSHMEM is expected to use MPI when using the uniqueId based initialization.
|
||||
- ``0``
|
||||
- | 0: Do not use MPI.
|
||||
| 1: Use MPI.
|
||||
- | ``0``: Do not use MPI.
|
||||
| ``1``: Use MPI.
|
||||
|
||||
* - | ``ROCSHMEM_DISABLE_MIXED_IPC``
|
||||
| Defines whether to force using the network conduit even when IPC is available.
|
||||
- ``0``
|
||||
- | 0: Use IPC when available.
|
||||
| 1: Force network conduit.
|
||||
- | ``0``: Use IPC when available.
|
||||
| ``1``: Force network conduit.
|
||||
|
||||
* - | ``ROCSHMEM_USE_IB_HCA``
|
||||
| Defines which NIC that this PE should be bound to
|
||||
| Defines which NIC that this PE should be bound to. The default value is an empty string and rocSHMEM auto-detects the most appropriate NIC.
|
||||
- `` ``
|
||||
- | Example value: `bnxt_re0`
|
||||
- | Example value: ``bnxt_re0``
|
||||
|
||||
* - | ``ROCSHMEM_BOOTSTRAP_SOCKET_IFNAME``
|
||||
| Chooses the interface to bootstrap rocSHMEM with.
|
||||
| Only valid when not using MPI.
|
||||
| The default value is an empty string and rocSHMEM auto-detects the most appropriate interface.
|
||||
- `` ``
|
||||
- | Example value: `eno8303`
|
||||
- | Example value: ``eno8303``
|
||||
|
||||
* - | ``ROCSHMEM_GDA_PROVIDER``
|
||||
| When rocSHMEM is compiled with support for multiple NIC vendors,
|
||||
| the enviroment variable selects the desired provider.
|
||||
| The default value is an empty string and rocSHMEM auto-detects the most appropriate NIC.
|
||||
- `` ``
|
||||
- | bnxt: Broadcom Thor 2
|
||||
| pensando: AMD Pensando Pollara
|
||||
| ionic: AMD Pensando Pollara (alias)
|
||||
| mlx5: Mellanox ConnectX-7
|
||||
- | ``bnxt``: Broadcom Thor 2
|
||||
| ``pensando``: AMD Pensando Pollara
|
||||
| ``ionic``: AMD Pensando Pollara (alias)
|
||||
| ``mlx5``: Mellanox ConnectX-7
|
||||
|
||||
* - | ``ROCSHMEM_GDA_ALTERNATE_QP_PORTS``
|
||||
| Enables/Disables having QPs alternate their mappings across rocSHMEM contexts.
|
||||
| Enables or disables alternating QP mappings across rocSHMEM contexts.
|
||||
- ``1``
|
||||
- | 0: Disabled.
|
||||
| 1: Enabled (helps saturate bandwidth on multiport bonded interfaces).
|
||||
- | ``0``: Disabled.
|
||||
| ``1``: Enabled. This helps saturate bandwidth on multiport bonded interfaces.
|
||||
|
||||
* - | ``ROCSHMEM_GDA_TRAFFIC_CLASS``
|
||||
| When using an NIC with an Ethernet link layer, this assigns the traffic class for our QPs
|
||||
| When using an NIC with an Ethernet link layer, this sets the traffic class for the QPs.
|
||||
- ``0``
|
||||
- The traffic class number.
|
||||
|
||||
* - | ``ROCSHMEM_GDA_PCIE_RELAXED_ORDERING``
|
||||
| Enables PCIe Relaxed Ordering when registering the symmetric heap with the RDMA NICs.
|
||||
- ``0``
|
||||
- | 0: Disabled.
|
||||
| 1: Enabled.
|
||||
- | ``0``: Disabled.
|
||||
| ``1``: Enabled.
|
||||
|
||||
|
||||
@@ -28,7 +28,7 @@ Requirements
|
||||
Other AMD GPUs might function with unknown limitations. For the complete list of supported hardware, see `ROCm System Requirements <https://rocm.docs.amd.com/projects/install-on-linux-internal/en/latest/reference/system-requirements.html>`_.
|
||||
|
||||
* The RO backend requires ROCm-aware Open MPI and UCX. When using the IPC or GDA backends, MPI is optional.
|
||||
For more information on installing ROCm-aware Open MPI and UCX please see :ref:`install-dependencies`.
|
||||
For more information about installing ROCm-aware Open MPI and UCX, see :ref:`install-dependencies`.
|
||||
|
||||
* Inter-node communication requires AMD Pollara IONIC, Broadcom Thor 2, or CX7 Infiniband NICs.
|
||||
|
||||
@@ -39,7 +39,7 @@ rocSHMEM supports the following network backends:
|
||||
|
||||
* The **IPC (Inter-Process Communication)** backend enables fast communication between GPUs on the same host using ROCm inter-process mechanisms. It does not support inter-node communication.
|
||||
* The **RO (Reverse Offload)** backend enables communication between GPUs on different nodes through a NIC, using a host-based proxy to forward communication orders to and from the GPU. RO is built on an MPI-RMA compatibility layer.
|
||||
* The **GDA (GPU Direct Async)** backend enables communication between GPUs on different nodes through a NIC. In this backend the GPU directly interacts with the NIC with no host (CPU) involvement in the critical path of communication.
|
||||
* The **GDA (GPU Direct Async)** backend enables communication between GPUs on different nodes through a NIC. In this backend, the GPU directly interacts with the NIC with no host (CPU) involvement in the critical path of communication.
|
||||
|
||||
You can activate IPC, RO, and GDA backends in the same rocSHMEM build.
|
||||
|
||||
@@ -66,7 +66,7 @@ On Ubuntu, you can install rocSHMEM by running:
|
||||
Building dependencies
|
||||
---------------------
|
||||
|
||||
GDA NIC Dependencies
|
||||
GDA NIC dependencies
|
||||
^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
- GDA on Mellanox NICs should work on any recent version of rdma-core.
|
||||
@@ -76,7 +76,7 @@ GDA NIC Dependencies
|
||||
Building rocSHMEM with MPI (Optional)
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
rocSHMEM requires ROCm-Aware Open MPI and UCX for the RO backend or if you want to run rocSHMEM.
|
||||
rocSHMEM requires ROCm-Aware Open MPI and UCX for the RO backend.
|
||||
MPI is optional with the IPC and GDA backends.
|
||||
Other MPI implementations, such as MPICH, have not been fully tested.
|
||||
|
||||
@@ -119,15 +119,15 @@ For more information about OpenMPI-UCX support, see
|
||||
Installing from source
|
||||
--------------------------------
|
||||
|
||||
You can select between three communication backends at build time for rocSHMEM: IPC, RO, and GDA.
|
||||
You can choose from three communication backends at build time for rocSHMEM: IPC, RO, and GDA.
|
||||
Backend can be combined during build time.
|
||||
|
||||
MPI is not required to build rocSHMEM, if you want to disable MPI you can pass
|
||||
the following flag to the build configs scripts `-DUSE_EXTERNAL_MPI=OFF`.
|
||||
However, it is important to note that this will disable the functional and unit
|
||||
tests as they required MPI to run.
|
||||
MPI is not required to build rocSHMEM. To disable MPI, pass
|
||||
the following flag to the build configuration scripts ``-DUSE_EXTERNAL_MPI=OFF``.
|
||||
However, this will disable the functional and unit
|
||||
tests, as they required MPI to run.
|
||||
|
||||
All Backends build
|
||||
All backends build
|
||||
^^^^^^^^^^^^^^^^^^
|
||||
|
||||
To build and install rocSHMEM with all three backends, run:
|
||||
@@ -144,7 +144,7 @@ The build script passes configuration options to CMake to set up a canonical bui
|
||||
|
||||
.. note::
|
||||
|
||||
This will build rocSHMEM with all backends, you can select the IPC, RO, GDA, or a combination at runtime. However there is a small performance penality for this portability. The other build scripts are recommended if you are trying to achive maximum performance.
|
||||
This builds rocSHMEM with all backends. You can select IPC, RO, GDA, or any combination at runtime. However, this portability can reduce performance, so the other build scripts are recommended if you need maximum performance.
|
||||
|
||||
GDA backend build
|
||||
^^^^^^^^^^^^^^^^^
|
||||
|
||||
@@ -29,7 +29,7 @@ subtrees:
|
||||
- file: api/ctx.rst
|
||||
title: Context management routines
|
||||
- file: api/env_variables.rst
|
||||
title: Environment Variables
|
||||
title: Environment variables
|
||||
- file: api/rma.rst
|
||||
title: Remote memory access routines
|
||||
- file: api/amo.rst
|
||||
|
||||
Αναφορά σε νέο ζήτημα
Block a user