SWDEV-490062 - Update documentation
Change-Id: Ib5297fdda2e05795b3b20436cc1de962e310b08b
Этот коммит содержится в:
Istvan Kiss authored and 
Istvan Kiss committed
1 родитель
36739655e4
Коммит
3d60bd3a64
108 files changed
+14036
-2987
No files matched your search
@@ -5,16 +5,16 @@
|
||||
.. _AMD_Compute_Language_Runtimes:
|
||||
|
||||
*******************************************************************************
|
||||
AMD Common Language Runtimes (CLR)
|
||||
AMD common language runtimes (CLR)
|
||||
*******************************************************************************
|
||||
|
||||
CLR contains source codes for AMD's compute languages runtimes: ``HIP`` and ``OpenCL™``.
|
||||
CLR is the part of HIP runtime which is supported on the AMD ROCm platform, it provides a header and runtime library built on top of HIP-Clang compiler.
|
||||
For developers and users, CLR implements HIP runtime APIs including streams, events, and memory APIs, which is a object library that is linked with the application.
|
||||
The source codes for all headers and the library implementation are available on GitHub in the `clr repository <https://github.com/ROCm/clr>`_.
|
||||
The source codes for all headers and the library implementation are available on GitHub in the `CLR repository <https://github.com/ROCm/clr>`_.
|
||||
|
||||
|
||||
Project Organisation
|
||||
Project organization
|
||||
====================
|
||||
|
||||
CLR includes the following source code,
|
||||
|
||||
@@ -5,13 +5,13 @@
|
||||
.. _hardware_implementation:
|
||||
|
||||
*******************************************************************************
|
||||
Hardware Implementation
|
||||
Hardware implementation
|
||||
*******************************************************************************
|
||||
|
||||
This chapter describes the typical hardware implementation of GPUs supported by
|
||||
HIP, and how the :ref:`inherent_thread_model` maps to the hardware.
|
||||
|
||||
Compute Units
|
||||
Compute units
|
||||
=============
|
||||
|
||||
The basic building block of a GPU is a compute unit (CU), also known
|
||||
@@ -79,7 +79,7 @@ instructions of the other branch have to be executed in the same way. The best
|
||||
performance can therefore be achieved when thread divergence is kept to a warp
|
||||
level, i.e. when all threads in a warp take the same execution path.
|
||||
|
||||
Vector Cache
|
||||
Vector cache
|
||||
------------
|
||||
|
||||
The usage of cache on a GPU differs from that on a CPU, as there is less cache
|
||||
@@ -88,7 +88,7 @@ warps in order to reduce the amount of accesses to device memory, and make that
|
||||
memory available for other warps that currently reside on the compute unit, that
|
||||
also need to load those values.
|
||||
|
||||
Local Data Share
|
||||
Local data share
|
||||
----------------
|
||||
|
||||
The local data share is memory that is accessible to all threads within a block.
|
||||
@@ -103,14 +103,14 @@ The scalar unit performs instructions that are uniform within a warp. It
|
||||
thereby improves efficiency and reduces the pressure on the vector ALUs and the
|
||||
vector register file.
|
||||
|
||||
CDNA Architecture
|
||||
CDNA architecture
|
||||
=================
|
||||
|
||||
The general structure of CUs stays mostly as it is in GCN
|
||||
architectures. The most prominent change is the addition of matrix ALUs, which
|
||||
can greatly improve the performance of algorithms involving matrix
|
||||
multiply-accumulate operations for
|
||||
:doc:`int8, float16, bfloat16 or float32<rocm:compatibility/precision-support>`.
|
||||
:doc:`int8, float16, bfloat16 or float32<rocm:reference/precision-support>`.
|
||||
|
||||
.. figure:: ../data/understand/hardware_implementation/cdna3_cu.png
|
||||
:alt: Block diagram showing the structure of a CDNA3 compute unit. It includes
|
||||
@@ -122,7 +122,7 @@ multiply-accumulate operations for
|
||||
|
||||
Block Diagram of a CDNA3 Compute Unit.
|
||||
|
||||
RDNA Architecture
|
||||
RDNA architecture
|
||||
=================
|
||||
|
||||
RDNA makes a fundamental change to CU design, by changing the
|
||||
@@ -145,7 +145,7 @@ an L0 cache.
|
||||
|
||||
Block Diagram of an RDNA3 work group processor.
|
||||
|
||||
Shader Engines
|
||||
Shader engines
|
||||
==============
|
||||
|
||||
For hardware implementation's sake, multiple CUs are grouped
|
||||
|
||||
@@ -5,28 +5,34 @@
|
||||
:keywords: AMD, ROCm, HIP, CUDA, API design
|
||||
|
||||
*******************************************************************************
|
||||
Understanding the HIP programming model
|
||||
HIP programming model
|
||||
*******************************************************************************
|
||||
|
||||
The HIP programming model makes it easy to map data-parallel C/C++ algorithms to
|
||||
massively parallel, wide single instruction, multiple data (SIMD) architectures,
|
||||
such as GPUs. A basic understanding of the underlying device architecture helps you
|
||||
such as GPUs.
|
||||
|
||||
While the model may be expressed in most imperative languages, (for example
|
||||
Python via PyHIP) this document will focus on the original C/C++ API of HIP.
|
||||
|
||||
A basic understanding of the underlying device architecture helps you
|
||||
make efficient use of HIP and general purpose graphics processing unit (GPGPU)
|
||||
programming in general.
|
||||
|
||||
RDNA & CDNA Architecture Summary
|
||||
RDNA & CDNA architecture summary
|
||||
================================
|
||||
|
||||
Most GPU architectures, like RDNA and CDNA, have a hierarchical structure.
|
||||
The innermost piece is a SIMD-enabled vector Arithmetic Logical Unit (ALU).
|
||||
In addition to the vector ALUs, most recent GPUs also house matrix ALUs for
|
||||
accelerating algorithms involving matrix multiply-accumulate operations.
|
||||
AMD GPUs also contain scalar ALUs, that can be used to reduce the load on the
|
||||
vector ALU by performing operations which are uniform for all threads of a warp.
|
||||
GPUs in general are made up of basic building blocks called compute units (CUs),
|
||||
that execute the threads of a kernel. These CUs provide the necessary resources
|
||||
for the threads: the Arithmetic Logical Units (ALUs), register files, caches and
|
||||
shared memory for efficient communication between the threads.
|
||||
|
||||
A set of ALUs, together with register files, caches and shared memory, comprise
|
||||
a larger block, often referred to as a compute unit (CU), e.g. in OpenCL and
|
||||
AMD block diagrams, or as streaming multiprocessor (SM).
|
||||
This design allows for efficient execution of kernels while also being able to
|
||||
scale from small GPUs embedded in APUs with few CUs up to GPUs designed for data
|
||||
centers with hundreds of CUs. Figure :ref:`rdna3_cu` and :ref:`cdna3_cu` show
|
||||
examples of such compute units.
|
||||
|
||||
For architecture details, check :ref:`hardware_implementation`.
|
||||
|
||||
.. _rdna3_cu:
|
||||
|
||||
@@ -50,35 +56,60 @@ AMD block diagrams, or as streaming multiprocessor (SM).
|
||||
|
||||
Block Diagram of a CDNA3 Compute Unit.
|
||||
|
||||
For implementation in hardware, multiple CUs are grouped together into
|
||||
a Shader Engine or Compute Engine, typically sharing some fixed function units or
|
||||
memory subsystem resources.
|
||||
Heterogeneous Programming
|
||||
=========================
|
||||
|
||||
.. _cdna2_gcd:
|
||||
The HIP programming model assumes two execution contexts. One is referred to as
|
||||
*host* while compute kernels execute on a *device*. These contexts have
|
||||
different capabilities, therefor slightly different rules apply. The *host*
|
||||
execution is defined by the C++ abstract machine, while *device* execution
|
||||
follows the :ref:`SIMT model<programming_model_simt>` of HIP. These execution contexts in
|
||||
code are signified by the ``__host__`` and ``__device__`` decorators. There are
|
||||
a few key differences between the two:
|
||||
|
||||
.. figure:: ../data/understand/programming_model/cdna2_gcd.png
|
||||
:alt: Block diagram showing four Compute Engines each with 28 Compute Units
|
||||
inside. These four Compute Engines share one block of L2 Cache. Around
|
||||
them are four Memory Controllers. To the top and bottom of all these are
|
||||
eight blocks of Infinity Fabric Links. Two Video Core Next blocks sit in
|
||||
the top corners. At the very bottom spans a colored section reading
|
||||
Infinity Fabric.
|
||||
* The C++ abstract machine assumes a unified memory address space, meaning that
|
||||
one can always access any given address in memory (assuming the absence of
|
||||
data races). HIP however introduces several memory namespaces, an address
|
||||
from one means nothing in another. Moreover, not all address spaces are
|
||||
accessible from all contexts.
|
||||
|
||||
Block Diagram of a CDNA2 Graphics Compute Die.
|
||||
Looking at :ref:`rdna3_cu` and :ref:`cdna3_cu`, you can see that
|
||||
every CU has an instance of storage backing the namespace ``__shared__``.
|
||||
Even if the host were to have access to these regions of
|
||||
memory, the performance benefits of the segmented memory subsystem are
|
||||
supported by the inability of asynchronous access from the host.
|
||||
|
||||
* Not all C++ language features map cleanly to typical device architectures,
|
||||
some are very expensive (meaning slow) to implement on GPU devices, therefor
|
||||
they are forbidden in device contexts to avoid users tapping into features
|
||||
that unexpectedly decimate their program's performance. Offload devices targeted
|
||||
by HIP aren't general purpose devices, at least not in the sense that a CPU is.
|
||||
HIP focuses on data parallel computations and as such caters to throughput
|
||||
optimized architectures, such as GPUs or accelerators derived from GPU
|
||||
architectures.
|
||||
|
||||
* Asynchrony is at the forefront of the HIP API. Computations launched on the device
|
||||
execute asynchronously with respect to the host, and it is the user's responsibility to
|
||||
synchronize their data dispatch/fetch with computations on the device.
|
||||
|
||||
.. note::
|
||||
HIP does perform implicit synchronization on occasions, more advanced than other
|
||||
APIs such as OpenCL or SYCL, in which the responsibility of synchronization mostly
|
||||
depends on the user.
|
||||
|
||||
.. _programming_model_simt:
|
||||
|
||||
Single Instruction Multiple Threads
|
||||
===================================
|
||||
Single instruction multiple threads (SIMT)
|
||||
==========================================
|
||||
|
||||
The single instruction, multiple threads (SIMT) programming model behind the
|
||||
HIP device-side execution is a middle-ground between SMT (Simultaneous Multi-Threading)
|
||||
programming known from multicore CPUs, and SIMD (Single Instruction, Multiple Data) programming
|
||||
mostly known from exploiting relevant instruction sets on CPUs (for example SSE/AVX/Neon).
|
||||
The SIMT programming model behind the HIP device-side execution is a middle-ground
|
||||
between SMT (Simultaneous Multi-Threading) programming known from multicore CPUs,
|
||||
and SIMD (Single Instruction, Multiple Data) programming mostly known from exploiting
|
||||
relevant instruction sets on CPUs (for example SSE/AVX/Neon).
|
||||
|
||||
A HIP device compiler maps SIMT code written in HIP C++ to an inherently SIMD
|
||||
architecture (like GPUs). This is done by scalarizing the entire kernel and issuing the scalar
|
||||
instructions of multiple kernel instances to each of the SIMD engine lanes, rather
|
||||
instructions of multiple kernel instances (called threads) to each of the SIMD engine lanes, rather
|
||||
than exploiting data parallelism within a single instance of a kernel and spreading
|
||||
identical instructions over the available SIMD engines.
|
||||
|
||||
@@ -97,9 +128,10 @@ Consider the following kernel:
|
||||
|
||||
The incoming four-vector of floating-point values ``b`` is multiplied by a
|
||||
scalar and then added element-wise to the four-vector floating-point values of
|
||||
``a``. On modern SIMD-capable architectures the four-vector ops are expected to
|
||||
compile to a single SIMD instruction. GPU execution of this kernel however will
|
||||
typically look the following:
|
||||
``a``. On modern SIMD-capable architectures, the four-vector ops are expected to
|
||||
compile to a single SIMD instruction. However, GPU execution of this kernel will
|
||||
typically break down the vector elements into 4 separate threads for parallel execution,
|
||||
as seen in the following figure:
|
||||
|
||||
.. _simt:
|
||||
|
||||
@@ -111,46 +143,221 @@ typically look the following:
|
||||
|
||||
Instruction flow of the sample SIMT program.
|
||||
|
||||
In HIP, lanes of a SIMD architecture are fed by mapping threads of a SIMT
|
||||
execution, one thread down each lane of a SIMD engine. Execution parallelism
|
||||
usually isn't exploited from the width of the built-in vector types, but via the
|
||||
thread id constants ``threadIdx.x``, ``blockIdx.x``, etc. For more details,
|
||||
refer to :ref:`inherent_thread_model`.
|
||||
In HIP, lanes of the SIMD architecture are fed by mapping threads of a SIMT
|
||||
execution, one thread down each lane of an SIMD engine. Execution parallelism
|
||||
usually isn't exploited from the width of the built-in vector types, but across multiple threads via the thread ID constants ``threadIdx.x``, ``blockIdx.x``, etc.
|
||||
|
||||
Heterogeneous Programming
|
||||
=========================
|
||||
.. _inherent_thread_model:
|
||||
|
||||
The HIP programming model assumes two execution contexts. One is referred to as
|
||||
*host* while compute kernels execute on a *device*. These contexts have
|
||||
different capabilities, therefor slightly different rules apply. The *host*
|
||||
execution is defined by the C++ abstract machine, while *device* execution
|
||||
follows the HIP model, primarily defined by SIMT. These execution contexts in
|
||||
code are signified by the ``__host__`` and ``__device__`` decorators. There are
|
||||
a few key differences between the two:
|
||||
Inherent thread model
|
||||
=====================
|
||||
|
||||
* The C++ abstract machine assumes a unified memory address space, meaning that
|
||||
one can always access any given address in memory (assuming the absence of
|
||||
data races). HIP however introduces several memory namespaces, an address
|
||||
from one means nothing in another. Moreover, not all address spaces are
|
||||
accessible from all contexts.
|
||||
The SIMT nature of HIP is captured by the ability to execute user-provided
|
||||
device programs, expressed as single-source C/C++ functions or sources compiled
|
||||
online/offline to binaries, in bulk.
|
||||
|
||||
If one were to look at :ref:`cdna2_gcd` and inside the :ref:`cdna3_cu`,
|
||||
every Compute Unit has an instance of storage backing the namespace
|
||||
``__shared__``. Even if the host were to have access to these regions of
|
||||
memory, the performance benefits of the segmented memory subsystem are
|
||||
supported by the inability of asynchronous access from the host.
|
||||
All threads of a kernel are uniquely identified by a set of integral values, called thread IDs.
|
||||
The set of integers identifying a thread relate to the hierarchy in which the threads execute.
|
||||
|
||||
* Not all C++ language features map cleanly to typical device architectures,
|
||||
some are very expensive (meaning slow) to implement on GPU devices, therefor
|
||||
they are forbidden in device contexts to avoid users tapping into features
|
||||
that unexpectedly decimate their program's performance. Offload devices targeted
|
||||
by HIP aren't general purpose devices, at least not in the sense a CPU is.
|
||||
HIP focuses on data parallel computations and as such caters to throughput
|
||||
optimized architectures, such as GPUs or accelerators derived from GPU
|
||||
architectures.
|
||||
The thread hierarchy inherent to how AMD GPUs operate is depicted in the
|
||||
following figure.
|
||||
|
||||
.. _inherent_thread_hierarchy:
|
||||
|
||||
.. figure:: ../data/understand/programming_model/thread_hierarchy.svg
|
||||
:alt: Diagram depicting nested rectangles of varying color. The outermost one
|
||||
titled "Grid", inside sets of uniform rectangles layered on one another
|
||||
titled "Block". Each "Block" containing sets of uniform rectangles
|
||||
layered on one another titled "Warp". Each of the "Warp" titled
|
||||
rectangles filled with downward pointing arrows inside.
|
||||
|
||||
Hierarchy of thread groups.
|
||||
|
||||
Warp (or Wavefront)
|
||||
The innermost grouping of threads is called a warp, or a wavefront in ISA terms. A warp
|
||||
is the most tightly coupled groups of threads, both physically and logically. Threads
|
||||
inside a warp are also called lanes, and the integral value identifying them is the lane ID.
|
||||
|
||||
.. tip::
|
||||
|
||||
Lane IDs aren't queried like other thread IDs, but are user-calculated. As a
|
||||
consequence, they are only as multidimensional as the user interprets the
|
||||
calculated values to be.
|
||||
|
||||
The size of a warp is architecture dependent and always fixed. For AMD GPUs
|
||||
the wavefront is typically 64 threads, though sometimes 32 threads. Warps are
|
||||
signified by the set of communication primitives at their disposal, as
|
||||
discussed in :ref:`warp-cross-lane`.
|
||||
|
||||
.. _inherent_thread_hierarchy_block:
|
||||
|
||||
Block
|
||||
The middle grouping is called a block or thread block. The defining feature
|
||||
of a block is that all threads in a block will share an instance of memory
|
||||
which they may use to share data or synchronize with one another.
|
||||
|
||||
The size of a block is user-configurable but is limited by the queryable
|
||||
capabilities of the executing hardware. The unique ID of the thread within a
|
||||
block is 3-dimensional as provided by the API. When linearizing thread IDs
|
||||
within a block, assume the "fast index" being dimension ``x``, followed by
|
||||
the ``y`` and ``z`` dimensions.
|
||||
|
||||
.. _inherent_thread_hierarchy_grid:
|
||||
|
||||
Grid
|
||||
The outermost grouping is called a grid. A grid manifests as a single
|
||||
dispatch of kernels for execution. The unique ID of each block within a grid
|
||||
is 3-dimensional, as provided by the API and is queryable by every thread
|
||||
within the block.
|
||||
|
||||
Cooperative groups thread model
|
||||
-------------------------------
|
||||
|
||||
The Cooperative groups API introduces new APIs to launch, group, subdivide,
|
||||
synchronize and identify threads, as well as some predefined group-collective
|
||||
algorithms, but most importantly a matching threading model to think in terms of.
|
||||
It relaxes some restrictions of the :ref:`inherent_thread_model` imposed by the
|
||||
strict 1:1 mapping of architectural details to the programming model. Cooperative
|
||||
groups let you define your own set of thread groups which may fit your user-cases
|
||||
better than the defaults defined by the hardware.
|
||||
|
||||
.. note::
|
||||
The implicit groups defined by kernel launch parameters are still available
|
||||
when working with cooperative groups.
|
||||
|
||||
For further information, see :doc:`Cooperative groups </how-to/cooperative_groups>`.
|
||||
|
||||
Memory model
|
||||
============
|
||||
|
||||
The hierarchy of threads introduced by the :ref:`inherent_thread_model` is induced
|
||||
by the memory subsystem of GPUs. The following figure summarizes the memory
|
||||
namespaces and how they relate to the various levels of the threading model.
|
||||
|
||||
.. _memory_hierarchy:
|
||||
|
||||
.. figure:: ../data/understand/programming_model/memory_hierarchy.svg
|
||||
:alt: Diagram depicting nested rectangles of varying color. The outermost one
|
||||
titled "Grid", inside it are two identical rectangles titled "Block",
|
||||
inside them are ones titled "Local" with multiple "Warp" titled rectangles.
|
||||
Blocks have not just Local inside, but also rectangles titled "Shared".
|
||||
Inside the Grid is a rectangle titled "Global" with three others inside:
|
||||
"Surface", "Texture" (same color) and "Constant" (different color).
|
||||
|
||||
Memory hierarchy.
|
||||
|
||||
Local or per-thread memory
|
||||
Read-write storage only visible to the threads defining the given variables,
|
||||
also called per-thread memory. The size of a block for a given kernel, and thereby
|
||||
the number of concurrent warps, are limited by local memory usage.
|
||||
This relates to an important aspect: occupancy. This is the default memory
|
||||
namespace.
|
||||
|
||||
Shared memory
|
||||
Read-write storage visible to all the threads in a given block.
|
||||
|
||||
Global
|
||||
Read-write storage visible to all threads in a given grid. There are
|
||||
specialized versions of global memory with different usage semantics which
|
||||
are typically backed by the same hardware storing global.
|
||||
|
||||
Constant
|
||||
Read-only storage visible to all threads in a given grid. It is a limited
|
||||
segment of global with queryable size.
|
||||
|
||||
Texture
|
||||
Read-only storage visible to all threads in a given grid and accessible
|
||||
through additional APIs.
|
||||
|
||||
Surface
|
||||
A read-write version of texture memory.
|
||||
|
||||
Execution model
|
||||
===============
|
||||
|
||||
HIP programs consist of two distinct scopes:
|
||||
|
||||
* The host-side API running on the host processor. There are two APIs available:
|
||||
|
||||
* The HIP runtime API which enables use of the single-source programming
|
||||
model.
|
||||
|
||||
* The HIP driver API which sits at a lower level and most importantly differs
|
||||
by removing some facilities provided by the runtime API, most
|
||||
importantly around kernel launching and argument setting. It is geared
|
||||
towards implementing abstractions atop, such as the runtime API itself.
|
||||
Offers two additional pieces of functionality not provided by the Runtime
|
||||
API: ``hipModule`` and ``hipCtx`` APIs. For further details, check
|
||||
:doc:`HIP driver API </how-to/hip_porting_driver_api>`.
|
||||
|
||||
* The device-side kernels running on GPUs. Both the host and the device-side
|
||||
APIs have synchronous and asynchronous functions in them.
|
||||
|
||||
.. note::
|
||||
|
||||
The HIP does not present two *separate* APIs link NVIDIA CUDA. HIP only extends
|
||||
the HIP runtime API with new APIs for ``hipModule`` and ``hipCtx``.
|
||||
|
||||
Host-side execution
|
||||
-------------------
|
||||
|
||||
The part of the host-side API which deals with device management and their
|
||||
queries are synchronous. All asynchronous APIs, such as kernel execution, data
|
||||
movement and potentially data allocation/freeing all happen in the context of
|
||||
device streams.
|
||||
|
||||
Streams are FIFO buffers of commands to execute relating to a given device.
|
||||
Commands which enqueue tasks on a stream all return promptly and the command is
|
||||
executed asynchronously. All side effects of a command on a stream are visible
|
||||
to all subsequent commands on the same stream. Multiple streams may point to
|
||||
the same device and those streams may be fed from multiple concurrent host-side
|
||||
threads. Execution on multiple streams may be concurrent but isn't required to
|
||||
be.
|
||||
|
||||
Asynchronous APIs involving a stream all return a stream event which may be
|
||||
used to synchronize the execution of multiple streams. A user may enqueue a
|
||||
barrier onto a stream referencing an event. The barrier will block until
|
||||
the command related to the event does not complete, at which point all
|
||||
side effects of the command shall be visible to commands following the barrier,
|
||||
even if those side effects manifest on different devices.
|
||||
|
||||
Streams also support executing user-defined functions as callbacks on the host.
|
||||
The stream will not launch subsequent commands until the callback completes.
|
||||
|
||||
Device-side execution
|
||||
---------------------
|
||||
|
||||
The SIMT programming model behind the HIP device-side execution is a
|
||||
middle-ground between SMT (Simultaneous Multi-Threading) programming known from
|
||||
multicore CPUs, and SIMD (Single Instruction, Multiple Data) programming
|
||||
mostly known from exploiting relevant instruction sets on CPUs (for example
|
||||
SSE/AVX/Neon).
|
||||
|
||||
Kernel launch
|
||||
-------------
|
||||
|
||||
Kernels may be launched in multiple ways all with different syntaxes and
|
||||
intended use-cases.
|
||||
|
||||
* Using the triple-chevron ``<<<...>>>`` operator on a ``__global__`` annotated
|
||||
function.
|
||||
|
||||
* Using ``hipLaunchKernelGGL()`` on a ``__global__`` annotated function.
|
||||
|
||||
.. tip::
|
||||
|
||||
This name by default is a macro expanding to triple-chevron. In cases where
|
||||
language syntax extensions are undesirable, or where launching templated
|
||||
and/or overloaded kernel functions define the
|
||||
``HIP_TEMPLATE_KERNEL_LAUNCH`` preprocessor macro before including the HIP
|
||||
headers to turn it into a templated function.
|
||||
|
||||
* Using the launch APIs supporting the triple-chevron syntax directly.
|
||||
|
||||
.. caution::
|
||||
|
||||
These APIs are intended to be used/generated by tools such as the HIP
|
||||
compiler itself and not intended towards end-user code. Should you be
|
||||
writing a tool having to launch device code using HIP, consider using these
|
||||
over the alternatives.
|
||||
|
||||
* Asynchrony is at the forefront of the HIP API. Computations launched on the device
|
||||
execute asynchronously with respect to the host, and it is the user's responsibility to
|
||||
synchronize their data dispatch/fetch with computations on the device. HIP
|
||||
does perform implicit synchronization on occasions, more advanced than other APIs such as
|
||||
OpenCL or SYCL, in which the responsibility of synchronization mostly depends on the user.
|
||||
@@ -1,258 +0,0 @@
|
||||
.. meta::
|
||||
:description: This chapter describes the HIP programming model, the contract
|
||||
between the programmer and the compiler/runtime executing the
|
||||
code.
|
||||
:keywords: AMD, ROCm, HIP, CUDA, C++ language extensions
|
||||
|
||||
*******************************************************************************
|
||||
Programming model reference
|
||||
*******************************************************************************
|
||||
|
||||
HIP defines a model for mapping single instruction, multiple threads (SIMT) programs
|
||||
onto various architectures, primarily GPUs. While the model may be expressed
|
||||
in most imperative languages, (for example Python via PyHIP) this document will focus on
|
||||
the original C/C++ API of HIP.
|
||||
|
||||
Threading Model
|
||||
===============
|
||||
|
||||
The SIMT nature of HIP is captured by the ability to execute user-provided
|
||||
device programs, expressed as single-source C/C++ functions or sources compiled
|
||||
online/offline to binaries in bulk.
|
||||
|
||||
Multiple instances of the device program (or kernel) are called threads and may
|
||||
execute in parallel. All uniquely identified by a set of integral values, or thread IDs.
|
||||
The set of integers identifying a thread relate to the hierarchy in which threads execute.
|
||||
|
||||
.. _inherent_thread_model:
|
||||
|
||||
Inherent Thread Model
|
||||
---------------------
|
||||
|
||||
The thread hierarchy inherent to how AMD GPUs operate is depicted in
|
||||
:numref:`inherent_thread_hierarchy`.
|
||||
|
||||
.. _inherent_thread_hierarchy:
|
||||
|
||||
.. figure:: ../data/understand/programming_model_reference/thread_hierarchy.svg
|
||||
:alt: Diagram depicting nested rectangles of varying color. The outermost one
|
||||
titled "Grid", inside sets of uniform rectangles layered on one another
|
||||
titled "Block". Each "Block" containing sets of uniform rectangles
|
||||
layered on one another titled "Warp". Each of the "Warp" titled
|
||||
rectangles filled with downward pointing arrows inside.
|
||||
|
||||
Hierarchy of thread groups.
|
||||
|
||||
Warp
|
||||
The innermost grouping is called a warp, or a wavefront in ISA terms. A warp
|
||||
is the most tightly coupled groups of threads, both physically and logically.
|
||||
|
||||
When referring to threads inside a warp, they may be called lanes, and the
|
||||
integral value identifying them the lane ID. Lane IDs aren't queried like
|
||||
other thread IDs, but are user-calculated. As a consequence they are only as
|
||||
multidimensional as the user interprets the calculated values to be.
|
||||
|
||||
The size of a warp is architecture dependent and always fixed. Warps are
|
||||
signified by the set of communication primitives at their disposal, as
|
||||
discussed in :ref:`warp-cross-lane`.
|
||||
|
||||
Block
|
||||
The middle grouping is called a block or thread block. The defining feature
|
||||
of a block is that all threads in a block will share an instance of memory
|
||||
which they may use to share data or synchronize with one another.
|
||||
|
||||
The size of a block is user-configurable but is maximized by the queryable
|
||||
capabilities of the executing hardware. The unique ID of the thread within a
|
||||
block is 3-dimensional as provided by the API. When linearizing thread IDs
|
||||
within a block, assume the "fast index" being dimension ``x``, followed by
|
||||
the ``y`` and ``z`` dimensions.
|
||||
|
||||
Grid
|
||||
The outermost grouping is called a grid. A grid manifests as a single
|
||||
dispatch of kernels for execution. The unique ID of each block within a grid
|
||||
is 3-dimensional, as provided by the API and is queryable by every thread
|
||||
within the block.
|
||||
|
||||
Cooperative Groups Thread Model
|
||||
-------------------------------
|
||||
|
||||
The Cooperative Groups API introduces new APIs to launch, group, subdivide,
|
||||
synchronize and identify threads, as well as some predefined group-collective
|
||||
algorithms, but most importantly a matching threading model to think in terms
|
||||
of. It relaxes some restrictions of the :ref:`inherent_thread_model`
|
||||
imposed by the strict 1:1 mapping of architectural details to the programming
|
||||
model.
|
||||
|
||||
The rich set of APIs introduced by Cooperative Groups allow the programmer to
|
||||
define their own set of thread groups which may fit their user-cases better than
|
||||
those defined by the hardware. The set of implicit groups by kernel launch
|
||||
parameters are still available.
|
||||
|
||||
The thread hierarchy abstraction of Cooperative Groups manifest as depicted in
|
||||
:numref:`coop_thread_hierarchy`.
|
||||
|
||||
.. _coop_thread_hierarchy:
|
||||
|
||||
.. figure:: ../data/understand/programming_model_reference/thread_hierarchy_coop.svg
|
||||
:alt: Diagram depicting nested rectangles of varying color. The outermost one
|
||||
titled "Grid", inside sets of different sized rectangles layered on
|
||||
one another titled "Block". Each "Block" containing sets of uniform
|
||||
rectangles layered on one another titled "Warp". Each of the "Warp"
|
||||
titled rectangles filled with downward pointing arrows inside.
|
||||
|
||||
Cooperative group thread hierarchy.
|
||||
|
||||
Multi Grid
|
||||
An abstraction of potentially multiple simultaneous launches of
|
||||
the same kernel over multiple devices. Grids inside a multi device kernel
|
||||
launch need not be of uniform size, thus allowing taking into account
|
||||
different device capabilities and preferences.
|
||||
|
||||
.. deprecated:: 5.0
|
||||
|
||||
Grid
|
||||
Same as the :ref:`inherent_thread_model` Grid entity. The ability to
|
||||
synchronize over a grid requires the kernel to be launched using the
|
||||
Cooperative Groups API.
|
||||
|
||||
Block
|
||||
Same as the :ref:`inherent_thread_model` Block entity.
|
||||
|
||||
.. note::
|
||||
|
||||
Explicit warp-level thread handling is absent from the Cooperative Groups API.
|
||||
In order to exploit the known hardware SIMD width on which built-in
|
||||
functionality translates to simpler logic, one may use the group partitioning
|
||||
part of the API, such as ``tiled_partition``.
|
||||
|
||||
Memory Model
|
||||
============
|
||||
|
||||
The hierarchy of threads introduced by :ref:`inherent_thread_model` is induced
|
||||
by the memory subsystem of GPUs. :numref:`memory_hierarchy` summarizes that memory namespaces and
|
||||
how they relate to the various levels of the threading model.
|
||||
|
||||
.. _memory_hierarchy:
|
||||
|
||||
.. figure:: ../data/understand/programming_model_reference/memory_hierarchy.svg
|
||||
:alt: Diagram depicting nested rectangles of varying color. The outermost one
|
||||
titled "Grid", inside it are two identical rectangles titled "Block",
|
||||
inside them are ones titled "Local" with multiple "Warp" titled rectangles.
|
||||
Blocks have not just Local inside, but also rectangles titled "Shared".
|
||||
Inside the Grid is a rectangle titled "Global" with three others inside:
|
||||
"Surface", "Texture" (same color) and "Constant" (different color).
|
||||
|
||||
Memory hierarchy.
|
||||
|
||||
Local or per-thread memory
|
||||
Read-write storage only visible to the threads defining the given variables,
|
||||
also called per-thread memory. The size of a block for a given kernel,
|
||||
the number of concurrent warps are limited by local memory usage.
|
||||
This relates to an important aspect: occupancy. This is the default memory
|
||||
namespace.
|
||||
|
||||
Shared memory
|
||||
Read-write storage visible to all the threads in a given block.
|
||||
|
||||
Global
|
||||
Read-write storage visible to all threads in a given grid. There are
|
||||
specialized versions of global memory with different usage semantics which
|
||||
are typically backed by the same hardware storing global.
|
||||
|
||||
Constant
|
||||
Read-only storage visible to all threads in a given grid. It is a limited
|
||||
segment of global with queryable size.
|
||||
|
||||
Texture
|
||||
Read-only storage visible to all threads in a given grid and accessible
|
||||
through additional APIs.
|
||||
|
||||
Surface
|
||||
A read-write version of texture memory.
|
||||
|
||||
Execution Model
|
||||
===============
|
||||
|
||||
HIP programs consist of two distinct scopes:
|
||||
|
||||
* The host-side API running on the host processor. There are to APIs available:
|
||||
|
||||
* The HIP runtime API which enables use of the single-source programming
|
||||
model.
|
||||
|
||||
* The HIP driver API which sits at a lower level and most importantly differs
|
||||
by removing some facilities provided by the runtime API, most
|
||||
importantly around kernel launching and argument setting. It is geared
|
||||
towards implementing abstractions atop, such as the runtime API itself.
|
||||
|
||||
* The device-side kernels running on GPUs. Both the host and the device-side
|
||||
APIs have synchronous and asynchronous functions in them.
|
||||
|
||||
Host-side execution
|
||||
-------------------
|
||||
|
||||
The part of the host-side API which deals with device management and their
|
||||
queries are synchronous. All asynchronous APIs, such as kernel execution, data
|
||||
movement and potentially data allocation/freeing all happen in the context of
|
||||
device streams.
|
||||
|
||||
Streams are FIFO buffers of commands to execute relating to a given device.
|
||||
Commands which enqueue tasks on a stream all return promptly and the command is
|
||||
executed asynchronously. All side effects of a command on a stream are visible
|
||||
to all subsequent commands on the same stream. Multiple streams may point to
|
||||
the same device and those streams may be fed from multiple concurrent host-side
|
||||
threads. Execution on multiple streams may be concurrent but isn't required to
|
||||
be.
|
||||
|
||||
Asynchronous APIs involving a stream all return a stream event which may be
|
||||
used to synchronize the execution of multiple streams. A user may enqueue a
|
||||
barrier onto a stream referencing an event. The barrier will block until
|
||||
the command related to the event does not complete, at which point all
|
||||
side effects of the command shall be visible to commands following the barrier,
|
||||
even if those side effects manifest on different devices.
|
||||
|
||||
Streams also support executing user-defined functions as callbacks on the host.
|
||||
The stream will not launch subsequent commands until the callback completes.
|
||||
|
||||
Device-side execution
|
||||
---------------------
|
||||
|
||||
The SIMT programming model behind the HIP device-side execution is a
|
||||
middle-ground between SMT (Simultaneous Multi-Threading) programming known from
|
||||
multicore CPUs, and SIMD (Single Instruction, Multiple Data) programming
|
||||
mostly known from exploiting relevant instruction sets on CPUs (for example
|
||||
SSE/AVX/Neon).
|
||||
|
||||
A HIP device compiler maps our SIMT code written in HIP C++ to an inherently
|
||||
SIMD architecture (like GPUs) not by exploiting data parallelism within a
|
||||
single instance of a kernel and spreading identical instructions over the SIMD
|
||||
engines at hand, but by scalarizing the entire kernel and issuing the scalar
|
||||
instructions of multiple kernel instances to each of the SIMD engine lanes.
|
||||
|
||||
Kernel launch
|
||||
-------------
|
||||
|
||||
Kernels may be launched in multiple ways all with different syntaxes and
|
||||
intended use-cases.
|
||||
|
||||
* Using the triple-chevron ``<<<...>>>`` operator on a ``__global__`` annotated
|
||||
function.
|
||||
|
||||
* Using ``hipLaunchKernelGGL()`` on a ``__global__`` annotated function.
|
||||
|
||||
.. tip::
|
||||
|
||||
This name by default is a macro expanding to triple-chevron. In cases where
|
||||
language syntax extensions are undesirable, or where launching templated
|
||||
and/or overloaded kernel functions define the
|
||||
``HIP_TEMPLATE_KERNEL_LAUNCH`` preprocessor macro before including the HIP
|
||||
headers to turn it into a templated function.
|
||||
|
||||
* Using the launch APIs supporting the triple-chevron syntax directly.
|
||||
|
||||
.. caution::
|
||||
|
||||
These APIs are intended to be used/generated by tools such as the HIP
|
||||
compiler itself and not intended towards end-user code. Should you be
|
||||
writing a tool having to launch device code using HIP, consider using these
|
||||
over the alternatives.
|
||||
@@ -0,0 +1,212 @@
|
||||
.. meta::
|
||||
:description: This chapter describes the texture fetching modes of the HIP ecosystem
|
||||
ROCm software.
|
||||
:keywords: AMD, ROCm, HIP, Texture, Texture Fetching
|
||||
|
||||
*******************************************************************************
|
||||
Texture fetching
|
||||
*******************************************************************************
|
||||
|
||||
`Textures <../doxygen/html/group___texture.html>`_ are more than just a buffer
|
||||
interpreted as a 1D, 2D, or 3D array.
|
||||
|
||||
As textures are associated with graphics, they are indexed using floating-point
|
||||
values. The index can be in the range of [0 to size-1] or [0 to 1].
|
||||
|
||||
Depending on the index, texture sampling or texture addressing is performed,
|
||||
which decides the return value.
|
||||
|
||||
**Texture sampling**: When a texture is indexed with a fraction, the queried
|
||||
value is often between two or more texels (texture elements). The sampling
|
||||
method defines what value to return in such cases.
|
||||
|
||||
**Texture addressing**: Sometimes, the index is outside the bounds of the
|
||||
texture. This condition might look like a problem but helps to put a texture on
|
||||
a surface multiple times or to create a visible sign of out-of-bounds indexing,
|
||||
in computer graphics. The addressing mode defines what value to return when
|
||||
indexing a texture out of bounds.
|
||||
|
||||
The different sampling and addressing modes are described in the following
|
||||
sections.
|
||||
|
||||
Here is the sample texture used in this document for demonstration purposes. It
|
||||
is 2x2 texels and indexed in the [0 to 1] range.
|
||||
|
||||
.. figure:: ../data/understand/textures/original.png
|
||||
:width: 150
|
||||
:alt: Sample texture
|
||||
:align: center
|
||||
|
||||
Texture used as example
|
||||
|
||||
Texture sampling
|
||||
===============================================================================
|
||||
|
||||
Texture sampling handles the usage of fractional indices. It is the method that
|
||||
describes, which nearby values will be used, and how they are combined into the
|
||||
resulting value.
|
||||
|
||||
The various texture sampling methods are discussed in the following sections.
|
||||
|
||||
.. _texture_fetching_nearest:
|
||||
|
||||
Nearest point sampling
|
||||
-------------------------------------------------------------------------------
|
||||
|
||||
In this method, the modulo of index is calculated as:
|
||||
|
||||
``tex(x) = T[floor(x)]``
|
||||
|
||||
This is also applicable for 2D and 3D variants.
|
||||
|
||||
This doesn't interpolate between neighboring values, which results in a
|
||||
pixelated look.
|
||||
|
||||
The following image shows a texture stretched to a 4x4 pixel quad but still
|
||||
indexed in the [0 to 1] range. The in-between values are the same as the values
|
||||
of the nearest texel.
|
||||
|
||||
.. figure:: ../data/understand/textures/nearest.png
|
||||
:width: 300
|
||||
:alt: Texture upscaled with nearest point sampling
|
||||
:align: center
|
||||
|
||||
Texture upscaled with nearest point sampling
|
||||
|
||||
.. _texture_fetching_linear:
|
||||
|
||||
Linear filtering
|
||||
-------------------------------------------------------------------------------
|
||||
|
||||
The linear filtering method does a linear interpolation between values. Linear
|
||||
interpolation is used to create a linear transition between two values. The
|
||||
formula used is ``(1-t)P1 + tP2`` where ``P1`` and ``P2`` are the values and
|
||||
``t`` is within the [0 to 1] range.
|
||||
|
||||
In the case of texture sampling the following formulas are used:
|
||||
|
||||
* For one dimensional textures: ``tex(x) = (1-α)T[i] + αT[i+1]``
|
||||
* For two dimensional textures: ``tex(x,y) = (1-α)(1-β)T[i,j] + α(1-β)T[i+1,j] + (1-α)βT[i,j+1] + αβT[i+1,j+1]``
|
||||
* For three dimensional textures: ``tex(x,y,z) = (1-α)(1-β)(1-γ)T[i,j,k] + α(1-β)(1-γ)T[i+1,j,k] + (1-α)β(1-γ)T[i,j+1,k] + αβ(1-γ)T[i+1,j+1,k] + (1-α)(1-β)γT[i,j,k+1] + α(1-β)γT[i+1,j,k+1] + (1-α)βγT[i,j+1,k+1] + αβγT[i+1,j+1,k+1]``
|
||||
|
||||
Where x, y, and, z are the floating-point indices. i, j, and, k are the integer
|
||||
indices and, α, β, and, γ values represent how far along the sampled point is on
|
||||
the three axes. These values are calculated by these formulas: ``i = floor(x')``, ``α = frac(x')``, ``x' = x - 0.5``, ``j = floor(y')``, ``β = frac(y')``, ``y' = y - 0.5``, ``k = floor(z')``, ``γ = frac(z')`` and ``z' = z - 0.5``
|
||||
|
||||
This following image shows a texture stretched out to a 4x4 pixel quad, but
|
||||
still indexed in the [0 to 1] range. The in-between values are interpolated
|
||||
between the neighboring texels.
|
||||
|
||||
.. figure:: ../data/understand/textures/linear.png
|
||||
:width: 300
|
||||
:alt: Texture upscaled with linear filtering
|
||||
:align: center
|
||||
|
||||
Texture upscaled with linear filtering
|
||||
|
||||
Texture addressing
|
||||
===============================================================================
|
||||
|
||||
Texture addressing mode handles the index that is out of bounds of the texture.
|
||||
This mode describes which values of the texture or a preset value to use when
|
||||
the index is out of bounds.
|
||||
|
||||
The following sections describe the various texture addressing methods.
|
||||
|
||||
.. _texture_fetching_border:
|
||||
|
||||
Address mode border
|
||||
-------------------------------------------------------------------------------
|
||||
|
||||
In this method, the texture fetching returns a border value when indexing out of
|
||||
bounds. The border value must be set before texture fetching.
|
||||
|
||||
The following image shows the texture on a 4x4 pixel quad, indexed in the
|
||||
[0 to 3] range. The out-of-bounds values are the border color, which is yellow.
|
||||
|
||||
.. figure:: ../data/understand/textures/border.png
|
||||
:width: 300
|
||||
:alt: Texture with yellow border color
|
||||
:align: center
|
||||
|
||||
Texture with yellow border color.
|
||||
|
||||
The purple lines are not part of the texture. They only denote the edge, where
|
||||
the addressing begins.
|
||||
|
||||
.. _texture_fetching_clamp:
|
||||
|
||||
Address mode clamp
|
||||
-------------------------------------------------------------------------------
|
||||
|
||||
This mode clamps the index between [0 to size-1]. Due to this, when indexing
|
||||
out-of-bounds, the values on the edge of the texture repeat. The clamp mode is
|
||||
the default addressing mode.
|
||||
|
||||
The following image shows the texture on a 4x4 pixel quad, indexed in the
|
||||
[0 to 3] range. The out-of-bounds values are repeating the values at the edge of
|
||||
the texture.
|
||||
|
||||
.. figure:: ../data/understand/textures/clamp.png
|
||||
:width: 300
|
||||
:alt: Texture with clamp addressing
|
||||
:align: center
|
||||
|
||||
Texture with clamp addressing
|
||||
|
||||
The purple lines are not part of the texture. They only denote the edge, where
|
||||
the addressing begins.
|
||||
|
||||
.. _texture_fetching_wrap:
|
||||
|
||||
Address mode wrap
|
||||
-------------------------------------------------------------------------------
|
||||
|
||||
Wrap mode addressing is only available for normalized texture coordinates. In
|
||||
this addressing mode, the fractional part of the index is used:
|
||||
|
||||
``tex(frac(x))``
|
||||
|
||||
This creates a repeating image effect.
|
||||
|
||||
The following image shows the texture on a 4x4 pixel quad, indexed in the
|
||||
[0 to 3] range. The out-of-bounds values are repeating the original texture.
|
||||
|
||||
.. figure:: ../data/understand/textures/wrap.png
|
||||
:width: 300
|
||||
:alt: Texture with wrap addressing
|
||||
:align: center
|
||||
|
||||
Texture with wrap addressing.
|
||||
|
||||
The purple lines are not part of the texture. They only denote the edge, where
|
||||
the addressing begins.
|
||||
|
||||
.. _texture_fetching_mirror:
|
||||
|
||||
Address mode mirror
|
||||
-------------------------------------------------------------------------------
|
||||
|
||||
Similar to the wrap mode the mirror mode is only available for normalized
|
||||
texture coordinates and also creates a repeating image, but mirroring the
|
||||
neighboring instances.
|
||||
|
||||
The formula is the following:
|
||||
|
||||
``tex(frac(x))``, if ``floor(x)`` is even,
|
||||
|
||||
``tex(1 - frac(x))``, if ``floor(x)`` is odd.
|
||||
|
||||
The following image shows the texture on a 4x4 pixel quad, indexed in The
|
||||
[0 to 3] range. The out-of-bounds values are repeating the original texture, but
|
||||
mirrored.
|
||||
|
||||
.. figure:: ../data/understand/textures/mirror.png
|
||||
:width: 300
|
||||
:alt: Texture with mirror addressing
|
||||
:align: center
|
||||
|
||||
Texture with mirror addressing
|
||||
|
||||
The purple lines are not part of the texture. They only denote the edge, where
|
||||
the addressing begins.
|
||||
Reference in new issue
Block a user