Merge branch 'amd-develop' into amd-master
Change-Id: I53d5a8916d769c4f0fe60d2ee3b240551da80b4f
(cherry picked from commit 01c523f6c9)
Dieser Commit ist enthalten in:
Maneesh Gupta
committet von
Maneesh Gupta
Maneesh Gupta
Ursprung
83dd6b4bec
Commit
cfdb828e6c
@@ -1,49 +1,177 @@
|
||||
# HIP Bugs
|
||||
# HIP Bugs
|
||||
|
||||
<!-- toc -->
|
||||
|
||||
- [Errors related to undefined reference to `__hcLaunchKernel__***__grid_launch_parm**](#errors-related-to-undefined-reference-to-hclaunchkernel__grid_launch_parm)
|
||||
- [Application hangs after a hipLaunchKernel call](#what-if-i-see-application-hangs-after-a-hiplaunchkernel-call)
|
||||
- [Errors related to undefined reference to `__hcLaunchKernel__***__grid_launch_parm**`](#errors-related-to-undefined-reference-to-__hclaunchkernel____grid_launch_parm)
|
||||
- [What is the current limitation of HIP Generic Grid Launch method?](#what-is-the-current-limitation-of-hip-generic-grid-launch-method)
|
||||
- [Errors related to `no matching constructor`](#errors-related-to-no-matching-constructor)
|
||||
- [HIP is more restrictive in enforcing restrictions](#hip-is-more-restrictive-in-enforcing-restrictions)
|
||||
|
||||
<!-- tocstop -->
|
||||
|
||||
### Errors related to undefined reference to `__hcLaunchKernel__***__grid_launch_parm**
|
||||
### Errors related to undefined reference to `__hcLaunchKernel__***__grid_launch_parm**`
|
||||
|
||||
Some common code practices may lead to hipcc generating a error with the form :
|
||||
undefined reference to `__hcLaunchKernel__ZN15vecAddNamespace6vecAddIidEEv16grid_launch_parmPT0_S3_S3_T_
|
||||
|
||||
To workaround, try:
|
||||
- Avoid calling hcLaunchKernel from a function with the __host__ attribute
|
||||
__host__ MyFunc(…) {
|
||||
hipLaunchKernel(myKernel, …)
|
||||
Suggested workarounds:
|
||||
- Avoid use of static with kernel definition:
|
||||
```c++
|
||||
static __global__ MyKernel
|
||||
- Avoid defining kernels in anonymous namespace
|
||||
```
|
||||
|
||||
- Avoid defining kernels in anonymous namespace :
|
||||
```c++
|
||||
namespace {
|
||||
__global__ MyKernel …
|
||||
- Avoid calling member functions
|
||||
|
||||
If hipLaunchKernel takes parameters that request explicitly memcpy, then it will cause application hang.
|
||||
Reason is that the hipLaunchKernel macro locks the stream.
|
||||
If kernel paramters are actually function calls which invoke other hip apis (i.e. memcpy) to the same stream, then deadlock occurs.
|
||||
|
||||
To workaround, try:
|
||||
Move the function calls so they occur outside the hipLaunchKernel macro, store results in temps, then use the tems inside the kernel.
|
||||
|
||||
__global__ MyKernel
|
||||
}
|
||||
```
|
||||
// Example pseudo code causing system hang:
|
||||
// "bottom[0]->gpu_data()" calls hipMemcpy() implicitly and using the same stream, cause deadlock condition.
|
||||
hipLaunchKernel(HIP_KERNEL_NAME(LRNComputeDiff),dim3(CAFFE_GET_BLOCKS(n_threads)), dim3(CAFFE_HIP_NUM_THREADS), 0, 0, n_threads,
|
||||
bottom[0]->gpu_data());
|
||||
|
||||
// Move "gpu_data()" ouside of hipLaunchKernel to avoid hang.
|
||||
auto bot_gpu_data = bottom[0]->gpu_data();
|
||||
hipLaunchKernel( LRNComputeDiff, dim3(CAFFE_GET_BLOCKS(n_threads)), dim3(CAFFE_HIP_NUM_THREADS), 0, 0, n_threads,
|
||||
bot_gpu_data);
|
||||
|
||||
```
|
||||
|
||||
### What is the current limitation of HIP Generic Grid Launch method?
|
||||
1. __global__ functions cannot be marked as static or put in an unnamed namespace i.e. they cannot be given internal linkage (this would clash with __attribute__((weak)));
|
||||
2. using the macro based dispatch mechanism i.e. hipLaunchKernel* only works for functions that take no more than 20 arguments (this limit can be increased up to 126, and is temporary until we can enable C++14 mode and use variadic generic lambdas); no such limitation applies do dispatching directly through grid_launch.
|
||||
2. using the macro based dispatch mechanism i.e. hipLaunchKernel* only works for functions that take no more than 20 arguments (this limit can be increased up to 126, and is temporary until we can enable C++14 mode and use variadic generic lambdas); no such limitation applies do dispatching directly through grid_launch.
|
||||
|
||||
|
||||
### Errors related to `no matching constructor`
|
||||
|
||||
The symptom is the compiler would complain about errors like `no matching constructor` for classes/structs passed as arguments into a GPU kernel. Often, this is caused by a design limitation in HCC where array-typed member variables inside a class/struct can’t be correctly passed into GPU kernels. To mitigate this issue, a custom serializer/deserializer pair is provided.
|
||||
|
||||
For example, `Foo` in the code snippets below contains an array-typed member variable `table`, which would fail the compiler if used as a kernel argument.
|
||||
|
||||
```
|
||||
struct Foo {
|
||||
// table is an array, which makes foo
|
||||
int table[3];
|
||||
};
|
||||
```
|
||||
|
||||
An workaround is to provide a custom serializer on CPU side, and append the contents of the array as kernel arguments:
|
||||
|
||||
```
|
||||
|
||||
struct Foo {
|
||||
int table[3];
|
||||
|
||||
// user-provided CPU serializer
|
||||
// must append the contents of the array member as kernel arguments
|
||||
#ifdef __HCC__
|
||||
__attribute__((annotate(“serialize”)))
|
||||
void __cxxamp_serialize(Kalmar::Serialize &s) const {
|
||||
for (int i = 0; i < 3; ++i)
|
||||
s.Append(sizeof(int), &table[i]);
|
||||
}
|
||||
#endif
|
||||
};
|
||||
```
|
||||
|
||||
Then, provide a custom deserializer on GPU side, to help reconstruct the array within GPU kernels. Notice that the deserializer can not be a function template, and should have scalar-typed parameters of the number equals to the length of the array-typed member variable. For example:
|
||||
|
||||
```
|
||||
struct Foo {
|
||||
int table[3];
|
||||
|
||||
// user-provided GPU deserializer
|
||||
// table has 3 int elements, so deserializer must have 3 int parameters.
|
||||
#ifdef __HCC__
|
||||
__attribute__((annotate(“user_deserialize”)))
|
||||
Foo(int x0, int x1, int x2) [[cpu]][[hc]] {
|
||||
table[0] = x0;
|
||||
table[1] = x1;
|
||||
table[2] = x2;
|
||||
}
|
||||
#endif
|
||||
|
||||
#ifdef __HCC__
|
||||
__attribute__((annotate(“serialize”)))
|
||||
void __cxxamp_serialize(Kalmar::Serialize &s) const {
|
||||
s.Append(sizeof(int), &table[0]);
|
||||
s.Append(sizeof(int), &table[1]);
|
||||
s.Append(sizeof(int), &table[2]);
|
||||
}
|
||||
#endif
|
||||
};
|
||||
```
|
||||
|
||||
|
||||
Rather than create serializer functions, another workaround is to pass the member fields from the structure as simple data types.
|
||||
|
||||
|
||||
### HIP is more restrictive in enforcing restrictions
|
||||
The language specification for HIP and CUDA forbid calling a
|
||||
`__device__` function in a `__host__` context. In practice, you may observe
|
||||
differences in the strictness of this restriction, with HIP exhibiting a tighter
|
||||
adherence to the specification and thus less tolerant of infringing code. The
|
||||
solution is to ensure that all functions which are called in a
|
||||
`__device__` context are correctly annotated to reflect it. An interesting case
|
||||
where these differences emerge is shown below. This relies on a the common
|
||||
[C++ Member Detector idiom][1], as it would be implemented pre C++11):
|
||||
|
||||
```c++
|
||||
#include <cassert>
|
||||
#include <type_traits>
|
||||
|
||||
struct aye { bool a[1]; };
|
||||
struct nay { bool a[2]; };
|
||||
|
||||
// Dual restriction is necessary in HIP if the detector is to work for
|
||||
// __device__ contexts as well as __host__ ones. NVCC is less strict.
|
||||
template<typename T>
|
||||
__host__ __device__
|
||||
const T& cref_t();
|
||||
|
||||
template<typename T>
|
||||
struct Has_call_operator {
|
||||
// Dual restriction is necessary in HIP if the detector is to work for
|
||||
// __device__ contexts as well as __host__ ones. NVCC is less strict.
|
||||
template<typename C>
|
||||
__host__ __device__
|
||||
static
|
||||
aye test(
|
||||
C const *,
|
||||
typename std::enable_if<
|
||||
(sizeof(cref_t<C>().operator()()) > 0)>::type* = nullptr);
|
||||
static
|
||||
nay test(...);
|
||||
|
||||
enum { value = sizeof(test(static_cast<T*>(0))) == sizeof(aye) };
|
||||
};
|
||||
|
||||
template<typename T, typename U, bool callable = has_call_operator<U>::value>
|
||||
struct Wrapper {
|
||||
template<typename V>
|
||||
V f() const { return T{1}; }
|
||||
};
|
||||
|
||||
|
||||
template<typename T, typename U>
|
||||
struct Wrapper<T, U, true> {
|
||||
template<typename V>
|
||||
V f() const { return T{10}; }
|
||||
};
|
||||
|
||||
// This specialisation will yield a compile-time error, if selected.
|
||||
template<typename T, typename U>
|
||||
struct Wrapper<T, U, false> {};
|
||||
|
||||
template<typename T>
|
||||
struct Functor;
|
||||
|
||||
template<> struct Functor<float> {
|
||||
__device__
|
||||
float operator()() const { return 42.0f; }
|
||||
};
|
||||
|
||||
__device__
|
||||
void this_will_not_compile_if_detector_is_not_marked_device()
|
||||
{
|
||||
float f = Wrapper<float, Functor<float>>().f<float>();
|
||||
}
|
||||
|
||||
__host__
|
||||
void this_will_not_compile_if_detector_is_marked_device_only()
|
||||
{
|
||||
float f = Wrapper<float, Functor<float>>().f<float>();
|
||||
}
|
||||
```
|
||||
[1]: https://en.wikibooks.org/wiki/More_C%2B%2B_Idioms/Member_Detector
|
||||
|
||||
@@ -4,7 +4,7 @@
|
||||
|
||||
- [What APIs and features does HIP support?](#what-apis-and-features-does-hip-support)
|
||||
- [What is not supported?](#what-is-not-supported)
|
||||
* [Run-time features](#run-time-features)
|
||||
* [Runtime/Driver API features](#runtimedriver-api-features)
|
||||
* [Kernel language features](#kernel-language-features)
|
||||
- [Is HIP a drop-in replacement for CUDA?](#is-hip-a-drop-in-replacement-for-cuda)
|
||||
- [What specific version of CUDA does HIP support?](#what-specific-version-of-cuda-does-hip-support)
|
||||
@@ -23,10 +23,11 @@
|
||||
- [On HCC, can I link HIP code with host code compiled with another compiler such as gcc, icc, or clang ?](#on-hcc-can-i-link-hip-code-with-host-code-compiled-with-another-compiler-such-as-gcc-icc-or-clang-)
|
||||
- [HIP detected my platform (hcc vs nvcc) incorrectly - what should I do?](#hip-detected-my-platform-hcc-vs-nvcc-incorrectly---what-should-i-do)
|
||||
- [Can I install both CUDA SDK and HCC on same machine?](#can-i-install-both-cuda-sdk-and-hcc-on-same-machine)
|
||||
- [On CUDA, can I mix CUDA code with HIP code?](#on-cuda-can-i-mix-cuda-code-with-hip-code)
|
||||
- [On HCC, can I use HC functionality with HIP?](#on-hcc-can-i-use-hc-functionality-with-hip)
|
||||
- [How do I trace HIP application flow?](#how-do-i-trace-hip-application-flow)
|
||||
* [Using CodeXL markers for HIP Functions](#using-codexl-markers-for-hip-functions)
|
||||
* [Using HIP_TRACE_API](#using-hip_trace_api)
|
||||
- [How do I enable HIP Generic Grid Launch option?](#how-do-i-enable-hip-generic-grid-launch-option)
|
||||
- [What if HIP generates error of "symbol multiply defined!" only on AMD machine?](#what-if-hip-generates-error-of-symbol-multiply-defined-only-on-amd-machine)
|
||||
- [How do I disable HIP Generic Grid Launch option?](#how-do-i-disable-hip-generic-grid-launch-option)
|
||||
|
||||
<!-- tocstop -->
|
||||
|
||||
|
||||
@@ -44,6 +44,7 @@
|
||||
- [Pragma Unroll](#pragma-unroll)
|
||||
- [In-Line Assembly](#in-line-assembly)
|
||||
- [C++ Support](#c-support)
|
||||
- [Kernel Compilation](#kernel-compilation)
|
||||
|
||||
<!-- tocstop -->
|
||||
|
||||
|
||||
@@ -21,6 +21,7 @@ and provides practical suggestions on how to port CUDA code and work through com
|
||||
* [Device-Architecture Properties](#device-architecture-properties)
|
||||
* [Table of Architecture Properties](#table-of-architecture-properties)
|
||||
- [Finding HIP](#finding-hip)
|
||||
- [hipLaunchKernel](#hiplaunchkernel)
|
||||
- [Compiler Options](#compiler-options)
|
||||
- [Linking Issues](#linking-issues)
|
||||
* [Linking With hipcc](#linking-with-hipcc)
|
||||
@@ -31,9 +32,11 @@ and provides practical suggestions on how to port CUDA code and work through com
|
||||
* [Using a Standard C++ Compiler](#using-a-standard-c-compiler)
|
||||
+ [cuda.h](#cudah)
|
||||
* [Choosing HIP File Extensions](#choosing-hip-file-extensions)
|
||||
* [Workarounds](#workarounds)
|
||||
+ [warpSize](#warpsize)
|
||||
+ [Textures and Cache Control](#textures-and-cache-control)
|
||||
- [Workarounds](#workarounds)
|
||||
* [warpSize](#warpsize)
|
||||
- [memcpyToSymbol](#memcpytosymbol)
|
||||
- [threadfence_system](#threadfence_system)
|
||||
* [Textures and Cache Control](#textures-and-cache-control)
|
||||
- [More Tips](#more-tips)
|
||||
* [HIPTRACE Mode](#hiptrace-mode)
|
||||
* [Environment Variables](#environment-variables)
|
||||
|
||||
@@ -4,26 +4,32 @@ This section describes the profiling and debugging capabilities that HIP provide
|
||||
Profiling information can viewed in the CodeXL visualization tool or printed directly to stderr as the application runs.
|
||||
This document starts with some of the general capabilities of CodeXL and then describes some of the additional HIP marker and debug features.
|
||||
|
||||
* [CodeXL Profiling](#codexl-profiling)
|
||||
* [Collecting and Viewing Traces](#collecting-and-viewing-traces)
|
||||
* [Using rocm-profiler timestamp profiling](#using-rocm-profiler-timestamp-profiling)
|
||||
* [Using rocm-profiler performance counter collection:](#using-rocm-profiler-performance-counter-collection)
|
||||
* [Using CodeXL to view profiling results:](#using-codexl-to-view-profiling-results)
|
||||
* [More information on CodeXL](#more-information-on-codexl)
|
||||
* [HIP Markers](#hip-markers)
|
||||
* [Profiling HIP APIs](#profiling-hip-apis)
|
||||
* [Adding markers to applications](#adding-markers-to-applications)
|
||||
* [Additional HIP Profiling Features](#additional-hip-profiling-features)
|
||||
* [Demangling C Kernel Names](#demangling-c-kernel-names)
|
||||
* [Controlling when profiling starts and ends](#controlling-when-profiling-starts-and-ends)
|
||||
* [Reducing timeline trace output file size](#reducing-timeline-trace-output-file-size)
|
||||
* [How to enable profiling at HIP build time](#how-to-enable-profiling-at-hip-build-time)
|
||||
* [Tracing and Debug](#tracing-and-debug)
|
||||
* [Tracing HIP APIs](#tracing-hip-apis)
|
||||
* [Color](#color)
|
||||
* [Using HIP_DB](#using-hip_db)
|
||||
* [Using ltrace](#using-ltrace)
|
||||
* [Chicken bits](#chicken-bits)
|
||||
<!-- toc -->
|
||||
|
||||
- [CodeXL Profiling](#codexl-profiling)
|
||||
* [Collecting and Viewing Traces](#collecting-and-viewing-traces)
|
||||
+ [Using rocm-profiler timestamp profiling](#using-rocm-profiler-timestamp-profiling)
|
||||
+ [Using rocm-profiler performance counter collection:](#using-rocm-profiler-performance-counter-collection)
|
||||
+ [Using CodeXL to view profiling results:](#using-codexl-to-view-profiling-results)
|
||||
+ [More information on CodeXL](#more-information-on-codexl)
|
||||
* [HIP Markers](#hip-markers)
|
||||
+ [Profiling HIP APIs](#profiling-hip-apis)
|
||||
+ [Adding markers to applications](#adding-markers-to-applications)
|
||||
* [Additional HIP Profiling Features](#additional-hip-profiling-features)
|
||||
+ [Demangling C++ Kernel Names](#demangling-c-kernel-names)
|
||||
+ [Controlling when profiling starts and ends](#controlling-when-profiling-starts-and-ends)
|
||||
+ [Reducing timeline trace output file size](#reducing-timeline-trace-output-file-size)
|
||||
+ [How to enable profiling at HIP build time](#how-to-enable-profiling-at-hip-build-time)
|
||||
- [Tracing and Debug](#tracing-and-debug)
|
||||
* [Tracing HIP APIs](#tracing-hip-apis)
|
||||
+ [Color](#color)
|
||||
* [Using HIP_DB](#using-hip_db)
|
||||
* [Using ltrace](#using-ltrace)
|
||||
* [Chicken bits](#chicken-bits)
|
||||
* [Debugging HIP Applications](#debugging-hip-applications)
|
||||
* [General Debugging Tips](#general-debugging-tips)
|
||||
|
||||
<!-- tocstop -->
|
||||
|
||||
## CodeXL Profiling
|
||||
|
||||
|
||||
In neuem Issue referenzieren
Einen Benutzer sperren