From a4e55e52ec63ed3e6b302abaea67a1f264666417 Mon Sep 17 00:00:00 2001
From: "Galantsev, Dmitrii" <dmitrii.galantsev@amd.com>
Date: Wed, 25 Sep 2024 15:10:41 -0500
Subject: [PATCH] README: Add known issues section

Change-Id: I298750fdafed556480271cfce31c3fc88984cf0b
Signed-off-by: Galantsev, Dmitrii <dmitrii.galantsev@amd.com>


[ROCm/rdc commit: 04be1211c1a5f401a88408d492b82fecb04cd1ac]
---
 projects/rdc/README.md | 53 ++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 53 insertions(+)

diff --git a/projects/rdc/README.md b/projects/rdc/README.md
index 435786564e..df1857d1e3 100644
--- a/projects/rdc/README.md
+++ b/projects/rdc/README.md
@@ -19,6 +19,16 @@ Or read the concise guide under authentication/readme.txt
 
 RDC can run on AMD ROCm supported platforms, please refer to the [List of Supported Operating Systems](https://rocm.docs.amd.com/projects/install-on-linux/en/latest/reference/system-requirements.html#supported-operating-systems)
 
+## Important notes
+
+### RocProfiler metrics usage
+
+When using rocprofiler fields (800-899) you must call
+`export HSA_TOOLS_LIB=/opt/rocm/lib/librocprofiler64.so.1`
+before starting a compute load.
+
+[***See: dmon-rocprofiler-fields-return-zeros***](#dmon-rocprofiler-fields-return-zeros)
+
 ## Building RDC from source
 
 ### Dependencies
@@ -168,6 +178,49 @@ RDCI provides command-line interface to all RDC features. This CLI can be run lo
     ./rdci dmon -u -i 0 -c 1 -e 100             ## monitor field 100 on gpu 0 for count of 1
     ./rdci dmon -u -i 0 -c 1 -e 1,2             ## monitor fields 1,2 on gpu 0 for count of 1
 
+## Known issues
+
+### dmon fields return N/A
+
+1. An optional library might be missing. Do you have
+   `/opt/rocm/lib/rdc/librdc_*.so`? Do you have the library it's related to?
+   (rocprofiler, rocruntime...)
+2. The GPU you're using might not be supported. As a rule of thumb - most
+   metrics should work on MI300 and up. Less metrics are supported for MI200.
+   NV21 (aka RX6800) and other consumer GPUs have even less metrics.
+
+### dmon rocprofiler fields return zeros
+
+Due to a rocprofiler limitation - you must set `HSA_TOOLS_LIB` environmental
+variable *before* running a compute job.
+
+If `HSA_TOOLS_LIB` is not set - most rocprofiler metrics will return all zeros.
+
+E.g. Correct output on MI300 using [gpu-burn](https://github.com/ROCm/HIP-Examples/tree/master/gpu-burn)
+
+    # terminal 1
+    rdcd -u
+    # terminal 2
+    export HSA_TOOLS_LIB=/opt/rocm/lib/librocprofiler64.so.1
+    gpu-burn
+    # terminal 3
+    rdci dmon -u -e 800,801,803 -i 0 -c 1
+    # output:
+    # GPU   MN_OCC_PER_CU       MN_OCC_PER_ACT_CU   ACTIVE_WAVES
+    # 0     1683.422            6479.242            32640.000
+
+### `HSA_STATUS_ERROR_OUT_OF_RESOURCES`
+
+error:
+
+    terminate called after throwing an instance of 'std::runtime_error'
+      what():  hsa error code: 4104 HSA_STATUS_ERROR_OUT_OF_RESOURCES: The runtime failed to allocate the necessary resources. This error may also occur when the core runtime library needs to spawn threads or create internal OS-specific events.
+    Aborted (core dumped)
+
+1. Missing groups. Run `groups`. You're expected to have `video` and `render`.
+   This can be fixed with `sudo usermod -aG video,render $USER` followed by
+   logging out and logging back in.
+
 ## Troubleshooting rdcd
 
 - Log messages that can provide useful debug information.