diff --git a/projects/rdc/README.md b/projects/rdc/README.md index 998de13aaa..883d17d2e6 100644 --- a/projects/rdc/README.md +++ b/projects/rdc/README.md @@ -8,10 +8,10 @@ The ROCm™ Data Center Tool simplifies the administration and addresses key inf - Integration with third-party tools - Open source -For up-to-date document and how to start using RDC from pre-built packages, please refer to [**user guide**](https://docs.amd.com/bundle/ROCm-Data-Center-Tool-User-Guide-v5.1/page/Overview.html) +For up-to-date document and how to start using RDC from pre-built packages, please refer to the [**ROCm DataCenter Tool User Guide**](https://docs.amd.com/bundle?name_filter.field=title&name_filter.value=ROCm%20DataCenter%20Tool%20User%20Guide&rpp=10&sort.field=title&sort.value=desc) # Supported platforms -RDC can run on AMD ROCm supported platforms, please refer to [List of Supported Operating Systems](https://rocmdocs.amd.com/en/latest/Current_Release_Notes/Current-Release-Notes.html) +RDC can run on AMD ROCm supported platforms, please refer to the **List of Supported Operating Systems** under [ROCm Release Notes](https://docs.amd.com/bundle?name_filter.field=title&name_filter.value=ROCm%20Release%20Notes&rpp=10&sort.field=title&sort.value=desc) # Building RDC from source @@ -27,12 +27,12 @@ RDC can run on AMD ROCm supported platforms, please refer to [List of Supported AMD ROCm platform (https://github.com/RadeonOpenCompute/ROCm) * It is recommended to install the complete AMD ROCm platform. - For installation instruction see https://rocmdocs.amd.com/en/latest/Current_Release_Notes/Current-Release-Notes.html + For installation instruction see https://docs.amd.com/category/Release%20Documentation * At the minimum, these two components are required (i) AMD ROCm SMI Library (https://github.com/RadeonOpenCompute/rocm_smi_lib) (ii) AMD ROCk Kernel driver (https://github.com/RadeonOpenCompute/ROCK-Kernel-Driver) * For profiler metrics, this component is required: - (i) AMD ROCm Tools (https://github.com/RadeonOpenCompute/rocm_smi_lib) + (i) AMD ROCm Tools (https://github.com/ROCm-Developer-Tools/rocmtools) ## Building gRPC and protoc **NOTE:** gRPC and protoc compiler must be built when building RDC from source as pre-built packages are not available. When installing RDC from a package, gRPC and protoc will be installed from the package. @@ -47,17 +47,19 @@ Download and build gRPC ```bash git clone -b v1.44.0 https://github.com/grpc/grpc cd grpc -git submodule update --init -mkdir -p cmake/build -cd cmake/build +git submodule update --init --recursive +mkdir -p build ``` -By default (without using CMAKE_INSTALL_PREFIX option), the following will install to /usr/local lib, include and bin directories +By default (without using CMAKE_INSTALL_PREFIX option), gRPC will install to /usr/local lib, include and bin directories. +It is highly recommended to install gRPC into a unique directory. +Below example installs gRPC into /opt/grpc ```bash -cmake -DgRPC_INSTALL=ON -DBUILD_SHARED_LIBS=ON <-DCMAKE_INSTALL_PREFIX=> ../.. -make -sudo make install -echo "/lib" | sudo tee /etc/ld.so.conf.d/grpc.conf +export GRPC_ROOT=/opt/grpc +cmake -B build -DgRPC_INSTALL=ON -DBUILD_SHARED_LIBS=ON -DCMAKE_INSTALL_PREFIX="$GRPC_ROOT" +make -C build -j $(nproc) +sudo make -C build install +echo "$GRPC_ROOT" | sudo tee /etc/ld.so.conf.d/grpc.conf ``` ## Building RDC @@ -67,10 +69,11 @@ Clone the RDC source code from GitHub and use CMake to build and install ```bash git clone https://github.com/RadeonOpenCompute/rdc cd rdc -mkdir -p build; cd build -cmake -DGRPC_ROOT="$GRPC_PROTOC_ROOT" <-DCMAKE_INSTALL_PREFIX=> .. -make -make install ## default installation location is /opt/rocm, specify with -DROCM_DIR +mkdir -p build +# default installation location is /opt/rocm, specify with -DROCM_DIR or -DCMAKE_INSTALL_PREFIX +cmake -B build -DGRPC_ROOT="$GRPC_ROOT" +make -C build -j $(nproc) +make -C build install ``` ## Building RDC library only without gRPC (optional) @@ -78,7 +81,7 @@ make install ## default installation location is /opt/rocm, specify with -DROCM_ If only the RDC libraries are needed (i.e. only "embedded mode" is required), the user can choose to not build rdci and rdcd. This will eliminate the need for gRPC and protoc. To build in this way, -DBUILD_STANDALONE=off should be passed on the the cmake command line: ```bash -cmake DBUILD_STANDALONE=off <-DCMAKE_INSTALL_PREFIX=> .. +cmake -B build -DBUILD_STANDALONE=off ``` ## Building RDC library without ROCM Run time (optional) @@ -86,26 +89,22 @@ cmake DBUILD_STANDALONE=off <-DCMAKE_INSTALL_PREFIX=> .. The user can choose to not build RDC diagnostic ROCM Run time. This will eliminate the need for ROCM Run time. To build in this way, -DBUILD_ROCRTEST=off should be passed on the the cmake command line: ```bash -cmake DBUILD_ROCRTEST=off <-DCMAKE_INSTALL_PREFIX=> .. +cmake -B build -DBUILD_ROCRTEST=off ``` ## Update System Library Path -The following commands need to be executed as root (sudo). It may be easiest to put them into a script and then run that script as root: - ```bash -RDC_LIB_DIR=/lib -GRPC_LIB_DIR=/lib -echo "$GRPC_LIB_DIR" > /etc/ld.so.conf.d/x86_64-librdc_client.conf -echo "$GRPC_LIB_DIR"64 >> /etc/ld.so.conf.d/x86_64-librdc_client.conf -echo "$RDC_LIB_DIR" >> /etc/ld.so.conf.d/x86_64-librdc_client.conf -echo "$RDC_LIB_DIR"64 >> /etc/ld.so.conf.d/x86_64-librdc_client.conf +RDC_LIB_DIR=/opt/rocm/rdc/lib +GRPC_LIB_DIR=/opt/grpc/lib +echo -e "${GRPC_LIB_DIR}\n${GRPC_LIB_DIR}64" | sudo tee /etc/ld.so.conf.d/x86_64-librdc_client.conf +echo -e "${RDC_LIB_DIR}\n${RDC_LIB_DIR}64" | sudo tee -a /etc/ld.so.conf.d/x86_64-librdc_client.conf ldconfig ``` # Running RDC RDC supports encrypted communications between clients and servers. The -communication can be configured to be *authenticated* or *not authenticated*. The [**user guide**](https://docs.amd.com/bundle/ROCm-Data-Center-Tool-User-Guide-v5.1/page/Developer_Handbook.html) has information on how to generate and install SSL keys and certificates for authentication. By default, authentication is enabled. +communication can be configured to be *authenticated* or *not authenticated*. The [**user guide**](https://docs.amd.com/bundle/ROCm-Data-Center-Tool-User-Guide-v5.3/page/Developer_Handbook.html) has information on how to generate and install SSL keys and certificates for authentication. By default, authentication is enabled. ## Starting ROCm™ Data Center Daemon (RDCD) For an RDC client application to monitor and/or control a remote system, the RDC server daemon, *rdcd*, must be running on the remote system. *rdcd* can be configured to run with (a) full-capabilities which includes ability to set or change GPU configuration or (b) monitor-only capabilities which limits to monitoring GPU metrics. @@ -114,27 +113,27 @@ For an RDC client application to monitor and/or control a remote system, the RDC When *rdcd* is started from a command-line the *capabilities* are determined by privilege of the *user* starting *rdcd* ```bash +## Note that this environment variable is automatically set in rdc.service ## If RDC_FI_PROF_* metrics are required - you MUST export ROCMTOOLS_METRICS_PATH before starting rdcd -export ROCMTOOLS_METRICS_PATH=/opt/rocm-/libexec/rocmtools/counters/derived_counters.xml - -cd rdc_install_prefix ## If specified in Building RDC section +## NOTE: Replace /opt/rocm with specific rocm version if needed +export ROCMTOOLS_METRICS_PATH=/opt/rocm/libexec/rocmtools/counters/derived_counters.xml ## To run with authentication. Ensure SSL keys are setup properly -## version will be the version number(ex:3.10.0) of ROCm where RDC was pacakged with -/opt/rocm-/rdc/bin/rdcd ## rdcd is started with monitor-only capabilities -sudo /opt/rocm-/rdc/bin/rdcd ## rdcd is started will full-capabilities +## version will be the version number(ex:3.10.0) of ROCm where RDC was packaged with +/opt/rocm/rdc/bin/rdcd ## rdcd is started with monitor-only capabilities +sudo /opt/rocm/rdc/bin/rdcd ## rdcd is started will full-capabilities ## To run without authentication. SSL key & certificates are not required. -## version will be the version number(ex:3.10.0) of ROCm where RDC was pacakged with -/opt/rocm-/rdc/bin/rdcd -u ## rdcd is started with monitor-only capabilities -sudo /opt/rocm-/rdc/bin/rdcd -u ## rdcd is started will full-capabilities +## version will be the version number(ex:3.10.0) of ROCm where RDC was packaged with +/opt/rocm/rdc/bin/rdcd -u ## rdcd is started with monitor-only capabilities +sudo /opt/rocm/rdc/bin/rdcd -u ## rdcd is started will full-capabilities ``` ### Start RDCD using systemd -*rdcd* can be started by using the systemctl command. You can copy /opt/rocm-\/rdc/lib/rdc.service, which is installed with RDC, to the systemd folder. This file has 2 lines that control what *capabilities* with which *rdcd* will run. If left uncommented, rdcd will run with full-capabilities. +*rdcd* can be started by using the systemctl command. You can copy /opt/rocm/rdc/lib/rdc.service, which is installed with RDC, to the systemd folder. This file has 2 lines that control what *capabilities* with which *rdcd* will run. If left uncommented, rdcd will run with full-capabilities. ```bash -## file: /opt/rocm-/rdc/lib/rdc.service +## file: /opt/rocm/rdc/lib/rdc.service ## Comment the following two lines to run with monitor-only capabilities CapabilityBoundingSet=CAP_DAC_OVERRIDE AmbientCapabilities=CAP_DAC_OVERRIDE @@ -145,15 +144,15 @@ systemctl start rdc ## start rdc as systemd service ``` ## Invoke RDC using ROCm™ Data Center Interface (RDCI) -RDCI provides command-line interface to all RDC features. This CLI can be run locally or remotely. Refer to [**user guide**](https://docs.amd.com/bundle/ROCm-Data-Center-Tool-User-Guide-v5.1/page/Feature_Overview.html) for the current list of features. +RDCI provides command-line interface to all RDC features. This CLI can be run locally or remotely. Refer to [**user guide**](https://docs.amd.com/bundle/ROCm-Data-Center-Tool-User-Guide-v5.3/page/Feature_Overview.html) for the current list of features. ```bash ## sample rdci commands to test RDC functionality ## discover devices in a local or remote compute node ## NOTE: option -u (for unauthenticated) is required if rdcd was started in this mode +## Assuming that rdc is installed into /opt/rocm -cd rdc_install_prefix ## If specified in Building RDC section -cd ./opt/rocm-/rdc/bin +cd /opt/rocm/rdc/bin ./rdci discovery -l <-u> ## list available GPUs in localhost ./rdci discovery -l <-u> ## list available GPUs in host machine ./rdci dmon <-u> -l ## list most GPU counters @@ -173,19 +172,20 @@ cd ./opt/rocm-/rdc/bin If rdcd was started as a systemd service, then use journalctl to view rdcd logs ```bash -journalctl -u rdc +sudo journalctl -u rdc ``` To run rdcd with debug log from command-line use version will be the version number(ex:3.10.0) of ROCm where RDC was packaged with ```bash -RDC_LOG=DEBUG /opt/rocm-/rdc/bin/rdcd +RDC_LOG=DEBUG /opt/rocm/rdc/bin/rdcd ``` RDC_LOG=DEBUG also works on rdci ERROR, INFO, DEBUG logging levels are supported +- Reading `RDC_FI_PROF_*` crashes rdcd - All `RDC_FI_PROF_*` metrics return N/A 1. Is `ROCMTOOLS_METRICS_PATH` set? diff --git a/projects/rdc/server/rdc.service.in b/projects/rdc/server/rdc.service.in index 5ae7666903..bd031de4a2 100755 --- a/projects/rdc/server/rdc.service.in +++ b/projects/rdc/server/rdc.service.in @@ -18,6 +18,11 @@ Group=rdc Type=simple +# Needed for accessing fields 700-799 provided by ROCMTools +# At the time of writing - rdcd will crash if these metrics are accessed +# without the following environment variable +Environment=ROCMTOOLS_METRICS_PATH=/@CPACK_PACKAGING_INSTALL_PREFIX@/libexec/rocmtools/counters/derived_counters.xml + CapabilityBoundingSet=CAP_DAC_OVERRIDE AmbientCapabilities=CAP_DAC_OVERRIDE