readthedocs updates (#877)

* readthedocs updates

* Adding License

* correcting table of contents path

* Move doc requirements to sphinx dir

* Compile requirements.txt

* Update path to reqs

* Adding missing python module

* changing sphinx version

* changing docutils version

* enabling sphinx extensions

* trying sphinx-rtd-theme

* Remove unused doc configs

* Remove unused html theme options

* Add files to toc

* temp commit to test

* updating environment.yml for CI build

* Update doc requirements

To include rocprofiler-sdk in projects.yaml

* Set external_projects_current_project as rocprofiler-sdk

* Exclude external projects

* Fix warning for missing static path

* updating conf.py

* Removing reST syntax

* Use rocm-docs-core doxygen integration

* Remove RST syntax from Markdown files

* Generate doxyfile post checkout on RTD

* Use custom RTD env

* Specify mambaforge

* Put conda before post checkout cmd

* Add doxyfile for RTD

* Run cmake from conf.py

* Update environment.yml

* Use mambaforge

* Fix path to environment.yml

* Call build doxyfile

* Add Developer API title to Doxyfile

* Config version header

* Fix typo in conf.py

* Format fix for conf.py

* Increasing timeout for build-docs-from-source

* Remove README as mainpage for doxyfile

* Fix formatting in conf.py

---------

Co-authored-by: Sam Wu <22262939+samjwu@users.noreply.github.com>

[ROCm/rocprofiler-sdk commit: a60c4def00]
This commit is contained in:
Gopesh Bhardwaj
2024-06-22 00:10:54 +05:30
committed by GitHub
szülő 0114bcad4b
commit 076f4d8427
23 fájl változott, egészen pontosan 305 új sor hozzáadva és 234 régi sor törölve
@@ -101,7 +101,7 @@ jobs:
python3 -m pip install -r requirements.txt
- name: Configure, Build, Install, and Package
timeout-minutes: 30
timeout-minutes: 60
shell: bash
run:
export CMAKE_PREFIX_PATH=/opt/rocm:${CMAKE_PREFIX_PATH};
@@ -0,0 +1,21 @@
# Read the Docs configuration file
# See https://docs.readthedocs.io/en/stable/config-file/v2.html for details
version: 2
sphinx:
configuration: source/docs/conf.py
formats: [htmlzip, pdf, epub]
python:
install:
- requirements: source/docs/sphinx/requirements.txt
build:
os: ubuntu-22.04
tools:
python: "mambaforge-22.9"
conda:
environment: source/docs/environment.yml
@@ -3,3 +3,5 @@
/_doxygen
/.gitinfo
/*.dox
/.sass-cache
/_toc.yml
@@ -94,7 +94,7 @@ conda activate rocprofiler-docs
which python
python -m pip install -r ${CMAKE_CURRENT_LIST_DIR}/requirements.txt
python -m pip install -r ${CMAKE_CURRENT_LIST_DIR}/sphinx/requirements.txt
WORK_DIR=${PROJECT_SOURCE_DIR}/source/docs
SOURCE_DIR=${PROJECT_SOURCE_DIR}
@@ -0,0 +1,23 @@
# Anywhere {branch} is used, the branch name will be substituted.
# These comments will also be removed.
defaults:
numbered: True
maxdepth: 4
root: index
subtrees:
- caption: Table of Contents
entries:
- file: features
- file: installation
- file: tool_library_overview
- file: callback_services
- file: buffered_services
- file: pc_sampling
- file: intercept_table
- file: _doxygen/html/index
- file: samples
- file: rocprofv3
- caption: License
entries:
- file: license
@@ -1,34 +0,0 @@
# About
```eval_rst
.. toctree::
:glob:
:maxdepth: 4
```
## Important Changes
[Roctracer](https://github.com/ROCm/roctracer) and [rocprofiler (v1)](https://github.com/ROCm/rocprofiler)
have been combined into a single rocprofiler SDK and re-designed from scratch. The new rocprofiler API has been designed with some
new restrictions to avoid problems that plagued the former implementations. These restrictions enable more efficient implementations
and much better thread-safety. The most important restriction is the window for tools to inform rocprofiler about which services
the tool wishes to use (where "services" refers to the capabilities for API tracing, kernel tracing, etc.).
In the former implementations, when one of the ROCm runtimes were initially loaded, a tool only had
to inform roctracer/rocprofiler that it wished to use its services at some point (e.g. calling `roctracer_init()`)
and were not required to specify which services it would eventually or potentially use. Thus, these libraries had to effectively prepare for
any service to be enable at any point in time -- which introduced unnecessary overhead when tools had no desire to use certain features and
made thread-safe data management difficult. For example, roctracer was required to _always_ install wrappers around _every_ runtime API function
and _always_ added extra overhead of indirection through the roctracer library and checks for the current service configuration (in a thread-safe manner).
In the re-designed implementation, rocprofiler introduces the concept of a "context". Contexts are effectively
bundles of service configurations. Rocprofiler gives each tool _one_ opportunity to create as many contexts as necessary --
for example, a tool can group all of the services into one context, create individual contexts for each service, or somewhere in between.
Due to this design choice change, rocprofiler now knows _exactly_ which services might be requested by the tool clients at any point in time.
This has several important implications:
- rocprofiler does not have to unnecessarily prepare for services that are never used -- if no registered contexts requested tracing the HSA API, no wrappers need to be generated
- rocprofiler can perform more extensive checks during service specification and inform tools about potential issues very early on
- rocprofiler can allow multiple tools to use certain services simulatenously
- rocprofiler was able to improve thread-safety without introducing parallel bottlenecks
- rocprofiler can manage internal data and allocations more efficiently
@@ -4,12 +4,6 @@ For the buffered approach, supported buffer record categories are enumerated in
## Buffered Tracing Services
```eval_rst
.. toctree::
:glob:
:maxdepth: 4
```
## Overview
In buffered approach, callbacks are receieved for batches of records from an internal (background) thread. Supported buffered tracing services are enumerated in `rocprofiler_buffer_tracing_kind_t`.
@@ -1,11 +1,5 @@
# Callback Tracing Services
```eval_rst
.. toctree::
:glob:
:maxdepth: 4
```
## Overview
## Code Object Tracing
@@ -9,9 +9,6 @@
# If extensions (or modules to document with autodoc) are in another directory,
# add these directories to sys.path here. If the directory is relative to the
# documentation root, use os.path.abspath to make it absolute, like shown here.
#
# import os
# sys.path.insert(0, os.path.abspath('.'))
import os
import sys
@@ -30,10 +27,36 @@ def install(package):
# Check if we're running on Read the Docs' servers
read_the_docs_build = os.environ.get("READTHEDOCS", None) == "True"
_srcdir = os.path.realpath(os.path.join(os.getcwd(), "../.."))
def build_doxyfile():
sp.run(
[
"cmake",
f"-DSOURCE_DIR={_srcdir}",
"-DPROJECT_NAME='Rocprofiler SDK'",
f"-P {_srcdir}/source/docs/generate-doxyfile.cmake",
]
)
def configure_version_header():
sp.run(
[
"cmake",
f"-S {_srcdir}/source/include/rocprofiler-sdk",
f"-B {_srcdir}/source/include/rocprofiler-sdk",
]
)
configure_version_header()
build_doxyfile()
# -- Project information -----------------------------------------------------
project = "Rocprofiler SDK"
copyright = "2023, Advanced Micro Devices, Inc."
copyright = "2023-2024, Advanced Micro Devices, Inc."
author = "Advanced Micro Devices, Inc."
project_root = os.path.normpath(os.path.join(os.getcwd(), "..", ".."))
@@ -41,123 +64,44 @@ version = open(os.path.join(project_root, "VERSION")).read().strip()
# The full version, including alpha/beta/rc tags
release = version
_docdir = os.path.realpath(os.getcwd())
_srcdir = os.path.realpath(os.path.join(os.getcwd(), ".."))
_sitedir = os.path.realpath(os.path.join(os.getcwd(), "..", "site"))
_staticdir = os.path.realpath(os.path.join(_docdir, "_static"))
_templatedir = os.path.realpath(os.path.join(_docdir, "_templates"))
if not os.path.exists(_staticdir):
os.makedirs(_staticdir)
if not os.path.exists(_templatedir):
os.makedirs(_templatedir)
# -- General configuration ---------------------------------------------------
install("sphinx_rtd_theme")
# Add any Sphinx extension module names here, as strings. They can be
# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
# ones.
extensions = [
"sphinx.ext.autodoc",
"sphinx.ext.doctest",
"sphinx.ext.todo",
"sphinx.ext.viewcode",
"sphinx.ext.githubpages",
"sphinx.ext.mathjax",
"sphinx.ext.autosummary",
"sphinx.ext.napoleon",
"sphinx_markdown_tables",
"recommonmark",
"sphinxcontrib.doxylink",
"rocm_docs",
"rocm_docs.doxygen",
]
source_suffix = {
".rst": "restructuredtext",
".md": "markdown",
doxygen_root = "."
doxysphinx_enabled = True
doxygen_project = {
"name": "rocprofiler-sdk",
"path": "_doxygen/xml",
}
doxyfile = "rocprofiler-sdk.dox"
from recommonmark.parser import CommonMarkParser
external_projects_current_project = "rocprofiler-sdk"
external_projects = []
source_parsers = {".md": CommonMarkParser}
master_doc = "index"
exclude_patterns = ["_build", "Thumbs.db", ".DS_Store", "README.md"]
external_toc_path = "./_toc.yml"
# Add any paths that contain templates here, relative to this directory.
templates_path = ["_templates"]
# The master toctree document.
master_doc = "index"
# List of patterns, relative to source directory, that match files and
# directories to ignore when looking for source files.
# This pattern also affects html_static_path and html_extra_path.
exclude_patterns = ["_build", "Thumbs.db", ".DS_Store", "README.md"]
default_role = None
suppress_warnings = ["etoc.toctree"]
# -- Options for HTML output -------------------------------------------------
# The theme to use for HTML and HTML Help pages. See the documentation for
# a list of builtin themes.
#
html_theme = "sphinx_rtd_theme"
html_theme = "rocm_docs_theme"
html_theme_options = {"flavor": "rocm"}
# Add any paths that contain custom static files (such as style sheets) here,
# relative to this directory. They are copied after the builtin static files,
# so a file named "default.css" will overwrite the builtin "default.css".
html_static_path = ["_static"]
html_theme_options = {
# "analytics_id": "G-...", # Provided by Google in your dashboard
"logo_only": False,
"display_version": True,
"prev_next_buttons_location": "bottom",
"style_external_links": False,
# 'style_nav_header_background': 'white',
# Toc options
"collapse_navigation": True,
"sticky_navigation": True,
"navigation_depth": 4,
"includehidden": True,
"titles_only": False,
}
doxygen_root = "_doxygen" # this is just a convenience variable
doxylink = {
"demo": ( # "demo" is the role name that you can later use in sphinx to reference this doxygen documentation (see below)
f"{doxygen_root}/tagfile.xml", # the first parameter of this tuple is the tagfile
f"{doxygen_root}/html", # the second parameter of this tuple is a relative path pointing from
# sphinx output directory to the doxygen output folder inside the output
# directory tree.
# Doxylink will use the tagfile to get the html file name of the symbol you want
# to link and then prefix it with this path to generate html links (<a>-tags).
),
}
from pygments.styles import get_all_styles
# The name of the Pygments (syntax highlighting) style to use.
styles = list(get_all_styles())
preferences = ("emacs", "pastie", "colorful")
for pref in preferences:
if pref in styles:
pygments_style = pref
break
from recommonmark.transform import AutoStructify
# app setup hook
def setup(app):
app.add_config_value(
"recommonmark_config",
{
"auto_toc_tree_section": "Contents",
"enable_eval_rst": True,
"enable_auto_doc_ref": False,
},
True,
)
app.add_transform(AutoStructify)
html_title = f"ROCprofiler-SDK {version} Documentation"
@@ -1,12 +0,0 @@
# Developer API
```eval_rst
.. toctree::
:glob:
:maxdepth: 4
_doxygen/html/topics
_doxygen/html/files
_doxygen/html/annotated
_doxygen/html/classes
```
@@ -127,16 +127,4 @@ dependencies:
- zlib=1.2.13=hd590300_5
- zstd=1.5.5=hfc55251_0
- pip:
- click==8.1.7
- click-log==0.4.0
- doxysphinx==3.3.4
- libsass==0.22.0
- lxml==4.9.3
- mpire==2.8.0
- pyjson5==1.6.4
- pyparsing==3.1.1
- python-dateutil==2.8.2
- six==1.16.0
- sphinx-markdown==1.0.2
- sphinxcontrib-doxylink==1.12.2
- tqdm==4.66.1
- -r ./sphinx/requirements.txt
@@ -1,11 +1,5 @@
# Features
```eval_rst
.. toctree::
:glob:
:maxdepth: 4
```
## Overview
- Improved tool initialization
@@ -1,20 +1,28 @@
# Welcome to the [ROCprofiler](https://github.com/ROCm/rocprofiler-sdk) Documentation!
```eval_rst
.. toctree::
:glob:
:maxdepth: 4
:caption: Table of Contents
## Important Changes
about
features
installation
tool_library_overview
callback_services
buffered_services
pc_sampling
intercept_table
developer_api
samples
rocprofv3
```
[Roctracer](https://github.com/ROCm/roctracer) and [rocprofiler (v1)](https://github.com/ROCm/rocprofiler)
have been combined into a single rocprofiler SDK and re-designed from scratch. The new rocprofiler API has been designed with some
new restrictions to avoid problems that plagued the former implementations. These restrictions enable more efficient implementations
and much better thread-safety. The most important restriction is the window for tools to inform rocprofiler about which services
the tool wishes to use (where "services" refers to the capabilities for API tracing, kernel tracing, etc.).
In the former implementations, when one of the ROCm runtimes were initially loaded, a tool only had
to inform roctracer/rocprofiler that it wished to use its services at some point (e.g. calling `roctracer_init()`)
and were not required to specify which services it would eventually or potentially use. Thus, these libraries had to effectively prepare for
any service to be enable at any point in time -- which introduced unnecessary overhead when tools had no desire to use certain features and
made thread-safe data management difficult. For example, roctracer was required to _always_ install wrappers around _every_ runtime API function
and _always_ added extra overhead of indirection through the roctracer library and checks for the current service configuration (in a thread-safe manner).
In the re-designed implementation, rocprofiler introduces the concept of a "context". Contexts are effectively
bundles of service configurations. Rocprofiler gives each tool _one_ opportunity to create as many contexts as necessary --
for example, a tool can group all of the services into one context, create individual contexts for each service, or somewhere in between.
Due to this design choice change, rocprofiler now knows _exactly_ which services might be requested by the tool clients at any point in time.
This has several important implications:
- rocprofiler does not have to unnecessarily prepare for services that are never used -- if no registered contexts requested tracing the HSA API, no wrappers need to be generated
- rocprofiler can perform more extensive checks during service specification and inform tools about potential issues very early on
- rocprofiler can allow multiple tools to use certain services simulatenously
- rocprofiler was able to improve thread-safety without introducing parallel bottlenecks
- rocprofiler can manage internal data and allocations more efficiently
@@ -1,11 +1,5 @@
# Installation
```eval_rst
.. toctree::
:glob:
:maxdepth: 4
```
## Operating System
ROCprofiler is only supported on Linux. The following distributions are tested:
@@ -1,9 +1,3 @@
# Runtime Intercept Tables
```eval_rst
.. toctree::
:glob:
:maxdepth: 4
```
Discussion on how access the raw runtime intercept tables of HSA and HIP (i.e. ExaTracer requirements by LTTng).
@@ -0,0 +1,5 @@
=======
License
=======
.. include:: ../../LICENSE
@@ -1,2 +0,0 @@
doxysphinx
jsonschema2md
@@ -4,7 +4,7 @@
# Project related configuration options
#---------------------------------------------------------------------------
DOXYFILE_ENCODING = UTF-8
PROJECT_NAME = @PROJECT_NAME@
PROJECT_NAME = @PROJECT_NAME@ Developer API
PROJECT_NUMBER = @ROCPROFILER_VERSION@
PROJECT_BRIEF = "ROCm Profiling API and tools"
PROJECT_LOGO =
@@ -172,7 +172,7 @@ INPUT_FILTER =
FILTER_PATTERNS =
FILTER_SOURCE_FILES = NO
FILTER_SOURCE_PATTERNS =
USE_MDFILE_AS_MAINPAGE = @SOURCE_DIR@/README.md
USE_MDFILE_AS_MAINPAGE =
FORTRAN_COMMENT_AFTER = 72
#---------------------------------------------------------------------------
# Configuration options related to source browsing
@@ -197,10 +197,10 @@ IGNORE_PREFIX =
GENERATE_HTML = YES
HTML_OUTPUT = html
HTML_FILE_EXTENSION = .html
HTML_HEADER =
HTML_FOOTER =
HTML_STYLESHEET =
HTML_EXTRA_STYLESHEET = ../../external/doxygen-awesome-css/doxygen-awesome.css
HTML_HEADER =
HTML_FOOTER =
HTML_STYLESHEET =
HTML_EXTRA_STYLESHEET =
HTML_EXTRA_FILES =
HTML_COLORSTYLE = LIGHT
HTML_COLORSTYLE_HUE = 220
@@ -235,7 +235,7 @@ QHG_LOCATION =
GENERATE_ECLIPSEHELP = NO
ECLIPSE_DOC_ID = org.doxygen.rocprofiler
DISABLE_INDEX = NO
GENERATE_TREEVIEW = YES
GENERATE_TREEVIEW = NO
FULL_SIDEBAR = NO
ENUM_VALUES_PER_LINE = 1
TREEVIEW_WIDTH = 300
@@ -298,7 +298,7 @@ MAN_LINKS = YES
#---------------------------------------------------------------------------
# Configuration options related to the XML output
#---------------------------------------------------------------------------
GENERATE_XML = NO
GENERATE_XML = YES
XML_OUTPUT = xml
XML_PROGRAMLISTING = YES
XML_NS_MEMB_FILE_SCOPE = YES
@@ -355,7 +355,7 @@ SKIP_FUNCTION_MACROS = NO
# Configuration options related to external references
#---------------------------------------------------------------------------
TAGFILES =
GENERATE_TAGFILE = _doxygen/tagfile.xml
GENERATE_TAGFILE = _doxygen/html/tagfile.xml
ALLEXTERNALS = NO
EXTERNAL_GROUPS = YES
EXTERNAL_PAGES = YES
@@ -385,7 +385,7 @@ GRAPHICAL_HIERARCHY = YES
DIRECTORY_GRAPH = YES
DIR_GRAPH_MAX_DEPTH = 1
DOT_IMAGE_FORMAT = svg
INTERACTIVE_SVG = NO
INTERACTIVE_SVG = YES
DOT_PATH = @DOT_EXECUTABLE@
DOTFILE_DIRS =
DIA_PATH =
@@ -178,7 +178,7 @@ To trace HIP runtime APIs, use:
rocprofv3 --hip-trace -- < app_relative_path >
```
**Note: The tracing and counter colleciton options generates an additional agent info file. See** [Agent Info](#agent-info)
**Note: The tracing and counter colleciton options generates an additional agent info file.
The above command generates a `hip_api_trace.csv` file prefixed with the process ID.
@@ -2,12 +2,6 @@
## Running Samples
```eval_rst
.. toctree::
:glob:
:maxdepth: 4
```
Samples and tool can be run in order to see the profiler in action. This section covers on how to build these samples and run the tool.
Once the rocm build is installed, samples are installed under:
@@ -0,0 +1 @@
rocm-docs-core[api_reference]==1.4.0
@@ -0,0 +1,169 @@
#
# This file is autogenerated by pip-compile with Python 3.10
# by the following command:
#
# pip-compile requirements.in
#
accessible-pygments==0.0.5
# via pydata-sphinx-theme
alabaster==0.7.16
# via sphinx
babel==2.15.0
# via
# pydata-sphinx-theme
# sphinx
beautifulsoup4==4.12.3
# via pydata-sphinx-theme
breathe==4.35.0
# via rocm-docs-core
certifi==2024.6.2
# via requests
cffi==1.16.0
# via
# cryptography
# pynacl
charset-normalizer==3.3.2
# via requests
click==8.1.7
# via
# click-log
# doxysphinx
# sphinx-external-toc
click-log==0.4.0
# via doxysphinx
cryptography==42.0.8
# via pyjwt
deprecated==1.2.14
# via pygithub
docutils==0.21.2
# via
# breathe
# myst-parser
# pydata-sphinx-theme
# sphinx
doxysphinx==3.3.8
# via rocm-docs-core
fastjsonschema==2.20.0
# via rocm-docs-core
gitdb==4.0.11
# via gitpython
gitpython==3.1.43
# via rocm-docs-core
idna==3.7
# via requests
imagesize==1.4.1
# via sphinx
jinja2==3.1.4
# via
# myst-parser
# sphinx
libsass==0.22.0
# via doxysphinx
lxml==4.9.4
# via doxysphinx
markdown-it-py==3.0.0
# via
# mdit-py-plugins
# myst-parser
markupsafe==2.1.5
# via jinja2
mdit-py-plugins==0.4.1
# via myst-parser
mdurl==0.1.2
# via markdown-it-py
mpire==2.10.2
# via doxysphinx
myst-parser==3.0.1
# via rocm-docs-core
numpy==1.26.4
# via doxysphinx
packaging==24.1
# via
# pydata-sphinx-theme
# sphinx
pycparser==2.22
# via cffi
pydata-sphinx-theme==0.15.3
# via
# rocm-docs-core
# sphinx-book-theme
pygithub==2.3.0
# via rocm-docs-core
pygments==2.18.0
# via
# accessible-pygments
# mpire
# pydata-sphinx-theme
# sphinx
pyjson5==1.6.6
# via doxysphinx
pyjwt[crypto]==2.8.0
# via pygithub
pynacl==1.5.0
# via pygithub
pyparsing==3.1.2
# via doxysphinx
pyyaml==6.0.1
# via
# myst-parser
# rocm-docs-core
# sphinx-external-toc
requests==2.32.3
# via
# pygithub
# sphinx
rocm-docs-core[api-reference]==1.4.0
# via -r requirements.in
smmap==5.0.1
# via gitdb
snowballstemmer==2.2.0
# via sphinx
soupsieve==2.5
# via beautifulsoup4
sphinx==7.3.7
# via
# breathe
# myst-parser
# pydata-sphinx-theme
# rocm-docs-core
# sphinx-book-theme
# sphinx-copybutton
# sphinx-design
# sphinx-external-toc
# sphinx-notfound-page
sphinx-book-theme==1.1.3
# via rocm-docs-core
sphinx-copybutton==0.5.2
# via rocm-docs-core
sphinx-design==0.6.0
# via rocm-docs-core
sphinx-external-toc==1.0.1
# via rocm-docs-core
sphinx-notfound-page==1.0.2
# via rocm-docs-core
sphinxcontrib-applehelp==1.0.8
# via sphinx
sphinxcontrib-devhelp==1.0.6
# via sphinx
sphinxcontrib-htmlhelp==2.0.5
# via sphinx
sphinxcontrib-jsmath==1.0.1
# via sphinx
sphinxcontrib-qthelp==1.0.7
# via sphinx
sphinxcontrib-serializinghtml==1.1.10
# via sphinx
tomli==2.0.1
# via sphinx
tqdm==4.66.4
# via mpire
typing-extensions==4.12.2
# via
# pydata-sphinx-theme
# pygithub
urllib3==2.2.2
# via
# pygithub
# requests
wrapt==1.16.0
# via deprecated
@@ -1,11 +1,5 @@
# Tool Library
```eval_rst
.. toctree::
:glob:
:maxdepth: 4
```
## Rocprofiler and ROCm Runtimes Design
The ROCm runtimes are now designed to directly communicate with a new library called rocprofiler-register during their initialization. This library does cursory checks