Jonathan R. Madsen
6825578603
* various tweaks
* build updates + cleanup + overlap guard + min addr range
* Library source reorg + miscellaneous tweaks
* Removed unnecessary fwd decls
* Print address range in --print-X pair mode
- hosttrace modifications
- disable instrumenting functions with overlapping sections or multiple entry points by default (control via --allow-overlapping option)
- disable instrumenting functions whose address range < 512 bytes unless a loop is present by default (control via --min-address-range option)
- disable instrumenting functions w/ loops whose address range < 64 bytes (control via --min-loop-address-range)
- Support for wrapping MPI function calls even in binary rewrite mode
- e.g. use gotcha to wrap MPI functions with hosttrace_push_trace and hosttrace_pop_trace
- New timemory only mode --> HOSTTRACE_USE_TIMEMORY=ON
- New timemory + perfetto mode --> HOSTTRACE_USE_PERFETTO=ON + HOSTTRACE_USE_TIMEMORY=ON
- Full support for all timemory components
- parallel-overhead example for measuring the overhead in a MT-parallelized application with very small instrumentation functions
- improvements to output directories for hosttrace exe
- improvements to output directories for hosttrace library
- new hosttrace options
- --print-instrumented <type> prints out the instrumented entities and exits
- --print-available <type> prints out the available instrumentation entities and exits
- --print-overlapping <type> prints out the overlapping entities and exits
- NOTE: <type> above refers to the information printed out, e.g. module name vs. function name vs. module and function name, etc.
[ROCm/rocprofiler-systems commit: 1f15b3070f]
hosttrace: application tracing with static/dynamic binary instrumentation and perfetto
1. load necessary modules
module load julia
module load dyninst
module load rocm/VERSION
2. to install julia packages, only need it the first time.
$ julia
type ']' to get into pkg mode, then
(@v1.6) pkg> add JSON, DataFrames, Dates, CSV, Chain, PrettyTables
3. hosttrace usage
export PATH=$PATH:$HTRACE_PATH/bin
hosttrace --help
4. instrument the binaries
hosttrace -L $HTRACE_PATH/bin/libhosttrace.so -o app.inst -- path_to_your_app
hosttrace -L $HTRACE_PATH/bin/libhosttrace.so -E 'hipApiName|hipGetCmdName' -o libamdhip64.so.4 -- /opt/rocm-VERSION/lib/libamdhip64.so.4
hosttrace -L $HTRACE_PATH/bin/libhosttrace.so -E 'rocr::atomic|rocr::core|rocr::HSA' -o libhsa-runtime64.so.1 -- /opt/rocm-VERSION/lib/libhsa-runtime64.so.1
5. run the app
export LD_LIBRARY_PATH=.:$LD_LIBRARY_PATH
rocprof --hip-trace --roctx-trace --stats ./app.inst
6. you may need to increase the buffer size if the hosttrace.perfetto-trace file is close to 1GB. for example to set it to 10GB.
export HOSTTRACE_BUFFER_SIZE_KB=10240000
7. merge the traces from rocprof and hosttrace
julia $HTRACE_PATH/bin/merge-trace.jl results.json hosttrace.perfetto-trace*
8. another mode of Perfetto tracing is to use system backend. To do it:
in a separate window
pkill traced; traced --background; perfetto --out ./htrace.out --txt -c $HTRACE/roctrace.cfg
then in the app running window do this before running rocprof or other app
export HOSTTRACE_BACKEND_SYSTEM=1
for the merge use the htrace.out
julia $HTRACE_PATH/bin/merge-trace.jl results.json htrace.out