Jonathan R. Madsen
1f15b3070f
* various tweaks * build updates + cleanup + overlap guard + min addr range * Library source reorg + miscellaneous tweaks * Removed unnecessary fwd decls * Print address range in --print-X pair mode - hosttrace modifications - disable instrumenting functions with overlapping sections or multiple entry points by default (control via --allow-overlapping option) - disable instrumenting functions whose address range < 512 bytes unless a loop is present by default (control via --min-address-range option) - disable instrumenting functions w/ loops whose address range < 64 bytes (control via --min-loop-address-range) - Support for wrapping MPI function calls even in binary rewrite mode - e.g. use gotcha to wrap MPI functions with hosttrace_push_trace and hosttrace_pop_trace - New timemory only mode --> HOSTTRACE_USE_TIMEMORY=ON - New timemory + perfetto mode --> HOSTTRACE_USE_PERFETTO=ON + HOSTTRACE_USE_TIMEMORY=ON - Full support for all timemory components - parallel-overhead example for measuring the overhead in a MT-parallelized application with very small instrumentation functions - improvements to output directories for hosttrace exe - improvements to output directories for hosttrace library - new hosttrace options - --print-instrumented <type> prints out the instrumented entities and exits - --print-available <type> prints out the available instrumentation entities and exits - --print-overlapping <type> prints out the overlapping entities and exits - NOTE: <type> above refers to the information printed out, e.g. module name vs. function name vs. module and function name, etc.
hosttrace: application tracing with static/dynamic binary instrumentation and perfetto
1. load necessary modules
module load julia
module load dyninst
module load rocm/VERSION
2. to install julia packages, only need it the first time.
$ julia
type ']' to get into pkg mode, then
(@v1.6) pkg> add JSON, DataFrames, Dates, CSV, Chain, PrettyTables
3. hosttrace usage
export PATH=$PATH:$HTRACE_PATH/bin
hosttrace --help
4. instrument the binaries
hosttrace -L $HTRACE_PATH/bin/libhosttrace.so -o app.inst -- path_to_your_app
hosttrace -L $HTRACE_PATH/bin/libhosttrace.so -E 'hipApiName|hipGetCmdName' -o libamdhip64.so.4 -- /opt/rocm-VERSION/lib/libamdhip64.so.4
hosttrace -L $HTRACE_PATH/bin/libhosttrace.so -E 'rocr::atomic|rocr::core|rocr::HSA' -o libhsa-runtime64.so.1 -- /opt/rocm-VERSION/lib/libhsa-runtime64.so.1
5. run the app
export LD_LIBRARY_PATH=.:$LD_LIBRARY_PATH
rocprof --hip-trace --roctx-trace --stats ./app.inst
6. you may need to increase the buffer size if the hosttrace.perfetto-trace file is close to 1GB. for example to set it to 10GB.
export HOSTTRACE_BUFFER_SIZE_KB=10240000
7. merge the traces from rocprof and hosttrace
julia $HTRACE_PATH/bin/merge-trace.jl results.json hosttrace.perfetto-trace*
8. another mode of Perfetto tracing is to use system backend. To do it:
in a separate window
pkill traced; traced --background; perfetto --out ./htrace.out --txt -c $HTRACE/roctrace.cfg
then in the app running window do this before running rocprof or other app
export HOSTTRACE_BACKEND_SYSTEM=1
for the merge use the htrace.out
julia $HTRACE_PATH/bin/merge-trace.jl results.json htrace.out