* Improved sampling performance
* Sampling tweaks
- backtrace::get() returns vector of string_view
- further performance improvements
- tweaked _use_label
- wrapped samples in perfetto "samples [omnitrace]" block
- samples in TID=0 in sampling mode are in separate thread row
* Fix empty HW counter desc + category for sampling
- fallback to to metric name if papi event info description not found
- add perfetto sampling category
* Limit the SIGALRM frequency
- found when using ROCm-enabled OpenMPI with rocHPL
- when wrapping pthread_rwlock_rdlock, pthread_rwlock_wrlock, and pthread_rwlock_unlock, omnitrace has been found to deadlock for some unknown reason
- New configuration variable: OMNITRACE_TRACE_THREAD_RW_LOCKS which defaults to false
- handle OMNITRACE_ENABLED=OFF by disabling everything
- use set_data to get wrappee in pthread_create_gotcha
- clear roctracer_data storage if roctracer not initialized
* HIP API perfetto args + updated perfetto categories
- Support for HIP API args field in perfetto
- PERFETTO_CATEGORIES -> OMNITRACE_PERFETTO_CATEGORIES
- Changed perfetto categories for several trace events and trace counters
- migrated several TRACE_EVENT_* to use omnitrace::tracing::{push,pop}_perfetto_ts(...)
* Tweaked category_region to encode the type of args as well as value
- Affects MPI args field in perfetto
* Improved testing in ubuntu-focal.yml
- "Test Install" step sources setup-env.sh
- "Test Install" step tests python support
- "Test Install" step tests reading ~/.omnitrace.cfg
- Avoid installing boost and tbb libs when building from submodule
* validate-perfetto-proto.py accepts -m / --categories
* Remove reference from category_region typeids
* Tweak opensuse action name
* Tweak the "Test Install" Step of ubuntu-focal
* Use concurrency + remove cancelling
* Combined ubuntu-focal-external with ubuntu-focal-external-rocm
* Update workflow names and README
* Update ubuntu-focal name
* More tweaks to the name
* Change names in formatting workflow
* Adds test which validates errors for missing configs
- updates timemory submodule with fix for protection against recursion
* Removed PASS_REGULAR_EXPRESSION
- this seems to have different behaviors depending on cmake version
* Rework submodule installation
- use add_subdirectory(... EXCLUDE_FROM_ALL) + explicit installation of deps
- install all library deps to lib/omnitrace
- internal builds of dyninst use libomnitrace-rt for binary rewriting
- support libdyninstAPI_RT not in LD_LIBRARY_PATH when dyninst built internally
* Update ubuntu-focal to test full dyninst install
* Use RelWithDebInfo because Dyninst segfaults with MinSizeRel
* Fix ubuntu-focal.yml install step
* Config updates
- See PR #69 for details
- change type of OMNITRACE_DL_VERBOSE
- add "deprecated" category to OMNITRACE_ROCM_SMI_DEVICES
- reduce size of perfetto shared memory size hint
- deprecate OMNITRACE_OUTPUT_FILE in favor of OMNITRACE_PERFETTO_FILE
- set papi event choices
- read config file after reading command line
- fix update of OMNITRACE_DL_VERBOSE
- mark several settings as hidden
- timemory update support hidden attribute for settings
- rework get_perfetto_output_filename()
- Hide settings from not available backends
* Rework omnitrace-avail to support dumping configurations
* Overwrite query, tests, output flag
- Support using -O flag when dumping config
- Support checking before overwriting existing config
- Support --force to overwrite existing config
- Fix get_component_info not including omnitrace components
- Testing for dumping config
* Update documentation on omnitrace-avail
* Fix issue with timemory format + "/__w/"
* Update output prefix keys docs
* Rename --dump-config to --generate-config
* Hide MPI related options
- OMNITRACE_PERFETTO_COMBINE_TRACES and OMNITRACE_COLLAPSE_PROCESSES are hidden w/o MPI support
* Fix to pid via mpi_gotcha
* OMNITRACE_VERSION defines
* call perfetto on hsa_activity_callback thread
* Test label tweak
* Config fixes
- Change type of OMNITRACE_DL_VERBOSE
- Update OMNITRACE_DL_VERBOSE properly
- Add OMNITRACE_ROCM_SMI_DEVICES to deprecated group
- Set default_process_suffix
* metadata for OMNITRACE_VERSION and OMNITRACE_HIP_VERSION
tracing NS + category region component
- made library.cpp impl more broadly available
- support for perfetto args
- MPI wrappers encode args and return type
- new categories / perfetto categories
- omnitrace_library category -> libomnitrace
- omnitrace_dl_library -> libomnitrace-dl
* adding perfetto-validation-script
* Rename validation script
* Update tests/validate_perfetto.py
Co-authored-by: Jonathan R. Madsen <jrmadsen@users.noreply.github.com>
* Update tests/validate_perfetto.py
Co-authored-by: Jonathan R. Madsen <jrmadsen@users.noreply.github.com>
* Update tests/validate_perfetto.py
Co-authored-by: Jonathan R. Madsen <jrmadsen@users.noreply.github.com>
* Update tests/validate_perfetto.py
Co-authored-by: Jonathan R. Madsen <jrmadsen@users.noreply.github.com>
* addressed the edits in the validation script
* addressed the edits in the validation script
* Perfetto validation script (#1)
* Fixed mismatch message in validate-timemory-json
* validate_perfetto.py -> validate-perfetto-proto.py
* Add python-source-validate-perfetto
- python-source-validate-perfetto uses validate-python-proto.py to validate perfetto output
- renamed python-source-check test to python-source-validate timemory
* Moved python-source-validate tests outside of cat command if block
- these tests don't rely on OMNITRACE_CAT_COMMAND
* CMake/CTest OMNITRACE_ADD_PYTHON_VALIDATION_TEST function
- generalized function for performing validation test with validate-{timemory-json,perfetto-proto}.py scripts
* Print perfetto validation
* Install perfetto python package in workflows
* cmake format
* Python formatting
* Python formatting CI
* Install perfetto python package in workflows
* Install dataclasses for perfetto in opensuse
* Install dataclasses for perfetto in ubuntu
- uninstalled dependency for perfetto in Python 3.6
Co-authored-by: Jonathan R. Madsen <jrmadsen@users.noreply.github.com>
Co-authored-by: Jonathan R. Madsen <jonathanrmadsen@gmail.com>
* Update timemory submodule
* Update timemory submodule
- updates timemory's pybind11 submodule due to VS 2022 debug issue
* Drop a commit in timemory submodule
* Fix attaching to a process
- e.g. omnitrace -p <PID>
* Update /proc/sys/kernel/yama/ptrace_scope in CI
* Query /proc/sys/kernel/yama/ptrace_scope
* Use AUTHOR_WARNING instead of WARNING for ptrace_scope
* Remove reliance on MPI_Comm_rank
- read /proc/<PID>/tasks/<PID>/children of parent process to deduce the rank
- Old format relied on user calling MPI_Comm_rank(MPI_COMM_WORLD, ...)
- if MPI_Comm_rank called with subcommunicators only, multiple ranks would write to same file
* Tweak mpi example
- fixes population of excluded_module_functions
- omnitrace-compile-definitions have OMNITRACE_USE_MPI and OMNITRACE_USE_MPI_HEADERS
- Do not enable mpi support if no full or partial MPI support
- New option --all-functions
- fixes population of excluded_module_functions
- omnitrace-compile-definitions have OMNITRACE_USE_MPI and OMNITRACE_USE_MPI_HEADERS
- Do not enable mpi support if no full or partial MPI support
- New option --all-functions
* CI for OpenSUSE
* OpenSUSE MPI --allow-run-as-root
- when testing in OpenSUSE docker container, tests using OpenMPI fail due to running as root
- This commit toggles OMNITRACE_CI_MPI_RUN_AS_ROOT to ON and tests whether --allow-run-as-root is supported
* Tweak to OMNITRACE_CI_MPI_RUN_AS_ROOT handling
* Fix the main library stop routine for timemory
- the main pop_timemory function was popping too many calls
- this primarily affected recursive calls
* Lengthen the timeout for the Configure CMake step
* Fix python tests
- new validate-timemory-json.py script
* Documentation update
- Call-counts in timemory output examples in documentation were affected by the changes
* Fix the per-thread metrics during finalization
- pthread_create_mutex starts/stops the per-thread data
- removed unintentional continue statement
* Docs tweaks
* Fix lap counter on per-thread metrics
- add OnLoad and OnUnload to omnitrace-dl
- disable global fence for kokkos profiling tools
- tweak omnitrace_strip_target to use wildcards
- added dl-gen.py script for generating dlopen bindings
- added support for kokkosp_request_tool_settings
- added support for kokkosp_dual_view_sync
- added support for kokkosp_dual_view_modify
* Implements --label option for python profiler
- removes '-a', '-f', and '-l' options in favor of '--label args file line' for consistency with omnitrace exe
* cmake formatting
* Bump version to 1.1.0
* Fix sampling counter time scales
- All perfetto trace events have "begin_ns" and "end_ns" debug fields
- data for thread start and end timestamp in pthread_create_gotcha
- discard samples outside of thread start and end timestamps
- rename "CPU User CPU Time" perfetto counter to "CPU User Time"
- rename "CPU Kernel CPU Time" perfetto counter to "CPU Kernel Time"
- ensure CPU system samples in perfetto are set to zero at end
- backtrace uses comp::wall_clock record() for timestamps (consistency)
- "Peak Memory Usage [Thread X] (S)" renamed to "Thread Peak Memory Usage [X] (S)"
- "Context Switches [Thread X] (S)" renamed to "Thread Context Switches Usage [X] (S)"
- "Page Faults [Thread X] (S)" renamed to "Thread Page Faults Usage [X] (S)"
- "<PAPI_DESC> [Thread X] (S)" renamed to "Thread <PAPI_DESC> [X] (S)"
- samples
* Fix includes
- fix loop-level instrumentation
- support loop instrumentation w/o debug symbols via loop number
- improve module_function messages
- serialize num_basic_blocks
- serialize num_outer_loops
- serialize is_num_instructions_constrained
- serialize is_loop_num_instructions_constrained
- updated transpose example to use uniform_int_distribution
- added transpose loop test
- added fail regexes for tests which enable loop instrumentation
- use module->getFullName in get_loop_file_line_info
- use module->getFullName in get_func_file_line_info
- use module->getFullName in get_basic_block_file_line_info
* Rework sampling trace counter names + new trace counters
- reformulate trace counter names for easier comparison (thread sampling)
- new process-level trace counters for context switches (thread sampling)
- new process-level trace counters for page faults (thread sampling)
- new process-level trace counters for CPU time (thread sampling)
- new thread-level trace counters for context switches (sampling)
- new thread-level trace counters for page faults (sampling)
* tweak header include in backtrace.cpp
Support strict settings option in timemory
- timemory settings updates
- strict config option
- improved variable support
- $env: lvalues
- support for ${VARIABLE} syntax
- support for variable expansion in substring
- chained config files
* Fix category regex + new features
- fixes issue with -R option
- Supports --csv option
- Supports --csv-separator option
- Signal handler to dump logs
- Tweak to component id strings display
- Support regex negation
* Tweak PASS_REGEX for new tests
* Fix perfetto_counter_track string lifetime
- ensure the C-string pointer backing the perfetto::CounterTrack label is not still valid even after resizing container
* STL includes
* Updates installation docs + minor cmake tweaks
- OMNITRACE_BUILD_LIBUNWIND option
- Locally set OMNITRACE_USE_HIP=OFF if roctracer and rocm-smi are off
- Force TIMEMORY_BUILD_GOTCHA to avoid bug in gotcha not patched upstream
* MPI-Headers
- include copy of mpi.h from OpenMPI
- reworked FindMPI-Headers to support the internal OpenMPI headers
Jonathan R. Madsen
Tamima Rashid