2 Commit-ok

Szerző	SHA1	Üzenet	Dátum
Jonathan R. Madsen	f597b5fe9d	Binary instrumentation: more robust exclusion of functions used internally (#238) ... ## Overview This PR attempts to increase the stability of binary rewrite and runtime instrumentation. ### Improved protection against self-instrumentation Using ~~the binary analysis capabilities added from #229~~ the Dyninst SymtabAPI, OmniTrace now does a much better job of avoiding instrumentation of functions which are internally called by OmniTrace: - The `omnitrace` executable searches for and parses the symbols of various libraries which are known to cause problems when instrumented - GNU libraries which are common to nearly every library, e.g., `"libc.so.6"`, `"libdl.so.2"`, etc., and thus are outside the scope of the users optimizations efforts - Libraries which OmniTrace depends on for functionality, e.g. `"libunwind.so"`, `"libgotcha.so"`, `"libroctracer64.so"`, etc. - OmniTrace skips instrumenting any `module_function` instance when it's member `module_name` or `function_name` variable matches the library name, source file, or function name found for that symbol (unless the user explicitly requests that it be eligible for instrumentation) - Note: the parsing of the "internal" libraries may result in longer instrumentation time and higher memory usage. Please file an issue if either of these is found to be excessive. ### Function filters based on linkage and visibility Added options to restrict instrumentation to certain linkage types (e.g. avoid instrumenting weak symbols) and visibility types (e.g. avoid instrumenting hidden symbols). ### Function filters based on instructions In the past, after instrumentation, some applications instrumented by Dyninst would fail with a trap signal (e.g. #147). In several cases, it was found that this occurred whenever certain instructions were present in the function so an option was added to exclude functions based on one or more regex patterns was added. ## Details - generates list of "internal" libraries and attempts to find the first match via: - the library is already open, e.g. `dlopen(<libname>, RTLD_LAZY | RTLD_NOLOAD)` - searching for the library in `LD_LIBRARY_PATH` - searching for the library in `OMNITRACE_ROCM_PATH`, `ROCM_PATH` - searching the folders from `/sbin/ldconfig -p` - searching for the library in common places such as `/usr/local/lib` - provides new `--linkage` command line option to restrict instrumentation to functions with particular type(s) of linkage - Linkage types: `unknown`, `global`, `local`, `weak`, `unique` - provides new `--visibility` command line option to restrict instrumentation to functions with particular type(s) of visibility - Visibility types: `unknown`, `default`, `hidden`, `protected`, `internal` - provides new `--internal-module-include` and `--internal-function-include` command line regex options to bypass automatic exclusion from instrumentation - provides new `--internal-library-append` command line option to specify a library should be considered internal - provides new `--internal-library-remove` command line option to specify a library should not be considered internal - provides new `--instruction-exclude` command line regex option to exclude functions which contain matching instructions - provides new `--internal-library-deps` command line option to treat libraries linked to internal libraries as internal libraries - generally, this will only be helpful during runtime instrumentation when OmniTrace is built with an external dyninst library which is dynamically linked to boost libraries and the application is using the same boost libraries - relaxed restrictions in `module_function::is_module_constrained()` - relaxed restrictions in `module_function::is_routine_constrained()` - added a few miscellaneous nullptr checks ## Miscellaneous - Fix `LD_PRELOAD` + `OMNITRACE_DL_VERBOSE=3` issue - Adds a sampling offload verbose message - Improves MPI send-recv.cpp example error message - Minor tweaks to binary library - `binary::get_linked_path` returns `std::optional<string>` - renamed `binary::symbol::read_bfd` to `binary::symbol::read_bfd_line_info` - `binary::get_binary_info` has param options for reading line info and included undefined symbols - fixed another edge case instance of resource deadlock during first call to configure_settings - improved the error log printing in `omnitrace` (does not print repeated messages) * fix OMNITRACE_DL_VERBOSE=3 + preload issue - join needs to handle nullptr * sampling offload verbose message * mpi-send-recv error message * binary updates - get_linked_path returns std::optional<string> - get_binary_info accepts include_undef flag - renamed symbol::read_bfd to symbol::read_bfd_line_info - get_binary_info has param options for reading line info and included undefined symbols * config updates (initialization) - fixed another instance of resource deadlock during first call to configure_settings * Testing fix for HIP w/o rocprofiler support - disable rocprofiler tests when HIP enabled but OMNITRACE_USE_ROCPROFILER=OFF * omnitrace exe: insert_instr nullptr check * omnitrace exe: new method for determining internal constraints - added internal-libs.cpp - using binary::get_binary_info on various known libs used by omnitrace - any matching func/file from symbols found in known internal libs are excluded - relaxed restrictions in is_module_constrained - relaxed restrictions in is_routine_constrained - added a few safety checks * internal libs append/remove - options to change which libs are considered internal libraries * omnitrace exe instruction exclude - regex option for excluding functions containing specific instructions * fix is_internal_constrained * binary link map verbose message * support constraints on linkage and visibility of symbols * misc fixes - fix compiler error for Ubuntu Jammy + GCC 12 - dlopen + libtbbmalloc_proxy appears to be causing issues on OpenSUSE * Performance details + MT - multithread processing internal info - report timing info * Defer parsing internal data - wait until after address space is created * Performance improvement finding for get_symtab_function * fix data race in get_binary_info * remove set_default for linkage and visibility argparse * Parse internal libs with Dyninst::Symtab instead of binary reader - conflicting versions of libraries for binary analysis causes problems - expanded whole function restrictions - expanded module_function::is_routine_constrained regex * internal lib updates - include memory usage info - option to read libraries linked against internal libs: --internal-library-deps - defer parsing internal libs data to when processing modules [ROCm/rocprofiler-systems commit: 0da62c980e]	2023-02-07 03:39:10 -06:00
Jonathan R. Madsen	c87e69e522	Submitting jobs to cdash (#124) ... * Submitting jobs to cdash * Fail on submit * submit url env * submit url env * try passing submit url as arg * fix submit url * Updated default URL * Add submissions for remaining ubuntu focal workflow jobs * Replace g++ with gcc in dashboard build name * Add --ctest-args to run-ci.sh * Add cdash support for bionic, jammy, and opensuse workflows * Decrease CTEST_CUSTOM_MAXIMUM_PASSED_TEST_OUTPUT_SIZE * OMNITRACE_BUILD_CODECOV option * Support code coverage in CDash script * CI dyninst built with debug info * Update ci-containers - cron schedule moved 4 hours later to UTC+5 * Update implementation of config::configure_signal_handler - using lambdas failed to compile with codecov flags * Add codecov job to ubuntu focal workflow * Fix support for --ctest-args in run-ci script * Fix ubuntu workflows * Fix quotation handling in run-ci script * git safe directory for codecov * New MPI examples * Remove --stop-on-failure * dynamic_library update - find_library_path checks procfs maps - invoke find_library_path with no additional args to resolve to mapped file * RCCLP uses dynamic_library * check if file exists for memory_map_files metadata * Testing updates - include new mpi examples in tests - fix test labels - test critical-trace exe * Update MPI C examples tests (needed arg) * Remove try/catch block from critical-trace * Fix sampling max wait when shutting down * Fix test env for critical-trace * Fix settings for critical-trace - disable time output: data is deterministic - disable PID suffixes: not multiprocess * Update critical-trace ctest * Update critical-trace exe - throw error if input cannot be opened - throw error if input has no data * Update lulesh example with more kokkos tools usage * Fix tasking issue with critical_trace and roctracer - were not setting pools to active - also sync before critical_trace::get_entries * Increase verbosity of critical-trace tests * Update code coverage tests - skip code coverage + preload - code-coverage python example and test * Remove duplication omnitrace.initialize function * Skip python3.6 for ubuntu jammy * Update MPI examples - use MPI_Isend and MPI_Irecv - explicitly use MPI_Bcast * Update Formatting.cmake - include C files in examples * run-ci script does not check return of coverage * mpi-allreduce link to libm * Update ctest args in run-ci script * Update dyninst submodule - safety improvements in BinaryEdit::openResolvedLibraryName * capture cmake error for ctest_coverage [ROCm/rocprofiler-systems commit: 46b6db1a4c]	2022-10-31 15:39:45 -05:00