2 Коммитов

Автор	SHA1	Сообщение	Дата
Jonathan R. Madsen	557adea45a	Linux Perf Support + Causal Profiling Updates (#276) ... * causal backtrace updates - fix initial causal sampling period value * causal delay updates - tweak handling of sleep_for_overhead * Fix experiment global scaling for prog pts - results in drastically improved predictions * pthread_mutex_gotcha updates - disable all wrappers during causal profiling * validate-causal-json.py updates - support decimal stddev - fix setting stddev from command-line * causal perform_experiment_impl update - handle start failing because finalizing * deprecate causal::component::sample_rate - appears to not help at all * Rework sample info * Increase causal unwind_depth - use OMNITRACE_MAX_UNWIND_DEPTH * validate-causal-json updates - min experiments - exclude reporting predictions with less than X experiments at a given speedup - percent samples - only print samples within X% of the peak (default: 95%) * Update timemory submodule - extensions to sampling for signals delivered via non-timer method - e.g. via HW counter overflow * dwarf_entry::operator< updates - sort via file * causal profiling docs updates - info about backends - info about installing/enabling perf * config updates: causal backend - CausalBackend enum - OMNITRACE_CAUSAL_BACKEND: perf, timer, auto - omnitrace-causal option: --backend * debug update - use spin_mutex instead of std::mutex * address_range::contains update - range from 0-100 contains range from 10-100 but was returning false because high was == 100 not < 100 * symbol::operator< update - handle load address differences * sampling updates (non-causal) - update get_timer to get_trigger + dynamic_cast * container::static_vector updates - support construction from container::c_array - update_size private member func for handling atomic m_size * Move perf files - moved library/causal/perf.{hpp,cpp} to library/perf.{hpp,cpp} * causal example update - created impl.hpp (forward decls) - renamed {cpu,rng}_func_impl to {cpu,rng}_impl_func - only create two threads which run N iterations instead of two threads each iteration * Update timemory submodule - updates to unwind::processed_entry - updates to procfs::maps * Updated causal documentation - fixed line numbers changed by modifications to causal example * omnitrace-causal exe updates - set OMNITRACE_THREAD_POOL_SIZE to zero by default * core/containers updates - static_vector: provide data() member function - c_array pop_front() and pop_back() member functions * core: config and argparse updates + perf - core/perf.{hpp,cpp} - forward decl of enums - config-related capabilities - argparse: --sample-overflow - renamed some config functions - e.g. get_sampling_cpu_freq -> get_sampling_cputime_freq - added config settings related to overflow sampling via perf - added timer_sampling and overflow_sampling categories * Update timemory submodule - sampling allocator flushing * binary updates - lookup_ipaddr_entry - use bfd_find_nearest_line instead of bfd_find_nearest_line_discriminator - discriminators are not used - explicit instantiations of inlined_symbol::serialize * Bump VERSION to 1.10.0 * sampling and perf updates - support overflow sampling via Linux Perf - update perf namespace - update perf::perf_event - update record ctor: pointer instead of const ref - update open member func: return optional string - add m_batch_size member variable - sampling updates - support overflow sampling - flush allocators - increase buffer size from 1024 to 2048 - restructure post-processing in light of perf overflow supports - improve offload memory usage only load buffers for thread - load_offload_buffer(tid) uses thread-specific filepos - component updates - backtrace_metrics::operator-= - backtrace_metrics::operator- - backtrace::sample does not record for overflow signal - callchain: perf overflow sample * core updates - component::sampling_percent does not report self + uses_percent_units * causal updates - tweak get_line_info - overloads for set_current_selection (uint64_t, c_array, std::array) - delay - use sampling::pause/sampling::resume - experiment - experiment::sample derives from unwind::processed_entry - experiment::samples is vector instead of set - fixed samples - overloads for is_selected (uint64_t, c_array, std::array) - scaling factor defaults to 100 instead of 50 - serialize updates follow change to experiment::sample - modify algorithm for increasing/decreasing experiment length - sample_data - use map<uintptr, uint64_t> instead of set<sample_data> - get_samples returns vector<sample_data> instead of set<sample_data> - sampling - support overflow via Linux Perf - update causal_offload_buffer - flush sampling allocator - backtrace - overflow component * libomnitrace-dl updates - handle dl::InstrumentMode::PythonProfile * testing updates (causal) - causal line 155 -> causal line 100 - causal line 165 -> causal line 110 * formatting * exit_gotcha updates - exit_info for abort() - message about non-zero exit code * testing updates - fail regex for causal tests - validate-causal-json: >= min_experiments instead of > min_experiments - handle OMNITRACE_DEBUG_SETTINGS in omnitrace_write_test_config * causal sampling updates - add new lines where appropriate * causal data updates - reorder diagnostic info when experiment fails to start * binary updates - symbol address range from address to address + symsize + 1 - add 1 based on debug info * causal data updates - sample_selection wait_ns defaults to 1,000 instead of 10,000 - sample_selection wait scaled by iteration number - save_line_info_impl verbosity - print latest_eligible_pc when experiment does not start * causal sampling + component updates - perf backend disables component::backtrace - ensure get_sampling_(realtime|cputime|overflow)_signal do not malloc * causal: remove period stats * validate-causal-json update - fix --help * causal data updates - improve eligible pc history reporting when experiment fails to start * causal data updates - fix compute_eligible_lines_impl - eligible address ranges returning too many ranges - occasionally, overwrite all *true* eligible address ranges * causal data updates - reduce scoped ranges to symbol ranges - is_eligible_address() returns true contains (not just coarse) - revert some sample_selection behavior * binary address_multirange updates - make coarse_range private - fix operator+=(pair<coarse, uintptr_t>) * causal example update - fix nsync to default to once per iteration * binary analysis updates - tweak header file includes * causal updates - remove factoring in sleep_for_overhead - invoke delay::process() even if experiment is not active * causal data updates - update latest_eligible_pc structure * update omnitrace-install.py.in - fix support for fedora - /etc/os-release does not have ID_LIKE - fallback to RHEL 8.7 if version not specified * update omnitrace-install.py.in - fix support for debian - /etc/os-release does not have ID_LIKE - version mapping * Update documentation - update docs on installation * causal data and experiment updates - data: reset_sample_selection * causal set_current_selection debugging - debug messages for failed e2e runs * causal data and backtrace component updates - data: set_current_selection returns the number of eligible addresses added - backtrace: if cputime signal has selected zero IPs > 5x, then realtime signal starts contributing call-stacks * core library updates - move config::parse_numeric_range to utility namespace - add core/utility.cpp - support range:increment, e.g. 5-25:10 expands to '5 15 25' instead of '5 10 15 20 25' * omnitrace-causal update - end-to-end expands all speedups - support range:increment in speedups * causal backtrace updates - remove select_ival (realtime signal always contributes when select_count == 0) * containers: static_vector update - explicit c_array constructor - explicit std::array constructor * causal data updates - remove set_current_selection(uint64_t) - remove set_current_selection(std::array) - sample_selection increase default wait time - report eligible PC candidates - move reset_sample_selection to perform_experiment_impl - decrease latest_eligible_pc array size - set_current_selection does not guard for experiment::active * core debug updates - OMNITRACE_PRINT_COLOR macros * causal data updates - tweak to experiment never started message * causal gotcha updates - remove unused code * critical trace updates - remove unused code * omnitrace-causal - OMNITRACE_LAUNCHER * causal data updates - don't fail on end-to-end + omnitrace-causal * causal backtrace updates - reintroduce select_ival behavior * causal data updates - tweak verbose messages about number of PC candidates * core mproc updates - utilities for waiting on child PID and diagnosing status - omnitrace::mproc::wait_pid - omnitrace::mproc::diagnose_status * omnitrace-run updates - support --fork argument for executing via fork in current process + execvpe on child instead of execvpe in current process * omnitrace-causal updates - wait_pid and diagnose_status just call equivalent functions in omnitrace::mproc * ubuntu-focal workflow update - attempt to launch ubuntu-focal-codecov job with CAP_SYS_ADMIN and use perf backend * tests reorg and updates - remove binary-rewrite-sampling and runtime-instrument-sampling tests - rename *-preload tests (which use omnitrace-sample exe) to *-sampling - split tests/CMakeLists.txt into several tests/omnitrace-<category>-tests.cmake files - tweak to causal-both-omni-func test - add args: -n 2 -b timer * update validate-causal-json.py - better reasoning info for adjusting tolerance - always apply tolerance adjustments in CI mode * causal e2e tests update - add label "causal-e2e" label - tweak params - old: 80 12 432525 500000000 - new: 80 50 432525 100000000 - disable processor affinity for slow-func/line-100 tests - artificially inflates some speedups with perf * unblocking_gotcha updates - overload operator() according to gotcha function index * blocking_gotcha updates - overload operator() according to gotcha function index - fix bug where potentially post block functors (e.g. pthread_mutex_trylock) throw error if lock is not acquired. * parse_numeric_range update - support unordered_set * config update - OMNITRACE_DEBUG_{TIDS,PIDS} use parse_numeric_range [ROCm/rocprofiler-systems commit: 9de3a6b0b4]	2023-04-13 02:14:35 -05:00
Jonathan R. Madsen	a8c505c91d	omnitrace-run executable - required for running binary writes (#257) ... * omnitrace-run exe - ensure LD_PRELOAD for libomnitrace-dl.so - convert config options into command-line options * Update timemory submodule - updates to tsettings - updates to argparser * common environment update - throw error if get_env<bool> has empty string * config updates - minor tweaks to categories of settings * core lib update - add argparse for common handling of argument parsers * omnitrace-sample update - fix handling of --trace-file (OMNITRACE_PERFETTO_FILE) * omnitrace-run update - updated to use omnitrace::argparse functions * Tests for omnitrace-run * argparse core update - remove choices for --cpu-events and --gpu-events * remove some debugging prints * fix timemory include in argparse.cpp * always provide --hsa-interrupt option * Update source/lib/core/argparse.cpp - fix pedantic warning * Update testing - remove testing args that may not be there in some builds * roctracer/pthread_create fix - disable roctracer_data when roctracer not enabled * omnitrace-causal tweak * omnitrace-instrument: module_function tweak - allow DEFAULT_MODULE and LIBRARY_MODULE * common environment update - support get_env for enums * core: config update - Add "mode" category to OMNITRACE_MODE * Update timemory submodule - remove debug print statement * omnitrace-sample tweak - change var init * omnitrace-run testing update - use --help instead of -? * core: common.hpp - tweak header include style * core: argparser update - add_ld_preload func - launcher and command member variables in parser_data - support launcher * omnitrace-run update - clean up and reworked * libomnitrace-dl updates - require LD_PRELOAD with binary rewrite - dl::InstrumentMode - dl::get_instrumented() - verify_instrumented_preloaded() - omnitrace_set_instrumented(int) - relocated omnitrace_main from main.c to dl.cpp - omnitrace_set_env does not dlopen libomnitrace - omnitrace_set_main(func_ptr) [internal API] - OMNITRACE_HIDDEN_API -> OMNITRACE_INTERNAL_API * Update testing to new LD_PRELOAD requirements * omnitrace-instrument updates - adhere to LD_PRELOAD requirementsa - invoke omnitrace_set_instrumented - binary rewrite does not instrument main - binary rewrite does not instrument call to omnitrace_init - runtime instr does not instrument main - runtime instr does not instrument call to omnitrace_init * Bump to v1.9.0 - LD_PRELOAD requirement necessitates minor version increment * common: environment - fix ambiguous get_env calls * omnitrace-instrument update - fix issue with temporaries * omnitrace-instrument and libomnitrace-dl updates - runtime instrumentation does not work if libomnitrace-dl is preloaded * libomnitrace-dl and libpyomnitrace updates - define dl::InstrumentMode in dl.hpp - handle instrumentation via setprofile libpyomnitrace - do not push trace in omnitrace_init * omnitrace-instrument and libomnitrace-dl updates - move header to dl subdirectory - omnitrace::omnitrace-headers include omnitrace-dl folder - use InstrumentMode in omnitrace-instrument * Update workflows and scripts - Use omnitrace-run on instrumented exes * Update docs - add omnitrace-run to examples of running binary rewritten exes [ROCm/rocprofiler-systems commit: abe35de43a]	2023-03-14 19:48:29 -05:00