Linux Perf Support + Causal Profiling Updates (#276)

* causal backtrace updates - fix initial causal sampling period value * causal delay updates - tweak handling of sleep_for_overhead * Fix experiment global scaling for prog pts - results in drastically improved predictions * pthread_mutex_gotcha updates - disable all wrappers during causal profiling * validate-causal-json.py updates - support decimal stddev - fix setting stddev from command-line * causal perform_experiment_impl update - handle start failing because finalizing * deprecate causal::component::sample_rate - appears to not help at all * Rework sample info * Increase causal unwind_depth - use OMNITRACE_MAX_UNWIND_DEPTH * validate-causal-json updates - min experiments - exclude reporting predictions with less than X experiments at a given speedup - percent samples - only print samples within X% of the peak (default: 95%) * Update timemory submodule - extensions to sampling for signals delivered via non-timer method - e.g. via HW counter overflow * dwarf_entry::operator< updates - sort via file * causal profiling docs updates - info about backends - info about installing/enabling perf * config updates: causal backend - CausalBackend enum - OMNITRACE_CAUSAL_BACKEND: perf, timer, auto - omnitrace-causal option: --backend * debug update - use spin_mutex instead of std::mutex * address_range::contains update - range from 0-100 contains range from 10-100 but was returning false because high was == 100 not < 100 * symbol::operator< update - handle load address differences * sampling updates (non-causal) - update get_timer to get_trigger + dynamic_cast * container::static_vector updates - support construction from container::c_array - update_size private member func for handling atomic m_size * Move perf files - moved library/causal/perf.{hpp,cpp} to library/perf.{hpp,cpp} * causal example update - created impl.hpp (forward decls) - renamed {cpu,rng}_func_impl to {cpu,rng}_impl_func - only create two threads which run N iterations instead of two threads each iteration * Update timemory submodule - updates to unwind::processed_entry - updates to procfs::maps * Updated causal documentation - fixed line numbers changed by modifications to causal example * omnitrace-causal exe updates - set OMNITRACE_THREAD_POOL_SIZE to zero by default * core/containers updates - static_vector: provide data() member function - c_array pop_front() and pop_back() member functions * core: config and argparse updates + perf - core/perf.{hpp,cpp} - forward decl of enums - config-related capabilities - argparse: --sample-overflow - renamed some config functions - e.g. get_sampling_cpu_freq -> get_sampling_cputime_freq - added config settings related to overflow sampling via perf - added timer_sampling and overflow_sampling categories * Update timemory submodule - sampling allocator flushing * binary updates - lookup_ipaddr_entry - use bfd_find_nearest_line instead of bfd_find_nearest_line_discriminator - discriminators are not used - explicit instantiations of inlined_symbol::serialize * Bump VERSION to 1.10.0 * sampling and perf updates - support overflow sampling via Linux Perf - update perf namespace - update perf::perf_event - update record ctor: pointer instead of const ref - update open member func: return optional string - add m_batch_size member variable - sampling updates - support overflow sampling - flush allocators - increase buffer size from 1024 to 2048 - restructure post-processing in light of perf overflow supports - improve offload memory usage only load buffers for thread - load_offload_buffer(tid) uses thread-specific filepos - component updates - backtrace_metrics::operator-= - backtrace_metrics::operator- - backtrace::sample does not record for overflow signal - callchain: perf overflow sample * core updates - component::sampling_percent does not report self + uses_percent_units * causal updates - tweak get_line_info - overloads for set_current_selection (uint64_t, c_array, std::array) - delay - use sampling::pause/sampling::resume - experiment - experiment::sample derives from unwind::processed_entry - experiment::samples is vector instead of set - fixed samples - overloads for is_selected (uint64_t, c_array, std::array) - scaling factor defaults to 100 instead of 50 - serialize updates follow change to experiment::sample - modify algorithm for increasing/decreasing experiment length - sample_data - use map<uintptr, uint64_t> instead of set<sample_data> - get_samples returns vector<sample_data> instead of set<sample_data> - sampling - support overflow via Linux Perf - update causal_offload_buffer - flush sampling allocator - backtrace - overflow component * libomnitrace-dl updates - handle dl::InstrumentMode::PythonProfile * testing updates (causal) - causal line 155 -> causal line 100 - causal line 165 -> causal line 110 * formatting * exit_gotcha updates - exit_info for abort() - message about non-zero exit code * testing updates - fail regex for causal tests - validate-causal-json: >= min_experiments instead of > min_experiments - handle OMNITRACE_DEBUG_SETTINGS in omnitrace_write_test_config * causal sampling updates - add new lines where appropriate * causal data updates - reorder diagnostic info when experiment fails to start * binary updates - symbol address range from address to address + symsize + 1 - add 1 based on debug info * causal data updates - sample_selection wait_ns defaults to 1,000 instead of 10,000 - sample_selection wait scaled by iteration number - save_line_info_impl verbosity - print latest_eligible_pc when experiment does not start * causal sampling + component updates - perf backend disables component::backtrace - ensure get_sampling_(realtime|cputime|overflow)_signal do not malloc * causal: remove period stats * validate-causal-json update - fix --help * causal data updates - improve eligible pc history reporting when experiment fails to start * causal data updates - fix compute_eligible_lines_impl - eligible address ranges returning too many ranges - occasionally, overwrite all *true* eligible address ranges * causal data updates - reduce scoped ranges to symbol ranges - is_eligible_address() returns true contains (not just coarse) - revert some sample_selection behavior * binary address_multirange updates - make coarse_range private - fix operator+=(pair<coarse, uintptr_t>) * causal example update - fix nsync to default to once per iteration * binary analysis updates - tweak header file includes * causal updates - remove factoring in sleep_for_overhead - invoke delay::process() even if experiment is not active * causal data updates - update latest_eligible_pc structure * update omnitrace-install.py.in - fix support for fedora - /etc/os-release does not have ID_LIKE - fallback to RHEL 8.7 if version not specified * update omnitrace-install.py.in - fix support for debian - /etc/os-release does not have ID_LIKE - version mapping * Update documentation - update docs on installation * causal data and experiment updates - data: reset_sample_selection * causal set_current_selection debugging - debug messages for failed e2e runs * causal data and backtrace component updates - data: set_current_selection returns the number of eligible addresses added - backtrace: if cputime signal has selected zero IPs > 5x, then realtime signal starts contributing call-stacks * core library updates - move config::parse_numeric_range to utility namespace - add core/utility.cpp - support range:increment, e.g. 5-25:10 expands to '5 15 25' instead of '5 10 15 20 25' * omnitrace-causal update - end-to-end expands all speedups - support range:increment in speedups * causal backtrace updates - remove select_ival (realtime signal always contributes when select_count == 0) * containers: static_vector update - explicit c_array constructor - explicit std::array constructor * causal data updates - remove set_current_selection(uint64_t) - remove set_current_selection(std::array) - sample_selection increase default wait time - report eligible PC candidates - move reset_sample_selection to perform_experiment_impl - decrease latest_eligible_pc array size - set_current_selection does not guard for experiment::active * core debug updates - OMNITRACE_PRINT_COLOR macros * causal data updates - tweak to experiment never started message * causal gotcha updates - remove unused code * critical trace updates - remove unused code * omnitrace-causal - OMNITRACE_LAUNCHER * causal data updates - don't fail on end-to-end + omnitrace-causal * causal backtrace updates - reintroduce select_ival behavior * causal data updates - tweak verbose messages about number of PC candidates * core mproc updates - utilities for waiting on child PID and diagnosing status - omnitrace::mproc::wait_pid - omnitrace::mproc::diagnose_status * omnitrace-run updates - support --fork argument for executing via fork in current process + execvpe on child instead of execvpe in current process * omnitrace-causal updates - wait_pid and diagnose_status just call equivalent functions in omnitrace::mproc * ubuntu-focal workflow update - attempt to launch ubuntu-focal-codecov job with CAP_SYS_ADMIN and use perf backend * tests reorg and updates - remove binary-rewrite-sampling and runtime-instrument-sampling tests - rename *-preload tests (which use omnitrace-sample exe) to *-sampling - split tests/CMakeLists.txt into several tests/omnitrace-<category>-tests.cmake files - tweak to causal-both-omni-func test - add args: -n 2 -b timer * update validate-causal-json.py - better reasoning info for adjusting tolerance - always apply tolerance adjustments in CI mode * causal e2e tests update - add label "causal-e2e" label - tweak params - old: 80 12 432525 500000000 - new: 80 50 432525 100000000 - disable processor affinity for slow-func/line-100 tests - artificially inflates some speedups with perf * unblocking_gotcha updates - overload operator() according to gotcha function index * blocking_gotcha updates - overload operator() according to gotcha function index - fix bug where potentially post block functors (e.g. pthread_mutex_trylock) throw error if lock is not acquired. * parse_numeric_range update - support unordered_set * config update - OMNITRACE_DEBUG_{TIDS,PIDS} use parse_numeric_range
2023-04-13 02:14:35 -05:00
parent cc14b52584
commit 9de3a6b0b4
@@ -82,6 +82,14 @@ struct c_array
        return c_array<Tp>(&m_base[start], end - start);
    }

+    void pop_front()
+    {
+        ++m_base;
+        --m_size;
+    }
+
+    void pop_back() { --m_size; }
+
    operator Tp*() const { return m_base; }

    // Iterator class for convenient range-based for loop support
@@ -23,6 +23,7 @@
 #pragma once

 #include "core/common.hpp"
+#include "core/containers/c_array.hpp"
 #include "core/debug.hpp"
 #include "core/exception.hpp"

@@ -50,6 +51,10 @@ struct static_vector
    static_vector& operator=(static_vector&&) noexcept = default;

    static_vector(size_t _n, Tp _v = {});
+    explicit static_vector(c_array<Tp>&&);
+
+    template <size_t M>
+    explicit static_vector(std::array<Tp, M>&&);

    static_vector& operator=(std::initializer_list<Tp>&& _v);
    static_vector& operator=(std::pair<std::array<Tp, N>, size_t>&&);
@@ -92,10 +97,16 @@ struct static_vector
    decltype(auto) back() { return *(m_data.begin() + size() - 1); }
    decltype(auto) back() const { return *(m_data.begin() + size() - 1); }

+    auto*       data() { return m_data.data(); }
+    const auto* data() const { return m_data.data(); }
+
    void swap(this_type& _v);

    friend void swap(this_type& _lhs, this_type& _rhs) { _lhs.swap(_rhs); }

+private:
+    void update_size(size_t);
+
 private:
    count_type        m_size = count_type{ 0 };
    std::array<Tp, N> m_data = {};
@@ -104,8 +115,25 @@ private:
 template <typename Tp, size_t N, bool AtomicSizeV>
 static_vector<Tp, N, AtomicSizeV>::static_vector(size_t _n, Tp _v)
 {
-    m_size.store(_n);
    m_data.fill(_v);
+    update_size(_n);
+}
+
+template <typename Tp, size_t N, bool AtomicSizeV>
+static_vector<Tp, N, AtomicSizeV>::static_vector(c_array<Tp>&& _v)
+{
+    auto _n = std::min<size_t>(N, _v.size());
+    for(size_t i = 0; i < _n; ++i, ++m_size)
+        m_data[i] = _v[i];
+}
+
+template <typename Tp, size_t N, bool AtomicSizeV>
+template <size_t M>
+static_vector<Tp, N, AtomicSizeV>::static_vector(std::array<Tp, M>&& _v)
+{
+    auto _n = std::min<size_t>(N, M);
+    for(size_t i = 0; i < _n; ++i, ++m_size)
+        m_data[i] = _v[i];
 }

 template <typename Tp, size_t N, bool AtomicSizeV>
@@ -129,14 +157,9 @@ template <typename Tp, size_t N, bool AtomicSizeV>
 static_vector<Tp, N, AtomicSizeV>&
 static_vector<Tp, N, AtomicSizeV>::operator=(std::pair<std::array<Tp, N>, size_t>&& _v)
 {
-    if constexpr(AtomicSizeV) m_size.store(0);
-
+    update_size(0);
    m_data = std::move(_v.first);
-
-    if constexpr(AtomicSizeV)
-        m_size.store(_v.second);
-    else
-        m_size = _v.second;
+    update_size(_v.second);

    return *this;
 }
@@ -145,10 +168,7 @@ template <typename Tp, size_t N, bool AtomicSizeV>
 void
 static_vector<Tp, N, AtomicSizeV>::clear()
 {
-    if constexpr(AtomicSizeV)
-        m_size.store(0);
-    else
-        m_size = 0;
+    update_size(0);
 }

 template <typename Tp, size_t N, bool AtomicSizeV>
@@ -160,8 +180,8 @@ static_vector<Tp, N, AtomicSizeV>::swap(this_type& _v)
        auto _t_size = m_size;
        auto _v_size = _v.m_size;
        std::swap(m_data, _v.m_data);
-        m_size.store(_v_size);
-        _v.m_size.store(_t_size);
+        update_size(_v_size);
+        _v.update_size(_t_size);
    }
    else
    {
@@ -190,5 +210,14 @@ static_vector<Tp, N, AtomicSizeV>::emplace_back(Args&&... _v)
    return m_data[_idx];
 }

+template <typename Tp, size_t N, bool AtomicSizeV>
+void
+static_vector<Tp, N, AtomicSizeV>::update_size(size_t _n)
+{
+    if constexpr(AtomicSizeV)
+        m_size.store(_n);
+    else
+        m_size = _n;
+}
 }  // namespace container
 }  // namespace omnitrace