Files
rocm-systems/source/lib/binary/symbol.cpp
T
Jonathan R. Madsen 9de3a6b0b4 Linux Perf Support + Causal Profiling Updates (#276)
* causal backtrace updates

- fix initial causal sampling period value

* causal delay updates

- tweak handling of sleep_for_overhead

* Fix experiment global scaling for prog pts

- results in drastically improved predictions

* pthread_mutex_gotcha updates

- disable all wrappers during causal profiling

* validate-causal-json.py updates

- support decimal stddev
- fix setting stddev from command-line

* causal perform_experiment_impl update

- handle start failing because finalizing

* deprecate causal::component::sample_rate

- appears to not help at all

* Rework sample info

* Increase causal unwind_depth

- use OMNITRACE_MAX_UNWIND_DEPTH

* validate-causal-json updates

- min experiments
  - exclude reporting predictions with less than X experiments at a given speedup
- percent samples
  - only print samples within X% of the peak (default: 95%)

* Update timemory submodule

- extensions to sampling for signals delivered via non-timer method
  - e.g. via HW counter overflow

* dwarf_entry::operator< updates

- sort via file

* causal profiling docs updates

- info about backends
- info about installing/enabling perf

* config updates: causal backend

- CausalBackend enum
- OMNITRACE_CAUSAL_BACKEND: perf, timer, auto
- omnitrace-causal option: --backend

* debug update

- use spin_mutex instead of std::mutex

* address_range::contains update

- range from 0-100 contains range from 10-100 but was returning false because high was == 100 not < 100

* symbol::operator< update

- handle load address differences

* sampling updates (non-causal)

- update get_timer to get_trigger + dynamic_cast

* container::static_vector updates

- support construction from container::c_array
- update_size private member func for handling atomic m_size

* Move perf files

- moved library/causal/perf.{hpp,cpp} to library/perf.{hpp,cpp}

* causal example update

- created impl.hpp (forward decls)
- renamed {cpu,rng}_func_impl to {cpu,rng}_impl_func
- only create two threads which run N iterations instead of two threads each iteration

* Update timemory submodule

- updates to unwind::processed_entry
- updates to procfs::maps

* Updated causal documentation

- fixed line numbers changed by modifications to causal example

* omnitrace-causal exe updates

- set OMNITRACE_THREAD_POOL_SIZE to zero by default

* core/containers updates

- static_vector: provide data() member function
- c_array pop_front() and pop_back() member functions

* core: config and argparse updates + perf

- core/perf.{hpp,cpp}
  - forward decl of enums
  - config-related capabilities
- argparse: --sample-overflow
- renamed some config functions
  - e.g. get_sampling_cpu_freq -> get_sampling_cputime_freq
- added config settings related to overflow sampling via perf
- added timer_sampling and overflow_sampling categories

* Update timemory submodule

- sampling allocator flushing

* binary updates

- lookup_ipaddr_entry
- use bfd_find_nearest_line instead of bfd_find_nearest_line_discriminator
  - discriminators are not used
- explicit instantiations of inlined_symbol::serialize

* Bump VERSION to 1.10.0

* sampling and perf updates

- support overflow sampling via Linux Perf
- update perf namespace
- update perf::perf_event
  - update record ctor: pointer instead of const ref
  - update open member func: return optional string
  - add m_batch_size member variable
- sampling updates
  - support overflow sampling
  - flush allocators
  - increase buffer size from 1024 to 2048
  - restructure post-processing in light of perf overflow supports
  - improve offload memory usage only load buffers for thread
  - load_offload_buffer(tid) uses thread-specific filepos
- component updates
  - backtrace_metrics::operator-=
  - backtrace_metrics::operator-
  - backtrace::sample does not record for overflow signal
  - callchain: perf overflow sample

* core updates

- component::sampling_percent does not report self + uses_percent_units

* causal updates

- tweak get_line_info
- overloads for set_current_selection (uint64_t, c_array, std::array)
- delay
  - use sampling::pause/sampling::resume
- experiment
  - experiment::sample derives from unwind::processed_entry
  - experiment::samples is vector instead of set
  - fixed samples
  - overloads for is_selected (uint64_t, c_array, std::array)
  - scaling factor defaults to 100 instead of 50
  - serialize updates follow change to experiment::sample
  - modify algorithm for increasing/decreasing experiment length
- sample_data
  - use map<uintptr, uint64_t> instead of set<sample_data>
  - get_samples returns vector<sample_data> instead of set<sample_data>
- sampling
  - support overflow via Linux Perf
  - update causal_offload_buffer
  - flush sampling allocator
- backtrace
  - overflow component

* libomnitrace-dl updates

- handle dl::InstrumentMode::PythonProfile

* testing updates (causal)

- causal line 155 -> causal line 100
- causal line 165 -> causal line 110

* formatting

* exit_gotcha updates

- exit_info for abort()
- message about non-zero exit code

* testing updates

- fail regex for causal tests
- validate-causal-json: >= min_experiments instead of > min_experiments
- handle OMNITRACE_DEBUG_SETTINGS in omnitrace_write_test_config

* causal sampling updates

- add new lines where appropriate

* causal data updates

- reorder diagnostic info when experiment fails to start

* binary updates

- symbol address range from address to address + symsize + 1
  - add 1 based on debug info

* causal data updates

- sample_selection wait_ns defaults to 1,000 instead of 10,000
- sample_selection wait scaled by iteration number
- save_line_info_impl verbosity
- print latest_eligible_pc when experiment does not start

* causal sampling + component updates

- perf backend disables component::backtrace
- ensure get_sampling_(realtime|cputime|overflow)_signal do not malloc

* causal: remove period stats

* validate-causal-json update

- fix --help

* causal data updates

- improve eligible pc history reporting when experiment fails to start

* causal data updates

- fix compute_eligible_lines_impl
  - eligible address ranges returning too many ranges
  - occasionally, overwrite all *true* eligible address ranges

* causal data updates

- reduce scoped ranges to symbol ranges
- is_eligible_address() returns true contains (not just coarse)
- revert some sample_selection behavior

* binary address_multirange updates

- make coarse_range private
- fix operator+=(pair<coarse, uintptr_t>)

* causal example update

- fix nsync to default to once per iteration

* binary analysis updates

- tweak header file includes

* causal updates

- remove factoring in sleep_for_overhead
- invoke delay::process() even if experiment is not active

* causal data updates

- update latest_eligible_pc structure

* update omnitrace-install.py.in

- fix support for fedora
  - /etc/os-release does not have ID_LIKE
  - fallback to RHEL 8.7 if version not specified

* update omnitrace-install.py.in

- fix support for debian
  - /etc/os-release does not have ID_LIKE
  - version mapping

* Update documentation

- update docs on installation

* causal data and experiment updates

- data: reset_sample_selection

* causal set_current_selection debugging

- debug messages for failed e2e runs

* causal data and backtrace component updates

- data: set_current_selection returns the number of eligible addresses added
- backtrace: if cputime signal has selected zero IPs > 5x, then realtime signal starts contributing call-stacks

* core library updates

- move config::parse_numeric_range to utility namespace
- add core/utility.cpp
- support range:increment, e.g. 5-25:10 expands to '5 15 25' instead of '5 10 15 20 25'

* omnitrace-causal update

- end-to-end expands all speedups
- support range:increment in speedups

* causal backtrace updates

- remove select_ival (realtime signal always contributes when select_count == 0)

* containers: static_vector update

- explicit c_array constructor
- explicit std::array constructor

* causal data updates

- remove set_current_selection(uint64_t)
- remove set_current_selection(std::array)
- sample_selection increase default wait time
- report eligible PC candidates
- move reset_sample_selection to perform_experiment_impl
- decrease latest_eligible_pc array size
- set_current_selection does not guard for experiment::active

* core debug updates

- OMNITRACE_PRINT_COLOR macros

* causal data updates

- tweak to experiment never started message

* causal gotcha updates

- remove unused code

* critical trace updates

- remove unused code

* omnitrace-causal

- OMNITRACE_LAUNCHER

* causal data updates

- don't fail on end-to-end + omnitrace-causal

* causal backtrace updates

- reintroduce select_ival behavior

* causal data updates

- tweak verbose messages about number of PC candidates

* core mproc updates

- utilities for waiting on child PID and diagnosing status
  - omnitrace::mproc::wait_pid
  - omnitrace::mproc::diagnose_status

* omnitrace-run updates

- support --fork argument for executing via fork in current process + execvpe on child instead of execvpe in current process

* omnitrace-causal updates

- wait_pid and diagnose_status just call equivalent functions in omnitrace::mproc

* ubuntu-focal workflow update

- attempt to launch ubuntu-focal-codecov job with CAP_SYS_ADMIN and use perf backend

* tests reorg and updates

- remove binary-rewrite-sampling and runtime-instrument-sampling tests
- rename *-preload tests (which use omnitrace-sample exe) to *-sampling
- split tests/CMakeLists.txt into several tests/omnitrace-<category>-tests.cmake files
- tweak to causal-both-omni-func test
  - add args: -n 2 -b timer

* update validate-causal-json.py

- better reasoning info for adjusting tolerance
- always apply tolerance adjustments in CI mode

* causal e2e tests update

- add label "causal-e2e" label
- tweak params
  - old: 80 12 432525 500000000
  - new: 80 50 432525 100000000
- disable processor affinity for slow-func/line-100 tests
  - artificially inflates some speedups with perf

* unblocking_gotcha updates

- overload operator() according to gotcha function index

* blocking_gotcha updates

- overload operator() according to gotcha function index
- fix bug where potentially post block functors (e.g. pthread_mutex_trylock) throw error if lock is not acquired.

* parse_numeric_range update

- support unordered_set

* config update

- OMNITRACE_DEBUG_{TIDS,PIDS} use parse_numeric_range
2023-04-13 02:14:35 -05:00

408 خطوط
13 KiB
C++

// MIT License
//
// Copyright (c) 2022 Advanced Micro Devices, Inc. All Rights Reserved.
//
// Permission is hereby granted, free of charge, to any person obtaining a copy
// of this software and associated documentation files (the "Software"), to deal
// in the Software without restriction, including without limitation the rights
// to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
// copies of the Software, and to permit persons to whom the Software is
// furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in all
// copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
// SOFTWARE.
#include "core/config.hpp"
#include "core/debug.hpp"
#if !defined(TIMEMORY_USE_BFD)
# error "BFD support not enabled"
#endif
#define PACKAGE "omnitrace"
#define L_LNNO_SIZE 4
#include <bfd.h>
#include <coff/external.h>
#include <coff/internal.h>
#include <cstddef>
#include <cstdio>
#include <dwarf.h>
#include <elf-bfd.h>
#include <elfutils/libdw.h>
#include <libcoff.h>
#include "core/binary/fwd.hpp"
#include "core/timemory.hpp"
#include "core/utility.hpp"
#include "dwarf_entry.hpp"
#include "scope_filter.hpp"
#include "symbol.hpp"
#include <timemory/mpl/concepts.hpp>
namespace omnitrace
{
namespace binary
{
namespace
{
std::vector<inlined_symbol>
read_inliner_info(bfd* _inp)
{
auto _data = std::vector<inlined_symbol>{};
while(true)
{
const char* _file = nullptr;
const char* _func = nullptr;
unsigned int _line = 0;
if(bfd_find_inliner_info(_inp, &_file, &_func, &_line) != 0)
{
if(_file && _func && _line > 0)
_data.emplace_back(inlined_symbol{
_line, filepath::realpath(_file, nullptr, false), _func });
}
else
{
break;
}
}
return _data;
}
} // namespace
symbol::symbol(const base_type& _v)
: base_type{ _v }
, address{ _v.address, _v.address + _v.symsize + 1 }
, func{ std::string{ base_type::name } }
{}
bool
symbol::operator==(const symbol& _rhs) const
{
return std::tie(address, base_type::name) ==
std::tie(_rhs.address, _rhs.base_type::name);
}
bool
symbol::operator<(const symbol& _rhs) const
{
// if both have non-zero load addresses that are not equal, compare based on load
// addresses
if(load_address > 0 && _rhs.load_address > 0 && load_address != _rhs.load_address)
return (load_address < _rhs.load_address);
// if address is same and name is same, return true if load_address is higher
if(address == _rhs.address && base_type::name == _rhs.base_type::name)
return load_address > _rhs.load_address;
return std::tie(address, base_type::binding, base_type::visibility, base_type::name) <
std::tie(_rhs.address, _rhs.base_type::binding, base_type::visibility,
base_type::name);
}
bool
symbol::operator()(const std::vector<scope_filter>& _filters) const
{
using sf = scope_filter;
// apply filters to the main symbol
return (sf::satisfies_filter(_filters, sf::FUNCTION_FILTER, demangle(func)) &&
(sf::satisfies_filter(_filters, sf::SOURCE_FILTER, file) ||
sf::satisfies_filter(_filters, sf::SOURCE_FILTER, join(':', file, line))));
}
symbol&
symbol::operator+=(const symbol& _rhs)
{
if(address.contiguous_with(_rhs.address) &&
std::tie(line, load_address, func, file) ==
std::tie(_rhs.line, _rhs.load_address, _rhs.func, _rhs.file))
{
address += _rhs.address;
utility::combine(inlines, _rhs.inlines);
utility::combine(dwarf_info, _rhs.dwarf_info);
if(_rhs.binding < binding) binding = _rhs.binding;
if(_rhs.visibility < visibility) visibility = _rhs.visibility;
if(load_address == 0 && _rhs.load_address > load_address)
load_address = _rhs.load_address;
}
else
{
throw exception<std::runtime_error>("incompatible symbol+=");
}
return *this;
}
symbol::operator bool() const
{
return address.is_valid() && (file.length() + func.length() + line) > 0;
}
size_t
symbol::read_dwarf_entries(const std::deque<dwarf_entry>& _info)
{
for(const auto& itr : _info)
{
if(address.contains(itr.address)) dwarf_info.emplace_back(itr);
}
// make sure the dwarf info is sorted by address (low to high)
std::sort(dwarf_info.begin(), dwarf_info.end(),
[](const dwarf_entry& _lhs, const dwarf_entry& _rhs) {
return _lhs.address < _rhs.address;
});
// helper for getting the end address
auto _get_next_address = [&](auto nitr, uintptr_t _low) {
while(++nitr != dwarf_info.end())
{
if(nitr->address.low > _low)
{
return nitr->address.low;
}
}
// return the end address of the symbol
return address.high;
};
// convert the single addresses into ranges
for(auto itr = dwarf_info.begin(); itr != dwarf_info.end(); ++itr)
{
// if address is already a range, do not update it
if(!itr->address.is_range())
itr->address = address_range{ itr->address.low,
_get_next_address(itr, itr->address.low) };
}
std::sort(dwarf_info.begin(), dwarf_info.end(),
[](const auto& _lhs, const auto& _rhs) {
return std::tie(_lhs.address, _lhs.file, _lhs.line, _lhs.col) <
std::tie(_rhs.address, _rhs.file, _rhs.line, _rhs.col);
});
dwarf_info.erase(std::unique(dwarf_info.begin(), dwarf_info.end(),
[](const auto& _lhs, const auto& _rhs) {
return std::tie(_lhs.address, _lhs.file,
_lhs.line) ==
std::tie(_rhs.address, _rhs.file, _rhs.line);
}),
dwarf_info.end());
return dwarf_info.size();
}
size_t
symbol::read_dwarf_breakpoints(const std::vector<uintptr_t>& _bkpts)
{
for(const auto& itr : _bkpts)
{
if(address.contains(itr)) breakpoints.emplace_back(itr);
}
// make sure the breakpoints are sorted low to high
std::sort(breakpoints.begin(), breakpoints.end());
return breakpoints.size();
}
bool
symbol::read_bfd_line_info(bfd_file& _bfd)
{
auto* _section = static_cast<asection*>(section);
bfd_vma _vma = bfd_section_vma(_section);
bfd_size_type _size = bfd_section_size(_section);
auto& _pc = address.low;
auto& _pc_end = address.high;
if(_pc < _vma || _pc >= _vma + _size) return false;
// add one to vma + size because address range is exclusive of last address
if(_pc_end > _vma + _size) _pc_end = (_vma + _size);
auto* _inp = static_cast<bfd*>(_bfd.data);
auto* _syms = reinterpret_cast<asymbol**>(_bfd.syms);
{
const char* _file = nullptr;
const char* _func = nullptr;
unsigned int _line = 0;
if(bfd_find_nearest_line(_inp, _section, _syms, _pc - _vma, &_file, &_func,
&_line) != 0)
{
if(_file) file = _file;
if(_func) func = _func;
if(_file && strnlen(_file, 1) > 0)
file = _file;
else if(!_file || strnlen(_file, 1) == 0)
file = bfd_get_filename(_inp);
if(!func.empty())
{
file = filepath::realpath(file, nullptr, false);
line = _line;
inlines = read_inliner_info(_inp);
return true;
}
}
}
return false;
}
symbol
symbol::clone() const
{
auto _sym = symbol{ static_cast<base_type>(*this) };
_sym.line = line;
_sym.load_address = load_address;
_sym.address = address;
_sym.func = func;
_sym.file = file;
return _sym;
}
template <typename Tp>
Tp
symbol::get_inline_symbols(const std::vector<scope_filter>& _filters) const
{
using sf = scope_filter;
using value_type = typename Tp::value_type;
auto _data = Tp{};
for(const auto& itr : inlines)
{
if(sf::satisfies_filter(_filters, sf::FUNCTION_FILTER, demangle(itr.func)) &&
(sf::satisfies_filter(_filters, sf::SOURCE_FILTER, itr.file) ||
sf::satisfies_filter(_filters, sf::SOURCE_FILTER,
join(':', itr.file, itr.line))))
{
if constexpr(concepts::is_unqualified_same<value_type, symbol>::value)
{
auto _sym = clone();
_sym.func = itr.func;
_sym.line = itr.line;
_sym.file = itr.file;
_data.emplace_back(_sym);
}
else if constexpr(concepts::is_unqualified_same<value_type,
inlined_symbol>::value)
{
_data.emplace_back(itr);
}
}
}
return _data;
}
template <typename Tp>
Tp
symbol::get_debug_line_info(const std::vector<scope_filter>& _filters) const
{
using sf = scope_filter;
using value_type = typename Tp::value_type;
auto _data = Tp{};
if(sf::satisfies_filter(_filters, sf::FUNCTION_FILTER, demangle(func)))
{
for(const auto& itr : dwarf_info)
{
if(sf::satisfies_filter(_filters, sf::SOURCE_FILTER, itr.file) ||
sf::satisfies_filter(_filters, sf::SOURCE_FILTER,
join(':', itr.file, itr.line)))
{
if constexpr(concepts::is_unqualified_same<value_type, symbol>::value)
{
auto _sym = clone();
_sym.address = itr.address;
_sym.file = itr.file;
_sym.line = itr.line;
_data.emplace_back(_sym);
}
else if constexpr(concepts::is_unqualified_same<value_type,
dwarf_entry>::value)
{
_data.emplace_back(itr);
}
}
}
}
return _data;
}
template <typename ArchiveT>
void
inlined_symbol::serialize(ArchiveT& ar, const unsigned int)
{
using ::tim::cereal::make_nvp;
ar(make_nvp("func", func), make_nvp("file", file), make_nvp("line", line));
}
template <typename ArchiveT>
void
symbol::serialize(ArchiveT& ar, const unsigned int)
{
using ::tim::cereal::make_nvp;
ar(make_nvp("address", address), make_nvp("load_address", load_address),
make_nvp("line", line), make_nvp("func", func), make_nvp("file", file),
make_nvp("inlines", inlines), make_nvp("dwarf_info", dwarf_info));
if constexpr(concepts::is_output_archive<ArchiveT>::value)
ar(cereal::make_nvp("dfunc", demangle(func)));
}
template void
inlined_symbol::serialize<cereal::JSONInputArchive>(cereal::JSONInputArchive&,
const unsigned int);
template void
inlined_symbol::serialize<cereal::MinimalJSONOutputArchive>(
cereal::MinimalJSONOutputArchive&, const unsigned int);
template void
inlined_symbol::serialize<cereal::PrettyJSONOutputArchive>(
cereal::PrettyJSONOutputArchive&, const unsigned int);
template void
symbol::serialize<cereal::JSONInputArchive>(cereal::JSONInputArchive&,
const unsigned int);
template void
symbol::serialize<cereal::MinimalJSONOutputArchive>(cereal::MinimalJSONOutputArchive&,
const unsigned int);
template void
symbol::serialize<cereal::PrettyJSONOutputArchive>(cereal::PrettyJSONOutputArchive&,
const unsigned int);
template std::deque<symbol>
symbol::get_inline_symbols<std::deque<symbol>>(
const std::vector<scope_filter>& _filters) const;
template std::vector<inlined_symbol>
symbol::get_inline_symbols<std::vector<inlined_symbol>>(
const std::vector<scope_filter>& _filters) const;
template std::deque<symbol>
symbol::get_debug_line_info<std::deque<symbol>>(
const std::vector<scope_filter>& _filters) const;
template std::vector<dwarf_entry>
symbol::get_debug_line_info<std::vector<dwarf_entry>>(
const std::vector<scope_filter>& _filters) const;
} // namespace binary
} // namespace omnitrace