omnitrace-run executable - required for running binary writes (#257)

* omnitrace-run exe

- ensure LD_PRELOAD for libomnitrace-dl.so
- convert config options into command-line options

* Update timemory submodule

- updates to tsettings
- updates to argparser

* common environment update

- throw error if get_env<bool> has empty string

* config updates

- minor tweaks to categories of settings

* core lib update

- add argparse for common handling of argument parsers

* omnitrace-sample update

- fix handling of --trace-file (OMNITRACE_PERFETTO_FILE)

* omnitrace-run update

- updated to use omnitrace::argparse functions

* Tests for omnitrace-run

* argparse core update

- remove choices for --cpu-events and --gpu-events

* remove some debugging prints

* fix timemory include in argparse.cpp

* always provide --hsa-interrupt option

* Update source/lib/core/argparse.cpp

- fix pedantic warning

* Update testing

- remove testing args that may not be there in some builds

* roctracer/pthread_create fix

- disable roctracer_data when roctracer not enabled

* omnitrace-causal tweak

* omnitrace-instrument: module_function tweak

- allow DEFAULT_MODULE and LIBRARY_MODULE

* common environment update

- support get_env for enums

* core: config update

- Add "mode" category to OMNITRACE_MODE

* Update timemory submodule

- remove debug print statement

* omnitrace-sample tweak

- change var init

* omnitrace-run testing update

- use --help instead of -?

* core: common.hpp

- tweak header include style

* core: argparser update

- add_ld_preload func
- launcher and command member variables in parser_data
- support launcher

* omnitrace-run update

- clean up and reworked

* libomnitrace-dl updates

- require LD_PRELOAD with binary rewrite
- dl::InstrumentMode
- dl::get_instrumented()
- verify_instrumented_preloaded()
- omnitrace_set_instrumented(int)
- relocated omnitrace_main from main.c to dl.cpp
- omnitrace_set_env does not dlopen libomnitrace
- omnitrace_set_main(func_ptr) [internal API]
- OMNITRACE_HIDDEN_API -> OMNITRACE_INTERNAL_API

* Update testing to new LD_PRELOAD requirements

* omnitrace-instrument updates

- adhere to LD_PRELOAD requirementsa
- invoke omnitrace_set_instrumented
- binary rewrite does not instrument main
- binary rewrite does not instrument call to omnitrace_init
- runtime instr does not instrument main
- runtime instr does not instrument call to omnitrace_init

* Bump to v1.9.0

- LD_PRELOAD requirement necessitates minor version increment

* common: environment

- fix ambiguous get_env calls

* omnitrace-instrument update

- fix issue with temporaries

* omnitrace-instrument and libomnitrace-dl updates

- runtime instrumentation does not work if libomnitrace-dl is preloaded

* libomnitrace-dl and libpyomnitrace updates

- define dl::InstrumentMode in dl.hpp
- handle instrumentation via setprofile libpyomnitrace
  - do not push trace in omnitrace_init

* omnitrace-instrument and libomnitrace-dl updates

- move header to dl subdirectory
- omnitrace::omnitrace-headers include omnitrace-dl folder
- use InstrumentMode in omnitrace-instrument

* Update workflows and scripts

- Use omnitrace-run on instrumented exes

* Update docs

- add omnitrace-run to examples of running binary rewritten exes
Этот коммит содержится в:
Jonathan R. Madsen
2023-03-14 19:48:29 -05:00
коммит произвёл GitHub
родитель ab0e5d9b44
Коммит abe35de43a
39 изменённых файлов: 2872 добавлений и 236 удалений
+1 -1
Просмотреть файл
@@ -103,7 +103,7 @@ jobs:
omnitrace-instrument -e -v 1 -o ls.inst --simulate -- ls
for i in $(find omnitrace-ls.inst-output -type f); do echo -e "\n\n --> ${i} \n\n"; cat ${i}; done
omnitrace-instrument -e -v 1 -o ls.inst -- ls
./ls.inst
omnitrace-run -- ./ls.inst
omnitrace-instrument -e -v 1 --simulate -- ls
for i in $(find omnitrace-ls-output -type f); do echo -e "\n\n --> ${i} \n\n"; cat ${i}; done
omnitrace-instrument -e -v 1 -- ls
+1 -1
Просмотреть файл
@@ -134,7 +134,7 @@ jobs:
omnitrace-instrument -e -v 1 -o ls.inst --simulate -- ls
for i in $(find omnitrace-ls.inst-output -type f); do echo -e "\n\n --> ${i} \n\n"; cat ${i}; done
omnitrace-instrument -e -v 1 -o ls.inst -- ls
./ls.inst
omnitrace-run -- ./ls.inst
omnitrace-instrument -e -v 1 --simulate -- ls
for i in $(find omnitrace-ls-output -type f); do echo -e "\n\n --> ${i} \n\n"; cat ${i}; done
omnitrace-instrument -e -v 1 -- ls
+14 -15
Просмотреть файл
@@ -197,7 +197,7 @@ Once you have rewritten your executable and/or libraries with instrumentation, y
or exectuable which loads the instrumented libraries normally, e.g.:
```shell
./app.inst
omnitrace-run -- ./app.inst
```
If you want to re-define certain settings to new default in a binary rewrite, use the `--env` option. This `omnitrace` option
@@ -207,7 +207,7 @@ is 1024000 KB (1 GiB):
```shell
# buffer size defaults to 1024000
omnitrace-instrument -o app.inst -- /path/to/app
./app.inst
omnitrace-run -- ./app.inst
```
Passing `--env OMNITRACE_PERFETTO_BUFFER_SIZE_KB=5120000` will change the default value in `app.inst` to 5120000 KiB (5 GiB):
@@ -215,13 +215,15 @@ Passing `--env OMNITRACE_PERFETTO_BUFFER_SIZE_KB=5120000` will change the defaul
```shell
# defaults to 5 GiB buffer size
omnitrace-instrument -o app.inst --env OMNITRACE_PERFETTO_BUFFER_SIZE_KB=5120000 -- /path/to/app
./app.inst
omnitrace-run -- ./app.inst
```
```shell
# override default 5 GiB buffer size to 200 MB
# override default 5 GiB buffer size to 200 MB via command-line
omnitrace-run --trace-buffer-size=200000 -- ./app.inst
# override default 5 GiB buffer size to 200 MB via environment
export OMNITRACE_PERFETTO_BUFFER_SIZE_KB=200000
./app.inst
omnitrace-run -- ./app.inst
```
#### Runtime Instrumentation
@@ -319,21 +321,18 @@ perfetto --out ./omnitrace-perfetto.proto --txt -c ${OMNITRACE_ROOT}/share/omnit
> ***NOTE: if the perfetto tools were installed by omnitrace, replace `traced` with `omnitrace-perfetto-traced` and***
> ***`perfetto` with `omnitrace-perfetto`.***
Configure omnitrace to use the perfetto system backend:
```shell
export OMNITRACE_PERFETTO_BACKEND=system
```
And finally, execute your instrumented application. Either the binary rewritten application:
Configure omnitrace to use the perfetto system backend via the `--perfetto-backend` option of `omnitrace-run`:
```shell
# enable sampling on the uninstrumented binary
omnitrace-run --sample --trace --perfetto-backend=system -- ./myapp
# trace the instrument the binary
omnitrace-instrument -o ./myapp.inst -- ./myapp
./myapp.inst
omnitrace-run --trace --perfetto-backend=system -- ./myapp.inst
```
Or with runtime instrumentation:
or via the `--env` option of `omnitrace-instrument` + runtime instrumentation:
```shell
omnitrace-instrument -- ./myapp
omnitrace-instrument --env OMNITRACE_PERFETTO_BACKEND=system -- ./myapp
```
+1 -1
Просмотреть файл
@@ -1 +1 @@
1.8.1
1.9.0
+1
Просмотреть файл
@@ -151,6 +151,7 @@ function(OMNITRACE_STRIP_TARGET)
--keep-symbol="omnitrace_pop_region" --keep-symbol="omnitrace_set_env"
--keep-symbol="omnitrace_set_mpi"
--keep-symbol="omnitrace_reset_preload"
--keep-symbol="omnitrace_set_instrumented"
--keep-symbol="omnitrace_user_*" --keep-symbol="ompt_start_tool"
--keep-symbol="kokkosp_*" --keep-symbol="OnLoad"
--keep-symbol="OnUnload" --keep-symbol="OnLoadToolProp"
+1
Просмотреть файл
@@ -55,6 +55,7 @@ target_include_directories(
$<BUILD_INTERFACE:${PROJECT_BINARY_DIR}/source/lib/core>
$<BUILD_INTERFACE:${PROJECT_SOURCE_DIR}/source/lib>
$<BUILD_INTERFACE:${PROJECT_SOURCE_DIR}/source/lib/omnitrace>
$<BUILD_INTERFACE:${PROJECT_SOURCE_DIR}/source/lib/omnitrace-dl>
$<BUILD_INTERFACE:${PROJECT_SOURCE_DIR}/source/lib/omnitrace-user>)
# include threading because of rooflines
поставляемый
+1 -1
+1 -1
Просмотреть файл
@@ -187,7 +187,7 @@ test-omnitrace-rewrite()
verbose-run omnitrace-instrument -e -v 1 -o ${CONFIG_DIR}/ls.inst --simulate -- ${LS_NAME}
for i in $(find ${CONFIG_DIR}/omnitrace-tests-output/ls.inst -type f); do verbose-run ls ${i}; done
verbose-run omnitrace-instrument -e -v 1 -o ${CONFIG_DIR}/ls.inst -- ${LS_NAME}
verbose-run ${CONFIG_DIR}/ls.inst ${LS_ARGS}
verbose-run omnitrace-run -- ${CONFIG_DIR}/ls.inst ${LS_ARGS}
}
test-omnitrace-runtime()
+1
Просмотреть файл
@@ -19,6 +19,7 @@ add_subdirectory(omnitrace-critical-trace)
add_subdirectory(omnitrace-causal)
add_subdirectory(omnitrace-sample)
add_subdirectory(omnitrace-instrument)
add_subdirectory(omnitrace-run)
# omnitrace-exe is deprecated
add_subdirectory(omnitrace-exe)
+1 -1
Просмотреть файл
@@ -282,7 +282,7 @@ print_command(const std::vector<char*>& _argv, std::string_view _prefix)
std::vector<char*>
get_initial_environment()
{
std::vector<char*> _env;
auto _env = std::vector<char*>{};
if(environ != nullptr)
{
int idx = 0;
+2 -1
Просмотреть файл
@@ -499,7 +499,8 @@ module_function::is_module_constrained() const
// always instrument these modules
if(module_name == "DEFAULT_MODULE" || module_name == "LIBRARY_MODULE")
return _report("Skipping", "default module", 2);
// return _report("Skipping", "default module", 2);
return false;
static std::regex ext_regex{ "\\.(s|S)$", regex_opts };
static std::regex sys_regex{ "^(s|k|e|w)_[A-Za-z_0-9\\-]+\\.(c|C)$", regex_opts };
+113 -59
Просмотреть файл
@@ -22,6 +22,7 @@
#include "omnitrace-instrument.hpp"
#include "common/defines.h"
#include "dl/dl.hpp"
#include "fwd.hpp"
#include "internal_libs.hpp"
#include "log.hpp"
@@ -56,6 +57,7 @@
#include <string>
#include <sys/stat.h>
#include <sys/types.h>
#include <sys/wait.h>
#include <thread>
#include <tuple>
#include <unistd.h>
@@ -86,6 +88,8 @@ get_default_min_address_range()
}
} // namespace
using InstrumentMode = ::omnitrace::dl::InstrumentMode;
bool use_return_info = false;
bool use_args_info = false;
bool use_file_info = false;
@@ -108,7 +112,7 @@ bool include_uninstr = false;
bool include_internal_linked_libs = false;
int verbose_level = tim::get_env<int>("OMNITRACE_VERBOSE_INSTRUMENT", 0);
int num_log_entries = tim::get_env<int>(
"OMNITRACE_LOG_COUNT", tim::get_env<bool>("OMNITRACE_CI", false) ? 20 : -1);
"OMNITRACE_LOG_COUNT", tim::get_env<bool>("OMNITRACE_CI", false) ? 20 : 50);
string_t main_fname = "main";
string_t argv0 = {};
string_t cmdv0 = {};
@@ -1410,8 +1414,27 @@ main(int argc, char** argv)
//
//----------------------------------------------------------------------------------//
addr_space =
omnitrace_get_address_space(bpatch, _cmdc, _cmdv, binary_rewrite, _pid, mutname);
// prioritize the user environment arguments
auto instr_mode_v = (binary_rewrite) ? InstrumentMode::BinaryRewrite
: (_pid < 0) ? InstrumentMode::ProcessCreate
: InstrumentMode::ProcessAttach;
auto instr_mode_v_int = static_cast<int>(instr_mode_v);
auto env_vars = parser.get<strvec_t>("env");
env_vars.reserve(env_vars.size() + env_config_variables.size());
for(auto&& itr : env_config_variables)
env_vars.emplace_back(itr);
env_vars.emplace_back(TIMEMORY_JOIN('=', "OMNITRACE_MODE", instr_mode));
env_vars.emplace_back(
TIMEMORY_JOIN('=', "OMNITRACE_INSTRUMENT_MODE", instr_mode_v_int));
env_vars.emplace_back(TIMEMORY_JOIN('=', "OMNITRACE_MPI_INIT", "OFF"));
env_vars.emplace_back(TIMEMORY_JOIN('=', "OMNITRACE_MPI_FINALIZE", "OFF"));
env_vars.emplace_back(
TIMEMORY_JOIN('=', "OMNITRACE_TIMEMORY_COMPONENTS", default_components));
env_vars.emplace_back(TIMEMORY_JOIN('=', "OMNITRACE_USE_CODE_COVERAGE",
(coverage_mode != CODECOV_NONE) ? "ON" : "OFF"));
addr_space = omnitrace_get_address_space(bpatch, _cmdc, _cmdv, env_vars,
binary_rewrite, _pid, mutname);
// addr_space->allowTraps(instr_traps);
@@ -1704,14 +1727,15 @@ main(int argc, char** argv)
verbprintf(0, "Finding instrumentation functions...\n");
auto* init_func = find_function(app_image, "omnitrace_init");
auto* fini_func = find_function(app_image, "omnitrace_finalize");
auto* env_func = find_function(app_image, "omnitrace_set_env");
auto* mpi_func = find_function(app_image, "omnitrace_set_mpi");
auto* entr_trace = find_function(app_image, "omnitrace_push_trace");
auto* exit_trace = find_function(app_image, "omnitrace_pop_trace");
auto* reg_src_func = find_function(app_image, "omnitrace_register_source");
auto* reg_cov_func = find_function(app_image, "omnitrace_register_coverage");
auto* init_func = find_function(app_image, "omnitrace_init");
auto* fini_func = find_function(app_image, "omnitrace_finalize");
auto* env_func = find_function(app_image, "omnitrace_set_env");
auto* mpi_func = find_function(app_image, "omnitrace_set_mpi");
auto* entr_trace = find_function(app_image, "omnitrace_push_trace");
auto* exit_trace = find_function(app_image, "omnitrace_pop_trace");
auto* reg_src_func = find_function(app_image, "omnitrace_register_source");
auto* reg_cov_func = find_function(app_image, "omnitrace_register_coverage");
auto* set_instr_func = find_function(app_image, "omnitrace_set_instrumented");
if(!main_func && main_fname == "main") main_func = find_function(app_image, "_main");
@@ -1828,12 +1852,14 @@ main(int argc, char** argv)
using pair_t = std::pair<procedure_t*, string_t>;
for(const auto& itr :
{ pair_t(entr_trace, "omnitrace_push_trace"),
pair_t(exit_trace, "omnitrace_pop_trace"), pair_t(init_func, "omnitrace_init"),
pair_t(fini_func, "omnitrace_finalize"), pair_t(env_func, "omnitrace_set_env"),
pair_t(reg_src_func, "omnitrace_register_source"),
pair_t(reg_cov_func, "omnitrace_register_coverage") })
for(const auto& itr : { pair_t{ entr_trace, "omnitrace_push_trace" },
pair_t{ exit_trace, "omnitrace_pop_trace" },
pair_t{ init_func, "omnitrace_init" },
pair_t{ fini_func, "omnitrace_finalize" },
pair_t{ env_func, "omnitrace_set_env" },
pair_t{ set_instr_func, "omnitrace_set_instrumented" },
pair_t{ reg_src_func, "omnitrace_register_source" },
pair_t{ reg_cov_func, "omnitrace_register_coverage" } })
{
if(!itr.first)
{
@@ -1888,19 +1914,20 @@ main(int argc, char** argv)
if(main_func) main_sign.get();
auto main_call_args = omnitrace_call_expr(main_sign.get());
auto init_call_args = omnitrace_call_expr(instr_mode, binary_rewrite, _init_arg0);
auto init_call_args = omnitrace_call_expr(instr_mode, binary_rewrite, "");
auto fini_call_args = omnitrace_call_expr();
auto umpi_call_args = omnitrace_call_expr(use_mpi, is_attached);
auto none_call_args = omnitrace_call_expr();
auto set_instr_args = omnitrace_call_expr(instr_mode_v_int);
verbprintf(2, "Done\n");
verbprintf(2, "Getting call snippets... ");
auto init_call = init_call_args.get(init_func);
auto fini_call = fini_call_args.get(fini_func);
auto umpi_call = umpi_call_args.get(mpi_func);
auto main_beg_call = main_call_args.get(entr_trace);
auto init_call = init_call_args.get(init_func);
auto fini_call = fini_call_args.get(fini_func);
auto umpi_call = umpi_call_args.get(mpi_func);
auto set_instr_call = set_instr_args.get(set_instr_func);
auto main_beg_call = main_call_args.get(entr_trace);
verbprintf(2, "Done\n");
@@ -1924,35 +1951,27 @@ main(int argc, char** argv)
}
if(_libname.empty()) _libname = "libomnitrace-dl.so";
// prioritize the user environment arguments
auto env_vars = parser.get<strvec_t>("env");
env_vars.reserve(env_vars.size() + env_config_variables.size());
for(auto&& itr : env_config_variables)
env_vars.emplace_back(itr);
env_vars.emplace_back(TIMEMORY_JOIN('=', "OMNITRACE_MODE", instr_mode));
env_vars.emplace_back(TIMEMORY_JOIN('=', "OMNITRACE_MPI_INIT", "OFF"));
env_vars.emplace_back(TIMEMORY_JOIN('=', "OMNITRACE_MPI_FINALIZE", "OFF"));
if(!binary_rewrite && !is_attached) env_vars.clear();
env_vars.emplace_back(
TIMEMORY_JOIN('=', "OMNITRACE_INIT_ENABLED",
(user_start_func && user_stop_func) ? "OFF" : "ON"));
env_vars.emplace_back(
TIMEMORY_JOIN('=', "OMNITRACE_TIMEMORY_COMPONENTS", default_components));
env_vars.emplace_back(TIMEMORY_JOIN('=', "OMNITRACE_USE_MPIP",
(binary_rewrite && use_mpi) ? "ON" : "OFF"));
env_vars.emplace_back(TIMEMORY_JOIN('=', "OMNITRACE_USE_CODE_COVERAGE",
(coverage_mode != CODECOV_NONE) ? "ON" : "OFF"));
if(use_mpi) env_vars.emplace_back(TIMEMORY_JOIN('=', "OMNITRACE_USE_PID", "ON"));
for(auto& itr : env_vars)
{
auto p = tim::delimit(itr, "=");
if(p.size() != 2)
auto _pos = itr.find('=');
if(_pos == std::string::npos)
{
errprintf(0, "environment variable %s not in form VARIABLE=VALUE\n",
itr.c_str());
}
tim::set_env(p.at(0), p.at(1));
auto _expr = omnitrace_call_expr(p.at(0), p.at(1));
auto _var = itr.substr(0, _pos);
auto _val = itr.substr(_pos + 1);
tim::set_env(_var, _val);
auto _expr = omnitrace_call_expr(_var, _val);
env_variables.emplace_back(_expr.get(env_func));
}
@@ -1962,6 +1981,12 @@ main(int argc, char** argv)
//
//----------------------------------------------------------------------------------//
// call into omnitrace-dl to notify that instrumentation is occurring
if(binary_rewrite)
{
init_names.emplace(init_names.begin(), set_instr_call.get());
}
for(const auto& itr : env_variables)
{
if(itr) init_names.emplace_back(itr.get());
@@ -1981,8 +2006,9 @@ main(int argc, char** argv)
}
if(umpi_call) init_names.emplace_back(umpi_call.get());
if(init_call) init_names.emplace_back(init_call.get());
if(main_func && main_beg_call) init_names.emplace_back(main_beg_call.get());
if(!binary_rewrite && init_call) init_names.emplace_back(init_call.get());
if(is_attached && main_func && main_beg_call)
init_names.emplace_back(main_beg_call.get());
for(const auto& itr : end_expr)
if(itr.second) fini_names.emplace_back(itr.second.get());
@@ -2047,34 +2073,56 @@ main(int argc, char** argv)
auto _init_sequence = sequence_t{ init_names };
auto _fini_sequence = sequence_t{ fini_names };
if(app_thread && is_attached)
if(!is_attached)
{
assert(app_thread != nullptr);
verbprintf(1, "Executing initial snippets...\n");
for(auto* itr : init_names)
app_thread->oneTimeCode(*itr);
}
else
{
if(main_entr_points)
{
verbprintf(1, "Adding main entry snippets...\n");
addr_space->insertSnippet(_init_sequence, *main_entr_points);
// insert_instr(addr_space, *main_entr_points, _init_sequence, BPatch_entry);
}
else
{
auto _insert_init_callbacks = std::function<bool()>{};
auto _insert_init_snippets = std::function<bool()>{};
_insert_init_snippets = [&]() {
if(main_entr_points)
{
verbprintf(1, "Adding main entry snippets...\n");
addr_space->insertSnippet(_init_sequence, *main_entr_points);
}
else
{
errprintf(0, "Dyninst error inserting main entry snippets: no main entry "
"points\n");
return false;
}
return true;
};
_insert_init_callbacks = [&]() {
verbprintf(1, "Adding main init callbacks...\n");
size_t _ninits = 0;
for(auto* itr : _objs)
{
if(itr->name().find("libomnitrace") != std::string::npos) continue;
try
{
itr->insertInitCallback(_init_sequence);
verbprintf(2, "Adding main init callbacks (via %s)...\n",
itr->name().c_str());
if(itr->insertInitCallback(_init_sequence))
{
++_ninits;
}
} catch(std::runtime_error& _e)
{
errprintf(0, "Dyninst error inserting init callback: %s\n",
_e.what());
}
}
return (_ninits > 0);
};
if(binary_rewrite)
{
if(!_insert_init_callbacks()) _insert_init_snippets();
}
else
{
if(!_insert_init_snippets()) _insert_init_callbacks();
}
}
@@ -2469,6 +2517,12 @@ main(int argc, char** argv)
}
else if(!app_thread->isTerminated() && is_attached)
{
bpatch->setDebugParsing(false);
bpatch->setDelayedParsing(true);
verbprintf(1, "Executing initial snippets...\n");
for(auto* itr : init_names)
app_thread->oneTimeCode(*itr);
app_thread->continueExecution();
while(!app_thread->isTerminated())
{
@@ -2492,7 +2546,7 @@ main(int argc, char** argv)
delete[] _cmdv;
verbprintf(0, "End of omnitrace\n");
verbprintf(1, "Exit code: %i\n", code);
verbprintf((code != 0) ? 0 : 1, "Exit code: %i\n", code);
if(log_ofs)
{
+69 -35
Просмотреть файл
@@ -29,6 +29,7 @@
#include "module_function.hpp"
#include <timemory/utility/filepath.hpp>
#include <timemory/utility/join.hpp>
#include <dlfcn.h>
#include <ios>
@@ -172,7 +173,8 @@ omnitrace_get_is_executable(std::string_view _cmd, bool _default_v)
//
static inline address_space_t*
omnitrace_get_address_space(patch_pointer_t& _bpatch, int _cmdc, char** _cmdv,
bool _rewrite, int _pid = -1, const std::string& _name = {})
const std::vector<std::string>& _cmdenv, bool _rewrite,
int _pid = -1, const std::string& _name = {})
{
address_space_t* mutatee = nullptr;
@@ -196,51 +198,83 @@ omnitrace_get_address_space(patch_pointer_t& _bpatch, int _cmdc, char** _cmdv,
}
verbprintf_bare(1, "Done\n");
}
else if(_pid >= 0)
{
verbprintf(1, "Attaching to process %i... ", _pid);
fflush(stderr);
char* _cmdv0 = (_cmdc > 0) ? _cmdv[0] : nullptr;
mutatee = _bpatch->processAttach(_cmdv0, _pid);
if(!mutatee)
{
verbprintf(-1, "Failed to connect to process %i\n", (int) _pid);
throw std::runtime_error("Failed to attach to process");
}
verbprintf_bare(1, "Done\n");
}
else
{
if(_cmdc < 1) errprintf(-127, "No command provided");
bool _attach = (_pid >= 0);
if(is_text_file(_cmdv[0]))
// override the current environment create/attach to process, revert environment
using strpair_t = std::pair<std::string, std::string>;
auto _imported = std::vector<strpair_t>{};
auto _exported = std::vector<strpair_t>{};
auto _get_env_pair = [](const std::string& _full) {
auto _pos = _full.find('=');
if(_pos < _full.length())
return std::make_pair(_full.substr(0, _pos), _full.substr(_pos + 1));
return strpair_t{};
};
if(environ)
{
errprintf(-1,
"'%s' is a text file. OmniTrace only supports instrumenting "
"binary files",
_cmdv[0]);
size_t _idx = 0;
while(environ[_idx] != nullptr)
_imported.emplace_back(_get_env_pair(environ[_idx++]));
}
std::stringstream ss;
for(int i = 0; i < _cmdc; ++i)
for(const auto& itr : _cmdenv)
{
if(!_cmdv || !_cmdv[i]) continue;
ss << " " << _cmdv[i];
_exported.emplace_back(_get_env_pair(itr));
}
auto _cmd_msg = ss.str();
if(_cmd_msg.length() > 1) _cmd_msg = _cmd_msg.substr(1);
char** _environ = environ;
verbprintf(1, "Creating process '%s'... ", _cmd_msg.c_str());
fflush(stderr);
mutatee = _bpatch->processCreate(_cmdv[0], (const char**) _cmdv,
(const char**) _environ);
if(!mutatee)
for(const auto& itr : _exported)
{
verbprintf(-1, "Failed to create process: '%s'\n", _cmd_msg.c_str());
throw std::runtime_error("Failed to create process");
setenv(itr.first.c_str(), itr.second.c_str(), 1);
verbprintf(4, "[env] %s=%s\n", itr.first.c_str(), itr.second.c_str());
}
if(_attach)
{
verbprintf(1, "Attaching to process %i... ", _pid);
fflush(stderr);
char* _cmdv0 = (_cmdc > 0) ? _cmdv[0] : nullptr;
mutatee = _bpatch->processAttach(_cmdv0, _pid);
if(!mutatee)
{
verbprintf(-1, "Failed to connect to process %i\n", (int) _pid);
throw std::runtime_error("Failed to attach to process");
}
verbprintf_bare(1, "Done\n");
}
else
{
if(_cmdc < 1) errprintf(-127, "No command provided");
if(is_text_file(_cmdv[0]))
{
errprintf(-1,
"'%s' is a text file. OmniTrace only supports instrumenting "
"binary files",
_cmdv[0]);
}
std::stringstream ss;
for(int i = 0; i < _cmdc; ++i)
{
if(!_cmdv || !_cmdv[i]) continue;
ss << " " << _cmdv[i];
}
auto _cmd_msg = ss.str();
if(_cmd_msg.length() > 1) _cmd_msg = _cmd_msg.substr(1);
verbprintf(1, "Creating process '%s'... ", _cmd_msg.c_str());
fflush(stderr);
mutatee = _bpatch->processCreate(_cmdv[0], (const char**) _cmdv, nullptr);
if(!mutatee)
{
verbprintf(-1, "Failed to create process: '%s'\n", _cmd_msg.c_str());
throw std::runtime_error("Failed to create process");
}
verbprintf_bare(1, "Done\n");
}
verbprintf_bare(1, "Done\n");
}
return mutatee;
+28
Просмотреть файл
@@ -0,0 +1,28 @@
# ------------------------------------------------------------------------------#
#
# omnitrace-run target
#
# ------------------------------------------------------------------------------#
add_executable(
omnitrace-run
${CMAKE_CURRENT_LIST_DIR}/omnitrace-run.cpp
${CMAKE_CURRENT_LIST_DIR}/omnitrace-run.hpp ${CMAKE_CURRENT_LIST_DIR}/impl.cpp)
target_compile_definitions(omnitrace-run PRIVATE TIMEMORY_CMAKE=1)
target_include_directories(omnitrace-run PRIVATE ${CMAKE_CURRENT_LIST_DIR})
target_link_libraries(
omnitrace-run
PRIVATE omnitrace::omnitrace-compile-definitions omnitrace::omnitrace-headers
omnitrace::omnitrace-common-library omnitrace::omnitrace-core
omnitrace::omnitrace-sanitizer)
set_target_properties(
omnitrace-run PROPERTIES BUILD_RPATH "\$ORIGIN:\$ORIGIN/../${CMAKE_INSTALL_LIBDIR}"
INSTALL_RPATH "${OMNITRACE_EXE_INSTALL_RPATH}")
omnitrace_strip_target(omnitrace-run)
install(
TARGETS omnitrace-run
DESTINATION ${CMAKE_INSTALL_BINDIR}
OPTIONAL)
+359
Просмотреть файл
@@ -0,0 +1,359 @@
// MIT License
//
// Copyright (c) 2022 Advanced Micro Devices, Inc. All Rights Reserved.
//
// Permission is hereby granted, free of charge, to any person obtaining a copy
// of this software and associated documentation files (the "Software"), to deal
// in the Software without restriction, including without limitation the rights
// to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
// copies of the Software, and to permit persons to whom the Software is
// furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in all
// copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
// SOFTWARE.
#include "omnitrace-run.hpp"
#include "common/defines.h"
#include "common/delimit.hpp"
#include "common/environment.hpp"
#include "common/join.hpp"
#include "common/setup.hpp"
#include "core/argparse.hpp"
#include "core/config.hpp"
#include "core/state.hpp"
#include "core/timemory.hpp"
#include <timemory/environment.hpp>
#include <timemory/environment/types.hpp>
#include <timemory/log/color.hpp>
#include <timemory/settings/types.hpp>
#include <timemory/settings/vsettings.hpp>
#include <timemory/signals/signal_handlers.hpp>
#include <timemory/utility/argparse.hpp>
#include <timemory/utility/console.hpp>
#include <timemory/utility/filepath.hpp>
#include <timemory/utility/join.hpp>
#include <array>
#include <cctype>
#include <chrono>
#include <cmath>
#include <cstdint>
#include <cstdio>
#include <cstdlib>
#include <cstring>
#include <iostream>
#include <regex>
#include <stdexcept>
#include <string>
#include <string_view>
#include <sys/wait.h>
#include <thread>
#include <unistd.h>
#include <vector>
namespace color = ::tim::log::color;
namespace filepath = ::tim::filepath; // NOLINT
namespace console = ::tim::utility::console;
namespace argparse = ::tim::argparse;
namespace signals = ::tim::signals;
using settings = ::omnitrace::settings;
using namespace ::timemory::join;
using ::tim::get_env;
using ::tim::log::stream;
namespace std
{
std::string
to_string(bool _v)
{
return (_v) ? "true" : "false";
}
} // namespace std
namespace
{
std::string
get_internal_libpath(const std::string& _lib)
{
auto _exe = std::string_view{ realpath("/proc/self/exe", nullptr) };
auto _pos = _exe.find_last_of('/');
auto _dir = std::string{ "./" };
if(_pos != std::string_view::npos) _dir = _exe.substr(0, _pos);
return omnitrace::common::join("/", _dir, "..", "lib", _lib);
}
parser_data_t&
get_initial_environment(parser_data_t& _data)
{
if(environ != nullptr)
{
int idx = 0;
while(environ[idx] != nullptr)
{
auto* _v = environ[idx++];
_data.initial.emplace(_v);
_data.current.emplace_back(strdup(_v));
}
}
return _data;
}
int
get_verbose(parser_data_t& _data)
{
auto& verbose = _data.verbose;
verbose = get_env("OMNITRACE_CAUSAL_VERBOSE",
get_env<int>("OMNITRACE_VERBOSE", verbose, false));
auto _debug =
get_env("OMNITRACE_CAUSAL_DEBUG", get_env<bool>("OMNITRACE_DEBUG", false, false));
if(_debug) verbose += 8;
return verbose;
}
std::string
get_realpath(const std::string& _v)
{
auto* _tmp = realpath(_v.c_str(), nullptr);
auto _ret = std::string{ _tmp };
free(_tmp);
return _ret;
}
auto
toggle_suppression(std::tuple<bool, bool> _inp)
{
auto _out =
std::make_tuple(settings::suppress_config(), settings::suppress_parsing());
std::tie(settings::suppress_config(), settings::suppress_parsing()) = _inp;
return _out;
}
// disable suppression when exe loads but store original values for restoration later
auto initial_suppression = toggle_suppression({ true, true });
} // namespace
void
print_command(const parser_data_t& _data, std::string_view _prefix)
{
auto verbose = _data.verbose;
const auto& _argv = _data.command;
if(verbose >= 1)
stream(std::cout, color::info())
<< _prefix << "Executing '" << join(array_config{ " " }, _argv) << "'...\n";
std::cerr << color::end() << std::flush;
}
void
prepare_command_for_run(char* _exe, parser_data_t& _data)
{
if(!_data.launcher.empty())
{
bool _injected = false;
auto _new_argv = std::vector<char*>{};
for(auto* itr : _data.command)
{
if(!_injected && std::regex_search(itr, std::regex{ _data.launcher }))
{
_new_argv.emplace_back(_exe);
_new_argv.emplace_back(strdup("--"));
_injected = true;
}
_new_argv.emplace_back(itr);
}
if(!_injected)
{
throw std::runtime_error(
join("", "omnitrace-run was unable to match \"", _data.launcher,
"\" to any arguments on the command line: \"",
join(array_config{ " ", "", "" }, _data.command), "\""));
}
std::swap(_data.command, _new_argv);
}
}
void
prepare_environment_for_run(parser_data_t& _data)
{
if(_data.launcher.empty())
{
omnitrace::argparse::add_ld_preload(_data);
}
}
void
print_updated_environment(parser_data_t& _data, std::string_view _prefix)
{
auto _verbose = get_verbose(_data);
if(_verbose < 0) return;
auto _env = _data.current;
const auto& _updated_envs = _data.updated;
std::sort(_env.begin(), _env.end(), [](auto* _lhs, auto* _rhs) {
if(!_lhs) return false;
if(!_rhs) return true;
return std::string_view{ _lhs } < std::string_view{ _rhs };
});
std::vector<std::string_view> _updates = {};
std::vector<std::string_view> _general = {};
for(auto* itr : _env)
{
if(itr == nullptr) continue;
auto _is_omni = (std::string_view{ itr }.find("OMNITRACE") == 0);
auto _updated = false;
for(const auto& vitr : _updated_envs)
{
if(std::string_view{ itr }.find(vitr) == 0)
{
_updated = true;
break;
}
}
if(_updated)
_updates.emplace_back(itr);
else if(_verbose >= 1 && _is_omni)
_general.emplace_back(itr);
}
if(_general.size() + _updates.size() == 0 || _verbose < 0) return;
std::cerr << std::endl;
for(auto& itr : _general)
stream(std::cerr, color::source()) << _prefix << itr << "\n";
for(auto& itr : _updates)
stream(std::cerr, color::source()) << _prefix << itr << "\n";
std::cerr << color::end() << std::flush;
}
parser_data_t&
parse_args(int argc, char** argv, parser_data_t& _parser_data)
{
get_initial_environment(_parser_data);
bool _do_parse_args = false;
for(int i = 1; i < argc; ++i)
{
auto _arg = std::string_view{ argv[i] };
if(_arg == "--" || _arg == "-?" || _arg == "-h" || _arg == "--help" ||
_arg == "--version")
_do_parse_args = true;
}
if(!_do_parse_args && argc > 1 && std::string_view{ argv[1] }.find('-') == 0)
_do_parse_args = true;
if(!_do_parse_args) return parse_command(argc, argv, _parser_data);
using parser_t = argparse::argument_parser;
using parser_err_t = typename parser_t::result_type;
toggle_suppression(initial_suppression);
omnitrace::argparse::init_parser(_parser_data);
// no need for backtraces
signals::disable_signal_detection(signals::signal_settings::get_enabled());
auto help_check = [](parser_t& p, int _argc, char** _argv) {
std::set<std::string> help_args = { "-h", "--help", "-?" };
return (p.exists("help") || _argc == 1 ||
(_argc > 1 && help_args.find(_argv[1]) != help_args.end()));
};
const auto* _desc = R"desc(
Command line interface to omnitrace configuration.
)desc";
auto parser = parser_t{ basename(argv[0]), _desc };
parser.on_error([](parser_t&, const parser_err_t& _err) {
stream(std::cerr, color::fatal()) << _err << "\n";
exit(EXIT_FAILURE);
});
parser.enable_help("", "Usage: omnitrace-run <OPTIONS> -- <COMMAND> <ARGS>");
parser.enable_version("omnitrace-run", "v" OMNITRACE_VERSION_STRING,
OMNITRACE_GIT_DESCRIBE, OMNITRACE_GIT_REVISION);
auto _cols = std::get<0>(console::get_columns());
if(_cols > parser.get_help_width() + 8)
parser.set_description_width(
std::min<int>(_cols - parser.get_help_width() - 8, 120));
// disable options related to causal profiling
_parser_data.processed_groups.emplace("causal");
omnitrace::argparse::add_core_arguments(parser, _parser_data);
omnitrace::argparse::add_extended_arguments(parser, _parser_data);
auto _inpv = std::vector<char*>{};
auto& _outv = _parser_data.command;
bool _hash = false;
for(int i = 0; i < argc; ++i)
{
if(argv[i] == nullptr)
{
continue;
}
else if(_hash)
{
_outv.emplace_back(strdup(argv[i]));
}
else if(std::string_view{ argv[i] } == "--")
{
_hash = true;
}
else
{
_inpv.emplace_back(strdup(argv[i]));
}
}
auto _cerr = parser.parse_args(_inpv.size(), _inpv.data());
if(_cerr)
{
std::cerr << _cerr.what() << std::endl;
exit(EXIT_FAILURE);
}
return _parser_data;
}
parser_data_t&
parse_command(int argc, char** argv, parser_data_t& _parser_data)
{
toggle_suppression(initial_suppression);
omnitrace::argparse::init_parser(_parser_data);
// no need for backtraces
signals::disable_signal_detection(signals::signal_settings::get_enabled());
auto& _outv = _parser_data.command;
bool _hash = false;
for(int i = 1; i < argc; ++i)
{
_outv.emplace_back(strdup(argv[i]));
}
return _parser_data;
}
+81
Просмотреть файл
@@ -0,0 +1,81 @@
// MIT License
//
// Copyright (c) 2022 Advanced Micro Devices, Inc. All Rights Reserved.
//
// Permission is hereby granted, free of charge, to any person obtaining a copy
// of this software and associated documentation files (the "Software"), to deal
// in the Software without restriction, including without limitation the rights
// to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
// copies of the Software, and to permit persons to whom the Software is
// furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in all
// copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
// SOFTWARE.
#include "omnitrace-run.hpp"
#include <timemory/log/color.hpp>
#include <timemory/log/macros.hpp>
#include <algorithm>
#include <cmath>
#include <cstdlib>
#include <iomanip>
#include <iostream>
#include <map>
#include <sstream>
#include <string_view>
#include <unistd.h>
namespace
{
auto* _getenv_at_load = getenv("TIMEMORY_LIBRARY_CTOR");
auto _setenv_at_load = setenv("TIMEMORY_LIBRARY_CTOR", "0", 0);
} // namespace
int
main(int argc, char** argv)
{
if(!_getenv_at_load)
unsetenv("TIMEMORY_LIBRARY_CTOR");
else
setenv("TIMEMORY_LIBRARY_CTOR", _getenv_at_load, 1);
auto _print_usage = [argv]() {
std::cerr << tim::log::color::fatal() << "Usage: " << argv[0]
<< " <OPTIONS> -- <COMMAND> <ARGS>" << std::endl;
};
if(argc == 1)
{
_print_usage();
return EXIT_FAILURE;
}
auto _parse_data = parser_data_t{};
parse_args(argc, argv, _parse_data);
prepare_command_for_run(argv[0], _parse_data);
prepare_environment_for_run(_parse_data);
auto& _argv = _parse_data.command;
auto& _envp = _parse_data.current;
if(!_argv.empty())
{
print_updated_environment(_parse_data, "OMNITRACE: ");
print_command(_parse_data, "OMNITRACE: ");
_argv.emplace_back(nullptr);
_envp.emplace_back(nullptr);
return execvpe(_argv.front(), _argv.data(), _envp.data());
}
_print_usage();
return EXIT_FAILURE;
}
+53
Просмотреть файл
@@ -0,0 +1,53 @@
// MIT License
//
// Copyright (c) 2022 Advanced Micro Devices, Inc. All Rights Reserved.
//
// Permission is hereby granted, free of charge, to any person obtaining a copy
// of this software and associated documentation files (the "Software"), to deal
// in the Software without restriction, including without limitation the rights
// to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
// copies of the Software, and to permit persons to whom the Software is
// furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in all
// copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
// SOFTWARE.
#pragma once
#include "core/argparse.hpp"
#include <csignal>
#include <map>
#include <sched.h>
#include <set>
#include <string>
#include <string_view>
#include <vector>
using parser_data_t = omnitrace::argparse::parser_data;
void
print_command(const parser_data_t&, std::string_view);
void
print_updated_environment(parser_data_t&, std::string_view);
void
prepare_command_for_run(char*, parser_data_t&);
void
prepare_environment_for_run(parser_data_t&);
parser_data_t&
parse_args(int argc, char** argv, parser_data_t&);
parser_data_t&
parse_command(int argc, char** argv, parser_data_t&);
+4 -4
Просмотреть файл
@@ -117,7 +117,7 @@ print_command(const std::vector<char*>& _argv)
std::vector<char*>
get_initial_environment()
{
std::vector<char*> _env;
auto _env = std::vector<char*>{};
if(environ != nullptr)
{
int idx = 0;
@@ -497,7 +497,7 @@ parse_args(int argc, char** argv, std::vector<char*>& _env)
.count(1)
.dtype("filepath")
.action([&](parser_t& p) {
update_env(_env, "OMNITRACE_PERFETTO_FILE", p.get<int64_t>("trace-file"));
update_env(_env, "OMNITRACE_PERFETTO_FILE", p.get<std::string>("trace-file"));
});
parser
.add_argument({ "--trace-buffer-size" },
@@ -634,7 +634,7 @@ parse_args(int argc, char** argv, std::vector<char*>& _env)
.requires({ "host" })
.action([&](parser_t& p) {
update_env(
_env, "OMNITRACE_PROCESS_SAMPLING_CPUS",
_env, "OMNITRACE_SAMPLING_CPUS",
join(array_config{ "," }, p.get<std::vector<std::string>>("cpus")));
});
parser
@@ -644,7 +644,7 @@ parse_args(int argc, char** argv, std::vector<char*>& _env)
.requires({ "device" })
.action([&](parser_t& p) {
update_env(
_env, "OMNITRACE_PROCESS_SAMPLING_GPUS",
_env, "OMNITRACE_SAMPLING_GPUS",
join(array_config{ "," }, p.get<std::vector<std::string>>("gpus")));
});
+116
Просмотреть файл
@@ -410,3 +410,119 @@ omnitrace_add_bin_test(
PASS_REGEX
"OMNITRACE_CONFIG_FILE(.*)OMNITRACE_ENABLED(.*)OMNITRACE_SUPPRESS_CONFIG(.*)OMNITRACE_SUPPRESS_PARSING(.*)OMNITRACE_VERBOSE"
)
omnitrace_add_bin_test(
NAME omnitrace-run-help
TARGET omnitrace-run
ARGS --help
TIMEOUT 45
LABELS "omnitrace-run")
file(MAKE_DIRECTORY "${PROJECT_BINARY_DIR}/omnitrace-tests-config")
file(
WRITE "${PROJECT_BINARY_DIR}/omnitrace-tests-config/empty.cfg"
"
#
# empty config file
#
")
add_executable(sleeper ${CMAKE_CURRENT_SOURCE_DIR}/sleeper.cpp)
set_target_properties(
sleeper PROPERTIES BUILD_TYPE RelWithDebInfo RUNTIME_OUTPUT_DIRECTORY
${PROJECT_BINARY_DIR}/bin/testing)
omnitrace_add_bin_test(
NAME omnitrace-run-args
TARGET omnitrace-run
ARGS --monochrome
--debug=false
-v
1
-c
%env{TWD}%/omnitrace-tests-config/empty.cfg
-o
omnitrace-tests-output
omnitrace-run-args-output/
-TPHD
-S
cputime
realtime
--trace-wait=1.0e-12
--trace-duration=5.0
--wait=1.0
--duration=3.0
--trace-file=perfetto-run-args-trace.proto
--trace-buffer-size=100
--trace-fill-policy=ring_buffer
--profile-format
console
json
text
--process-freq
1000
--process-wait
0.0
--process-duration
10
--cpus
0-4
--gpus
0
-f
1000
--sampling-wait
1.0
--sampling-duration
10
-t
0-3
--sample-cputime
1000
1.0
0-3
--sample-realtime
10
0.5
0-3
-I
all
-E
mutex-locks
rw-locks
spin-locks
-C
perf::INSTRUCTIONS
--inlines
--hsa-interrupt
0
--use-causal=false
--use-kokkosp
--num-threads-hint=4
--sampling-allocator-size=32
--ci
--dl-verbose=3
--perfetto-annotations=off
--kokkosp-kernel-logger
--kokkosp-name-length-max=1024
--kokkosp-prefix="[kokkos]"
--tmpdir
${CMAKE_BINARY_DIR}/omnitrace-tests-config/tmpdir
--perfetto-backend
inprocess
--use-pid
false
--time-output
off
--thread-pool-size
0
--timemory-components
wall_clock
cpu_clock
peak_rss
page_rss
--
$<TARGET_FILE:sleeper>
5
TIMEOUT 45
LABELS "omnitrace-run")
+28
Просмотреть файл
@@ -0,0 +1,28 @@
#include <chrono>
#include <ratio>
#include <sstream>
#include <thread>
using clock_type = std::chrono::steady_clock;
int
main(int argc, char** argv)
{
double _val = 0.0;
if(argc > 0)
{
auto _ss = std::stringstream{};
_ss << argv[1];
_ss >> _val;
}
intmax_t _nsec = _val * std::nano::den;
auto _end = clock_type::now() + std::chrono::nanoseconds{ _nsec };
while(clock_type::now() < _end)
{
std::this_thread::sleep_for(std::chrono::nanoseconds{ _nsec / 10 });
}
return 0;
}
+1 -1
Просмотреть файл
@@ -97,7 +97,7 @@ e.g., omnitrace's meaning of the term "module" when instrumenting Python.
- Binary rewrites only instrument the functions defined in the target binary, whereas runtime instrumentation can/will instrument functions defined in the shared libraries which are linked into the target binary
- When using binary instrumentation with MPI, avoid runtime instrumentation
- Runtime instrumentation requires a fork + ptrace: which is generally incompatible with how MPI applications spawn their processes
- Binary rewrite the executable using MPI (and, optionally, libraries used by the executable) and execute the generated instrumented executable instead of the original, e.g. `mpirun -n 2 ./myexe` should be `mpirun -n 2 ./myexe.inst` where `myexe.inst` is the generated instrumented `myexe` executable.
- Binary rewrite the executable using MPI (and, optionally, libraries used by the executable) and execute the generated instrumented executable via `omnitrace-run` instead of the original, e.g. `mpirun -n 2 ./myexe` should be `mpirun -n 2 omnitrace-run -- ./myexe.inst` where `myexe.inst` is the generated instrumented `myexe` executable.
## Data Collection Mode(s)
+4 -4
Просмотреть файл
@@ -335,7 +335,7 @@ is consistent with the LLVM style of using a standalone double-hyphen (`--`). Al
are interpreted as belonging to omnitrace and all arguments following the double-hyphen are interpreted as the
application and it's arguments. In binary rewrite mode, all application arguments after the first argument
are ignored, i.e. `./omnitrace-instrument -o ls.inst -- ls -l` interprets `ls` as the target to instrument (ignores the `-l` argument)
and generates a `ls.inst` executable that you can subsequently run `ls.inst -l` with.
and generates a `ls.inst` executable that you can subsequently run `omnitrace-run -- ls.inst -l` with.
## Runtime Instrumentation
@@ -781,7 +781,7 @@ For example, if the following sequence of commands are run:
omnitrace-instrument -o ./foo.inst -- ./foo
export OMNITRACE_USE_SAMPLING=ON
export OMNITRACE_SAMPLING_FREQ=5
./foo.inst
omnitrace-run -- ./foo.inst
```
These configuration settings will not be preserved in another session, whereas:
@@ -794,7 +794,7 @@ will preserve those environment variables:
```shell
# will sample 5x per second
./foo.samp
omnitrace-run -- ./foo.samp
```
while still allowing the subsequent session to override those defaults:
@@ -802,7 +802,7 @@ while still allowing the subsequent session to override those defaults:
```shell
# will sample 100x per second
export OMNITRACE_SAMPLING_FREQ=100
./foo.samp
omnitrace-run -- ./foo.samp
```
### Troubleshooting
+1 -1
Просмотреть файл
@@ -15,7 +15,7 @@ Call-stack sampling can be activated with either a binary instrumented via the `
```console
omnitrace-instrument -M sampling -o foo.inst -- foo
./foo.inst
omnitrace-run -- ./foo.inst
```
- Runtime instrumentation with only instrumentation necessary to start/stop sampling
+1 -5
Просмотреть файл
@@ -169,11 +169,7 @@ custom_push_region(const char* name)
```console
$ omnitrace-instrument -l --min-instructions=8 -E custom_push_region -o -- ./user-api
...
$ export OMNITRACE_USE_TIMEMORY=ON
$ export OMNITRACE_USE_PID=OFF
$ export OMNITRACE_TIME_OUTPUT=OFF
$ export OMNITRACE_OUTPUT_PATH=omnitrace-example-output
$ ./user-api.inst 20 4 100
$ omnitrace-run --profile --use-pid off --time-output off -- ./user-api.inst 20 4 100
Pushing custom region :: ./user-api.inst
[omnitrace][omnitrace_init_tooling] Instrumentation mode: Trace
+29 -5
Просмотреть файл
@@ -26,8 +26,10 @@
#include <cstdlib>
#include <cstring>
#include <iostream>
#include <stdexcept>
#include <string>
#include <string_view>
#include <type_traits>
namespace omnitrace
{
@@ -36,7 +38,7 @@ inline namespace common
namespace
{
inline std::string
get_env(std::string_view env_id, std::string_view _default)
get_env_impl(std::string_view env_id, std::string_view _default)
{
if(env_id.empty()) return std::string{ _default };
char* env_var = ::std::getenv(env_id.data());
@@ -45,13 +47,13 @@ get_env(std::string_view env_id, std::string_view _default)
}
inline std::string
get_env(std::string_view env_id, const char* _default)
get_env_impl(std::string_view env_id, const char* _default)
{
return get_env(env_id, std::string_view{ _default });
return get_env_impl(env_id, std::string_view{ _default });
}
inline int
get_env(std::string_view env_id, int _default)
get_env_impl(std::string_view env_id, int _default)
{
if(env_id.empty()) return _default;
char* env_var = ::std::getenv(env_id.data());
@@ -73,15 +75,21 @@ get_env(std::string_view env_id, int _default)
}
inline bool
get_env(std::string_view env_id, bool _default)
get_env_impl(std::string_view env_id, bool _default)
{
if(env_id.empty()) return _default;
char* env_var = ::std::getenv(env_id.data());
if(env_var)
{
if(std::string_view{ env_var }.empty())
throw std::runtime_error(std::string{ "No boolean value provided for " } +
std::string{ env_id });
if(std::string_view{ env_var }.find_first_not_of("0123456789") ==
std::string_view::npos)
{
return static_cast<bool>(std::stoi(env_var));
}
else
{
for(size_t i = 0; i < strlen(env_var); ++i)
@@ -93,6 +101,22 @@ get_env(std::string_view env_id, bool _default)
}
return _default;
}
template <typename Tp>
inline auto
get_env(std::string_view env_id, Tp&& _default)
{
if constexpr(std::is_enum<Tp>::value)
{
using Up = std::underlying_type_t<Tp>;
// cast to underlying type -> get_env -> cast to enum type
return static_cast<Tp>(get_env_impl(env_id, static_cast<Up>(_default)));
}
else
{
return get_env_impl(env_id, std::forward<Tp>(_default));
}
}
} // namespace
} // namespace common
} // namespace omnitrace
+2
Просмотреть файл
@@ -3,6 +3,7 @@ configure_file(${CMAKE_CURRENT_SOURCE_DIR}/defines.hpp.in
${CMAKE_CURRENT_BINARY_DIR}/defines.hpp @ONLY)
set(core_sources
${CMAKE_CURRENT_LIST_DIR}/argparse.cpp
${CMAKE_CURRENT_LIST_DIR}/categories.cpp
${CMAKE_CURRENT_LIST_DIR}/config.cpp
${CMAKE_CURRENT_LIST_DIR}/constraint.cpp
@@ -16,6 +17,7 @@ set(core_sources
${CMAKE_CURRENT_LIST_DIR}/timemory.cpp)
set(core_headers
${CMAKE_CURRENT_LIST_DIR}/argparse.hpp
${CMAKE_CURRENT_LIST_DIR}/categories.hpp
${CMAKE_CURRENT_LIST_DIR}/common.hpp
${CMAKE_CURRENT_LIST_DIR}/concepts.hpp
Разница между файлами не показана из-за своего большого размера Загрузить разницу
+93
Просмотреть файл
@@ -0,0 +1,93 @@
// MIT License
//
// Copyright (c) 2022 Advanced Micro Devices, Inc. All Rights Reserved.
//
// Permission is hereby granted, free of charge, to any person obtaining a copy
// of this software and associated documentation files (the "Software"), to deal
// in the Software without restriction, including without limitation the rights
// to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
// copies of the Software, and to permit persons to whom the Software is
// furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in all
// copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
// SOFTWARE.
#pragma once
#include "defines.hpp"
#include <timemory/settings/vsettings.hpp>
#include <timemory/utility/argparse.hpp>
#include <functional>
#include <set>
#include <vector>
namespace omnitrace
{
namespace argparse
{
struct parser_data;
using parser_t = ::tim::argparse::argument_parser;
using vsetting_t = ::tim::vsettings;
using vsettings_set_t = std::set<vsetting_t*>;
using strset_t = std::set<std::string>;
using strvec_t = std::vector<std::string>;
using setting_filter_t = std::function<bool(vsetting_t*, const parser_data&)>;
using environ_filter_t = std::function<bool(std::string_view, const parser_data&)>;
using grouping_filter_t = std::function<bool(std::string_view, const parser_data&)>;
bool
default_setting_filter(vsetting_t*, const parser_data&);
bool
default_environ_filter(std::string_view, const parser_data&);
bool
default_grouping_filter(std::string_view, const parser_data&);
struct parser_data
{
bool monochrome = false;
bool debug = false;
int verbose = 0;
std::string dl_libpath = {};
std::string omni_libpath = {};
std::string launcher = {};
vsettings_set_t processed_settings = {};
std::set<std::string> processed_environs = {};
std::set<std::string> processed_groups = {};
std::vector<char*> current = {};
std::vector<char*> command = {};
std::set<std::string_view> updated = {};
std::set<std::string> initial = {};
grouping_filter_t grouping_filter = default_grouping_filter;
setting_filter_t setting_filter = default_setting_filter;
environ_filter_t environ_filter = default_environ_filter;
};
parser_data&
init_parser(parser_data&);
parser_data&
add_ld_preload(parser_data&);
parser_data&
add_core_arguments(parser_t&, parser_data&);
parser_data&
add_group_arguments(parser_t&, const std::string&, parser_data&, bool _add_group = false);
parser_data&
add_extended_arguments(parser_t&, parser_data&);
} // namespace argparse
} // namespace omnitrace
+3 -3
Просмотреть файл
@@ -22,10 +22,10 @@
#pragma once
#include "categories.hpp"
#include "common/join.hpp"
#include "concepts.hpp"
#include "defines.hpp"
#include "core/categories.hpp"
#include "core/concepts.hpp"
#include "core/defines.hpp"
#include <timemory/api.hpp>
#include <timemory/api/macros.hpp>
+29 -13
Просмотреть файл
@@ -281,7 +281,7 @@ configure_settings(bool _init)
std::string, "OMNITRACE_MODE",
"Data collection mode. Used to set default values for OMNITRACE_USE_* options. "
"Typically set by omnitrace binary instrumenter.",
std::string{ "trace" }, "backend", "advanced")
std::string{ "trace" }, "backend", "advanced", "mode")
->set_choices({ "trace", "sampling", "causal", "coverage" });
OMNITRACE_CONFIG_SETTING(bool, "OMNITRACE_CI",
@@ -308,7 +308,7 @@ configure_settings(bool _init)
"threads that get sampled, omnitrace can start all the background threads during "
"initialization",
get_env<size_t>("OMNITRACE_NUM_THREADS", 1), "threading", "performance",
"sampling", "debugging", "advanced");
"sampling", "parallelism", "advanced");
OMNITRACE_CONFIG_SETTING(bool, "OMNITRACE_USE_PERFETTO", "Enable perfetto backend",
_default_perfetto_v, "backend", "perfetto");
@@ -680,13 +680,13 @@ configure_settings(bool _init)
"Enable collecting profiling and trace data for these "
"categories and disable all other categories",
"", "trace", "profile", "perfetto", "timemory", "data",
"advanced")
"category", "advanced")
->set_choices(get_available_categories<std::vector<std::string>>());
OMNITRACE_CONFIG_SETTING(
std::string, "OMNITRACE_DISABLE_CATEGORIES",
"Disable collecting profiling and trace data for these categories", "", "trace",
"profile", "perfetto", "timemory", "data", "advanced")
"profile", "perfetto", "timemory", "data", "category", "advanced")
->set_choices(get_available_categories<std::vector<std::string>>());
OMNITRACE_CONFIG_SETTING(bool, "OMNITRACE_PERFETTO_ANNOTATIONS",
@@ -705,19 +705,18 @@ configure_settings(bool _init)
OMNITRACE_CONFIG_EXT_SETTING(int64_t, "OMNITRACE_CRITICAL_TRACE_COUNT",
"Number of critical trace to export (0 == all)",
int64_t{ 0 }, "data", "critical_trace",
"omnitrace-critical-trace", "perfetto", "advanced");
int64_t{ 0 }, "critical_trace",
"omnitrace-critical-trace", "advanced");
OMNITRACE_CONFIG_SETTING(uint64_t, "OMNITRACE_CRITICAL_TRACE_BUFFER_COUNT",
"Number of critical trace records to store in thread-local "
"memory before submitting to shared buffer",
uint64_t{ 2000 }, "data", "critical_trace", "advanced");
uint64_t{ 2000 }, "critical_trace", "advanced");
OMNITRACE_CONFIG_EXT_SETTING(
int64_t, "OMNITRACE_CRITICAL_TRACE_PER_ROW",
"How many critical traces per row in perfetto (0 == all in one row)",
int64_t{ 0 }, "io", "critical_trace", "omnitrace-critical-trace", "perfetto",
"advanced");
int64_t{ 0 }, "critical_trace", "omnitrace-critical-trace", "advanced");
OMNITRACE_CONFIG_SETTING(
std::string, "OMNITRACE_TIMEMORY_COMPONENTS",
@@ -1506,9 +1505,15 @@ print_banner(std::ostream& _os)
)banner";
auto _tag = std::string_view{ OMNITRACE_GIT_DESCRIBE };
auto _rev = std::string_view{ OMNITRACE_GIT_REVISION };
std::stringstream _version_info{};
#if OMNITRACE_HIP_VERSION_MAJOR > 0
auto _hip = JOIN('.', OMNITRACE_HIP_VERSION_MAJOR, OMNITRACE_HIP_VERSION_MINOR, "x");
#else
auto _hip = std::string_view{};
#endif
std::stringstream _version_info{};
_version_info << "omnitrace v" << OMNITRACE_VERSION_STRING;
if(!_tag.empty() || !_rev.empty())
if(!_tag.empty() || !_rev.empty() || !_hip.empty())
{
_version_info << " (";
if(!_tag.empty())
@@ -1516,10 +1521,21 @@ print_banner(std::ostream& _os)
_version_info << "tag: " << OMNITRACE_GIT_DESCRIBE;
if(!_rev.empty()) _version_info << ", ";
}
if(!_rev.empty()) _version_info << "rev: " << OMNITRACE_GIT_REVISION;
_version_info << ")";
if(!_rev.empty())
{
_version_info << "rev: " << OMNITRACE_GIT_REVISION;
if(!_hip.empty()) _version_info << ", ";
}
if(!_hip.empty())
{
_version_info << "rocm: " << _hip;
}
}
if(!_version_info.str().empty()) _version_info << ")";
tim::log::stream(_os, tim::log::color::info()) << _banner << _version_info.str();
_os << std::endl;
}
+4 -4
Просмотреть файл
@@ -17,8 +17,8 @@ add_library(omnitrace::omnitrace-dl-library ALIAS omnitrace-dl-library)
target_sources(
omnitrace-dl-library
PRIVATE ${CMAKE_CURRENT_SOURCE_DIR}/dl.cpp ${CMAKE_CURRENT_SOURCE_DIR}/dl.hpp
${CMAKE_CURRENT_SOURCE_DIR}/main.c)
PRIVATE ${CMAKE_CURRENT_SOURCE_DIR}/dl.cpp ${CMAKE_CURRENT_SOURCE_DIR}/main.c
${CMAKE_CURRENT_SOURCE_DIR}/dl/dl.hpp)
target_include_directories(
omnitrace-dl-library
PUBLIC $<BUILD_INTERFACE:${CMAKE_CURRENT_SOURCE_DIR}>
@@ -44,7 +44,7 @@ set_target_properties(
omnitrace_strip_target(omnitrace-dl-library)
install(FILES ${CMAKE_CURRENT_SOURCE_DIR}/dl.hpp
DESTINATION ${CMAKE_INSTALL_INCLUDEDIR}/omnitrace)
install(FILES ${CMAKE_CURRENT_SOURCE_DIR}/dl/dl.hpp
DESTINATION ${CMAKE_INSTALL_INCLUDEDIR}/omnitrace/dl)
install(TARGETS omnitrace-dl-library DESTINATION ${CMAKE_INSTALL_LIBDIR})
+364 -29
Просмотреть файл
@@ -38,12 +38,21 @@
#include "common/invoke.hpp"
#include "common/join.hpp"
#include "common/setup.hpp"
#include "dl.hpp"
#include "dl/dl.hpp"
#include "omnitrace/categories.h"
#include "omnitrace/types.h"
#include <timemory/utility/filepath.hpp>
#include <cassert>
#include <chrono>
#include <gnu/libc-version.h>
#include <link.h>
#include <linux/limits.h>
#include <sys/types.h>
#include <sys/wait.h>
#include <thread>
#include <unistd.h>
//--------------------------------------------------------------------------------------//
@@ -69,6 +78,8 @@
//--------------------------------------------------------------------------------------//
using main_func_t = int (*)(int, char**, char**);
std::ostream&
operator<<(std::ostream& _os, const SpaceHandle& _handle)
{
@@ -78,7 +89,7 @@ operator<<(std::ostream& _os, const SpaceHandle& _handle)
namespace omnitrace
{
inline namespace dl
namespace dl
{
namespace
{
@@ -97,6 +108,16 @@ get_omnitrace_dl_env()
: get_env("OMNITRACE_DL_VERBOSE", get_omnitrace_env());
}
inline bool&
get_omnitrace_is_preloaded()
{
static bool _v = []() {
auto&& _preload_libs = get_env("LD_PRELOAD", std::string{});
return (_preload_libs.find("libomnitrace-dl.so") != std::string::npos);
}();
return _v;
}
inline bool
get_omnitrace_preload()
{
@@ -136,6 +157,12 @@ get_omnitrace_root_pid()
return get_env("OMNITRACE_ROOT_PROCESS", _pid);
}
void
omnitrace_preinit() OMNITRACE_INTERNAL_API;
void
omnitrace_postinit(std::string exe = {}) OMNITRACE_INTERNAL_API;
pid_t _omnitrace_root_pid = get_omnitrace_root_pid();
// environment priority:
@@ -189,7 +216,7 @@ const char* _omnitrace_dl_dlopen_descr = "RTLD_LAZY | RTLD_LOCAL";
#endif
/// This class contains function pointers for omnitrace's instrumentation functions
struct OMNITRACE_HIDDEN_API indirect
struct OMNITRACE_INTERNAL_API indirect
{
OMNITRACE_INLINE indirect(const std::string& _omnilib, const std::string& _userlib,
const std::string& _dllib)
@@ -456,7 +483,7 @@ private:
};
inline indirect&
get_indirect() OMNITRACE_HIDDEN_API;
get_indirect() OMNITRACE_INTERNAL_API;
indirect&
get_indirect()
@@ -519,6 +546,13 @@ get_thread_status()
return _v;
}
InstrumentMode&
get_instrumented()
{
static auto _v = get_env("OMNITRACE_INSTRUMENT_MODE", InstrumentMode::None);
return _v;
}
// ensure finalization is called
bool _omnitrace_dl_fini = (std::atexit([]() {
if(get_active()) omnitrace_finalize();
@@ -555,7 +589,7 @@ bool _omnitrace_dl_fini = (std::atexit([]() {
fflush(stderr); \
}
using omnitrace::get_indirect;
using omnitrace::dl::get_indirect;
namespace dl = omnitrace::dl;
extern "C"
@@ -598,6 +632,9 @@ extern "C"
return;
}
if(dl::get_instrumented() < dl::InstrumentMode::PythonProfile)
dl::omnitrace_preinit();
bool _invoked = false;
OMNITRACE_DL_INVOKE_STATUS(_invoked, get_indirect().omnitrace_init_f, a, b, c);
if(_invoked)
@@ -605,6 +642,8 @@ extern "C"
dl::get_active() = true;
dl::get_inited() = true;
dl::_omnitrace_dl_verbose = dl::get_omnitrace_dl_env();
if(dl::get_instrumented() < dl::InstrumentMode::PythonProfile)
dl::omnitrace_postinit((c) ? std::string{ c } : std::string{});
}
}
@@ -727,8 +766,9 @@ extern "C"
OMNITRACE_DL_IGNORE(2, "already initialized and active", a, b);
return;
}
OMNITRACE_DL_LOG(2, "%s(%s, %s)\n", __FUNCTION__, a, b);
setenv(a, b, 0);
OMNITRACE_DL_INVOKE(get_indirect().omnitrace_set_env_f, a, b);
// OMNITRACE_DL_INVOKE(get_indirect().omnitrace_set_env_f, a, b);
}
void omnitrace_set_mpi(bool a, bool b)
@@ -840,6 +880,22 @@ extern "C"
_annotations, _annotation_count);
}
void omnitrace_set_instrumented(int _mode)
{
OMNITRACE_DL_LOG(2, "%s(%i)\n", __FUNCTION__, _mode);
auto _mode_v = static_cast<dl::InstrumentMode>(_mode);
if(_mode_v < dl::InstrumentMode::None || _mode_v >= dl::InstrumentMode::Last)
{
OMNITRACE_DL_LOG(-127,
"%s(mode=%i) invoked with invalid instrumentation mode. "
"mode should be %i >= mode < %i\n",
__FUNCTION__, _mode,
static_cast<int>(dl::InstrumentMode::None),
static_cast<int>(dl::InstrumentMode::Last));
}
dl::get_instrumented() = _mode_v;
}
//----------------------------------------------------------------------------------//
//
// KokkosP
@@ -1060,18 +1116,153 @@ extern "C"
namespace omnitrace
{
inline namespace dl
namespace dl
{
namespace
{
bool
omnitrace_preload() OMNITRACE_HIDDEN_API;
omnitrace_preload() OMNITRACE_INTERNAL_API;
std::vector<std::string>
get_link_map(const char*,
std::vector<int>&& = { (RTLD_LAZY | RTLD_NOLOAD) }) OMNITRACE_INTERNAL_API;
const char*
get_default_mode() OMNITRACE_INTERNAL_API;
void
verify_instrumented_preloaded() OMNITRACE_INTERNAL_API;
std::vector<std::string>
get_link_map(const char* _name, std::vector<int>&& _open_modes)
{
void* _handle = nullptr;
bool _noload = false;
for(auto _mode : _open_modes)
{
_handle = dlopen(_name, _mode);
_noload = (_mode & RTLD_NOLOAD) == RTLD_NOLOAD;
if(_handle) break;
}
auto _chain = std::vector<std::string>{};
if(_handle)
{
struct link_map* _link_map = nullptr;
dlinfo(_handle, RTLD_DI_LINKMAP, &_link_map);
struct link_map* _next = _link_map->l_next;
while(_next)
{
if(_next->l_name != nullptr && !std::string_view{ _next->l_name }.empty())
{
_chain.emplace_back(_next->l_name);
}
_next = _next->l_next;
}
if(_noload == false) dlclose(_handle);
}
return _chain;
}
const char*
get_default_mode()
{
if(get_env("OMNITRACE_USE_CAUSAL", false)) return "causal";
auto _link_map = get_link_map(nullptr);
for(const auto& itr : _link_map)
{
if(itr.find("libomnitrace-rt.so") != std::string::npos ||
itr.find("libdyninstAPI_RT.so") != std::string::npos)
return "trace";
}
return "sampling";
}
void
omnitrace_preinit()
{
switch(get_instrumented())
{
case InstrumentMode::None:
case InstrumentMode::BinaryRewrite:
case InstrumentMode::ProcessCreate:
case InstrumentMode::ProcessAttach:
{
auto _use_mpip = get_env("OMNITRACE_USE_MPIP", false);
auto _use_mpi = get_env("OMNITRACE_USE_MPI", _use_mpip);
auto _causal = get_env("OMNITRACE_USE_CAUSAL", false);
auto _mode = get_env("OMNITRACE_MODE", get_default_mode());
if(_use_mpi && !(_causal && _mode == "causal"))
{
// only make this call if true bc otherwise, if
// false, it will disable the MPIP component and
// we may intercept the MPI init call later.
// If _use_mpi defaults to true above, calling this
// will override can current env or config value for
// OMNITRACE_USE_PID.
omnitrace_set_mpi(_use_mpi, dl::get_instrumented() ==
dl::InstrumentMode::ProcessAttach);
}
break;
}
case InstrumentMode::PythonProfile:
case InstrumentMode::Last: break;
}
}
void
omnitrace_postinit(std::string _exe)
{
switch(get_instrumented())
{
case InstrumentMode::None:
case InstrumentMode::BinaryRewrite:
case InstrumentMode::ProcessCreate:
case InstrumentMode::ProcessAttach:
{
if(_exe.empty())
_exe = tim::filepath::readlink(join('/', "/proc", getpid(), "exe"));
omnitrace_init_tooling();
if(_exe.empty())
omnitrace_push_trace("main");
else
omnitrace_push_trace(basename(_exe.c_str()));
break;
}
case InstrumentMode::PythonProfile:
{
omnitrace_init_tooling();
break;
}
case InstrumentMode::Last: break;
}
}
bool
omnitrace_preload()
{
auto _preload = get_omnitrace_preload() && get_env("OMNITRACE_ENABLED", true);
auto _use_mpi = get_env("OMNITRACE_USE_MPI", get_env("OMNITRACE_USE_MPIP", false));
auto _preload = get_omnitrace_is_preloaded() && get_omnitrace_preload() &&
get_env("OMNITRACE_ENABLED", true);
auto _link_map = get_link_map(nullptr);
auto _instr_mode =
get_env("OMNITRACE_INSTRUMENT_MODE", dl::InstrumentMode::BinaryRewrite);
for(const auto& itr : _link_map)
{
if(itr.find("libomnitrace-rt.so") != std::string::npos ||
itr.find("libdyninstAPI_RT.so") != std::string::npos)
{
omnitrace_set_instrumented(static_cast<int>(_instr_mode));
break;
}
}
verify_instrumented_preloaded();
static bool _once = false;
if(_once) return _preload;
@@ -1081,30 +1272,174 @@ omnitrace_preload()
{
reset_omnitrace_preload();
omnitrace_preinit_library();
auto _causal = get_env("OMNITRACE_USE_CAUSAL", false);
auto _mode = get_env("OMNITRACE_MODE", (_causal) ? "causal" : "sampling");
OMNITRACE_DL_LOG(1, "[%s] invoking %s(%s)\n", __FUNCTION__, "omnitrace_init",
::omnitrace::join(::omnitrace::QuoteStrings{}, ", ", _mode,
false, "omnitrace")
.c_str());
if(_use_mpi && !(_causal && _mode == "causal"))
{
// only make this call if true bc otherwise, if
// false, it will disable the MPIP component and
// we may intercept the MPI init call later.
// If _use_mpi defaults to true above, calling this
// will override can current env or config value for
// OMNITRACE_USE_PID.
omnitrace_set_mpi(_use_mpi, false);
}
omnitrace_init(_mode.c_str(), false, nullptr);
omnitrace_init_tooling();
}
return _preload;
}
bool _handle_preload = omnitrace::dl::omnitrace_preload();
void
verify_instrumented_preloaded()
{
// if preloaded then we are fine
if(get_omnitrace_is_preloaded()) return;
// value returned by get_instrumented is set by either:
// - the search of the linked libraries
// - via the instrumenter
// if binary rewrite or runtime instrumentation, there is an opportunity for
// LD_PRELOAD
switch(dl::get_instrumented())
{
case dl::InstrumentMode::None:
case dl::InstrumentMode::ProcessAttach:
case dl::InstrumentMode::ProcessCreate:
{
return;
}
case dl::InstrumentMode::BinaryRewrite:
{
break;
}
case dl::InstrumentMode::Last:
{
throw std::runtime_error(
"Invalid instrumentation type: InstrumentMode::Last");
}
}
static const char* _notice = R"notice(
NNNNNNNN NNNNNNNN OOOOOOOOO TTTTTTTTTTTTTTTTTTTTTTTIIIIIIIIII CCCCCCCCCCCCCEEEEEEEEEEEEEEEEEEEEEE
N:::::::N N::::::N OO:::::::::OO T:::::::::::::::::::::TI::::::::I CCC::::::::::::CE::::::::::::::::::::E
N::::::::N N::::::N OO:::::::::::::OO T:::::::::::::::::::::TI::::::::I CC:::::::::::::::CE::::::::::::::::::::E
N:::::::::N N::::::NO:::::::OOO:::::::OT:::::TT:::::::TT:::::TII::::::IIC:::::CCCCCCCC::::CEE::::::EEEEEEEEE::::E
N::::::::::N N::::::NO::::::O O::::::OTTTTTT T:::::T TTTTTT I::::I C:::::C CCCCCC E:::::E EEEEEE
N:::::::::::N N::::::NO:::::O O:::::O T:::::T I::::IC:::::C E:::::E
N:::::::N::::N N::::::NO:::::O O:::::O T:::::T I::::IC:::::C E::::::EEEEEEEEEE
N::::::N N::::N N::::::NO:::::O O:::::O T:::::T I::::IC:::::C E:::::::::::::::E
N::::::N N::::N:::::::NO:::::O O:::::O T:::::T I::::IC:::::C E:::::::::::::::E
N::::::N N:::::::::::NO:::::O O:::::O T:::::T I::::IC:::::C E::::::EEEEEEEEEE
N::::::N N::::::::::NO:::::O O:::::O T:::::T I::::IC:::::C E:::::E
N::::::N N:::::::::NO::::::O O::::::O T:::::T I::::I C:::::C CCCCCC E:::::E EEEEEE
N::::::N N::::::::NO:::::::OOO:::::::O TT:::::::TT II::::::IIC:::::CCCCCCCC::::CEE::::::EEEEEEEE:::::E
N::::::N N:::::::N OO:::::::::::::OO T:::::::::T I::::::::I CC:::::::::::::::CE::::::::::::::::::::E
N::::::N N::::::N OO:::::::::OO T:::::::::T I::::::::I CCC::::::::::::CE::::::::::::::::::::E
NNNNNNNN NNNNNNN OOOOOOOOO TTTTTTTTTTT IIIIIIIIII CCCCCCCCCCCCCEEEEEEEEEEEEEEEEEEEEEE
_ _ _____ ______
| | | |/ ____| ____|
| | | | (___ | |__
| | | |\___ \| __|
| |__| |____) | |____
\____/|_____/|______|
____ __ __ _ _ _____ _______ _____ _____ ______ _____ _ _ _ _
/ __ \| \/ | \ | |_ _|__ __| __ \ /\ / ____| ____| | __ \| | | | \ | |
| | | | \ / | \| | | | | | | |__) | / \ | | | |__ ______| |__) | | | | \| |
| | | | |\/| | . ` | | | | | | _ / / /\ \| | | __|______| _ /| | | | . ` |
| |__| | | | | |\ |_| |_ | | | | \ \ / ____ \ |____| |____ | | \ \| |__| | |\ |
\____/|_| |_|_| \_|_____| |_| |_| \_\/_/ \_\_____|______| |_| \_\\____/|_| \_|
Due to a variety of edge cases we've encountered, OmniTrace now requires that binary rewritten executables and libraries be launched
with the 'omnitrace-run' executable.
In order to launch the executable with 'omnitrace-run', prefix the current command with 'omnitrace-run' and a standalone double hyphen ('--').
For MPI applications, place 'omnitrace-run --' after the MPI command.
E.g.:
<EXECUTABLE> <ARGS...>
mpirun -n 2 <EXECUTABLE> <ARGS...>
should be:
omnitrace-run -- <EXECUTABLE> <ARGS...>
mpirun -n 2 omnitrace-run -- <EXECUTABLE> <ARGS...>
Note: the command-line arguments passed to 'omnitrace-run' (which are specified before the double hyphen) will override configuration variables
and/or any configuration values specified to 'omnitrace-instrument' via the '--config' or '--env' options.
E.g.:
$ omnitrace-instrument -o ./sleep.inst --env OMNITRACE_SAMPLING_DELAY=5.0 -- sleep
$ echo "OMNITRACE_SAMPLING_FREQ = 500" > omnitrace.cfg
$ export OMNITRACE_CONFIG_FILE=omnitrace.cfg
$ omnitrace-run --sampling-freq=100 --sampling-delay=1.0 -- ./sleep.inst 10
In the first command, a default sampling delay of 5 seconds in embedded into the instrumented 'sleep.inst'.
In the second command, the sampling frequency will be set to 500 interrupts per second when OmniTrace reads the config file
In the fourth command, the sampling frequency and sampling delay are overridden to 100 interrupts per second and 1 second, respectively, when sleep.inst runs
Thanks for using OmniTrace and happy optimizing!
)notice";
// emit notice
std::cerr << _notice << std::endl;
std::quick_exit(EXIT_FAILURE);
}
bool _handle_preload = omnitrace_preload();
main_func_t main_real = nullptr;
} // namespace
} // namespace dl
} // namespace omnitrace
extern "C"
{
int omnitrace_main(int argc, char** argv, char** envp) OMNITRACE_INTERNAL_API;
void omnitrace_set_main(main_func_t) OMNITRACE_INTERNAL_API;
void omnitrace_set_main(main_func_t _main_real)
{
::omnitrace::dl::main_real = _main_real;
}
int omnitrace_main(int argc, char** argv, char** envp)
{
OMNITRACE_DL_LOG(0, "%s\n", __FUNCTION__);
using ::omnitrace::common::get_env;
using ::omnitrace::dl::get_default_mode;
// prevent re-entry
static int _reentry = 0;
if(_reentry > 0) return -1;
_reentry = 1;
if(!::omnitrace::dl::main_real)
throw std::runtime_error("[omnitrace][dl] Unsuccessful wrapping of main: "
"nullptr to real main function");
if(envp)
{
size_t _idx = 0;
while(envp[_idx] != nullptr)
{
auto _env_v = std::string_view{ envp[_idx++] };
if(_env_v.find("OMNITRACE") != 0 &&
_env_v.find("libomnitrace") == std::string_view::npos)
continue;
auto _pos = _env_v.find('=');
if(_pos < _env_v.length())
{
auto _var = std::string{ _env_v }.substr(0, _pos);
auto _val = std::string{ _env_v }.substr(_pos + 1);
OMNITRACE_DL_LOG(1, "%s(%s, %s)\n", "omnitrace_set_env", _var.c_str(),
_val.c_str());
setenv(_var.c_str(), _val.c_str(), 0);
}
}
}
auto _mode = get_env("OMNITRACE_MODE", get_default_mode());
omnitrace_init(_mode.c_str(),
dl::get_instrumented() == dl::InstrumentMode::BinaryRewrite,
argv[0]);
int ret = (*::omnitrace::dl::main_real)(argc, argv, envp);
omnitrace_pop_trace(basename(argv[0]));
omnitrace_finalize();
return ret;
}
}
+17
Просмотреть файл
@@ -76,6 +76,7 @@ extern "C"
void omnitrace_set_env(const char* env_name,
const char* env_val) OMNITRACE_PUBLIC_API;
void omnitrace_set_mpi(bool use, bool attached) OMNITRACE_PUBLIC_API;
void omnitrace_set_instrumented(int) OMNITRACE_PUBLIC_API;
void omnitrace_push_trace(const char* name) OMNITRACE_PUBLIC_API;
void omnitrace_pop_trace(const char* name) OMNITRACE_PUBLIC_API;
int omnitrace_push_region(const char*) OMNITRACE_PUBLIC_API;
@@ -191,4 +192,20 @@ extern "C"
#endif
}
namespace omnitrace
{
namespace dl
{
enum class InstrumentMode : int
{
None = -1,
BinaryRewrite = 0,
ProcessCreate = 1, // runtime instrumentation at start of process
ProcessAttach = 2, // runtime instrumentation of running process
PythonProfile = 3, // python setprofile
Last,
};
}
} // namespace omnitrace
#endif // OMNITRACE_DL_HPP_ 1
+12 -38
Просмотреть файл
@@ -22,8 +22,9 @@
#define _GNU_SOURCE
#define OMNITRACE_PUBLIC_API __attribute__((visibility("default")));
#define OMNITRACE_HIDDEN_API __attribute__((visibility("hidden")));
#define OMNITRACE_PUBLIC_API __attribute__((visibility("default")));
#define OMNITRACE_HIDDEN_API __attribute__((visibility("hidden")));
#define OMNITRACE_INTERNAL_API __attribute__((visibility("internal")));
#include <dlfcn.h>
#include <stdbool.h>
@@ -35,25 +36,18 @@
//
// local type definitions
//
typedef int (*main_func_t)(int, char**, char**);
typedef int (*start_main_t)(int (*)(int, char**, char**), int, char**,
int (*)(int, char**, char**), void (*)(void), void (*)(void),
void*);
//
// local variables
//
static int (*main_real)(int, char**, char**); // Trampoline for the real main()
//
// local function declarations
//
int
omnitrace_main(int, char**, char**) OMNITRACE_HIDDEN_API;
int
omnitrace_libc_start_main(int (*)(int, char**, char**), int, char**,
int (*)(int, char**, char**), void (*)(void), void (*)(void),
void*) OMNITRACE_HIDDEN_API;
void*) OMNITRACE_INTERNAL_API;
int
__libc_start_main(int (*)(int, char**, char**), int, char**, int (*)(int, char**, char**),
@@ -80,32 +74,13 @@ omnitrace_init_tooling(void);
extern void
omnitrace_init(const char*, bool, const char*);
//
// local function definitions
//
int
omnitrace_main(int argc, char** argv, char** envp)
{
// prevent re-entry
static int _reentry = 0;
if(_reentry > 0) return -1;
_reentry = 1;
extern char*
basename(const char*);
// set the relevant environment variables
// omnitrace_update_env(&envp);
extern void omnitrace_set_main(main_func_t) OMNITRACE_INTERNAL_API;
const char* mode = getenv("OMNITRACE_MODE");
omnitrace_init(mode ? mode : "sampling", false, argv[0]);
omnitrace_init_tooling();
omnitrace_push_trace(basename(argv[0]));
int ret = main_real(argc, argv, envp);
omnitrace_pop_trace(basename(argv[0]));
omnitrace_finalize();
return ret;
}
extern int
omnitrace_main(int argc, char** argv, char** envp) OMNITRACE_INTERNAL_API;
int
omnitrace_libc_start_main(int (*_main)(int, char**, char**), int _argc, char** _argv,
@@ -123,7 +98,7 @@ omnitrace_libc_start_main(int (*_main)(int, char**, char**), int _argc, char** _
void* _this_func = __builtin_return_address(0);
// Save the real main function address
main_real = _main;
omnitrace_set_main(_main);
// Find the real __libc_start_main()
start_main_t user_main = dlsym(RTLD_NEXT, "__libc_start_main");
@@ -136,8 +111,7 @@ omnitrace_libc_start_main(int (*_main)(int, char**, char**), int _argc, char** _
if(_preload == 0)
{
// call original main
return user_main(main_real, _argc, _argv, _init, _fini, _rtld_fini,
_stack_end);
return user_main(_main, _argc, _argv, _init, _fini, _rtld_fini, _stack_end);
}
else
{
+4 -3
Просмотреть файл
@@ -150,7 +150,7 @@ ensure_finalization(bool _static_init = false)
_tid->system_value);
}
if(get_env("OMNITRACE_MONOCHROME", false)) tim::log::monochrome() = true;
if(common::get_env("OMNITRACE_MONOCHROME", false)) tim::log::monochrome() = true;
(void) tim::manager::instance();
(void) tim::settings::shared_instance();
@@ -637,7 +637,7 @@ extern "C" void
omnitrace_reset_preload_hidden(void)
{
tim::set_env("OMNITRACE_PRELOAD", "0", 1);
auto&& _preload_libs = get_env("LD_PRELOAD", std::string{});
auto&& _preload_libs = common::get_env("LD_PRELOAD", std::string{});
if(_preload_libs.find("libomnitrace") != std::string::npos)
{
auto _modified_preload = std::string{};
@@ -732,7 +732,8 @@ omnitrace_finalize_hidden(void)
if(dmp::rank() == 0)
{
OMNITRACE_PRINT_F("\n");
config::print_settings(get_env<bool>("OMNITRACE_PRINT_ENV", get_debug()));
config::print_settings(
tim::get_env<bool>("OMNITRACE_PRINT_ENV", get_debug()));
}
}
+2
Просмотреть файл
@@ -36,6 +36,7 @@
#include <timemory/backends/threading.hpp>
#include <timemory/components/macros.hpp>
#include <timemory/components/timing/wall_clock.hpp>
#include <timemory/mpl/types.hpp>
#include <timemory/sampling/allocator.hpp>
#include <timemory/utility/types.hpp>
@@ -76,6 +77,7 @@ inline void
start_bundle(bundle_t& _bundle, Args&&... _args)
{
if(!get_use_timemory() && !get_use_perfetto()) return;
trait::runtime_enabled<comp::roctracer_data>::set(get_use_roctracer());
OMNITRACE_BASIC_VERBOSE_F(3, "starting bundle '%s'...\n", _bundle.key().c_str());
if constexpr(sizeof...(Args) > 0)
{
+3 -1
Просмотреть файл
@@ -21,7 +21,7 @@
// SOFTWARE.
#include "libpyomnitrace.hpp"
#include "dl.hpp"
#include "dl/dl.hpp"
#include "library/coverage.hpp"
#include "library/coverage/impl.hpp"
#include "omnitrace/categories.h"
@@ -123,6 +123,8 @@ PYBIND11_MODULE(libpyomnitrace, omni)
if(_is_initialized)
throw std::runtime_error("Error! omnitrace is already initialized");
_is_initialized = true;
omnitrace_set_instrumented(
static_cast<int>(omnitrace::dl::InstrumentMode::PythonProfile));
omnitrace_set_mpi(_get_use_mpi(), false);
std::string _cmd = {};
std::string _cmd_line = {};
+4 -4
Просмотреть файл
@@ -281,7 +281,7 @@ if(OMNITRACE_USE_MPI OR OMNITRACE_USE_MPI_HEADERS)
ENVIRONMENT
"${_flat_environment};OMNITRACE_USE_SAMPLING=OFF;OMNITRACE_STRICT_CONFIG=OFF;OMNITRACE_USE_MPIP=ON"
REWRITE_RUN_PASS_REGEX
">>> main(.*\n.*)>>> MPI_Init_thread(.*\n.*)>>> pthread_create(.*\n.*)>>> MPI_Comm_size(.*\n.*)>>> MPI_Comm_rank(.*\n.*)>>> MPI_Barrier(.*\n.*)>>> MPI_Alltoall"
">>> mpi-flat-mpip.inst(.*\n.*)>>> MPI_Init_thread(.*\n.*)>>> pthread_create(.*\n.*)>>> MPI_Comm_size(.*\n.*)>>> MPI_Comm_rank(.*\n.*)>>> MPI_Barrier(.*\n.*)>>> MPI_Alltoall"
)
omnitrace_add_test(
@@ -303,7 +303,7 @@ if(OMNITRACE_USE_MPI OR OMNITRACE_USE_MPI_HEADERS)
0
ENVIRONMENT "${_flat_environment};OMNITRACE_USE_SAMPLING=OFF"
REWRITE_RUN_PASS_REGEX
">>> main(.*\n.*)>>> MPI_Init_thread(.*\n.*)>>> pthread_create(.*\n.*)>>> MPI_Comm_size(.*\n.*)>>> MPI_Comm_rank(.*\n.*)>>> MPI_Barrier(.*\n.*)>>> MPI_Alltoall"
">>> mpi-flat.inst(.*\n.*)>>> MPI_Init_thread(.*\n.*)>>> pthread_create(.*\n.*)>>> MPI_Comm_size(.*\n.*)>>> MPI_Comm_rank(.*\n.*)>>> MPI_Barrier(.*\n.*)>>> MPI_Alltoall"
)
set(_mpip_environment
@@ -724,7 +724,7 @@ omnitrace_add_validation_test(
LABELS "time-window"
FAIL_REGEX "outer_d"
ARGS -l
main
trace-time-window.inst
outer_a
outer_b
outer_c
@@ -749,7 +749,7 @@ omnitrace_add_validation_test(
LABELS "time-window"
FAIL_REGEX "outer_d"
ARGS -l
main
trace-time-window
outer_a
outer_b
outer_c
+5 -5
Просмотреть файл
@@ -257,7 +257,7 @@ endif()
function(OMNITRACE_WRITE_TEST_CONFIG _FILE _ENV)
set(_ENV_ONLY
"OMNITRACE_(MODE|USE_MPIP|DEBUG_SETTINGS|FORCE_ROCPROFILER_INIT|DEFAULT_MIN_INSTRUCTIONS|MONOCHROME|VERBOSE)="
"OMNITRACE_(CI|MODE|USE_MPIP|DEBUG_SETTINGS|FORCE_ROCPROFILER_INIT|DEFAULT_MIN_INSTRUCTIONS|MONOCHROME|VERBOSE)="
)
set(_FILE_CONTENTS)
set(_ENV_CONTENTS)
@@ -434,8 +434,8 @@ function(OMNITRACE_ADD_TEST)
add_test(
NAME ${TEST_NAME}-binary-rewrite-run
COMMAND
${COMMAND_PREFIX} $<TARGET_FILE_DIR:${TEST_TARGET}>/${TEST_NAME}.inst
${TEST_RUN_ARGS}
${COMMAND_PREFIX} $<TARGET_FILE:omnitrace-run> --
$<TARGET_FILE_DIR:${TEST_TARGET}>/${TEST_NAME}.inst ${TEST_RUN_ARGS}
WORKING_DIRECTORY ${PROJECT_BINARY_DIR})
endif()
@@ -451,8 +451,8 @@ function(OMNITRACE_ADD_TEST)
add_test(
NAME ${TEST_NAME}-binary-rewrite-sampling-run
COMMAND
${COMMAND_PREFIX} $<TARGET_FILE_DIR:${TEST_TARGET}>/${TEST_NAME}.samp
${TEST_RUN_ARGS}
${COMMAND_PREFIX} $<TARGET_FILE:omnitrace-run> --
$<TARGET_FILE_DIR:${TEST_TARGET}>/${TEST_NAME}.samp ${TEST_RUN_ARGS}
WORKING_DIRECTORY ${PROJECT_BINARY_DIR})
endif()