39f17ae8b8
* renamed omnitrace_thread_data to thread_data
* initial implementation
* Numerous fixes and updates
- Updated timemory submodule
- Updated perfetto submodule (pulls in fixes for TRACE_EVENT)
- pthread_gotcha only after omnitrace_init_tooling
- omnitrace banner
- config settings for rocm-smi freq and devices
- critical_trace::get_entries
- OMNITRACE_BASIC_PRINT
- rocm_smi perfetto category
- redirect roctracer warnings for ROCm 4.5.0
- property specializations for rocm-smi components
- units fixes data_tracker types
- roctracer entries for pthread_create and start_thread
- omnitrace-avail defaults to settings, not components
- settings have conforming names
- settings warn about duplicates
- ptl named threads
- decreased max freq for sampler SIGALRM
- rocm-smi names thread
- rocm-smi avoids call to hipGetDeviceCount
- name roctracer activity callback threads
- fixed binary rewrite test output names
* Update lulesh example
- supports non-UVM GPU
* Lulesh tweaks + formatting
* KokkosP + Mode + Roctracer sampling deadlock fix
- kokkosp support
- omnitrace_init_library
- config::print_settings()
- config::get_mode()
- omnitrace::Mode
- omnitrace-avail improvements (removes settings)
- handle get_verbose() < 0
- disable dyninst InstrStackFrames by default
- handle perf_event_paranoid > 1 by disabling PAPI
- SIGALRM max freq to 5.0
- Name threads
- rocm-smi handles get_use_perfetto() and get_use_timemory()
- HSA_ENABLE_INTERRUPT=0 when roctracer + sampling (fixes deadlock)
* Tests, API renaming, roctracer
- disable renaming of thread 0
- verbprintf_bare
- enable dyninst merge tramp
- tweaked some omnitrace exe verbose levels
- reworked roctracer::setup and roctracer::shutdown
- rocm_smi::data::poll checks get_state()
- omnitrace_trace_finalize -> omnitrace_finalize
- omnitrace_trace_init -> omnitrace_init
- omnitrace_trace_set_env -> omnitrace_set_env
- omnitrace_trace_set_mpi -> omnitrace_set_mpi
- sampling mode does not disable timemory
- disable roctracer before shutting down rocm-smi
- lulesh tests w/ and w/o kokkosp
- lulesh tests for perfetto only
- with --dynamic-callsites --traps --allow-overlapping
- lulesh tests for timemory only
- with --stdlib --dynamic-callsites --traps --allow-overlapping
* Update timemory submodule
- fix for TIMEMORY_PROPERTY_SPECIALIZATION
* get_verbose() handling + timemory submodule update
- Findroctracer.cmake uses find_package(hsakmt)
* Stability fixes + rework roctracer + perfetto
- reworked roctracer start up
- critical_trace perfetto basic values
- perfetto sampling category
- sampler checks signals
- peak_rss in sampling
- pthread_gotcha::shutdown()
- rocm_smi::device_count()
- HSA_TOOLS_LIB is set
- HSA_ENABLE_INTERRUPT in omnitrace exe
- omnitrace exe verbosity level changes
- Avoid instrumenting Impl ns in Kokkos
- gpu::device_count prefers rocm_smi instead of hip
- ptl blocks signals
- fixed pthread_gotcha roctracer_data values
- removed runtime-instrument-sampling tests
- timemory submodule update
* cmake formatting
* timemory + roctracer updates
- fix timemory issue with papi_common
- fix timemory issue with units
- define roctracer::is_setup()
* Miscellaneous tweaks
- Disable sampling during runtime instrument
- Fixed warnings about dynamic callsites
- Fixed backtrace output when timemory disabled
- Test tweaks
* cmake-format
* omnitrace_target_compile_definitions
* timemory submodule update
* config, omnitrace, State, mpi_gotcha updates
- use OMNITRACE_THROW instead of direct throw
- is_attached()
- is_binary_rewrite()
- get_is_continuous_integration()
- get_debug_init()
- get_debug_finalize()
- max_thread_bookmarks default to 1
- State::Init
- app_thread oneTimeCode
- runtime instrumentation uses waitpid
- fixed init_names
- include main in MPI runs
- fixed sampling setup when disabled
- reworked mpi_gotcha
- disabled critical trace in transpose test
* cmake-format
* handle rocm_smi::device_count() exception
* CI timeouts
* Re-enable runtime-instrument + sampling
141 line
7.2 KiB
C++
141 line
7.2 KiB
C++
// MIT License
|
|
//
|
|
// Copyright (c) 2022 Advanced Micro Devices, Inc. All Rights Reserved.
|
|
//
|
|
// Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
// of this software and associated documentation files (the "Software"), to deal
|
|
// in the Software without restriction, including without limitation the rights
|
|
// to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
// copies of the Software, and to permit persons to whom the Software is
|
|
// furnished to do so, subject to the following conditions:
|
|
//
|
|
// The above copyright notice and this permission notice shall be included in all
|
|
// copies or substantial portions of the Software.
|
|
//
|
|
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
// SOFTWARE.
|
|
|
|
#pragma once
|
|
|
|
#include "library/defines.hpp"
|
|
|
|
#include <timemory/api.hpp>
|
|
#include <timemory/backends/dmp.hpp>
|
|
#include <timemory/backends/process.hpp>
|
|
#include <timemory/utility/utility.hpp>
|
|
|
|
#include <cstdio>
|
|
|
|
namespace omnitrace
|
|
{
|
|
inline namespace config
|
|
{
|
|
bool
|
|
get_debug();
|
|
|
|
bool
|
|
get_debug_tid();
|
|
|
|
bool
|
|
get_debug_pid();
|
|
|
|
bool
|
|
get_critical_trace_debug();
|
|
} // namespace config
|
|
} // namespace omnitrace
|
|
|
|
#if defined(TIMEMORY_USE_MPI)
|
|
# define OMNITRACE_CONDITIONAL_PRINT(COND, ...) \
|
|
if((COND) && get_debug_tid() && get_debug_pid()) \
|
|
{ \
|
|
fflush(stderr); \
|
|
tim::auto_lock_t _lk{ tim::type_mutex<decltype(std::cerr)>() }; \
|
|
fprintf(stderr, "[omnitrace][%i][%li] ", static_cast<int>(tim::dmp::rank()), \
|
|
tim::threading::get_id()); \
|
|
fprintf(stderr, __VA_ARGS__); \
|
|
fflush(stderr); \
|
|
}
|
|
#else
|
|
# define OMNITRACE_CONDITIONAL_PRINT(COND, ...) \
|
|
if((COND) && get_debug_tid() && get_debug_pid()) \
|
|
{ \
|
|
fflush(stderr); \
|
|
tim::auto_lock_t _lk{ tim::type_mutex<decltype(std::cerr)>() }; \
|
|
fprintf(stderr, "[omnitrace][%i][%li] ", \
|
|
static_cast<int>(tim::process::get_id()), tim::threading::get_id()); \
|
|
fprintf(stderr, __VA_ARGS__); \
|
|
fflush(stderr); \
|
|
}
|
|
#endif
|
|
|
|
#define OMNITRACE_CONDITIONAL_BASIC_PRINT(COND, ...) \
|
|
if((COND) && get_debug_tid() && get_debug_pid()) \
|
|
{ \
|
|
fflush(stderr); \
|
|
tim::auto_lock_t _lk{ tim::type_mutex<decltype(std::cerr)>() }; \
|
|
fprintf(stderr, "[omnitrace] "); \
|
|
fprintf(stderr, __VA_ARGS__); \
|
|
fflush(stderr); \
|
|
}
|
|
|
|
#if defined(TIMEMORY_USE_MPI)
|
|
# define OMNITRACE_CONDITIONAL_THROW(COND, ...) \
|
|
if(COND) \
|
|
{ \
|
|
char _msg_buffer[2048]; \
|
|
snprintf(_msg_buffer, 2048, "[omnitrace][%i][%li] ", \
|
|
static_cast<int>(tim::dmp::rank()), tim::threading::get_id()); \
|
|
auto len = strlen(_msg_buffer); \
|
|
snprintf(_msg_buffer + len, 2048 - len, __VA_ARGS__); \
|
|
throw std::runtime_error(_msg_buffer); \
|
|
}
|
|
#else
|
|
# define OMNITRACE_CONDITIONAL_THROW(COND, ...) \
|
|
if(COND) \
|
|
{ \
|
|
char _msg_buffer[2048]; \
|
|
snprintf(_msg_buffer, 2048, "[omnitrace][%i][%li] ", \
|
|
static_cast<int>(tim::process::get_id()), \
|
|
tim::threading::get_id()); \
|
|
auto len = strlen(_msg_buffer); \
|
|
snprintf(_msg_buffer + len, 2048 - len, __VA_ARGS__); \
|
|
throw std::runtime_error(_msg_buffer); \
|
|
}
|
|
#endif
|
|
|
|
#define OMNITRACE_CONDITIONAL_BASIC_THROW(COND, ...) \
|
|
if(COND) \
|
|
{ \
|
|
char _msg_buffer[2048]; \
|
|
snprintf(_msg_buffer, 2048, "[omnitrace] " __VA_ARGS__); \
|
|
auto len = strlen(_msg_buffer); \
|
|
snprintf(_msg_buffer + len, 2048 - len, __VA_ARGS__); \
|
|
throw std::runtime_error(_msg_buffer); \
|
|
}
|
|
|
|
#define OMNITRACE_DEBUG(...) \
|
|
OMNITRACE_CONDITIONAL_PRINT(::omnitrace::get_debug(), __VA_ARGS__)
|
|
#define OMNITRACE_PRINT(...) OMNITRACE_CONDITIONAL_PRINT(true, __VA_ARGS__)
|
|
#define OMNITRACE_CT_DEBUG(...) \
|
|
OMNITRACE_CONDITIONAL_PRINT(::omnitrace::get_critical_trace_debug(), __VA_ARGS__)
|
|
#define OMNITRACE_BASIC_PRINT(...) OMNITRACE_CONDITIONAL_BASIC_PRINT(true, __VA_ARGS__)
|
|
|
|
#define OMNITRACE_THROW(...) OMNITRACE_CONDITIONAL_THROW(true, __VA_ARGS__)
|
|
#define OMNITRACE_BASIC_THROW(...) OMNITRACE_CONDITIONAL_BASIC_THROW(true, __VA_ARGS__)
|
|
|
|
#include <string>
|
|
|
|
namespace std
|
|
{
|
|
inline std::string
|
|
to_string(bool _v)
|
|
{
|
|
return (_v) ? "true" : "false";
|
|
}
|
|
} // namespace std
|