Files
rocm-systems/include/library/debug.hpp
T
Jonathan R. Madsen 39f17ae8b8 rocm-smi and KokkosTools support (#23)
* renamed omnitrace_thread_data to thread_data

* initial implementation

* Numerous fixes and updates

- Updated timemory submodule
- Updated perfetto submodule (pulls in fixes for TRACE_EVENT)
- pthread_gotcha only after omnitrace_init_tooling
- omnitrace banner
- config settings for rocm-smi freq and devices
- critical_trace::get_entries
- OMNITRACE_BASIC_PRINT
- rocm_smi perfetto category
- redirect roctracer warnings for ROCm 4.5.0
- property specializations for rocm-smi components
- units fixes data_tracker types
- roctracer entries for pthread_create and start_thread
- omnitrace-avail defaults to settings, not components
- settings have conforming names
- settings warn about duplicates
- ptl named threads
- decreased max freq for sampler SIGALRM
- rocm-smi names thread
- rocm-smi avoids call to hipGetDeviceCount
- name roctracer activity callback threads
- fixed binary rewrite test output names

* Update lulesh example

- supports non-UVM GPU

* Lulesh tweaks + formatting

* KokkosP + Mode + Roctracer sampling deadlock fix

- kokkosp support
- omnitrace_init_library
- config::print_settings()
- config::get_mode()
- omnitrace::Mode
- omnitrace-avail improvements (removes settings)
- handle get_verbose() < 0
- disable dyninst InstrStackFrames by default
- handle perf_event_paranoid > 1 by disabling PAPI
- SIGALRM max freq to 5.0
- Name threads
- rocm-smi handles get_use_perfetto() and get_use_timemory()
- HSA_ENABLE_INTERRUPT=0 when roctracer + sampling (fixes deadlock)

* Tests, API renaming, roctracer

- disable renaming of thread 0
- verbprintf_bare
- enable dyninst merge tramp
- tweaked some omnitrace exe verbose levels
- reworked roctracer::setup and roctracer::shutdown
- rocm_smi::data::poll checks get_state()
- omnitrace_trace_finalize -> omnitrace_finalize
- omnitrace_trace_init -> omnitrace_init
- omnitrace_trace_set_env -> omnitrace_set_env
- omnitrace_trace_set_mpi -> omnitrace_set_mpi
- sampling mode does not disable timemory
- disable roctracer before shutting down rocm-smi
- lulesh tests w/ and w/o kokkosp
- lulesh tests for perfetto only
    - with --dynamic-callsites --traps --allow-overlapping
- lulesh tests for timemory only
    - with --stdlib --dynamic-callsites --traps --allow-overlapping

* Update timemory submodule

- fix for TIMEMORY_PROPERTY_SPECIALIZATION

* get_verbose() handling + timemory submodule update

- Findroctracer.cmake uses find_package(hsakmt)

* Stability fixes + rework roctracer + perfetto

- reworked roctracer start up
- critical_trace perfetto basic values
- perfetto sampling category
- sampler checks signals
- peak_rss in sampling
- pthread_gotcha::shutdown()
- rocm_smi::device_count()
- HSA_TOOLS_LIB is set
- HSA_ENABLE_INTERRUPT in omnitrace exe
- omnitrace exe verbosity level changes
- Avoid instrumenting Impl ns in Kokkos
- gpu::device_count prefers rocm_smi instead of hip
- ptl blocks signals
- fixed pthread_gotcha roctracer_data values
- removed runtime-instrument-sampling tests
- timemory submodule update

* cmake formatting

* timemory + roctracer updates

- fix timemory issue with papi_common
- fix timemory issue with units
- define roctracer::is_setup()

* Miscellaneous tweaks

- Disable sampling during runtime instrument
- Fixed warnings about dynamic callsites
- Fixed backtrace output when timemory disabled
- Test tweaks

* cmake-format

* omnitrace_target_compile_definitions

* timemory submodule update

* config, omnitrace, State, mpi_gotcha updates

- use OMNITRACE_THROW instead of direct throw
- is_attached()
- is_binary_rewrite()
- get_is_continuous_integration()
- get_debug_init()
- get_debug_finalize()
- max_thread_bookmarks default to 1
- State::Init
- app_thread oneTimeCode
- runtime instrumentation uses waitpid
- fixed init_names
- include main in MPI runs
- fixed sampling setup when disabled
- reworked mpi_gotcha
- disabled critical trace in transpose test

* cmake-format

* handle rocm_smi::device_count() exception

* CI timeouts

* Re-enable runtime-instrument + sampling
2022-02-08 17:42:17 -06:00

141 line
7.2 KiB
C++

// MIT License
//
// Copyright (c) 2022 Advanced Micro Devices, Inc. All Rights Reserved.
//
// Permission is hereby granted, free of charge, to any person obtaining a copy
// of this software and associated documentation files (the "Software"), to deal
// in the Software without restriction, including without limitation the rights
// to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
// copies of the Software, and to permit persons to whom the Software is
// furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in all
// copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
// SOFTWARE.
#pragma once
#include "library/defines.hpp"
#include <timemory/api.hpp>
#include <timemory/backends/dmp.hpp>
#include <timemory/backends/process.hpp>
#include <timemory/utility/utility.hpp>
#include <cstdio>
namespace omnitrace
{
inline namespace config
{
bool
get_debug();
bool
get_debug_tid();
bool
get_debug_pid();
bool
get_critical_trace_debug();
} // namespace config
} // namespace omnitrace
#if defined(TIMEMORY_USE_MPI)
# define OMNITRACE_CONDITIONAL_PRINT(COND, ...) \
if((COND) && get_debug_tid() && get_debug_pid()) \
{ \
fflush(stderr); \
tim::auto_lock_t _lk{ tim::type_mutex<decltype(std::cerr)>() }; \
fprintf(stderr, "[omnitrace][%i][%li] ", static_cast<int>(tim::dmp::rank()), \
tim::threading::get_id()); \
fprintf(stderr, __VA_ARGS__); \
fflush(stderr); \
}
#else
# define OMNITRACE_CONDITIONAL_PRINT(COND, ...) \
if((COND) && get_debug_tid() && get_debug_pid()) \
{ \
fflush(stderr); \
tim::auto_lock_t _lk{ tim::type_mutex<decltype(std::cerr)>() }; \
fprintf(stderr, "[omnitrace][%i][%li] ", \
static_cast<int>(tim::process::get_id()), tim::threading::get_id()); \
fprintf(stderr, __VA_ARGS__); \
fflush(stderr); \
}
#endif
#define OMNITRACE_CONDITIONAL_BASIC_PRINT(COND, ...) \
if((COND) && get_debug_tid() && get_debug_pid()) \
{ \
fflush(stderr); \
tim::auto_lock_t _lk{ tim::type_mutex<decltype(std::cerr)>() }; \
fprintf(stderr, "[omnitrace] "); \
fprintf(stderr, __VA_ARGS__); \
fflush(stderr); \
}
#if defined(TIMEMORY_USE_MPI)
# define OMNITRACE_CONDITIONAL_THROW(COND, ...) \
if(COND) \
{ \
char _msg_buffer[2048]; \
snprintf(_msg_buffer, 2048, "[omnitrace][%i][%li] ", \
static_cast<int>(tim::dmp::rank()), tim::threading::get_id()); \
auto len = strlen(_msg_buffer); \
snprintf(_msg_buffer + len, 2048 - len, __VA_ARGS__); \
throw std::runtime_error(_msg_buffer); \
}
#else
# define OMNITRACE_CONDITIONAL_THROW(COND, ...) \
if(COND) \
{ \
char _msg_buffer[2048]; \
snprintf(_msg_buffer, 2048, "[omnitrace][%i][%li] ", \
static_cast<int>(tim::process::get_id()), \
tim::threading::get_id()); \
auto len = strlen(_msg_buffer); \
snprintf(_msg_buffer + len, 2048 - len, __VA_ARGS__); \
throw std::runtime_error(_msg_buffer); \
}
#endif
#define OMNITRACE_CONDITIONAL_BASIC_THROW(COND, ...) \
if(COND) \
{ \
char _msg_buffer[2048]; \
snprintf(_msg_buffer, 2048, "[omnitrace] " __VA_ARGS__); \
auto len = strlen(_msg_buffer); \
snprintf(_msg_buffer + len, 2048 - len, __VA_ARGS__); \
throw std::runtime_error(_msg_buffer); \
}
#define OMNITRACE_DEBUG(...) \
OMNITRACE_CONDITIONAL_PRINT(::omnitrace::get_debug(), __VA_ARGS__)
#define OMNITRACE_PRINT(...) OMNITRACE_CONDITIONAL_PRINT(true, __VA_ARGS__)
#define OMNITRACE_CT_DEBUG(...) \
OMNITRACE_CONDITIONAL_PRINT(::omnitrace::get_critical_trace_debug(), __VA_ARGS__)
#define OMNITRACE_BASIC_PRINT(...) OMNITRACE_CONDITIONAL_BASIC_PRINT(true, __VA_ARGS__)
#define OMNITRACE_THROW(...) OMNITRACE_CONDITIONAL_THROW(true, __VA_ARGS__)
#define OMNITRACE_BASIC_THROW(...) OMNITRACE_CONDITIONAL_BASIC_THROW(true, __VA_ARGS__)
#include <string>
namespace std
{
inline std::string
to_string(bool _v)
{
return (_v) ? "true" : "false";
}
} // namespace std