Files
rocm-systems/include/avail.hpp
T
Jonathan R. Madsen 778af2a760 Sampling support + testing + omnitrace namespace (#19)
* omnitrace namespace

* Kokkos + Lulesh example/tests

* Sampling support + more

- OMNITRACE_BUILD_TESTING option
- sampling support
- pthread_gotcha
- fixes to labels for mpi_gotcha, fork_gotcha, omnitrace_component
- tasking::block_signals, tasking::unblock_signals
- instrumentation mode option in omnitrace exe
- argument option groups in omnitrace exe
- categories in omnitrace settings
- remove TIMEMORY_ prefixed options

* Release workflow updates

* Updated settings printing

* Fixed defaults in README

* Tweak setting defaults in README

* CMake fixes

* cmake-format

* clang-format

* LULESH_USE_MPI OFF

* LULESH_USE_MPI fix

* timemory add_secondary fix

* timemory ambiguous internal namespace fix

* Update timemory submodule

* Handle output path/prefix in omnitrace

- updated timemory
- updated test environment

* sampling + papi fix

* Fix to sampling without PAPI

* Fix for using too many processors in CI

* formatting

* Updated CI

- minor cmake tweaks
- updated timemory submodule

* Updated CI

* Updated CI

* CI + timemory updates

- data race fixes

* CI updates + debug for sampling

* Sampling updates

- moved tasking::{block,unblock}_signals to sampling namespace
- improvements to sampling w.r.t. thread-locality

* Minimum OMNITRACE_THREAD_COUNT of 128

* Handle multiple dims in sampler data

* Configure libunwind support for timemory

* Improved safeguards for sampling

- updated CI
- lulesh runtime-instrument test tweak

* formatting

* CI updates + sampler updates + misc

- fixed stack-buffer-overflow in omnitrace (get_*file_line_info)
- test labels
- steady_clock instead of system_clock in sampler
- update dyninst submodule with upgradePlaceholder fix
- disable OMNITRACE_BUILD_TESTING by default

* Updated timemory submodule

- hidden visibility for timemory
- storage finalizers do not capture this

* Update timemory submodule

- component visibility updates

* Reworked header includes

- use <...> for timemory headers
- always include <library/defines.hpp>

* Rename some config options

* Update PTL submodule

* Update kokkos submodule

* Updated sampling

* Updated CI

* Reworked instrumentation exe

- lowered min-address-range threshold to 256
- extended whole function exclude

* CI fix + timemory submodule update

- TIMEMORY_VISIBLE on component base
- RelWithDebugInfo -> RelWithDebInfo
- Info output for parallel-overhead

* Sampling flags + transpose update + CI update

- disable critical trace for parallel-overhead in CI
- SA_RESTART only in sampler
- reworked transpose example to use fewer threads

* CI update

- removed ubuntu-focal-external-debug
- reduced data artifacts upload

* CI timeouts

- updated timemory submodule
- minor tweaks to omnitrace exe logging

* LICENSE updates (partial)

* CI Test stage timeout extension

* Docker and Packaging updates

* Miscellaneous fixes/tweaks

- gpu.hpp / gpu.cpp
- disable roctracer component if no devices
- re-enable InstrStackFrames by default
- disable sampling by default
- pthread_gotcha::m_enable_sampling is false by default
- timemory submodule update w/ sampler and pop(tid) updates
- fix minor bug in sampler logic
- CMake: OMNITRACE_USE_HIP option
- roctracer + timemory fix

* Replaced OMNITRACE_USE_ROCTRACER with OMNITRACE_USE_HIP where appropriate

* cmake format

* Sampler deadlock fixes

* Removed debug messages from sampler

* Fix for MPI detection + test tweaks + misc

* Sampler deadlock fixes + misc

- removed papi_tot_ins
- pthread_gotcha blocks signals globally until sampler is setup
- metadata specialization for sampling components
- OMNITRACE_INSTRUMENTATION_MODE -> OMNITRACE_MODE
- default sampling delay increased to 0.05 from 1.0e-6
- removed {block,unblock}_signals from critical_trace and ptl
    - no longer necessary to use
- sampling delay minimum is 1.0e-3
- OMNITRACE_BUILD_HIDDEN_VISIBILITY

* omnitrace-avail + libunwind update + restructure

- restructured omnitrace components
- build custom omnitrace-avail executable
- updated libunwind to avoid malloc in get_unw_backtrace

* Fix remaining reorganization issues

- removed some duplicate code
- fixed some trait specializations after implicit instatiation
- formatting

* ensure_storage fix + avail improvements

- fix ensure_storage when component not avail
- suppress irrelevant info in omnitrace-avail

* Delay settings initialization

- slight tweak to tests w/ MPI

* Disable OpenMPI testing w/ ubuntu-bionic

- MPI testing is hanging bc of network interface issue on system:

> [[20462,1],0]: A high-performance Open MPI point-to-point messaging module
> was unable to find any relevant network interfaces:
> Module: OpenFabrics (openib)
>   Host: fv-az19-371
> Another transport will be used instead, although this may result in
> lower performance.
> NOTE: You can disable this warning by setting the MCA parameter
> btl_base_warn_component_unused to 0.
2022-01-24 20:49:17 -06:00

338 строки
11 KiB
C++

// MIT License
//
// Copyright (c) 2020, The Regents of the University of California,
// through Lawrence Berkeley National Laboratory (subject to receipt of any
// required approvals from the U.S. Dept. of Energy). All rights reserved.
//
// Permission is hereby granted, free of charge, to any person obtaining a copy
// of this software and associated documentation files (the "Software"), to
// deal in the Software without restriction, including without limitation the
// rights to use, copy, modify, merge, publish, distribute, sublicense, and
// copies of the Software, and to permit persons to whom the Software is
// furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in
// all copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
// FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
// IN THE SOFTWARE.
/** \file timemory/tools/available.hpp
* \headerfile tools/available.hpp "tools/available.hpp"
* Handles serializing the settings
*
*/
#pragma once
#define TIMEMORY_DISABLE_BANNER
#define TIMEMORY_DISABLE_COMPONENT_STORAGE_INIT
#include "timemory/settings/macros.hpp"
#include "timemory/tpls/cereal/archives.hpp"
#include "timemory/tpls/cereal/cereal/external/base64.hpp"
#include "timemory/utility/demangle.hpp"
#include <algorithm>
#include <array>
#include <functional>
#include <iomanip>
#include <sstream>
#include <stack>
#include <string>
#include <tuple>
#include <utility>
#include <vector>
#if !defined(TIMEMORY_CEREAL_EPILOGUE_FUNCTION_NAME)
# define TIMEMORY_CEREAL_EPILOGUE_FUNCTION_NAME epilogue
#endif
#if !defined(TIMEMORY_CEREAL_PROLOGUE_FUNCTION_NAME)
# define TIMEMORY_CEREAL_PROLOGUE_FUNCTION_NAME prologue
#endif
//======================================================================================//
namespace tim
{
namespace cereal
{
class SettingsTextArchive
: public OutputArchive<SettingsTextArchive>
, public traits::TextArchive
{
public:
using width_type = std::vector<uint64_t>;
using value_type = std::string;
using entry_type = std::map<std::string, value_type>;
using array_type = std::vector<entry_type>;
using unique_set = std::set<std::string>;
using int_stack = std::stack<uint32_t>;
public:
//! Construct, outputting to the provided stream
/// \param stream The array of output data
SettingsTextArchive(array_type& stream, unique_set exclude)
: OutputArchive<SettingsTextArchive>(this)
, output_stream(&stream)
, exclude_stream(std::move(exclude))
{
name_counter.push(0);
}
~SettingsTextArchive() override = default;
void saveBinaryValue(const void* data, size_t size, const char* name = nullptr)
{
setNextName(name);
writeName();
auto base64string =
base64::encode(reinterpret_cast<const unsigned char*>(data), size);
saveValue(base64string);
}
void startNode() { name_counter.push(0); }
void finishNode() { name_counter.pop(); }
//! Sets the name for the next node created with startNode
void setNextName(const char* name)
{
if(exclude_stream.count(name) > 0) return;
if((current_entry != nullptr) && value_keys.count(name) > 0)
{
current_entry->insert({ name, "" });
current_value = &((*current_entry)[name]);
return;
}
if(value_keys.count(name) > 0)
{
return;
}
current_value = nullptr;
output_stream->push_back(entry_type{});
current_entry = &(output_stream->back());
current_entry->insert({ "identifier", name });
std::string func = name;
const std::string prefix = TIMEMORY_SETTINGS_PREFIX;
func = func.erase(0, prefix.length());
std::transform(func.begin(), func.end(), func.begin(),
[](char& c) { return tolower(c); });
{
std::stringstream ss;
ss << "settings::" << func << "()";
current_entry->insert({ "static_accessor", ss.str() });
}
{
std::stringstream ss;
ss << "settings::instance()->get_" << func << "()";
current_entry->insert({ "member_accessor", ss.str() });
}
{
std::stringstream ss;
ss << "settings." << func;
current_entry->insert({ "python_accessor", ss.str() });
}
}
void setNextType(const char*) {}
public:
template <typename Tp>
inline void saveValue(Tp _val)
{
std::stringstream ssval;
ssval << std::boolalpha << _val;
if(current_value)
{
*current_value = ssval.str();
}
}
void writeName() {}
void makeArray() {}
private:
value_type* current_value = nullptr;
entry_type* current_entry = nullptr;
array_type* output_stream = nullptr;
unique_set exclude_stream = {};
int_stack name_counter;
unique_set value_keys = { "name", "value", "description", "count",
"environ", "max_count", "cmdline", "data_type",
"initial", "categories" };
};
//======================================================================================//
//
// prologue and epilogue functions
//
//======================================================================================//
//--------------------------------------------------------------------------------------//
//! Prologue for NVPs for settings archive
/*! NVPs do not start or finish nodes - they just set up the names */
template <typename T>
inline void
TIMEMORY_CEREAL_PROLOGUE_FUNCTION_NAME(SettingsTextArchive&, const NameValuePair<T>&)
{}
//--------------------------------------------------------------------------------------//
//! Epilogue for NVPs for settings archive
/*! NVPs do not start or finish nodes - they just set up the names */
template <typename T>
inline void
TIMEMORY_CEREAL_EPILOGUE_FUNCTION_NAME(SettingsTextArchive&, const NameValuePair<T>&)
{}
//--------------------------------------------------------------------------------------//
//! Prologue for deferred data for settings archive
/*! Do nothing for the defer wrapper */
template <typename T>
inline void
TIMEMORY_CEREAL_PROLOGUE_FUNCTION_NAME(SettingsTextArchive&, const DeferredData<T>&)
{}
//--------------------------------------------------------------------------------------//
//! Epilogue for deferred for settings archive
/*! NVPs do not start or finish nodes - they just set up the names */
template <typename T>
inline void
TIMEMORY_CEREAL_EPILOGUE_FUNCTION_NAME(SettingsTextArchive&, const DeferredData<T>&)
{}
//--------------------------------------------------------------------------------------//
//! Prologue for SizeTags for settings archive
/*! SizeTags are ignored */
template <typename T>
inline void
TIMEMORY_CEREAL_PROLOGUE_FUNCTION_NAME(SettingsTextArchive& ar, const SizeTag<T>&)
{
ar.makeArray();
}
//--------------------------------------------------------------------------------------//
//! Epilogue for SizeTags for settings archive
/*! SizeTags are ignored */
template <typename T>
inline void
TIMEMORY_CEREAL_EPILOGUE_FUNCTION_NAME(SettingsTextArchive&, const SizeTag<T>&)
{}
//--------------------------------------------------------------------------------------//
//! Prologue for all other types for settings archive
/*! Starts a new node, named either automatically or by some NVP,
that may be given data by the type about to be archived*/
template <typename T>
inline void
TIMEMORY_CEREAL_PROLOGUE_FUNCTION_NAME(SettingsTextArchive& ar, const T&)
{
ar.startNode();
}
//--------------------------------------------------------------------------------------//
//! Epilogue for all other types other for settings archive
/*! Finishes the node created in the prologue*/
template <typename T>
inline void
TIMEMORY_CEREAL_EPILOGUE_FUNCTION_NAME(SettingsTextArchive& ar, const T&)
{
ar.finishNode();
}
//--------------------------------------------------------------------------------------//
//! Prologue for arithmetic types for settings archive
inline void
TIMEMORY_CEREAL_PROLOGUE_FUNCTION_NAME(SettingsTextArchive&, const std::nullptr_t&)
{}
//--------------------------------------------------------------------------------------//
//! Epilogue for arithmetic types for settings archive
inline void
TIMEMORY_CEREAL_EPILOGUE_FUNCTION_NAME(SettingsTextArchive&, const std::nullptr_t&)
{}
//======================================================================================//
//
// Common serialization functions
//
//======================================================================================//
//! Serializing NVP types
template <typename T>
inline void
TIMEMORY_CEREAL_SAVE_FUNCTION_NAME(SettingsTextArchive& ar, const NameValuePair<T>& t)
{
ar.setNextName(t.name);
if(std::is_same<T, std::string>::value)
{
ar.setNextType("string");
}
else
{
ar.setNextType(tim::demangle<T>().c_str());
}
ar(t.value);
}
template <typename CharT, typename Traits, typename Alloc>
inline void
TIMEMORY_CEREAL_SAVE_FUNCTION_NAME(
SettingsTextArchive& ar,
const NameValuePair<std::basic_string<CharT, Traits, Alloc>>& t)
{
ar.setNextName(t.name);
ar.setNextType("string");
ar(t.value);
}
//! Saving for nullptr
inline void
TIMEMORY_CEREAL_SAVE_FUNCTION_NAME(SettingsTextArchive&, const std::nullptr_t&)
{}
//! Saving for arithmetic
template <typename T, traits::EnableIf<std::is_arithmetic<T>::value> = traits::sfinae>
inline void
TIMEMORY_CEREAL_SAVE_FUNCTION_NAME(SettingsTextArchive& ar, const T& t)
{
if(std::is_same<T, std::string>::value) ar.setNextType("string");
ar.saveValue(t);
}
//! saving string
template <typename CharT, typename Traits, typename Alloc>
inline void
TIMEMORY_CEREAL_SAVE_FUNCTION_NAME(SettingsTextArchive& ar,
const std::basic_string<CharT, Traits, Alloc>& str)
{
ar.setNextType("string");
ar.saveValue(str);
}
//--------------------------------------------------------------------------------------//
//! Saving SizeTags
template <typename T>
inline void
TIMEMORY_CEREAL_SAVE_FUNCTION_NAME(SettingsTextArchive&, const SizeTag<T>&)
{
// nothing to do here, we don't explicitly save the size
}
} // namespace cereal
} // namespace tim
// register archives for polymorphic support
TIMEMORY_CEREAL_REGISTER_ARCHIVE(SettingsTextArchive)