Gbaraldi/att tool (#766)

* Enabling codeobj and thread trace samples * Updating aqlprofile_v2 header * Codeobj and thread trace samples with output log files * Fixing clang format * Cmake formatting * Adding coverage to codeobj * Comment trace sample * Adding ATT Parser API * Fixing forwarding to aqlprofile * Clang formatting * Clang tidy * Adding option to print memory kernels * Clang format * Remove default from switch case * Separating client/main on codeobj sample for ASAn * Formatting * Gbaraldi/att tool rebase (#801) * Enabling codeobj and thread trace samples * Updating aqlprofile_v2 header * Codeobj and thread trace samples with output log files * Fixing clang format * Cmake formatting * Adding coverage to codeobj * Comment trace sample * Removing python from workflow * Adding ATT Parser API * Fixing forwarding to aqlprofile * Clang formatting * Clang tidy * Adding option to print memory kernels * Clang format * Remove default from switch case * Separating client/main on codeobj sample for ASAn * Formatting * Enabling codeobj and thread trace samples * Updating aqlprofile_v2 header * Codeobj and thread trace samples with output log files * Fixing clang format * Cmake formatting * Adding coverage to codeobj * Comment trace sample * Adding ATT Parser API * Fixing forwarding to aqlprofile * Clang formatting * Clang tidy * Adding option to print memory kernels * Clang format * Remove default from switch case * Separating client/main on codeobj sample for ASAn * Formatting * Fix codeobj library * Allow thread trace in parallel with other service * Zeroing the HSA signals * Adding exception wrappers in ATT sample * Removed force configure * Remove force configure from ISA decode * Removing codecov flag * Gbaraldi/att tool tests (#828) * Adding tests for codeobj ISA decode * Adding ATT tests * Adding ATT integration tests * Formatting * Changing codeobj binary extension * Renaming codeobj library spaces * Fixing samples * Formatting * Formatting * Fixing int test * Fixing linker error * Fixing memory fault * Moving kernel ot inside namespace * ASAN linking fix * Removing unecessary headers * Formatting * Fixing target_cu * Remove codeobj binary * Revert "Remove codeobj binary" This reverts commit 7d286f89d8096bc36925cd79cd742a5e6d10d179. * Enable memory snapshot * adding comgr --------- Co-authored-by: Ammar ELWazir <ammar.elwazir@amd.com>
2024-05-03 18:45:47 -03:00
@@ -0,0 +1,24 @@
+# ##############################################################################
+# # Copyright (c) 2024 Advanced Micro Devices, Inc. # # Permission is hereby
+# granted, free of charge, to any person obtaining a copy # of this software and
+# associated documentation files (the "Software"), to # deal in the Software
+# without restriction, including without limitation the # rights to use, copy,
+# modify, merge, publish, distribute, sublicense, and/or # sell copies of the
+# Software, and to permit persons to whom the Software is # furnished to do so,
+# subject to the following conditions: # # The above copyright notice and this
+# permission notice shall be included in # all copies or substantial portions of
+# the Software. # # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY
+# KIND, EXPRESS OR # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+# MERCHANTABILITY, # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO
+# EVENT SHALL THE # AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM,
+# DAMAGES OR OTHER # LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR
+# OTHERWISE, ARISING # FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE
+# USE OR OTHER DEALINGS # IN THE SOFTWARE.
+# ##############################################################################
+
+set(CODEOBJ_PARSER_HEADERS code_printing.hpp disassembly.hpp segment.hpp)
+
+install(
+    FILES ${CODEOBJ_PARSER_HEADERS}
+    DESTINATION ${CMAKE_INSTALL_INCLUDEDIR}/rocprofiler-sdk-codeobj
+    COMPONENT development)
@@ -0,0 +1,272 @@
+// MIT License
+//
+// Copyright (c) 2024 Advanced Micro Devices, Inc. All rights reserved.
+//
+// Permission is hereby granted, free of charge, to any person obtaining a copy
+// of this software and associated documentation files (the "Software"), to deal
+// in the Software without restriction, including without limitation the rights
+// to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+// copies of the Software, and to permit persons to whom the Software is
+// furnished to do so, subject to the following conditions:
+//
+// The above copyright notice and this permission notice shall be included in all
+// copies or substantial portions of the Software.
+//
+// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+// SOFTWARE.
+
+#pragma once
+
+#include <map>
+#include <memory>
+#include <optional>
+#include <string>
+#include <unordered_map>
+#include <vector>
+#include "disassembly.hpp"
+#include "segment.hpp"
+
+namespace rocprofiler
+{
+namespace codeobj
+{
+namespace disassembly
+{
+struct Instruction
+{
+    Instruction() = default;
+    Instruction(std::string&& _inst, size_t _size)
+    : inst(std::move(_inst))
+    , size(_size)
+    {}
+    std::string inst;
+    std::string comment;
+    uint64_t    faddr;
+    uint64_t    vaddr;
+    uint64_t    ld_addr;
+    size_t      size;
+};
+
+struct DSourceLine
+{
+    uint64_t    vaddr;
+    uint64_t    size;
+    std::string str;
+    uint64_t    begin() const { return vaddr; }
+    bool        inrange(uint64_t addr) const { return addr >= vaddr && addr < vaddr + size; }
+};
+
+class CodeobjDecoderComponent
+{
+public:
+    CodeobjDecoderComponent(const void* codeobj_data, uint64_t codeobj_size);
+    ~CodeobjDecoderComponent();
+
+    std::optional<uint64_t> va2fo(uint64_t vaddr)
+    {
+        if(disassembly) return disassembly->va2fo(vaddr);
+        return {};
+    };
+
+    std::shared_ptr<Instruction> disassemble_instruction(uint64_t faddr, uint64_t vaddr);
+    int                          m_fd;
+
+    cached_ordered_vector<DSourceLine> m_line_number_map;
+    std::map<uint64_t, SymbolInfo>     m_symbol_map{};
+
+    std::string                               m_uri;
+    std::vector<std::shared_ptr<Instruction>> instructions{};
+    std::unique_ptr<DisassemblyInstance>      disassembly{};
+};
+
+class LoadedCodeobjDecoder
+{
+public:
+    LoadedCodeobjDecoder(const char* filepath, uint64_t load_addr, uint64_t memsize);
+    LoadedCodeobjDecoder(const void* data, uint64_t size, uint64_t load_addr, size_t memsize);
+    std::shared_ptr<Instruction> add_to_map(uint64_t ld_addr);
+
+    std::shared_ptr<Instruction> get(uint64_t addr);
+    uint64_t                     begin() const { return load_addr; };
+    uint64_t                     end() const { return load_end; }
+    uint64_t                     size() const { return load_end - load_addr; }
+    bool inrange(uint64_t addr) const { return addr >= begin() && addr < end(); }
+
+    const char* getSymbolName(uint64_t addr) const
+    {
+        if(!decoder) return nullptr;
+
+        auto it = decoder->m_symbol_map.find(addr - load_addr);
+        if(it != decoder->m_symbol_map.end()) return it->second.name.data();
+
+        return nullptr;
+    }
+
+    std::map<uint64_t, SymbolInfo>& getSymbolMap() const
+    {
+        if(!decoder) throw std::exception();
+        return decoder->m_symbol_map;
+    }
+    std::vector<std::pair<uint64_t, uint64_t>> elf_segments{};
+    const uint64_t                             load_addr;
+
+private:
+    uint64_t load_end = 0;
+
+    std::unordered_map<uint64_t, std::shared_ptr<Instruction>> decoded_map;
+    std::unique_ptr<CodeobjDecoderComponent>                   decoder{nullptr};
+};
+
+/**
+ * @brief Maps ID and offsets into instructions
+ */
+class CodeobjMap
+{
+public:
+    CodeobjMap() = default;
+
+    virtual void addDecoder(const char*         filepath,
+                            codeobj_marker_id_t id,
+                            uint64_t            load_addr,
+                            uint64_t            memsize)
+    {
+        decoders[id] = std::make_shared<LoadedCodeobjDecoder>(filepath, load_addr, memsize);
+    }
+
+    virtual void addDecoder(const void*         data,
+                            size_t              memory_size,
+                            codeobj_marker_id_t id,
+                            uint64_t            load_addr,
+                            uint64_t            memsize)
+    {
+        decoders[id] =
+            std::make_shared<LoadedCodeobjDecoder>(data, memory_size, load_addr, memsize);
+    }
+
+    virtual bool removeDecoderbyId(codeobj_marker_id_t id) { return decoders.erase(id) != 0; }
+
+    std::shared_ptr<Instruction> get(codeobj_marker_id_t id, uint64_t offset)
+    {
+        auto& decoder = decoders.at(id);
+        return decoder->get(decoder->begin() + offset);
+    }
+
+    const char* getSymbolName(codeobj_marker_id_t id, uint64_t offset)
+    {
+        auto&    decoder = decoders.at(id);
+        uint64_t vaddr   = decoder->begin() + offset;
+        if(decoder->inrange(vaddr)) return decoder->getSymbolName(vaddr);
+        return nullptr;
+    }
+
+protected:
+    std::unordered_map<codeobj_marker_id_t, std::shared_ptr<LoadedCodeobjDecoder>> decoders{};
+};
+
+/**
+ * @brief Translates virtual addresses to elf file offsets
+ */
+class CodeobjAddressTranslate : public CodeobjMap
+{
+    using Super = CodeobjMap;
+
+public:
+    CodeobjAddressTranslate() = default;
+
+    virtual void addDecoder(const char*         filepath,
+                            codeobj_marker_id_t id,
+                            uint64_t            load_addr,
+                            uint64_t            memsize) override
+    {
+        this->Super::addDecoder(filepath, id, load_addr, memsize);
+        auto ptr = decoders.at(id);
+        table.insert({ptr->begin(), ptr->size(), id, 0});
+    }
+
+    virtual void addDecoder(const void*         data,
+                            size_t              memory_size,
+                            codeobj_marker_id_t id,
+                            uint64_t            load_addr,
+                            uint64_t            memsize) override
+    {
+        this->Super::addDecoder(data, memory_size, id, load_addr, memsize);
+        auto ptr = decoders.at(id);
+        table.insert({ptr->begin(), ptr->size(), id, 0});
+    }
+
+    virtual bool removeDecoder(codeobj_marker_id_t id, uint64_t load_addr)
+    {
+        return table.remove(load_addr) && this->Super::removeDecoderbyId(id);
+    }
+
+    std::shared_ptr<Instruction> get(uint64_t vaddr)
+    {
+        auto& addr_range = table.find_codeobj_in_range(vaddr);
+        return this->Super::get(addr_range.id, vaddr - addr_range.vbegin);
+    }
+
+    std::shared_ptr<Instruction> get(codeobj_marker_id_t id, uint64_t offset)
+    {
+        if(id == 0)
+            return get(offset);
+        else
+            return this->Super::get(id, offset);
+    }
+
+    const char* getSymbolName(uint64_t vaddr)
+    {
+        for(auto& [_, decoder] : decoders)
+        {
+            if(!decoder->inrange(vaddr)) continue;
+            return decoder->getSymbolName(vaddr);
+        }
+        return nullptr;
+    }
+
+    void getSymbolMap(const std::shared_ptr<LoadedCodeobjDecoder>& dec,
+                      std::unordered_map<uint64_t, SymbolInfo>&    symbols) const
+    {
+        try
+        {
+            auto& smap = dec->getSymbolMap();
+            for(auto& [vaddr, sym] : smap)
+                symbols[vaddr + dec->load_addr] = sym;
+        } catch(std::exception& e)
+        {
+            return;
+        };
+    }
+
+    std::unordered_map<uint64_t, SymbolInfo> getSymbolMap() const
+    {
+        std::unordered_map<uint64_t, SymbolInfo> symbols;
+
+        for(auto& [_, dec] : decoders)
+            this->getSymbolMap(dec, symbols);
+
+        return symbols;
+    }
+
+    std::unordered_map<uint64_t, SymbolInfo> getSymbolMap(codeobj_marker_id_t id) const
+    {
+        std::unordered_map<uint64_t, SymbolInfo> symbols;
+
+        auto it = decoders.find(id);
+        if(it == decoders.end()) return symbols;
+
+        this->getSymbolMap(it->second, symbols);
+        return symbols;
+    }
+
+private:
+    CodeobjTableTranslator table;
+};
+
+}  // namespace disassembly
+}  // namespace codeobj
+}  // namespace rocprofiler
@@ -0,0 +1,78 @@
+// MIT License
+//
+// Copyright (c) 2024 Advanced Micro Devices, Inc. All rights reserved.
+//
+// Permission is hereby granted, free of charge, to any person obtaining a copy
+// of this software and associated documentation files (the "Software"), to deal
+// in the Software without restriction, including without limitation the rights
+// to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+// copies of the Software, and to permit persons to whom the Software is
+// furnished to do so, subject to the following conditions:
+//
+// The above copyright notice and this permission notice shall be included in all
+// copies or substantial portions of the Software.
+//
+// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+// SOFTWARE.
+
+#pragma once
+
+#include <amd_comgr/amd_comgr.h>
+#include <limits>
+#include <memory>
+#include <string>
+#include <vector>
+
+namespace rocprofiler
+{
+namespace codeobj
+{
+namespace disassembly
+{
+class CodeObjectBinary
+{
+public:
+    CodeObjectBinary(const std::string& uri);
+    std::string       m_uri;
+    std::vector<char> buffer;
+};
+
+struct SymbolInfo
+{
+    std::string name{};
+    uint64_t    faddr    = 0;
+    uint64_t    vaddr    = 0;
+    uint64_t    mem_size = 0;
+};
+
+class DisassemblyInstance
+{
+public:
+    DisassemblyInstance(const char* codeobj_data, uint64_t codeobj_size);
+    ~DisassemblyInstance();
+
+    std::pair<std::string, size_t>  ReadInstruction(uint64_t faddr);
+    std::map<uint64_t, SymbolInfo>& GetKernelMap();
+
+    static uint64_t memory_callback(uint64_t from, char* to, uint64_t size, void* user_data);
+    static void     inst_callback(const char* instruction, void* user_data);
+    static amd_comgr_status_t symbol_callback(amd_comgr_symbol_t symbol, void* user_data);
+
+    std::optional<uint64_t>                    va2fo(uint64_t va);
+    std::vector<std::pair<uint64_t, uint64_t>> getSegments();
+
+    std::vector<char>              buffer;
+    std::string                    last_instruction;
+    amd_comgr_disassembly_info_t   info;
+    amd_comgr_data_t               data;
+    std::map<uint64_t, SymbolInfo> symbol_map;
+};
+
+}  // namespace disassembly
+}  // namespace codeobj
+}  // namespace rocprofiler
@@ -0,0 +1,162 @@
+// MIT License
+//
+// Copyright (c) 2024 Advanced Micro Devices, Inc. All rights reserved.
+//
+// Permission is hereby granted, free of charge, to any person obtaining a copy
+// of this software and associated documentation files (the "Software"), to deal
+// in the Software without restriction, including without limitation the rights
+// to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+// copies of the Software, and to permit persons to whom the Software is
+// furnished to do so, subject to the following conditions:
+//
+// The above copyright notice and this permission notice shall be included in all
+// copies or substantial portions of the Software.
+//
+// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+// SOFTWARE.
+
+#pragma once
+#include <algorithm>
+#include <iostream>
+#include <random>
+#include <string>
+#include <unordered_set>
+#include <vector>
+
+using codeobj_marker_id_t = size_t;
+
+template <typename Type>
+class ordered_vector : public std::vector<Type>
+{
+    using Super = std::vector<Type>;
+
+public:
+    void insert(const Type& elem)
+    {
+        size_t loc = lower_bound(elem.begin());
+        if(this->size() && get(loc).begin() < elem.begin()) loc++;
+        this->Super::insert(this->begin() + loc, elem);
+    }
+    bool remove(const Type& elem)
+    {
+        if(!this->size()) return false;
+        size_t loc = lower_bound(elem.begin());
+        if(get(loc) != elem) return false;
+
+        this->Super::erase(this->begin() + loc);
+        return true;
+    }
+    bool remove(uint64_t elem_begin)
+    {
+        if(!this->size()) return false;
+        size_t loc = lower_bound(elem_begin);
+        if(get(loc).begin() != elem_begin) return false;
+
+        this->Super::erase(this->begin() + loc);
+        return true;
+    }
+    size_t lower_bound(size_t addr) const
+    {
+        if(!this->size()) return 0;
+        return binary_search(addr, 0, this->size() - 1);
+    }
+
+    size_t binary_search(size_t addr, size_t s, size_t e) const
+    {
+        if(s >= e)
+            return s;
+        else if(s + 1 == e)
+            return (get(e).begin() <= addr) ? e : s;
+
+        size_t mid = (s + e) / 2;
+        if(get(mid).begin() <= addr)
+            return binary_search(addr, mid, e);
+        else
+            return binary_search(addr, s, mid);
+    }
+    const Type& get(size_t i) const { return this->operator[](i); }
+};
+
+/**
+ * @brief Finds a candidate codeobj for the given vaddr
+ */
+template <typename Type>
+class cached_ordered_vector : public ordered_vector<Type>
+{
+    using Super = ordered_vector<Type>;
+
+public:
+    cached_ordered_vector<Type>() { reset(); }
+
+    const Type& find_obj(uint64_t addr)
+    {
+        if(testCache(addr)) return get(cached_segment);
+
+        size_t lb = this->lower_bound(addr);
+        if(lb >= this->size() || !get(lb).inrange(addr))
+            throw std::string("segment addr out of range");
+
+        cached_segment = lb;
+        return get(cached_segment);
+    }
+
+    uint64_t find_addr(uint64_t addr) { return find_obj(addr).begin(); }
+
+    bool testCache(uint64_t addr) const
+    {
+        return this->cached_segment < this->size() && get(cached_segment).inrange(addr);
+    }
+
+    const Type& get(size_t index) const { return this->data()[index]; }
+
+    void insert(const Type& elem) { this->Super::insert(elem); }
+    void insert_list(std::vector<Type> arange)
+    {
+        for(auto& elem : arange)
+            push_back(elem);
+        std::sort(this->begin(), this->end(), [](const Type& a, const Type& b) {
+            return a.begin() < b.begin();
+        });
+    };
+
+    void reset() { cached_segment = ~0; }
+    void clear()
+    {
+        reset();
+        this->Super::clear();
+    }
+    bool remove(uint64_t addr)
+    {
+        reset();
+        return this->Super::remove(addr);
+    }
+
+private:
+    size_t cached_segment = ~0;
+};
+
+struct address_range_t
+{
+    uint64_t            vbegin;
+    uint64_t            size;
+    codeobj_marker_id_t id;
+    uint64_t            offset;
+
+    bool     operator<(const address_range_t& other) const { return vbegin < other.vbegin; }
+    bool     inrange(uint64_t addr) const { return addr >= vbegin && addr < vbegin + size; };
+    uint64_t begin() const { return vbegin; }
+};
+
+/**
+ * @brief Finds a candidate codeobj for the given vaddr
+ */
+class CodeobjTableTranslator : public cached_ordered_vector<address_range_t>
+{
+public:
+    const address_range_t& find_codeobj_in_range(uint64_t addr) { return this->find_obj(addr); }
+};