Граф коммитов

2 Коммитов

Автор SHA1 Сообщение Дата
systems-assistant[bot] 3f001b0305 [rocpd] Refactor to use python to convert rocpd to CSV + add CSV tests + remove old cpp implementation (#159)
* Write agent info to CSV

* Write kernel to CSV

* Write memory copy to CSV

* Write memory allocation to CSV

* Write hip api to CSV

* Write hsa api to CSV

* Write marker api to CSV

* Write counters to CSV

* Write scratch memory to CSV

* Write rccl api to CSV

* Write rocdecode api to CSV

* Write rocjpeg api to CSV

* Remove info_process joins

* Format agent id

* Compose full file name is sql writer function

* Add missing fields to kernel traces csv

* Rename vgpr_count to arch_vgpr_count

* Fix kernel name

* Skip empty query results

* Format csv.py

* Delete c++ CSV writer

* Add CSV header comparison test

* Fix comment spacing in csv.py

* Change ALLOC to ALLOCATE in memory allocation writer

* Do not append trace to agent info file name

* Revert changes for VGPR_Count

* Fix csv validation test

* Add sorting by guid

* Use EXISTS to check query results are not empty

* Merge API-specific queries

* Optimize regions query

* Column name mapping for agent info

* Pass config to sql writer

* Move agent id string building to a separate function

* add titled_headers argument

* Remove titled-columns argument

* Improvements for regions csv

* fix CSV validation test

* improve CSV validation test

* remove roctxMarkA from csv validation test

* fix capability field titles in agent info

* remove filter.py from query as that is still experimental

* Remove some aliases, now that query will auto-title the column headers

---------

Co-authored-by: Aleksei Tumakaev <atumakae@amd.com>
Co-authored-by: Young Hui <young.hui@amd.com>
Co-authored-by: Young Hui - AMD <145490163+yhuiYH@users.noreply.github.com>
2025-09-19 10:15:57 -04:00
Hui, Young 68ae6cf65f [rocpd] Adding summary module to generate summaries from rocpd database + query submodule + rocpd command-line tools (#488)
* adding summary.py to generate tmp <category_region>_summary views

* migrating CSV summary to SDK method of writing CSVs

  - Add domain_view to summary.py
  - omit the C++ code of writing CSV because it gets revered later anyway

* Add summary subparser and write_sql_view_to_csv function

* adding all <>_summary views generation to summary.py

* add summary_per_rank feature

* add --summary-per-rank

* reconstruct generate_summary_view and create_domain_view

-introduce by_rank

* remove sqr and variance in summary views

* use RocpdImportData instead of connection

* two fixes on summary.py

--modify the generate_summary_view function to return a tuple with view name and sql code

add if_not_exits parameter to generete_summary_view

* Refactor summary.py to allow output path and filename args, and apply time_window
- clean up summary table column headers
- only generate by-rank views if that param is specified

* Add ProcessID to Hostname output and csv, so users can identify the system in the by-rank summaries

* Summary.py, just add hostname to by-rank summaries, instead of creating mapping table

* Summary - migrate csv writer to pandas, for more future flexibility

* Adding a few simple tests for summary.py

* Linting fixes

* add region_categories to summary options

  -  Automatically retrieve region categories from the database if argument is None

* add backticks for view_names

* fix tests after rebase

* Made code review changes
- fixed whitespace in CMakelists.txt
- adding query.py module & subparser in __main__.py
- refactor summary function to return query
- used query.py to output csv
- used query.py to also output summary to console
- provided new command line options to select summary output to csv or console

* Made fix to jinja template in query.py, as suggested by copilot

* Consolidated output calls to query in export_view function based on feedback
- refactored: helpers, query functions, create view functions
- extended formats to include what query supports (md, html, pdf, json)
- added json format to query, and changed orient=records
- adding jinja2 and reportlab to requirements.txt

* Add version_info for rocpd and roctx

* Add rocpd commandline tool

* Add executable permissions to source/bin/rocpd.py

* Removed rocpd2query, and cleaned up --help examples

---------

Co-authored-by: acanadas <acanadas@amd.com>
Co-authored-by: Jin Tao <jintao12@amd.com>
Co-authored-by: a-canadasruiz <Araceli.CanadasRuiz@amd.com>
Co-authored-by: Jonathan R. Madsen <Jonathan.Madsen@amd.com>

[ROCm/rocprofiler-sdk commit: 3954cedd25]
2025-07-24 16:12:06 -05:00