From a009146494af218719d452abcb2421bb23cdf191 Mon Sep 17 00:00:00 2001 From: Benjamin Welton Date: Fri, 17 Nov 2023 01:49:51 -0800 Subject: [PATCH] Buffered Counter Collection API (#179) * Added buffer counter collection API. Initial testing added into counter-collection sample. Added support for constant metrics in counter collection (#194) * Added support for constant metrics in counter collection Adds support and test cases for constant metrics (such as max wave size) and adds the metric kernel duration (though this is still not yet calculated). * Minor doc updates * Simple counter unit tests (#199) * Simple counter unit tests Unit tests and some minor fixes for simple and derived counter evaluation * Added unit tests for reduction operations (#200) * Added unit tests for reduction operations * added tests for combo (constant+regular) counters (#201) source formatting (clang-format v11) (#202) Co-authored-by: bwelton source formatting (clang-format v11) (#203) Co-authored-by: bwelton Local changes source formatting (clang-format v11) (#205) Co-authored-by: bwelton Minor doc fix Remove kernel_duration, migrate over set_dimensions to after HSA init source formatting (clang-format v11) (#207) Co-authored-by: bwelton Added output to ROCPROFILER_SAMPLE_OUTPUT_FILE: * Remove integer based counter in return struct This casues a lot of complications and seems to provide limit benefit of just treating all counters as doubles. For ease of use, drop the integer based counter. * source formatting (clang-format v11) (#217) Co-authored-by: bwelton * Add correlation id support to counters (#218) Adds correlation id support to counter collection. Requires tracing to be enabled to return any useful value currently (since we do not have HIP kernel tracing yet). * source formatting (clang-format v11) (#223) Co-authored-by: bwelton * Add sample that attempts to fetch all counters On whatever machine this test is run on, all counters available on the platform will attempted to be fetched from a kernel execution. Each counter will be fetched one time to check that the counter can be fetched on the platform and that the counter is returning the correct instance count (however due to the lack of transparency from AQL profiler this check is not functional for some counters). We do not do any implicit reduction on any counter, the result is that we see more counters than the number of events being requested. Below is the status of all counters on MI210. All counters appear functional with the changes in this PR. However, the instance count retruned will be greater than that returned by rocprofiler_query_counter_instance_count. Got 516 counters collected Counter ID: 0 (size) expected 1 instances and got 1 Counter ID: 1 (processor_id_low) expected 1 instances and got 1 Counter ID: 2 (capability) expected 1 instances and got 1 Counter ID: 3 (local_mem_size) expected 1 instances and got 1 Counter ID: 4 (min_latency) expected 1 instances and got 1 Counter ID: 5 (weight) expected 1 instances and got 1 Counter ID: 6 (node_from) expected 1 instances and got 1 Counter ID: 7 (version_major) expected 1 instances and got 1 Counter ID: 8 (version_minor) expected 1 instances and got 1 Counter ID: 9 (mem_clk_max) expected 1 instances and got 1 Counter ID: 10 (num_xcc) expected 1 instances and got 1 Counter ID: 11 (width) expected 1 instances and got 1 Counter ID: 12 (flags) expected 1 instances and got 1 Counter ID: 13 (size_in_bytes) expected 1 instances and got 1 Counter ID: 14 (array_count) expected 1 instances and got 1 Counter ID: 15 (num_gws) expected 1 instances and got 1 Counter ID: 16 (simd_id_base) expected 1 instances and got 1 Counter ID: 17 (max_waves_per_simd) expected 1 instances and got 1 Counter ID: 18 (sdma_fw_version) expected 1 instances and got 1 Counter ID: 19 (gfx_target_version) expected 1 instances and got 1 Counter ID: 20 (max_bandwidth) expected 1 instances and got 1 Counter ID: 21 (cpu_core_id_base) expected 1 instances and got 1 Counter ID: 22 (cache_line_size) expected 1 instances and got 1 Counter ID: 23 (level) expected 1 instances and got 1 Counter ID: 24 (min_bandwidth) expected 1 instances and got 1 Counter ID: 25 (location_id) expected 1 instances and got 1 Counter ID: 26 (wave_front_size) expected 1 instances and got 1 Counter ID: 27 (lds_size_in_kb) expected 1 instances and got 1 Counter ID: 28 (simd_count) expected 1 instances and got 1 Counter ID: 29 (fw_version) expected 1 instances and got 1 Counter ID: 30 (recommended_transfer_size) expected 1 instances and got 1 Counter ID: 31 (simd_per_cu) expected 1 instances and got 1 Counter ID: 32 (association) expected 1 instances and got 1 Counter ID: 33 (mem_banks_count) expected 1 instances and got 1 Counter ID: 34 (latency) expected 1 instances and got 1 Counter ID: 35 (max_latency) expected 1 instances and got 1 Counter ID: 36 (cpu_cores_count) expected 1 instances and got 1 Counter ID: 37 (io_links_count) expected 1 instances and got 1 Counter ID: 38 (domain) expected 1 instances and got 1 Counter ID: 39 (max_engine_clk_fcompute) expected 1 instances and got 1 Counter ID: 40 (caches_count) expected 1 instances and got 1 Counter ID: 41 (simd_arrays_per_engine) expected 1 instances and got 1 Counter ID: 42 (cache_lines_per_tag) expected 1 instances and got 1 Counter ID: 43 (gds_size_in_kb) expected 1 instances and got 1 Counter ID: 44 (cu_per_simd_array) expected 1 instances and got 1 Counter ID: 45 (type) expected 1 instances and got 1 Counter ID: 46 (max_slots_scratch_cu) expected 1 instances and got 1 Counter ID: 47 (vendor_id) expected 1 instances and got 1 Counter ID: 48 (device_id) expected 1 instances and got 1 Counter ID: 49 (heap_type) expected 1 instances and got 1 Counter ID: 50 (drm_render_minor) expected 1 instances and got 1 Counter ID: 51 (num_sdma_engines) expected 1 instances and got 1 Counter ID: 52 (node_to) expected 1 instances and got 1 Counter ID: 53 (num_sdma_xgmi_engines) expected 1 instances and got 1 Counter ID: 54 (num_sdma_queues_per_engine) expected 1 instances and got 1 Counter ID: 55 (hive_id) expected 1 instances and got 1 Counter ID: 56 (num_cp_queues) expected 1 instances and got 1 Counter ID: 57 (max_engine_clk_ccompute) expected 1 instances and got 1 Counter ID: 517 (MAX_WAVE_SIZE) expected 1 instances and got 1 Counter ID: 518 (SE_NUM) expected 1 instances and got 1 Counter ID: 519 (SIMD_NUM) expected 1 instances and got 1 Counter ID: 520 (CU_NUM) expected 1 instances and got 1 [ERROR]Counter ID: 521 (SQ_WAIT_INST_LDS) expected 1 instances and got 8 [ERROR]Counter ID: 522 (TCP_TCP_TA_DATA_STALL_CYCLES) expected 16 instances and got 128 Counter ID: 523 (GRBM_COUNT) expected 1 instances and got 1 Counter ID: 524 (GRBM_GUI_ACTIVE) expected 1 instances and got 1 Counter ID: 525 (GRBM_CP_BUSY) expected 1 instances and got 1 Counter ID: 526 (GRBM_SPI_BUSY) expected 1 instances and got 1 Counter ID: 527 (GRBM_TA_BUSY) expected 1 instances and got 1 Counter ID: 528 (GRBM_TC_BUSY) expected 1 instances and got 1 Counter ID: 529 (GRBM_CPC_BUSY) expected 1 instances and got 1 Counter ID: 530 (GRBM_CPF_BUSY) expected 1 instances and got 1 Counter ID: 531 (GRBM_UTCL2_BUSY) expected 1 instances and got 1 Counter ID: 532 (GRBM_EA_BUSY) expected 1 instances and got 1 Counter ID: 533 (CPC_ME1_BUSY_FOR_PACKET_DECODE) expected 1 instances and got 1 Counter ID: 534 (CPC_UTCL1_STALL_ON_TRANSLATION) expected 1 instances and got 1 Counter ID: 535 (CPC_CPC_STAT_BUSY) expected 1 instances and got 1 Counter ID: 536 (CPC_CPC_STAT_IDLE) expected 1 instances and got 1 Counter ID: 537 (CPC_CPC_STAT_STALL) expected 1 instances and got 1 Counter ID: 538 (CPC_CPC_TCIU_BUSY) expected 1 instances and got 1 Counter ID: 539 (CPC_CPC_TCIU_IDLE) expected 1 instances and got 1 Counter ID: 540 (CPC_CPC_UTCL2IU_BUSY) expected 1 instances and got 1 Counter ID: 541 (CPC_CPC_UTCL2IU_IDLE) expected 1 instances and got 1 Counter ID: 542 (CPC_CPC_UTCL2IU_STALL) expected 1 instances and got 1 Counter ID: 543 (CPC_ME1_DC0_SPI_BUSY) expected 1 instances and got 1 Counter ID: 544 (CPF_CMP_UTCL1_STALL_ON_TRANSLATION) expected 1 instances and got 1 Counter ID: 545 (CPF_CPF_STAT_BUSY) expected 1 instances and got 1 Counter ID: 546 (CPF_CPF_STAT_IDLE) expected 1 instances and got 1 Counter ID: 547 (CPF_CPF_STAT_STALL) expected 1 instances and got 1 Counter ID: 548 (CPF_CPF_TCIU_BUSY) expected 1 instances and got 1 Counter ID: 549 (CPF_CPF_TCIU_IDLE) expected 1 instances and got 1 Counter ID: 550 (CPF_CPF_TCIU_STALL) expected 1 instances and got 1 [ERROR]Counter ID: 551 (SPI_CSN_WINDOW_VALID) expected 1 instances and got 8 [ERROR]Counter ID: 552 (SPI_CSN_BUSY) expected 1 instances and got 8 [ERROR]Counter ID: 553 (SPI_CSN_NUM_THREADGROUPS) expected 1 instances and got 8 [ERROR]Counter ID: 554 (SPI_CSN_WAVE) expected 1 instances and got 8 [ERROR]Counter ID: 555 (SPI_RA_REQ_NO_ALLOC) expected 1 instances and got 8 [ERROR]Counter ID: 556 (SPI_RA_REQ_NO_ALLOC_CSN) expected 1 instances and got 8 [ERROR]Counter ID: 557 (SPI_RA_RES_STALL_CSN) expected 1 instances and got 8 [ERROR]Counter ID: 558 (SPI_RA_TMP_STALL_CSN) expected 1 instances and got 8 [ERROR]Counter ID: 559 (SPI_RA_WAVE_SIMD_FULL_CSN) expected 1 instances and got 8 [ERROR]Counter ID: 560 (SPI_RA_VGPR_SIMD_FULL_CSN) expected 1 instances and got 8 [ERROR]Counter ID: 561 (SPI_RA_SGPR_SIMD_FULL_CSN) expected 1 instances and got 8 [ERROR]Counter ID: 562 (SPI_RA_LDS_CU_FULL_CSN) expected 1 instances and got 8 [ERROR]Counter ID: 563 (SPI_RA_BAR_CU_FULL_CSN) expected 1 instances and got 8 [ERROR]Counter ID: 564 (SPI_RA_BULKY_CU_FULL_CSN) expected 1 instances and got 8 [ERROR]Counter ID: 565 (SPI_RA_TGLIM_CU_FULL_CSN) expected 1 instances and got 8 [ERROR]Counter ID: 566 (SPI_RA_WVLIM_STALL_CSN) expected 1 instances and got 8 [ERROR]Counter ID: 567 (SPI_SWC_CSC_WR) expected 1 instances and got 8 [ERROR]Counter ID: 568 (SPI_VWC_CSC_WR) expected 1 instances and got 8 [ERROR]Counter ID: 569 (SQ_ACCUM_PREV) expected 1 instances and got 8 [ERROR]Counter ID: 570 (SQ_CYCLES) expected 1 instances and got 8 [ERROR]Counter ID: 571 (SQ_BUSY_CYCLES) expected 1 instances and got 8 [ERROR]Counter ID: 572 (SQ_WAVES) expected 1 instances and got 8 [ERROR]Counter ID: 573 (SQ_LEVEL_WAVES) expected 1 instances and got 8 [ERROR]Counter ID: 574 (SQ_WAVES_EQ_64) expected 1 instances and got 8 [ERROR]Counter ID: 575 (SQ_WAVES_LT_64) expected 1 instances and got 8 [ERROR]Counter ID: 576 (SQ_WAVES_LT_48) expected 1 instances and got 8 [ERROR]Counter ID: 577 (SQ_WAVES_LT_32) expected 1 instances and got 8 [ERROR]Counter ID: 578 (SQ_WAVES_LT_16) expected 1 instances and got 8 [ERROR]Counter ID: 579 (SQ_BUSY_CU_CYCLES) expected 1 instances and got 8 [ERROR]Counter ID: 580 (SQ_ITEMS) expected 1 instances and got 8 [ERROR]Counter ID: 581 (SQ_INSTS) expected 1 instances and got 8 [ERROR]Counter ID: 582 (SQ_INSTS_VALU) expected 1 instances and got 8 [ERROR]Counter ID: 583 (SQ_INSTS_VALU_ADD_F16) expected 1 instances and got 8 [ERROR]Counter ID: 584 (SQ_INSTS_VALU_MUL_F16) expected 1 instances and got 8 [ERROR]Counter ID: 585 (SQ_INSTS_VALU_FMA_F16) expected 1 instances and got 8 [ERROR]Counter ID: 586 (SQ_INSTS_VALU_TRANS_F16) expected 1 instances and got 8 [ERROR]Counter ID: 587 (SQ_INSTS_VALU_ADD_F32) expected 1 instances and got 8 [ERROR]Counter ID: 588 (SQ_INSTS_VALU_MUL_F32) expected 1 instances and got 8 [ERROR]Counter ID: 589 (SQ_INSTS_VALU_FMA_F32) expected 1 instances and got 8 [ERROR]Counter ID: 590 (SQ_INSTS_VALU_TRANS_F32) expected 1 instances and got 8 [ERROR]Counter ID: 591 (SQ_INSTS_VALU_ADD_F64) expected 1 instances and got 8 [ERROR]Counter ID: 592 (SQ_INSTS_VALU_MUL_F64) expected 1 instances and got 8 [ERROR]Counter ID: 593 (SQ_INSTS_VALU_FMA_F64) expected 1 instances and got 8 [ERROR]Counter ID: 594 (SQ_INSTS_VALU_TRANS_F64) expected 1 instances and got 8 [ERROR]Counter ID: 595 (SQ_INSTS_VALU_INT32) expected 1 instances and got 8 [ERROR]Counter ID: 596 (SQ_INSTS_VALU_INT64) expected 1 instances and got 8 [ERROR]Counter ID: 597 (SQ_INSTS_VALU_CVT) expected 1 instances and got 8 [ERROR]Counter ID: 598 (SQ_INSTS_VALU_MFMA_I8) expected 1 instances and got 8 [ERROR]Counter ID: 599 (SQ_INSTS_VALU_MFMA_F16) expected 1 instances and got 8 [ERROR]Counter ID: 600 (SQ_INSTS_VALU_MFMA_BF16) expected 1 instances and got 8 [ERROR]Counter ID: 601 (SQ_INSTS_VALU_MFMA_F32) expected 1 instances and got 8 [ERROR]Counter ID: 602 (SQ_INSTS_VALU_MFMA_F64) expected 1 instances and got 8 [ERROR]Counter ID: 603 (SQ_INSTS_VALU_MFMA_MOPS_I8) expected 1 instances and got 8 [ERROR]Counter ID: 604 (SQ_INSTS_VALU_MFMA_MOPS_F16) expected 1 instances and got 8 [ERROR]Counter ID: 605 (SQ_INSTS_VALU_MFMA_MOPS_BF16) expected 1 instances and got 8 [ERROR]Counter ID: 606 (SQ_INSTS_VALU_MFMA_MOPS_F32) expected 1 instances and got 8 [ERROR]Counter ID: 607 (SQ_INSTS_VALU_MFMA_MOPS_F64) expected 1 instances and got 8 [ERROR]Counter ID: 608 (SQ_INSTS_MFMA) expected 1 instances and got 8 [ERROR]Counter ID: 609 (SQ_INSTS_VMEM_WR) expected 1 instances and got 8 [ERROR]Counter ID: 610 (SQ_INSTS_VMEM_RD) expected 1 instances and got 8 [ERROR]Counter ID: 611 (SQ_INSTS_VMEM) expected 1 instances and got 8 [ERROR]Counter ID: 612 (SQ_INSTS_SALU) expected 1 instances and got 8 [ERROR]Counter ID: 613 (SQ_INSTS_SMEM) expected 1 instances and got 8 [ERROR]Counter ID: 614 (SQ_INSTS_FLAT) expected 1 instances and got 8 [ERROR]Counter ID: 615 (SQ_INSTS_FLAT_LDS_ONLY) expected 1 instances and got 8 [ERROR]Counter ID: 616 (SQ_INSTS_LDS) expected 1 instances and got 8 [ERROR]Counter ID: 617 (SQ_INSTS_GDS) expected 1 instances and got 8 [ERROR]Counter ID: 618 (SQ_INSTS_EXP_GDS) expected 1 instances and got 8 [ERROR]Counter ID: 619 (SQ_INSTS_BRANCH) expected 1 instances and got 8 [ERROR]Counter ID: 620 (SQ_INSTS_SENDMSG) expected 1 instances and got 8 [ERROR]Counter ID: 621 (SQ_INSTS_VSKIPPED) expected 1 instances and got 8 [ERROR]Counter ID: 622 (SQ_INST_LEVEL_VMEM) expected 1 instances and got 8 [ERROR]Counter ID: 623 (SQ_INST_LEVEL_SMEM) expected 1 instances and got 8 [ERROR]Counter ID: 624 (SQ_INST_LEVEL_LDS) expected 1 instances and got 8 [ERROR]Counter ID: 625 (SQ_VALU_MFMA_BUSY_CYCLES) expected 1 instances and got 8 [ERROR]Counter ID: 626 (SQ_WAVE_CYCLES) expected 1 instances and got 8 [ERROR]Counter ID: 627 (SQ_WAIT_ANY) expected 1 instances and got 8 [ERROR]Counter ID: 628 (SQ_WAIT_INST_ANY) expected 1 instances and got 8 [ERROR]Counter ID: 629 (SQ_ACTIVE_INST_ANY) expected 1 instances and got 8 [ERROR]Counter ID: 630 (SQ_ACTIVE_INST_VMEM) expected 1 instances and got 8 [ERROR]Counter ID: 631 (SQ_ACTIVE_INST_LDS) expected 1 instances and got 8 [ERROR]Counter ID: 632 (SQ_ACTIVE_INST_VALU) expected 1 instances and got 8 [ERROR]Counter ID: 633 (SQ_ACTIVE_INST_SCA) expected 1 instances and got 8 [ERROR]Counter ID: 634 (SQ_ACTIVE_INST_EXP_GDS) expected 1 instances and got 8 [ERROR]Counter ID: 635 (SQ_ACTIVE_INST_MISC) expected 1 instances and got 8 [ERROR]Counter ID: 636 (SQ_ACTIVE_INST_FLAT) expected 1 instances and got 8 [ERROR]Counter ID: 637 (SQ_INST_CYCLES_VMEM_WR) expected 1 instances and got 8 [ERROR]Counter ID: 638 (SQ_INST_CYCLES_VMEM_RD) expected 1 instances and got 8 [ERROR]Counter ID: 639 (SQ_INST_CYCLES_SMEM) expected 1 instances and got 8 [ERROR]Counter ID: 640 (SQ_INST_CYCLES_SALU) expected 1 instances and got 8 [ERROR]Counter ID: 641 (SQ_THREAD_CYCLES_VALU) expected 1 instances and got 8 [ERROR]Counter ID: 642 (SQ_IFETCH) expected 1 instances and got 8 [ERROR]Counter ID: 643 (SQ_IFETCH_LEVEL) expected 1 instances and got 8 [ERROR]Counter ID: 644 (SQ_LDS_BANK_CONFLICT) expected 1 instances and got 8 [ERROR]Counter ID: 645 (SQ_LDS_ADDR_CONFLICT) expected 1 instances and got 8 [ERROR]Counter ID: 646 (SQ_LDS_UNALIGNED_STALL) expected 1 instances and got 8 [ERROR]Counter ID: 647 (SQ_LDS_MEM_VIOLATIONS) expected 1 instances and got 8 [ERROR]Counter ID: 648 (SQ_LDS_ATOMIC_RETURN) expected 1 instances and got 8 [ERROR]Counter ID: 649 (SQ_LDS_IDX_ACTIVE) expected 1 instances and got 8 [ERROR]Counter ID: 650 (SQ_ACCUM_PREV_HIRES) expected 1 instances and got 8 [ERROR]Counter ID: 651 (SQ_WAVES_RESTORED) expected 1 instances and got 8 [ERROR]Counter ID: 652 (SQ_WAVES_SAVED) expected 1 instances and got 8 [ERROR]Counter ID: 653 (SQ_INSTS_SMEM_NORM) expected 1 instances and got 8 [ERROR]Counter ID: 654 (SQC_DCACHE_INPUT_VALID_READYB) expected 1 instances and got 8 [ERROR]Counter ID: 655 (SQC_TC_REQ) expected 1 instances and got 8 [ERROR]Counter ID: 656 (SQC_TC_INST_REQ) expected 1 instances and got 8 [ERROR]Counter ID: 657 (SQC_TC_DATA_READ_REQ) expected 1 instances and got 8 [ERROR]Counter ID: 658 (SQC_TC_DATA_WRITE_REQ) expected 1 instances and got 8 [ERROR]Counter ID: 659 (SQC_TC_DATA_ATOMIC_REQ) expected 1 instances and got 8 [ERROR]Counter ID: 660 (SQC_TC_STALL) expected 1 instances and got 8 [ERROR]Counter ID: 661 (SQC_ICACHE_REQ) expected 1 instances and got 8 [ERROR]Counter ID: 662 (SQC_ICACHE_HITS) expected 1 instances and got 8 [ERROR]Counter ID: 663 (SQC_ICACHE_MISSES) expected 1 instances and got 8 [ERROR]Counter ID: 664 (SQC_ICACHE_MISSES_DUPLICATE) expected 1 instances and got 8 [ERROR]Counter ID: 665 (SQC_DCACHE_REQ) expected 1 instances and got 8 [ERROR]Counter ID: 666 (SQC_DCACHE_HITS) expected 1 instances and got 8 [ERROR]Counter ID: 667 (SQC_DCACHE_MISSES) expected 1 instances and got 8 [ERROR]Counter ID: 668 (SQC_DCACHE_MISSES_DUPLICATE) expected 1 instances and got 8 [ERROR]Counter ID: 669 (SQC_DCACHE_ATOMIC) expected 1 instances and got 8 [ERROR]Counter ID: 670 (SQC_DCACHE_REQ_READ_1) expected 1 instances and got 8 [ERROR]Counter ID: 671 (SQC_DCACHE_REQ_READ_2) expected 1 instances and got 8 [ERROR]Counter ID: 672 (SQC_DCACHE_REQ_READ_4) expected 1 instances and got 8 [ERROR]Counter ID: 673 (SQC_DCACHE_REQ_READ_8) expected 1 instances and got 8 [ERROR]Counter ID: 674 (SQC_DCACHE_REQ_READ_16) expected 1 instances and got 8 [ERROR]Counter ID: 675 (TA_TA_BUSY) expected 16 instances and got 128 [ERROR]Counter ID: 676 (TA_TOTAL_WAVEFRONTS) expected 16 instances and got 128 [ERROR]Counter ID: 677 (TA_BUFFER_WAVEFRONTS) expected 16 instances and got 128 [ERROR]Counter ID: 678 (TA_BUFFER_READ_WAVEFRONTS) expected 16 instances and got 128 [ERROR]Counter ID: 679 (TA_BUFFER_WRITE_WAVEFRONTS) expected 16 instances and got 128 [ERROR]Counter ID: 680 (TA_BUFFER_ATOMIC_WAVEFRONTS) expected 16 instances and got 128 [ERROR]Counter ID: 681 (TA_BUFFER_TOTAL_CYCLES) expected 16 instances and got 128 [ERROR]Counter ID: 682 (TA_BUFFER_COALESCED_READ_CYCLES) expected 16 instances and got 128 [ERROR]Counter ID: 683 (TA_BUFFER_COALESCED_WRITE_CYCLES) expected 16 instances and got 128 [ERROR]Counter ID: 684 (TA_ADDR_STALLED_BY_TC_CYCLES) expected 16 instances and got 128 [ERROR]Counter ID: 685 (TA_ADDR_STALLED_BY_TD_CYCLES) expected 16 instances and got 128 [ERROR]Counter ID: 686 (TA_DATA_STALLED_BY_TC_CYCLES) expected 16 instances and got 128 [ERROR]Counter ID: 687 (TA_FLAT_WAVEFRONTS) expected 16 instances and got 128 [ERROR]Counter ID: 688 (TA_FLAT_READ_WAVEFRONTS) expected 16 instances and got 128 [ERROR]Counter ID: 689 (TA_FLAT_WRITE_WAVEFRONTS) expected 16 instances and got 128 [ERROR]Counter ID: 690 (TA_FLAT_ATOMIC_WAVEFRONTS) expected 16 instances and got 128 [ERROR]Counter ID: 691 (TD_TD_BUSY) expected 16 instances and got 128 [ERROR]Counter ID: 692 (TD_TC_STALL) expected 16 instances and got 128 [ERROR]Counter ID: 693 (TD_SPI_STALL) expected 16 instances and got 128 [ERROR]Counter ID: 694 (TD_LOAD_WAVEFRONT) expected 16 instances and got 128 [ERROR]Counter ID: 695 (TD_ATOMIC_WAVEFRONT) expected 16 instances and got 128 [ERROR]Counter ID: 696 (TD_STORE_WAVEFRONT) expected 16 instances and got 128 [ERROR]Counter ID: 697 (TD_COALESCABLE_WAVEFRONT) expected 16 instances and got 128 [ERROR]Counter ID: 698 (TCP_GATE_EN1) expected 16 instances and got 128 [ERROR]Counter ID: 699 (TCP_GATE_EN2) expected 16 instances and got 128 [ERROR]Counter ID: 700 (TCP_TD_TCP_STALL_CYCLES) expected 16 instances and got 128 [ERROR]Counter ID: 701 (TCP_TCR_TCP_STALL_CYCLES) expected 16 instances and got 128 [ERROR]Counter ID: 702 (TCP_READ_TAGCONFLICT_STALL_CYCLES) expected 16 instances and got 128 [ERROR]Counter ID: 703 (TCP_WRITE_TAGCONFLICT_STALL_CYCLES) expected 16 instances and got 128 [ERROR]Counter ID: 704 (TCP_ATOMIC_TAGCONFLICT_STALL_CYCLES) expected 16 instances and got 128 [ERROR]Counter ID: 705 (TCP_PENDING_STALL_CYCLES) expected 16 instances and got 128 [ERROR]Counter ID: 706 (TCP_TA_TCP_STATE_READ) expected 16 instances and got 128 [ERROR]Counter ID: 707 (TCP_VOLATILE) expected 16 instances and got 128 [ERROR]Counter ID: 708 (TCP_TOTAL_ACCESSES) expected 16 instances and got 128 [ERROR]Counter ID: 709 (TCP_TOTAL_READ) expected 16 instances and got 128 [ERROR]Counter ID: 710 (TCP_TOTAL_WRITE) expected 16 instances and got 128 [ERROR]Counter ID: 711 (TCP_TOTAL_ATOMIC_WITH_RET) expected 16 instances and got 128 [ERROR]Counter ID: 712 (TCP_TOTAL_ATOMIC_WITHOUT_RET) expected 16 instances and got 128 [ERROR]Counter ID: 713 (TCP_TOTAL_WRITEBACK_INVALIDATES) expected 16 instances and got 128 [ERROR]Counter ID: 714 (TCP_UTCL1_REQUEST) expected 16 instances and got 128 [ERROR]Counter ID: 715 (TCP_UTCL1_TRANSLATION_MISS) expected 16 instances and got 128 [ERROR]Counter ID: 716 (TCP_UTCL1_TRANSLATION_HIT) expected 16 instances and got 128 [ERROR]Counter ID: 717 (TCP_UTCL1_PERMISSION_MISS) expected 16 instances and got 128 [ERROR]Counter ID: 718 (TCP_TOTAL_CACHE_ACCESSES) expected 16 instances and got 128 [ERROR]Counter ID: 719 (TCP_TCP_LATENCY) expected 16 instances and got 128 [ERROR]Counter ID: 720 (TCP_TCC_READ_REQ_LATENCY) expected 16 instances and got 128 [ERROR]Counter ID: 721 (TCP_TCC_WRITE_REQ_LATENCY) expected 16 instances and got 128 [ERROR]Counter ID: 722 (TCP_TCC_READ_REQ) expected 16 instances and got 128 [ERROR]Counter ID: 723 (TCP_TCC_WRITE_REQ) expected 16 instances and got 128 [ERROR]Counter ID: 724 (TCP_TCC_ATOMIC_WITH_RET_REQ) expected 16 instances and got 128 [ERROR]Counter ID: 725 (TCP_TCC_ATOMIC_WITHOUT_RET_REQ) expected 16 instances and got 128 [ERROR]Counter ID: 726 (TCP_TCC_NC_READ_REQ) expected 16 instances and got 128 [ERROR]Counter ID: 727 (TCP_TCC_NC_WRITE_REQ) expected 16 instances and got 128 [ERROR]Counter ID: 728 (TCP_TCC_NC_ATOMIC_REQ) expected 16 instances and got 128 [ERROR]Counter ID: 729 (TCP_TCC_UC_READ_REQ) expected 16 instances and got 128 [ERROR]Counter ID: 730 (TCP_TCC_UC_WRITE_REQ) expected 16 instances and got 128 [ERROR]Counter ID: 731 (TCP_TCC_UC_ATOMIC_REQ) expected 16 instances and got 128 [ERROR]Counter ID: 732 (TCP_TCC_CC_READ_REQ) expected 16 instances and got 128 [ERROR]Counter ID: 733 (TCP_TCC_CC_WRITE_REQ) expected 16 instances and got 128 [ERROR]Counter ID: 734 (TCP_TCC_CC_ATOMIC_REQ) expected 16 instances and got 128 [ERROR]Counter ID: 735 (TCP_TCC_RW_READ_REQ) expected 16 instances and got 128 [ERROR]Counter ID: 736 (TCP_TCC_RW_WRITE_REQ) expected 16 instances and got 128 [ERROR]Counter ID: 737 (TCP_TCC_RW_ATOMIC_REQ) expected 16 instances and got 128 Counter ID: 738 (TCA_CYCLE) expected 32 instances and got 32 Counter ID: 739 (TCA_BUSY) expected 32 instances and got 32 Counter ID: 740 (TCC_CYCLE) expected 32 instances and got 32 Counter ID: 741 (TCC_BUSY) expected 32 instances and got 32 Counter ID: 742 (TCC_REQ) expected 32 instances and got 32 Counter ID: 743 (TCC_STREAMING_REQ) expected 32 instances and got 32 Counter ID: 744 (TCC_NC_REQ) expected 32 instances and got 32 Counter ID: 745 (TCC_UC_REQ) expected 32 instances and got 32 Counter ID: 746 (TCC_CC_REQ) expected 32 instances and got 32 Counter ID: 747 (TCC_RW_REQ) expected 32 instances and got 32 Counter ID: 748 (TCC_PROBE) expected 32 instances and got 32 Counter ID: 749 (TCC_PROBE_ALL) expected 32 instances and got 32 Counter ID: 750 (TCC_READ) expected 32 instances and got 32 Counter ID: 751 (TCC_WRITE) expected 32 instances and got 32 Counter ID: 752 (TCC_ATOMIC) expected 32 instances and got 32 Counter ID: 753 (TCC_HIT) expected 32 instances and got 32 Counter ID: 754 (TCC_MISS) expected 32 instances and got 32 Counter ID: 755 (TCC_WRITEBACK) expected 32 instances and got 32 Counter ID: 756 (TCC_EA_WRREQ) expected 32 instances and got 32 Counter ID: 757 (TCC_EA_WRREQ_64B) expected 32 instances and got 32 Counter ID: 758 (TCC_EA_WR_UNCACHED_32B) expected 32 instances and got 32 Counter ID: 759 (TCC_EA_WRREQ_STALL) expected 32 instances and got 32 Counter ID: 760 (TCC_EA_WRREQ_IO_CREDIT_STALL) expected 32 instances and got 32 Counter ID: 761 (TCC_EA_WRREQ_GMI_CREDIT_STALL) expected 32 instances and got 32 Counter ID: 762 (TCC_EA_WRREQ_DRAM_CREDIT_STALL) expected 32 instances and got 32 Counter ID: 763 (TCC_TOO_MANY_EA_WRREQS_STALL) expected 32 instances and got 32 Counter ID: 764 (TCC_EA_WRREQ_LEVEL) expected 32 instances and got 32 Counter ID: 765 (TCC_EA_ATOMIC) expected 32 instances and got 32 Counter ID: 766 (TCC_EA_ATOMIC_LEVEL) expected 32 instances and got 32 Counter ID: 767 (TCC_EA_RDREQ) expected 32 instances and got 32 Counter ID: 768 (TCC_EA_RDREQ_32B) expected 32 instances and got 32 Counter ID: 769 (TCC_EA_RD_UNCACHED_32B) expected 32 instances and got 32 Counter ID: 770 (TCC_EA_RDREQ_IO_CREDIT_STALL) expected 32 instances and got 32 Counter ID: 771 (TCC_EA_RDREQ_GMI_CREDIT_STALL) expected 32 instances and got 32 Counter ID: 772 (TCC_EA_RDREQ_DRAM_CREDIT_STALL) expected 32 instances and got 32 Counter ID: 773 (TCC_EA_RDREQ_LEVEL) expected 32 instances and got 32 Counter ID: 774 (TCC_TAG_STALL) expected 32 instances and got 32 Counter ID: 775 (TCC_NORMAL_WRITEBACK) expected 32 instances and got 32 Counter ID: 776 (TCC_ALL_TC_OP_WB_WRITEBACK) expected 32 instances and got 32 Counter ID: 777 (TCC_NORMAL_EVICT) expected 32 instances and got 32 Counter ID: 778 (TCC_ALL_TC_OP_INV_EVICT) expected 32 instances and got 32 Counter ID: 779 (TCC_EA_RDREQ_DRAM) expected 32 instances and got 32 Counter ID: 780 (TCC_EA_WRREQ_DRAM) expected 32 instances and got 32 [ERROR]Counter ID: 1893 (MeanOccupancyPerCU) expected 1 instances and got 8 [ERROR]Counter ID: 1894 (MeanOccupancyPerActiveCU) expected 1 instances and got 8 [ERROR]Counter ID: 1895 (TA_BUSY_avr) expected 16 instances and got 1 [ERROR]Counter ID: 1896 (TA_BUSY_max) expected 16 instances and got 1 [ERROR]Counter ID: 1897 (TA_BUSY_min) expected 16 instances and got 1 [ERROR]Counter ID: 1898 (TA_TA_BUSY_sum) expected 16 instances and got 1 [ERROR]Counter ID: 1899 (TA_TOTAL_WAVEFRONTS_sum) expected 16 instances and got 1 [ERROR]Counter ID: 1900 (TA_ADDR_STALLED_BY_TC_CYCLES_sum) expected 16 instances and got 1 [ERROR]Counter ID: 1901 (TA_ADDR_STALLED_BY_TD_CYCLES_sum) expected 16 instances and got 1 [ERROR]Counter ID: 1902 (TA_DATA_STALLED_BY_TC_CYCLES_sum) expected 16 instances and got 1 [ERROR]Counter ID: 1903 (TA_FLAT_WAVEFRONTS_sum) expected 16 instances and got 1 [ERROR]Counter ID: 1904 (TA_FLAT_READ_WAVEFRONTS_sum) expected 16 instances and got 1 [ERROR]Counter ID: 1905 (TA_FLAT_WRITE_WAVEFRONTS_sum) expected 16 instances and got 1 [ERROR]Counter ID: 1906 (TA_FLAT_ATOMIC_WAVEFRONTS_sum) expected 16 instances and got 1 [ERROR]Counter ID: 1907 (TA_BUFFER_WAVEFRONTS_sum) expected 16 instances and got 1 [ERROR]Counter ID: 1908 (TA_BUFFER_READ_WAVEFRONTS_sum) expected 16 instances and got 1 [ERROR]Counter ID: 1909 (TA_BUFFER_WRITE_WAVEFRONTS_sum) expected 16 instances and got 1 [ERROR]Counter ID: 1910 (TA_BUFFER_ATOMIC_WAVEFRONTS_sum) expected 16 instances and got 1 [ERROR]Counter ID: 1911 (TA_BUFFER_TOTAL_CYCLES_sum) expected 16 instances and got 1 [ERROR]Counter ID: 1912 (TA_BUFFER_COALESCED_READ_CYCLES_sum) expected 16 instances and got 1 [ERROR]Counter ID: 1913 (TA_BUFFER_COALESCED_WRITE_CYCLES_sum) expected 16 instances and got 1 [ERROR]Counter ID: 1914 (TD_TD_BUSY_sum) expected 16 instances and got 1 [ERROR]Counter ID: 1915 (TD_TC_STALL_sum) expected 16 instances and got 1 [ERROR]Counter ID: 1916 (TD_LOAD_WAVEFRONT_sum) expected 16 instances and got 1 [ERROR]Counter ID: 1917 (TD_ATOMIC_WAVEFRONT_sum) expected 16 instances and got 1 [ERROR]Counter ID: 1918 (TD_STORE_WAVEFRONT_sum) expected 16 instances and got 1 [ERROR]Counter ID: 1919 (TD_COALESCABLE_WAVEFRONT_sum) expected 16 instances and got 1 [ERROR]Counter ID: 1920 (TD_SPI_STALL_sum) expected 16 instances and got 1 [ERROR]Counter ID: 1921 (TCP_GATE_EN1_sum) expected 16 instances and got 1 [ERROR]Counter ID: 1922 (TCP_GATE_EN2_sum) expected 16 instances and got 1 [ERROR]Counter ID: 1923 (TCP_TD_TCP_STALL_CYCLES_sum) expected 16 instances and got 1 [ERROR]Counter ID: 1924 (TCP_TCR_TCP_STALL_CYCLES_sum) expected 16 instances and got 1 [ERROR]Counter ID: 1925 (TCP_READ_TAGCONFLICT_STALL_CYCLES_sum) expected 16 instances and got 1 [ERROR]Counter ID: 1926 (TCP_WRITE_TAGCONFLICT_STALL_CYCLES_sum) expected 16 instances and got 1 [ERROR]Counter ID: 1927 (TCP_ATOMIC_TAGCONFLICT_STALL_CYCLES_sum) expected 16 instances and got 1 [ERROR]Counter ID: 1928 (TCP_VOLATILE_sum) expected 16 instances and got 1 [ERROR]Counter ID: 1929 (TCP_TOTAL_ACCESSES_sum) expected 16 instances and got 1 [ERROR]Counter ID: 1930 (TCP_TOTAL_READ_sum) expected 16 instances and got 1 [ERROR]Counter ID: 1931 (TCP_TOTAL_WRITE_sum) expected 16 instances and got 1 [ERROR]Counter ID: 1932 (TCP_TOTAL_ATOMIC_WITH_RET_sum) expected 16 instances and got 1 [ERROR]Counter ID: 1933 (TCP_TOTAL_ATOMIC_WITHOUT_RET_sum) expected 16 instances and got 1 [ERROR]Counter ID: 1934 (TCP_TOTAL_WRITEBACK_INVALIDATES_sum) expected 16 instances and got 1 [ERROR]Counter ID: 1935 (TCP_UTCL1_REQUEST_sum) expected 16 instances and got 1 [ERROR]Counter ID: 1936 (TCP_UTCL1_TRANSLATION_MISS_sum) expected 16 instances and got 1 [ERROR]Counter ID: 1937 (TCP_UTCL1_TRANSLATION_HIT_sum) expected 16 instances and got 1 [ERROR]Counter ID: 1938 (TCP_UTCL1_PERMISSION_MISS_sum) expected 16 instances and got 1 [ERROR]Counter ID: 1939 (TCP_TOTAL_CACHE_ACCESSES_sum) expected 16 instances and got 1 [ERROR]Counter ID: 1940 (TCP_TCP_LATENCY_sum) expected 16 instances and got 1 [ERROR]Counter ID: 1941 (TCP_TA_TCP_STATE_READ_sum) expected 16 instances and got 1 [ERROR]Counter ID: 1942 (TCP_TCC_READ_REQ_LATENCY_sum) expected 16 instances and got 1 [ERROR]Counter ID: 1943 (TCP_TCC_WRITE_REQ_LATENCY_sum) expected 16 instances and got 1 [ERROR]Counter ID: 1944 (TCP_TCC_READ_REQ_sum) expected 16 instances and got 1 [ERROR]Counter ID: 1945 (TCP_TCC_WRITE_REQ_sum) expected 16 instances and got 1 [ERROR]Counter ID: 1946 (TCP_TCC_ATOMIC_WITH_RET_REQ_sum) expected 16 instances and got 1 [ERROR]Counter ID: 1947 (TCP_TCC_ATOMIC_WITHOUT_RET_REQ_sum) expected 16 instances and got 1 [ERROR]Counter ID: 1948 (TCP_TCC_NC_READ_REQ_sum) expected 16 instances and got 1 [ERROR]Counter ID: 1949 (TCP_TCC_NC_WRITE_REQ_sum) expected 16 instances and got 1 [ERROR]Counter ID: 1950 (TCP_TCC_NC_ATOMIC_REQ_sum) expected 16 instances and got 1 [ERROR]Counter ID: 1951 (TCP_TCC_UC_READ_REQ_sum) expected 16 instances and got 1 [ERROR]Counter ID: 1952 (TCP_TCC_UC_WRITE_REQ_sum) expected 16 instances and got 1 [ERROR]Counter ID: 1953 (TCP_TCC_UC_ATOMIC_REQ_sum) expected 16 instances and got 1 [ERROR]Counter ID: 1954 (TCP_TCC_CC_READ_REQ_sum) expected 16 instances and got 1 [ERROR]Counter ID: 1955 (TCP_TCC_CC_WRITE_REQ_sum) expected 16 instances and got 1 [ERROR]Counter ID: 1956 (TCP_TCC_CC_ATOMIC_REQ_sum) expected 16 instances and got 1 [ERROR]Counter ID: 1957 (TCP_TCC_RW_READ_REQ_sum) expected 16 instances and got 1 [ERROR]Counter ID: 1958 (TCP_TCC_RW_WRITE_REQ_sum) expected 16 instances and got 1 [ERROR]Counter ID: 1959 (TCP_TCC_RW_ATOMIC_REQ_sum) expected 16 instances and got 1 [ERROR]Counter ID: 1960 (TCP_PENDING_STALL_CYCLES_sum) expected 16 instances and got 1 [ERROR]Counter ID: 1961 (TCA_CYCLE_sum) expected 32 instances and got 1 [ERROR]Counter ID: 1962 (TCA_BUSY_sum) expected 32 instances and got 1 [ERROR]Counter ID: 1963 (TCC_BUSY_avr) expected 32 instances and got 1 [ERROR]Counter ID: 1964 (TCC_WRREQ_STALL_max) expected 32 instances and got 1 [ERROR]Counter ID: 1965 (TCC_CYCLE_sum) expected 32 instances and got 1 [ERROR]Counter ID: 1966 (TCC_BUSY_sum) expected 32 instances and got 1 [ERROR]Counter ID: 1967 (TCC_REQ_sum) expected 32 instances and got 1 [ERROR]Counter ID: 1968 (TCC_STREAMING_REQ_sum) expected 32 instances and got 1 [ERROR]Counter ID: 1969 (TCC_NC_REQ_sum) expected 32 instances and got 1 [ERROR]Counter ID: 1970 (TCC_UC_REQ_sum) expected 32 instances and got 1 [ERROR]Counter ID: 1971 (TCC_CC_REQ_sum) expected 32 instances and got 1 [ERROR]Counter ID: 1972 (TCC_RW_REQ_sum) expected 32 instances and got 1 [ERROR]Counter ID: 1973 (TCC_PROBE_sum) expected 32 instances and got 1 [ERROR]Counter ID: 1974 (TCC_PROBE_ALL_sum) expected 32 instances and got 1 [ERROR]Counter ID: 1975 (TCC_READ_sum) expected 32 instances and got 1 [ERROR]Counter ID: 1976 (TCC_WRITE_sum) expected 32 instances and got 1 [ERROR]Counter ID: 1977 (TCC_ATOMIC_sum) expected 32 instances and got 1 [ERROR]Counter ID: 1978 (TCC_HIT_sum) expected 32 instances and got 1 [ERROR]Counter ID: 1979 (TCC_MISS_sum) expected 32 instances and got 1 [ERROR]Counter ID: 1980 (TCC_WRITEBACK_sum) expected 32 instances and got 1 [ERROR]Counter ID: 1981 (TCC_EA_WRREQ_sum) expected 32 instances and got 1 [ERROR]Counter ID: 1982 (TCC_EA_WRREQ_64B_sum) expected 32 instances and got 1 [ERROR]Counter ID: 1983 (TCC_EA_WR_UNCACHED_32B_sum) expected 32 instances and got 1 [ERROR]Counter ID: 1984 (TCC_EA_WRREQ_STALL_sum) expected 32 instances and got 1 [ERROR]Counter ID: 1985 (TCC_EA_WRREQ_IO_CREDIT_STALL_sum) expected 32 instances and got 1 [ERROR]Counter ID: 1986 (TCC_EA_WRREQ_GMI_CREDIT_STALL_sum) expected 32 instances and got 1 [ERROR]Counter ID: 1987 (TCC_EA_WRREQ_DRAM_CREDIT_STALL_sum) expected 32 instances and got 1 [ERROR]Counter ID: 1988 (TCC_TOO_MANY_EA_WRREQS_STALL_sum) expected 32 instances and got 1 [ERROR]Counter ID: 1989 (TCC_EA_WRREQ_LEVEL_sum) expected 32 instances and got 1 [ERROR]Counter ID: 1990 (TCC_EA_RDREQ_LEVEL_sum) expected 32 instances and got 1 [ERROR]Counter ID: 1991 (TCC_EA_ATOMIC_sum) expected 32 instances and got 1 [ERROR]Counter ID: 1992 (TCC_EA_ATOMIC_LEVEL_sum) expected 32 instances and got 1 [ERROR]Counter ID: 1993 (TCC_EA_RDREQ_sum) expected 32 instances and got 1 [ERROR]Counter ID: 1994 (TCC_EA_RDREQ_32B_sum) expected 32 instances and got 1 [ERROR]Counter ID: 1995 (TCC_EA_RD_UNCACHED_32B_sum) expected 32 instances and got 1 [ERROR]Counter ID: 1996 (TCC_EA_RDREQ_IO_CREDIT_STALL_sum) expected 32 instances and got 1 [ERROR]Counter ID: 1997 (TCC_EA_RDREQ_GMI_CREDIT_STALL_sum) expected 32 instances and got 1 [ERROR]Counter ID: 1998 (TCC_EA_RDREQ_DRAM_CREDIT_STALL_sum) expected 32 instances and got 1 [ERROR]Counter ID: 1999 (TCC_TAG_STALL_sum) expected 32 instances and got 1 [ERROR]Counter ID: 2000 (TCC_NORMAL_WRITEBACK_sum) expected 32 instances and got 1 [ERROR]Counter ID: 2001 (TCC_ALL_TC_OP_WB_WRITEBACK_sum) expected 32 instances and got 1 [ERROR]Counter ID: 2002 (TCC_NORMAL_EVICT_sum) expected 32 instances and got 1 [ERROR]Counter ID: 2003 (TCC_ALL_TC_OP_INV_EVICT_sum) expected 32 instances and got 1 [ERROR]Counter ID: 2004 (TCC_EA_RDREQ_DRAM_sum) expected 32 instances and got 1 [ERROR]Counter ID: 2005 (TCC_EA_WRREQ_DRAM_sum) expected 32 instances and got 1 [ERROR]Counter ID: 2006 (FETCH_SIZE) expected 32 instances and got 1 [ERROR]Counter ID: 2007 (WRITE_SIZE) expected 32 instances and got 1 [ERROR]Counter ID: 2008 (WRITE_REQ_32B) expected 32 instances and got 1 [ERROR]Counter ID: 2009 (CU_OCCUPANCY) expected 1 instances and got 8 Counter ID: 2010 (CU_UTILIZATION) expected 1 instances and got 1 [ERROR]Counter ID: 2011 (TOTAL_16_OPS) expected 1 instances and got 8 [ERROR]Counter ID: 2012 (TOTAL_32_OPS) expected 1 instances and got 8 [ERROR]Counter ID: 2013 (TOTAL_64_OPS) expected 1 instances and got 8 Counter ID: 2014 (AggSysCycles) expected 1 instances and got 1 Counter ID: 2015 (GpuUtil) expected 1 instances and got 1 Counter ID: 2016 (CpUtil) expected 1 instances and got 1 Counter ID: 2017 (SpiUtil) expected 1 instances and got 1 Counter ID: 2018 (TaUtil) expected 1 instances and got 1 Counter ID: 2019 (TcUtil) expected 1 instances and got 1 Counter ID: 2020 (EaUtil) expected 1 instances and got 1 [ERROR]Counter ID: 2021 (InstrFetchLatency) expected 1 instances and got 8 [ERROR]Counter ID: 2022 (WaveOccupancy) expected 1 instances and got 8 [ERROR]Counter ID: 2023 (WaveDuration) expected 1 instances and got 8 [ERROR]Counter ID: 2024 (WaveDepWait) expected 1 instances and got 8 [ERROR]Counter ID: 2025 (WaveIssueWait) expected 1 instances and got 8 [ERROR]Counter ID: 2026 (WaveExec) expected 1 instances and got 8 [ERROR]Counter ID: 2027 (ValuIops) expected 1 instances and got 8 [ERROR]Counter ID: 2028 (MfmaFlops) expected 1 instances and got 8 [ERROR]Counter ID: 2029 (MfmaFlopsF16) expected 1 instances and got 8 [ERROR]Counter ID: 2030 (MfmaFlopsBF16) expected 1 instances and got 8 [ERROR]Counter ID: 2031 (MfmaFlopsF32) expected 1 instances and got 8 [ERROR]Counter ID: 2032 (MfmaFlopsF64) expected 1 instances and got 8 [ERROR]Counter ID: 2033 (ScaPipeIssueUtil) expected 1 instances and got 8 [ERROR]Counter ID: 2034 (ValuPipeIssueUtil) expected 1 instances and got 8 [ERROR]Counter ID: 2035 (VmemPipeIssueUtil) expected 1 instances and got 8 [ERROR]Counter ID: 2036 (MfmaUtil) expected 1 instances and got 8 [ERROR]Counter ID: 2037 (AvgNumActiveThreads) expected 1 instances and got 8 [ERROR]Counter ID: 2038 (VmemLatency) expected 1 instances and got 8 [ERROR]Counter ID: 2039 (SmemLatency) expected 1 instances and got 8 [ERROR]Counter ID: 2040 (LdsUtil) expected 1 instances and got 8 [ERROR]Counter ID: 2041 (LdsPipeIssueUtil) expected 1 instances and got 8 [ERROR]Counter ID: 2042 (LdsLatency) expected 1 instances and got 8 [ERROR]Counter ID: 2043 (LdsBankConflict) expected 1 instances and got 8 [ERROR]Counter ID: 2044 (L1iCacheHitRate) expected 1 instances and got 8 [ERROR]Counter ID: 2045 (sL1dCacheHitRate) expected 1 instances and got 8 [ERROR]Counter ID: 2046 (vL1dBufCoalesceRate) expected 16 instances and got 1 [ERROR]Counter ID: 2047 (vL1dCacheUtil) expected 16 instances and got 1 [ERROR]Counter ID: 2048 (vL1dCacheTcbHitRate) expected 16 instances and got 1 [ERROR]Counter ID: 2049 (vL1dCacheWaveLatency) expected 16 instances and got 1 [ERROR]Counter ID: 2050 (vL1dReadFromL2Latency) expected 16 instances and got 1 [ERROR]Counter ID: 2051 (vL1dWriteToL2Latency) expected 16 instances and got 1 [ERROR]Counter ID: 2052 (vL1dRdTagConfStallRate) expected 16 instances and got 1 [ERROR]Counter ID: 2053 (vL1dWrTagConfStallRate) expected 16 instances and got 1 [ERROR]Counter ID: 2054 (vL1dAtomicTagConfStallRate) expected 16 instances and got 1 [ERROR]Counter ID: 2055 (vL1dMissReqStallRate) expected 16 instances and got 1 [ERROR]Counter ID: 2056 (vL1dDataPendRate) expected 16 instances and got 1 [ERROR]Counter ID: 2057 (vL1dDataRetStallRate) expected 16 instances and got 1 [ERROR]Counter ID: 2058 (L2CacheHitRate) expected 32 instances and got 1 [ERROR]Counter ID: 2059 (L2CacheTagRamStallRate) expected 32 instances and got 1 [ERROR]Counter ID: 2060 (EaRdLatency) expected 32 instances and got 1 [ERROR]Counter ID: 2061 (EaRdIoStallRate) expected 32 instances and got 1 [ERROR]Counter ID: 2062 (EaRdGmiStallRate) expected 32 instances and got 1 [ERROR]Counter ID: 2063 (EaRdDramStallRate) expected 32 instances and got 1 [ERROR]Counter ID: 2064 (EaWrLatency) expected 32 instances and got 1 [ERROR]Counter ID: 2065 (EaWrIoStallRate) expected 32 instances and got 1 [ERROR]Counter ID: 2066 (EaWrGmiStallRate) expected 32 instances and got 1 [ERROR]Counter ID: 2067 (EaWrDramStallRate) expected 32 instances and got 1 [ERROR]Counter ID: 2068 (EaWrStarveRate) expected 32 instances and got 1 [ERROR]Counter ID: 2069 (EaAtomicLatency) expected 32 instances and got 1 [ERROR]Counter ID: 2070 (TCP_TCP_TA_DATA_STALL_CYCLES_sum) expected 16 instances and got 1 [ERROR]Counter ID: 2071 (TCP_TCP_TA_DATA_STALL_CYCLES_max) expected 16 instances and got 1 [ERROR]Counter ID: 2072 (VFetchInsts) expected 16 instances and got 8 [ERROR]Counter ID: 2073 (VWriteInsts) expected 16 instances and got 8 [ERROR]Counter ID: 2074 (FlatVMemInsts) expected 1 instances and got 8 [ERROR]Counter ID: 2075 (LDSInsts) expected 1 instances and got 8 [ERROR]Counter ID: 2076 (FlatLDSInsts) expected 1 instances and got 8 [ERROR]Counter ID: 2077 (VALUUtilization) expected 1 instances and got 8 [ERROR]Counter ID: 2078 (VALUBusy) expected 1 instances and got 8 [ERROR]Counter ID: 2079 (SALUBusy) expected 1 instances and got 8 [ERROR]Counter ID: 2080 (FetchSize) expected 32 instances and got 1 [ERROR]Counter ID: 2081 (WriteSize) expected 32 instances and got 1 [ERROR]Counter ID: 2082 (MemWrites32B) expected 32 instances and got 1 [ERROR]Counter ID: 2083 (L2CacheHit) expected 32 instances and got 1 [ERROR]Counter ID: 2084 (MemUnitStalled) expected 16 instances and got 1 [ERROR]Counter ID: 2085 (WriteUnitStalled) expected 32 instances and got 1 [ERROR]Counter ID: 2086 (LDSBankConflict) expected 1 instances and got 8 * source formatting (clang-format v11) (#225) Co-authored-by: bwelton * cmake formatting (cmake-format) (#224) Co-authored-by: bwelton * Minor fixes * source formatting (clang-format v11) (#226) Co-authored-by: bwelton * Minor test change --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: bwelton [ROCm/rocprofiler-sdk commit: e8a5845661801c9d443576a98b32c81c3a43e815] --- .../samples/counter_collection/CMakeLists.txt | 46 +- .../samples/counter_collection/client.cpp | 227 +++-- .../test_counter_client.cpp | 277 ++++++ .../source/include/rocprofiler/counters.h | 9 +- .../include/rocprofiler/dispatch_profile.h | 60 +- .../source/include/rocprofiler/fwd.h | 8 +- .../source/lib/rocprofiler/agent.cpp | 8 + .../source/lib/rocprofiler/agent.hpp | 10 + .../source/lib/rocprofiler/aql/helpers.cpp | 1 + .../lib/rocprofiler/aql/packet_construct.cpp | 8 +- .../lib/rocprofiler/aql/packet_construct.hpp | 4 +- .../source/lib/rocprofiler/counters.cpp | 39 +- .../source/lib/rocprofiler/counters/core.cpp | 377 ++++--- .../source/lib/rocprofiler/counters/core.hpp | 38 +- .../lib/rocprofiler/counters/dimensions.cpp | 87 +- .../lib/rocprofiler/counters/dimensions.hpp | 2 +- .../lib/rocprofiler/counters/evaluate_ast.cpp | 218 ++++- .../lib/rocprofiler/counters/evaluate_ast.hpp | 91 +- .../lib/rocprofiler/counters/metrics.cpp | 58 +- .../rocprofiler/counters/parser/raw_ast.hpp | 4 +- .../counters/parser/tests/CMakeLists.txt | 5 +- .../counters/parser/tests/parser_test.cpp | 18 + .../counters/tests/evaluate_ast_test.cpp | 918 ++++++++++++++++-- .../counters/tests/metrics_test.cpp | 5 +- .../rocprofiler/counters/tests/metrics_test.h | 3 +- .../counters/xml/basic_counters.xml | 63 +- .../counters/xml/derived_counters.xml | 16 +- .../lib/rocprofiler/dispatch_profile.cpp | 22 +- .../source/lib/rocprofiler/hsa/aql_packet.cpp | 12 +- .../source/lib/rocprofiler/hsa/aql_packet.hpp | 1 + .../source/lib/rocprofiler/hsa/queue.cpp | 17 +- .../source/lib/rocprofiler/hsa/queue.hpp | 26 +- .../lib/rocprofiler/hsa/queue_controller.cpp | 22 +- .../lib/rocprofiler/hsa/queue_controller.hpp | 10 +- .../source/lib/rocprofiler/profile_config.cpp | 10 +- 35 files changed, 2088 insertions(+), 632 deletions(-) create mode 100644 projects/rocprofiler-sdk/samples/counter_collection/test_counter_client.cpp diff --git a/projects/rocprofiler-sdk/samples/counter_collection/CMakeLists.txt b/projects/rocprofiler-sdk/samples/counter_collection/CMakeLists.txt index 3ad3ab924b..66302525cb 100644 --- a/projects/rocprofiler-sdk/samples/counter_collection/CMakeLists.txt +++ b/projects/rocprofiler-sdk/samples/counter_collection/CMakeLists.txt @@ -29,22 +29,44 @@ if(NOT TARGET rocprofiler::rocprofiler) find_package(rocprofiler REQUIRED) endif() -add_library(counter-collection-client SHARED) -target_sources(counter-collection-client PRIVATE client.cpp client.hpp) -target_link_libraries(counter-collection-client PRIVATE rocprofiler::rocprofiler) - +add_library(counter-collection-buffer-client SHARED) +target_sources(counter-collection-buffer-client PRIVATE client.cpp client.hpp) +target_link_libraries(counter-collection-buffer-client PRIVATE rocprofiler::rocprofiler) set_source_files_properties(main.cpp PROPERTIES LANGUAGE HIP) -find_package(Threads REQUIRED) +add_executable(counter-collection-buffer) +target_sources(counter-collection-buffer PRIVATE main.cpp) +target_link_libraries(counter-collection-buffer PRIVATE counter-collection-buffer-client + Threads::Threads) -add_executable(counter-collection) -target_sources(counter-collection PRIVATE main.cpp) -target_link_libraries(counter-collection PRIVATE counter-collection-client - Threads::Threads) - -add_test(NAME counter-collection COMMAND $) +add_test(NAME counter-collection-buffer COMMAND $) set_tests_properties( - counter-collection + counter-collection-buffer + PROPERTIES + TIMEOUT + 300 + LABELS + "samples" + ENVIRONMENT + "${ROCPROFILER_MEMCHECK_PRELOAD_ENV};HSA_TOOLS_LIB=$" + FAIL_REGULAR_EXPRESSION + "threw an exception") + +add_library(counter-collection-test-counter-client SHARED) +target_sources(counter-collection-test-counter-client PRIVATE test_counter_client.cpp + client.hpp) +target_link_libraries(counter-collection-test-counter-client + PRIVATE rocprofiler::rocprofiler) +add_executable(counter-collection-test-counters) +target_sources(counter-collection-test-counters PRIVATE main.cpp) +target_link_libraries(counter-collection-test-counters + PRIVATE counter-collection-test-counter-client Threads::Threads) + +add_test(NAME counter-collection-test-counters + COMMAND $) + +set_tests_properties( + counter-collection-test-counters PROPERTIES TIMEOUT 300 diff --git a/projects/rocprofiler-sdk/samples/counter_collection/client.cpp b/projects/rocprofiler-sdk/samples/counter_collection/client.cpp index 80cd0a5fe6..4fd71ad803 100644 --- a/projects/rocprofiler-sdk/samples/counter_collection/client.cpp +++ b/projects/rocprofiler-sdk/samples/counter_collection/client.cpp @@ -22,8 +22,13 @@ #include "client.hpp" +#include +#include +#include #include -#include // std::stringstream +#include +#include +#include #include #include @@ -60,101 +65,163 @@ get_client_ctx() return ctx; } +rocprofiler_buffer_id_t& +get_buffer() +{ + static rocprofiler_buffer_id_t buf = {}; + return buf; +} + +std::ostream* +get_output_stream() +{ + static std::ostream* isTerm = []() -> std::ostream* { + if(auto* outfile = getenv("ROCPROFILER_SAMPLE_OUTPUT_FILE")) + { + if(outfile == "stdout") + return static_cast(&std::cout); + else if(outfile == "stderr") + return &std::cerr; + } + return nullptr; + }(); + static std::unique_ptr stream; + + if(isTerm) return isTerm; + if(stream) return stream.get(); + std::string filename = "counter_collection.log"; + if(auto* outfile = getenv("ROCPROFILER_SAMPLE_OUTPUT_FILE")) + { + filename = outfile; + } + stream = std::make_unique(filename); + return stream.get(); +} + void -test_callback(rocprofiler_queue_id_t queue_id, - rocprofiler_agent_t agent_id, - rocprofiler_correlation_id_t corr_id, - const hsa_kernel_dispatch_packet_t*, - void*, - rocprofiler_record_counter_t* out_counters, - size_t out_size, - rocprofiler_profile_config_id_t) +buffered_callback(rocprofiler_context_id_t, + rocprofiler_buffer_id_t, + rocprofiler_record_header_t** headers, + size_t num_headers, + void*, + uint64_t) { static int enter_count = 0; enter_count++; - // Limit output to avoid massive log size if(enter_count % 100 != 0) return; - std::stringstream ss; - for(size_t i = 0; i < out_size; i++) + for(size_t i = 0; i < num_headers; ++i) { - ss << "(Id: " << out_counters[i].id << " Value [D]: " << out_counters[i].derived_counter - << ", Value [I]: " << out_counters[i].hw_counter << "),"; + auto* header = headers[i]; + if(header->category == ROCPROFILER_BUFFER_CATEGORY_COUNTERS && header->kind == 0) + { + auto* record = static_cast(header->payload); + ss << "(Id: " << record->id << " Value [D]: " << record->counter_value + << " Corr_Id: " << record->corr_id.internal << "),"; + } } - // Callback containing counter data. - std::clog << "[" << __FUNCTION__ << "] " << queue_id.handle << " | " << agent_id.id.handle - << " | " << corr_id.internal << "|" << ss.str() << "\n"; + + *get_output_stream() << "[" << __FUNCTION__ << "] " << ss.str() << "\n"; +} + +void +dispatch_callback(rocprofiler_queue_id_t queue_id, + const rocprofiler_agent_t* agent, + rocprofiler_correlation_id_t correlation_id, + const hsa_kernel_dispatch_packet_t* dispatch_packet, + void* callback_data_args, + rocprofiler_profile_config_id_t* config) +{ + /** + * This simple example uses the same profile counter set for all agents. + * We store this in a cache to prevent constructing many identical profile counter + * sets. We first check the cache to see if we have already constructed a counter" + * set for the agent. If we have, return it. Otherwise, construct a new profile counter + * set. + */ + static std::shared_mutex m_mutex = {}; + static std::unordered_map profile_cache = {}; + + auto search_cache = [&]() { + if(auto pos = profile_cache.find(agent->id.handle); pos != profile_cache.end()) + { + *config = pos->second; + return true; + } + return false; + }; + + { + auto rlock = std::shared_lock{m_mutex}; + if(search_cache()) return; + } + + auto wlock = std::unique_lock{m_mutex}; + if(search_cache()) return; + + std::set counters_to_collect = {"SQ_WAVES"}; + std::vector gpu_counters; + ROCPROFILER_CALL( + rocprofiler_iterate_agent_supported_counters( + *agent, + [](rocprofiler_counter_id_t* counters, size_t num_counters, void* user_data) { + std::vector* vec = + static_cast*>(user_data); + for(size_t i = 0; i < num_counters; i++) + { + vec->push_back(counters[i]); + } + return ROCPROFILER_STATUS_SUCCESS; + }, + static_cast(&gpu_counters)), + "Could not fetch supported counters"); + + std::vector collect_counters; + for(auto& counter : gpu_counters) + { + const char* name; + size_t size; + ROCPROFILER_CALL(rocprofiler_query_counter_name(counter, &name, &size), + "Could not query name"); + if(counters_to_collect.count(std::string(name)) > 0) + { + std::clog << "Counter: " << counter.handle << " " << name << "\n"; + collect_counters.push_back(counter); + } + } + + rocprofiler_profile_config_id_t profile; + ROCPROFILER_CALL(rocprofiler_create_profile_config( + *agent, collect_counters.data(), collect_counters.size(), &profile), + "Could not construct profile cfg"); + + profile_cache.emplace(agent->id.handle, profile); + *config = profile; } int tool_init(rocprofiler_client_finalize_t, void*) { - std::set counters_to_collect = {"SQ_WAVES"}; - - std::vector gpu_agents; - auto agent_query = [](const rocprofiler_agent_t** agents, size_t num_agents, void* user_data) { - std::vector* vec = - static_cast*>(user_data); - for(size_t i = 0; i < num_agents; i++) - { - const rocprofiler_agent_t* agent = agents[i]; - if(agent->type == ROCPROFILER_AGENT_TYPE_GPU) - { - vec->push_back(*agent); - } - } - return ROCPROFILER_STATUS_SUCCESS; - }; - - ROCPROFILER_CALL(rocprofiler_query_available_agents( - agent_query, sizeof(rocprofiler_agent_t), static_cast(&gpu_agents)), - "Could not query agents"); - ROCPROFILER_CALL(rocprofiler_create_context(&get_client_ctx()), "context creation failed"); - std::vector profile_configs; + ROCPROFILER_CALL(rocprofiler_create_buffer(get_client_ctx(), + 4096, + 2048, + ROCPROFILER_BUFFER_POLICY_LOSSLESS, + buffered_callback, + nullptr, + &get_buffer()), + "buffer creation failed"); - for(auto& agent : gpu_agents) - { - std::vector collect_counters; - std::vector gpu_counters; - std::clog << agent.name << "\n"; - ROCPROFILER_CALL( - rocprofiler_iterate_agent_supported_counters( - agent, - [](rocprofiler_counter_id_t* counters, size_t num_counters, void* user_data) { - std::vector* vec = - static_cast*>(user_data); - for(size_t i = 0; i < num_counters; i++) - { - vec->push_back(counters[i]); - } - return ROCPROFILER_STATUS_SUCCESS; - }, - static_cast(&gpu_counters)), - "Could not fetch supported counters"); - - for(auto& counter : gpu_counters) - { - const char* name; - size_t size; - ROCPROFILER_CALL(rocprofiler_query_counter_name(counter, &name, &size), - "Could not query name"); - if(counters_to_collect.count(std::string(name)) > 0) - { - std::clog << "Counter: " << counter.handle << " " << name << "\n"; - collect_counters.push_back(counter); - } - } - rocprofiler_profile_config_id_t profile; - ROCPROFILER_CALL(rocprofiler_create_profile_config( - agent, collect_counters.data(), collect_counters.size(), &profile), - "Could not construct profile cfg"); - ROCPROFILER_CALL(rocprofiler_configure_dispatch_profile_counting_service( - get_client_ctx(), profile, test_callback, nullptr), - "Could not setup dispatch service"); - profile_configs.push_back(profile); - } + auto client_thread = rocprofiler_callback_thread_t{}; + ROCPROFILER_CALL(rocprofiler_create_callback_thread(&client_thread), + "failure creating callback thread"); + get_output_stream(); + ROCPROFILER_CALL(rocprofiler_assign_callback_thread(get_buffer(), client_thread), + "failed to assign thread for buffer"); + ROCPROFILER_CALL(rocprofiler_configure_buffered_dispatch_profile_counting_service( + get_client_ctx(), get_buffer(), dispatch_callback, nullptr), + "Could not setup buffered service"); rocprofiler_start_context(get_client_ctx()); // no errors diff --git a/projects/rocprofiler-sdk/samples/counter_collection/test_counter_client.cpp b/projects/rocprofiler-sdk/samples/counter_collection/test_counter_client.cpp new file mode 100644 index 0000000000..7ebcfdf5c2 --- /dev/null +++ b/projects/rocprofiler-sdk/samples/counter_collection/test_counter_client.cpp @@ -0,0 +1,277 @@ +#include "client.hpp" + +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include +#include + +/** + * Tests the collection of all counters on the agent the test is run on. + */ + +#define ROCPROFILER_CALL(result, msg) \ + { \ + rocprofiler_status_t CHECKSTATUS = result; \ + if(CHECKSTATUS != ROCPROFILER_STATUS_SUCCESS) \ + { \ + std::string status_msg = rocprofiler_get_status_string(CHECKSTATUS); \ + std::cerr << "[" #result "][" << __FILE__ << ":" << __LINE__ << "] " << msg \ + << " failed with error code " << CHECKSTATUS << ": " << status_msg \ + << std::endl; \ + std::stringstream errmsg{}; \ + errmsg << "[" #result "][" << __FILE__ << ":" << __LINE__ << "] " << msg " failure (" \ + << status_msg << ")"; \ + throw std::runtime_error(errmsg.str()); \ + } \ + } + +int +start() +{ + return 1; +} + +namespace +{ +rocprofiler_context_id_t& +get_client_ctx() +{ + static rocprofiler_context_id_t ctx; + return ctx; +} + +rocprofiler_buffer_id_t& +get_buffer() +{ + static rocprofiler_buffer_id_t buf = {}; + return buf; +} + +struct CaptureRecords +{ + std::shared_mutex m_mutex{}; + // + std::map expected{}; + std::map expected_counter_names{}; + std::vector remaining{}; + // + std::map captured{}; +}; + +CaptureRecords* REC = new CaptureRecords; + +CaptureRecords* +get_capture() +{ + return REC; +} + +void +buffered_callback(rocprofiler_context_id_t, + rocprofiler_buffer_id_t, + rocprofiler_record_header_t** headers, + size_t num_headers, + void*, + uint64_t) +{ + auto& cap = *get_capture(); + auto wlock = std::unique_lock{cap.m_mutex}; + std::map seen_counters; + for(size_t i = 0; i < num_headers; ++i) + { + auto* header = headers[i]; + if(header->category == ROCPROFILER_BUFFER_CATEGORY_COUNTERS && header->kind == 0) + { + rocprofiler_counter_id_t counter; + auto* record = static_cast(header->payload); + rocprofiler_query_record_counter_id(record->id, &counter); + if(counter.handle == 517) + { + std::clog << "HERE"; + } + seen_counters.emplace(counter.handle, 0).first->second++; + } + } + + for(const auto& [counter_id, instances] : seen_counters) + { + cap.captured.emplace(counter_id, 0).first->second += instances; + } +} + +void +dispatch_callback(rocprofiler_queue_id_t queue_id, + const rocprofiler_agent_t* agent, + rocprofiler_correlation_id_t correlation_id, + const hsa_kernel_dispatch_packet_t* dispatch_packet, + void* callback_data_args, + rocprofiler_profile_config_id_t* config) +{ + auto& cap = *get_capture(); + auto wlock = std::unique_lock{cap.m_mutex}; + + if(cap.expected.empty()) + { + std::vector counters_needed; + ROCPROFILER_CALL( + rocprofiler_iterate_agent_supported_counters( + *agent, + [](rocprofiler_counter_id_t* counters, size_t num_counters, void* user_data) { + std::vector* vec = + static_cast*>(user_data); + for(size_t i = 0; i < num_counters; i++) + { + vec->push_back(counters[i]); + } + return ROCPROFILER_STATUS_SUCCESS; + }, + static_cast(&counters_needed)), + "Could not fetch supported counters"); + + for(auto& found_counter : counters_needed) + { + size_t expected = 0; + rocprofiler_query_counter_instance_count(*agent, found_counter, &expected); + cap.remaining.push_back(found_counter); + cap.expected.emplace(found_counter.handle, expected); + const char* name; + size_t name_size; + ROCPROFILER_CALL(rocprofiler_query_counter_name(found_counter, &name, &name_size), + "Could not query name"); + cap.expected_counter_names.emplace(found_counter.handle, std::string(name)); + } + if(cap.expected.empty()) throw std::runtime_error("No counters found for agent"); + } + if(cap.remaining.empty()) return; + + rocprofiler_profile_config_id_t profile; + + ROCPROFILER_CALL( + rocprofiler_create_profile_config(*agent, &(cap.remaining.back()), 1, &profile), + "Could not construct profile cfg"); + + cap.remaining.pop_back(); + *config = profile; +} + +int +tool_init(rocprofiler_client_finalize_t, void*) +{ + get_capture(); + ROCPROFILER_CALL(rocprofiler_create_context(&get_client_ctx()), "context creation failed"); + + ROCPROFILER_CALL(rocprofiler_create_buffer(get_client_ctx(), + 4096, + 2048, + ROCPROFILER_BUFFER_POLICY_LOSSLESS, + buffered_callback, + nullptr, + &get_buffer()), + "buffer creation failed"); + + auto client_thread = rocprofiler_callback_thread_t{}; + ROCPROFILER_CALL(rocprofiler_create_callback_thread(&client_thread), + "failure creating callback thread"); + + ROCPROFILER_CALL(rocprofiler_assign_callback_thread(get_buffer(), client_thread), + "failed to assign thread for buffer"); + ROCPROFILER_CALL(rocprofiler_configure_buffered_dispatch_profile_counting_service( + get_client_ctx(), get_buffer(), dispatch_callback, nullptr), + "Could not setup buffered service"); + rocprofiler_start_context(get_client_ctx()); + + // no errors + return 0; +} + +void +tool_fini(void*) +{ + rocprofiler_flush_buffer(get_buffer()); + rocprofiler_stop_context(get_client_ctx()); + // Flush buffer isn't waiting.... + sleep(2); + + std::clog << "In tool fini\n"; + + auto cap_ptr = get_capture(); + auto& cap = *get_capture(); + auto wlock = std::unique_lock{cap.m_mutex}; + + if(cap.captured.size() != cap.expected.size()) + { + std::clog << "[ERROR] Expected " << cap.expected.size() << " counters collected but got " + << cap.captured.size() << "\n"; + } + + for(const auto& [counter_id, expected] : cap.expected) + { + std::string name = "UNKNOWN"; + if(auto pos = cap.expected_counter_names.find(counter_id); + pos != cap.expected_counter_names.end()) + { + name = pos->second; + } + + std::optional actual_size; + + if(auto pos = cap.captured.find(counter_id); pos != cap.captured.end()) + { + actual_size = pos->second; + } + + if(actual_size && *actual_size != expected) + { + std::clog << (*actual_size == expected ? "" : "[ERROR]") << "Counter ID: " << counter_id + << " (" << name << ")" + << " expected " << expected << " instances and got " << *actual_size << "\n"; + } + else + { + std::clog << "[ERROR] Counter ID: " << counter_id << " (" << name + << ") is missing from output\n"; + } + } +} + +} // namespace + +extern "C" rocprofiler_tool_configure_result_t* +rocprofiler_configure(uint32_t version, + const char* runtime_version, + uint32_t, + rocprofiler_client_id_t* id) +{ + // set the client name + id->name = "CounterClientSample"; + + // compute major/minor/patch version info + uint32_t major = version / 10000; + uint32_t minor = (version % 10000) / 100; + uint32_t patch = version % 100; + + // generate info string + auto info = std::stringstream{}; + info << id->name << " is using rocprofiler v" << major << "." << minor << "." << patch << " (" + << runtime_version << ")"; + + std::clog << info.str() << std::endl; + + // create configure data + static auto cfg = + rocprofiler_tool_configure_result_t{sizeof(rocprofiler_tool_configure_result_t), + &tool_init, + &tool_fini, + static_cast(nullptr)}; + + // return pointer to configure data + return &cfg; +} diff --git a/projects/rocprofiler-sdk/source/include/rocprofiler/counters.h b/projects/rocprofiler-sdk/source/include/rocprofiler/counters.h index 412586998a..bd7481cffa 100644 --- a/projects/rocprofiler-sdk/source/include/rocprofiler/counters.h +++ b/projects/rocprofiler-sdk/source/include/rocprofiler/counters.h @@ -85,12 +85,17 @@ rocprofiler_query_counter_name(rocprofiler_counter_id_t counter_id, const char** ROCPROFILER_NONNULL(2, 3); /** - * @brief Query Counter Instances Count. + * @brief This call returns the number of instances specific counter contains. + * WARNING: There is a restriction on this call in the alpha/beta release + * of rocprof. This call will not return correct instance information in + * tool_init and must be called as part of the dispatch callback for accurate + * instance counting information. The reason for this restriction is that HSA + * is not yet loaded on tool_init. * * @param [in] agent rocprofiler agent * @param [in] counter_id counter id (obtained from iterate_agent_supported_counters) * @param [out] instance_count number of instances the counter has - * @return ::rocprofiler_status_t + * @return rocprofiler_status_t */ rocprofiler_status_t ROCPROFILER_API rocprofiler_query_counter_instance_count(rocprofiler_agent_t agent, diff --git a/projects/rocprofiler-sdk/source/include/rocprofiler/dispatch_profile.h b/projects/rocprofiler-sdk/source/include/rocprofiler/dispatch_profile.h index 37ce4856b0..200c22ad03 100644 --- a/projects/rocprofiler-sdk/source/include/rocprofiler/dispatch_profile.h +++ b/projects/rocprofiler-sdk/source/include/rocprofiler/dispatch_profile.h @@ -38,62 +38,46 @@ ROCPROFILER_EXTERN_C_INIT */ /** - * @brief ROCProfiler Profile Counting Data. + * @brief Kernel Dispatch Callback. This is a callback that is invoked before the kernel + * is enqueued into the HSA queue. What counters to collect for a kernel are set + * via passing back a profile config (config) in this callback. These counters + * will be collected and emplaced in the buffer rocprofiler_buffer_id_t used when + * setting up this callback. * - */ -typedef struct -{ - rocprofiler_timestamp_t start_timestamp; - rocprofiler_timestamp_t end_timestamp; - /** - * Counters, including identifiers to get counter information and Counters - * values - * - * Should it be a record per counter? - */ - rocprofiler_record_counter_t* counters; - uint64_t counters_count; - rocprofiler_correlation_id_t correlation_id; -} rocprofiler_dispatch_profile_counting_record_t; - -/** - * @brief Kernel Dispatch Callback - * - * @param [out] queue_id - * @param [out] agent_id - * @param [out] correlation_id - * @param [out] dispatch_packet It can be used to get the kernel descriptor and then using - * code_object tracing, we can get the kernel name. `dispatch_packet->reserved2` is the - * correlation_id used to correlate the dispatch packet with the corresponding API call. - * @param [out] callback_data_args - * @param [in] config + * @param [in] queue_id Queue the kernel dispatach packet is being enqueued onto + * @param [in] agent Agent of this queue + * @param [in] correlation_id Correlation ID for this dispatch + * @param [in] dispatch_packet Kernel dispatch packet about to be enqueued into HSA + * @param [in] callback_data_args Callback supplied via buffered_dispatch_profile_counting_service + * @param [out] config Profile config detailing the counters to collect for this kernel */ typedef void (*rocprofiler_profile_counting_dispatch_callback_t)( rocprofiler_queue_id_t queue_id, - rocprofiler_agent_t agent_id, + const rocprofiler_agent_t* agent, rocprofiler_correlation_id_t correlation_id, const hsa_kernel_dispatch_packet_t* dispatch_packet, void* callback_data_args, - rocprofiler_record_counter_t* records, - size_t record_count, - rocprofiler_profile_config_id_t config); + rocprofiler_profile_config_id_t* config); /** - * @brief Configure Dispatch Profile Counting Service. + * @brief Configure buffered dispatch profile Counting Service. + * Collects the counters in dispatch packets and stores them + * in buffer_id. The buffer may contain packets from more than + * one dispatch (denoted by correlation id). Will trigger the + * callback based on the parameters setup in buffer_id_t. * * @param [in] context_id context id - * @param [in] profile profile config to use for dispatch - * @param [in] callback callback + * @param [in] buffer_id id of the buffer to use for the counting service + * @param [in] callback callback to perform when dispatch is enqueued * @param [in] callback_data_args callback data * @return ::rocprofiler_status_t */ rocprofiler_status_t ROCPROFILER_API -rocprofiler_configure_dispatch_profile_counting_service( +rocprofiler_configure_buffered_dispatch_profile_counting_service( rocprofiler_context_id_t context_id, - rocprofiler_profile_config_id_t profile, + rocprofiler_buffer_id_t buffer_id, rocprofiler_profile_counting_dispatch_callback_t callback, void* callback_data_args); - /** @} */ ROCPROFILER_EXTERN_C_FINI diff --git a/projects/rocprofiler-sdk/source/include/rocprofiler/fwd.h b/projects/rocprofiler-sdk/source/include/rocprofiler/fwd.h index 9b986ecf95..87c6c8d406 100644 --- a/projects/rocprofiler-sdk/source/include/rocprofiler/fwd.h +++ b/projects/rocprofiler-sdk/source/include/rocprofiler/fwd.h @@ -86,6 +86,7 @@ typedef enum // NOLINT(performance-enum-size) ROCPROFILER_BUFFER_CATEGORY_NONE = 0, ROCPROFILER_BUFFER_CATEGORY_TRACING, ROCPROFILER_BUFFER_CATEGORY_PC_SAMPLING, + ROCPROFILER_BUFFER_CATEGORY_COUNTERS, ROCPROFILER_BUFFER_CATEGORY_LAST, } rocprofiler_buffer_category_t; @@ -453,11 +454,8 @@ typedef struct typedef struct { rocprofiler_counter_instance_id_t id; - union - { - int64_t hw_counter; //<< physical hardware counter - double derived_counter; //<< derived counter value - }; + double counter_value; //<< counter value + rocprofiler_correlation_id_t corr_id; } rocprofiler_record_counter_t; /** diff --git a/projects/rocprofiler-sdk/source/lib/rocprofiler/agent.cpp b/projects/rocprofiler-sdk/source/lib/rocprofiler/agent.cpp index a182bf2212..d15ba6ecd3 100644 --- a/projects/rocprofiler-sdk/source/lib/rocprofiler/agent.cpp +++ b/projects/rocprofiler-sdk/source/lib/rocprofiler/agent.cpp @@ -246,6 +246,7 @@ read_property(const MapT& data, const std::string& label, Tp& value) { using mutable_type = std::remove_const_t; + get_agent_available_properties().insert(label); if constexpr(std::is_enum::value) { using value_type = std::underlying_type_t; @@ -813,6 +814,13 @@ get_agent_cache(hsa_agent_t agent) return std::nullopt; } + +std::unordered_set& +get_agent_available_properties() +{ + static std::unordered_set _prop; + return _prop; +} } // namespace agent } // namespace rocprofiler diff --git a/projects/rocprofiler-sdk/source/lib/rocprofiler/agent.hpp b/projects/rocprofiler-sdk/source/lib/rocprofiler/agent.hpp index e0d2aa1346..979144a692 100644 --- a/projects/rocprofiler-sdk/source/lib/rocprofiler/agent.hpp +++ b/projects/rocprofiler-sdk/source/lib/rocprofiler/agent.hpp @@ -29,6 +29,7 @@ #include #include +#include #include namespace rocprofiler @@ -52,5 +53,14 @@ get_agent_cache(const rocprofiler_agent_t* agent); std::optional get_agent_cache(hsa_agent_t agent); + +/** + * @brief A set containing all properties that may have been + * set during decoding of the properties file. + * + * @return std::unordered_set of all property names + */ +std::unordered_set& +get_agent_available_properties(); } // namespace agent } // namespace rocprofiler diff --git a/projects/rocprofiler-sdk/source/lib/rocprofiler/aql/helpers.cpp b/projects/rocprofiler-sdk/source/lib/rocprofiler/aql/helpers.cpp index e4a92507cd..f02437d042 100644 --- a/projects/rocprofiler-sdk/source/lib/rocprofiler/aql/helpers.cpp +++ b/projects/rocprofiler-sdk/source/lib/rocprofiler/aql/helpers.cpp @@ -39,6 +39,7 @@ get_query_info(hsa_agent_t agent, const counters::Metric& metric) if(hsa_ven_amd_aqlprofile_get_info(&profile, HSA_VEN_AMD_AQLPROFILE_INFO_BLOCK_ID, &query) != HSA_STATUS_SUCCESS) { + DLOG(FATAL) << fmt::format("AQL failed to query info for counter {}", metric); throw std::runtime_error(fmt::format("AQL failed to query info for counter {}", metric)); } return query; diff --git a/projects/rocprofiler-sdk/source/lib/rocprofiler/aql/packet_construct.cpp b/projects/rocprofiler-sdk/source/lib/rocprofiler/aql/packet_construct.cpp index 02d7278866..bef067bff6 100644 --- a/projects/rocprofiler-sdk/source/lib/rocprofiler/aql/packet_construct.cpp +++ b/projects/rocprofiler-sdk/source/lib/rocprofiler/aql/packet_construct.cpp @@ -34,11 +34,6 @@ AQLPacketConstruct::AQLPacketConstruct(const hsa::AgentCache& agen const std::vector& metrics) : _agent(agent) { - if(metrics.empty()) - { - throw std::runtime_error("No metrics supplied"); - } - // Validate that the counter exists and construct the block instances // for the counter. for(const auto& x : metrics) @@ -80,8 +75,9 @@ AQLPacketConstruct::construct_packet(const AmdExtTable& ext) const auto& pkt = *pkt_ptr; if(_events.empty()) { - throw std::runtime_error("Constructing packet with no events"); + return pkt_ptr; } + pkt.empty = false; pkt.profile = hsa_ven_amd_aqlprofile_profile_t{ _agent.get_hsa_agent(), diff --git a/projects/rocprofiler-sdk/source/lib/rocprofiler/aql/packet_construct.hpp b/projects/rocprofiler-sdk/source/lib/rocprofiler/aql/packet_construct.hpp index e918e69310..3dcc5a3ee3 100644 --- a/projects/rocprofiler-sdk/source/lib/rocprofiler/aql/packet_construct.hpp +++ b/projects/rocprofiler-sdk/source/lib/rocprofiler/aql/packet_construct.hpp @@ -54,6 +54,7 @@ public: std::unique_ptr construct_packet(const AmdExtTable&) const; const counters::Metric* event_to_metric(const hsa_ven_amd_aqlprofile_event_t& event) const; + std::vector get_all_events() const; private: struct AQLProfileMetric @@ -62,8 +63,7 @@ private: std::vector instances; }; - std::vector get_all_events() const; - void can_collect(); + void can_collect(); const hsa::AgentCache& _agent; std::vector _metrics; diff --git a/projects/rocprofiler-sdk/source/lib/rocprofiler/counters.cpp b/projects/rocprofiler-sdk/source/lib/rocprofiler/counters.cpp index 4e89a29f8c..070e136373 100644 --- a/projects/rocprofiler-sdk/source/lib/rocprofiler/counters.cpp +++ b/projects/rocprofiler-sdk/source/lib/rocprofiler/counters.cpp @@ -22,6 +22,8 @@ #include +#include + #include "lib/common/synchronized.hpp" #include "lib/rocprofiler/aql/helpers.hpp" #include "lib/rocprofiler/counters/evaluate_ast.hpp" @@ -56,11 +58,17 @@ rocprofiler_query_counter_name(rocprofiler_counter_id_t counter_id, const char** } /** - * @brief Query Counter Instances Count. + * @brief This call returns the number of instances specific counter contains. + * WARNING: There is a restriction on this call in the alpha/beta release + * of rocprof. This call will not return correct instance information in + * tool_init and must be called as part of the dispatch callback for accurate + * instance counting information. The reason for this restriction is that HSA + * is not yet loaded on tool_init. * - * @param [in] counter_id - * @param [out] instance_count - * @return ::rocprofiler_status_t + * @param [in] agent rocprofiler agent + * @param [in] counter_id counter id (obtained from iterate_agent_supported_counters) + * @param [out] instance_count number of instances the counter has + * @return rocprofiler_status_t */ rocprofiler_status_t ROCPROFILER_API rocprofiler_query_counter_instance_count(rocprofiler_agent_t agent, @@ -72,8 +80,8 @@ rocprofiler_query_counter_instance_count(rocprofiler_agent_t agent, if(!metric_ptr) return ROCPROFILER_STATUS_ERROR_COUNTER_NOT_FOUND; *instance_count = 0; - // Special counters like KERNEL_DURATION are not real counters and wont - // have any query info. + // Special counters do not have hardware metrics and will always have an instance + // count of 1 (i.e. MAX_WAVE_SIZE) if(!metric_ptr->special().empty()) { *instance_count = 1; @@ -83,8 +91,8 @@ rocprofiler_query_counter_instance_count(rocprofiler_agent_t agent, // Returns the set of hardware counters needed to evaluate the metric. // For derived metrics, this can be more than one counter. In that case, // we return the maximum instance count among all underlying counters. - auto req_counters = - rocprofiler::counters::get_required_hardware_counters(std::string(agent.name), *metric_ptr); + auto req_counters = rocprofiler::counters::get_required_hardware_counters( + rocprofiler::counters::get_ast_map(), std::string(agent.name), *metric_ptr); if(!req_counters) return ROCPROFILER_STATUS_ERROR_COUNTER_NOT_FOUND; // NOTE: to look up instance information, we require HSA be init'd. Reason @@ -104,8 +112,19 @@ rocprofiler_query_counter_instance_count(rocprofiler_agent_t agent, *instance_count = std::max(size_t(1), *instance_count); continue; } - auto query_info = rocprofiler::aql::get_query_info(maybe_agent->get_hsa_agent(), counter); - *instance_count = std::max(static_cast(query_info.instance_count), *instance_count); + + try + { + auto dims = rocprofiler::counters::getBlockDimensions(maybe_agent->name(), counter); + for(const auto& dim : dims) + { + *instance_count = std::max(static_cast(dim.size()), *instance_count); + } + } catch(std::runtime_error& err) + { + LOG(ERROR) << fmt::format("Could not lookup instance count for counter {}", counter); + return ROCPROFILER_STATUS_ERROR_COUNTER_NOT_FOUND; + } } return ROCPROFILER_STATUS_SUCCESS; diff --git a/projects/rocprofiler-sdk/source/lib/rocprofiler/counters/core.cpp b/projects/rocprofiler-sdk/source/lib/rocprofiler/counters/core.cpp index cb93f6d2de..9bbda75398 100644 --- a/projects/rocprofiler-sdk/source/lib/rocprofiler/counters/core.cpp +++ b/projects/rocprofiler-sdk/source/lib/rocprofiler/counters/core.cpp @@ -23,8 +23,10 @@ #include "lib/rocprofiler/counters/core.hpp" #include "lib/common/synchronized.hpp" +#include "lib/rocprofiler/agent.hpp" #include "lib/rocprofiler/aql/helpers.hpp" #include "lib/rocprofiler/aql/packet_construct.hpp" +#include "lib/rocprofiler/buffer.hpp" #include "lib/rocprofiler/context/context.hpp" #include "lib/rocprofiler/hsa/queue_controller.hpp" #include "lib/rocprofiler/registration.hpp" @@ -35,139 +37,18 @@ namespace rocprofiler { namespace counters { -/** - * Callback we get from HSA interceptor when a kernel packet is being enqueued. - * - * We return an AQLPacket containing the start/stop/read packets for injection. - */ -std::unique_ptr -queue_cb(const std::shared_ptr& info, - const hsa::Queue& queue, - hsa::ClientID, - hsa::rocprofiler_packet) -{ - if(!info) return nullptr; - - std::unique_ptr ret_pkt; - - // Check packet cache - info->packets.wlock([&](auto& pkt_vector) { - // Delay packet generator construction until first HSA packet is processed - // This ensures that HSA exists - if(!info->pkt_generator) - { - // One time setup of profile config - if(info->profile_cfg.reqired_hw_counters.empty()) - { - auto& config = info->profile_cfg; - auto agent_name = std::string(config.agent.name); - for(const auto& metric : config.metrics) - { - auto req_counters = - rocprofiler::counters::get_required_hardware_counters(agent_name, metric); - if(!req_counters) - { - throw std::runtime_error( - fmt::format("Could not find counter {}", metric.name())); - } - config.reqired_hw_counters.insert(req_counters->begin(), req_counters->end()); - - const auto& asts = rocprofiler::counters::get_ast_map(); - const auto* agent_map = rocprofiler::common::get_val(asts, agent_name); - if(!agent_map) - throw std::runtime_error( - fmt::format("Coult not build AST for {}", agent_name)); - const auto* counter_ast = - rocprofiler::common::get_val(*agent_map, metric.name()); - if(!counter_ast) - { - throw std::runtime_error( - fmt::format("Coult not find AST for {}", metric.name())); - } - config.asts.push_back(*counter_ast); - } - } - - info->pkt_generator = std::make_unique( - queue.get_agent(), - std::vector{info->profile_cfg.reqired_hw_counters.begin(), - info->profile_cfg.reqired_hw_counters.end()}); - } - - if(!pkt_vector.empty()) - { - ret_pkt = std::move(pkt_vector.back()); - pkt_vector.pop_back(); - } - }); - - if(!ret_pkt) - { - // If we do not have a packet in the cache, create one. - ret_pkt = - info->pkt_generator->construct_packet(hsa::get_queue_controller().get_ext_table()); - } - return ret_pkt; -} - -/** - * Callback called by HSA interceptor when the kernel has completed processing. - */ -void -completed_cb(const std::shared_ptr& info, - const hsa::Queue& queue, - hsa::ClientID, - hsa::rocprofiler_packet kernel, - std::unique_ptr pkt) -{ - if(!info) return; - - // auto out_buf = pkt->profile.output_buffer.ptr; - // Read data and create user return.... - auto decoded_pkt = EvaluateAST::read_pkt(info->pkt_generator.get(), *pkt); - - // return AQL packet for reuse. - - info->packets.wlock([&](auto& pkt_vector) { - if(pkt) - { - pkt_vector.emplace_back(std::move(pkt)); - } - }); - - if(!info->user_cb) return; - - std::vector out; - for(auto& ast : info->profile_cfg.asts) - { - auto* ret = ast.evaluate(decoded_pkt); - CHECK(ret); - out.insert(out.end(), ret->begin(), ret->end()); - } - - // Maybe move to its own thread? - info->user_cb(queue.get_id(), - info->profile_cfg.agent, - rocprofiler_correlation_id_t{}, - &kernel.kernel_dispatch, - info->callback_args, - out.data(), // Date pointer does here. - out.size(), // Number of objects - info->profile_cfg.id); -} - class CounterController { public: // Adds a counter collection profile to our global cache. // Note: these profiles can be used across multiple contexts // and are independent of the context. - uint64_t add_profile(profile_config&& config) + uint64_t add_profile(std::shared_ptr&& config) { static std::atomic profile_val = 1; uint64_t ret = 0; _configs.wlock([&](auto& data) { - config.id = rocprofiler_profile_config_id_t{.handle = profile_val}; + config->id = rocprofiler_profile_config_id_t{.handle = profile_val}; data.emplace(profile_val, std::move(config)); ret = profile_val; profile_val++; @@ -184,17 +65,13 @@ public: // to contain the counters that need to be collected (specified in profile_id) and // the AQL packet generator for injecting packets. Note: the service is created // in the stop state. - bool configure_dispatch(rocprofiler_context_id_t context_id, - uint64_t profile_id, - rocprofiler_profile_counting_dispatch_callback_t callback, - void* callback_args) const + static bool configure_dispatch(rocprofiler_context_id_t context_id, + rocprofiler_buffer_id_t buffer, + rocprofiler_profile_counting_dispatch_callback_t callback, + void* callback_args) { auto& ctx = *rocprofiler::context::get_registered_contexts().at(context_id.handle); - // Note: A single profile config could be used on multiple contexts - profile_config cfg; - _configs.rlock([&](const auto& map) { cfg = map.at(profile_id); }); - if(!ctx.counter_collection) { ctx.counter_collection = @@ -202,19 +79,28 @@ public: } auto& cb = *ctx.counter_collection->callbacks.emplace_back( - std::make_shared()); + std::make_shared()); - cb.user_cb = callback; - - // Secondary copy of the config to be shared with async callback - cb.profile_cfg = cfg; + cb.user_cb = callback; cb.callback_args = callback_args; cb.context = context_id; + cb.buffer = buffer; + cb.internal_context = + rocprofiler::context::get_registered_contexts().at(context_id.handle).get(); + return true; } + std::shared_ptr get_profile_cfg(rocprofiler_profile_config_id_t id) + { + std::shared_ptr cfg; + _configs.rlock([&](const auto& map) { cfg = map.at(id.handle); }); + return cfg; + } + private: - rocprofiler::common::Synchronized> _configs; + rocprofiler::common::Synchronized>> + _configs; }; CounterController& @@ -225,7 +111,7 @@ get_controller() } uint64_t -create_counter_profile(profile_config&& config) +create_counter_profile(std::shared_ptr&& config) { return get_controller().add_profile(std::move(config)); } @@ -236,6 +122,190 @@ destroy_counter_profile(uint64_t id) get_controller().destroy_profile(id); } +/** + * Callback we get from HSA interceptor when a kernel packet is being enqueued. + * + * We return an AQLPacket containing the start/stop/read packets for injection. + */ +std::unique_ptr +queue_cb(const std::shared_ptr& info, + const hsa::Queue& queue, + hsa::ClientID, + const hsa::rocprofiler_packet& pkt, + const hsa::Queue::queue_info_session_t::external_corr_id_map_t& extern_corr_ids, + const context::correlation_id* correlation_id) +{ + if(!info || !info->user_cb) return nullptr; + + auto _corr_id_v = + rocprofiler_correlation_id_t{.internal = 0, .external = context::null_user_data}; + if(const auto* _corr_id = correlation_id) + { + _corr_id_v.internal = _corr_id->internal; + if(const auto* extrenal = + rocprofiler::common::get_val(extern_corr_ids, info->internal_context)) + { + _corr_id_v.external = *extrenal; + } + } + + rocprofiler_profile_config_id_t req_profile = {.handle = 0}; + info->user_cb(queue.get_id(), + queue.get_agent().get_rocp_agent(), + _corr_id_v, + &pkt.kernel_dispatch, + info->callback_args, + &req_profile); + if(req_profile.handle == 0) return nullptr; + + auto prof_config = get_controller().get_profile_cfg(req_profile); + CHECK(prof_config); + + std::unique_ptr ret_pkt; + + // Check packet cache + prof_config->packets.wlock([&](auto& pkt_vector) { + // Delay packet generator construction until first HSA packet is processed + // This ensures that HSA exists + if(!prof_config->pkt_generator) + { + // One time setup of profile config + if(prof_config->reqired_hw_counters.empty()) + { + auto& config = *prof_config; + auto agent_name = std::string(config.agent.name); + for(const auto& metric : config.metrics) + { + auto req_counters = + get_required_hardware_counters(get_ast_map(), agent_name, metric); + + if(!req_counters) + { + throw std::runtime_error( + fmt::format("Could not find counter {}", metric.name())); + } + + // Special metrics are those that are not hw counters but other + // constants like MAX_WAVE_SIZE + for(const auto& req_metric : *req_counters) + { + if(req_metric.special().empty()) + { + config.reqired_hw_counters.insert(req_metric); + } + else + { + config.required_special_counters.insert(req_metric); + } + } + + const auto& asts = get_ast_map(); + const auto* agent_map = rocprofiler::common::get_val(asts, agent_name); + if(!agent_map) + throw std::runtime_error( + fmt::format("Coult not build AST for {}", agent_name)); + const auto* counter_ast = + rocprofiler::common::get_val(*agent_map, metric.name()); + if(!counter_ast) + { + throw std::runtime_error( + fmt::format("Coult not find AST for {}", metric.name())); + } + config.asts.push_back(*counter_ast); + config.asts.back().set_dimensions(); + } + } + + prof_config->pkt_generator = std::make_unique( + queue.get_agent(), + std::vector{prof_config->reqired_hw_counters.begin(), + prof_config->reqired_hw_counters.end()}); + } + + if(!pkt_vector.empty()) + { + ret_pkt = std::move(pkt_vector.back()); + pkt_vector.pop_back(); + } + }); + + if(!ret_pkt) + { + // If we do not have a packet in the cache, create one. + ret_pkt = prof_config->pkt_generator->construct_packet( + hsa::get_queue_controller().get_ext_table()); + } + + info->packet_return_map.wlock([&](auto& data) { data.emplace(ret_pkt.get(), prof_config); }); + + return ret_pkt; +} + +/** + * Callback called by HSA interceptor when the kernel has completed processing. + */ +void +completed_cb(const std::shared_ptr& info, + const hsa::Queue&, + hsa::ClientID, + hsa::rocprofiler_packet, + const hsa::Queue::queue_info_session_t& session, + std::unique_ptr pkt) +{ + if(!info || !pkt) return; + + std::shared_ptr prof_config; + // Get the Profile Config + info->packet_return_map.wlock([&](auto& data) { + prof_config = data.at(pkt.get()); + data.erase(pkt.get()); + }); + + auto decoded_pkt = EvaluateAST::read_pkt(prof_config->pkt_generator.get(), *pkt); + EvaluateAST::read_special_counters( + prof_config->agent, prof_config->required_special_counters, decoded_pkt); + + prof_config->packets.wlock([&](auto& pkt_vector) { + if(pkt) + { + pkt_vector.emplace_back(std::move(pkt)); + } + }); + + if(!info->buffer) return; + + std::vector out; + rocprofiler::buffer::instance* buf = nullptr; + + buf = CHECK_NOTNULL(buffer::get_buffer(info->buffer->handle)); + + auto _corr_id_v = + rocprofiler_correlation_id_t{.internal = 0, .external = context::null_user_data}; + if(const auto* _corr_id = session.correlation_id) + { + _corr_id_v.internal = _corr_id->internal; + if(const auto* extrenal = + rocprofiler::common::get_val(session.extern_corr_ids, info->internal_context)) + { + _corr_id_v.external = *extrenal; + } + } + + for(auto& ast : prof_config->asts) + { + std::vector>> cache; + auto* ret = ast.evaluate(decoded_pkt, cache); + CHECK(ret); + ast.set_out_id(*ret); + + for(auto& val : *ret) + { + val.corr_id = _corr_id_v; + buf->emplace(ROCPROFILER_BUFFER_CATEGORY_COUNTERS, 0, val); + } + } +} + void start_context(context::context* ctx) { @@ -251,16 +321,21 @@ start_context(context::context* ctx) // Insert our callbacks into HSA Interceptor. This // turns on counter instrumentation. cb->queue_id = controller.add_callback( - cb->profile_cfg.agent, - [=](const hsa::Queue& q, hsa::ClientID c, hsa::rocprofiler_packet kern_pkt) { - return queue_cb(cb, q, c, kern_pkt); + std::nullopt, + [=](const hsa::Queue& q, + hsa::ClientID c, + const hsa::rocprofiler_packet& kern_pkt, + const hsa::Queue::queue_info_session_t::external_corr_id_map_t& extern_corr_ids, + const context::correlation_id* correlation_id) { + return queue_cb(cb, q, c, kern_pkt, extern_corr_ids, correlation_id); }, // Completion CB - [=](const hsa::Queue& q, - hsa::ClientID c, - hsa::rocprofiler_packet kern_pkt, - std::unique_ptr aql) { - completed_cb(cb, q, c, kern_pkt, std::move(aql)); + [=](const hsa::Queue& q, + hsa::ClientID c, + hsa::rocprofiler_packet kern_pkt, + const hsa::Queue::queue_info_session_t& session, + std::unique_ptr aql) { + completed_cb(cb, q, c, kern_pkt, session, std::move(aql)); }); } enabled = true; @@ -287,12 +362,12 @@ stop_context(context::context* ctx) } bool -configure_dispatch(rocprofiler_context_id_t context_id, - uint64_t profile_id, - rocprofiler_profile_counting_dispatch_callback_t callback, - void* callback_args) +configure_buffered_dispatch(rocprofiler_context_id_t context_id, + rocprofiler_buffer_id_t buffer, + rocprofiler_profile_counting_dispatch_callback_t callback, + void* callback_args) { - return get_controller().configure_dispatch(context_id, profile_id, callback, callback_args); + return get_controller().configure_dispatch(context_id, buffer, callback, callback_args); } } // namespace counters diff --git a/projects/rocprofiler-sdk/source/lib/rocprofiler/counters/core.hpp b/projects/rocprofiler-sdk/source/lib/rocprofiler/counters/core.hpp index 5ff53dfe53..64f022a388 100644 --- a/projects/rocprofiler-sdk/source/lib/rocprofiler/counters/core.hpp +++ b/projects/rocprofiler-sdk/source/lib/rocprofiler/counters/core.hpp @@ -51,46 +51,56 @@ struct profile_config // metrics (derived metrics are broken down into hw counters // in this vector). std::set reqired_hw_counters{}; + // Counters that are not hardware based but based on either a + // static value (such as those in agent) + std::set required_special_counters{}; // ASTs to evaluate std::vector asts{}; rocprofiler_profile_config_id_t id{.handle = 0}; -}; - -// Internal counter struct that stores the state needed to handle an intercepted -// HSA kernel packet. -struct counter_callback_info -{ // Packet generator to create AQL packets for insertion std::unique_ptr pkt_generator{nullptr}; // A packet cache of AQL packets. This allows reuse of AQL packets (preventing costly // allocation of new packets/destruction). rocprofiler::common::Synchronized>> packets{}; +}; + +// Internal counter struct that stores the state needed to handle an intercepted +// HSA kernel packet. +struct counter_callback_info +{ // User callback rocprofiler_profile_counting_dispatch_callback_t user_cb{nullptr}; - // Profile configuration used for this callback containing the counters - // to collect and the evaluation ASTs - profile_config profile_cfg{}; // User id void* callback_args{nullptr}; // Link to the context this is associated with rocprofiler_context_id_t context{.handle = 0}; + // Link to the internal context this is associated with + const context::context* internal_context{nullptr}; // HSA Queue ClientID. This is an ID we get when we insert a callback into the // HSA queue interceptor. This ID can be used to disable the callback. rocprofiler::hsa::ClientID queue_id{-1}; + + // Buffer to use for storing counter data. Used if callback is not set. + std::optional buffer; + + // Facilitates the return of an AQL Packet to the profile config that constructed it. + rocprofiler::common::Synchronized< + std::unordered_map>> + packet_return_map{}; }; uint64_t -create_counter_profile(profile_config&& config); +create_counter_profile(std::shared_ptr&& config); void destroy_counter_profile(uint64_t id); bool -configure_dispatch(rocprofiler_context_id_t context_id, - uint64_t profile_id, - rocprofiler_profile_counting_dispatch_callback_t callback, - void* callback_args); +configure_buffered_dispatch(rocprofiler_context_id_t context_id, + rocprofiler_buffer_id_t buffer, + rocprofiler_profile_counting_dispatch_callback_t callback, + void* callback_args); void start_context(context::context*); diff --git a/projects/rocprofiler-sdk/source/lib/rocprofiler/counters/dimensions.cpp b/projects/rocprofiler-sdk/source/lib/rocprofiler/counters/dimensions.cpp index e74b9b5ec7..b2dd9e2949 100644 --- a/projects/rocprofiler-sdk/source/lib/rocprofiler/counters/dimensions.cpp +++ b/projects/rocprofiler-sdk/source/lib/rocprofiler/counters/dimensions.cpp @@ -32,66 +32,15 @@ #include "lib/common/synchronized.hpp" #include "lib/common/utility.hpp" #include "lib/rocprofiler/aql/helpers.hpp" +#include "lib/rocprofiler/aql/packet_construct.hpp" #include "lib/rocprofiler/hsa/queue_controller.hpp" namespace rocprofiler { namespace counters { -// namespace -// { -// void -// create_block_dimensions(std::string& agent, -// std::string& block_name, -// std::vector& dimension_list) -// { -// static std::atomic id = 0; -// // query hsa/aqlprofile/kfd etc here to get dimension sizes -// // create MetricDimension objects and push_back() in dimension_list -// } - -// } // namespace - -// BlockDimensionMap& -// getBlockDimensionsMap(std::string& agent) -// { -// static std::unique_ptr map = [&]() { -// auto data = std::make_unique(); -// // TODO: populate this vector with list of all blocks -// std::vector block_names; - -// for(auto& block : block_names) -// { -// auto& dimensions = data->emplace(block, -// std::vector()).first->second; create_block_dimensions(agent, block, -// dimensions); -// } -// return data; -// }(); -// return *map; -// } - -// const AgentBlockDimensionsMap& -// getAgentBlockDimensionsMap() -// { -// static std::unique_ptr map = [&]() { -// auto data = std::make_unique(); -// // TODO: fill this up with agent iteration or through xml -// std::vector agent_names; - -// // insert the BlockDimensionMap for each agent -// for(auto& agent : agent_names) -// { -// auto& val = getBlockDimensionsMap(agent); -// data->emplace(agent, val); -// } -// return data; -// }(); -// return *map; -// } - std::vector -getBlockDimensions(const std::string& agent, const Metric& metric) +getBlockDimensions(std::string_view agent, const Metric& metric) { if(!metric.special().empty()) { @@ -104,23 +53,23 @@ getBlockDimensions(const std::string& agent, const Metric& metric) { if(maybe_agent.name() == agent) { - return std::vector{ - {dimension_map().at(ROCPROFILER_DIMENSION_SHADER_ENGINE), - maybe_agent.get_rocp_agent()->num_shader_banks, - ROCPROFILER_DIMENSION_SHADER_ENGINE}, - {dimension_map().at(ROCPROFILER_DIMENSION_XCC), - maybe_agent.get_rocp_agent()->num_xcc, - ROCPROFILER_DIMENSION_XCC}, - {dimension_map().at(ROCPROFILER_DIMENSION_CU), - maybe_agent.get_rocp_agent()->cu_count, - ROCPROFILER_DIMENSION_CU}, - {dimension_map().at(ROCPROFILER_DIMENSION_AGENT), - maybe_agent.get_rocp_agent()->id.handle, - ROCPROFILER_DIMENSION_AGENT}}; - - // auto query_info = aql::get_query_info(maybe_agent.get_agent(), metric); + // To be returned when instance counting is functional with AQL profiler // return std::vector{ - // {metric.block(), query_info.instance_count, ROCPROFILER_DIMENSION_NONE}}; + // {dimension_map().at(ROCPROFILER_DIMENSION_SHADER_ENGINE), + // maybe_agent.get_rocp_agent()->num_shader_banks, + // ROCPROFILER_DIMENSION_SHADER_ENGINE}, + // {dimension_map().at(ROCPROFILER_DIMENSION_XCC), + // maybe_agent.get_rocp_agent()->num_xcc, + // ROCPROFILER_DIMENSION_XCC}, + // {dimension_map().at(ROCPROFILER_DIMENSION_CU), + // maybe_agent.get_rocp_agent()->cu_count, + // ROCPROFILER_DIMENSION_CU}, + // {dimension_map().at(ROCPROFILER_DIMENSION_AGENT), + // maybe_agent.get_rocp_agent()->id.handle, + // ROCPROFILER_DIMENSION_AGENT}}; + aql::AQLPacketConstruct pkt_gen(maybe_agent, {metric}); + return std::vector{ + {metric.block(), pkt_gen.get_all_events().size(), ROCPROFILER_DIMENSION_NONE}}; } } diff --git a/projects/rocprofiler-sdk/source/lib/rocprofiler/counters/dimensions.hpp b/projects/rocprofiler-sdk/source/lib/rocprofiler/counters/dimensions.hpp index 4483f9f46a..b49a04788d 100644 --- a/projects/rocprofiler-sdk/source/lib/rocprofiler/counters/dimensions.hpp +++ b/projects/rocprofiler-sdk/source/lib/rocprofiler/counters/dimensions.hpp @@ -93,7 +93,7 @@ private: // get all dimensions for an agent, block_name std::vector -getBlockDimensions(const std::string& agent, const counters::Metric&); +getBlockDimensions(std::string_view agent, const counters::Metric&); // // get a specific dimension by id // const MetricDimension& diff --git a/projects/rocprofiler-sdk/source/lib/rocprofiler/counters/evaluate_ast.cpp b/projects/rocprofiler-sdk/source/lib/rocprofiler/counters/evaluate_ast.cpp index f2374b62c6..3df3753710 100644 --- a/projects/rocprofiler-sdk/source/lib/rocprofiler/counters/evaluate_ast.cpp +++ b/projects/rocprofiler-sdk/source/lib/rocprofiler/counters/evaluate_ast.cpp @@ -45,8 +45,9 @@ get_reduce_op_type_from_string(const std::string& op) static const std::unordered_map reduce_op_string_to_type = { {"min", REDUCE_MIN}, {"max", REDUCE_MAX}, {"sum", REDUCE_SUM}, {"avr", REDUCE_AVG}}; - ReduceOperation type = REDUCE_NONE; - const auto* reduce_op_type = rocprofiler::common::get_val(reduce_op_string_to_type, op); + ReduceOperation type = REDUCE_NONE; + if(op.empty()) return REDUCE_NONE; + const auto* reduce_op_type = rocprofiler::common::get_val(reduce_op_string_to_type, op); if(reduce_op_type) type = *reduce_op_type; return type; } @@ -54,7 +55,7 @@ get_reduce_op_type_from_string(const std::string& op) std::vector* perform_reduction(ReduceOperation reduce_op, std::vector* input_array) { - rocprofiler_record_counter_t result{.id = 0, .derived_counter = 0}; + rocprofiler_record_counter_t result{.id = 0, .counter_value = 0}; if(input_array->empty()) return input_array; switch(reduce_op) { @@ -63,7 +64,7 @@ perform_reduction(ReduceOperation reduce_op, std::vectorbegin(), input_array->end(), [](auto& a, auto& b) { - return a.derived_counter < b.derived_counter; + return a.counter_value < b.counter_value; }); break; } @@ -71,33 +72,33 @@ perform_reduction(ReduceOperation reduce_op, std::vectorbegin(), input_array->end(), [](auto& a, auto& b) { - return a.derived_counter > b.derived_counter; + return a.counter_value > b.counter_value; }); break; } case REDUCE_SUM: { - result = std::accumulate( - input_array->begin(), - input_array->end(), - rocprofiler_record_counter_t{.id = 0, .derived_counter = 0}, - [](auto& a, auto& b) { - return rocprofiler_record_counter_t{ - .id = a.id, .derived_counter = a.derived_counter + b.derived_counter}; - }); + result = std::accumulate(input_array->begin(), + input_array->end(), + rocprofiler_record_counter_t{.id = 0, .counter_value = 0}, + [](auto& a, auto& b) { + return rocprofiler_record_counter_t{ + .id = a.id, + .counter_value = a.counter_value + b.counter_value}; + }); break; } case REDUCE_AVG: { - result = std::accumulate( - input_array->begin(), - input_array->end(), - rocprofiler_record_counter_t{.id = 0, .derived_counter = 0}, - [](auto& a, auto& b) { - return rocprofiler_record_counter_t{ - .id = a.id, .derived_counter = a.derived_counter + b.derived_counter}; - }); - result.derived_counter /= input_array->size(); + result = std::accumulate(input_array->begin(), + input_array->end(), + rocprofiler_record_counter_t{.id = 0, .counter_value = 0}, + [](auto& a, auto& b) { + return rocprofiler_record_counter_t{ + .id = a.id, + .counter_value = a.counter_value + b.counter_value}; + }); + result.counter_value /= input_array->size(); break; } } @@ -142,7 +143,9 @@ get_ast_map() try { auto& evaluate_ast_node = - eval_map.emplace(metric.name(), EvaluateAST(by_name, *ast, gfx)) + eval_map + .emplace(metric.name(), + EvaluateAST({.handle = metric.id()}, by_name, *ast, gfx)) .first->second; evaluate_ast_node.validate_raw_ast( by_name); // TODO: refactor and consolidate internal post-construction @@ -156,17 +159,10 @@ get_ast_map() yy_delete_buffer(buf); delete ast; } - // Set dimensions after all ASTs loaded for arch. + for(auto& [name, ast] : eval_map) { - try - { - ast.set_dimensions(); - } catch(std::exception& e) - { - LOG(ERROR) << "Could not set dimensions for " << name << " failed with " - << e.what(); - } + ast.expand_derived(eval_map); } } @@ -176,9 +172,10 @@ get_ast_map() } std::optional> -get_required_hardware_counters(const std::string& agent, const Metric& metric) +get_required_hardware_counters(const std::unordered_map& asts, + const std::string& agent, + const Metric& metric) { - const auto& asts = get_ast_map(); const auto* agent_map = rocprofiler::common::get_val(asts, agent); if(!agent_map) return std::nullopt; const auto* counter_ast = rocprofiler::common::get_val(*agent_map, metric.name()); @@ -189,13 +186,15 @@ get_required_hardware_counters(const std::string& agent, const Metric& metric) return required_counters; } -EvaluateAST::EvaluateAST(const std::unordered_map& metrics, +EvaluateAST::EvaluateAST(rocprofiler_counter_id_t out_id, + const std::unordered_map& metrics, const RawAST& ast, std::string agent) : _type(ast.type) , _reduce_op(get_reduce_op_type_from_string(ast.reduce_op)) , _agent(std::move(agent)) , _reduce_dimension_set(ast.reduce_dimension_set) +, _out_id(out_id) { if(_type == NodeType::REFERENCE_NODE) { @@ -212,12 +211,13 @@ EvaluateAST::EvaluateAST(const std::unordered_map& metrics, if(_type == NodeType::NUMBER_NODE) { _raw_value = std::get(ast.value); - _static_value.push_back({.id = 0, .hw_counter = std::get(ast.value)}); + _static_value.push_back( + {.id = 0, .counter_value = static_cast(std::get(ast.value))}); } for(const auto& nextAst : ast.counter_set) { - _children.emplace_back(metrics, *nextAst, _agent); + _children.emplace_back(_out_id, metrics, *nextAst, _agent); } } @@ -242,6 +242,7 @@ EvaluateAST::set_dimensions() { case NONE: case RANGE_NODE: + case CONSTANT_NODE: case NUMBER_NODE: break; case ADDITION_NODE: case SUBTRACTION_NODE: @@ -326,6 +327,7 @@ EvaluateAST::validate_raw_ast(const std::unordered_map& met { case NONE: case RANGE_NODE: + case CONSTANT_NODE: case NUMBER_NODE: break; case ADDITION_NODE: case SUBTRACTION_NODE: @@ -376,6 +378,77 @@ EvaluateAST::validate_raw_ast(const std::unordered_map& met return ret; } +namespace +{ +using property_function_t = int64_t (*)(const rocprofiler_agent_t&); +#define GEN_MAP_ENTRY(name, value) \ + { \ + name, property_function_t([](const rocprofiler_agent_t& agent_info) { \ + return static_cast(value); \ + }) \ + } + +int64_t +get_agent_property(const std::string& property, const rocprofiler_agent_t& agent) +{ + static std::unordered_map props = { + GEN_MAP_ENTRY("cpu_cores_count", agent_info.cpu_cores_count), + GEN_MAP_ENTRY("simd_count", agent_info.simd_count), + GEN_MAP_ENTRY("mem_banks_count", agent_info.mem_banks_count), + GEN_MAP_ENTRY("caches_count", agent_info.caches_count), + GEN_MAP_ENTRY("io_links_count", agent_info.io_links_count), + GEN_MAP_ENTRY("cpu_core_id_base", agent_info.cpu_core_id_base), + GEN_MAP_ENTRY("simd_id_base", agent_info.simd_id_base), + GEN_MAP_ENTRY("max_waves_per_simd", agent_info.max_waves_per_simd), + GEN_MAP_ENTRY("lds_size_in_kb", agent_info.lds_size_in_kb), + GEN_MAP_ENTRY("gds_size_in_kb", agent_info.gds_size_in_kb), + GEN_MAP_ENTRY("num_gws", agent_info.num_gws), + GEN_MAP_ENTRY("wave_front_size", agent_info.wave_front_size), + GEN_MAP_ENTRY("array_count", agent_info.array_count), + GEN_MAP_ENTRY("simd_arrays_per_engine", agent_info.simd_arrays_per_engine), + GEN_MAP_ENTRY("cu_per_simd_array", agent_info.cu_per_simd_array), + GEN_MAP_ENTRY("simd_per_cu", agent_info.simd_per_cu), + GEN_MAP_ENTRY("max_slots_scratch_cu", agent_info.max_slots_scratch_cu), + GEN_MAP_ENTRY("gfx_target_version", agent_info.gfx_target_version), + GEN_MAP_ENTRY("vendor_id", agent_info.vendor_id), + GEN_MAP_ENTRY("device_id", agent_info.device_id), + GEN_MAP_ENTRY("location_id", agent_info.location_id), + GEN_MAP_ENTRY("domain", agent_info.domain), + GEN_MAP_ENTRY("drm_render_minor", agent_info.drm_render_minor), + GEN_MAP_ENTRY("hive_id", agent_info.hive_id), + GEN_MAP_ENTRY("num_sdma_engines", agent_info.num_sdma_engines), + GEN_MAP_ENTRY("num_sdma_xgmi_engines", agent_info.num_sdma_xgmi_engines), + GEN_MAP_ENTRY("num_sdma_queues_per_engine", agent_info.num_sdma_queues_per_engine), + GEN_MAP_ENTRY("num_cp_queues", agent_info.num_cp_queues), + GEN_MAP_ENTRY("max_engine_clk_ccompute", agent_info.max_engine_clk_ccompute), + }; + if(const auto* func = rocprofiler::common::get_val(props, property)) + { + return (*func)(agent); + } + + LOG(ERROR) << fmt::format("Unsupported special property {}", property); + return 0.0; +} +} // namespace + +void +EvaluateAST::read_special_counters( + const rocprofiler_agent_t& agent, + const std::set& required_special_counters, + std::unordered_map>& out_map) +{ + for(const auto& metric : required_special_counters) + { + if(!out_map[metric.id()].empty()) out_map[metric.id()].clear(); + auto& record = out_map[metric.id()].emplace_back(); + set_counter_in_rec(record.id, {.handle = metric.id()}); + set_dim_in_rec(record.id, ROCPROFILER_DIMENSION_NONE, 0); + + record.counter_value = get_agent_property(metric.name(), agent); + } +} + std::unordered_map> EvaluateAST::read_pkt(const aql::AQLPacketConstruct* pkt_gen, hsa::AQLPacket& pkt) { @@ -386,8 +459,9 @@ EvaluateAST::read_pkt(const aql::AQLPacketConstruct* pkt_gen, hsa::AQLPacket& pk }; std::unordered_map> ret; + if(pkt.empty) return ret; it_data aql_data{.data = &ret, .pkt_gen = pkt_gen}; - + ; hsa_status_t status = hsa_ven_amd_aqlprofile_iterate_data( &pkt.profile, [](hsa_ven_amd_aqlprofile_info_type_t info_type, @@ -405,7 +479,7 @@ EvaluateAST::read_pkt(const aql::AQLPacketConstruct* pkt_gen, hsa::AQLPacket& pk // Actual dimension info needs to be used here in the future set_dim_in_rec(next_rec.id, ROCPROFILER_DIMENSION_NONE, vec.size() - 1); // Note: in the near future we need to use hw_counter here instead - next_rec.derived_counter = info_data->pmc_data.result; + next_rec.counter_value = info_data->pmc_data.result; return HSA_STATUS_SUCCESS; }, &aql_data); @@ -413,17 +487,66 @@ EvaluateAST::read_pkt(const aql::AQLPacketConstruct* pkt_gen, hsa::AQLPacket& pk return ret; } +void +EvaluateAST::set_out_id(std::vector& results) const +{ + for(auto& record : results) + { + set_counter_in_rec(record.id, _out_id); + } +} + +void +EvaluateAST::expand_derived(std::unordered_map& asts) +{ + if(_expanded) return; + _expanded = true; + for(auto& child : _children) + { + if(auto* ptr = rocprofiler::common::get_val(asts, child.metric().name())) + { + ptr->expand_derived(asts); + child = *ptr; + } + else + { + child.expand_derived(asts); + } + } + + /** + * This covers cases where a derived metric is not a child at all. I.e. + * . This will expand + * WRITE_REQ_32B to its proper expression. + */ + if(!_metric.expression().empty()) + { + if(auto* ptr = rocprofiler::common::get_val(asts, _metric.name())) + { + ptr->expand_derived(asts); + _children = ptr->children(); + _type = ptr->type(); + _reduce_op = ptr->reduce_op(); + } + } +} + // convert to buffer at some point std::vector* EvaluateAST::evaluate( - std::unordered_map>& results_map) + std::unordered_map>& results_map, + std::vector>>& cache) { auto perform_op = [&](auto&& op) { - auto* r1 = _children[0].evaluate(results_map); - auto* r2 = _children[1].evaluate(results_map); + auto* r1 = _children.at(0).evaluate(results_map, cache); + auto* r2 = _children.at(1).evaluate(results_map, cache); if(r1->size() < r2->size()) swap(r1, r2); + cache.emplace_back(std::make_unique>()); + *cache.back() = *r1; + r1 = cache.back().get(); + CHECK(!r1->empty() && !r2->empty()); if(r2->size() == 1) @@ -452,29 +575,30 @@ EvaluateAST::evaluate( switch(_type) { case NONE: + case CONSTANT_NODE: case RANGE_NODE: break; case NUMBER_NODE: return &_static_value; case ADDITION_NODE: return perform_op([](auto& a, auto& b) { return rocprofiler_record_counter_t{ - .id = a.id, .derived_counter = a.derived_counter + b.derived_counter}; + .id = a.id, .counter_value = a.counter_value + b.counter_value}; }); case SUBTRACTION_NODE: return perform_op([](auto& a, auto& b) { return rocprofiler_record_counter_t{ - .id = a.id, .derived_counter = a.derived_counter - b.derived_counter}; + .id = a.id, .counter_value = a.counter_value - b.counter_value}; }); case MULTIPLY_NODE: return perform_op([](auto& a, auto& b) { return rocprofiler_record_counter_t{ - .id = a.id, .derived_counter = a.derived_counter * b.derived_counter}; + .id = a.id, .counter_value = a.counter_value * b.counter_value}; }); case DIVIDE_NODE: return perform_op([](auto& a, auto& b) { return rocprofiler_record_counter_t{ .id = a.id, - .derived_counter = - (b.derived_counter == 0 ? 0 : a.derived_counter / b.derived_counter)}; + .counter_value = + (b.counter_value == 0 ? 0 : a.counter_value / b.counter_value)}; }); case REFERENCE_NODE: { diff --git a/projects/rocprofiler-sdk/source/lib/rocprofiler/counters/evaluate_ast.hpp b/projects/rocprofiler-sdk/source/lib/rocprofiler/counters/evaluate_ast.hpp index ee7ece4c8f..4a25809986 100644 --- a/projects/rocprofiler-sdk/source/lib/rocprofiler/counters/evaluate_ast.hpp +++ b/projects/rocprofiler-sdk/source/lib/rocprofiler/counters/evaluate_ast.hpp @@ -66,30 +66,106 @@ enum ReduceOperation class EvaluateAST { public: - EvaluateAST(const std::unordered_map& metrics, + EvaluateAST(rocprofiler_counter_id_t out_id, + const std::unordered_map& metrics, const RawAST& ast, std::string agent); + /** + * @brief Evaluates the AST, returning a pointer to the location where the output + * results are stored. The output results will reuse a vector contained in + * result map (and the map should be treated as tainted after this call). + * For simple base counters, evaluate performs no copy operations. For + * derived counters, the number of data copies is contingent on the complexity + * of the counter. + * + * @param [in] results_map Results decoded from the AQL packet + * @param [in] cache Used to store results generated from derived counter + * computations. This is needed to avoid destroying data + * in the result map that may be used by other evaluate calls. + * + * @return std::vector* A pointer to the output records. + * This pointer SHOULD NOT BE FREE'D/DELETED BY THE CALLER. + */ std::vector* evaluate( - std::unordered_map>& results_map); + std::unordered_map>& results_map, + std::vector>>& cache); + /** + * @brief Expand derived counter ASTs contained within this AST to full hardware counter + * representations. + * + * @param [in] asts all ASTs read for this agent. + */ + void expand_derived(std::unordered_map& asts); + + /** + * @brief Set the dimensions for this AST and its sub-nodes. Returns the dimension + * of this AST. Can throw if the AST is invalid (i.e. dimension mismatch in + * child nodes of this AST). This is done in a recursive fashion. + * + * @return DimensionTypes dimension of the output of this AST. + */ DimensionTypes set_dimensions(); bool validate_raw_ast(const std::unordered_map& metrics); + /** + * @brief Get the base hardware counters required to evaluate the expressions in the + * AST structure. This is primarily useful if the AST contains derived metrics + * which will be converted into the base metrics needed to evaluate the derived. + * + * @param [in] asts All constructed ASTs returned by get_ast_map() + * @param [out] counters Base metrics that need to be collected to evaluate this AST + */ void get_required_counters(const std::unordered_map& asts, std::set& counters) const; + /** + * @brief Read the AQL packet and construct rocprofiler_record_counter_t. This call + * does not perform any evaluation, only dumping the packet contents into + * rocprofiler_record_counter_t. + * + * @param [in] pkt_gen packet generator used to generate the AQL packet. This packet + * generator contains information, such as the ordering of instances + * contained in the return packet, that is required to decode what + * data goes with what base counters. + * @param [in] pkt AQL packet structure to decode + * @return std::unordered_map> map of + * {metric->id(), vector} + * + */ static std::unordered_map> read_pkt( const aql::AQLPacketConstruct* pkt_gen, hsa::AQLPacket& pkt); + /** + * @brief Insert special counter values, such as constants of the agent (i.e. max waves) + * and kernel duration into the output map. + * + * @param [in] agent Agent of the output + * @param [in] required_special_counters Special counters that are required for eval + * @param [out] out_map Where the special counter values will be written + */ + static void read_special_counters( + const rocprofiler_agent_t& agent, + const std::set& required_special_counters, + std::unordered_map>& out_map); + NodeType type() const { return _type; } ReduceOperation reduce_op() const { return _reduce_op; } const std::vector& children() const { return _children; } const Metric& metric() const { return _metric; } DimensionTypes dimension_types() const { return _dimension_types; } + /** + * @brief When an evaluation is complete, set the output id of the results. This is called + * externally to reduce the number of times the id is set to only the end result. + * + * @param [in] results computed results that will have their id modified to be counter _out_id + */ + void set_out_id(std::vector& results) const; + private: NodeType _type{NONE}; ReduceOperation _reduce_op{REDUCE_NONE}; @@ -100,6 +176,8 @@ private: DimensionTypes _dimension_types{DIMENSION_LAST}; std::vector _static_value; std::unordered_set _reduce_dimension_set; + bool _expanded{false}; + rocprofiler_counter_id_t _out_id{.handle = 0}; }; using EvaluateASTMap = std::unordered_map; @@ -116,7 +194,9 @@ get_ast_map(); * specific metric (may be multiple HW counters if a derrived metric). */ std::optional> -get_required_hardware_counters(const std::string& agent, const Metric& metric); +get_required_hardware_counters(const std::unordered_map& asts, + const std::string& agent, + const Metric& metric); } // namespace counters } // namespace rocprofiler @@ -135,12 +215,11 @@ struct formatter auto format(rocprofiler_record_counter_t const& data, Ctx& ctx) const { return fmt::format_to(ctx.out(), - "(CounterId: {}, Dimension: {:x}, Value [D]: {}, Value [I]: {})", + "(CounterId: {}, Dimension: {:x}, Value [D]: {})", rocprofiler::counters::rec_to_counter_id(data.id).handle, rocprofiler::counters::rec_to_dim_pos( data.id, rocprofiler::counters::ROCPROFILER_DIMENSION_NONE), - data.derived_counter, - data.hw_counter); + data.counter_value); } }; } // namespace fmt diff --git a/projects/rocprofiler-sdk/source/lib/rocprofiler/counters/metrics.cpp b/projects/rocprofiler-sdk/source/lib/rocprofiler/counters/metrics.cpp index dfe4823792..aa4412b5f6 100644 --- a/projects/rocprofiler-sdk/source/lib/rocprofiler/counters/metrics.cpp +++ b/projects/rocprofiler-sdk/source/lib/rocprofiler/counters/metrics.cpp @@ -27,6 +27,7 @@ #include "lib/common/synchronized.hpp" #include "lib/common/utility.hpp" #include "lib/common/xml.hpp" +#include "lib/rocprofiler/agent.hpp" #include "dimensions.hpp" #include "glog/logging.h" @@ -44,18 +45,56 @@ namespace counters { namespace { +uint64_t& +current_id() +{ + static uint64_t id = 0; + return id; +} + +/** + * Constant/speical metrics are treated as psudo-metrics in that they + * are given their own metric id. MAX_WAVE_SIZE for example is not collected + * by AQL Profiler but is a constant from the topology. It will still have + * a counter associated with it. Nearly all metrics contained in + * rocprofiler_agent_t will have a counter id associated with it and can be + * used in derived counters (exact support properties that can be used can + * be viewed in evaluate_ast.cpp:get_agent_property()). + */ +const std::vector& +get_constants() +{ + static std::vector constants; + if(!constants.empty()) return constants; + // Ensure topology is read + rocprofiler::agent::get_agents(); + for(const auto& prop : rocprofiler::agent::get_agent_available_properties()) + { + constants.emplace_back("constant", + prop, + "", + "", + fmt::format("Constant value {} from agent properties", prop), + "", + "yes", + current_id()); + current_id()++; + } + return constants; +} + // Future TODO: inheritance? does it work for derived_counters.xml? MetricMap -loadXml(const std::string& filename) +loadXml(const std::string& filename, bool load_constants = false) { - static std::atomic id = 0; - MetricMap ret; + MetricMap ret; DLOG(INFO) << "Loading Counter Config: " << filename; // todo: return unique_ptr.... auto xml = common::Xml::Create(filename); LOG_IF(FATAL, !xml) << "Could not open XML Counter Config File (set env ROCPROFILER_METRICS_PATH)"; + const auto& constant_metrics = get_constants(); for(const auto& [gfx_name, nodes] : xml->GetAllNodes()) { /** @@ -82,11 +121,18 @@ loadXml(const std::string& filename) node->opts["descr"], node->opts["expr"], node->opts["special"], - id); - id++; + current_id()); + current_id()++; + } + + if(load_constants) + { + metricVec.insert(metricVec.end(), constant_metrics.begin(), constant_metrics.end()); } } + LOG_IF(FATAL, current_id() > 65536) + << "Counter count exceeds 16 bits, which may break counter id output"; return ret; } @@ -126,7 +172,7 @@ getDerivedHardwareMetrics() MetricMap getBaseHardwareMetrics() { - return loadXml(findViaEnvironment("basic_counters.xml")); + return loadXml(findViaEnvironment("basic_counters.xml"), true); } const MetricIdMap& diff --git a/projects/rocprofiler-sdk/source/lib/rocprofiler/counters/parser/raw_ast.hpp b/projects/rocprofiler-sdk/source/lib/rocprofiler/counters/parser/raw_ast.hpp index b234bc7f85..2ce87581f0 100644 --- a/projects/rocprofiler-sdk/source/lib/rocprofiler/counters/parser/raw_ast.hpp +++ b/projects/rocprofiler-sdk/source/lib/rocprofiler/counters/parser/raw_ast.hpp @@ -54,6 +54,7 @@ enum NodeType REFERENCE_NODE, SELECT_NODE, SUBTRACTION_NODE, + CONSTANT_NODE, }; struct LinkedList @@ -76,7 +77,7 @@ struct RawAST { // Node type NodeType type{NONE}; // Operation to perform on the counter set - std::string reduce_op; + std::string reduce_op{}; // Stores either the name or digit dependening on whether this // is a name or number @@ -143,6 +144,7 @@ struct RawAST , reduce_op(CHECK_NOTNULL(op)) , counter_set({counter}) { + CHECK_EQ(t, REDUCE_NODE); if(dimensions) { while(dimensions) diff --git a/projects/rocprofiler-sdk/source/lib/rocprofiler/counters/parser/tests/CMakeLists.txt b/projects/rocprofiler-sdk/source/lib/rocprofiler/counters/parser/tests/CMakeLists.txt index 82b423ae5f..a1c256772b 100644 --- a/projects/rocprofiler-sdk/source/lib/rocprofiler/counters/parser/tests/CMakeLists.txt +++ b/projects/rocprofiler-sdk/source/lib/rocprofiler/counters/parser/tests/CMakeLists.txt @@ -1,6 +1,3 @@ -# -# -# rocprofiler_deactivate_clang_tidy() include(GoogleTest) @@ -14,7 +11,7 @@ target_sources(parser-test PRIVATE ${ROCPROFILER_LIB_PARSER_TEST_SOURCES}) target_link_libraries( parser-test PRIVATE rocprofiler::rocprofiler-common-library - rocprofiler::rocprofiler-static-library GTest::gtest GTest::gtest_main) + rocprofiler::rocprofiler-object-library GTest::gtest GTest::gtest_main) gtest_add_tests( TARGET parser-test diff --git a/projects/rocprofiler-sdk/source/lib/rocprofiler/counters/parser/tests/parser_test.cpp b/projects/rocprofiler-sdk/source/lib/rocprofiler/counters/parser/tests/parser_test.cpp index 06357f7b42..6c889d7b1a 100644 --- a/projects/rocprofiler-sdk/source/lib/rocprofiler/counters/parser/tests/parser_test.cpp +++ b/projects/rocprofiler-sdk/source/lib/rocprofiler/counters/parser/tests/parser_test.cpp @@ -240,3 +240,21 @@ TEST(parser, parse_derived_counters) } } } + +// TEST(parser, parse_complex_counters) +// { +// std::map expressionToExpected = { +// {"(TCC_EA_WRREQ_sum-TCC_EA_WRREQ_64B_sum)+(TCC_EA1_WRREQ_sum-TCC_EA1_WRREQ_64B_sum)+(TCC_EA_WRREQ_64B_sum+TCC_EA1_WRREQ_64B_sum)*2",""} +// }; + +// for(auto [op, expected] : expressionToExpected) +// { +// RawAST* ast = nullptr; +// auto* buf = yy_scan_string(op.c_str()); +// yyparse(&ast); +// ASSERT_TRUE(ast); +// EXPECT_EQ(fmt::format("{}", *ast), expected); +// yy_delete_buffer(buf); +// delete ast; +// } +// } diff --git a/projects/rocprofiler-sdk/source/lib/rocprofiler/counters/tests/evaluate_ast_test.cpp b/projects/rocprofiler-sdk/source/lib/rocprofiler/counters/tests/evaluate_ast_test.cpp index d2371858ed..7758f8f01e 100644 --- a/projects/rocprofiler-sdk/source/lib/rocprofiler/counters/tests/evaluate_ast_test.cpp +++ b/projects/rocprofiler-sdk/source/lib/rocprofiler/counters/tests/evaluate_ast_test.cpp @@ -20,6 +20,7 @@ // OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE // SOFTWARE. +#include #include #include @@ -27,6 +28,7 @@ #include #include "evaluate_ast_test.hpp" +#include "lib/rocprofiler/agent.hpp" #include "lib/rocprofiler/counters/parser/reader.hpp" namespace @@ -75,7 +77,7 @@ TEST(evaluate_ast, basic_copy) yyparse(&ast); ASSERT_TRUE(ast); - auto eval_ast = EvaluateAST(metrics, *ast, "gfx9"); + auto eval_ast = EvaluateAST({.handle = 0}, metrics, *ast, "gfx9"); EXPECT_TRUE(isIdentical(eval_ast, *ast)); yy_delete_buffer(buf); @@ -98,7 +100,7 @@ TEST(evaluate_ast, counter_expansion) : metric.expression().c_str()); yyparse(&ast); ASSERT_TRUE(ast); - asts.emplace(val, std::move(EvaluateAST(metrics, *ast, "gfx9"))); + asts.emplace(val, std::move(EvaluateAST({.handle = metric.id()}, metrics, *ast, "gfx9"))); yy_delete_buffer(buf); delete ast; } @@ -133,7 +135,7 @@ TEST(evaluate_ast, counter_expansion_multi_derived) : metric.expression().c_str()); yyparse(&ast); ASSERT_TRUE(ast); - asts.emplace(val, std::move(EvaluateAST(metrics, *ast, "gfx9"))); + asts.emplace(val, std::move(EvaluateAST({.handle = metric.id()}, metrics, *ast, "gfx9"))); yy_delete_buffer(buf); delete ast; } @@ -168,7 +170,7 @@ TEST(evaluate_ast, counter_expansion_order) : metric.expression().c_str()); yyparse(&ast); ASSERT_TRUE(ast); - asts.emplace(val, std::move(EvaluateAST(metrics, *ast, "gfx9"))); + asts.emplace(val, std::move(EvaluateAST({.handle = metric.id()}, metrics, *ast, "gfx9"))); yy_delete_buffer(buf); delete ast; } @@ -186,100 +188,832 @@ TEST(evaluate_ast, counter_expansion_order) } } -// TEST(evaluate_ast, counter_expansion_function) -// { -// std::unordered_map metrics = { -// {"SQ_WAVES", Metric("gfx9", "SQ_WAVES", "a", "a", "a", "", "", 0)}, -// {"TCC_HIT", Metric("gfx9", "TCC_HIT", "b", "b", "b", "", "", 1)}, -// {"VLL", Metric("gfx9", "VLL", "b", "b", "b", "", "", 4)}, -// {"TEST_DERRIVED", Metric("gfx9", "TEST_DERRIVED", "C", "C", "C", "SQ_WAVES+VLL", "", -// 2)}}; +TEST(evaluate_ast, counter_expansion_function) +{ + std::unordered_map metrics = { + {"SQ_WAVES", Metric("gfx9", "SQ_WAVES", "a", "a", "a", "", "", 0)}, + {"TCC_HIT", Metric("gfx9", "TCC_HIT", "b", "b", "b", "", "", 1)}, + {"VLL", Metric("gfx9", "VLL", "b", "b", "b", "", "", 4)}, + {"TEST_DERRIVED", Metric("gfx9", "TEST_DERRIVED", "C", "C", "C", "SQ_WAVES+VLL", "", 2)}}; -// std::unordered_map asts; -// for(auto [val, metric] : metrics) -// { -// RawAST* ast = nullptr; -// auto buf = yy_scan_string(metric.expression().empty() ? metric.name().c_str() -// : metric.expression().c_str()); -// yyparse(&ast); -// ASSERT_TRUE(ast) << metric.expression() << " " << metric.name(); -// asts.emplace(val, std::move(EvaluateAST(metrics, *ast, "gfx9"))); -// yy_delete_buffer(buf); -// delete ast; -// } -// } + std::unordered_map asts; + for(auto [val, metric] : metrics) + { + RawAST* ast = nullptr; + auto buf = yy_scan_string(metric.expression().empty() ? metric.name().c_str() + : metric.expression().c_str()); + yyparse(&ast); + ASSERT_TRUE(ast) << metric.expression() << " " << metric.name(); + asts.emplace(val, std::move(EvaluateAST({.handle = metric.id()}, metrics, *ast, "gfx9"))); + yy_delete_buffer(buf); + delete ast; + } +} -// TEST(evaluate_ast, evaluate_simple_math) -// { -// std::unordered_map metrics; -// std::unordered_map results_map; -// std::unordered_map> expected_values; +namespace +{ +void +add_constants(std::unordered_map& metrics, uint64_t start_id) +{ + // Ensure topology is read + rocprofiler::agent::get_agents(); + for(const auto& prop : rocprofiler::agent::get_agent_available_properties()) + { + LOG(ERROR) << prop; + metrics[prop] = {"constant", + prop, + "", + "", + fmt::format("Constant value {} from agent properties", prop), + "", + "yes", + start_id}; + start_id++; + } +} -// uint64_t id = 0; -// for(auto& data : test_data_evaluate_simple_math) -// { -// metrics.emplace( -// data.name, Metric("gfx9", data.name, "Block", "0", "", data.expr, "", id)); -// metric_result res = {id, data.sample_values}; -// results_map.emplace(id, res); -// expected_values.emplace(id, data.expected_values); -// ++id; -// } +} // namespace -// std::unordered_map asts; -// for(auto [val, metric] : metrics) -// { -// RawAST* ast = nullptr; -// auto buf = yy_scan_string(metric.expression().empty() ? metric.name().c_str() -// : metric.expression().c_str()); -// yyparse(&ast); -// ASSERT_TRUE(ast); -// asts.emplace(val, std::move(EvaluateAST(metrics, *ast, "gfx9"))); -// yy_delete_buffer(buf); -// delete ast; -// } +TEST(evaluate_ast, counter_constants) +{ + // Test the construction of counter constants and their evaluation + std::unordered_map metrics = { + {"MAX_WAVE_SIZE", Metric("gfx9", "MAX_WAVE_SIZE", "a", "a", "a", "wave_front_size", "", 0)}, + {"SE_NUM", + Metric("gfx9", "SE_NUM", "b", "b", "b", "array_count/simd_arrays_per_engine", "", 4)}, + {"SIMD_NUM", Metric("gfx9", "SIMD_NUM", "C", "C", "C", "simd_per_cu/CU_NUM", "", 2)}, + {"CU_NUM", + Metric("gfx9", "CU_NUM", "D", "D", "D", "cu_per_simd_array*array_count", "", 5)}}; + add_constants(metrics, 6); + std::unordered_map> asts; -// for(auto [metric_name, ast] : asts) -// { -// double value = ast.evaluate(results_map); -// uint64_t metric_id = metrics.at(metric_name).id(); -// EXPECT_EQ(value, expected_values.at(metric_id)[0]); -// } -// } + // Check counter constants can be loaded + for(const auto& [val, metric] : metrics) + { + RawAST* ast = nullptr; + auto buf = yy_scan_string(metric.expression().empty() ? metric.name().c_str() + : metric.expression().c_str()); + yyparse(&ast); + ASSERT_TRUE(ast) << metric.expression() << " " << metric.name(); + asts.emplace("gfx9", std::unordered_map{}) + .first->second.emplace( + val, std::move(EvaluateAST({.handle = metric.id()}, metrics, *ast, "gfx9"))); + yy_delete_buffer(buf); + delete ast; + } -// TEST(evaluate_ast, evaluate_evaluate_simple_reduce) -// { -// std::unordered_map metrics; -// std::unordered_map results_map; -// std::unordered_map> expected_values; + // Set some random values for test data + auto test_data = rocprofiler_agent_t{}; + test_data.wave_front_size = 32; + test_data.array_count = 8; + test_data.simd_arrays_per_engine = 5; + test_data.simd_per_cu = 104; + test_data.cu_per_simd_array = 156; -// uint64_t id = 0; -// for(auto& data : test_data_evaluate_simple_reduce) -// { -// metrics.emplace( -// data.name, Metric("gfx9", data.name, "Block", "0", "", data.expr, "", id)); -// metric_result res = {id, data.sample_values}; -// results_map.emplace(id, res); -// expected_values.emplace(id, data.expected_values); -// ++id; -// } + // Check that required counters is calculated correctly + std::unordered_map> required_counters = { + {"MAX_WAVE_SIZE", {"wave_front_size"}}, + {"SE_NUM", {"array_count", "simd_arrays_per_engine"}}, + {"SIMD_NUM", {"simd_per_cu", "cu_per_simd_array", "array_count"}}, + {"CU_NUM", {"cu_per_simd_array", "array_count"}}, + }; -// std::unordered_map asts; -// for(auto [val, metric] : metrics) -// { -// RawAST* ast = nullptr; -// auto buf = yy_scan_string(metric.expression().empty() ? metric.name().c_str() -// : metric.expression().c_str()); -// yyparse(&ast); -// ASSERT_TRUE(ast); -// asts.emplace(val, EvaluateAST(metrics, *ast, "gfx9")); -// yy_delete_buffer(buf); -// delete ast; -// } + // Check that the values are being read from agent_t correctly + std::unordered_map raw_agent_values = { + {"wave_front_size", 32}, + {"array_count", 8}, + {"simd_arrays_per_engine", 5}, + {"simd_per_cu", 104}, + {"cu_per_simd_array", 156}, + }; -// for(auto [metric_name, ast]: asts){ -// double value = ast.evaluate(results_map); -// uint64_t metric_id = metrics.at(metric_name).id(); -// EXPECT_EQ(value, expected_values.at(metric_id)[0]); -// } -// } + // Check that the evaluation of the special counters is correct + std::unordered_map final_computed_values = { + {"MAX_WAVE_SIZE", 32}, + {"SE_NUM", 8.0 / 5.0}, + {"SIMD_NUM", 104.0 / (156.0 * 8.0)}, + {"CU_NUM", 156 * 8}, + }; + + for(const auto& [name, expected] : required_counters) + { + auto eval_counters = + rocprofiler::counters::get_required_hardware_counters(asts, "gfx9", metrics[name]); + LOG(INFO) << name; + ASSERT_TRUE(eval_counters); + EXPECT_EQ(eval_counters->size(), expected.size()); + for(const auto& c : *eval_counters) + { + EXPECT_NE(expected.find(c.name()), expected.end()); + EXPECT_TRUE(!c.special().empty()); + } + + // Check that special counters are being decoded properly by the AST + std::unordered_map> decode; + EvaluateAST::read_special_counters(test_data, *eval_counters, decode); + for(const auto& c : *eval_counters) + { + auto* ptr = rocprofiler::common::get_val(decode, c.id()); + ASSERT_TRUE(ptr) << c.name(); + // Check that the value matches agent_t and that its dim/id is set correctly + ASSERT_EQ(ptr->size(), 1); + EXPECT_EQ(ptr->at(0).counter_value, raw_agent_values[c.name()]); + EXPECT_EQ(rocprofiler::counters::rec_to_counter_id(ptr->at(0).id).handle, c.id()); + EXPECT_EQ(rocprofiler::counters::rec_to_dim_pos( + ptr->at(0).id, + rocprofiler::counters::rocprofiler_profile_counter_instance_types:: + ROCPROFILER_DIMENSION_NONE), + 0); + } + asts.at("gfx9").at(name).expand_derived(asts.at("gfx9")); + std::vector>> cache; + auto ret = asts.at("gfx9").at(name).evaluate(decode, cache); + EXPECT_EQ(ret->size(), 1); + EXPECT_FLOAT_EQ(ret->at(0).counter_value, final_computed_values[name]); + + asts.at("gfx9").at(name).set_out_id(*ret); + EXPECT_EQ(rocprofiler::counters::rec_to_counter_id(ret->at(0).id).handle, + metrics[name].id()); + EXPECT_EQ(rocprofiler::counters::rec_to_dim_pos( + ret->at(0).id, + rocprofiler::counters::rocprofiler_profile_counter_instance_types:: + ROCPROFILER_DIMENSION_NONE), + 0); + } +} + +namespace +{ +std::vector +construct_test_data_dim( + rocprofiler_counter_instance_id_t base_id, + std::vector dims, + size_t dim_size) +{ + if(dims.empty()) return {}; + std::vector ret; + auto my_dim = dims.back(); + dims.pop_back(); + for(size_t i = 0; i < dim_size; i++) + { + auto& record = ret.emplace_back(); + record.id = base_id; + rocprofiler::counters::set_dim_in_rec(record.id, my_dim, i); + record.counter_value = + static_cast(rand()) / (static_cast(RAND_MAX / 50000)) + 1.0; + auto recursive_dim = construct_test_data_dim(record.id, dims, dim_size); + ret.insert(ret.end(), recursive_dim.begin(), recursive_dim.end()); + } + return ret; +} + +std::vector +minus_vec(std::vector a, + const std::vector& b) +{ + for(size_t i = 0; i < a.size(); i++) + { + a[i].counter_value -= b[i].counter_value; + } + return a; +}; + +std::vector +plus_vec(std::vector a, + const std::vector& b) +{ + for(size_t i = 0; i < a.size(); i++) + { + a[i].counter_value += b[i].counter_value; + } + return a; +}; + +std::vector +times_vec(std::vector a, + const std::vector& b) +{ + for(size_t i = 0; i < a.size(); i++) + { + a[i].counter_value *= b[i].counter_value; + } + return a; +}; + +std::vector +divide_vec(std::vector a, + const std::vector& b) +{ + for(size_t i = 0; i < a.size(); i++) + { + a[i].counter_value /= b[i].counter_value; + } + return a; +}; + +} // namespace + +TEST(evaluate_ast, evaluate_simple_counters) +{ + using namespace rocprofiler::counters; + + auto get_base_rec_id = [](uint64_t counter_id) { + rocprofiler_counter_instance_id_t base_id = 0; + set_counter_in_rec(base_id, {.handle = counter_id}); + return base_id; + }; + + std::unordered_map metrics = { + {"VOORHEES", Metric("gfx9", "VOORHEES", "a", "a", "a", "", "", 0)}, + {"KRUEGER", Metric("gfx9", "KRUEGER", "a", "a", "a", "", "", 1)}, + {"MYERS", Metric("gfx9", "MYERS", "a", "a", "a", "", "", 2)}, + {"BATES", Metric("gfx9", "BATES", "a", "a", "a", "VOORHEES+KRUEGER", "", 3)}, + {"KRAMER", Metric("gfx9", "KRAMER", "a", "a", "a", "MYERS*BATES", "", 4)}, + {"TORRANCE", Metric("gfx9", "TORRANCE", "a", "a", "a", "KRAMER/KRUEGER", "", 5)}, + {"GHOSTFACE", + Metric("gfx9", "GHOSTFACE", "a", "a", "a", "VOORHEES-(KRUEGER+MYERS)", "", 6)}}; + + std::unordered_map> base_counter_data = { + {"VOORHEES", construct_test_data_dim(get_base_rec_id(0), {ROCPROFILER_DIMENSION_NONE}, 8)}, + {"KRUEGER", construct_test_data_dim(get_base_rec_id(1), {ROCPROFILER_DIMENSION_NONE}, 8)}, + {"MYERS", construct_test_data_dim(get_base_rec_id(2), {ROCPROFILER_DIMENSION_NONE}, 8)}, + }; + + std::unordered_map> asts; + for(const auto& [val, metric] : metrics) + { + RawAST* ast = nullptr; + auto buf = yy_scan_string(metric.expression().empty() ? metric.name().c_str() + : metric.expression().c_str()); + yyparse(&ast); + ASSERT_TRUE(ast) << metric.expression() << " " << metric.name(); + asts.emplace("gfx9", std::unordered_map{}) + .first->second.emplace( + val, std::move(EvaluateAST({.handle = metric.id()}, metrics, *ast, "gfx9"))); + yy_delete_buffer(buf); + delete ast; + } + + // Check base counter evaluation + for(const auto& [name, expected] : base_counter_data) + { + auto eval_counters = + rocprofiler::counters::get_required_hardware_counters(asts, "gfx9", metrics[name]); + ASSERT_TRUE(eval_counters); + ASSERT_EQ(eval_counters->size(), 1); + EXPECT_EQ(eval_counters->begin()->name(), name); + EXPECT_TRUE(eval_counters->begin()->special().empty()); + std::unordered_map> decode = { + {metrics[name].id(), expected}}; + std::vector>> cache; + auto ret = asts.at("gfx9").at(name).evaluate(decode, cache); + EXPECT_EQ(ret->size(), expected.size()); + int pos = 0; + for(const auto& v : *ret) + { + EXPECT_EQ(v.id, expected[pos].id); + EXPECT_EQ(v.counter_value, expected[pos].counter_value); + pos++; + } + } + + // Check derived counter evaluation + std::vector, int64_t>> + derived_counters = { + {"BATES", plus_vec(base_counter_data["VOORHEES"], base_counter_data["KRUEGER"]), 2}, + {"KRAMER", + times_vec(base_counter_data["MYERS"], + plus_vec(base_counter_data["VOORHEES"], base_counter_data["KRUEGER"])), + 3}, + {"TORRANCE", + divide_vec( + times_vec(base_counter_data["MYERS"], + plus_vec(base_counter_data["VOORHEES"], base_counter_data["KRUEGER"])), + base_counter_data["KRUEGER"]), + 3}, + {"GHOSTFACE", + minus_vec(base_counter_data["VOORHEES"], + plus_vec(base_counter_data["KRUEGER"], base_counter_data["MYERS"])), + 3}, + }; + + std::unordered_map> base_counter_decode; + for(const auto& [name, base_counter_v] : base_counter_data) + { + base_counter_decode[metrics[name].id()] = base_counter_v; + } + + for(auto& [name, expected, eval_count] : derived_counters) + { + LOG(INFO) << name; + auto eval_counters = + rocprofiler::counters::get_required_hardware_counters(asts, "gfx9", metrics[name]); + ASSERT_TRUE(eval_counters); + ASSERT_EQ(eval_counters->size(), eval_count); + std::vector>> cache; + asts.at("gfx9").at(name).expand_derived(asts.at("gfx9")); + auto ret = asts.at("gfx9").at(name).evaluate(base_counter_decode, cache); + EXPECT_EQ(ret->size(), expected.size()); + int pos = 0; + asts.at("gfx9").at(name).set_out_id(*ret); + for(const auto& v : *ret) + { + set_counter_in_rec(expected[pos].id, {.handle = metrics[name].id()}); + EXPECT_EQ(v.id, expected[pos].id); + EXPECT_FLOAT_EQ(v.counter_value, expected[pos].counter_value); + pos++; + } + } +} + +namespace +{ +void +run_reduce_test( + std::unordered_map& metrics, + std::unordered_map>& base_counter_data, + std::vector, int64_t>>& + derived_counters) +{ + std::unordered_map> asts; + for(const auto& [val, metric] : metrics) + { + RawAST* ast = nullptr; + auto buf = yy_scan_string(metric.expression().empty() ? metric.name().c_str() + : metric.expression().c_str()); + yyparse(&ast); + ASSERT_TRUE(ast) << metric.expression() << " " << metric.name(); + asts.emplace("gfx9", std::unordered_map{}) + .first->second.emplace( + val, std::move(EvaluateAST({.handle = metric.id()}, metrics, *ast, "gfx9"))); + yy_delete_buffer(buf); + delete ast; + } + + std::unordered_map> base_counter_decode; + for(const auto& [name, base_counter_v] : base_counter_data) + { + base_counter_decode[metrics[name].id()] = base_counter_v; + } + + for(auto& [name, expected, eval_count] : derived_counters) + { + LOG(INFO) << name; + auto eval_counters = + rocprofiler::counters::get_required_hardware_counters(asts, "gfx9", metrics[name]); + ASSERT_TRUE(eval_counters); + ASSERT_EQ(eval_counters->size(), eval_count); + std::vector>> cache; + asts.at("gfx9").at(name).expand_derived(asts.at("gfx9")); + auto ret = asts.at("gfx9").at(name).evaluate(base_counter_decode, cache); + EXPECT_EQ(ret->size(), expected.size()); + ASSERT_EQ(expected.size(), 1); + int pos = 0; + asts.at("gfx9").at(name).set_out_id(*ret); + for(const auto& v : *ret) + { + set_counter_in_rec(expected[pos].id, {.handle = metrics[name].id()}); + EXPECT_EQ(v.id, expected[pos].id); + EXPECT_FLOAT_EQ(v.counter_value, expected[pos].counter_value); + pos++; + } + } +} +}; // namespace + +TEST(evaluate_ast, counter_reduction_sum) +{ + using namespace rocprofiler::counters; + + auto get_base_rec_id = [](uint64_t counter_id) { + rocprofiler_counter_instance_id_t base_id = 0; + set_counter_in_rec(base_id, {.handle = counter_id}); + return base_id; + }; + + auto sum_vec = [](auto& a) { + for(size_t i = 1; i < a.size(); i++) + { + a[0].counter_value += a[i].counter_value; + } + a.resize(1); + CHECK(a.size() == 1); + return a; + }; + + std::unordered_map metrics = { + {"VOORHEES", Metric("gfx9", "VOORHEES", "a", "a", "a", "", "", 0)}, + {"KRUEGER", Metric("gfx9", "KRUEGER", "a", "a", "a", "", "", 1)}, + {"MYERS", Metric("gfx9", "MYERS", "a", "a", "a", "", "", 2)}, + {"MYERS_REDUCED", + Metric("gfx9", "MYERS_REDUCED", "a", "a", "a", "reduce(MYERS,sum)", "", 3)}, + {"BATES", + Metric( + "gfx9", "BATES", "a", "a", "a", "reduce(VOORHEES, sum)+reduce(KRUEGER, sum)", "", 4)}, + {"KRAMER", + Metric("gfx9", + "KRAMER", + "a", + "a", + "a", + "5*reduce(VOORHEES, sum)+reduce(KRUEGER, sum)", + "", + 5)}, + {"GHOSTFACE", + Metric("gfx9", + "GHOSTFACE", + "a", + "a", + "a", + "reduce(VOORHEES, sum)+(reduce(KRUEGER, sum)/5)", + "", + 6)}, + }; + + std::unordered_map> base_counter_data = { + {"VOORHEES", construct_test_data_dim(get_base_rec_id(0), {ROCPROFILER_DIMENSION_NONE}, 8)}, + {"KRUEGER", construct_test_data_dim(get_base_rec_id(1), {ROCPROFILER_DIMENSION_NONE}, 8)}, + {"MYERS", construct_test_data_dim(get_base_rec_id(2), {ROCPROFILER_DIMENSION_NONE}, 8)}, + }; + + // Check derived counter evaluation + std::vector, int64_t>> + derived_counters = { + {"MYERS_REDUCED", sum_vec(base_counter_data["MYERS"]), 1}, + {"BATES", + plus_vec(sum_vec(base_counter_data["VOORHEES"]), + sum_vec(base_counter_data["KRUEGER"])), + 2}, + {"KRAMER", + plus_vec(times_vec(std::vector{{.id = 0, + .counter_value = 5.0}}, + sum_vec(base_counter_data["VOORHEES"])), + sum_vec(base_counter_data["KRUEGER"])), + 2}, + {"GHOSTFACE", + plus_vec(sum_vec(base_counter_data["VOORHEES"]), + divide_vec(sum_vec(base_counter_data["KRUEGER"]), + std::vector{ + {.id = 0, .counter_value = 5.0}})), + 2}, + }; + + run_reduce_test(metrics, base_counter_data, derived_counters); +} + +TEST(evaluate_ast, counter_reduction_min) +{ + using namespace rocprofiler::counters; + + auto get_base_rec_id = [](uint64_t counter_id) { + rocprofiler_counter_instance_id_t base_id = 0; + set_counter_in_rec(base_id, {.handle = counter_id}); + return base_id; + }; + + auto min_vec = [](auto& a) { + a[0].counter_value = std::min_element(a.begin(), a.end(), [](const auto& b, const auto& c) { + return b.counter_value < c.counter_value; + })->counter_value; + a.resize(1); + CHECK(a.size() == 1); + return a; + }; + + std::unordered_map metrics = { + {"VOORHEES", Metric("gfx9", "VOORHEES", "a", "a", "a", "", "", 0)}, + {"KRUEGER", Metric("gfx9", "KRUEGER", "a", "a", "a", "", "", 1)}, + {"MYERS", Metric("gfx9", "MYERS", "a", "a", "a", "", "", 2)}, + {"MYERS_REDUCED", + Metric("gfx9", "MYERS_REDUCED", "a", "a", "a", "reduce(MYERS,min)", "", 3)}, + {"BATES", + Metric( + "gfx9", "BATES", "a", "a", "a", "reduce(VOORHEES, min)+reduce(KRUEGER, min)", "", 4)}, + {"KRAMER", + Metric("gfx9", + "KRAMER", + "a", + "a", + "a", + "5*reduce(VOORHEES, min)+reduce(KRUEGER, min)", + "", + 5)}, + {"GHOSTFACE", + Metric("gfx9", + "GHOSTFACE", + "a", + "a", + "a", + "reduce(VOORHEES, min)+(reduce(KRUEGER, min)/5)", + "", + 6)}, + }; + + std::unordered_map> base_counter_data = { + {"VOORHEES", construct_test_data_dim(get_base_rec_id(0), {ROCPROFILER_DIMENSION_NONE}, 8)}, + {"KRUEGER", construct_test_data_dim(get_base_rec_id(1), {ROCPROFILER_DIMENSION_NONE}, 8)}, + {"MYERS", construct_test_data_dim(get_base_rec_id(2), {ROCPROFILER_DIMENSION_NONE}, 8)}, + }; + + // Check derived counter evaluation + std::vector, int64_t>> + derived_counters = { + {"MYERS_REDUCED", min_vec(base_counter_data["MYERS"]), 1}, + {"BATES", + plus_vec(min_vec(base_counter_data["VOORHEES"]), + min_vec(base_counter_data["KRUEGER"])), + 2}, + {"KRAMER", + plus_vec(times_vec(std::vector{{.id = 0, + .counter_value = 5.0}}, + min_vec(base_counter_data["VOORHEES"])), + min_vec(base_counter_data["KRUEGER"])), + 2}, + {"GHOSTFACE", + plus_vec(min_vec(base_counter_data["VOORHEES"]), + divide_vec(min_vec(base_counter_data["KRUEGER"]), + std::vector{ + {.id = 0, .counter_value = 5.0}})), + 2}, + }; + + run_reduce_test(metrics, base_counter_data, derived_counters); +} + +TEST(evaluate_ast, counter_reduction_max) +{ + using namespace rocprofiler::counters; + + auto get_base_rec_id = [](uint64_t counter_id) { + rocprofiler_counter_instance_id_t base_id = 0; + set_counter_in_rec(base_id, {.handle = counter_id}); + return base_id; + }; + + auto max_vec = [](auto& a) { + a[0].counter_value = std::max_element(a.begin(), a.end(), [](const auto& b, const auto& c) { + return b.counter_value < c.counter_value; + })->counter_value; + a.resize(1); + CHECK(a.size() == 1); + return a; + }; + + std::unordered_map metrics = { + {"VOORHEES", Metric("gfx9", "VOORHEES", "a", "a", "a", "", "", 0)}, + {"KRUEGER", Metric("gfx9", "KRUEGER", "a", "a", "a", "", "", 1)}, + {"MYERS", Metric("gfx9", "MYERS", "a", "a", "a", "", "", 2)}, + {"MYERS_REDUCED", + Metric("gfx9", "MYERS_REDUCED", "a", "a", "a", "reduce(MYERS,max)", "", 3)}, + {"BATES", + Metric( + "gfx9", "BATES", "a", "a", "a", "reduce(VOORHEES, max)+reduce(KRUEGER, max)", "", 4)}, + {"KRAMER", + Metric("gfx9", + "KRAMER", + "a", + "a", + "a", + "5*reduce(VOORHEES, max)+reduce(KRUEGER, max)", + "", + 5)}, + {"GHOSTFACE", + Metric("gfx9", + "GHOSTFACE", + "a", + "a", + "a", + "reduce(VOORHEES, max)+(reduce(KRUEGER, max)/5)", + "", + 6)}, + }; + + std::unordered_map> base_counter_data = { + {"VOORHEES", construct_test_data_dim(get_base_rec_id(0), {ROCPROFILER_DIMENSION_NONE}, 8)}, + {"KRUEGER", construct_test_data_dim(get_base_rec_id(1), {ROCPROFILER_DIMENSION_NONE}, 8)}, + {"MYERS", construct_test_data_dim(get_base_rec_id(2), {ROCPROFILER_DIMENSION_NONE}, 8)}, + }; + + // Check derived counter evaluation + std::vector, int64_t>> + derived_counters = { + {"MYERS_REDUCED", max_vec(base_counter_data["MYERS"]), 1}, + {"BATES", + plus_vec(max_vec(base_counter_data["VOORHEES"]), + max_vec(base_counter_data["KRUEGER"])), + 2}, + {"KRAMER", + plus_vec(times_vec(std::vector{{.id = 0, + .counter_value = 5.0}}, + max_vec(base_counter_data["VOORHEES"])), + max_vec(base_counter_data["KRUEGER"])), + 2}, + {"GHOSTFACE", + plus_vec(max_vec(base_counter_data["VOORHEES"]), + divide_vec(max_vec(base_counter_data["KRUEGER"]), + std::vector{ + {.id = 0, .counter_value = 5.0}})), + 2}, + }; + + run_reduce_test(metrics, base_counter_data, derived_counters); +} + +TEST(evaluate_ast, counter_reduction_avg) +{ + using namespace rocprofiler::counters; + + auto get_base_rec_id = [](uint64_t counter_id) { + rocprofiler_counter_instance_id_t base_id = 0; + set_counter_in_rec(base_id, {.handle = counter_id}); + return base_id; + }; + + auto avg_vec = [](auto& a) { + for(size_t i = 1; i < a.size(); i++) + { + a[0].counter_value += a[i].counter_value; + } + a[0].counter_value /= a.size(); + a.resize(1); + CHECK(a.size() == 1); + return a; + }; + + std::unordered_map metrics = { + {"VOORHEES", Metric("gfx9", "VOORHEES", "a", "a", "a", "", "", 0)}, + {"KRUEGER", Metric("gfx9", "KRUEGER", "a", "a", "a", "", "", 1)}, + {"MYERS", Metric("gfx9", "MYERS", "a", "a", "a", "", "", 2)}, + {"MYERS_REDUCED", + Metric("gfx9", "MYERS_REDUCED", "a", "a", "a", "reduce(MYERS, avr)", "", 3)}, + {"BATES", + Metric( + "gfx9", "BATES", "a", "a", "a", "reduce(VOORHEES, avr)+reduce(KRUEGER, avr)", "", 4)}, + {"KRAMER", + Metric("gfx9", + "KRAMER", + "a", + "a", + "a", + "5*reduce(VOORHEES, avr)+reduce(KRUEGER, avr)", + "", + 5)}, + {"GHOSTFACE", + Metric("gfx9", + "GHOSTFACE", + "a", + "a", + "a", + "reduce(VOORHEES, avr)+(reduce(KRUEGER, avr)/5)", + "", + 6)}, + }; + + std::unordered_map> base_counter_data = { + {"VOORHEES", construct_test_data_dim(get_base_rec_id(0), {ROCPROFILER_DIMENSION_NONE}, 8)}, + {"KRUEGER", construct_test_data_dim(get_base_rec_id(1), {ROCPROFILER_DIMENSION_NONE}, 8)}, + {"MYERS", construct_test_data_dim(get_base_rec_id(2), {ROCPROFILER_DIMENSION_NONE}, 8)}, + }; + + // Check derived counter evaluation + std::vector, int64_t>> + derived_counters = { + {"MYERS_REDUCED", avg_vec(base_counter_data["MYERS"]), 1}, + {"BATES", + plus_vec(avg_vec(base_counter_data["VOORHEES"]), + avg_vec(base_counter_data["KRUEGER"])), + 2}, + {"KRAMER", + plus_vec(times_vec(std::vector{{.id = 0, + .counter_value = 5.0}}, + avg_vec(base_counter_data["VOORHEES"])), + avg_vec(base_counter_data["KRUEGER"])), + 2}, + {"GHOSTFACE", + plus_vec(avg_vec(base_counter_data["VOORHEES"]), + divide_vec(avg_vec(base_counter_data["KRUEGER"]), + std::vector{ + {.id = 0, .counter_value = 5.0}})), + 2}, + }; + + run_reduce_test(metrics, base_counter_data, derived_counters); +} + +TEST(evaluate_ast, evaluate_mixed_counters) +{ + using namespace rocprofiler::counters; + + auto get_base_rec_id = [](uint64_t counter_id) { + rocprofiler_counter_instance_id_t base_id = 0; + set_counter_in_rec(base_id, {.handle = counter_id}); + return base_id; + }; + + auto sum_vec = [](auto& a) { + for(size_t i = 1; i < a.size(); i++) + { + a[0].counter_value += a[i].counter_value; + } + a.resize(1); + CHECK(a.size() == 1); + return a; + }; + + // Set some random values for test data + auto test_data = rocprofiler_agent_t{}; + test_data.wave_front_size = 32; + test_data.array_count = 8; + test_data.simd_arrays_per_engine = 5; + test_data.simd_per_cu = 104; + test_data.cu_per_simd_array = 156; + + std::unordered_map metrics = { + {"MAX_WAVE_SIZE", Metric("gfx9", "MAX_WAVE_SIZE", "a", "a", "a", "wave_front_size", "", 0)}, + {"SE_NUM", + Metric("gfx9", "SE_NUM", "b", "b", "b", "array_count/simd_arrays_per_engine", "", 1)}, + {"CU_NUM", Metric("gfx9", "CU_NUM", "D", "D", "D", "cu_per_simd_array*array_count", "", 2)}, + {"SIMD_NUM", Metric("gfx9", "SIMD_NUM", "C", "C", "C", "simd_per_cu/CU_NUM", "", 3)}, + {"VOORHEES", Metric("gfx9", "VOORHEES", "a", "a", "a", "", "", 4)}, + {"KRUEGER", Metric("gfx9", "KRUEGER", "a", "a", "a", "", "", 5)}, + {"BATES", + Metric("gfx9", "BATES", "a", "a", "a", "MAX_WAVE_SIZE*reduce(VOORHEES,sum)", "", 6)}, + {"KRAMER", Metric("gfx9", "KRAMER", "a", "a", "a", "reduce(KRUEGER,sum)*SE_NUM", "", 7)}, + {"TORRANCE", + Metric("gfx9", "TORRANCE", "a", "a", "a", "reduce(KRUEGER,sum)*SIMD_NUM", "", 8)}}; + add_constants(metrics, 9); + + std::unordered_map> base_counter_data = { + {"VOORHEES", construct_test_data_dim(get_base_rec_id(0), {ROCPROFILER_DIMENSION_NONE}, 8)}, + {"KRUEGER", construct_test_data_dim(get_base_rec_id(1), {ROCPROFILER_DIMENSION_NONE}, 8)}, + }; + + std::vector, int64_t>> + derived_counters = { + {"BATES", + times_vec(std::vector{{.id = 0, .counter_value = 32}}, + sum_vec(base_counter_data["VOORHEES"])), + 2}, + {"KRAMER", + times_vec( + sum_vec(base_counter_data["KRUEGER"]), + std::vector{{.id = 0, .counter_value = 8.0 / 5.0}}), + 3}, + {"TORRANCE", + times_vec(sum_vec(base_counter_data["KRUEGER"]), + std::vector{ + {.id = 0, .counter_value = 104.0 / (156.0 * 8.0)}}), + 4}, + }; + + std::unordered_map> asts; + for(const auto& [val, metric] : metrics) + { + RawAST* ast = nullptr; + auto buf = yy_scan_string(metric.expression().empty() ? metric.name().c_str() + : metric.expression().c_str()); + yyparse(&ast); + ASSERT_TRUE(ast) << metric.expression() << " " << metric.name(); + asts.emplace("gfx9", std::unordered_map{}) + .first->second.emplace( + val, std::move(EvaluateAST({.handle = metric.id()}, metrics, *ast, "gfx9"))); + yy_delete_buffer(buf); + delete ast; + } + + std::unordered_map> base_counter_decode; + for(const auto& special_counter : {"MAX_WAVE_SIZE", "SE_NUM", "SIMD_NUM"}) + { + auto eval_counters = rocprofiler::counters::get_required_hardware_counters( + asts, "gfx9", metrics[special_counter]); + EvaluateAST::read_special_counters(test_data, *eval_counters, base_counter_decode); + } + + for(const auto& [name, base_counter_v] : base_counter_data) + { + base_counter_decode[metrics[name].id()] = base_counter_v; + } + + for(auto& [name, expected, eval_count] : derived_counters) + { + LOG(INFO) << name; + auto eval_counters = + rocprofiler::counters::get_required_hardware_counters(asts, "gfx9", metrics[name]); + ASSERT_TRUE(eval_counters); + ASSERT_EQ(eval_counters->size(), eval_count); + std::vector>> cache; + asts.at("gfx9").at(name).expand_derived(asts.at("gfx9")); + auto ret = asts.at("gfx9").at(name).evaluate(base_counter_decode, cache); + EXPECT_EQ(ret->size(), expected.size()); + ASSERT_EQ(expected.size(), 1); + int pos = 0; + asts.at("gfx9").at(name).set_out_id(*ret); + for(const auto& v : *ret) + { + set_counter_in_rec(expected[pos].id, {.handle = metrics[name].id()}); + EXPECT_EQ(v.id, expected[pos].id); + EXPECT_FLOAT_EQ(v.counter_value, expected[pos].counter_value); + pos++; + } + } +} diff --git a/projects/rocprofiler-sdk/source/lib/rocprofiler/counters/tests/metrics_test.cpp b/projects/rocprofiler-sdk/source/lib/rocprofiler/counters/tests/metrics_test.cpp index 318a08d950..9938ff35e4 100644 --- a/projects/rocprofiler-sdk/source/lib/rocprofiler/counters/tests/metrics_test.cpp +++ b/projects/rocprofiler-sdk/source/lib/rocprofiler/counters/tests/metrics_test.cpp @@ -25,6 +25,7 @@ #include #include +#include "lib/rocprofiler/agent.hpp" #include "lib/rocprofiler/counters/metrics.hpp" namespace @@ -63,7 +64,9 @@ TEST(metrics, base_load) ASSERT_EQ(test_data.count("gfx908"), 1); auto rocp_data_v = rocp_data.at("gfx908"); auto test_data_v = test_data.at("gfx908"); - EXPECT_EQ(rocp_data_v.size(), test_data_v.size()); + // get_agent_available_properties() is the metrics added for fields in agent.hpp + EXPECT_EQ(rocp_data_v.size(), + test_data_v.size() + rocprofiler::agent::get_agent_available_properties().size()); auto find = [&rocp_data_v](const auto& v) -> std::optional { for(const auto& ditr : rocp_data_v) { diff --git a/projects/rocprofiler-sdk/source/lib/rocprofiler/counters/tests/metrics_test.h b/projects/rocprofiler-sdk/source/lib/rocprofiler/counters/tests/metrics_test.h index c53bb0e084..1c62bf6c0b 100644 --- a/projects/rocprofiler-sdk/source/lib/rocprofiler/counters/tests/metrics_test.h +++ b/projects/rocprofiler-sdk/source/lib/rocprofiler/counters/tests/metrics_test.h @@ -32,7 +32,6 @@ static const std::unordered_map>> basic_gfx908 = { {"gfx908", {{"MAX_WAVE_SIZE", "", "", "1", "Max wave size constant"}, - {"KERNEL_DURATION", "", "", "1", "The duration of the kernel dispatch"}, {"SE_NUM", "", "", "1", "SE_NUM"}, {"SIMD_NUM", "", "", "1", "SIMD Number"}, {"CU_NUM", "", "", "1", "CU_NUM"}, @@ -212,7 +211,7 @@ static const std::unordered_map - - - - - + + + + @@ -38,11 +37,10 @@ - - - - - + + + + @@ -80,11 +78,10 @@ # EA1 - - - - - + + + + @@ -118,11 +115,10 @@ - - - - - + + + + @@ -386,11 +382,10 @@ - - - - - + + + + @@ -659,11 +654,10 @@ - - - - - + + + + @@ -724,11 +718,10 @@ - - - - - + + + + diff --git a/projects/rocprofiler-sdk/source/lib/rocprofiler/counters/xml/derived_counters.xml b/projects/rocprofiler-sdk/source/lib/rocprofiler/counters/xml/derived_counters.xml index bbe3281c05..d88099f707 100755 --- a/projects/rocprofiler-sdk/source/lib/rocprofiler/counters/xml/derived_counters.xml +++ b/projects/rocprofiler-sdk/source/lib/rocprofiler/counters/xml/derived_counters.xml @@ -1,5 +1,5 @@ - + @@ -33,7 +33,7 @@ - + @@ -103,7 +103,7 @@ - + @@ -171,7 +171,7 @@ - + @@ -305,7 +305,7 @@ - + @@ -370,7 +370,7 @@ - + @@ -436,7 +436,7 @@ - + @@ -471,7 +471,7 @@ - + diff --git a/projects/rocprofiler-sdk/source/lib/rocprofiler/dispatch_profile.cpp b/projects/rocprofiler-sdk/source/lib/rocprofiler/dispatch_profile.cpp index 8b3e0cfbde..150b8736bd 100644 --- a/projects/rocprofiler-sdk/source/lib/rocprofiler/dispatch_profile.cpp +++ b/projects/rocprofiler-sdk/source/lib/rocprofiler/dispatch_profile.cpp @@ -30,24 +30,26 @@ extern "C" { /** - * @brief Configure Dispatch Profile Counting Service. + * @brief Configure buffered dispatch profile Counting Service. + * Collects the counters in dispatch packets and stores them + * in buffer_id. The buffer may contain packets from more than + * one dispatch (denoted by correlation id). Will trigger the + * callback based on the parameters setup in buffer_id_t. * - * @param [in] context_id - * @param [in] agent_id - * @param [in] buffer_id - * @param [in] callback - * @param [in] callback_data_args + * @param [in] context_id context id + * @param [in] buffer_id id of the buffer to use for the counting service + * @param [in] profile profile config to use for dispatch * @return ::rocprofiler_status_t */ rocprofiler_status_t ROCPROFILER_API -rocprofiler_configure_dispatch_profile_counting_service( +rocprofiler_configure_buffered_dispatch_profile_counting_service( rocprofiler_context_id_t context_id, - rocprofiler_profile_config_id_t profile, + rocprofiler_buffer_id_t buffer_id, rocprofiler_profile_counting_dispatch_callback_t callback, void* callback_data_args) { - return rocprofiler::counters::configure_dispatch( - context_id, profile.handle, callback, callback_data_args) + return rocprofiler::counters::configure_buffered_dispatch( + context_id, buffer_id, callback, callback_data_args) ? ROCPROFILER_STATUS_SUCCESS : ROCPROFILER_STATUS_ERROR; } diff --git a/projects/rocprofiler-sdk/source/lib/rocprofiler/hsa/aql_packet.cpp b/projects/rocprofiler-sdk/source/lib/rocprofiler/hsa/aql_packet.cpp index 3c861c0c7a..23a28e9fea 100644 --- a/projects/rocprofiler-sdk/source/lib/rocprofiler/hsa/aql_packet.cpp +++ b/projects/rocprofiler-sdk/source/lib/rocprofiler/hsa/aql_packet.cpp @@ -30,7 +30,11 @@ namespace hsa { AQLPacket::~AQLPacket() { - if(!command_buf_mallocd) + if(!profile.command_buffer.ptr) + { + // pass, nothing malloced + } + else if(!command_buf_mallocd) { free_func(profile.command_buffer.ptr); } @@ -39,7 +43,11 @@ AQLPacket::~AQLPacket() ::free(profile.command_buffer.ptr); } - if(!output_buffer_malloced) + if(!profile.output_buffer.ptr) + { + // pass, nothing malloced + } + else if(!output_buffer_malloced) { free_func(profile.output_buffer.ptr); } diff --git a/projects/rocprofiler-sdk/source/lib/rocprofiler/hsa/aql_packet.hpp b/projects/rocprofiler-sdk/source/lib/rocprofiler/hsa/aql_packet.hpp index 105a966221..238e2fd88f 100644 --- a/projects/rocprofiler-sdk/source/lib/rocprofiler/hsa/aql_packet.hpp +++ b/projects/rocprofiler-sdk/source/lib/rocprofiler/hsa/aql_packet.hpp @@ -60,6 +60,7 @@ struct AQLPacket bool command_buf_mallocd = false; bool output_buffer_malloced = false; memory_pool_free_func_t free_func = nullptr; + bool empty = {true}; }; inline AQLPacket::AQLPacket(memory_pool_free_func_t func) diff --git a/projects/rocprofiler-sdk/source/lib/rocprofiler/hsa/queue.cpp b/projects/rocprofiler-sdk/source/lib/rocprofiler/hsa/queue.cpp index 8942561baf..7cf69153cf 100644 --- a/projects/rocprofiler-sdk/source/lib/rocprofiler/hsa/queue.cpp +++ b/projects/rocprofiler-sdk/source/lib/rocprofiler/hsa/queue.cpp @@ -71,7 +71,7 @@ signal_limiter() { // Limit the maximun number of HSA signals created. // There is a hard limit to the maximum that can exist. - static common::active_capacity_gate _gate{96}; + static common::active_capacity_gate _gate(16); return _gate; } @@ -215,12 +215,16 @@ AsyncSignalHandler(hsa_signal_value_t /*signal_v*/, void* data) cb_pair.second(queue_info_session.queue, client_id, queue_info_session.kernel_pkt, + queue_info_session, std::move(queue_info_session.inst_pkt)); } else { - cb_pair.second( - queue_info_session.queue, client_id, queue_info_session.kernel_pkt, nullptr); + cb_pair.second(queue_info_session.queue, + client_id, + queue_info_session.kernel_pkt, + queue_info_session, + nullptr); } } }); @@ -374,7 +378,8 @@ WriteInterceptor(const void* packets, queue.signal_callback([&](const auto& map) { for(const auto& [client_id, cb_pair] : map) { - if(auto maybe_pkt = cb_pair.first(queue, client_id, kernel_pkt)) + if(auto maybe_pkt = + cb_pair.first(queue, client_id, kernel_pkt, extern_corr_ids, corr_id)) { LOG_IF(FATAL, inst_pkt) << "We do not support two injections into the HSA queue"; @@ -387,7 +392,7 @@ WriteInterceptor(const void* packets, constexpr auto dummy_signal = hsa_signal_t{.handle = 0}; // Write instrumentation start packet (if one exists) - if(inst_pkt) + if(inst_pkt && !inst_pkt->empty) { inst_pkt->start.header = HSA_PACKET_TYPE_VENDOR_SPECIFIC << HSA_PACKET_HEADER_TYPE; AddVendorSpecificPacket(inst_pkt->start, dummy_signal, transformed_packets); @@ -410,7 +415,7 @@ WriteInterceptor(const void* packets, // Adding a barrier packet with the original packet's completion signal. queue.create_signal(0, &interrupt_signal); - if(inst_pkt) + if(inst_pkt && !inst_pkt->empty) { inst_pkt->stop.header = HSA_PACKET_TYPE_VENDOR_SPECIFIC << HSA_PACKET_HEADER_TYPE; AddVendorSpecificPacket(inst_pkt->stop, dummy_signal, transformed_packets); diff --git a/projects/rocprofiler-sdk/source/lib/rocprofiler/hsa/queue.hpp b/projects/rocprofiler-sdk/source/lib/rocprofiler/hsa/queue.hpp index c633b438dd..1290ea211b 100644 --- a/projects/rocprofiler-sdk/source/lib/rocprofiler/hsa/queue.hpp +++ b/projects/rocprofiler-sdk/source/lib/rocprofiler/hsa/queue.hpp @@ -99,15 +99,6 @@ public: using context_t = context::context; using context_array_t = common::container::small_vector; using callback_t = void (*)(hsa_status_t status, hsa_queue_t* source, void* data); - // Function prototype used to notify consumers that a kernel has been - // enqueued. An AQL packet can be returned that will be injected into - // the queue. - using queue_cb_t = std::function< - std::unique_ptr(const Queue&, ClientID, const rocprofiler_packet&)>; - // Signals the completion of the kernel packet. - using completed_cb_t = std::function< - void(const Queue&, ClientID, const rocprofiler_packet&, std::unique_ptr)>; - using callback_map_t = std::unordered_map>; // Internal session information that is used by write interceptor // to track state of the intercepted kernel. @@ -128,6 +119,23 @@ public: external_corr_id_map_t extern_corr_ids = {}; }; + // Function prototype used to notify consumers that a kernel has been + // enqueued. An AQL packet can be returned that will be injected into + // the queue. + using queue_cb_t = std::function( + const Queue&, + ClientID, + const rocprofiler_packet&, + const queue_info_session_t::external_corr_id_map_t&, + const context::correlation_id*)>; + // Signals the completion of the kernel packet. + using completed_cb_t = std::function)>; + using callback_map_t = std::unordered_map>; + Queue(const AgentCache& agent, uint32_t size, hsa_queue_type32_t type, diff --git a/projects/rocprofiler-sdk/source/lib/rocprofiler/hsa/queue_controller.cpp b/projects/rocprofiler-sdk/source/lib/rocprofiler/hsa/queue_controller.cpp index 1033b9a8b0..dea8d859ba 100644 --- a/projects/rocprofiler-sdk/source/lib/rocprofiler/hsa/queue_controller.cpp +++ b/projects/rocprofiler-sdk/source/lib/rocprofiler/hsa/queue_controller.cpp @@ -87,7 +87,7 @@ QueueController::add_queue(hsa_queue_t* id, std::unique_ptr queue) for(const auto& [cbid, cb_tuple] : callbacks) { auto& [agent, qcb, ccb] = cb_tuple; - if(agent.id.handle == agent_id) + if(agent.id.handle == ALL_AGENTS.id.handle || agent.id.handle == agent_id) { map[id]->register_callback(cbid, qcb, ccb); } @@ -103,20 +103,28 @@ QueueController::destory_queue(hsa_queue_t* id) } ClientID -QueueController::add_callback(const rocprofiler_agent_t& agent, - Queue::queue_cb_t qcb, - Queue::completed_cb_t ccb) +QueueController::add_callback(std::optional agent, + Queue::queue_cb_t qcb, + Queue::completed_cb_t ccb) { static std::atomic client_id = 1; ClientID return_id; _callback_cache.wlock([&](auto& cb_cache) { - return_id = client_id; - cb_cache[client_id] = std::tuple(agent, qcb, ccb); + return_id = client_id; + if(agent) + { + cb_cache[client_id] = std::tuple(*agent, qcb, ccb); + } + else + { + cb_cache[client_id] = std::tuple(ALL_AGENTS, qcb, ccb); + } client_id++; + _queues.wlock([&](auto& map) { for(auto& [_, queue] : map) { - if(queue->get_agent().get_rocp_agent()->id.handle == agent.id.handle) + if(!agent || queue->get_agent().get_rocp_agent()->id.handle == agent->id.handle) { queue->register_callback(return_id, qcb, ccb); } diff --git a/projects/rocprofiler-sdk/source/lib/rocprofiler/hsa/queue_controller.hpp b/projects/rocprofiler-sdk/source/lib/rocprofiler/hsa/queue_controller.hpp index f66c34851d..7167f1bb5f 100644 --- a/projects/rocprofiler-sdk/source/lib/rocprofiler/hsa/queue_controller.hpp +++ b/projects/rocprofiler-sdk/source/lib/rocprofiler/hsa/queue_controller.hpp @@ -27,6 +27,7 @@ #include "lib/rocprofiler/hsa/queue.hpp" #include +#include #include #include @@ -48,8 +49,11 @@ public: void destory_queue(hsa_queue_t*); // Add callback to queues associated with the agent. Returns a client - // id that can be used by callers to remove the callback. - ClientID add_callback(const rocprofiler_agent_t&, Queue::queue_cb_t, Queue::completed_cb_t); + // id that can be used by callers to remove the callback. If no agent + // is specified, callback will be applied to all agents. + ClientID add_callback(std::optional, + Queue::queue_cb_t, + Queue::completed_cb_t); void remove_callback(ClientID); const CoreApiTable& get_core_table() const { return _core_table; } @@ -62,6 +66,8 @@ public: const Queue* get_queue(const hsa_queue_t&) const; private: + static constexpr rocprofiler_agent_t ALL_AGENTS{ + .id = {.handle = std::numeric_limits::max()}}; using agent_callback_tuple_t = std::tuple; using queue_map_t = std::unordered_map>; diff --git a/projects/rocprofiler-sdk/source/lib/rocprofiler/profile_config.cpp b/projects/rocprofiler-sdk/source/lib/rocprofiler/profile_config.cpp index 572a73567a..6fcaf91e8a 100644 --- a/projects/rocprofiler-sdk/source/lib/rocprofiler/profile_config.cpp +++ b/projects/rocprofiler-sdk/source/lib/rocprofiler/profile_config.cpp @@ -46,8 +46,10 @@ rocprofiler_create_profile_config(rocprofiler_agent_t agent, size_t counters_count, rocprofiler_profile_config_id_t* config_id) { - rocprofiler::counters::profile_config config; - const auto& id_map = rocprofiler::counters::getMetricIdMap(); + std::shared_ptr config = + std::make_shared(); + + const auto& id_map = rocprofiler::counters::getMetricIdMap(); for(size_t i = 0; i < counters_count; i++) { @@ -55,10 +57,10 @@ rocprofiler_create_profile_config(rocprofiler_agent_t agent, const auto* metric_ptr = rocprofiler::common::get_val(id_map, counter_id.handle); if(!metric_ptr) return ROCPROFILER_STATUS_ERROR_COUNTER_NOT_FOUND; - config.metrics.push_back(*metric_ptr); + config->metrics.push_back(*metric_ptr); } - config.agent = agent; + config->agent = agent; config_id->handle = rocprofiler::counters::create_counter_profile(std::move(config)); return ROCPROFILER_STATUS_SUCCESS;