AI/rocm-systems
2
0
Vork 0
Code Kwesties Pull-aanvragen Actions Paketten Projecten Publicaties Wiki Activiteit

Replace instances of rocm-libraries with rocm-systems in workflows

Bladeren bron
This commit is contained in:
amd-jmacaran
2025-07-16 11:55:53 -04:00
bovenliggende 895d57d9e0
commit 30c74c10c4
15 gewijzigde bestanden met toevoegingen van 116 en 116 verwijderingen
Download Patch-bestand Download Diff-bestand Expand all files Collapse all files
docs/continuous-integration.md
+1 -1
Bestand weergeven
@@ -3,7 +3,7 @@
> [!IMPORTANT]
> This document is currently in **draft** and may be subject to change.
This document is to detail the various continuous integration (CI) systems that are run on the rocm-libraries monorepo.
This document is to detail the various continuous integration (CI) systems that are run on the rocm-systems monorepo.
## Table of Contents
1. [Azure Pipelines](#azure-pipelines)
docs/how-to-cherry-pick-monorepo-changes-to-release-staging.md
+15 -15
Bestand weergeven
@@ -3,17 +3,17 @@
> [!IMPORTANT]
> This document is currently in **draft** and may be subject to change.
When a project has been migrated into the ROCm monorepo, day-to-day work happens on the monorepo’s `develop` branch.
When a project has been migrated into the ROCm monorepo, day-to-day work happens on the monorepo’s `develop` branch.
Down-stream teams, however, still consume the original (pre-monorepo) repositories, particularly their `release-staging/rocm-rel-x.y` branches, through a variety of mechanisms.
This document explains how to move a change from the monorepo into those release-staging branches while guaranteeing that every commit on a release-staging branch also exists in the monorepo.
This document explains how to move a change from the monorepo into those release-staging branches while guaranteeing that every commit on a release-staging branch also exists in the monorepo.
## 1. Land the change in the monorepo's develop branch
1. Create a pull request in `ROCm/rocm-libraries` that targets `develop`.
2. When merging, choose **Squash & Merge** (if the change can be represented as a single logical commit).
1. Create a pull request in `ROCm/rocm-systems` that targets `develop`.
2. When merging, choose **Squash & Merge** (if the change can be represented as a single logical commit).
Why? A single commit is easier to cherry-pick later.
Result: The commit is now on `ROCm/rocm-libraries:develop`.
Result: The commit is now on `ROCm/rocm-systems:develop`.
## 2. Cherry-pick into the monorepo's release-staging branch
@@ -26,11 +26,11 @@ $ git checkout -b cherry-pick-foo-rel-x.y origin/release-staging/rocm-rel-x.y
2. Cherry-pick the commit:
```
$ git cherry-pick abcd1234
$ git cherry-pick abcd1234
```
3. Resolve any merge conflicts (rare if the branch is close to develop).
4. Push the branch and open a PR that targets `ROCm/rocm-libraries:release-staging/rocm-rel-x.y`.
4. Push the branch and open a PR that targets `ROCm/rocm-systems:release-staging/rocm-rel-x.y`.
5. Request reviews, obtain approvals, and merge.
## 3. Wait for the automatic “fan-out” sync
@@ -39,29 +39,29 @@ Every ~15 minutes, a CI job copies new commits from the monorepo back into the c
After merging your PR:
1. Monitor the CI job or simply wait ~15 minutes.
1. Monitor the CI job or simply wait ~15 minutes.
2. Go to the original (pre-monorepo) repository and verify the commits have been reflected onto the `develop` and `release-staging/rocm-rel-x.y` branches.
## FAQ
Q : Can I cherry-pick multiple commits at once?
Q : Can I cherry-pick multiple commits at once?
A : Yes, but prefer a squash merge in the monorepo so you only need to pick one.
Q : What if the auto-sync hasn’t copied the commit?
A : Verify the CI status in `rocm-libraries`. If failed, ask the infra team; the commit will re-sync after a successful run.
Q : What if the auto-sync hasn’t copied the commit?
A : Verify the CI status in `rocm-systems`. If failed, ask the infra team; the commit will re-sync after a successful run.
Q : Can I push directly to the release-staging branch?
Q : Can I push directly to the release-staging branch?
A : No. Always go through a PR so CI and reviewers can validate the cherry-pick.
Q : What if commits have been pushed to develop that make a cherry-pick incompatible with release-staging?
Q : What if commits have been pushed to develop that make a cherry-pick incompatible with release-staging?
A : It's likely that this fix/change will also be landed in develop at some point, else we risk divergent features/support. So, it's recommended to still land the change in develop first, and cherry-pick to release-staging, resolving any merge conflicts that arise. If for some reason the develop branch has diverged so far from the release-staging for your component that a cherry-pick is irreconcilable, land the changes in develop and release-staging using fully separate PRs, and add references to the other for traceability.
## Summary
In short:
1. Merge change to monorepo `develop`.
2. Cherry-pick to monorepo `release-staging/rocm-rel-x.y`.
1. Merge change to monorepo `develop`.
2. Cherry-pick to monorepo `release-staging/rocm-rel-x.y`.
3. Wait for the fan-out sync and verify the changes are reflected in the original repository.
Following this process keeps release branches in sync with the monorepo while allowing critical fixes to flow to down-stream consumers.
docs/migration-process.md
+51 -51
Bestand weergeven
@@ -1,16 +1,16 @@
# Migration from Single Repo to Monorepo
## Introduction
This document outlines the process for migrating from a single library repository to this monorepo. It covers the necessary steps to ensure a smooth transition, including pre-conditions, conflict resolution, and changes to repository management.
## Pre-conditions
To ensure consistency and maintainability during the migration, the following pre-conditions must be satisfied:
# Migration from Single Repo to Monorepo
## Introduction
This document outlines the process for migrating from a single library repository to this monorepo. It covers the necessary steps to ensure a smooth transition, including pre-conditions, conflict resolution, and changes to repository management.
## Pre-conditions
To ensure consistency and maintainability during the migration, the following pre-conditions must be satisfied:
1. **Identify Next Repo to Migrate:**
- Please refer to the main [README.md](/README.md) on the order of repositories being migrated.
- This is usually discussed in advance in meetings with the technical leads of that project.
2. **Identify Branches and Pull Requests:**
2. **Identify Branches and Pull Requests:**
- Determine branches and active pull requests that will be affected by the migration.
- Typically, this is limited to the pull requests targeting `develop` and `release-staging` branches.
- Any point-fixes for previous releases will not be migrated over.
@@ -18,51 +18,51 @@ To ensure consistency and maintainability during the migration, the following pr
3. **Pause Merges:**
- There are GitHub Actions that automatically synchronize changes from the individual repos to the monorepo.
- These automated actions need to be paused by disabling the workflow on the GitHub UI.
- develop branch workflow: https://github.com/ROCm/rocm-libraries/actions/workflows/update-subtrees.yml
- release-staging branch workflow: https://github.com/ROCm/rocm-libraries/actions/workflows/update-release-staging-subtree.yml
- develop branch workflow: https://github.com/ROCm/rocm-systems/actions/workflows/update-subtrees.yml
- release-staging branch workflow: https://github.com/ROCm/rocm-systems/actions/workflows/update-release-staging-subtree.yml
- Announce the pause to key stakeholders and ask them to propagate the news.
## Migration Process
### Step 1: Pull Request Management
1. **Automated Import of Pull Requests:**
## Migration Process
### Step 1: Pull Request Management
1. **Automated Import of Pull Requests:**
- Pull requests without merge conflicts will be automatically imported with a GitHub Action, only executable by maintainers and admins.
- This GitHub action will create a feature branch on the monorepo, pulling in the changes from the PR on the original repo using `git subtree`.
- These imported pull requests will have the `imported pr` label applied.
- After running the action successfully, close the PR on the original repo.
- GitHub Action: https://github.com/ROCm/rocm-libraries/actions/workflows/pr-import.yml
- Example Imported Pull Request on Monorepo: https://github.com/ROCm/rocm-libraries/pull/206
- GitHub Action: https://github.com/ROCm/rocm-systems/actions/workflows/pr-import.yml
- Example Imported Pull Request on Monorepo: https://github.com/ROCm/rocm-systems/pull/206
- Corresponding Pull Request on Original Repo: https://github.com/ROCm/Tensile/pull/2135
2. **Conflict Resolution:**
- For pull requests with merge conflicts, add a comment explaining the merge conflict and blocking issue preventing import.
2. **Conflict Resolution:**
- For pull requests with merge conflicts, add a comment explaining the merge conflict and blocking issue preventing import.
- Collaborate with contributors to import these PRs after the migration period, or the contributor can reopen the pull request themselves on the monorepo.
3. **NPI Development:**
- Repeat this import process for the monorepo on GitHub EMU for npi work.
### Step 2: Issue and Comment Import
1. **Issue Import:**
### Step 2: Issue and Comment Import
1. **Issue Import:**
- Import all open issues from both public and EMU repositories with a GitHub Action, only executable by maintainers and admins.
- Comments are copied over in the imported issues.
- GitHub Action: https://github.com/ROCm/rocm-libraries/actions/workflows/issue-import.yml
- GitHub Action: https://github.com/ROCm/rocm-systems/actions/workflows/issue-import.yml
- Ensure issue status and labels are preserved during migration.
- Look for any weird unicode characters that get mangled during the automated import.
- After running the action successfully, close the issue with a comment on the original repo.
- Example Imported Issue on Monorepo: https://github.com/ROCm/rocm-libraries/issues/100
- Example Imported Issue on Monorepo: https://github.com/ROCm/rocm-systems/issues/100
- Corresponding Issue on Original Repo: https://github.com/ROCm/rocThrust/issues/501
### Step 3: Path-Based Commit History
1. **Use of Git Filter-Repo:**
### Step 3: Path-Based Commit History
1. **Use of Git Filter-Repo:**
- Utilize `git filter-repo` at the migration point to add path-based commit history.
- As changing the contents of a commit will change the output the hash function, commit SHA will change.
- The filter-repo tool is used to add a snippet at the end of the old commit to refer to the old commit SHA.
- It is not possible to preserve the same commit SHA if the metadata is changed to point to new paths, as the hash function output changes.
- Example directory view: https://github.com/ROCm/rocm-libraries/commits/develop/projects/rocrand/library
- Example commit view: https://github.com/ROCm/rocm-libraries/commit/ea8b6884a0f2a0ec80ff7811bc5ec042600790e9
- Example directory view: https://github.com/ROCm/rocm-systems/commits/develop/projects/rocrand/library
- Example commit view: https://github.com/ROCm/rocm-systems/commit/ea8b6884a0f2a0ec80ff7811bc5ec042600790e9
2. **command sequence example**
@@ -74,11 +74,11 @@ pushd hipBLAS-common
git checkout develop
git pull origin
git checkout -b filtered/hipblas-common
git filter-repo --path-rename '':'projects/hipblas-common/' --commit-callback "original_hash = commit.original_id.decode(); original_message = commit.message.decode(); new_message = f'{original_message}\\n\\n[ROCm/hipBLAS-common commit: {original_hash}]' if original_message.strip() else f'[ROCm/hipBLAS-common commit: {original_hash}]'; commit.message = new_message.encode()" --force
git remote add monorepo git@github.com:ROCm/rocm-libraries.git
git filter-repo --path-rename '':'projects/hipblas-common/' --commit-callback "original_hash = commit.original_id.decode(); original_message = commit.message.decode(); new_message = f'{original_message}\\n\\n[ROCm/hipBLAS-common commit: {original_hash}]' if original_message.strip() else f'[ROCm/hipBLAS-common commit: {original_hash}]'; commit.message = new_message.encode()" --force
git remote add monorepo git@github.com:ROCm/rocm-systems.git
git push monorepo filtered/hipblas-common
popd
git clone git@github.com:ROCm/rocm-libraries.git
git clone git@github.com:ROCm/rocm-systems.git
git checkout develop
git pull origin
git branch backup/develop-hipblas-common
@@ -95,17 +95,17 @@ git push origin develop
```
### Step 4: CI/CD Triggers
1. **CI/CD Trigger Points:**
- Modify the existing CI/CD systems to be triggered off changes to this project in the monorepo.
### Step 5: Repository Adjustments
1. **Default Branch Deprecation:**
1. **Default Branch Deprecation:**
- Change the default branch of the original repository with a clear deprecation notice.
- Example: https://github.com/ROCm/rocPRIM/tree/develop_deprecated
2. **Disable Dependabot Updates:**
2. **Disable Dependabot Updates:**
- Cease automatic dependency updates in the old repository to streamline the focus on the monorepo.
- Clear the contents in this file on the original repo: https://github.com/ROCm/rocPRIM/blob/develop_deprecated/.github/dependabot.yml
- In the original repo settings, go to Security -> Advanced Security and disable all the Dependabot settings.
@@ -120,26 +120,26 @@ git push origin develop
- Update the true/false values in the [`repos-config.json`](/.github/repos-config.json) file that automated workflows use to determine which way the source gets synchronized..
- `auto_subtree_pull` should now be false, `auto_subtree_push` should now be true for this migrated project.
- Make this change on both the `develop` and `release-staging` branches.
- https://github.com/ROCm/rocm-libraries/blob/develop/.github/repos-config.json
- https://github.com/ROCm/rocm-libraries/blob/release-staging/rocm-rel-7.0/.github/repos-config.json
- https://github.com/ROCm/rocm-systems/blob/develop/.github/repos-config.json
- https://github.com/ROCm/rocm-systems/blob/release-staging/rocm-rel-7.0/.github/repos-config.json
2. **Update the monorepo README.md:**
- Update the migration status on the monorepo's main readme to indicate the migration has been completed.
- https://github.com/ROCm/rocm-libraries/blob/develop/README.md
- https://github.com/ROCm/rocm-systems/blob/develop/README.md
## Post-Migration Activities
## Post-Migration Activities
1. **Re-enable synchronization jobs:**
- Re-enable any automated workflows that were paused.
2. **Communication:**
- Communicate to key stakeholders the successful completion of the migration.
- Continue daily meetings and active written communications to offer support for any issues that arise.
- Continue daily meetings and active written communications to offer support for any issues that arise.
3. **Automated Patching of Original Repos:**
- During the migration period, when a pull request is merged on the monorepo, the contents of the pull request will be split into patches to be pushed onto the original repos.
- This supports potential pull requests that touch multiple projects.
- Example pull request on the monorepo: https://github.com/ROCm/rocm-libraries/pull/230
- Example pull request on the monorepo: https://github.com/ROCm/rocm-systems/pull/230
- Corresponding patches on the original repos:
- https://github.com/ROCm/hipCUB/commit/50438ec4971def627729ea3d9dc1485e52b09e48
- https://github.com/ROCm/hipRAND/commit/74afe303def580290a8e5b149ea13ae739bc4c61
@@ -149,8 +149,8 @@ git push origin develop
4. **Monitoring:**
- Monitor the monorepo for any issues or discrepancies.
- If the automated patching for a PR failed to make it to the original repo, use this GitHub Action: https://github.com/ROCm/rocm-libraries/actions/workflows/pr-merge-sync-patches-manual.yml
## Conclusion
This migration process aims to assist the ROCm development teams transition from many repos to a monorepo by addressing the topics above. By following these outlined steps, we aim to maintain and improve the quality of our development workflow post-migration.
- If the automated patching for a PR failed to make it to the original repo, use this GitHub Action: https://github.com/ROCm/rocm-systems/actions/workflows/pr-merge-sync-patches-manual.yml
## Conclusion
This migration process aims to assist the ROCm development teams transition from many repos to a monorepo by addressing the topics above. By following these outlined steps, we aim to maintain and improve the quality of our development workflow post-migration.