Replace instances of rocm-libraries with rocm-systems in workflows

This commit is contained in:
amd-jmacaran
2025-07-16 11:55:53 -04:00
rodzic 895d57d9e0
commit 30c74c10c4
15 zmienionych plików z 116 dodań i 116 usunięć
+2 -2
Wyświetl plik
@@ -22,9 +22,9 @@ Outputs:
Example Usage:
To run in debug mode and perform a dry-run (no changes made):
python pr_auto_label.py --repo ROCm/rocm-libraries --pr <pr-number> --dry-run --debug
python pr_auto_label.py --repo ROCm/rocm-systems --pr <pr-number> --dry-run --debug
To run in debug mode and apply label changes:
python pr_auto_label.py --repo ROCm/rocm-libraries --pr <pr-number> --debug
python pr_auto_label.py --repo ROCm/rocm-systems --pr <pr-number> --debug
"""
import argparse
@@ -28,7 +28,7 @@ Outputs:
Example Usage:
To run in auto-push situations in debug mode and perform a dry-run (no changes made):
python pr_detect_changed_subtrees.py --repo ROCm/rocm-libraries --pr 123 --require-auto-push --debug --dry-run
python pr_detect_changed_subtrees.py --repo ROCm/rocm-systems --pr 123 --require-auto-push --debug --dry-run
"""
import argparse
@@ -23,7 +23,7 @@ Arguments:
--debug : If set, enables detailed debug logging.
Example Usage:
python pr_merge_sync_patches.py --repo ROCm/rocm-libraries --pr 123 --subtrees "$(printf 'projects/rocBLAS\nprojects/hipBLASLt\nshared/rocSPARSE')" --dry-run --debug
python pr_merge_sync_patches.py --repo ROCm/rocm-systems --pr 123 --subtrees "$(printf 'projects/rocBLAS\nprojects/hipBLASLt\nshared/rocSPARSE')" --dry-run --debug
"""
import argparse
@@ -256,7 +256,7 @@ def apply_patches_and_squash(entry: RepoEntry, monorepo_url: str, monorepo_pr: i
_apply_patch(subrepo_path, patch_path, Path(rel_path), Path.cwd(), prefix)
# Final squash
commit_msg = f"[rocm-libraries] {monorepo_url}#{monorepo_pr} (commit {merge_sha[:7]})\n\n" + \
commit_msg = f"[rocm-systems] {monorepo_url}#{monorepo_pr} (commit {merge_sha[:7]})\n\n" + \
_run_git(["log", "-1", "--pretty=%B", merge_sha])
_run_git(["reset", "--soft", base_commit], cwd=subrepo_path)
_run_git(["commit", "-m", commit_msg, "--author", f"{author_name} <{author_email}>"], cwd=subrepo_path)
+15 -15
Wyświetl plik
@@ -24,40 +24,40 @@ def set_github_output(d: Mapping[str, str]):
return
with open(step_output_file, "a") as f:
f.writelines(f"{k}={v}" + "\n" for k, v in d.items())
def retrieve_projects(args):
# TODO(geomin12): #590 Enable TheRock CI for forked PRs
if args.get("is_forked_pr"):
logging.info("Warning: not enabling any projects due to is_forked_pr. Builds/tests for forked PRs are disabled pending: https://github.com/ROCm/rocm-libraries/issues/590")
logging.info("Warning: not enabling any projects due to is_forked_pr. Builds/tests for forked PRs are disabled pending: https://github.com/ROCm/rocm-systems/issues/590")
return []
if args.get("is_pull_request"):
subtrees = args.get("input_subtrees").split("\n")
if args.get("is_workflow_dispatch"):
if args.get("input_projects") == "all":
subtrees = list(subtree_to_project_map.keys())
else:
subtrees = args.get("input_projects").split()
# If a push event to develop happens, we run tests on all subtrees
if args.get("is_push"):
subtrees = list(subtree_to_project_map.keys())
projects = set()
# collect the associated subtree to project
for subtree in subtrees:
if subtree in subtree_to_project_map:
projects.add(subtree_to_project_map.get(subtree))
# retrieve the subtrees to checkout, cmake options to build, and projects to test
# retrieve the subtrees to checkout, cmake options to build, and projects to test
project_to_run = []
for project in projects:
if project in project_map:
project_to_run.append(project_map.get(project))
return project_to_run
@@ -72,16 +72,16 @@ if __name__ == "__main__":
args["is_pull_request"] = github_event_name == "pull_request"
args["is_push"] = github_event_name == "push"
args["is_workflow_dispatch"] = github_event_name == "workflow_dispatch"
is_forked_pr = os.getenv("IS_FORKED_PR")
args["is_forked_pr"] = is_forked_pr == "true"
input_subtrees = os.getenv("SUBTREES", "")
args["input_subtrees"] = input_subtrees
input_projects = os.getenv("PROJECTS", "")
args["input_projects"] = input_projects
logging.info(f"Retrieved arguments {args}")
run(args)
+17 -17
Wyświetl plik
@@ -34,7 +34,7 @@ concurrency:
jobs:
dispatch-azure-ci:
name: Trigger Azure CI
if: github.repository == 'ROCm/rocm-libraries'
if: github.repository == 'ROCm/rocm-systems'
runs-on: ubuntu-latest
steps:
- name: Generate a token
@@ -45,13 +45,13 @@ jobs:
private-key: ${{ secrets.APP_PRIVATE_KEY }}
owner: ${{ github.repository_owner }}
repositories: |
rocm-libraries
rocm-systems
- name: Wait until refs/pull/${{ github.event.pull_request.number }}/merge exists
run: |
merge_ref="refs/pull/${{ github.event.pull_request.number }}/merge"
check_merge_ref() {
git ls-remote "https://github.com/ROCm/rocm-libraries" "$merge_ref" | grep -q "$merge_ref"
git ls-remote "https://github.com/ROCm/rocm-systems" "$merge_ref" | grep -q "$merge_ref"
}
max_attempts=10
@@ -103,7 +103,7 @@ jobs:
if: steps.detect.outputs.subtrees
run: |
pr_number=${{ github.event.pull_request.number }}
pr_filter_query="branchName=refs/pull/$pr_number/merge&repositoryType=GitHub&repositoryId=ROCm/rocm-libraries&api-version=7.1"
pr_filter_query="branchName=refs/pull/$pr_number/merge&repositoryType=GitHub&repositoryId=ROCm/rocm-systems&api-version=7.1"
res=$(curl -sSX GET "https://dev.azure.com/ROCm-CI/ROCm-CI/_apis/build/builds?$pr_filter_query" \
-H "Content-Type: application/json")
@@ -111,12 +111,12 @@ jobs:
runs=$(echo "$res" | jq -r ".value[] | select(.status == \"inProgress\" or .status == \"notStarted\") | .id")
if [ -z "$runs" ]; then
echo "No in-progress/not-started runs found for ROCm/rocm-libraries PR #$pr_number"
echo "No in-progress/not-started runs found for ROCm/rocm-systems PR #$pr_number"
echo "status=false" >> $GITHUB_OUTPUT
exit 0
fi
echo "Found in-progress/not-started runs for ROCm/rocm-libraries PR #$pr_number: $runs"
echo "Found in-progress/not-started runs for ROCm/rocm-systems PR #$pr_number: $runs"
echo "status=true" >> $GITHUB_OUTPUT
for run_id in $runs; do
@@ -139,8 +139,8 @@ jobs:
if: steps.detect.outputs.subtrees && steps.cancel-in-progress.outputs.status == 'false'
run: |
pr_number=${{ github.event.pull_request.number }}
pr_merge_sha=$(curl -sSX GET "https://api.github.com/repos/ROCm/rocm-libraries/git/ref/pull/${pr_number}/merge" | jq -r '.object.sha')
pr_filter_query="branchName=refs/pull/$pr_number/merge&repositoryType=GitHub&repositoryId=ROCm/rocm-libraries&api-version=7.1"
pr_merge_sha=$(curl -sSX GET "https://api.github.com/repos/ROCm/rocm-systems/git/ref/pull/${pr_number}/merge" | jq -r '.object.sha')
pr_filter_query="branchName=refs/pull/$pr_number/merge&repositoryType=GitHub&repositoryId=ROCm/rocm-systems&api-version=7.1"
res=$(curl -sSX GET "https://dev.azure.com/ROCm-CI/ROCm-CI/_apis/build/builds?$pr_filter_query" \
-H "Content-Type: application/json")
@@ -160,13 +160,13 @@ jobs:
success_project_names=$(echo "$success_runs_info" | jq -r '.name')
if [ -z "$failed_run_ids" ]; then
echo "No failed/cancelled runs found for ROCm/rocm-libraries PR #$pr_number at merge commit $pr_merge_sha"
echo "No failed/cancelled runs found for ROCm/rocm-systems PR #$pr_number at merge commit $pr_merge_sha"
echo "status=false" >> $GITHUB_OUTPUT
exit 0
fi
echo "Found failed/cancelled runs for ROCm/rocm-libraries PR #$pr_number at merge commit $pr_merge_sha: ${failed_run_ids[*]}"
echo "Found successful runs for ROCm/rocm-libraries PR #$pr_number at merge commit $pr_merge_sha: ${success_run_ids[*]}"
echo "Found failed/cancelled runs for ROCm/rocm-systems PR #$pr_number at merge commit $pr_merge_sha: ${failed_run_ids[*]}"
echo "Found successful runs for ROCm/rocm-systems PR #$pr_number at merge commit $pr_merge_sha: ${success_run_ids[*]}"
echo "status=true" >> $GITHUB_OUTPUT
new_run_ids=()
@@ -278,13 +278,13 @@ jobs:
pr_number=${{ github.event.pull_request.number }}
pr_head_sha=${{ github.event.pull_request.head.sha }}
pr_merge_sha=$(curl -sSX GET "https://api.github.com/repos/ROCm/rocm-libraries/git/ref/pull/${pr_number}/merge" | jq -r '.object.sha')
pr_merge_sha=$(curl -sSX GET "https://api.github.com/repos/ROCm/rocm-systems/git/ref/pull/${pr_number}/merge" | jq -r '.object.sha')
newline=$'\n'
summary="PR: [${{ github.event.pull_request.title }} #$pr_number](${{ github.event.pull_request.html_url }})${newline}${newline}"
summary+="HEAD: [$pr_head_sha](https://github.com/ROCm/rocm-libraries/commit/$pr_head_sha)${newline}${newline}"
summary+="MERGE: [$pr_merge_sha](https://github.com/ROCm/rocm-libraries/commit/$pr_merge_sha)${newline}${newline}"
summary+="HEAD: [$pr_head_sha](https://github.com/ROCm/rocm-systems/commit/$pr_head_sha)${newline}${newline}"
summary+="MERGE: [$pr_merge_sha](https://github.com/ROCm/rocm-systems/commit/$pr_merge_sha)${newline}${newline}"
summary+="### Pipelines triggered for this PR${newline}${newline}"
summary+="| Project | Run ID | Status |${newline}"
summary+="|--------------|--------|--------|${newline}"
@@ -301,7 +301,7 @@ jobs:
summary+="${newline}${newline}"
summary+="### Rerun instructions${newline}${newline}"
summary+="To request Azure to rerun jobs, click the \`Re-run all jobs\` button on the [corresponding \`Trigger Azure CI\` run](https://github.com/ROCm/rocm-libraries/actions/runs/${{ github.run_id }}).${newline}${newline}"
summary+="To request Azure to rerun jobs, click the \`Re-run all jobs\` button on the [corresponding \`Trigger Azure CI\` run](https://github.com/ROCm/rocm-systems/actions/runs/${{ github.run_id }}).${newline}${newline}"
summary+="If there are any pending runs for this PR, they will be cancelled, and new runs will be started.${newline}${newline}"
summary+="If there are no pending runs, but there are existing failed or cancelled runs for this PR and merge SHA, the existing runs will be rerun.${newline}${newline}"
summary+="Otherwise, new runs will be started.${newline}${newline}"
@@ -318,7 +318,7 @@ jobs:
text+="${run_ids[i]}=pending;"
done
gh_output=$(gh api repos/ROCm/rocm-libraries/check-runs \
gh_output=$(gh api repos/ROCm/rocm-systems/check-runs \
-f "name=Azure CI Summary" \
-f "head_sha=$pr_head_sha" \
-f "status=in_progress" \
@@ -327,4 +327,4 @@ jobs:
-f "output[text]=$text")
echo "Created summary check with ID: $(echo "$gh_output" | jq -r '.id')"
echo "Summary check URL: https://github.com/ROCm/rocm-libraries/pull/561/checks?check_run_id=$(echo "$gh_output" | jq -r '.id')"
echo "Summary check URL: https://github.com/ROCm/rocm-systems/pull/561/checks?check_run_id=$(echo "$gh_output" | jq -r '.id')"
+1 -1
Wyświetl plik
@@ -4,7 +4,7 @@ on:
workflow_dispatch:
env:
MONOREPO_URL: github.com/ROCm/rocm-libraries.git
MONOREPO_URL: github.com/ROCm/rocm-systems.git
MONOREPO_BRANCH: develop
jobs:
+4 -4
Wyświetl plik
@@ -31,7 +31,7 @@ jobs:
AMDGPU_FAMILIES: "gfx94X-dcgpu"
TEATIME_FORCE_INTERACTIVE: 0
steps:
- name: Generate a token for rocm-libraries
- name: Generate a token for rocm-systems
id: generate-token
uses: actions/create-github-app-token@df432ceedc7162793a195dd1713ff69aefc7379e # v2.0.6
with:
@@ -39,14 +39,14 @@ jobs:
private-key: ${{ secrets.APP_PRIVATE_KEY }}
owner: ${{ github.repository_owner }}
- name: "Checking out repository for rocm-libraries"
- name: "Checking out repository for rocm-systems"
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
sparse-checkout: |
.github
${{ inputs.subtree_checkout }}
token: ${{ steps.generate-token.outputs.token }}
- name: Checkout TheRock repository
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
@@ -66,7 +66,7 @@ jobs:
echo "Git version: $(git --version)"
git config --global --add safe.directory $PWD
git config fetch.parallel 10
- name: Fetch sources
run: |
./TheRock/build_tools/fetch_sources.py --jobs 12 --no-include-math-libs
@@ -33,7 +33,7 @@ jobs:
# To get a fast signal of windows building for TheRock, adding gfx110X
AMDGPU_FAMILIES: "gfx110X-dgpu"
steps:
- name: Generate a token for rocm-libraries
- name: Generate a token for rocm-systems
id: generate-token
uses: actions/create-github-app-token@df432ceedc7162793a195dd1713ff69aefc7379e # v2.0.6
with:
@@ -41,14 +41,14 @@ jobs:
private-key: ${{ secrets.APP_PRIVATE_KEY }}
owner: ${{ github.repository_owner }}
- name: "Checking out repository for rocm-libraries"
- name: "Checking out repository for rocm-systems"
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
sparse-checkout: |
.github
${{ inputs.subtree_checkout }}
token: ${{ steps.generate-token.outputs.token }}
- name: Checkout TheRock repository
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
+1 -1
Wyświetl plik
@@ -62,7 +62,7 @@ jobs:
# - name: Patch monorepo
# run: |
# python TheRock/build_tools/github_actions/patch_monorepo.py \
# --repo rocm-libraries
# --repo rocm-systems
- name: Detect changed subtrees
id: detect
@@ -6,7 +6,7 @@ on:
- cron: '0 * * * *'
env:
MONOREPO_URL: github.com/ROCm/rocm-libraries.git
MONOREPO_URL: github.com/ROCm/rocm-systems.git
MONOREPO_BRANCH: release-staging/rocm-rel-7.0
concurrency:
+1 -1
Wyświetl plik
@@ -6,7 +6,7 @@ on:
- cron: '0 * * * *'
env:
MONOREPO_URL: github.com/ROCm/rocm-libraries.git
MONOREPO_URL: github.com/ROCm/rocm-systems.git
MONOREPO_BRANCH: develop
concurrency:
+1 -1
Wyświetl plik
@@ -75,7 +75,7 @@ This table provides the current status of the migration of specific ROCm systems
Note TheRock CI performs multi-component testing on top of builds leveraging [TheRock](https://github.com/ROCm/TheRock) build system.
[![The Rock CI](https://github.com/ROCm/rocm-libraries/actions/workflows/therock-ci.yml/badge.svg?branch%3Adevelop+event%3Apush)](https://github.com/ROCm/rocm-libraries/actions/workflows/therock-ci.yml?query=branch%3Adevelop+event%3Apush)
[![The Rock CI](https://github.com/ROCm/rocm-systems/actions/workflows/therock-ci.yml/badge.svg?branch%3Adevelop+event%3Apush)](https://github.com/ROCm/rocm-systems/actions/workflows/therock-ci.yml?query=branch%3Adevelop+event%3Apush)
---
+1 -1
Wyświetl plik
@@ -3,7 +3,7 @@
> [!IMPORTANT]
> This document is currently in **draft** and may be subject to change.
This document is to detail the various continuous integration (CI) systems that are run on the rocm-libraries monorepo.
This document is to detail the various continuous integration (CI) systems that are run on the rocm-systems monorepo.
## Table of Contents
1. [Azure Pipelines](#azure-pipelines)
@@ -3,17 +3,17 @@
> [!IMPORTANT]
> This document is currently in **draft** and may be subject to change.
When a project has been migrated into the ROCm monorepo, day-to-day work happens on the monorepos `develop` branch.
When a project has been migrated into the ROCm monorepo, day-to-day work happens on the monorepos `develop` branch.
Down-stream teams, however, still consume the original (pre-monorepo) repositories, particularly their `release-staging/rocm-rel-x.y` branches, through a variety of mechanisms.
This document explains how to move a change from the monorepo into those release-staging branches while guaranteeing that every commit on a release-staging branch also exists in the monorepo.
This document explains how to move a change from the monorepo into those release-staging branches while guaranteeing that every commit on a release-staging branch also exists in the monorepo.
## 1. Land the change in the monorepo's develop branch
1. Create a pull request in `ROCm/rocm-libraries` that targets `develop`.
2. When merging, choose **Squash & Merge** (if the change can be represented as a single logical commit).
1. Create a pull request in `ROCm/rocm-systems` that targets `develop`.
2. When merging, choose **Squash & Merge** (if the change can be represented as a single logical commit).
Why? A single commit is easier to cherry-pick later.
Result: The commit is now on `ROCm/rocm-libraries:develop`.
Result: The commit is now on `ROCm/rocm-systems:develop`.
## 2. Cherry-pick into the monorepo's release-staging branch
@@ -26,11 +26,11 @@ $ git checkout -b cherry-pick-foo-rel-x.y origin/release-staging/rocm-rel-x.y
2. Cherry-pick the commit:
```
$ git cherry-pick abcd1234
$ git cherry-pick abcd1234
```
3. Resolve any merge conflicts (rare if the branch is close to develop).
4. Push the branch and open a PR that targets `ROCm/rocm-libraries:release-staging/rocm-rel-x.y`.
4. Push the branch and open a PR that targets `ROCm/rocm-systems:release-staging/rocm-rel-x.y`.
5. Request reviews, obtain approvals, and merge.
## 3. Wait for the automatic “fan-out” sync
@@ -39,29 +39,29 @@ Every ~15 minutes, a CI job copies new commits from the monorepo back into the c
After merging your PR:
1. Monitor the CI job or simply wait ~15 minutes.
1. Monitor the CI job or simply wait ~15 minutes.
2. Go to the original (pre-monorepo) repository and verify the commits have been reflected onto the `develop` and `release-staging/rocm-rel-x.y` branches.
## FAQ
Q : Can I cherry-pick multiple commits at once?
Q : Can I cherry-pick multiple commits at once?
A : Yes, but prefer a squash merge in the monorepo so you only need to pick one.
Q : What if the auto-sync hasnt copied the commit?
A : Verify the CI status in `rocm-libraries`. If failed, ask the infra team; the commit will re-sync after a successful run.
Q : What if the auto-sync hasnt copied the commit?
A : Verify the CI status in `rocm-systems`. If failed, ask the infra team; the commit will re-sync after a successful run.
Q : Can I push directly to the release-staging branch?
Q : Can I push directly to the release-staging branch?
A : No. Always go through a PR so CI and reviewers can validate the cherry-pick.
Q : What if commits have been pushed to develop that make a cherry-pick incompatible with release-staging?
Q : What if commits have been pushed to develop that make a cherry-pick incompatible with release-staging?
A : It's likely that this fix/change will also be landed in develop at some point, else we risk divergent features/support. So, it's recommended to still land the change in develop first, and cherry-pick to release-staging, resolving any merge conflicts that arise. If for some reason the develop branch has diverged so far from the release-staging for your component that a cherry-pick is irreconcilable, land the changes in develop and release-staging using fully separate PRs, and add references to the other for traceability.
## Summary
In short:
1. Merge change to monorepo `develop`.
2. Cherry-pick to monorepo `release-staging/rocm-rel-x.y`.
1. Merge change to monorepo `develop`.
2. Cherry-pick to monorepo `release-staging/rocm-rel-x.y`.
3. Wait for the fan-out sync and verify the changes are reflected in the original repository.
Following this process keeps release branches in sync with the monorepo while allowing critical fixes to flow to down-stream consumers.
+51 -51
Wyświetl plik
@@ -1,16 +1,16 @@
# Migration from Single Repo to Monorepo
## Introduction
This document outlines the process for migrating from a single library repository to this monorepo. It covers the necessary steps to ensure a smooth transition, including pre-conditions, conflict resolution, and changes to repository management.
## Pre-conditions
To ensure consistency and maintainability during the migration, the following pre-conditions must be satisfied:
# Migration from Single Repo to Monorepo
## Introduction
This document outlines the process for migrating from a single library repository to this monorepo. It covers the necessary steps to ensure a smooth transition, including pre-conditions, conflict resolution, and changes to repository management.
## Pre-conditions
To ensure consistency and maintainability during the migration, the following pre-conditions must be satisfied:
1. **Identify Next Repo to Migrate:**
- Please refer to the main [README.md](/README.md) on the order of repositories being migrated.
- This is usually discussed in advance in meetings with the technical leads of that project.
2. **Identify Branches and Pull Requests:**
2. **Identify Branches and Pull Requests:**
- Determine branches and active pull requests that will be affected by the migration.
- Typically, this is limited to the pull requests targeting `develop` and `release-staging` branches.
- Any point-fixes for previous releases will not be migrated over.
@@ -18,51 +18,51 @@ To ensure consistency and maintainability during the migration, the following pr
3. **Pause Merges:**
- There are GitHub Actions that automatically synchronize changes from the individual repos to the monorepo.
- These automated actions need to be paused by disabling the workflow on the GitHub UI.
- develop branch workflow: https://github.com/ROCm/rocm-libraries/actions/workflows/update-subtrees.yml
- release-staging branch workflow: https://github.com/ROCm/rocm-libraries/actions/workflows/update-release-staging-subtree.yml
- develop branch workflow: https://github.com/ROCm/rocm-systems/actions/workflows/update-subtrees.yml
- release-staging branch workflow: https://github.com/ROCm/rocm-systems/actions/workflows/update-release-staging-subtree.yml
- Announce the pause to key stakeholders and ask them to propagate the news.
## Migration Process
### Step 1: Pull Request Management
1. **Automated Import of Pull Requests:**
## Migration Process
### Step 1: Pull Request Management
1. **Automated Import of Pull Requests:**
- Pull requests without merge conflicts will be automatically imported with a GitHub Action, only executable by maintainers and admins.
- This GitHub action will create a feature branch on the monorepo, pulling in the changes from the PR on the original repo using `git subtree`.
- These imported pull requests will have the `imported pr` label applied.
- After running the action successfully, close the PR on the original repo.
- GitHub Action: https://github.com/ROCm/rocm-libraries/actions/workflows/pr-import.yml
- Example Imported Pull Request on Monorepo: https://github.com/ROCm/rocm-libraries/pull/206
- GitHub Action: https://github.com/ROCm/rocm-systems/actions/workflows/pr-import.yml
- Example Imported Pull Request on Monorepo: https://github.com/ROCm/rocm-systems/pull/206
- Corresponding Pull Request on Original Repo: https://github.com/ROCm/Tensile/pull/2135
2. **Conflict Resolution:**
- For pull requests with merge conflicts, add a comment explaining the merge conflict and blocking issue preventing import.
2. **Conflict Resolution:**
- For pull requests with merge conflicts, add a comment explaining the merge conflict and blocking issue preventing import.
- Collaborate with contributors to import these PRs after the migration period, or the contributor can reopen the pull request themselves on the monorepo.
3. **NPI Development:**
- Repeat this import process for the monorepo on GitHub EMU for npi work.
### Step 2: Issue and Comment Import
1. **Issue Import:**
### Step 2: Issue and Comment Import
1. **Issue Import:**
- Import all open issues from both public and EMU repositories with a GitHub Action, only executable by maintainers and admins.
- Comments are copied over in the imported issues.
- GitHub Action: https://github.com/ROCm/rocm-libraries/actions/workflows/issue-import.yml
- GitHub Action: https://github.com/ROCm/rocm-systems/actions/workflows/issue-import.yml
- Ensure issue status and labels are preserved during migration.
- Look for any weird unicode characters that get mangled during the automated import.
- After running the action successfully, close the issue with a comment on the original repo.
- Example Imported Issue on Monorepo: https://github.com/ROCm/rocm-libraries/issues/100
- Example Imported Issue on Monorepo: https://github.com/ROCm/rocm-systems/issues/100
- Corresponding Issue on Original Repo: https://github.com/ROCm/rocThrust/issues/501
### Step 3: Path-Based Commit History
1. **Use of Git Filter-Repo:**
### Step 3: Path-Based Commit History
1. **Use of Git Filter-Repo:**
- Utilize `git filter-repo` at the migration point to add path-based commit history.
- As changing the contents of a commit will change the output the hash function, commit SHA will change.
- The filter-repo tool is used to add a snippet at the end of the old commit to refer to the old commit SHA.
- It is not possible to preserve the same commit SHA if the metadata is changed to point to new paths, as the hash function output changes.
- Example directory view: https://github.com/ROCm/rocm-libraries/commits/develop/projects/rocrand/library
- Example commit view: https://github.com/ROCm/rocm-libraries/commit/ea8b6884a0f2a0ec80ff7811bc5ec042600790e9
- Example directory view: https://github.com/ROCm/rocm-systems/commits/develop/projects/rocrand/library
- Example commit view: https://github.com/ROCm/rocm-systems/commit/ea8b6884a0f2a0ec80ff7811bc5ec042600790e9
2. **command sequence example**
@@ -74,11 +74,11 @@ pushd hipBLAS-common
git checkout develop
git pull origin
git checkout -b filtered/hipblas-common
git filter-repo --path-rename '':'projects/hipblas-common/' --commit-callback "original_hash = commit.original_id.decode(); original_message = commit.message.decode(); new_message = f'{original_message}\\n\\n[ROCm/hipBLAS-common commit: {original_hash}]' if original_message.strip() else f'[ROCm/hipBLAS-common commit: {original_hash}]'; commit.message = new_message.encode()" --force
git remote add monorepo git@github.com:ROCm/rocm-libraries.git
git filter-repo --path-rename '':'projects/hipblas-common/' --commit-callback "original_hash = commit.original_id.decode(); original_message = commit.message.decode(); new_message = f'{original_message}\\n\\n[ROCm/hipBLAS-common commit: {original_hash}]' if original_message.strip() else f'[ROCm/hipBLAS-common commit: {original_hash}]'; commit.message = new_message.encode()" --force
git remote add monorepo git@github.com:ROCm/rocm-systems.git
git push monorepo filtered/hipblas-common
popd
git clone git@github.com:ROCm/rocm-libraries.git
git clone git@github.com:ROCm/rocm-systems.git
git checkout develop
git pull origin
git branch backup/develop-hipblas-common
@@ -95,17 +95,17 @@ git push origin develop
```
### Step 4: CI/CD Triggers
1. **CI/CD Trigger Points:**
- Modify the existing CI/CD systems to be triggered off changes to this project in the monorepo.
### Step 5: Repository Adjustments
1. **Default Branch Deprecation:**
1. **Default Branch Deprecation:**
- Change the default branch of the original repository with a clear deprecation notice.
- Example: https://github.com/ROCm/rocPRIM/tree/develop_deprecated
2. **Disable Dependabot Updates:**
2. **Disable Dependabot Updates:**
- Cease automatic dependency updates in the old repository to streamline the focus on the monorepo.
- Clear the contents in this file on the original repo: https://github.com/ROCm/rocPRIM/blob/develop_deprecated/.github/dependabot.yml
- In the original repo settings, go to Security -> Advanced Security and disable all the Dependabot settings.
@@ -120,26 +120,26 @@ git push origin develop
- Update the true/false values in the [`repos-config.json`](/.github/repos-config.json) file that automated workflows use to determine which way the source gets synchronized..
- `auto_subtree_pull` should now be false, `auto_subtree_push` should now be true for this migrated project.
- Make this change on both the `develop` and `release-staging` branches.
- https://github.com/ROCm/rocm-libraries/blob/develop/.github/repos-config.json
- https://github.com/ROCm/rocm-libraries/blob/release-staging/rocm-rel-7.0/.github/repos-config.json
- https://github.com/ROCm/rocm-systems/blob/develop/.github/repos-config.json
- https://github.com/ROCm/rocm-systems/blob/release-staging/rocm-rel-7.0/.github/repos-config.json
2. **Update the monorepo README.md:**
- Update the migration status on the monorepo's main readme to indicate the migration has been completed.
- https://github.com/ROCm/rocm-libraries/blob/develop/README.md
- https://github.com/ROCm/rocm-systems/blob/develop/README.md
## Post-Migration Activities
## Post-Migration Activities
1. **Re-enable synchronization jobs:**
- Re-enable any automated workflows that were paused.
2. **Communication:**
- Communicate to key stakeholders the successful completion of the migration.
- Continue daily meetings and active written communications to offer support for any issues that arise.
- Continue daily meetings and active written communications to offer support for any issues that arise.
3. **Automated Patching of Original Repos:**
- During the migration period, when a pull request is merged on the monorepo, the contents of the pull request will be split into patches to be pushed onto the original repos.
- This supports potential pull requests that touch multiple projects.
- Example pull request on the monorepo: https://github.com/ROCm/rocm-libraries/pull/230
- Example pull request on the monorepo: https://github.com/ROCm/rocm-systems/pull/230
- Corresponding patches on the original repos:
- https://github.com/ROCm/hipCUB/commit/50438ec4971def627729ea3d9dc1485e52b09e48
- https://github.com/ROCm/hipRAND/commit/74afe303def580290a8e5b149ea13ae739bc4c61
@@ -149,8 +149,8 @@ git push origin develop
4. **Monitoring:**
- Monitor the monorepo for any issues or discrepancies.
- If the automated patching for a PR failed to make it to the original repo, use this GitHub Action: https://github.com/ROCm/rocm-libraries/actions/workflows/pr-merge-sync-patches-manual.yml
## Conclusion
This migration process aims to assist the ROCm development teams transition from many repos to a monorepo by addressing the topics above. By following these outlined steps, we aim to maintain and improve the quality of our development workflow post-migration.
- If the automated patching for a PR failed to make it to the original repo, use this GitHub Action: https://github.com/ROCm/rocm-systems/actions/workflows/pr-merge-sync-patches-manual.yml
## Conclusion
This migration process aims to assist the ROCm development teams transition from many repos to a monorepo by addressing the topics above. By following these outlined steps, we aim to maintain and improve the quality of our development workflow post-migration.