From eb8ede7f659da6c07d1f475aff9b0d46e5b49a79 Mon Sep 17 00:00:00 2001 From: Eduardo Broto Date: Sun, 23 Aug 2020 11:29:06 +0200 Subject: [PATCH 1/2] Improve documentation related to the sync process --- CONTRIBUTING.md | 56 +++++++++++++++++++++++++++++++++++++++++-------- 1 file changed, 47 insertions(+), 9 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 5f7b1e85ee9..631064cd92c 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -189,6 +189,35 @@ Clippy in the `rust-lang/rust` repository. For general information about `subtree`s in the Rust repository see [Rust's `CONTRIBUTING.md`][subtree]. +### Patching git-subtree to work with big repos + +Currently there's a bug in `git-subtree` that prevents it from working properly +with the [`rust-lang/rust`] repo. There's an open PR to fix that, but it's stall. +Before continuing with the following steps, we need to manually apply that fix to +our local copy of `git-subtree`. + +You can get the patched version of `git-subtree` from [here][gitgitgadget-pr]. +Put this file under `/usr/lib/git-core` (taking a backup of the previous file) +and make sure it has the proper permissions: + +```bash +sudo cp --backup /path/to/patched/git-subtree.sh /usr/lib/git-core/git-subtree +sudo chmod --reference=/usr/lib/git-core/git-subtree~ /usr/lib/git-core/git-subtree +sudo chown root:root /usr/lib/git-core/git-subtree +``` + +_Note:_ The first time running `git subtree push` a cache has to be built. This +involves going through the complete Clippy history once. For this you have to +increase the stack limit though, which you can do with `ulimit -s 60000`. +Make sure to run the `ulimit` command from the same session you call git subtree. + +_Note:_ If you are a Debian user, `dash` is the shell used by default for scripts instead of `sh`. +This shell has a hardcoded recursion limit set to 1000. In order to make this process work, +you need to force the script to run `bash` instead. You can do this by editing the first +line of the `git-subtree` script and changing `sh` to `bash`. + +### Performing the sync + Here is a TL;DR version of the sync process (all of the following commands have to be run inside the `rust` directory): @@ -198,6 +227,7 @@ to be run inside the `rust` directory): # Make sure to change `your-github-name` to your github name in the following command git subtree push -P src/tools/clippy git@github.com:your-github-name/rust-clippy sync-from-rust ``` + _Note:_ This will directly push to the remote repository. You can also push to your local copy by replacing the remote address with `/path/to/rust-clippy` directory. @@ -213,14 +243,28 @@ to be run inside the `rust` directory): 3. Open a PR to `rust-lang/rust-clippy` and wait for it to get merged (to accelerate the process ping the `@rust-lang/clippy` team in your PR and/or ~~annoy~~ ask them in the [Discord] channel.) -4. Sync the `rust-lang/rust-clippy` master to the rust-copy of Clippy: + +### Syncing back changes in Clippy to [`rust-lang/rust`] + +To avoid flooding the [`rust-lang/rust`] PR queue, changes in Clippy's repo are synced back +in a bi-weekly basis if there's no urgent changes. This is done starting on the day of +the Rust stable release and then every two other weeks. That way we guarantee that +every feature in Clippy is available for 2 weeks in nightly, before it can get to beta. +For reference, the first sync following this cadence was performed the 2020-08-27. + +1. Make sure Clippy itself is up-to-date by following the steps outlined in the previous +section if necessary. + +2. Sync the `rust-lang/rust-clippy` master to the rust-copy of Clippy: ```bash git checkout -b sync-from-clippy git subtree pull -P src/tools/clippy https://github.com/rust-lang/rust-clippy master ``` -5. Open a PR to [`rust-lang/rust`] +3. Open a PR to [`rust-lang/rust`] -Also, you may want to define remotes, so you don't have to type out the remote +### Defining remotes + +You may want to define remotes, so you don't have to type out the remote addresses on every sync. You can do this with the following commands (these commands still have to be run inside the `rust` directory): @@ -241,12 +285,6 @@ You can then sync with the remote names from above, e.g.: $ git subtree push -P src/tools/clippy clippy-local sync-from-rust ``` -_Note:_ The first time running `git subtree push` a cache has to be built. This -involves going through the complete Clippy history once. For this you have to -increase the stack limit though, which you can do with `ulimit -s 60000`. For -this to work, you will need the fix of `git subtree` available -[here][gitgitgadget-pr]. - [gitgitgadget-pr]: https://github.com/gitgitgadget/git/pull/493 [subtree]: https://rustc-dev-guide.rust-lang.org/contributing.html#external-dependencies-subtree [`rust-lang/rust`]: https://github.com/rust-lang/rust From b05077ea75f4585551650fc2d40a196ae437393d Mon Sep 17 00:00:00 2001 From: Eduardo Broto Date: Sun, 23 Aug 2020 22:20:55 +0200 Subject: [PATCH 2/2] Apply suggestions from PR review --- CONTRIBUTING.md | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 631064cd92c..54777810abb 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -192,7 +192,7 @@ For general information about `subtree`s in the Rust repository see [Rust's ### Patching git-subtree to work with big repos Currently there's a bug in `git-subtree` that prevents it from working properly -with the [`rust-lang/rust`] repo. There's an open PR to fix that, but it's stall. +with the [`rust-lang/rust`] repo. There's an open PR to fix that, but it's stale. Before continuing with the following steps, we need to manually apply that fix to our local copy of `git-subtree`. @@ -203,7 +203,7 @@ and make sure it has the proper permissions: ```bash sudo cp --backup /path/to/patched/git-subtree.sh /usr/lib/git-core/git-subtree sudo chmod --reference=/usr/lib/git-core/git-subtree~ /usr/lib/git-core/git-subtree -sudo chown root:root /usr/lib/git-core/git-subtree +sudo chown --reference=/usr/lib/git-core/git-subtree~ /usr/lib/git-core/git-subtree ``` _Note:_ The first time running `git subtree push` a cache has to be built. This @@ -248,9 +248,11 @@ to be run inside the `rust` directory): To avoid flooding the [`rust-lang/rust`] PR queue, changes in Clippy's repo are synced back in a bi-weekly basis if there's no urgent changes. This is done starting on the day of -the Rust stable release and then every two other weeks. That way we guarantee that +the Rust stable release and then every other week. That way we guarantee that every feature in Clippy is available for 2 weeks in nightly, before it can get to beta. -For reference, the first sync following this cadence was performed the 2020-08-27. +For reference, the first sync following this cadence was performed the 2020-08-27. + +All of the following commands have to be run inside the `rust` directory. 1. Make sure Clippy itself is up-to-date by following the steps outlined in the previous section if necessary.