Rebuilding Git Integration Environment

About ten days ago, after getting to the office, I found that my home box that I use as the primary integration machine was not responding. Later in the day, wife calls from home saying that the box is dead and does not even respond to the power button.

So until I had a chance to look at the box and resurrect it, I had to build the Git Integration Environment elsewhere. As Linus famously said once, only wimps use backup and real men just upload their important stuff and let the rest of the world mirror it ;-). All of my integration branches are pushed to multiple places, and one of the public repositories even have all of the topic branches, so it is just the matter of fetching them back. Other people could even try to imitate what I do by following this procedure.

First, grab all the branches I had in the primary box from the mirror:

$ git init git.git
$ cd git.git
$ git fetch --update-head-ok git://github.com/gitster/git refs/*:refs/*
$ git reset --hard master

Unfortunately, GitHub adds garbage refs to my repository that I didn't push; I can eradicate them like this, but this step is not strictly necessary:

$ for x in $(git for-each-ref --format='%(refname)' refs/pull/)

  do

    git update-ref -d $x

  done

After this, I can start building Git from tips of topic branches as usual, but my usual procedure is to make heavy use of helper scripts in the 'todo' branch that is checked out in the Meta directory (e.g. I run Meta/Make instead of make from the top-level, and that Make script is in the 'todo' branch).

$ git init Meta

$ (cd Meta && git pull git://git.kernel.org/pub/scm/git/git.git todo:master)
$ Meta/Make test install

Among the helper scripts on the 'todo' branch (and checked out in the Meta directory) is the Reintegrate script.  This is used to maintain redo-jch.sh and redo-pu.sh scripts I rebuild 'jch' and 'pu' branches with.

$ Meta/Reintegrate master..jch >Meta/redo-jch.sh

$ Meta/Reintegrate jch..pu >Meta/redo-pu.sh

$ chmod +x Meta/redo-*.sh

What is still missing after the above is the contents of the rerere database. This mechanism remembers the manual conflict resolution I had to perform when I built the integration branches (i.e. 'next', 'pu' and 'jch') earlier. Luckily, there is a script to do so in contrib/ since Sep 2008:

$ PATH=$PATH:. contrib/rerere-train.sh master..pu

This populates the rerere database by learning from the merge commits between 'master' and 'pu' branches, so after that, I can try rebuilding 'pu', like this:

$ git checkout master^0 ;# I am not really rebuilding--just "try rebuilding"

$ Meta/redo-jch.sh ;# replay all the merges that should go to the jch branch

$ Meta/redo-pu.sh ;# and then the merges to the pu branch on top

$ git diff HEAD pu ;# there should be no differences

After this, I can look at individual topics, decide which topics are worthy to be merged to 'next' or 'master', and advance tips of these integration branches with appropriate invocations of the git merge commands.

By the way, it turns out that the breakage was a failed power supply. I ordered one from NewEgg, which arrived early last week, but I procrastinated until this morning to replace it. After the necessary hardware swap,  running the above procedure again to get the environment back was more or less trivial.

So, it's Baaack ;-)

Git 1.7.7.7, 1.7.8.6 and 1.7.9.7

New releases for the three older maintenance tracks have been tagged and tarballs uploaded. They were mostly to push out small documentation fixes that have already been in the 'master' branch for the next feature release, except for v1.7.9.7, which has a fix for performance regression in "git fetch" introduced some time ago where the command wastes cycles to verify the connectivity of objects received from the other end, being unnecessarily pessimistic.

Git 1.7.10

After slipping for about a week, 1.7.10 final has been tagged.

The most notable change is that from this release, git merge will start opening the editor just like git commit does. See earlier articles for details.


Git v1.7.10 Release Notes

=========================


Compatibility Notes
-------------------


 * From this release on, the "git merge" command in an interactive
   session will start an editor when it automatically resolves the
   merge for the user to explain the resulting commit, just like the
   "git commit" command does when it wasn't given a commit message.


   If you have a script that runs "git merge" and keeps its standard
   input and output attached to the user's terminal, and if you do not
   want the user to explain the resulting merge commits, you can
   export GIT_MERGE_AUTOEDIT environment variable set to "no", like
   this:


        #!/bin/sh
        GIT_MERGE_AUTOEDIT=no
        export GIT_MERGE_AUTOEDIT


   to disable this behavior (if you want your users to explain their
   merge commits, you do not have to do anything).  Alternatively, you
   can give the "--no-edit" option to individual invocations of the
   "git merge" command if you know everybody who uses your script has
   Git v1.7.8 or newer.


 * The "--binary/-b" options to "git am" have been a no-op for quite a
   while and were deprecated in mid 2008 (v1.6.0).  When you give these
   options to "git am", it will now warn and ask you not to use them.


 * When you do not tell which branches and tags to push to the "git
   push" command in any way, the command used "matching refs" rule to
   update remote branches and tags with branches and tags with the
   same name you locally have.  In future versions of Git, this will
   change to push out only your current branch according to either the
   "upstream" or the "current" rule.  Although "upstream" may be more
   powerful once the user understands Git better, the semantics
   "current" gives is simpler and easier to understand for beginners
   and may be a safer and better default option.  We haven't decided
   yet which one to switch to.




Updates since v1.7.9
--------------------


UI, Workflows & Features


 * various "gitk" updates.
   - show the path to the top level directory in the window title
   - update preference edit dialog
   - display file list correctly when directories are given on command line
   - make "git-describe" output in the log message into a clickable link
   - avoid matching the UNIX timestamp part when searching all fields
   - give preference to symbolic font names like sans & monospace
   - allow comparing two commits using a mark
   - "gitk" honors log.showroot configuration.


 * Teams for localizing the messages from the Porcelain layer of
   commands are starting to form, thanks to Jiang Xin who volunteered
   to be the localization coordinator.  Translated messages for
   simplified Chinese, Swedish and Portuguese are available.


 * The configuration mechanism learned an "include" facility; an
   assignment to the include.path pseudo-variable causes the named
   file to be included in-place when Git looks up configuration
   variables.


 * A content filter (clean/smudge) used to be just a way to make the
   recorded contents "more useful", and allowed to fail; a filter can
   now optionally be marked as "required".


 * Options whose names begin with "--no-" (e.g. the "--no-verify"
   option of the "git commit" command) can be negated by omitting
   "no-" from its name, e.g. "git commit --verify".


 * "git am" learned to pass "-b" option to underlying "git mailinfo", so
   that a bracketed string other than "PATCH" at the beginning can be kept.


 * "git clone" learned "--single-branch" option to limit cloning to a
   single branch (surprise!); tags that do not point into the history
   of the branch are not fetched.

 * "git clone" learned to detach the HEAD in the resulting repository

   when the user specifies a tag with "--branch" (e.g., "--branch=v1.0").

   Clone also learned to print the usual "detached HEAD" advice in such

   a case, similar to "git checkout v1.0".

 * When showing a patch while ignoring whitespace changes, the context

   lines are taken from the postimage, in order to make it easier to

   view the output.

 * "git diff --stat" learned to adjust the width of the output on

   wider terminals, and give more columns to pathnames as needed.

 * "diff-highlight" filter (in contrib/) was updated to produce more

   aesthetically pleasing output.

 * "fsck" learned "--no-dangling" option to omit dangling object

   information.

 * "git log -G" and "git log -S" learned to pay attention to the "-i"

   option.  With "-i", "log -G" ignores the case when finding patch

   hunks that introduce or remove a string that matches the given

   pattern.  Similarly with "-i", "log -S" ignores the case when

   finding the commit the given block of text appears or disappears

   from the file.

 * "git merge" in an interactive session learned to spawn the editor

   by default to let the user edit the auto-generated merge message,

   to encourage people to explain their merges better. Legacy scripts

   can export GIT_MERGE_AUTOEDIT=no to retain the historical behavior.

   Both "git merge" and "git pull" can be given --no-edit from the

   command line to accept the auto-generated merge message.

 * The advice message given when the user didn't give enough clue on

   what to merge to "git pull" and "git merge" has been updated to

   be more concise and easier to understand.

 * "git push" learned the "--prune" option, similar to "git fetch".

 * The whole directory that houses a top-level superproject managed by

   "git submodule" can be moved to another place.

 * "git symbolic-ref" learned the "--short" option to abbreviate the

   refname it shows unambiguously.

 * "git tag --list" can be given "--points-at

This time, it will be really final rc before Git 1.7.10

The upcoming version 1.7.10 of Git was scheduled to be tagged on Sunday, but I did not want to make important decisions or announces over the weekend to avoid making people wonder if anything I say is an April Fools joke.

That was an excuse for slipping the release a bit, but as a nice side effect, it paid off. Over the weekend, a few patches and a pull request worthy to be in the upcoming release came. A couple of rather embarrassing last-minute bugs were found and fixed in the updated gitk, and we have Portuguese translation of the messages.

So I am issuing another release candidate 1.7.10-rc4, hoping that people can test it out to find any other last-minute regressions, so that we can release the real 1.7.10 by the end of this week.

Hopefully Final rc before Git 1.7.10

Git 1.7.10-rc3 has been tagged and users are strongly encouraged to download and test it before the final release happens.  The tarballs are found at http://code.google.com/p/git-core/downloads/list and people fetching them using Git can find it in my public repositories.


Among the many improvements in the upcoming release is an update to the git merge command. While this has been received very favorably, it may surprise you when you run it for the first time, and is worth mentioning.


Traditionally, when the git merge command attempted to merge two or more histories, it prepared a canned commit log message based on what is being merged, and recorded the result in a merge commit without any user intervention, if the automatic merge logic successfully computed the resulting tree without conflict. When the automatic logic did not manage to come up with a clean merge result, it gave up, leaving the conflicted state in the index and in the working tree files, and asked the user to resolve them and run the git commit command to record the outcome.

Most merges do cleanly resolve, and this behavior resulted in people making their merges too easily and lightly, even when the reason why the merge was made in the first place  should be explained. Nobody explained why the merge was made in a merge commit, because in order to do so after git merge already made the commit, they have to go back and run git commit --amend to do so.

Recently in a discussion on the Git mailing list, Linus admitted (and I agreed) that this was one of the design mistakes we made early in the history of Git. And in 1.7.10 and later, the git merge command that is run in an interactive session (i.e. both its standard input and its standard output connected to a terminal) will open an editor before creating a commit to record the merge result, to give the user a chance to explain the merge, just like the git commit command the user runs after resolving a conflicted merge already does.


A quick way to update your scripts that shouldn't let its users explain the merges created by them is to set and export an environment variable, like this.

#!/bin/sh

GIT_MERGE_AUTOEDIT=no
export GIT_MERGE_AUTOEDIT

# whatever your script did originally

...

git merge foo

git merge bar

...

For a more detailed discussion, see these earlier posts.

Git 1.7.9.5

Well, I lied when I said 1.7.9.4 would be the last maintenance release before the 1.7.10 final.
Here is a small update that contains mostly documentation fixes with a couple of back-merged low impact fixes that have been cooking on the 'master' front to become part of the upcoming 1.7.10 release.

Have fun.

Git 1.7.10-rc2

Not much has changed since the previous rc, but you know the drill.
Please test to catch any last minute regressions.


The release tarballs are found at:

    http://code.google.com/p/git-core/downloads/list

and their SHA-1 checksums are:

b32514ad69bb3100b6b5038bc88798d56ebe1e1d  git-1.7.10.rc2.tar.gz

f66bc63ed3e98df6c7ad205e06878e9cfc9084fc  git-htmldocs-1.7.10.rc2.tar.gz

130601149f97e9414467bb3d6a722aa37b8205af  git-manpages-1.7.10.rc2.tar.gz

Compatibility Notes

From this release on, the "git merge" command in an interactive session will start an editor when it automatically resolves the merge for the user to explain the resulting commit, just like the "git commit" command does when it wasn't given a commit message. If you have a script that runs "git merge" and keeps its standard input and output attached to the user's terminal, and if you do not want the user to explain the resulting merge commits, you can export GIT_MERGE_AUTOEDIT environment variable set to "no", like this:

   

        #!/bin/sh

        GIT_MERGE_AUTOEDIT=no

        export GIT_MERGE_AUTOEDIT

to disable this behaviour (if you want your users to explain their   merge commits, you do not have to do anything).  Alternatively, you can give the "--no-edit" option to individual invocations of the "git merge" command if you know everybody who uses your script has Git v1.7.8 or newer.

The "--binary/-b" options to "git am" have been a no-op for quite a while and were deprecated in mid 2008 (v1.6.0).  When you give these options to "git am", it will now warn and ask you not to use them.

When you do not tell which branches and tags to push to the "git push" command in any way, the command used "matching refs" rule to update remote branches and tags with branches and tags with the same name you locally have.  In future versions of Git, this will change to use the "upstream" rule to update the branch at the remote you would "pull" from into your current branch with your local current branch.  The release after 1.7.10 will start issuing a warning about this change, to encourage you to tell the command what to push out, e.g. by setting push.default configuration.

Toilet Roll Effect

To put this in a more formal-sounding way, it can be observed when there are two independent cues that stop or start your behavior, and the behavior is something whose change tends to be one-way.

In order to wipe certain parts after you have finished doing certain things, you wind some length of paper around your hands, cut the paper from the roll, and use it.

Let's say you are used to use 1-ply toilet rolls, but you noticed that 2-ply rolls were on sale at the market, so that is what you bought recently.

What happens?

Because you are so used to the hand-motion to wind the 1-ply toilet rolls (say, you always wind 4 times), you still wind the same length of the stuff. Your backside starts to be comfy with the extra fluffiness your hands are giving it by using more paper. After all, you are winding the thicker 2-ply paper the same number of times. You may start winding the stuff a bit fewer number of times, but the rate of such decrease is slower. Perhaps you learn to wind only 3 times, but it would take time for you to go down to 2, which is the half of the original.

Next week, you notice that 1-ply rolls are on sale, so you switch again. By this time, your hands are used to winding 3 times, but your backside now complains because it no longer is rewarded by the same fluffiness. After all, you are giving it less paper. So you end up start winding the stuff even more, until your backside gets comfy enough. Your hands may learn to wind 5 times by the time this happens.

Notice that there are two cues that makes you stop winding when you do this. How many times you wind the paper around your hand, and how much fluffiness your backside feels by being wiped by it. And the change in the amount of paper you would use tends to be one-way (using more is easier).

Imagine if you switch to 2-ply rolls again. And then switch back to 1-ply rolls. The consumption of your toilet rolls tends to increase, and increase more if you switch between 1-ply and 2-ply rolls more often.

Exactly the same thing happens to cigarette usage, by the way. Two competing cues are how often you take a break, and how much chemical effect you get from a puff. Start from a weak brand, and your break schedule may settle for a break per 3 hours. Switch to a stronger brand, and your body gets used to greater chemical effect with the same 1 break per 3 hours schedule. Switch back to the weaker brand, and now your body will complain and wants more nicotine, and your break schedule ends up being more frequent. Switch back to the stronger one again, with the more frequent schedule, your body gets trained to more nicotine. Rinse and repeat...

Please discuss: what "git push" should do when you do not say what to push?

[Update is here]


There is a proposal to change the default behaviour of git push on the Git mailing list. The goal of this message is to encourage you to discuss it before it happens (or the change is aborted, depending on the outcome of the discussion).

In the current setting (i.e. push.default=matching), git push without argument will push all branches that exist locally and remotely with the same name. This is usually appropriate when a developer pushes to his own public repository, but may be confusing if not dangerous when using a shared repository. The proposal is to change the default to 'upstream', i.e. push only the current branch, and push it to the branch git pull would pull from. Another candidate is 'current'; this pushes only the current branch to the remote branch of the same name.

For more details on the behavior of Git with these values, read the documentation about push.default in 'man git-config'.

You may be negatively affected when such a change happens if you do not see anything in the output from git config push.default and if you rely on the default that pushes all your matching branches. On the other hand, you may want to see the default behaviour to change, especially if you are using shared repositories. In either case, please join the discussion to give us more data point and help us decide the future of Git. Also, if you think your friends and colleagues will be affected by this change, either positively or negatively, please tell them about this discussion.

What has been discussed so far can be seen in this thread:

http://thread.gmane.org/gmane.comp.version-control.git/192547/focus=192694

Previous relevant discussions include:

http://thread.gmane.org/gmane.comp.version-control.git/123350/focus=123541

http://thread.gmane.org/gmane.comp.version-control.git/166743

To join the discussion, send your messages to:

git@vger.kernel.org

The list accepts messages from non-subscribers, and you do not have to ask "please Cc me, I am not subscribed", as it's customary to Cc: posters when replying on this list.

Thanks.

---

Administrivia: Comments on this post is disabled; the discussion should happen on the mailing list.

Summary of discussion on "git push" default change

So far, the messages on the git mailing list show that the proposed change of the default away from the traditional 'matching' is received overwhelmingly positively.

We've known it from the beginning that matching is not for many people, and definitely not for new people (otherwise, we wouldn't have made noises about it in the first place). It was not even the primary objective of the discussion thread to decide if we are going to switch away from 'matching' by voting. Waking up those who are sleeping, so that they do not have to be surprised with "I didn't know that was happening!" was.

The new default we are switching to is not about how many people prefer it for their own use. It is about what default is the least confusing to the new users. A default whose behaviour is easy to explain, easy to follow and easy to understand is the goal. Once people understand what they want and realize the hardcoded default does not work well for them, it is easy for them to configure their push.default to something else, like 'matching'. And 'matching' was a bad default for that purpose; it was the hardest to explain and understand in the context of the workflows of many new people.

Many said that they like 'upstream' solely based on their personal preference, but a few people did justify their preference of 'upstream' over 'current' based on their experience in teaching new people and observing the sharp edges that hurt them. And they all sounded reasonable.

In order to show how the world after phase #2 of the transition [1] would look like to developers, testers and early adopters, I am planning to merge Matthieu's patch mm/push-default-switch-warning topic [2] to the 'next' branch, together with Christopher's ct/advise-push-default topic [3]; hopefully these topics can be merged to the master branch soon after 1.7.10 final.

Thanks.

To people who helped spreading the initial RFD message: please do feel free to distribute this message to the same channels, too.

[References]

1 http://article.gmane.org/gmane.comp.version-control.git/193308
2 https://github.com/gitster/git/commit/5293b54
3 https://github.com/gitster/git/commit/f25950f