tests/test-mq.out

named branches: server branchmap wire protocol support (issue736)

The repository command, 'branchmap', returns a dictionary, branchname
-> [branchheads], and will be implemented for localrepo, httprepo and
sshrepo.

The following wire format is used for returning data:

branchname1 branch1head2 branch1head2 ...
branchname2 ...
...

Branch names are URL encoded to escape white space, and branch heads
are sent as hex encoded node ids. All branches and all their heads are
sent.

The background and motivation for this command is the desire for a
richer named branch semantics when pushing changesets. The details are
explained in the original proposal which is included below.


1. BACKGROUND

The algorithm currently implemented in Mercurial only considers the
graph theoretical heads when determining whether new heads are
created, rather than using the branch heads as a count (the algorithm
considers a branch head effectively closed when it is merged into
another branch or a new named branch is started from that point
onward).

Our particular problem with the algorithm is that we'd like to see the
following case working without forcing a push:

Upsteam has:

(0:dev) ---- (1:dev)
\
 `--- (2:stable)

Someone merges stable into dev:

(0:dev) ---- (1:dev) ------(3:dev)
\                         /
 `--- (2:stable) --------´

This can be pushed without --force (as it should).
Now someone else does some coding on stable (a bug fix, say):

(0:dev) ---- (1:dev) ------(3:dev)
\                          /
 `--- (2:stable) ---------´---------(4:stable)

This time we need --force to push.

We allow this to be pushed without using --force by getting all the
remote branch heads (by extending the wire protocol with a new
function).

We would, furthermore, also prefer if it is impossible to push a new
branch without --force (or a later --newbranch option so --force isn't
shoe-horned into too many disparate functions, if need be), except of
course in the case where the remote repository is empty.

This is what our patches accomplish.


2. ALTERNATIVES

We have, of course, considered some alternatives to reconstructing
enough information to decide whether we are creating new remote branch
heads, before we added the new wire protocol command.

2.1. LOOKUP ON REMOTE

The main alternative is to use the information from remote.heads() and
remote.lookup() to try to reconstruct enough graph information to
decide whether we are creating new heads. This is not adequate as
illustrated below.

Remember that each lookup is typically a request-response pair over
SSH or HTTP(S).

If we have a simple repository at the remote end like this:

(0:dev) ---- (1:dev) ---- (3:stable)
\
 `--- (2:dev)

then remote.heads() will yield [2, 3]. Assume we have nodes [0, 1, 2]
locally and want to create a new node, 4:dev, as a descendant from
(1:dev), which should be OK as 1:dev is a branch head.

If we do remote.lookup('dev') we will get [2]. Thus, we can get
information about whether a branch exists on the remote server or not,
but this does not solve our problem of figuring out whether we are
creating new heads or not.

Pushing 4:dev ought to be OK, since after the push, we still only have
two heads on branch a.

Using remote.lookup() and remote.heads() is thus not adequate to
consistently decide whether we are creating new remote heads (e.g. in
this situation the latter would never return 1:dev).

2.2. USING INCOMING TO RECONSTRUCT THE GRAPH

An alternative would be to use information equivalent to hg incoming
to get the full remote graph in addition to the local graph.

To do this, we would have to get a changegroup(subset) bundle
representing the remote end (which may be a substantial amount of
data), getting the branch heads from an instantiated bundlerepository,
deleting the bundle, and finally, we can compute the prepush logic.

While this is backwards compatible, it will cause a possibly
substantial slowdown of the push command as it first needs to pull in
all changes.


3. FURTHER ARGUMENTS IN FAVOUR OF THE BRANCHMAP WIRE-PROTOCOL EXTENSION

Currently, the commands incoming and pull, work based on the tip of a
given branch if used with "-r branchname", making it hard to get all
revisions of a certain branch only (if it has multiple heads). This
can be solved by requesting the remote's branchheads and letting the
revisions to be used with the command be these heads. This can be done
by extending the commands with a new option, e.g.:

hg pull -b branchname

which will be turned into the equivalent of:

hg pull -r branchhead1 -r branchhead2 -r branchhead3

We have a simple follow-up patch that can do this ready as well
(although not submitted yet as it is pending the acceptance of the
branch patch).


4. WRAP-UP

We generally find that the branchmap wire protocol extension can
provide better named branch support to Mercurial. Currently, some
things, like the initial push scenario in this mail, are fairly
counter-intuitive, and the more often you have to force push, the more
it is likely you will get a lot of spurious and unnecessary merge
nodes. Also, restricting incoming and pull to all changes on a branch
rather than changes on the tip-most head would be a sensible extension
to making named branches a first class citizen in Mercurial.
Currently, named branches sometimes feel like a late-coming unwanted
step-child.

We have run it in a production environment for a while, with fewer
multiple heads occurring in our repositories and fewer confused users
as a result.

Also, it fixes the long-standing issue 736.

Co-contributor: Sune Foldager <[e-mail]>

% help
mq extension - patch management and development

This extension lets you work with a stack of patches in a Mercurial
repository. It manages two stacks of patches - all known patches, and
applied patches (subset of known patches).

Known patches are represented as patch files in the .hg/patches
directory. Applied patches are both patch files and changesets.

Common tasks (use "hg help command" for more details):

prepare repository to work with patches   qinit
create new patch                          qnew
import existing patch                     qimport

print patch series                        qseries
print applied patches                     qapplied
print name of top applied patch           qtop

add known patch to applied stack          qpush
remove patch from applied stack           qpop
refresh contents of top applied patch     qrefresh

list of commands:

 qapplied     print the patches already applied
 qclone       clone main and patch repository at same time
 qcommit      commit changes in the queue repository
 qdelete      remove patches from queue
 qdiff        diff of the current patch and subsequent modifications
 qfinish      move applied patches into repository history
 qfold        fold the named patches into the current patch
 qgoto        push or pop patches until named patch is at top of stack
 qguard       set or print guards for a patch
 qheader      print the header of the topmost or specified patch
 qimport      import a patch
 qinit        init a new queue repository
 qnew         create a new patch
 qnext        print the name of the next patch
 qpop         pop the current patch off the stack
 qprev        print the name of the previous patch
 qpush        push the next patch onto the stack
 qrefresh     update the current patch
 qrename      rename a patch
 qrestore     restore the queue state saved by a revision
 qsave        save current queue state
 qselect      set or print guarded patches to push
 qseries      print the entire series file
 qtop         print the name of the current patch
 qunapplied   print the patches not yet applied
 strip        strip a revision and all its descendants from the repository

enabled extensions:

 mq   patch management and development

use "hg -v help mq" to show aliases and global options
adding a
updating working directory
1 files updated, 0 files merged, 0 files removed, 0 files unresolved
adding b/z
% qinit
% -R qinit
% qinit -c
A .hgignore
A series
% qinit; qinit -c
  .hgignore:
^\.hg
^\.mq
syntax: glob
status
guards
  series:
abort: repository already exists!
% qinit; <stuff>; qinit -c
adding .hg/patches/A
adding .hg/patches/B
A .hgignore
A A
A B
A series
  .hgignore:
status
bleh
  series:
A
B
% qrefresh
foo bar

diff -r  xa
--- a/a
+++ b/a
@@ -1,1 +1,2 @@
 a
+a
% empty qrefresh
revision:
patch:
foo bar

working dir diff:
--- a/a
+++ b/a
@@ -1,1 +1,2 @@
 a
+a
% qpop
patch queue now empty
% qpush
applying test.patch
now at: test.patch
% pop/push outside repo
patch queue now empty
applying test.patch
now at: test.patch
% qrefresh in subdir
% pop/push -a in subdir
patch queue now empty
applying test.patch
applying test2.patch
now at: test2.patch
% qseries
test.patch
test2.patch
now at: test.patch
0 A test.patch: foo bar
1 U test2.patch: 
applying test2.patch
now at: test2.patch
% qapplied
test.patch
test2.patch
% qtop
test2.patch
% qprev
test.patch
% qnext
all patches applied
% pop, qnext, qprev, qapplied
now at: test.patch
test2.patch
only one patch applied
test.patch
% commit should fail
abort: cannot commit over an applied mq patch
% push should fail
pushing to ../../k
abort: source has mq patches applied
% import should fail
abort: cannot import over an applied patch
% qunapplied
test2.patch
% qpush/qpop with index
applying test2.patch
now at: test2.patch
now at: test.patch
applying test1b.patch
now at: test1b.patch
applying test2.patch
now at: test2.patch
now at: test1b.patch
now at: test.patch
applying test1b.patch
applying test2.patch
now at: test2.patch
% push should succeed
patch queue now empty
pushing to ../../k
searching for changes
adding changesets
adding manifests
adding file changes
added 1 changesets with 1 changes to 1 files
% qpush/qpop error codes
applying test.patch
applying test1b.patch
applying test2.patch
now at: test2.patch
  % pops all patches and succeeds
patch queue now empty
  qpop -a succeeds
  % does nothing and succeeds
no patches applied
  qpop -a succeeds
  % fails - nothing else to pop
no patches applied
  qpop fails
  % pushes a patch and succeeds
applying test.patch
now at: test.patch
  qpush succeeds
  % pops a patch and succeeds
patch queue now empty
  qpop succeeds
  % pushes up to test1b.patch and succeeds
applying test.patch
applying test1b.patch
now at: test1b.patch
  qpush test1b.patch succeeds
  % does nothing and succeeds
qpush: test1b.patch is already at the top
  qpush test1b.patch succeeds
  % does nothing and succeeds
qpop: test1b.patch is already at the top
  qpop test1b.patch succeeds
  % fails - can't push to this patch
abort: cannot push to a previous patch: test.patch
  qpush test.patch fails
  % fails - can't pop to this patch
abort: patch test2.patch is not applied
  qpop test2.patch fails
  % pops up to test.patch and succeeds
now at: test.patch
  qpop test.patch succeeds
  % pushes all patches and succeeds
applying test1b.patch
applying test2.patch
now at: test2.patch
  qpush -a succeeds
  % does nothing and succeeds
all patches are currently applied
  qpush -a succeeds
  % fails - nothing else to push
patch series already fully applied
  qpush fails
  % does nothing and succeeds
qpush: test2.patch is already at the top
  qpush test2.patch succeeds
% strip
adding x
0 files updated, 0 files merged, 1 files removed, 0 files unresolved
saving bundle to 
adding changesets
adding manifests
adding file changes
added 1 changesets with 1 changes to 1 files
(run 'hg update' to get a working copy)
% strip with local changes, should complain
1 files updated, 0 files merged, 0 files removed, 0 files unresolved
abort: local changes found
% --force strip with local changes
0 files updated, 0 files merged, 1 files removed, 0 files unresolved
saving bundle to 
% cd b; hg qrefresh
adding a
foo

diff -r cb9a9f314b8b a
--- a/a
+++ b/a
@@ -1,1 +1,2 @@
 a
+a
diff -r cb9a9f314b8b b/f
--- /dev/null
+++ b/b/f
@@ -0,0 +1,1 @@
+f
% hg qrefresh .
foo

diff -r cb9a9f314b8b b/f
--- /dev/null
+++ b/b/f
@@ -0,0 +1,1 @@
+f
M a
% qpush failure
patch queue now empty
applying foo
applying bar
file foo already exists
1 out of 1 hunks FAILED -- saving rejects to file foo.rej
patch failed, unable to continue (try -v)
patch failed, rejects left in working dir
errors during apply, please fix and refresh bar
? foo
? foo.rej
% mq tags
0 qparent
1 qbase foo
2 qtip bar tip
% bad node in status
now at: foo
changeset:   0:cb9a9f314b8b
mq status file refers to unknown node
tag:         tip
user:        test
date:        Thu Jan 01 00:00:00 1970 +0000
summary:     a

mq status file refers to unknown node
default                        0:cb9a9f314b8b
abort: trying to pop unknown node
new file

diff --git a/new b/new
new file mode 100755
--- /dev/null
+++ b/new
@@ -0,0 +1,1 @@
+foo
copy file

diff --git a/new b/copy
copy from new
copy to copy
now at: new
applying copy
now at: copy
diff --git a/new b/copy
copy from new
copy to copy
diff --git a/new b/copy
copy from new
copy to copy
1 files updated, 0 files merged, 2 files removed, 0 files unresolved
created new head
2 files updated, 0 files merged, 1 files removed, 0 files unresolved
adding branch
adding changesets
adding manifests
adding file changes
added 1 changesets with 1 changes to 1 files
patch queue now empty
(working directory not at a head)
applying bar
now at: bar
diff --git a/bar b/bar
new file mode 100644
--- /dev/null
+++ b/bar
@@ -0,0 +1,1 @@
+bar
diff --git a/foo b/baz
rename from foo
rename to baz
2 baz (foo)
diff --git a/bar b/bar
new file mode 100644
--- /dev/null
+++ b/bar
@@ -0,0 +1,1 @@
+bar
diff --git a/foo b/baz
rename from foo
rename to baz
2 baz (foo)
diff --git a/bar b/bar
diff --git a/foo b/baz

1 files updated, 0 files merged, 2 files removed, 0 files unresolved
2 files updated, 0 files merged, 1 files removed, 0 files unresolved
adding branch
adding changesets
adding manifests
adding file changes
added 1 changesets with 1 changes to 1 files
patch queue now empty
(working directory not at a head)
applying bar
now at: bar
diff --git a/foo b/bleh
rename from foo
rename to bleh
diff --git a/quux b/quux
new file mode 100644
--- /dev/null
+++ b/quux
@@ -0,0 +1,1 @@
+bar
3 bleh (foo)
diff --git a/foo b/barney
rename from foo
rename to barney
diff --git a/fred b/fred
new file mode 100644
--- /dev/null
+++ b/fred
@@ -0,0 +1,1 @@
+bar
3 barney (foo)
% refresh omitting an added file
C newfile
A newfile
now at: bar
% create a git patch
diff --git a/alexander b/alexander
% create a git binary patch
8ba2a2f3e77b55d03051ff9c24ad65e7  bucephalus
diff --git a/bucephalus b/bucephalus
% check binary patches can be popped and pushed
now at: addalexander
applying addbucephalus
now at: addbucephalus
8ba2a2f3e77b55d03051ff9c24ad65e7  bucephalus
% strip again
1 files updated, 0 files merged, 0 files removed, 0 files unresolved
created new head
merging foo
0 files updated, 1 files merged, 0 files removed, 0 files unresolved
(branch merge, don't forget to commit)
changeset:   3:99615015637b
tag:         tip
parent:      2:20cbbe65cff7
parent:      1:d2871fc282d4
user:        test
date:        Thu Jan 01 00:00:00 1970 +0000
summary:     merge

changeset:   2:20cbbe65cff7
parent:      0:53245c60e682
user:        test
date:        Thu Jan 01 00:00:00 1970 +0000
summary:     change foo 2

changeset:   1:d2871fc282d4
user:        test
date:        Thu Jan 01 00:00:00 1970 +0000
summary:     change foo 1

changeset:   0:53245c60e682
user:        test
date:        Thu Jan 01 00:00:00 1970 +0000
summary:     add foo

1 files updated, 0 files merged, 0 files removed, 0 files unresolved
saving bundle to 
saving bundle to 
adding branch
adding changesets
adding manifests
adding file changes
added 1 changesets with 1 changes to 1 files
changeset:   1:20cbbe65cff7
tag:         tip
user:        test
date:        Thu Jan 01 00:00:00 1970 +0000
summary:     change foo 2

changeset:   0:53245c60e682
user:        test
date:        Thu Jan 01 00:00:00 1970 +0000
summary:     add foo

% qclone
abort: versioned patch repository not found (see qinit -c)
adding .hg/patches/patch1
main repo:
    rev 1: change foo
    rev 0: add foo
patch repo:
    rev 0: checkpoint
updating working directory
3 files updated, 0 files merged, 0 files removed, 0 files unresolved
1 files updated, 0 files merged, 0 files removed, 0 files unresolved
main repo:
    rev 0: add foo
patch repo:
    rev 0: checkpoint
patch queue now empty
main repo:
    rev 0: add foo
patch repo:
    rev 0: checkpoint
updating working directory
3 files updated, 0 files merged, 0 files removed, 0 files unresolved
1 files updated, 0 files merged, 0 files removed, 0 files unresolved
main repo:
    rev 0: add foo
patch repo:
    rev 0: checkpoint
% test applying on an empty file (issue 1033)
adding a
patch queue now empty
applying changea
now at: changea
% test qpush with --force, issue1087
adding bye.txt
adding hello.txt
patch queue now empty
% qpush should fail, local changes
abort: local changes found, refresh first
% apply force, should not discard changes with empty patch
applying empty
patch empty is empty
now at: empty
diff -r bf5fc3f07a0a hello.txt
--- a/hello.txt
+++ b/hello.txt
@@ -1,1 +1,2 @@
 hello
+world
diff -r 9ecee4f634e3 hello.txt
--- a/hello.txt
+++ b/hello.txt
@@ -1,1 +1,2 @@
 hello
+world
changeset:   1:bf5fc3f07a0a
tag:         qtip
tag:         tip
tag:         empty
tag:         qbase
user:        test
date:        Thu Jan 01 00:00:00 1970 +0000
summary:     imported patch empty


patch queue now empty
% qpush should fail, local changes
abort: local changes found, refresh first
% apply force, should discard changes in hello, but not bye
applying empty
now at: empty
M bye.txt
diff -r ba252371dbc1 bye.txt
--- a/bye.txt
+++ b/bye.txt
@@ -1,1 +1,2 @@
 bye
+universe
diff -r 9ecee4f634e3 bye.txt
--- a/bye.txt
+++ b/bye.txt
@@ -1,1 +1,2 @@
 bye
+universe
diff -r 9ecee4f634e3 hello.txt
--- a/hello.txt
+++ b/hello.txt
@@ -1,1 +1,3 @@
 hello
+world
+universe
% test popping revisions not in working dir ancestry
0 A empty
1 files updated, 0 files merged, 0 files removed, 0 files unresolved
patch queue now empty