infinitepush: move the extension to core from fb-hgext
This patch moves the infinitepush extension from fb-hgext to core. The
extension is used to store incoming bundles during a push in bundlestore rather
than applying them to the revlog.
The extension was copied from the repository revision at
f27f094e91553d3cae5167c0b1c42ae940f888d5 and following changes were made:
* added `from __future__ import absolute_import` where missing
* fixed module imports to follow the core style
* minor fixes for test-check-code.t
* registered the configs
* adding the testedwith value to match core's convention
* removed double newlines to make test-check-commit.t happy
* added one line doc about extension and marked it as experimental
Only one test file test-infinitepush-bundlestore.t is moved to core and
following changes are made to file:
* remove dependency of library.sh
* split the tests into two tests i.e. test-infinitepush.t and
test-infinitepush-bundlestore.t
* removed testing related to other facebook's extensions pushrebase, inhibit,
fbamend
library-infinitepush.sh is also copied from fb-hgext from the same revision and
following changes are made:
* change the path to infinitepush extension as it's in core with this patch
* removed sql handling from the file as we are not testing that initially
Currently at this revision, test-check-module-imports.t does not pass as there
is import of a module from fb/hgext in one the of the file which will be removed
in the next patch.
This extension right now has a lot of things which we don't require in core like
`--to`, `--create` flags to `hg bookmark`, logic related to remotenames
extension and another facebook's extensions, custom bundle2parts which can be
prevented by using bookmarks bundle part and also logic related to sql store
which is probably we don't want initially.
The next patches in this series will remove all the unwanted and unrequired
things from the extension and will make this a nice one.
The end goal is to have a very lighweight extension with no or very less
wrapping on the client side.
Differential Revision: https://phab.mercurial-scm.org/D2096
Testing infinipush extension and the confi options provided by it
Setup
$ . "$TESTDIR/library-infinitepush.sh"
$ cp $HGRCPATH $TESTTMP/defaulthgrc
$ setupcommon
$ hg init repo
$ cd repo
$ setupserver
$ echo initialcommit > initialcommit
$ hg ci -Aqm "initialcommit"
$ hg phase --public .
$ cd ..
$ hg clone ssh://user@dummy/repo client -q
Create two heads. Push first head alone, then two heads together. Make sure that
multihead push works.
$ cd client
$ echo multihead1 > multihead1
$ hg add multihead1
$ hg ci -m "multihead1"
$ hg up null
0 files updated, 0 files merged, 2 files removed, 0 files unresolved
$ echo multihead2 > multihead2
$ hg ci -Am "multihead2"
adding multihead2
created new head
$ hg push -r . --bundle-store
pushing to ssh://user@dummy/repo
searching for changes
remote: pushing 1 commit:
remote: ee4802bf6864 multihead2
$ hg push -r '1:2' --bundle-store
pushing to ssh://user@dummy/repo
searching for changes
remote: pushing 2 commits:
remote: bc22f9a30a82 multihead1
remote: ee4802bf6864 multihead2
$ scratchnodes
bc22f9a30a821118244deacbd732e394ed0b686c ab1bc557aa090a9e4145512c734b6e8a828393a5
ee4802bf6864326a6b3dcfff5a03abc2a0a69b8f ab1bc557aa090a9e4145512c734b6e8a828393a5
Create two new scratch bookmarks
$ hg up 0
1 files updated, 0 files merged, 1 files removed, 0 files unresolved
$ echo scratchfirstpart > scratchfirstpart
$ hg ci -Am "scratchfirstpart"
adding scratchfirstpart
created new head
$ hg push -r . --to scratch/firstpart --create
pushing to ssh://user@dummy/repo
searching for changes
remote: pushing 1 commit:
remote: 176993b87e39 scratchfirstpart
$ hg up 0
0 files updated, 0 files merged, 1 files removed, 0 files unresolved
$ echo scratchsecondpart > scratchsecondpart
$ hg ci -Am "scratchsecondpart"
adding scratchsecondpart
created new head
$ hg push -r . --to scratch/secondpart --create
pushing to ssh://user@dummy/repo
searching for changes
remote: pushing 1 commit:
remote: 8db3891c220e scratchsecondpart
Pull two bookmarks from the second client
$ cd ..
$ hg clone ssh://user@dummy/repo client2 -q
$ cd client2
$ hg pull -B scratch/firstpart -B scratch/secondpart
pulling from ssh://user@dummy/repo
searching for changes
adding changesets
adding manifests
adding file changes
added 1 changesets with 1 changes to 1 files
adding changesets
adding manifests
adding file changes
added 1 changesets with 1 changes to 1 files (+1 heads)
new changesets * (glob)
(run 'hg heads' to see heads, 'hg merge' to merge)
$ hg log -r scratch/secondpart -T '{node}'
8db3891c220e216f6da214e8254bd4371f55efca (no-eol)
$ hg log -r scratch/firstpart -T '{node}'
176993b87e39bd88d66a2cccadabe33f0b346339 (no-eol)
Make two commits to the scratch branch
$ echo testpullbycommithash1 > testpullbycommithash1
$ hg ci -Am "testpullbycommithash1"
adding testpullbycommithash1
created new head
$ hg log -r '.' -T '{node}\n' > ../testpullbycommithash1
$ echo testpullbycommithash2 > testpullbycommithash2
$ hg ci -Aqm "testpullbycommithash2"
$ hg push -r . --to scratch/mybranch --create -q
Create third client and pull by commit hash.
Make sure testpullbycommithash2 has not fetched
$ cd ..
$ hg clone ssh://user@dummy/repo client3 -q
$ cd client3
$ hg pull -r `cat ../testpullbycommithash1`
pulling from ssh://user@dummy/repo
searching for changes
adding changesets
adding manifests
adding file changes
added 1 changesets with 1 changes to 1 files
new changesets 33910bfe6ffe
(run 'hg update' to get a working copy)
$ hg log -G -T '{desc} {phase} {bookmarks}'
o testpullbycommithash1 draft
|
@ initialcommit public
Make public commit in the repo and pull it.
Make sure phase on the client is public.
$ cd ../repo
$ echo publiccommit > publiccommit
$ hg ci -Aqm "publiccommit"
$ hg phase --public .
$ cd ../client3
$ hg pull
pulling from ssh://user@dummy/repo
searching for changes
adding changesets
adding manifests
adding file changes
added 1 changesets with 1 changes to 1 files (+1 heads)
new changesets a79b6597f322
(run 'hg heads' to see heads, 'hg merge' to merge)
$ hg log -G -T '{desc} {phase} {bookmarks} {node|short}'
o publiccommit public a79b6597f322
|
| o testpullbycommithash1 draft 33910bfe6ffe
|/
@ initialcommit public 67145f466344
$ hg up a79b6597f322
1 files updated, 0 files merged, 0 files removed, 0 files unresolved
$ echo scratchontopofpublic > scratchontopofpublic
$ hg ci -Aqm "scratchontopofpublic"
$ hg push -r . --to scratch/scratchontopofpublic --create
pushing to ssh://user@dummy/repo
searching for changes
remote: pushing 1 commit:
remote: c70aee6da07d scratchontopofpublic
$ cd ../client2
$ hg pull -B scratch/scratchontopofpublic
pulling from ssh://user@dummy/repo
searching for changes
adding changesets
adding manifests
adding file changes
added 1 changesets with 1 changes to 1 files (+1 heads)
adding changesets
adding manifests
adding file changes
added 1 changesets with 1 changes to 1 files
new changesets a79b6597f322:c70aee6da07d
(run 'hg heads .' to see heads, 'hg merge' to merge)
$ hg log -r scratch/scratchontopofpublic -T '{phase}'
draft (no-eol)
Strip scratchontopofpublic commit and do hg update
$ hg log -r tip -T '{node}\n'
c70aee6da07d7cdb9897375473690df3a8563339
$ echo "[extensions]" >> .hg/hgrc
$ echo "strip=" >> .hg/hgrc
$ hg strip -q tip
$ hg up c70aee6da07d7cdb9897375473690df3a8563339
'c70aee6da07d7cdb9897375473690df3a8563339' does not exist locally - looking for it remotely...
pulling from ssh://user@dummy/repo
searching for changes
adding changesets
adding manifests
adding file changes
added 1 changesets with 1 changes to 1 files
new changesets c70aee6da07d
(run 'hg update' to get a working copy)
'c70aee6da07d7cdb9897375473690df3a8563339' found remotely
2 files updated, 0 files merged, 2 files removed, 0 files unresolved
Trying to pull from bad path
$ hg strip -q tip
$ hg --config paths.default=badpath up c70aee6da07d7cdb9897375473690df3a8563339
'c70aee6da07d7cdb9897375473690df3a8563339' does not exist locally - looking for it remotely...
pulling from $TESTTMP/client2/badpath (glob)
pull failed: repository $TESTTMP/client2/badpath not found
abort: unknown revision 'c70aee6da07d7cdb9897375473690df3a8563339'!
[255]
Strip commit and pull it using hg update with bookmark name
$ hg strip -q d8fde0ddfc96
$ hg book -d scratch/mybranch
$ hg up scratch/mybranch
'scratch/mybranch' does not exist locally - looking for it remotely...
pulling from ssh://user@dummy/repo
searching for changes
adding changesets
adding manifests
adding file changes
added 1 changesets with 1 changes to 2 files
new changesets d8fde0ddfc96
(run 'hg update' to get a working copy)
'scratch/mybranch' found remotely
2 files updated, 0 files merged, 1 files removed, 0 files unresolved
(activating bookmark scratch/mybranch)
$ hg log -r scratch/mybranch -T '{node}'
d8fde0ddfc962183977f92d2bc52d303b8840f9d (no-eol)
Test debugfillinfinitepushmetadata
$ cd ../repo
$ hg debugfillinfinitepushmetadata
abort: nodes are not specified
[255]
$ hg debugfillinfinitepushmetadata --node randomnode
abort: node randomnode is not found
[255]
$ hg debugfillinfinitepushmetadata --node d8fde0ddfc962183977f92d2bc52d303b8840f9d
$ cat .hg/scratchbranches/index/nodemetadatamap/d8fde0ddfc962183977f92d2bc52d303b8840f9d
{"changed_files": {"testpullbycommithash2": {"adds": 1, "isbinary": false, "removes": 0, "status": "added"}}} (no-eol)
$ cd ../client
$ hg up d8fde0ddfc962183977f92d2bc52d303b8840f9d
'd8fde0ddfc962183977f92d2bc52d303b8840f9d' does not exist locally - looking for it remotely...
pulling from ssh://user@dummy/repo
searching for changes
adding changesets
adding manifests
adding file changes
added 2 changesets with 2 changes to 2 files (+1 heads)
new changesets 33910bfe6ffe:d8fde0ddfc96
(run 'hg heads .' to see heads, 'hg merge' to merge)
'd8fde0ddfc962183977f92d2bc52d303b8840f9d' found remotely
2 files updated, 0 files merged, 1 files removed, 0 files unresolved
$ echo file > file
$ hg add file
$ hg rm testpullbycommithash2
$ hg ci -m 'add and rm files'
$ hg log -r . -T '{node}\n'
3edfe7e9089ab9f728eb8e0d0c62a5d18cf19239
$ hg cp file cpfile
$ hg mv file mvfile
$ hg ci -m 'cpfile and mvfile'
$ hg log -r . -T '{node}\n'
c7ac39f638c6b39bcdacf868fa21b6195670f8ae
$ hg push -r . --bundle-store
pushing to ssh://user@dummy/repo
searching for changes
remote: pushing 4 commits:
remote: 33910bfe6ffe testpullbycommithash1
remote: d8fde0ddfc96 testpullbycommithash2
remote: 3edfe7e9089a add and rm files
remote: c7ac39f638c6 cpfile and mvfile
$ cd ../repo
$ hg debugfillinfinitepushmetadata --node 3edfe7e9089ab9f728eb8e0d0c62a5d18cf19239 --node c7ac39f638c6b39bcdacf868fa21b6195670f8ae
$ cat .hg/scratchbranches/index/nodemetadatamap/3edfe7e9089ab9f728eb8e0d0c62a5d18cf19239
{"changed_files": {"file": {"adds": 1, "isbinary": false, "removes": 0, "status": "added"}, "testpullbycommithash2": {"adds": 0, "isbinary": false, "removes": 1, "status": "removed"}}} (no-eol)
$ cat .hg/scratchbranches/index/nodemetadatamap/c7ac39f638c6b39bcdacf868fa21b6195670f8ae
{"changed_files": {"cpfile": {"adds": 1, "copies": "file", "isbinary": false, "removes": 0, "status": "added"}, "file": {"adds": 0, "isbinary": false, "removes": 1, "status": "removed"}, "mvfile": {"adds": 1, "copies": "file", "isbinary": false, "removes": 0, "status": "added"}}} (no-eol)
Test infinitepush.metadatafilelimit number
$ cd ../client
$ echo file > file
$ hg add file
$ echo file1 > file1
$ hg add file1
$ echo file2 > file2
$ hg add file2
$ hg ci -m 'add many files'
$ hg log -r . -T '{node}'
09904fb20c53ff351bd3b1d47681f569a4dab7e5 (no-eol)
$ hg push -r . --bundle-store
pushing to ssh://user@dummy/repo
searching for changes
remote: pushing 5 commits:
remote: 33910bfe6ffe testpullbycommithash1
remote: d8fde0ddfc96 testpullbycommithash2
remote: 3edfe7e9089a add and rm files
remote: c7ac39f638c6 cpfile and mvfile
remote: 09904fb20c53 add many files
$ cd ../repo
$ hg debugfillinfinitepushmetadata --node 09904fb20c53ff351bd3b1d47681f569a4dab7e5 --config infinitepush.metadatafilelimit=2
$ cat .hg/scratchbranches/index/nodemetadatamap/09904fb20c53ff351bd3b1d47681f569a4dab7e5
{"changed_files": {"file": {"adds": 1, "isbinary": false, "removes": 0, "status": "added"}, "file1": {"adds": 1, "isbinary": false, "removes": 0, "status": "added"}}, "changed_files_truncated": true} (no-eol)
Test infinitepush.fillmetadatabranchpattern
$ cd ../repo
$ cat >> .hg/hgrc << EOF
> [infinitepush]
> fillmetadatabranchpattern=re:scratch/fillmetadata/.*
> EOF
$ cd ../client
$ echo tofillmetadata > tofillmetadata
$ hg ci -Aqm "tofillmetadata"
$ hg log -r . -T '{node}\n'
d2b0410d4da084bc534b1d90df0de9eb21583496
$ hg push -r . --to scratch/fillmetadata/fill --create
pushing to ssh://user@dummy/repo
searching for changes
remote: pushing 6 commits:
remote: 33910bfe6ffe testpullbycommithash1
remote: d8fde0ddfc96 testpullbycommithash2
remote: 3edfe7e9089a add and rm files
remote: c7ac39f638c6 cpfile and mvfile
remote: 09904fb20c53 add many files
remote: d2b0410d4da0 tofillmetadata
Make sure background process finished
$ sleep 3
$ cd ../repo
$ cat .hg/scratchbranches/index/nodemetadatamap/d2b0410d4da084bc534b1d90df0de9eb21583496
{"changed_files": {"tofillmetadata": {"adds": 1, "isbinary": false, "removes": 0, "status": "added"}}} (no-eol)