comparison tests/test-clone-stream.t @ 50524:58adcabc295f

stream-clone: introduce the notion of an experimental "v3" version We introduce a new experimental "v3" stream protocol, disabled by default. In practice the "v3-exp" protocol introduced in this changeset is identical to v2, but this changeset, lay the groundwork for having a new protocol: configuration, capability exchange, test coverage, etc. The actual protocol work will starts in the coming changesets.
author Pierre-Yves David <pierre-yves.david@octobus.net>
date Fri, 19 May 2023 14:49:50 +0200
parents 3c0da0fa0eed
children 116da6bb7e3b
comparison
equal deleted inserted replaced
50523:a6543983b8f4 50524:58adcabc295f
1 #require serve no-reposimplestore no-chg 1 #require serve no-reposimplestore no-chg
2 2
3 #testcases stream-legacy stream-bundle2 3 #testcases stream-legacy stream-bundle2-v2 stream-bundle2-v3
4 4
5 #if stream-legacy 5 #if stream-legacy
6 $ cat << EOF >> $HGRCPATH 6 $ cat << EOF >> $HGRCPATH
7 > [server] 7 > [server]
8 > bundle2.stream = no 8 > bundle2.stream = no
9 > EOF
10 #endif
11 #if stream-bundle2-v3
12 $ cat << EOF >> $HGRCPATH
13 > [experimental]
14 > stream-v3 = yes
9 > EOF 15 > EOF
10 #endif 16 #endif
11 17
12 Initialize repository 18 Initialize repository
13 19
172 0040: 74 65 64 20 62 75 74 20 73 65 72 76 65 72 20 64 |ted but server d| 178 0040: 74 65 64 20 62 75 74 20 73 65 72 76 65 72 20 64 |ted but server d|
173 0050: 6f 65 73 20 6e 6f 74 20 61 6c 6c 6f 77 20 74 68 |oes not allow th| 179 0050: 6f 65 73 20 6e 6f 74 20 61 6c 6c 6f 77 20 74 68 |oes not allow th|
174 0060: 69 73 20 66 |is f| 180 0060: 69 73 20 66 |is f|
175 181
176 #endif 182 #endif
177 #if stream-bundle2 183 #if stream-bundle2-v2
178 $ hg debugcapabilities http://localhost:$HGPORT 184 $ hg debugcapabilities http://localhost:$HGPORT
179 Main capabilities: 185 Main capabilities:
180 batch 186 batch
181 branchmap 187 branchmap
182 $USUAL_BUNDLE2_CAPS_SERVER$ 188 $USUAL_BUNDLE2_CAPS_SERVER$
240 0040: 74 65 64 20 62 75 74 20 73 65 72 76 65 72 20 64 |ted but server d| 246 0040: 74 65 64 20 62 75 74 20 73 65 72 76 65 72 20 64 |ted but server d|
241 0050: 6f 65 73 20 6e 6f 74 20 61 6c 6c 6f 77 20 74 68 |oes not allow th| 247 0050: 6f 65 73 20 6e 6f 74 20 61 6c 6c 6f 77 20 74 68 |oes not allow th|
242 0060: 69 73 20 66 |is f| 248 0060: 69 73 20 66 |is f|
243 249
244 #endif 250 #endif
251 #if stream-bundle2-v3
252 $ hg debugcapabilities http://localhost:$HGPORT
253 Main capabilities:
254 batch
255 branchmap
256 $USUAL_BUNDLE2_CAPS_SERVER$
257 changegroupsubset
258 compression=$BUNDLE2_COMPRESSIONS$
259 getbundle
260 httpheader=1024
261 httpmediatype=0.1rx,0.1tx,0.2tx
262 known
263 lookup
264 pushkey
265 unbundle=HG10GZ,HG10BZ,HG10UN
266 unbundlehash
267 Bundle2 capabilities:
268 HG20
269 bookmarks
270 changegroup
271 01
272 02
273 03
274 checkheads
275 related
276 digests
277 md5
278 sha1
279 sha512
280 error
281 abort
282 unsupportedcontent
283 pushraced
284 pushkey
285 hgtagsfnodes
286 listkeys
287 phases
288 heads
289 pushkey
290 remote-changegroup
291 http
292 https
293
294 $ hg clone --stream -U http://localhost:$HGPORT server-disabled
295 warning: stream clone requested but server has them disabled
296 requesting all changes
297 adding changesets
298 adding manifests
299 adding file changes
300 added 3 changesets with 1088 changes to 1088 files
301 new changesets 96ee1d7354c4:5223b5e3265f
302
303 $ get-with-headers.py $LOCALIP:$HGPORT '?cmd=getbundle' content-type --bodyfile body --hgproto 0.2 --requestheader "x-hgarg-1=bundlecaps=HG20%2Cbundle2%3DHG20%250Abookmarks%250Achangegroup%253D01%252C02%252C03%250Adigests%253Dmd5%252Csha1%252Csha512%250Aerror%253Dabort%252Cunsupportedcontent%252Cpushraced%252Cpushkey%250Ahgtagsfnodes%250Alistkeys%250Aphases%253Dheads%250Apushkey%250Aremote-changegroup%253Dhttp%252Chttps&cg=0&common=0000000000000000000000000000000000000000&heads=c17445101a72edac06facd130d14808dfbd5c7c2&stream=1"
304 200 Script output follows
305 content-type: application/mercurial-0.2
306
307
308 $ f --size body --hexdump --bytes 100
309 body: size=140
310 0000: 04 6e 6f 6e 65 48 47 32 30 00 00 00 00 00 00 00 |.noneHG20.......|
311 0010: 73 0b 45 52 52 4f 52 3a 41 42 4f 52 54 00 00 00 |s.ERROR:ABORT...|
312 0020: 00 01 01 07 3c 04 16 6d 65 73 73 61 67 65 73 74 |....<..messagest|
313 0030: 72 65 61 6d 20 64 61 74 61 20 72 65 71 75 65 73 |ream data reques|
314 0040: 74 65 64 20 62 75 74 20 73 65 72 76 65 72 20 64 |ted but server d|
315 0050: 6f 65 73 20 6e 6f 74 20 61 6c 6c 6f 77 20 74 68 |oes not allow th|
316 0060: 69 73 20 66 |is f|
317
318 #endif
245 319
246 $ killdaemons.py 320 $ killdaemons.py
247 $ cd server 321 $ cd server
248 $ hg serve -p $HGPORT -d --pid-file=hg.pid --error errors.txt 322 $ hg serve -p $HGPORT -d --pid-file=hg.pid --error errors.txt
249 $ cat hg.pid > $DAEMON_PIDS 323 $ cat hg.pid > $DAEMON_PIDS
260 transferred 98.8 KB in * seconds (* */sec) (glob) (zstd !) 334 transferred 98.8 KB in * seconds (* */sec) (glob) (zstd !)
261 searching for changes 335 searching for changes
262 no changes found 336 no changes found
263 $ cat server/errors.txt 337 $ cat server/errors.txt
264 #endif 338 #endif
265 #if stream-bundle2 339 #if stream-bundle2-v2
340 $ hg clone --stream -U http://localhost:$HGPORT clone1
341 streaming all changes
342 1093 files to transfer, 102 KB of data (no-zstd !)
343 transferred 102 KB in * seconds (* */sec) (glob) (no-zstd !)
344 1093 files to transfer, 98.9 KB of data (zstd !)
345 transferred 98.9 KB in * seconds (* */sec) (glob) (zstd !)
346
347 $ ls -1 clone1/.hg/cache
348 branch2-base
349 branch2-immutable
350 branch2-served
351 branch2-served.hidden
352 branch2-visible
353 branch2-visible-hidden
354 rbc-names-v1
355 rbc-revs-v1
356 tags2
357 tags2-served
358 $ cat server/errors.txt
359 #endif
360 #if stream-bundle2-v3
266 $ hg clone --stream -U http://localhost:$HGPORT clone1 361 $ hg clone --stream -U http://localhost:$HGPORT clone1
267 streaming all changes 362 streaming all changes
268 1093 files to transfer, 102 KB of data (no-zstd !) 363 1093 files to transfer, 102 KB of data (no-zstd !)
269 transferred 102 KB in * seconds (* */sec) (glob) (no-zstd !) 364 transferred 102 KB in * seconds (* */sec) (glob) (no-zstd !)
270 1093 files to transfer, 98.9 KB of data (zstd !) 365 1093 files to transfer, 98.9 KB of data (zstd !)
384 1090 files to transfer, 98.8 KB of data (zstd !) 479 1090 files to transfer, 98.8 KB of data (zstd !)
385 transferred 98.8 KB in * seconds (* */sec) (glob) (zstd !) 480 transferred 98.8 KB in * seconds (* */sec) (glob) (zstd !)
386 searching for changes 481 searching for changes
387 no changes found 482 no changes found
388 #endif 483 #endif
389 #if stream-bundle2 484 #if stream-bundle2-v2
485 $ hg clone --uncompressed -U http://localhost:$HGPORT clone1-uncompressed
486 streaming all changes
487 1093 files to transfer, 102 KB of data (no-zstd !)
488 transferred 102 KB in * seconds (* */sec) (glob) (no-zstd !)
489 1093 files to transfer, 98.9 KB of data (zstd !)
490 transferred 98.9 KB in * seconds (* */sec) (glob) (zstd !)
491 #endif
492 #if stream-bundle2-v3
390 $ hg clone --uncompressed -U http://localhost:$HGPORT clone1-uncompressed 493 $ hg clone --uncompressed -U http://localhost:$HGPORT clone1-uncompressed
391 streaming all changes 494 streaming all changes
392 1093 files to transfer, 102 KB of data (no-zstd !) 495 1093 files to transfer, 102 KB of data (no-zstd !)
393 transferred 102 KB in * seconds (* */sec) (glob) (no-zstd !) 496 transferred 102 KB in * seconds (* */sec) (glob) (no-zstd !)
394 1093 files to transfer, 98.9 KB of data (zstd !) 497 1093 files to transfer, 98.9 KB of data (zstd !)
423 bundle2-input-bundle: 2 parts total 526 bundle2-input-bundle: 2 parts total
424 checking for updated bookmarks 527 checking for updated bookmarks
425 updating the branch cache 528 updating the branch cache
426 (sent 5 HTTP requests and * bytes; received * bytes in responses) (glob) 529 (sent 5 HTTP requests and * bytes; received * bytes in responses) (glob)
427 #endif 530 #endif
428 #if stream-bundle2 531 #if stream-bundle2-v2
429 $ hg --debug --config worker.backgroundclose=true --config worker.backgroundcloseminfilecount=1 clone --stream -U http://localhost:$HGPORT clone-background | grep -v adding 532 $ hg --debug --config worker.backgroundclose=true --config worker.backgroundcloseminfilecount=1 clone --stream -U http://localhost:$HGPORT clone-background | grep -v adding
430 using http://localhost:$HGPORT/ 533 using http://localhost:$HGPORT/
431 sending capabilities command 534 sending capabilities command
432 query 1; heads 535 query 1; heads
433 sending batch command 536 sending batch command
450 bundle2-input-bundle: 2 parts total 553 bundle2-input-bundle: 2 parts total
451 checking for updated bookmarks 554 checking for updated bookmarks
452 updating the branch cache 555 updating the branch cache
453 (sent 3 HTTP requests and * bytes; received * bytes in responses) (glob) 556 (sent 3 HTTP requests and * bytes; received * bytes in responses) (glob)
454 #endif 557 #endif
558 #if stream-bundle2-v3
559 $ hg --debug --config worker.backgroundclose=true --config worker.backgroundcloseminfilecount=1 clone --stream -U http://localhost:$HGPORT clone-background | grep -v adding
560 using http://localhost:$HGPORT/
561 sending capabilities command
562 query 1; heads
563 sending batch command
564 streaming all changes
565 sending getbundle command
566 bundle2-input-bundle: with-transaction
567 bundle2-input-part: "stream3" (params: 3 mandatory) supported
568 applying stream bundle
569 1093 files to transfer, 102 KB of data (no-zstd !)
570 1093 files to transfer, 98.9 KB of data (zstd !)
571 starting 4 threads for background file closing
572 starting 4 threads for background file closing
573 updating the branch cache
574 transferred 102 KB in * seconds (* */sec) (glob) (no-zstd !)
575 bundle2-input-part: total payload size 118984 (no-zstd !)
576 transferred 98.9 KB in * seconds (* */sec) (glob) (zstd !)
577 bundle2-input-part: total payload size 116145 (zstd no-bigendian !)
578 bundle2-input-part: total payload size 116140 (zstd bigendian !)
579 bundle2-input-part: "listkeys" (params: 1 mandatory) supported
580 bundle2-input-bundle: 2 parts total
581 checking for updated bookmarks
582 updating the branch cache
583 (sent 3 HTTP requests and * bytes; received * bytes in responses) (glob)
584 #endif
455 585
456 Cannot stream clone when there are secret changesets 586 Cannot stream clone when there are secret changesets
457 587
458 $ hg -R server phase --force --secret -r tip 588 $ hg -R server phase --force --secret -r tip
459 $ hg clone --stream -U http://localhost:$HGPORT secret-denied 589 $ hg clone --stream -U http://localhost:$HGPORT secret-denied
482 1090 files to transfer, 98.8 KB of data (zstd !) 612 1090 files to transfer, 98.8 KB of data (zstd !)
483 transferred 98.8 KB in * seconds (* */sec) (glob) (zstd !) 613 transferred 98.8 KB in * seconds (* */sec) (glob) (zstd !)
484 searching for changes 614 searching for changes
485 no changes found 615 no changes found
486 #endif 616 #endif
487 #if stream-bundle2 617 #if stream-bundle2-v2
618 $ hg clone --stream -U http://localhost:$HGPORT secret-allowed
619 streaming all changes
620 1093 files to transfer, 102 KB of data (no-zstd !)
621 transferred 102 KB in * seconds (* */sec) (glob) (no-zstd !)
622 1093 files to transfer, 98.9 KB of data (zstd !)
623 transferred 98.9 KB in * seconds (* */sec) (glob) (zstd !)
624 #endif
625 #if stream-bundle2-v3
488 $ hg clone --stream -U http://localhost:$HGPORT secret-allowed 626 $ hg clone --stream -U http://localhost:$HGPORT secret-allowed
489 streaming all changes 627 streaming all changes
490 1093 files to transfer, 102 KB of data (no-zstd !) 628 1093 files to transfer, 102 KB of data (no-zstd !)
491 transferred 102 KB in * seconds (* */sec) (glob) (no-zstd !) 629 transferred 102 KB in * seconds (* */sec) (glob) (no-zstd !)
492 1093 files to transfer, 98.9 KB of data (zstd !) 630 1093 files to transfer, 98.9 KB of data (zstd !)
636 searching for changes 774 searching for changes
637 no changes found 775 no changes found
638 updating to branch default 776 updating to branch default
639 1088 files updated, 0 files merged, 0 files removed, 0 files unresolved 777 1088 files updated, 0 files merged, 0 files removed, 0 files unresolved
640 #endif 778 #endif
641 #if stream-bundle2 779 #if stream-bundle2-v2
780 $ hg clone --stream http://localhost:$HGPORT with-bookmarks
781 streaming all changes
782 1096 files to transfer, 102 KB of data (no-zstd !)
783 transferred 102 KB in * seconds (* */sec) (glob) (no-zstd !)
784 1096 files to transfer, 99.1 KB of data (zstd !)
785 transferred 99.1 KB in * seconds (* */sec) (glob) (zstd !)
786 updating to branch default
787 1088 files updated, 0 files merged, 0 files removed, 0 files unresolved
788 #endif
789 #if stream-bundle2-v3
642 $ hg clone --stream http://localhost:$HGPORT with-bookmarks 790 $ hg clone --stream http://localhost:$HGPORT with-bookmarks
643 streaming all changes 791 streaming all changes
644 1096 files to transfer, 102 KB of data (no-zstd !) 792 1096 files to transfer, 102 KB of data (no-zstd !)
645 transferred 102 KB in * seconds (* */sec) (glob) (no-zstd !) 793 transferred 102 KB in * seconds (* */sec) (glob) (no-zstd !)
646 1096 files to transfer, 99.1 KB of data (zstd !) 794 1096 files to transfer, 99.1 KB of data (zstd !)
672 searching for changes 820 searching for changes
673 no changes found 821 no changes found
674 updating to branch default 822 updating to branch default
675 1088 files updated, 0 files merged, 0 files removed, 0 files unresolved 823 1088 files updated, 0 files merged, 0 files removed, 0 files unresolved
676 #endif 824 #endif
677 #if stream-bundle2 825 #if stream-bundle2-v2
826 $ hg clone --stream http://localhost:$HGPORT phase-publish
827 streaming all changes
828 1096 files to transfer, 102 KB of data (no-zstd !)
829 transferred 102 KB in * seconds (* */sec) (glob) (no-zstd !)
830 1096 files to transfer, 99.1 KB of data (zstd !)
831 transferred 99.1 KB in * seconds (* */sec) (glob) (zstd !)
832 updating to branch default
833 1088 files updated, 0 files merged, 0 files removed, 0 files unresolved
834 #endif
835 #if stream-bundle2-v3
678 $ hg clone --stream http://localhost:$HGPORT phase-publish 836 $ hg clone --stream http://localhost:$HGPORT phase-publish
679 streaming all changes 837 streaming all changes
680 1096 files to transfer, 102 KB of data (no-zstd !) 838 1096 files to transfer, 102 KB of data (no-zstd !)
681 transferred 102 KB in * seconds (* */sec) (glob) (no-zstd !) 839 transferred 102 KB in * seconds (* */sec) (glob) (no-zstd !)
682 1096 files to transfer, 99.1 KB of data (zstd !) 840 1096 files to transfer, 99.1 KB of data (zstd !)
718 $ hg -R phase-no-publish phase -r 'all()' 876 $ hg -R phase-no-publish phase -r 'all()'
719 0: public 877 0: public
720 1: public 878 1: public
721 2: public 879 2: public
722 #endif 880 #endif
723 #if stream-bundle2 881 #if stream-bundle2-v2
724 $ hg clone --stream http://localhost:$HGPORT phase-no-publish 882 $ hg clone --stream http://localhost:$HGPORT phase-no-publish
725 streaming all changes 883 streaming all changes
726 1097 files to transfer, 102 KB of data (no-zstd !) 884 1097 files to transfer, 102 KB of data (no-zstd !)
727 transferred 102 KB in * seconds (* */sec) (glob) (no-zstd !) 885 transferred 102 KB in * seconds (* */sec) (glob) (no-zstd !)
728 1097 files to transfer, 99.1 KB of data (zstd !) 886 1097 files to transfer, 99.1 KB of data (zstd !)
732 $ hg -R phase-no-publish phase -r 'all()' 890 $ hg -R phase-no-publish phase -r 'all()'
733 0: draft 891 0: draft
734 1: draft 892 1: draft
735 2: draft 893 2: draft
736 #endif 894 #endif
895 #if stream-bundle2-v3
896 $ hg clone --stream http://localhost:$HGPORT phase-no-publish
897 streaming all changes
898 1097 files to transfer, 102 KB of data (no-zstd !)
899 transferred 102 KB in * seconds (* */sec) (glob) (no-zstd !)
900 1097 files to transfer, 99.1 KB of data (zstd !)
901 transferred 99.1 KB in * seconds (* */sec) (glob) (zstd !)
902 updating to branch default
903 1088 files updated, 0 files merged, 0 files removed, 0 files unresolved
904 $ hg -R phase-no-publish phase -r 'all()'
905 0: draft
906 1: draft
907 2: draft
908 #endif
737 $ hg verify -R phase-no-publish -q 909 $ hg verify -R phase-no-publish -q
738 910
739 $ killdaemons.py 911 $ killdaemons.py
740 912
741 #if stream-legacy 913 #if stream-legacy
742 914
743 With v1 of the stream protocol, changeset are always cloned as public. There's 915 With v1 of the stream protocol, changeset are always cloned as public. There's
744 no obsolescence markers exchange in stream v1. 916 no obsolescence markers exchange in stream v1.
745 917
746 #endif 918 #endif
747 #if stream-bundle2 919 #if stream-bundle2-v2
748 920
749 Stream repository with obsolescence 921 Stream repository with obsolescence
750 ----------------------------------- 922 -----------------------------------
751 923
752 Clone non-publishing with obsolescence 924 Clone non-publishing with obsolescence
792 [100] 964 [100]
793 965
794 $ killdaemons.py 966 $ killdaemons.py
795 967
796 #endif 968 #endif
969 #if stream-bundle2-v3
970
971 Stream repository with obsolescence
972 -----------------------------------
973
974 Clone non-publishing with obsolescence
975
976 $ cat >> $HGRCPATH << EOF
977 > [experimental]
978 > evolution=all
979 > EOF
980
981 $ cd server
982 $ echo foo > foo
983 $ hg -q commit -m 'about to be pruned'
984 $ hg debugobsolete `hg log -r . -T '{node}'` -d '0 0' -u test --record-parents
985 1 new obsolescence markers
986 obsoleted 1 changesets
987 $ hg up null -q
988 $ hg log -T '{rev}: {phase}\n'
989 2: draft
990 1: draft
991 0: draft
992 $ hg serve -p $HGPORT -d --pid-file=hg.pid
993 $ cat hg.pid > $DAEMON_PIDS
994 $ cd ..
995
996 $ hg clone -U --stream http://localhost:$HGPORT with-obsolescence
997 streaming all changes
998 1098 files to transfer, 102 KB of data (no-zstd !)
999 transferred 102 KB in * seconds (* */sec) (glob) (no-zstd !)
1000 1098 files to transfer, 99.5 KB of data (zstd !)
1001 transferred 99.5 KB in * seconds (* */sec) (glob) (zstd !)
1002 $ hg -R with-obsolescence log -T '{rev}: {phase}\n'
1003 2: draft
1004 1: draft
1005 0: draft
1006 $ hg debugobsolete -R with-obsolescence
1007 8c206a663911c1f97f2f9d7382e417ae55872cfa 0 {5223b5e3265f0df40bb743da62249413d74ac70f} (Thu Jan 01 00:00:00 1970 +0000) {'user': 'test'}
1008 $ hg verify -R with-obsolescence -q
1009
1010 $ hg clone -U --stream --config experimental.evolution=0 http://localhost:$HGPORT with-obsolescence-no-evolution
1011 streaming all changes
1012 remote: abort: server has obsolescence markers, but client cannot receive them via stream clone
1013 abort: pull failed on remote
1014 [100]
1015
1016 $ killdaemons.py
1017
1018 #endif
797 1019
798 Cloning a repo with no requirements doesn't give some obscure error 1020 Cloning a repo with no requirements doesn't give some obscure error
799 1021
800 $ mkdir -p empty-repo/.hg 1022 $ mkdir -p empty-repo/.hg
801 $ hg clone -q --stream ssh://user@dummy/empty-repo empty-repo2 1023 $ hg clone -q --stream ssh://user@dummy/empty-repo empty-repo2