view tests/test-mq-merge.t @ 39561:d06834e0f48e

wireprotov2peer: stream decoded responses

Previously, wire protocol version 2 would buffer all response data.
Only once all data was received did we CBOR decode it and resolve
the future associated with the command. This was obviously not
desirable. In future commits that introduce large response payloads,
this caused significant memory bloat and slowed down client
operations due to waiting on the server.

This commit refactors the response handling code so that response
data can be streamed.

Command response objects now contain a buffered CBOR decoder. As
new data arrives, it is fed into the decoder. Decoded objects are
made available to the generator as they are decoded.

Because there is a separate thread processing incoming frames and
feeding data into the response object, there is the potential for
race conditions when mutating response objects. So a lock has been
added to guard access to critical state variables.

Because the generator emitting decoded objects needs to wait on
those objects to become available, we've added an Event for the
generator to wait on so it doesn't busy loop. This does mean
there is the potential for deadlocks. And I'm pretty sure they can
occur in some scenarios. We already have a handful of TODOs around
this. But I've added some more. Fixing this will likely require
moving the background thread receiving frames into clienthandler.
We likely would have done this anyway when implementing the client
bits for the SSH transport.

Test output changes because the initial CBOR map holding the overall
response state is now always handled internally by the response
object.

Differential Revision: https://phab.mercurial-scm.org/D4474

Setup extension:

  $ cat <<EOF >> $HGRCPATH
  > [extensions]
  > mq =
  > [mq]
  > git = keep
  > EOF

Test merge with mq changeset as the second parent:

  $ hg init m
  $ cd m
  $ touch a b c
  $ hg add a
  $ hg commit -m a
  $ hg add b
  $ hg qnew -d "0 0" b
  $ hg update 0
  0 files updated, 0 files merged, 1 files removed, 0 files unresolved
  $ hg add c
  $ hg commit -m c
  created new head
  $ hg merge
  1 files updated, 0 files merged, 0 files removed, 0 files unresolved
  (branch merge, don't forget to commit)
  $ hg commit -m merge
  abort: cannot commit over an applied mq patch
  [255]
  $ cd ..

Issue529: mq aborts when merging patch deleting files

  $ checkundo()
  > {
  >     if [ -f .hg/store/undo ]; then
  >         echo ".hg/store/undo still exists"
  >     fi
  > }

Commit two dummy files in "init" changeset:

  $ hg init t
  $ cd t
  $ echo a > a
  $ echo b > b
  $ hg ci -Am init
  adding a
  adding b
  $ hg tag -l init

Create a patch removing a:

  $ hg qnew rm_a
  $ hg rm a
  $ hg qrefresh -m "rm a"

Save the patch queue so we can merge it later:

  $ hg qsave -c -e
  copy $TESTTMP/t/.hg/patches to $TESTTMP/t/.hg/patches.1
  $ checkundo

Update b and commit in an "update" changeset:

  $ hg up -C init
  1 files updated, 0 files merged, 0 files removed, 0 files unresolved
  $ echo b >> b
  $ hg st
  M b
  $ hg ci -m update
  created new head

# Here, qpush used to abort with :
# The system cannot find the file specified => a
  $ hg manifest
  a
  b

  $ hg qpush -a -m
  merging with queue at: $TESTTMP/t/.hg/patches.1
  applying rm_a
  now at: rm_a

  $ checkundo
  $ hg manifest
  b

Ensure status is correct after merge:

  $ hg qpop -a
  popping rm_a
  popping .hg.patches.merge.marker
  patch queue now empty

  $ cd ..

Classic MQ merge sequence *with an explicit named queue*:

  $ hg init t2
  $ cd t2
  $ echo '[diff]' > .hg/hgrc
  $ echo 'nodates = 1' >> .hg/hgrc
  $ echo a > a
  $ hg ci -Am init
  adding a
  $ echo b > a
  $ hg ci -m changea
  $ hg up -C 0
  1 files updated, 0 files merged, 0 files removed, 0 files unresolved
  $ hg cp a aa
  $ echo c >> a
  $ hg qnew --git -f -e patcha
  $ echo d >> a
  $ hg qnew -d '0 0' -f -e patcha2

Create the reference queue:

  $ hg qsave -c -e -n refqueue
  copy $TESTTMP/t2/.hg/patches to $TESTTMP/t2/.hg/refqueue
  $ hg up -C 1
  1 files updated, 0 files merged, 1 files removed, 0 files unresolved

Merge:

  $ HGMERGE=internal:other hg qpush -a -m -n refqueue
  merging with queue at: $TESTTMP/t2/.hg/refqueue
  applying patcha
  patching file a
  Hunk #1 succeeded at 2 with fuzz 1 (offset 0 lines).
  fuzz found when applying patch, stopping
  patch didn't work out, merging patcha
  1 files updated, 0 files merged, 1 files removed, 0 files unresolved
  0 files updated, 2 files merged, 0 files removed, 0 files unresolved
  (branch merge, don't forget to commit)
  applying patcha2
  now at: patcha2

Check patcha is still a git patch:

  $ cat .hg/patches/patcha
  # HG changeset patch
  # Parent  d3873e73d99ef67873dac33fbcc66268d5d2b6f4
  
  diff --git a/a b/a
  --- a/a
  +++ b/a
  @@ -1,1 +1,2 @@
  -b
  +a
  +c
  diff --git a/a b/aa
  copy from a
  copy to aa
  --- a/a
  +++ b/aa
  @@ -1,1 +1,1 @@
  -b
  +a

Check patcha2 is still a regular patch:

  $ cat .hg/patches/patcha2
  # HG changeset patch
  # Date 0 0
  # Parent  ???????????????????????????????????????? (glob)
  
  diff -r ???????????? -r ???????????? a (glob)
  --- a/a
  +++ b/a
  @@ -1,2 +1,3 @@
   a
   c
  +d

  $ cd ..