view tests/test-mq-qrefresh-interactive.t @ 44363:f7459da77f23

nodemap: introduce an option to use mmap to read the nodemap mapping

The performance and memory benefit is much greater if we don't have to copy all
the data in memory for each information. So we introduce an option (on by
default) to read the data using mmap.

This changeset is the last one definition the API for index support nodemap
data. (they have to be able to use the mmaping).

Below are some benchmark comparing the best we currently have in 5.3 with the
final step of this series (using the persistent nodemap implementation in
Rust). The benchmark run `hg perfindex` with various revset and the following
variants:

Before:

 * do not use the persistent nodemap
 * use the CPython implementation of the index for nodemap
 * use mmapping of the changelog index

After:

 * use the MixedIndex Rust code, with the NodeTree object for nodemap access
   (still in review)
 * use the persistent nodemap data from disk
 * access the persistent nodemap data through mmap
 * use mmapping of the changelog index

The persistent nodemap greatly speed up  most operation on very large
repositories. Some of the previously very fast lookup end up a bit slower because
the persistent nodemap has to be setup. However the absolute slowdown is very
small and won't matters in the big picture.

Here are some numbers (in seconds) for the reference copy of mozilla-try:

Revset            Before   After     abs-change speedup

-10000:           0.004622 0.005532   0.000910  ×   0.83
-10:              0.000050 0.000132   0.000082  ×   0.37
tip               0.000052 0.000085   0.000033  ×   0.61
0 + (-10000:)     0.028222 0.005337  -0.022885  ×   5.29
0                 0.023521 0.000084  -0.023437  × 280.01
(-10000:) + 0     0.235539 0.005308  -0.230231  ×  44.37
(-10:) + :9       0.232883 0.000180  -0.232703  ×1293.79
(-10000:) + (:99) 0.238735 0.005358  -0.233377  ×  44.55
:99 + (-10000:)   0.317942 0.005593  -0.312349  ×  56.84
:9 + (-10:)       0.313372 0.000179  -0.313193  ×1750.68
:9                0.316450 0.000143  -0.316307  ×2212.93


On smaller repositories, the cost of nodemap related operation is not as big, so
the win is much more modest. Yet it helps shaving a handful of millisecond here
and there.

Here are some numbers (in seconds) for the reference copy of mercurial:

Revset            Before   After     abs-change speedup
-10:              0.000065 0.000097   0.000032  × 0.67
tip               0.000063 0.000078   0.000015  × 0.80
0                 0.000561 0.000079  -0.000482  × 7.10
-10000:           0.004609 0.003648  -0.000961  × 1.26
0 + (-10000:)     0.005023 0.003715  -0.001307  × 1.35
(-10:) + :9       0.002187 0.000108  -0.002079  ×20.25
(-10000:) + 0     0.006252 0.003716  -0.002536  × 1.68
(-10000:) + (:99) 0.006367 0.003707  -0.002660  × 1.71
:9 + (-10:)       0.003846 0.000110  -0.003736  ×34.96
:9                0.003854 0.000099  -0.003755  ×38.92
:99 + (-10000:)   0.007644 0.003778  -0.003866  × 2.02

Differential Revision: https://phab.mercurial-scm.org/D7894

Create configuration

  $ echo "[ui]" >> $HGRCPATH
  $ echo "interactive=true" >> $HGRCPATH

help qrefresh (no record)

  $ echo "[extensions]" >> $HGRCPATH
  $ echo "mq=" >> $HGRCPATH
  $ hg help qrefresh
  hg qrefresh [-I] [-X] [-e] [-m TEXT] [-l FILE] [-s] [FILE]...
  
  update the current patch
  
      If any file patterns are provided, the refreshed patch will contain only
      the modifications that match those patterns; the remaining modifications
      will remain in the working directory.
  
      If -s/--short is specified, files currently included in the patch will be
      refreshed just like matched files and remain in the patch.
  
      If -e/--edit is specified, Mercurial will start your configured editor for
      you to enter a message. In case qrefresh fails, you will find a backup of
      your message in ".hg/last-message.txt".
  
      hg add/remove/copy/rename work as usual, though you might want to use git-
      style patches (-g/--git or [diff] git=1) to track copies and renames. See
      the diffs help topic for more information on the git diff format.
  
      Returns 0 on success.
  
  options ([+] can be repeated):
  
   -e --edit                invoke editor on commit messages
   -g --git                 use git extended diff format
   -s --short               refresh only files already in the patch and
                            specified files
   -U --currentuser         add/update author field in patch with current user
   -u --user USER           add/update author field in patch with given user
   -D --currentdate         add/update date field in patch with current date
   -d --date DATE           add/update date field in patch with given date
   -I --include PATTERN [+] include names matching the given patterns
   -X --exclude PATTERN [+] exclude names matching the given patterns
   -m --message TEXT        use text as commit message
   -l --logfile FILE        read commit message from file
  
  (some details hidden, use --verbose to show complete help)

help qrefresh (record)

  $ echo "record=" >> $HGRCPATH
  $ hg help qrefresh
  hg qrefresh [-I] [-X] [-e] [-m TEXT] [-l FILE] [-s] [FILE]...
  
  update the current patch
  
      If any file patterns are provided, the refreshed patch will contain only
      the modifications that match those patterns; the remaining modifications
      will remain in the working directory.
  
      If -s/--short is specified, files currently included in the patch will be
      refreshed just like matched files and remain in the patch.
  
      If -e/--edit is specified, Mercurial will start your configured editor for
      you to enter a message. In case qrefresh fails, you will find a backup of
      your message in ".hg/last-message.txt".
  
      hg add/remove/copy/rename work as usual, though you might want to use git-
      style patches (-g/--git or [diff] git=1) to track copies and renames. See
      the diffs help topic for more information on the git diff format.
  
      Returns 0 on success.
  
  options ([+] can be repeated):
  
   -e --edit                invoke editor on commit messages
   -g --git                 use git extended diff format
   -s --short               refresh only files already in the patch and
                            specified files
   -U --currentuser         add/update author field in patch with current user
   -u --user USER           add/update author field in patch with given user
   -D --currentdate         add/update date field in patch with current date
   -d --date DATE           add/update date field in patch with given date
   -I --include PATTERN [+] include names matching the given patterns
   -X --exclude PATTERN [+] exclude names matching the given patterns
   -m --message TEXT        use text as commit message
   -l --logfile FILE        read commit message from file
   -i --interactive         interactively select changes to refresh
  
  (some details hidden, use --verbose to show complete help)

  $ hg init a
  $ cd a

Base commit

  $ cat > 1.txt <<EOF
  > 1
  > 2
  > 3
  > 4
  > 5
  > EOF
  $ cat > 2.txt <<EOF
  > a
  > b
  > c
  > d
  > e
  > f
  > EOF

  $ mkdir dir
  $ cat > dir/a.txt <<EOF
  > hello world
  > 
  > someone
  > up
  > there
  > loves
  > me
  > EOF

  $ hg add 1.txt 2.txt dir/a.txt
  $ hg commit -m aaa
  $ hg qrecord --config ui.interactive=false patch
  abort: running non-interactively, use qnew instead
  [255]
  $ hg qnew -i --config ui.interactive=false patch
  abort: running non-interactively
  [255]
  $ hg qnew -d '0 0' patch

Changing files

  $ sed -e 's/2/2 2/;s/4/4 4/' 1.txt > 1.txt.new
  $ sed -e 's/b/b b/' 2.txt > 2.txt.new
  $ sed -e 's/hello world/hello world!/' dir/a.txt > dir/a.txt.new

  $ mv -f 1.txt.new 1.txt
  $ mv -f 2.txt.new 2.txt
  $ mv -f dir/a.txt.new dir/a.txt

Whole diff

  $ hg diff --nodates
  diff -r ed27675cb5df 1.txt
  --- a/1.txt
  +++ b/1.txt
  @@ -1,5 +1,5 @@
   1
  -2
  +2 2
   3
  -4
  +4 4
   5
  diff -r ed27675cb5df 2.txt
  --- a/2.txt
  +++ b/2.txt
  @@ -1,5 +1,5 @@
   a
  -b
  +b b
   c
   d
   e
  diff -r ed27675cb5df dir/a.txt
  --- a/dir/a.txt
  +++ b/dir/a.txt
  @@ -1,4 +1,4 @@
  -hello world
  +hello world!
   
   someone
   up

partial qrefresh

  $ hg qrefresh -i --config ui.interactive=false
  abort: running non-interactively
  [255]
  $ hg qrefresh -i -d '0 0' <<EOF
  > y
  > y
  > n
  > y
  > y
  > n
  > EOF
  diff --git a/1.txt b/1.txt
  2 hunks, 2 lines changed
  examine changes to '1.txt'?
  (enter ? for help) [Ynesfdaq?] y
  
  @@ -1,3 +1,3 @@
   1
  -2
  +2 2
   3
  record change 1/4 to '1.txt'?
  (enter ? for help) [Ynesfdaq?] y
  
  @@ -3,3 +3,3 @@
   3
  -4
  +4 4
   5
  record change 2/4 to '1.txt'?
  (enter ? for help) [Ynesfdaq?] n
  
  diff --git a/2.txt b/2.txt
  1 hunks, 1 lines changed
  examine changes to '2.txt'?
  (enter ? for help) [Ynesfdaq?] y
  
  @@ -1,5 +1,5 @@
   a
  -b
  +b b
   c
   d
   e
  record change 3/4 to '2.txt'?
  (enter ? for help) [Ynesfdaq?] y
  
  diff --git a/dir/a.txt b/dir/a.txt
  1 hunks, 1 lines changed
  examine changes to 'dir/a.txt'?
  (enter ? for help) [Ynesfdaq?] n
  

After partial qrefresh 'tip'

  $ hg tip -p
  changeset:   1:0738af1a8211
  tag:         patch
  tag:         qbase
  tag:         qtip
  tag:         tip
  user:        test
  date:        Thu Jan 01 00:00:00 1970 +0000
  summary:     [mq]: patch
  
  diff -r 1fd39ab63a33 -r 0738af1a8211 1.txt
  --- a/1.txt	Thu Jan 01 00:00:00 1970 +0000
  +++ b/1.txt	Thu Jan 01 00:00:00 1970 +0000
  @@ -1,5 +1,5 @@
   1
  -2
  +2 2
   3
   4
   5
  diff -r 1fd39ab63a33 -r 0738af1a8211 2.txt
  --- a/2.txt	Thu Jan 01 00:00:00 1970 +0000
  +++ b/2.txt	Thu Jan 01 00:00:00 1970 +0000
  @@ -1,5 +1,5 @@
   a
  -b
  +b b
   c
   d
   e
  
After partial qrefresh 'diff'

  $ hg diff --nodates
  diff -r 0738af1a8211 1.txt
  --- a/1.txt
  +++ b/1.txt
  @@ -1,5 +1,5 @@
   1
   2 2
   3
  -4
  +4 4
   5
  diff -r 0738af1a8211 dir/a.txt
  --- a/dir/a.txt
  +++ b/dir/a.txt
  @@ -1,4 +1,4 @@
  -hello world
  +hello world!
   
   someone
   up

qrefresh interactively everything else

  $ hg qrefresh -i -d '0 0' <<EOF
  > y
  > y
  > y
  > y
  > EOF
  diff --git a/1.txt b/1.txt
  1 hunks, 1 lines changed
  examine changes to '1.txt'?
  (enter ? for help) [Ynesfdaq?] y
  
  @@ -1,5 +1,5 @@
   1
   2 2
   3
  -4
  +4 4
   5
  record change 1/2 to '1.txt'?
  (enter ? for help) [Ynesfdaq?] y
  
  diff --git a/dir/a.txt b/dir/a.txt
  1 hunks, 1 lines changed
  examine changes to 'dir/a.txt'?
  (enter ? for help) [Ynesfdaq?] y
  
  @@ -1,4 +1,4 @@
  -hello world
  +hello world!
   
   someone
   up
  record change 2/2 to 'dir/a.txt'?
  (enter ? for help) [Ynesfdaq?] y
  

After final qrefresh 'tip'

  $ hg tip -p
  changeset:   1:2c3f66afeed9
  tag:         patch
  tag:         qbase
  tag:         qtip
  tag:         tip
  user:        test
  date:        Thu Jan 01 00:00:00 1970 +0000
  summary:     [mq]: patch
  
  diff -r 1fd39ab63a33 -r 2c3f66afeed9 1.txt
  --- a/1.txt	Thu Jan 01 00:00:00 1970 +0000
  +++ b/1.txt	Thu Jan 01 00:00:00 1970 +0000
  @@ -1,5 +1,5 @@
   1
  -2
  +2 2
   3
  -4
  +4 4
   5
  diff -r 1fd39ab63a33 -r 2c3f66afeed9 2.txt
  --- a/2.txt	Thu Jan 01 00:00:00 1970 +0000
  +++ b/2.txt	Thu Jan 01 00:00:00 1970 +0000
  @@ -1,5 +1,5 @@
   a
  -b
  +b b
   c
   d
   e
  diff -r 1fd39ab63a33 -r 2c3f66afeed9 dir/a.txt
  --- a/dir/a.txt	Thu Jan 01 00:00:00 1970 +0000
  +++ b/dir/a.txt	Thu Jan 01 00:00:00 1970 +0000
  @@ -1,4 +1,4 @@
  -hello world
  +hello world!
   
   someone
   up
  

After qrefresh 'diff'

  $ hg diff --nodates

  $ cd ..