tests/test-import-eol.t

util: lower water mark when removing nodes after cost limit reached

See the inline comment for the reasoning here. This is a pretty
common strategy for garbage collectors, other cache-like primtives.

The performance impact is substantial:

$ hg perflrucachedict --size 4 --gets 1000000 --sets 1000000 --mixed 1000000 --costlimit 100
! inserts w/ cost limit
! wall 1.659181 comb 1.650000 user 1.650000 sys 0.000000 (best of 7)
! wall 1.722122 comb 1.720000 user 1.720000 sys 0.000000 (best of 6)
! mixed w/ cost limit
! wall 1.139955 comb 1.140000 user 1.140000 sys 0.000000 (best of 9)
! wall 1.182513 comb 1.180000 user 1.180000 sys 0.000000 (best of 9)

$ hg perflrucachedict --size 1000 --gets 1000000 --sets 1000000 --mixed 1000000 --costlimit 10000
! inserts
! wall 0.679546 comb 0.680000 user 0.680000 sys 0.000000 (best of 15)
! sets
! wall 0.825147 comb 0.830000 user 0.830000 sys 0.000000 (best of 13)
! inserts w/ cost limit
! wall 25.105273 comb 25.080000 user 25.080000 sys 0.000000 (best of 3)
! wall  1.724397 comb  1.720000 user  1.720000 sys 0.000000 (best of 6)
! mixed
! wall 0.807096 comb 0.810000 user 0.810000 sys 0.000000 (best of 13)
! mixed w/ cost limit
! wall 12.104470 comb 12.070000 user 12.070000 sys 0.000000 (best of 3)
! wall  1.190563 comb  1.190000 user  1.190000 sys 0.000000 (best of 9)

$ hg perflrucachedict --size 1000 --gets 1000000 --sets 1000000 --mixed 1000000 --costlimit 10000 --mixedgetfreq 90
! inserts
! wall 0.711177 comb 0.710000 user 0.710000 sys 0.000000 (best of 14)
! sets
! wall 0.846992 comb 0.850000 user 0.850000 sys 0.000000 (best of 12)
! inserts w/ cost limit
! wall 25.963028 comb 25.960000 user 25.960000 sys 0.000000 (best of 3)
! wall  2.184311 comb  2.180000 user  2.180000 sys 0.000000 (best of 5)
! mixed
! wall 0.728256 comb 0.730000 user 0.730000 sys 0.000000 (best of 14)
! mixed w/ cost limit
! wall 3.174256 comb 3.170000 user 3.170000 sys 0.000000 (best of 4)
! wall 0.773186 comb 0.770000 user 0.770000 sys 0.000000 (best of 13)

$ hg perflrucachedict --size 100000 --gets 1000000 --sets 1000000 --mixed 1000000 --mixedgetfreq 90 --costlimit 5000000
! gets
! wall 1.191368 comb 1.190000 user 1.190000 sys 0.000000 (best of 9)
! wall 1.195304 comb 1.190000 user 1.190000 sys 0.000000 (best of 9)
! inserts
! wall 0.950995 comb 0.950000 user 0.950000 sys 0.000000 (best of 11)
! inserts w/ cost limit
! wall 1.589732 comb 1.590000 user 1.590000 sys 0.000000 (best of 7)
! sets
! wall 1.094941 comb 1.100000 user 1.090000 sys 0.010000 (best of 9)
! mixed
! wall 0.936420 comb 0.940000 user 0.930000 sys 0.010000 (best of 10)
! mixed w/ cost limit
! wall 0.882780 comb 0.870000 user 0.870000 sys 0.000000 (best of 11)

This puts us ~2x slower than caches without cost accounting. And for
read-heavy workloads (the prime use cases for caches), performance is
nearly identical.

In the worst case (pure write workloads with cost accounting enabled),
we're looking at ~1.5us per insert on large caches. That seems "fast
enough."

Differential Revision: https://phab.mercurial-scm.org/D4505

  $ cat > makepatch.py <<EOF
  > import sys
  > f = open(sys.argv[2], 'wb')
  > w = f.write
  > w(b'test message\n')
  > w(b'diff --git a/a b/a\n')
  > w(b'--- a/a\n')
  > w(b'+++ b/a\n')
  > w(b'@@ -1,5 +1,5 @@\n')
  > w(b' a\n')
  > w(b'-bbb\r\n')
  > w(b'+yyyy\r\n')
  > w(b' cc\r\n')
  > w({'empty:lf': b' \n',
  >    'empty:crlf': b' \r\n',
  >    'empty:stripped-lf': b'\n',
  >    'empty:stripped-crlf': b'\r\n'}[sys.argv[1]])
  > w(b' d\n')
  > w(b'-e\n')
  > w(b'\ No newline at end of file\n')
  > w(b'+z\r\n')
  > w(b'\ No newline at end of file\r\n')
  > EOF

  $ hg init repo
  $ cd repo
  $ echo '\.diff' > .hgignore


Test different --eol values

  $ $PYTHON -c 'open("a", "wb").write(b"a\nbbb\ncc\n\nd\ne")'
  $ hg ci -Am adda
  adding .hgignore
  adding a
  $ $PYTHON ../makepatch.py empty:lf eol.diff
  $ $PYTHON ../makepatch.py empty:crlf eol-empty-crlf.diff
  $ $PYTHON ../makepatch.py empty:stripped-lf eol-empty-stripped-lf.diff
  $ $PYTHON ../makepatch.py empty:stripped-crlf eol-empty-stripped-crlf.diff

invalid eol

  $ hg --config patch.eol='LFCR' import eol.diff
  applying eol.diff
  abort: unsupported line endings type: LFCR
  [255]
  $ hg revert -a


force LF

  $ hg --traceback --config patch.eol='LF' import eol.diff
  applying eol.diff
  $ hg id
  9e4ef7b3d4af tip
  $ cat a
  a
  yyyy
  cc
  
  d
  e (no-eol)
  $ hg st

 (test empty-line variants: all of them should generate the same revision)

  $ hg up -qC 0
  $ hg --config patch.eol='LF' import eol-empty-crlf.diff
  applying eol-empty-crlf.diff
  $ hg id
  9e4ef7b3d4af tip

  $ hg up -qC 0
  $ hg --config patch.eol='LF' import eol-empty-stripped-lf.diff
  applying eol-empty-stripped-lf.diff
  $ hg id
  9e4ef7b3d4af tip

  $ hg up -qC 0
  $ hg --config patch.eol='LF' import eol-empty-stripped-crlf.diff
  applying eol-empty-stripped-crlf.diff
  $ hg id
  9e4ef7b3d4af tip

force CRLF

  $ hg up -C 0
  1 files updated, 0 files merged, 0 files removed, 0 files unresolved
  $ hg --traceback --config patch.eol='CRLF' import eol.diff
  applying eol.diff
  $ cat a
  a\r (esc)
  yyyy\r (esc)
  cc\r (esc)
  \r (esc)
  d\r (esc)
  e (no-eol)
  $ hg st


auto EOL on LF file

  $ hg up -C 0
  1 files updated, 0 files merged, 0 files removed, 0 files unresolved
  $ hg --traceback --config patch.eol='auto' import eol.diff
  applying eol.diff
  $ cat a
  a
  yyyy
  cc
  
  d
  e (no-eol)
  $ hg st


auto EOL on CRLF file

  $ $PYTHON -c 'open("a", "wb").write(b"a\r\nbbb\r\ncc\r\n\r\nd\r\ne")'
  $ hg commit -m 'switch EOLs in a'
  $ hg --traceback --config patch.eol='auto' import eol.diff
  applying eol.diff
  $ cat a
  a\r (esc)
  yyyy\r (esc)
  cc\r (esc)
  \r (esc)
  d\r (esc)
  e (no-eol)
  $ hg st


auto EOL on new file or source without any EOL

  $ $PYTHON -c 'open("noeol", "wb").write(b"noeol")'
  $ hg add noeol
  $ hg commit -m 'add noeol'
  $ $PYTHON -c 'open("noeol", "wb").write(b"noeol\r\nnoeol\n")'
  $ $PYTHON -c 'open("neweol", "wb").write(b"neweol\nneweol\r\n")'
  $ hg add neweol
  $ hg diff --git > noeol.diff
  $ hg revert --no-backup noeol neweol
  $ rm neweol
  $ hg --traceback --config patch.eol='auto' import -m noeol noeol.diff
  applying noeol.diff
  $ cat noeol
  noeol\r (esc)
  noeol
  $ cat neweol
  neweol
  neweol\r (esc)
  $ hg st


Test --eol and binary patches

  $ $PYTHON -c 'open("b", "wb").write(b"a\x00\nb\r\nd")'
  $ hg ci -Am addb
  adding b
  $ $PYTHON -c 'open("b", "wb").write(b"a\x00\nc\r\nd")'
  $ hg diff --git > bin.diff
  $ hg revert --no-backup b

binary patch with --eol

  $ hg import --config patch.eol='CRLF' -m changeb bin.diff
  applying bin.diff
  $ cat b
  a\x00 (esc)
  c\r (esc)
  d (no-eol)
  $ hg st
  $ cd ..