tests/test-websub.t

streamclone: use backgroundfilecloser (issue4889)

Closing files that have been appended to is slow on Windows/NTFS.
CloseHandle() calls on this platform often take 1-10ms - and that's
on my i7-6700K Skylake processor with a modern and fast SSD. Contrast
with other I/O operations, such as writing data, which take <100us.

This means that creating/appending thousands of files can add
significant overhead. For example, cloning mozilla-central creates
~232,000 revlog files. Assuming 1ms per CloseHandle(), that yields
232s (3:52) of wall time waiting for file closes!

The impact of this overhead can be measured most directly when applying
stream clone bundles. Applying these files is effectively uncompressing
a tar archive (read: it's very fast).

Using a RAM disk (read: no I/O wait), the difference in wall time for a
`hg debugapplystreamclonebundle` for a ~1731 MB mozilla-central bundle
between Windows and Linux from the same machine is drastic:

Linux:    ~12.8s (128MB/s)
Windows: ~352.0s (4.7MB/s)

Windows is ~27.5x slower. Yikes!

After this patch:

Linux:    ~12.8s (128MB/s)
Windows: ~102.1s (16.1MB/s)

Windows is now ~3.4x faster. Unfortunately, it is still ~8x slower than
Linux. Profiling reveals a few hot code paths that could likely be
improved. But those are for other patches.

This patch introduces test-clone-uncompressed.t because existing tests
of `clone --uncompressed` are scattered about and adding a variation for
background thread closing to e.g. test-http.t doesn't feel correct.

#require serve

  $ hg init test
  $ cd test

  $ cat > .hg/hgrc <<EOF
  > [extensions]
  > # this is only necessary to check that the mapping from
  > # interhg to websub works
  > interhg =
  > 
  > [websub]
  > issues = s|Issue(\d+)|<a href="http://bts.example.org/issue\1">Issue\1</a>|
  > 
  > [interhg]
  > # check that we maintain some interhg backwards compatibility...
  > # yes, 'x' is a weird delimiter...
  > markbugs = sxbugx<i class="\x">bug</i>x
  > EOF

  $ touch foo
  $ hg add foo
  $ hg commit -d '1 0' -m 'Issue123: fixed the bug!'

  $ hg serve -n test -p $HGPORT -d --pid-file=hg.pid -A access.log -E errors.log
  $ cat hg.pid >> $DAEMON_PIDS

log

  $ get-with-headers.py localhost:$HGPORT "rev/tip" | grep bts
  <div class="description"><a href="http://bts.example.org/issue123">Issue123</a>: fixed the <i class="x">bug</i>!</div>
errors

  $ cat errors.log

  $ cd ..