.hgignore
author Gregory Szorc <gregory.szorc@gmail.com>
Sat, 20 Jan 2018 22:55:42 -0800
changeset 35793 4fb2bb61597c
parent 35569 964212780daf
child 38003 1335bbfb066f
permissions -rw-r--r--
bundle2: increase payload part chunk size to 32kb Bundle2 payload parts are framed chunks. Esentially, we obtain data in equal size chunks of size `preferedchunksize` and emit those to a generator. That generator is fed into a compressor (which can be the no-op compressor, which just re-emits the generator). And the output from the compressor likely goes to a file descriptor or socket. What this means is that small chunk sizes create more Python objects and Python function calls than larger chunk sizes. And as we know, Python object and function call overhead in performance sensitive code matters (at least with CPython). This commit increases the bundle2 part payload chunk size from 4k to 32k. Practically speaking, this means that the chunks we feed into a compressor (implemented in C code) or feed directly into a file handle or socket write() are larger. It's possible the chunks might be larger than what the receiver can handle in one logical operation. But at that point, we're in C code, which is much more efficient at dealing with splitting up the chunk and making multiple function calls than Python is. A downside to larger chunks is that the receiver has to wait for that much data to arrive (either raw or from a decompressor) before it can process the chunk. But 32kb still feels like a small buffer to have to wait for. And in many cases, the client will convert from 8 read(4096) to 1 read(32768). That's happening in Python land. So we cut down on the number of Python objects and function calls, making the client faster as well. I don't think there are any significant concerns to increasing the payload chunk size to 32kb. The impact of this change on performance significant. Using `curl` to obtain a stream clone bundle2 payload from a server on localhost serving the mozilla-unified repository: before: 20.78 user; 7.71 system; 80.5 MB/s after: 13.90 user; 3.51 system; 132 MB/s legacy: 9.72 user; 8.16 system; 132 MB/s bundle2 stream clone generation is still more resource intensive than legacy stream clone (that's likely because of the use of a util.chunkbuffer). But the throughput is the same. We might be in territory we're this is effectively a benchmark of the networking stack or Python's syscall throughput. From the client perspective, `hg clone -U --stream`: before: 33.50 user; 7.95 system; 53.3 MB/s after: 22.82 user; 7.33 system; 72.7 MB/s legacy: 29.96 user; 7.94 system; 58.0 MB/s And for `hg clone --stream` with a working directory update of ~230k files: after: 119.55 user; 26.47 system; 0:57.08 wall legacy: 126.98 user; 26.94 system; 1:05.56 wall So, it appears that bundle2's stream clone is now definitively faster than legacy stream clone! Differential Revision: https://phab.mercurial-scm.org/D1932
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
1270
fc3b41570082 Switch to new syntax for .hgignore files.
Bryan O'Sullivan <bos@serpentine.com>
parents: 981
diff changeset
     1
syntax: glob
fc3b41570082 Switch to new syntax for .hgignore files.
Bryan O'Sullivan <bos@serpentine.com>
parents: 981
diff changeset
     2
fc3b41570082 Switch to new syntax for .hgignore files.
Bryan O'Sullivan <bos@serpentine.com>
parents: 981
diff changeset
     3
*.elc
19991
ce5d711475a3 Makefile: do update on a temporary copy of a po file
Simon Heimberg <simohe@besonet.ch>
parents: 19426
diff changeset
     4
*.tmp
1270
fc3b41570082 Switch to new syntax for .hgignore files.
Bryan O'Sullivan <bos@serpentine.com>
parents: 981
diff changeset
     5
*.orig
fc3b41570082 Switch to new syntax for .hgignore files.
Bryan O'Sullivan <bos@serpentine.com>
parents: 981
diff changeset
     6
*.rej
fc3b41570082 Switch to new syntax for .hgignore files.
Bryan O'Sullivan <bos@serpentine.com>
parents: 981
diff changeset
     7
*~
5019
e6cc4d4f5a81 Ignore mergebackup files generated by mpatch
Bryan O'Sullivan <bos@serpentine.com>
parents: 2987
diff changeset
     8
*.mergebackup
2341
dbbe7f72d15a contrib: add restricted shell.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2151
diff changeset
     9
*.o
1270
fc3b41570082 Switch to new syntax for .hgignore files.
Bryan O'Sullivan <bos@serpentine.com>
parents: 981
diff changeset
    10
*.so
13643
537899158396 hgignore: ignore dll files generated under cygwin
Craig Leres <leres@ee.lbl.gov>
parents: 13346
diff changeset
    11
*.dll
17115
b52d08b930c5 hgignore: simply ignore all *.exe's everywhere
Adrian Buehlmann <adrian@cadifra.com>
parents: 16537
diff changeset
    12
*.exe
6551
4a064e1977f8 ignore *.pyd files
Adrian Buehlmann <adrian@cadifra.com>
parents: 5693
diff changeset
    13
*.pyd
1270
fc3b41570082 Switch to new syntax for .hgignore files.
Bryan O'Sullivan <bos@serpentine.com>
parents: 981
diff changeset
    14
*.pyc
13346
91fe769ac84e hgignore: ignore more bytecode
Simon Heimberg <simohe@besonet.ch>
parents: 11380
diff changeset
    15
*.pyo
91fe769ac84e hgignore: ignore more bytecode
Simon Heimberg <simohe@besonet.ch>
parents: 11380
diff changeset
    16
*$py.class
1270
fc3b41570082 Switch to new syntax for .hgignore files.
Bryan O'Sullivan <bos@serpentine.com>
parents: 981
diff changeset
    17
*.swp
fc3b41570082 Switch to new syntax for .hgignore files.
Bryan O'Sullivan <bos@serpentine.com>
parents: 981
diff changeset
    18
*.prof
17274
2426ec322cd1 hgignore: ignore zip files
Adrian Buehlmann <adrian@cadifra.com>
parents: 17115
diff changeset
    19
*.zip
8147
441dc7becd43 win32text: be more careful about rejecting violating changesets
Bryan O'Sullivan <bos@serpentine.com>
parents: 7706
diff changeset
    20
\#*\#
441dc7becd43 win32text: be more careful about rejecting violating changesets
Bryan O'Sullivan <bos@serpentine.com>
parents: 7706
diff changeset
    21
.\#*
2068
4a49daa3a40c let run-tests run optional code coverage tests.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 1817
diff changeset
    22
tests/.coverage*
27634
a1eff44c432b tests: write recent run times to a file named tests/.testtimes
Bryan O'Sullivan <bos@serpentine.com>
parents: 22945
diff changeset
    23
tests/.testtimes*
28255
f75f7d39cca3 testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents: 28061
diff changeset
    24
tests/.hypothesis
f75f7d39cca3 testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents: 28061
diff changeset
    25
tests/hypothesis-generated
2151
0ce3cd330996 Ignore annotated coverage output of run-tests.py -C
Thomas Arendsen Hein <thomas@intevation.de>
parents: 2068
diff changeset
    26
tests/annotated
35190
bd8875b6473c run-tests: mechanism to report exceptions during test execution
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33033
diff changeset
    27
tests/exceptions
1270
fc3b41570082 Switch to new syntax for .hgignore files.
Bryan O'Sullivan <bos@serpentine.com>
parents: 981
diff changeset
    28
tests/*.err
15859
44a371823f83 tests: add htmlcov option
Markus Zapke-Gründemann <info@keimlink.de>
parents: 15702
diff changeset
    29
tests/htmlcov
1270
fc3b41570082 Switch to new syntax for .hgignore files.
Bryan O'Sullivan <bos@serpentine.com>
parents: 981
diff changeset
    30
build
28061
3753d2e1759b hgignore: ignore chg binary
Yuya Nishihara <yuya@tcha.org>
parents: 27634
diff changeset
    31
contrib/chg/chg
2341
dbbe7f72d15a contrib: add restricted shell.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2151
diff changeset
    32
contrib/hgsh/hgsh
21874
8da01b6e7b49 contrib/vagrant: use Vagrant for running tests on virtual machine
anatoly techtonik <techtonik@gmail.com>
parents: 21560
diff changeset
    33
contrib/vagrant/.vagrant
33025
5c1283713293 make: templatize Debian build target a la e63dfbbdbd07
Anton Shestakov <av6@dwimlabs.net>
parents: 32080
diff changeset
    34
contrib/docker/debian-*
29031
e63dfbbdbd07 make: turn ubuntu docker into template
Sean Farley <sean@farley.io>
parents: 28430
diff changeset
    35
contrib/docker/ubuntu-*
1270
fc3b41570082 Switch to new syntax for .hgignore files.
Bryan O'Sullivan <bos@serpentine.com>
parents: 981
diff changeset
    36
dist
21560
a9e1ab2da30d packaging: move output directory from build/ to packages/
Matt Mackall <mpm@selenic.com>
parents: 19991
diff changeset
    37
packages
19426
e43184680461 doc: make man and html from translated documents
Takumi IINO <trot.thunder@gmail.com>
parents: 17732
diff changeset
    38
doc/common.txt
1270
fc3b41570082 Switch to new syntax for .hgignore files.
Bryan O'Sullivan <bos@serpentine.com>
parents: 981
diff changeset
    39
doc/*.[0-9]
19426
e43184680461 doc: make man and html from translated documents
Takumi IINO <trot.thunder@gmail.com>
parents: 17732
diff changeset
    40
doc/*.[0-9].txt
1817
e02fea13c69d Ignore generated documentation
Thomas Arendsen Hein <thomas@intevation.de>
parents: 1270
diff changeset
    41
doc/*.[0-9].gendoc.txt
1270
fc3b41570082 Switch to new syntax for .hgignore files.
Bryan O'Sullivan <bos@serpentine.com>
parents: 981
diff changeset
    42
doc/*.[0-9].{x,ht}ml
fc3b41570082 Switch to new syntax for .hgignore files.
Bryan O'Sullivan <bos@serpentine.com>
parents: 981
diff changeset
    43
MANIFEST
14560
0980239cb20c hgignore: ignore MANIFEST.in (generated since 2ce7dfe17bc5)
Thomas Arendsen Hein <thomas@intevation.de>
parents: 14538
diff changeset
    44
MANIFEST.in
1270
fc3b41570082 Switch to new syntax for .hgignore files.
Bryan O'Sullivan <bos@serpentine.com>
parents: 981
diff changeset
    45
patches
28430
17b85d739b62 setup: create a module for the modulepolicy
timeless <timeless@mozdev.org>
parents: 28255
diff changeset
    46
mercurial/__modulepolicy__.py
1270
fc3b41570082 Switch to new syntax for .hgignore files.
Bryan O'Sullivan <bos@serpentine.com>
parents: 981
diff changeset
    47
mercurial/__version__.py
17732
93d97a212559 exewrapper: adapt for legacy HackableMercurial
Adrian Buehlmann <adrian@cadifra.com>
parents: 17274
diff changeset
    48
mercurial/hgpythonlib.h
11380
2bb6dbf04757 Ignore mercurial.egg-info build output
Lee Cantey <lcantey@gmail.com>
parents: 8147
diff changeset
    49
mercurial.egg-info
2987
12d1475b48df Ignore .DS_Store directories
Lee Cantey <lcantey@gmail.com>
parents: 2386
diff changeset
    50
.DS_Store
5693
5d0b94d3ad0c .hgignore += tags & cscope files
Kirill Smelkov <kirr@mns.spb.ru>
parents: 5622
diff changeset
    51
tags
5d0b94d3ad0c .hgignore += tags & cscope files
Kirill Smelkov <kirr@mns.spb.ru>
parents: 5622
diff changeset
    52
cscope.*
22945
3232f92360d4 hgignore: ignore the PyCharm workspace folder
Angel Ezquerra <angel.ezquerra@gmail.com>
parents: 21874
diff changeset
    53
.idea/*
30406
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents: 29031
diff changeset
    54
.asv/*
7648
02e358a3a8a7 i18n: let Makefile generate i18n/hg.pot
Martin Geisler <mg@daimi.au.dk>
parents: 7439
diff changeset
    55
i18n/hg.pot
7649
a489e3a94443 i18n: new build_mo command for setup.py
Martin Geisler <mg@daimi.au.dk>
parents: 7648
diff changeset
    56
locale/*/LC_MESSAGES/hg.mo
14538
3818c67a501e setup: add command to generate index of extensions
Yuya Nishihara <yuya@tcha.org>
parents: 13643
diff changeset
    57
hgext/__index__.py
1270
fc3b41570082 Switch to new syntax for .hgignore files.
Bryan O'Sullivan <bos@serpentine.com>
parents: 981
diff changeset
    58
35569
964212780daf rust: implementation of `hg`
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35190
diff changeset
    59
rust/target/
964212780daf rust: implementation of `hg`
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35190
diff changeset
    60
32080
fcddcf442e4f packaging: add make target for linux wheels
Boris Feld <boris.feld@octobus.net>
parents: 30406
diff changeset
    61
# Generated wheels
fcddcf442e4f packaging: add make target for linux wheels
Boris Feld <boris.feld@octobus.net>
parents: 30406
diff changeset
    62
wheelhouse/
fcddcf442e4f packaging: add make target for linux wheels
Boris Feld <boris.feld@octobus.net>
parents: 30406
diff changeset
    63
1270
fc3b41570082 Switch to new syntax for .hgignore files.
Bryan O'Sullivan <bos@serpentine.com>
parents: 981
diff changeset
    64
syntax: regexp
794
cdf61d3d3904 Ignore test error files, protect dot in .pc/
Thomas Arendsen Hein <thomas@intevation.de>
parents: 740
diff changeset
    65
^\.pc/
7439
7dd44ad43914 Ignore eclipse droppings
Brendan Cully <brendan@kublai.com>
parents: 6551
diff changeset
    66
^\.(pydev)?project
15702
64a80204433f ignore: add files present in the hackable-hg windows distribution
Laurens Holst <laurens.hg@grauw.nl>
parents: 14560
diff changeset
    67
64a80204433f ignore: add files present in the hackable-hg windows distribution
Laurens Holst <laurens.hg@grauw.nl>
parents: 14560
diff changeset
    68
# hackable windows distribution additions
16537
5068d0974278 update .hgignore for hackable with Python 2.7
Mads Kiilerich <mads@kiilerich.com>
parents: 15859
diff changeset
    69
^hg-python
15702
64a80204433f ignore: add files present in the hackable-hg windows distribution
Laurens Holst <laurens.hg@grauw.nl>
parents: 14560
diff changeset
    70
^hg.py$