Pierre-Yves David <pierre-yves.david@fb.com> [Wed, 23 Sep 2015 11:44:52 -0700] rev 26393
bundle2: rename error exception class for unsupported feature
The original name explicitly mention "Part", however it is also used outside of
parts related feature. We rename from 'UnsupportedPartError' to
'BundleUnknownFeatureError' to fix this.
Pierre-Yves David <pierre-yves.david@fb.com> [Wed, 23 Sep 2015 11:33:30 -0700] rev 26392
changegroup: use a different compression key for BZ in HG10
For "space saving", bundle1 "strip" the first two bytes of the BZ stream since
they always are 'BZ'. So the current code boostrap the uncompressor with 'BZ'.
This hack is impractical in more generic case so we move it in a dedicated
"decompression".
Anton Shestakov <av6@dwimlabs.net> [Sat, 26 Sep 2015 17:24:12 +0800] rev 26391
monoblue: provide links to branches, tags and bookmarks by name
This is adapted from cd842821db2c, that was added to paper for 3.5 release.
It adds another way to refer to branches, tags and bookmarks in urls: by name.
It's still possible to navigate to a specific changeset hash, but now you can
get more descriptive urls straight from /summary page, for example.
branchentry template (and so the whole branches table on /summary and
/branches) lost the column that had a plain changeset hash, because tags and
bookmarks don't have this column and also because there is already a way to
address branch by its changeset hash (changeset link just next to it). Maybe we
can instead bring this column with a plain changeset hash to tags and
bookmarks, but this, more terse, new look feels fine.
Anton Shestakov <av6@dwimlabs.net> [Sat, 26 Sep 2015 17:15:58 +0800] rev 26390
gitweb: provide links to branches, tags and bookmarks by name
This is adapted from cd842821db2c, that was added to paper for 3.5 release.
It adds another way to refer to branches, tags and bookmarks in urls: by name.
It's still possible to navigate to a specific changeset hash, but now you can
get more descriptive urls straight from /summary page, for example.
branchentry template (and so the whole branches table on /summary and
/branches) lost the column that had a plain changeset hash, because tags and
bookmarks don't have this column and also because there is already a way to
address branch by its changeset hash (changeset link just next to it). Maybe we
can instead bring this column with a plain changeset hash to tags and
bookmarks, but this, more terse, new look feels fine.
timeless@mozdev.org [Fri, 25 Sep 2015 03:44:15 -0400] rev 26389
cmdutil: remove HG: prefix from translation strings
timeless@mozdev.org [Fri, 25 Sep 2015 03:47:48 -0400] rev 26388
i18n-zh_CN: annotate broken HG: translations
Siddharth Agarwal <sid0@fb.com> [Thu, 24 Sep 2015 22:07:55 -0700] rev 26387
lock: recognize parent locks while acquiring
This is part of a series that will allow locks to be inherited by subprocesses
in limited circumstances. This patch enables the logic introduced in previous
patches.
Siddharth Agarwal <sid0@fb.com> [Thu, 24 Sep 2015 22:00:51 -0700] rev 26386
test-lock.py: fix testing for forks
The earlier test worked only because the held count went up to 2, so the first
release brought it down to 1. Making a copy of the lock fixes that issue.
Siddharth Agarwal <sid0@fb.com> [Thu, 24 Sep 2015 20:40:00 -0700] rev 26385
test-lock.py: allow PID to be changed in test state
This will be used in upcoming patches to create locks that appear as if they're
being created by child processes.
Siddharth Agarwal <sid0@fb.com> [Thu, 24 Sep 2015 20:22:59 -0700] rev 26384
test-lock.py: add a lock wrapper that allows faking the PID
This will be used in upcoming patches to create locks that appear as if they're
being created by child processes.
Siddharth Agarwal <sid0@fb.com> [Thu, 24 Sep 2015 21:26:37 -0700] rev 26383
lock: add a wrapper to os.getpid() to make testing easier
This will allow us to fake locks across processes more easily.
Siddharth Agarwal <sid0@fb.com> [Thu, 24 Sep 2015 19:52:34 -0700] rev 26382
test-lock.py: move temp dir generation to testcase
In upcoming patches we'll want to create multiple test state objects with a
common test directory.
Siddharth Agarwal <sid0@fb.com> [Thu, 24 Sep 2015 17:33:13 -0700] rev 26381
test-lock.py: copy-edit assertions about file existing
Before: expected lock to exists but actually did not exists
After: expected lock to exist but actually did not exist
Gregory Szorc <gregory.szorc@gmail.com> [Sat, 26 Sep 2015 21:43:13 -0700] rev 26380
revlog: don't flush data file after every added revision
The current behavior of revlogs is to flush the data file when writing
data to it. Tracing system calls revealed that changegroup processing
incurred numerous write(2) calls for values much smaller than the
default buffer size (Python defaults to 4096, but it can be adjusted
based on detected block size at run time by CPython).
The reason we flush revlogs is so readers have all data available.
For example, the current code in revlog.py will re-open the revlog
file (instead of seeking an existing file handle) to read the text
of a revision. This happens when starting a new delta chain when
adding several revisions from changegroups, for example. Yes, this
is likely sub-optimal (we should probably be sharing file descriptors
between readers and writers to avoid the flushing and associated
overhead of re-opening files).
While flushing revlogs is necessary, it appears all callers are
diligent about flushing files before a read is performed (see
buildtext() in _addrevision()), making the flush in
_writeentry() redundant and unncessary. So, we remove it. In practice,
this means we incur a write(2) a) when the buffer is full (typically
4096 bytes) b) when a new delta chain is created rather than after
every added revision. This applies to every revlog, but by volume
it mostly impacts filelogs.
Removing the redundant flush from _writeentry() significantly
reduces the number of write(2) calls during changegroup processing on
my Linux machine. When applying a changegroup of the hg repo based on
my local repo, the total number of write(2) calls during application
of the mercurial/localrepo.py revlogs dropped from 1,320 to 217 with
this patch applied. Total I/O related system calls dropped from 1,577
to 474.
When unbundling a mozilla-central gzipped bundle (264,403 changesets
with 1,492,215 changes to 222,507 files), total write(2) calls
dropped from 1,252,881 to 827,106 and total system calls dropped from
3,601,259 to 3,178,636 - a reduction of 425,775!
While the system call reduction is significant, it appears
to have no impact on wall time on my Linux and Windows machines. Still,
fewer syscalls is fewer syscalls. Surely this can't hurt. If nothing
else, it makes examining remaining system call usage simpler and opens
the door to experimenting with the performance impact of different
buffer sizes.
Gregory Szorc <gregory.szorc@gmail.com> [Sun, 27 Sep 2015 16:08:18 -0700] rev 26379
revlog: use existing file handle when reading during _addrevision
_addrevision() may need to read from revlogs as part of computing
deltas. Previously, we would flush existing file handles and open
a new, short-lived file handle to perform the reading.
If we have an existing file handle, it seems logical to reuse it
for reading instead of opening a new file handle. This patch
makes that the new behavior.
After this patch, revlog files are only reopened when adding
revisions if the revlog is switched from inline to non-inline.
On Linux when unbundling a bundle of the mozilla-central repo, this
patch has the following impact on system call counts:
Call Before After Delta
write 827,639 673,390 -154,249
open 700,103 684,089 -16,014
read 74,489 74,489 0
fstat 493,924 461,896 -32,028
close 249,131 233,117 -16,014
stat 242,001 242,001 0
lstat 18,676 18,676 0
lseek 20,268 20,268 0
ioctl 14,652 13,173 -1,479
TOTAL 3,180,758 2,930,679 -250,079
It's worth noting that many of the open() calls fail due to missing
files. That's why there are many more open() calls than close().
Despite the significant system call reduction, this change does not
seem to have a significant performance impact on Linux.
On Windows 10 (not a VM, on a SSD), this patch appears to reduce
unbundle time for mozilla-central from ~960s to ~920s. This isn't
as significant as I was hoping. But a decrease it is nonetheless.
Still, Windows unbundle performance is still >2x slower than Linux.
Despite the lack of significant gains, fewer system calls is fewer
system calls. If nothing else, this will narrow the focus of potential
areas to optimize in the future.
Gregory Szorc <gregory.szorc@gmail.com> [Sun, 27 Sep 2015 15:59:19 -0700] rev 26378
revlog: always open revlogs for reading and appending
An upcoming patch will teach revlogs to use the existing file
handle to read revision data instead of opening a new file handle
just for quick reads. For this to work, files must be opened for
reading as well.
This patch is merely cosmetic: there are no behavior changes.