Augie Fackler <augie@google.com> [Mon, 03 Dec 2018 18:07:09 -0500] rev 40810
fuzz: grep away HAVE_GETC_UNLOCKED in pyconfig.h to avoid msan badness
Per discussion with Greg Smith and the patches on
https://bugs.python.org/
issue35214. This, combined with the previous
patch, fixes msan builds on oss-fuzz.
Differential Revision: https://phab.mercurial-scm.org/D5363
Augie Fackler <augie@google.com> [Tue, 13 Nov 2018 09:19:05 -0500] rev 40809
fuzz: more correctly specify CFLAGS and LDFLAGS when building Python
Gets us closer to a working msan build alongside our asan build.
Differential Revision: https://phab.mercurial-scm.org/D5362
Matt Harbison <matt_harbison@yahoo.com> [Tue, 04 Dec 2018 00:19:33 -0500] rev 40808
tests: stabilize test-blackbox.t on Windows
I didn't look into why the error is more detailed, but that seems like it's a
good thing (other than for recording tests).
Matt Harbison <matt_harbison@yahoo.com> [Tue, 04 Dec 2018 00:16:12 -0500] rev 40807
tests: stabilize for recent wcache changes
This goes with
47e3f554df35::
d5622dfe4ba3. I'm not sure if it was really
expected that there would be no wcache directory if neither execbit nor symlink
is supported.
Matt Harbison <matt_harbison@yahoo.com> [Mon, 03 Dec 2018 12:48:42 -0500] rev 40806
extdiff: avoid double backslashes in the displayed tool path on Windows
This shows the tool path in the help, and changed in
67b180c0e263. uirepr()
already does the same thing, but that undoes the mangling in its call to repr().
Boris Feld <boris.feld@octobus.net> [Wed, 28 Nov 2018 05:06:58 +0100] rev 40805
contrib: add a helper script that help to build interesting repositories
The script is dedicated to building a couple of repositories that should be
interesting to run discovery from one another. It seems a common enough need
to contribute it upstream.
Pulkit Goyal <pulkit@yandex-team.ru> [Mon, 03 Dec 2018 19:42:46 +0300] rev 40804
py3: listify filter() to call len() on it
Differential Revision: https://phab.mercurial-scm.org/D5354
Martin von Zweigbergk <martinvonz@google.com> [Mon, 03 Dec 2018 11:14:44 -0800] rev 40803
rebase: fix dir/file conflict detection when using in-mem merge
Differential Revision: https://phab.mercurial-scm.org/D5360
Martin von Zweigbergk <martinvonz@google.com> [Mon, 03 Dec 2018 11:11:34 -0800] rev 40802
tests: show that in-mem rebase does not find path dir/file conflicts
Differential Revision: https://phab.mercurial-scm.org/D5359
Matt Harbison <matt_harbison@yahoo.com> [Mon, 03 Dec 2018 20:59:48 -0500] rev 40801
extdiff: register the configuration generated commands with a help category
Otherwise, 'extdiff' shows up under file management and the rest of the commands
are at the bottom under 'Uncategorized'.
Yuya Nishihara <yuya@tcha.org> [Sun, 18 Nov 2018 18:35:31 +0900] rev 40800
loggingutil: document openlogfile()
This function will be used later for command-server logging.
Yuya Nishihara <yuya@tcha.org> [Sun, 18 Nov 2018 18:25:37 +0900] rev 40799
loggingutil: extract openlogfile() and proxylogger to new module
This module isn't placed under the "utils" package since it needs "ui" to
process things. It's called "loggingutil", not "logutil" because the word
"log" is too obscure in our codebase.
Yuya Nishihara <yuya@tcha.org> [Sun, 18 Nov 2018 18:21:39 +0900] rev 40798
blackbox: pass in options to _openlogfile() as arguments
This prepares for extracting utility function from the blackbox module.
Yuya Nishihara <yuya@tcha.org> [Sat, 17 Nov 2018 22:10:27 +0900] rev 40797
blackbox: just try writing to repo.vfs and update lastlogger on success
This is simpler and more robust. Before, an empty ".hg" directory would be
created if it's removed after checking vfs.isdir('.').
Yuya Nishihara <yuya@tcha.org> [Tue, 20 Nov 2018 22:31:12 +0900] rev 40796
vfs: add option to not create parent directories implicitly
In blackbox, we don't want to create a ".hg" directory by mistake. This
provides a race-safe option to achieve that.
Boris Feld <boris.feld@octobus.net> [Thu, 15 Nov 2018 02:55:33 +0100] rev 40795
repo: add a `wcachevfs` to access the `.hg/wcache/` directory
This wvfs will allow us to migrate various cache to the new `wcache` directory.
Helping with cache issues with "share".
Boris Feld <boris.feld@octobus.net> [Thu, 15 Nov 2018 02:46:31 +0100] rev 40794
cache: create `wcache` directory at init time
The cache directory will be needed very quickly, so it seems simpler to create
it early to make sure it has the same owner and permission than the other
directory in the repository.
Boris Feld <boris.feld@octobus.net> [Thu, 15 Nov 2018 02:38:55 +0100] rev 40793
cache: create `cache` directory at init time
The cache directory will be needed very quickly, so it seems simpler to create
it early to make sure it has the same owner and permission than the other
directory in the repository.
Boris Feld <boris.feld@octobus.net> [Thu, 15 Nov 2018 17:08:23 +0100] rev 40792
check-exec: write file in 'wcache' instead of 'cache'
Some cache are relevant or affected by the working copy used. So the `.hg/cache`
directory is not the best place for them because multiple shared repository can
end up fighting over them.
To address this issue, we introduce a new 'wcache' directory to host this kind
of cache.
The first user are the `checkisexec` type file. These files describe property of
the working copy and fit the use-case well.
Martin von Zweigbergk <martinvonz@google.com> [Mon, 03 Dec 2018 09:36:40 -0800] rev 40791
rebase: abort in-mem rebase if there's a dirty merge state
In-memory merge uses the on-disk merge state, so we should not allow
it run in-memory merge when the merge state is not clean. We should
probably not use the on-disk merge state when running in-memory merge,
but chaning that is not suitable for the stable branch.
Differential Revision: https://phab.mercurial-scm.org/D5357
Martin von Zweigbergk <martinvonz@google.com> [Fri, 30 Nov 2018 16:21:37 -0800] rev 40790
rebase: preserve working copy when redoing in-mem rebase on disk
When in-memory rebase runs into conflicts, we retry it on disk. But
before we do that, we abort the in-memory rebase. That is done because
even though it's mostly in memory, there are still a few state files
written (e.g. the merge state). We should make it not write those
files so we don't need to abort, but for the stable branch, let's
explicitly clear the state we need to clear instead of running the
usual abort code.
Differential Revision: https://phab.mercurial-scm.org/D5356
Martin von Zweigbergk <martinvonz@google.com> [Fri, 30 Nov 2018 15:08:43 -0800] rev 40789
tests: show that in-mem rebase falling back loses state
Both working copy changes and the merge state is lost when in-memory
rebase falls back to on-disk mode.
Differential Revision: https://phab.mercurial-scm.org/D5355
Yuya Nishihara <yuya@tcha.org> [Mon, 03 Dec 2018 21:45:15 +0900] rev 40788
commandserver: get around ETIMEDOUT raised by selectors2
selector.select() should exits with an empty event list on timed out, but
selectors2 raises OSError if timeout expires while recovering from EINTR.
Spotted while debugging new chg feature.
Yuya Nishihara <yuya@tcha.org> [Mon, 03 Dec 2018 21:31:19 +0900] rev 40787
selectors2: backport minimal fix of timeout handling from 2.0.1
The original code would raise TypeError since OSError() doesn't support
keyword arguments.
We can't simply import the selectors 2.0.1, which still spawns "uname -p"
through platform.system(). We could switch to the unreleased version, but
I decided to not right now to minimize the change.
Boris Feld <boris.feld@octobus.net> [Fri, 23 Nov 2018 06:09:44 +0100] rev 40786
mmapindex: set default to 1MB
mmapping index is more efficient if we only need a small part of it.
The 1MB value has been picked arbitrarily, a lower value might be better.
On a large repository with a 60MB index, we see the following performance
gain:
hg perfindex
before: ! wall 0.032023 comb 0.040000 user 0.000000 sys 0.040000 (best of 100)
after: ! wall 0.000196 comb 0.000000 user 0.000000 sys 0.000000 (best of 1060)
The speed boost benefit all cases, including the one where the full index
needs to be parsed.
hg perfindex --rev 0
before: ! wall 0.040673 comb 0.030000 user 0.000000 sys 0.030000 (best of 100)
after ! wall 0.010713 comb 0.020000 user 0.010000 sys 0.010000 (best of 212)
This gain reflect in higher level operation:
hg perfbookmarks --clear-revlogs
before: ! wall 0.161339 comb 0.160000 user 0.130000 sys 0.030000 (best of 56)
after: ! wall 0.123228 comb 0.120000 user 0.120000 sys 0.000000 (best of 68)
Boris Feld <boris.feld@octobus.net> [Fri, 23 Nov 2018 06:07:33 +0100] rev 40785
mmapindex: move the 'mmapindexthreshold' option out of experimental
The option is useful and should be advertised more. We move it out of
experimental as a first step. The `storage` section is selected as this is
related to how the storage is accessed. A new 'performance' section might be
more appropriate.
We move from 'mmapindexthreshold` to `mmap-threshold` as non-index item are
also suitable for mmap (eg: the rev-branch-cache).
If relevant, we can introduce sub-option `mmap-threshold.revlog-index` later.
Boris Feld <boris.feld@octobus.net> [Sat, 01 Dec 2018 15:57:27 +0100] rev 40784
perf: add a --rev attribute to perfindex
This allow for benchmarking the time necessary to look for other version than
the tip.
Boris Feld <boris.feld@octobus.net> [Fri, 23 Nov 2018 06:03:38 +0100] rev 40783
perf: update perfindex to be more realistic
The previous code was creating a revlog manually, we now use the actual
`localrepo` method to create it.
We have to jump though extra hops to work around the impact of filecache.
Martin von Zweigbergk <martinvonz@google.com> [Sun, 02 Dec 2018 13:09:46 -0800] rev 40782
match: drop unnecessary wrapping of regex in group
It seems the regexes have been wrapped in an unnamed group since
b6c42714d900 (Add locate command., 2005-07-05). In that commit, the
grouping was needed because there was a "head" ('^') added before the
group and a "tail" (os.sep) added after it. It seems the head was
moved inside the group in
1c0c413cccdd (Get add and locate to use new
repo and dirstate walk code., 2005-07-18) and the tail was moved
inside the group in
89985a1b3427 (Clean up walk and changes code to
use normalised names properly., 2005-07-31), So it seems to me that
we've carried around the unnecessary group for 13 years. This patch
removes it.
Differential Revision: https://phab.mercurial-scm.org/D5352
Martin von Zweigbergk <martinvonz@google.com> [Sun, 02 Dec 2018 13:45:20 -0800] rev 40781
match: use _BASE_SIZE instead of magic value 4
Differential Revision: https://phab.mercurial-scm.org/D5351
Martin von Zweigbergk <martinvonz@google.com> [Sun, 02 Dec 2018 13:44:49 -0800] rev 40780
match: make "groupsize" include the trailing "|"
I think this is a little easier to follow and it will simplify later
patches too.
Differential Revision: https://phab.mercurial-scm.org/D5350
Martin von Zweigbergk <martinvonz@google.com> [Sun, 02 Dec 2018 13:09:43 -0800] rev 40779
match: fix an unaligned (but harmless) indent
Differential Revision: https://phab.mercurial-scm.org/D5349
Boris Feld <boris.feld@octobus.net> [Thu, 22 Nov 2018 17:41:10 +0100] rev 40778
match: raise an Abort error instead of OverflowError
This case of OverflowError (one single pattern being too large) has never been
properly caught in the past.
Boris Feld <boris.feld@octobus.net> [Thu, 22 Nov 2018 21:02:02 +0100] rev 40777
match: avoid translating glob to matcher multiple times for large sets
For hgignore with many globs, the resulting regexp might not fit under the 20K
length limit. So the patterns need to be broken up in smaller pieces.
Before this change, the logic was re-starting the full process from scratch
for each smaller pieces, including the translation of globs into regexp.
Effectively doing the work over and over.
If the 20K limit is reached, we are likely in a case where there is many such
glob, so exporting them is especially expensive and we should be careful not
to do that work more than once.
To work around this, we now translate glob to regexp once and for all. Then,
we assemble the resulting individual regexp into valid blocks.
This raises a very significant performance win for large `.hgignore file`:
Before: ! wall 0.153153 comb 0.150000 user 0.150000 sys 0.000000 (median of 66)
After: ! wall 0.059793 comb 0.060000 user 0.060000 sys 0.000000 (median of 100)
Boris Feld <boris.feld@octobus.net> [Thu, 22 Nov 2018 17:25:49 +0100] rev 40776
match: extract function that group regexps
Boris Feld <boris.feld@octobus.net> [Thu, 22 Nov 2018 17:16:05 +0100] rev 40775
match: test for overflow error in pattern
If a single pattern is too large to handle, we raise an exception. This case is
now doctested.
Boris Feld <boris.feld@octobus.net> [Thu, 22 Nov 2018 17:20:32 +0100] rev 40774
match: extract a literal constant into a symbolic one
Matt Harbison <matt_harbison@yahoo.com> [Sat, 01 Dec 2018 21:42:48 -0500] rev 40773
tests: apply binary mode to output in seq.py
I noticed this when playing with running tests using WSL, and iterating over the
output yielded '0\r', '1\r',... Most of the other *.py tools do this, and `seq`
on MSYS lacks '\r' in the output, so this is more consistent.
Boris Feld <boris.feld@octobus.net> [Fri, 23 Nov 2018 01:09:37 +0100] rev 40772
perf: add a `--clear-caches` to `perfbranchmapupdate`
This flag will help to measure the time we spend loading various cache that
support the branchmap update.
Example for an 500 000 revisions repository:
hg perfbranchmapupdate --base 'not tip' --target 'tip'
! wall 0.000860 comb 0.000000 user 0.000000 sys 0.000000 (best of 336)
hg perfbranchmapupdate --base 'not tip' --target 'tip' --clear-caches
! wall 0.029494 comb 0.030000 user 0.030000 sys 0.000000 (best of 100)
Boris Feld <boris.feld@octobus.net> [Wed, 21 Nov 2018 21:11:47 +0000] rev 40771
perf: start from an existing branchmap if possible
If the --base set if a superset of one of the cached branchmap, we should use as
a starting point. This greatly help the overall runtime of
`hg perfbranchmapupdate`
For example, for a repository with about 500 000 revisions, using this trick
make the command runtime move from about 200 second to about 10 seconds. A 20x
gain.
Boris Feld <boris.feld@octobus.net> [Wed, 21 Nov 2018 20:35:22 +0000] rev 40770
perf: rely on repoview for perfbranchmapupdate
Using 'repoview' matching the base and target subset make the benchmark more
realistic. It also unlocks optimization to make the command initialization
faster.
Boris Feld <boris.feld@octobus.net> [Wed, 21 Nov 2018 22:56:06 +0100] rev 40769
perf: pre-indent some code in `perfbranchmapupdate`
This make the next patch easier to read.
Boris Feld <boris.feld@octobus.net> [Wed, 21 Nov 2018 12:02:25 +0000] rev 40768
perf: add a `perfbranchmapupdate` command
This command benchmark the time necessary to update the branchmap between two
sets of revisions. This changeset introduce a first version, doing nothing fancy
regarding cache or other internal details.
Anton Shestakov <av6@dwimlabs.net> [Mon, 05 Nov 2018 13:52:19 +0800] rev 40767
push: config option to control behavior when pushing to a publishing server
Pushing to a publishing server by mistake can lead to a difficult situation to
solve because evolution doesn't work on public changesets. This new
experimental config tries to help avoiding unintentionally (or at least being
aware of) pushing to publishing remotes.
`hg push --publish` can be used to make push succeed even when auto-publish is
set to 'abort'.
Pulkit Goyal <pulkit@yandex-team.ru> [Fri, 30 Nov 2018 17:42:55 +0300] rev 40766
narrowcommands: remove an unrequired `repo.narrowpats` call
We call that few lines above and do nothing significant in between which can
change the narrowpats. So let's use values returned by that call.
Differential Revision: https://phab.mercurial-scm.org/D5348
Augie Fackler <augie@google.com> [Thu, 29 Nov 2018 16:44:01 -0500] rev 40765
manifest: reject lines shorter than 42 bytes, not 22
Yuya correctly spotted during the review of
f27f8e9ef1e73 that we're
dealing with hexlified hashes here, and so it should be 42 bytes not
22.
Differential Revision: https://phab.mercurial-scm.org/D5347
Yuya Nishihara <yuya@tcha.org> [Sun, 11 Nov 2018 20:05:38 +0900] rev 40764
blackbox: initialize logger with repo instance
The blackboxlogger is unusable without a repo. Let's simply initialize it
with a repo instance.
Yuya Nishihara <yuya@tcha.org> [Sat, 17 Nov 2018 20:56:25 +0900] rev 40763
blackbox: do not nullify repo to deactivate the logger on failure
The _repo will be a mandatory attribute. Instead, make the logger to not
track any events.
Yuya Nishihara <yuya@tcha.org> [Sun, 11 Nov 2018 20:02:34 +0900] rev 40762
blackbox: extract global last logger to proxylogger class
So the blackboxlogger can be instantiated with a repo.
Yuya Nishihara <yuya@tcha.org> [Sun, 11 Nov 2018 19:36:21 +0900] rev 40761
ui: pass in bytes opts dict to logger.log()
This is the convention of the Mercurial API.
Yuya Nishihara <yuya@tcha.org> [Sun, 11 Nov 2018 19:35:33 +0900] rev 40760
ui: pass in formatted message to logger.log()
This makes sure that all logger instances will handle the message arguments
properly.
Yuya Nishihara <yuya@tcha.org> [Sun, 11 Nov 2018 17:34:46 +0900] rev 40759
blackbox: send debug message to logger by core ui
Since the core ui.log() may recurse into ui.log() through ui.debug(), it
must guard against recursion.
The ui extension class can finally be removed.
Yuya Nishihara <yuya@tcha.org> [Sat, 17 Nov 2018 20:23:50 +0900] rev 40758
blackbox: change the way of deactivating the logger on write error
This prepares for the upcoming code move. The recursion guard will be ported
to the core ui.
Martin von Zweigbergk <martinvonz@google.com> [Wed, 28 Nov 2018 10:12:50 -0800] rev 40757
match: remove obsolete catching of OverflowError
Since
0f6a1bdf89fb (match: handle large regexes, 2007-08-19), we catch
an OverflowError from the regex engine and split up the regex if that
happens. In
59a9dc9562e2 (ignore: split up huge patterns, 2008-02-11),
that was extended to raise an OverflowError in our code even if the
regex engine doesn't raise it. It's unclear if there was a range of
regex sizes where the OverflowError would be raised from the regex
engine but that were still below the limit we added in our
code. Either way, both limitations were probably removed in Python
2.7.4 when the regex code width was extended from 16bit to 32bit (or
Py_UCS4) integer (thanks to Yuya for finding that out).
If at least the first limitation was removed, we no longer should be
using OverflowError for flow control, so this patch changes that.
Differential Revision: https://phab.mercurial-scm.org/D5309
Boris Feld <boris.feld@octobus.net> [Tue, 27 Nov 2018 02:10:14 +0100] rev 40756
sparse: raise a move verbose index error from the C code
If we don't like a value we should print it.
Pulkit Goyal <pulkit@yandex-team.ru> [Fri, 05 Oct 2018 23:10:56 +0300] rev 40755
narrow: drop the bundle2 capability since we have server capabilities (BC)
This patch drops the narrow bundle2 capabilities since we introduced narrow
server capabilities which are more nice and now used everywhere.
I am not sure what it can affect, so on safe side I marked this as BC. Also I
removed the NARROWCAP constant as that kind of conflicts with the same name
constant in wireprototypes.py.
Differential Revision: https://phab.mercurial-scm.org/D4892
Boris Feld <boris.feld@octobus.net> [Sun, 02 Jul 2017 04:06:24 +0200] rev 40754
vfs: extract the audit path logic into a submethod
This will make it possible to apply it in more cases.
Boris Feld <boris.feld@octobus.net> [Thu, 22 Nov 2018 20:01:28 +0100] rev 40753
subrepo-git: use an official origvfs when appropriate
The origvfs has the auditor properly set and can move file without issue.
The current code is currently working without errors because rename are not
audited, yet.
Boris Feld <boris.feld@octobus.net> [Thu, 22 Nov 2018 19:26:05 +0100] rev 40752
revert: extract origvfs logic in a sub-function
The subrepo's "revert" logic could benefit from it.
Boris Feld <boris.feld@octobus.net> [Thu, 22 Nov 2018 18:44:07 +0100] rev 40751
vfs: treat 'undo.' file the same as 'journal.' file
They are the same kind of file, they are protected by the store lock, but
directly lives inside the '.hg' directory.
No warnings were ever raised about them because `vfs.rename` is not audited.
Something we are trying to change.
Boris Feld <boris.feld@octobus.net> [Thu, 22 Nov 2018 21:00:13 +0100] rev 40750
perf: add a perfignore command
The command is meant to benchmark operations related to hgignore. Right now the
command is benchmarking the loading time of the hgignore rules.
Pulkit Goyal <pulkit@yandex-team.ru> [Mon, 26 Nov 2018 15:36:06 +0300] rev 40749
py3: use pycompat.xrange instead of xrange
xrange does not exists on Python 3.
Differential Revision: https://phab.mercurial-scm.org/D5302
Pulkit Goyal <pulkit@yandex-team.ru> [Tue, 27 Nov 2018 16:16:13 +0300] rev 40748
store: write fncache only once if there are both adds and removes
Differential Revision: https://phab.mercurial-scm.org/D5307
Boris Feld <boris.feld@octobus.net> [Tue, 20 Nov 2018 17:44:24 +0000] rev 40747
perf: disable revlogs clearing in `perftags` by default
This aligns things with what `perfbookmarks` does. I decided to disable the
revlogs clearing by default to focus on the core logic by default, ignoring
side effects.
If we prefer to emphasize the side effect, we can instead keep this on in
`perftags` and enable it by default in `perfbookmarks`.