Matt Harbison <matt_harbison@yahoo.com> [Sun, 09 Dec 2018 18:35:35 -0500] rev 40889
py3: adapt test-largefiles-wireproto.t to python3
Matt Harbison <matt_harbison@yahoo.com> [Sun, 09 Dec 2018 13:53:08 -0500] rev 40888
py3: byteify test-push-race.t
Matt Harbison <matt_harbison@yahoo.com> [Sun, 09 Dec 2018 13:33:36 -0500] rev 40887
py3: convert filename to bytes in test-hardlinks.t
Windows was complaining about passing str to CreateFileA() in win32.py.
Martin von Zweigbergk <martinvonz@google.com> [Sat, 08 Dec 2018 23:41:54 -0800] rev 40886
narrow: keep bookmarks temporarily stripped for as long as commits are
The narrow extension also has support for shallowness and for
inserting older commits on pull. It works by temporarily stripping
newer commits, adding the older commits, then re-applying the stripped
bundle. The regular Mercurial server uses that when you widen,
although it shouldn't be necessary there. Our Google-internal server
does it when the user requests an older commit.
Our Google-internal tests fail since 7caf632e30c3 (filecache:
unimplement __set__() and __delete__() (API), 2018-10-20). I haven't
quite understood the problem, but it's related to the way we
temporarily hide bookmarks while the commits they point to are
stripped. When a transaction is started, Mercurial tries to read
various things from the repo for the transaction summary. That leads
to computation of hidden commits, which leads to an attempt to find
commits pinned by bookmarks. This is the reason we temporarily hide
the bookmarks. I think the aforementioned commit makes the restored
bookmarks visible earlier than before (which seems like an
improvement), so we can no longer incorrectly rely on the
repo._bookmarks field being cached too long (IIUC).
This patch makes it so we restore the temporarily hidden bookmarks
only after the temporary bundle has been re-applied. It also adapts
the code to update the repo.__bookmarks field using the pattern
described in the aforementioned commit instead of writing directly to
the fiels.
I have spent many hours trying to understand what was going on here,
but I still don't know if this can also happen without our custom
server. So this patch unfortunately does not add any tests; I have
only been able to test the fix using our Google-internal tests.
Differential Revision: https://phab.mercurial-scm.org/D5398
Martin von Zweigbergk <martinvonz@google.com> [Sat, 08 Dec 2018 23:04:11 -0800] rev 40885
narrow: drop obsolete support for old Mercurial
Not needed since narrow is shipped with core.
Differential Revision: https://phab.mercurial-scm.org/D5397
Martin von Zweigbergk <martinvonz@google.com> [Thu, 06 Dec 2018 10:15:41 -0800] rev 40884
tests: test that narrow preserves bookmarks on widen
When widening inserts older commits in the changelog, we have to
preserve bookmarks so they are not removed by the call to
repair.strip() we temporarily do. We didn't have any tests for that
before.
Differential Revision: https://phab.mercurial-scm.org/D5396
Martin von Zweigbergk <martinvonz@google.com> [Fri, 07 Dec 2018 14:23:17 -0800] rev 40883
rebase: clarify that commits that become empty are skipped
Our message just said that it created no changes, but we didn't
explicitly say that that meant that the the commit was not rebased. It
also wasn't clear why it created no changes, so this patch also
clarifies that that was because the changes were already in the
destination.
Differential Revision: https://phab.mercurial-scm.org/D5395
Martin von Zweigbergk <martinvonz@google.com> [Fri, 07 Dec 2018 14:17:09 -0800] rev 40882
rebase: format commit in usual way when explaining that it became empty
We use the _ctxdesc() for other similar messages, so it seems we
should use it here too.
Differential Revision: https://phab.mercurial-scm.org/D5394
Yuya Nishihara <yuya@tcha.org> [Mon, 29 Oct 2018 21:50:53 +0900] rev 40881
rust: propagate error of index_get_parents() properly
Before, rustla_contains() would return 0 on error, and the exception would
be cleared or noticed somewhere else. We need to propagate the error from
AncestorsIterator up to the FFI surface.
Yuya Nishihara <yuya@tcha.org> [Sun, 02 Dec 2018 22:20:38 +0900] rev 40880
rust: look up HgRevlogIndex_GetParents() from symbol table
And removes the unused index_get_parents_checked() function.
I expect the Index struct will be turned into a pyobject type, though I
haven't written any PoC-level patches yet.
Yuya Nishihara <yuya@tcha.org> [Sun, 02 Dec 2018 22:10:37 +0900] rev 40879
revlog: add public CPython function to get parent revisions
Since this is a public function, it validates the input revision, and supports
nullrev. index_get_parents_checked() will be replaced by this function.
Yuya Nishihara <yuya@tcha.org> [Sun, 02 Dec 2018 21:41:24 +0900] rev 40878
revlog: rename indexType to HgRevlogIndex_Type as it's a global symbol
It follows the CPython naming convention, but uses Hg instead of Py.
Yuya Nishihara <yuya@tcha.org> [Sun, 02 Dec 2018 21:37:42 +0900] rev 40877
revlog: export symbol of indexType
The idea is to wrap the index object with rust-cpython. I haven't tried it,
but it should be doable. We'll probably need a better interface than raw
function pointers to do more in Rust.
Yuya Nishihara <yuya@tcha.org> [Sun, 02 Dec 2018 21:33:43 +0900] rev 40876
setup: fix path of rust source dependencies
Appears that they were screwed up while moving the source files around.
Since rust-direct-ffi sources are automatically added to the depends list,
we only need to list hg-core sources.
Martin von Zweigbergk <martinvonz@google.com> [Sun, 04 Mar 2018 08:50:12 -0800] rev 40875
rebase: remove now-unnecessary arguments to _abort()
Since _abort() is now an instance method, we don't need to pass state
that's available on "self".
Differential Revision: https://phab.mercurial-scm.org/D5393
Martin von Zweigbergk <martinvonz@google.com> [Sun, 04 Mar 2018 08:41:29 -0800] rev 40874
rebase: move abort() onto rebaseruntime
The function depends on a lot of the state of rebaseruntime, so it
makes sense for it to be an instance method. This will let us remove
many of the arguments to the method. That will be done in a later
patch.
Differential Revision: https://phab.mercurial-scm.org/D5392
Martin von Zweigbergk <martinvonz@google.com> [Wed, 21 Mar 2018 16:46:28 -0700] rev 40873
cleanupnodes: trust caller when "moves" is not None
If "moves" (indicating how to move bookmarks) is None, we fill it out
based on "replacements" (indicating which obsmarkers to add). If
"moves" is not None, we would still add items based on
"replacements". This makes it impossible to pass "moves={}" and not
move bookmarks, which surprised me. The only caller that currently
passes a value for "moves" was the rebase extension and there we were
already adding bookmark moves corresponding to obsmarker additions, so
it should not be impacted.
Differential Revision: https://phab.mercurial-scm.org/D5391
Martin von Zweigbergk <martinvonz@google.com> [Wed, 05 Dec 2018 14:17:15 -0800] rev 40872
shelve: change transaction description from "commit" to "shelve"
"commit" was probably a copy&paste mistake.
Differential Revision: https://phab.mercurial-scm.org/D5390
Martin von Zweigbergk <martinvonz@google.com> [Wed, 05 Dec 2018 14:08:01 -0800] rev 40871
shelve: drop unnecessary backup of dirstate for phase-based case
Regular shelve has a hack using an uncommitted transaction that's then
aborted at the end of the operation. It preserves the dirstate across
the abort, however, by saving a backup copy of it. Phase-based shelve
instead commits the transaction, so the hack shouldn't be necessary
there.
Differential Revision: https://phab.mercurial-scm.org/D5389
Martin von Zweigbergk <martinvonz@google.com> [Wed, 05 Dec 2018 14:46:09 -0800] rev 40870
tests: split test-shelve.t in two
test-shelve.t dominated run time for all shelve tests.
Before:
# Ran 9 tests, 1 skipped, 0 failed.
real 0m43.568s
user 2m15.822s
sys 0m40.857s
After:
# Ran 11 tests, 1 skipped, 0 failed.
real 0m24.574s
user 2m21.354s
sys 0m40.435s
Differential Revision: https://phab.mercurial-scm.org/D5388
Martin von Zweigbergk <martinvonz@google.com> [Tue, 04 Dec 2018 22:16:13 -0800] rev 40869
remotefilelog: rely on progress helper for keeping track of position
The progress helper class keeps track of its current position, so we
don't need a "count" variable for that.
Differential Revision: https://phab.mercurial-scm.org/D5386
Martin von Zweigbergk <martinvonz@google.com> [Tue, 04 Dec 2018 22:15:42 -0800] rev 40868
remotefilelog: reduce use of "count" container
We had already given the initial value of "count[0]" a name, so just
use that when applicable.
Differential Revision: https://phab.mercurial-scm.org/D5385
Martin von Zweigbergk <martinvonz@google.com> [Tue, 04 Dec 2018 22:12:47 -0800] rev 40867
remotefilelog: replace a "a=[expr]; b=a[0]" by "b=expr; a = [b]"
Simpler is better.
Differential Revision: https://phab.mercurial-scm.org/D5384
Martin von Zweigbergk <martinvonz@google.com> [Tue, 04 Dec 2018 22:09:33 -0800] rev 40866
remotefilelog: remove an unnecessary update of "count" container
The "count" container is never used after this point, so there is no
need to update it.
Differential Revision: https://phab.mercurial-scm.org/D5383
Martin von Zweigbergk <martinvonz@google.com> [Tue, 04 Dec 2018 21:58:46 -0800] rev 40865
remotefilelog: avoid temporarily using "count" variable as synonym for "total"
The "count" variable is generally used for updating progress, but
early in fileserverclient.request(), its used to mean the total
count. We already have another "total" variable for that, so it seems
much clearer to use that.
Differential Revision: https://phab.mercurial-scm.org/D5382
Martin von Zweigbergk <martinvonz@google.com> [Wed, 05 Dec 2018 09:30:01 -0800] rev 40864
remotefilelog: use progress helper in fileserverclient
Differential Revision: https://phab.mercurial-scm.org/D5381
Martin von Zweigbergk <martinvonz@google.com> [Tue, 04 Dec 2018 16:32:59 -0800] rev 40863
remotefilelog: use progress helper in shallowrepo
Differential Revision: https://phab.mercurial-scm.org/D5380
Martin von Zweigbergk <martinvonz@google.com> [Tue, 04 Dec 2018 16:31:04 -0800] rev 40862
remotefilelog: use progress helper in shallowbundle
Differential Revision: https://phab.mercurial-scm.org/D5379
Martin von Zweigbergk <martinvonz@google.com> [Tue, 04 Dec 2018 16:29:05 -0800] rev 40861
remotefilelog: use progress helper in repack
Differential Revision: https://phab.mercurial-scm.org/D5378
Martin von Zweigbergk <martinvonz@google.com> [Tue, 04 Dec 2018 16:23:00 -0800] rev 40860
remotefilelog: use progress helper in remotefilelogserver
Differential Revision: https://phab.mercurial-scm.org/D5377
Martin von Zweigbergk <martinvonz@google.com> [Tue, 04 Dec 2018 16:09:20 -0800] rev 40859
remotefilelog: use progress helper in basestore
Differential Revision: https://phab.mercurial-scm.org/D5376
Martin von Zweigbergk <martinvonz@google.com> [Tue, 04 Dec 2018 16:02:56 -0800] rev 40858
remotefilelog: use progress helper when analyzing repos
Differential Revision: https://phab.mercurial-scm.org/D5375
Martin von Zweigbergk <martinvonz@google.com> [Tue, 04 Dec 2018 15:59:47 -0800] rev 40857
fastannotate: use progress helper
Differential Revision: https://phab.mercurial-scm.org/D5374
Martin von Zweigbergk <martinvonz@google.com> [Tue, 04 Dec 2018 15:57:11 -0800] rev 40856
debugbuildannotatecache: use progress helper
Differential Revision: https://phab.mercurial-scm.org/D5373
Boris Feld <boris.feld@octobus.net> [Fri, 13 Jul 2018 11:45:15 +0200] rev 40855
upgrade: clarify "aggressivemergedelta" handling
We rename "aggressivemergedelta" argument to "forceaggressivemergedelta". The
previous argument naming could infer an absolute control on the behavior.
However, the code show we respect the config option if set.
Boris Feld <boris.feld@octobus.net> [Fri, 13 Jul 2018 02:54:52 +0200] rev 40854
upgrade: test that fncache is valid after repository upgrade
The behavior was correct. Now it is correct and tested.
Augie Fackler <augie@google.com> [Wed, 05 Dec 2018 12:25:41 -0500] rev 40853
tests: update `rm` invocation for new location of checklink
This should fix the test on FreeBSD.
Differential Revision: https://phab.mercurial-scm.org/D5387
idlsoft <idlsoft@gmail.com> [Mon, 03 Dec 2018 14:17:38 -0500] rev 40852
bookflow: new extension for bookmark-based branching
This extension should be helpful for feature branches - based workflows.
At my company we first considered branches, but weren't sure about creating a lot of permanent objects.
We tried bookmarks, but found some scenarios to be difficult to control.
The main problem, was the active bookmark being moved on update.
Disabling that made everything a lot more predictable.
Bookmarks move on commit, and updating means switching between them.
The extension also implements a few minor features to better guide the workflow:
- hg bookmark NAME can be ambiguous (create or move), unlike hg branch. The extension requires -rev to move.
- require an active bookmark on commit.
- some bookmarks can be protected (like @), useful for teams, that require code reviews.
- block creation of new branches.
The initial implementation requires no changes in the core, but it does rely on some implementation details.
I thought it may be useful to discuss the functionality first, and then focus making the code more robust.
Differential Revision: https://phab.mercurial-scm.org/D4312
Georges Racinet <gracinet@anybox.fr> [Tue, 04 Dec 2018 17:03:43 +0000] rev 40851
rust: make clean takes care of rust/target
This isn't the prettiest way of doing it, but it doesn't require looking
up cargo, or wondering whether that should be part of setup.py clean.
Differential Revision: https://phab.mercurial-scm.org/D5369
Georges Racinet <georges@racinet.fr> [Tue, 04 Dec 2018 14:01:13 +0000] rev 40850
rust: rename local variables in AncestorsIterator::next
It was confusing to have p1 and parents.1 ; (p1, p2) is clearer.
Differential Revision: https://phab.mercurial-scm.org/D5365
Augie Fackler <augie@google.com> [Tue, 04 Dec 2018 17:34:10 -0500] rev 40849
tests: stabilize test-inherit-mode.t on FreeBSD and macOS again
This is the same fix as 90e26ef4cbb1, just repeated on the new file
location.
Differential Revision: https://phab.mercurial-scm.org/D5371
Martin von Zweigbergk <martinvonz@google.com> [Tue, 04 Dec 2018 08:50:33 -0800] rev 40848
tests: drop redundant "#if execbit" from test-rebase-inmemory.t
The whole file is already guarded by "#require execbit".
Differential Revision: https://phab.mercurial-scm.org/D5366
Martin von Zweigbergk <martinvonz@google.com> [Mon, 03 Dec 2018 16:56:09 -0800] rev 40847
narrow: don't resurrect old commits when narrowing (don't strip obsmarkers)
If you have an old obsolescence-chain of commits that has been pruned
and you narrow your repo so that some of those commits get stripped
(because they affected the removed paths), then we would currently
resurrect the commit that came before (along the obsmarker chain) the
last stripped commit. That happens by the usual rules for
obsmarker-stripping. However, it's quite surprising when it happens
when you narrow your repo. This patch makes narrowing not strip
obsmarkers.
Differential Revision: https://phab.mercurial-scm.org/D5364
Yuya Nishihara <yuya@tcha.org> [Sun, 18 Nov 2018 19:55:53 +0900] rev 40846
commandserver: turn server debug messages into logs
They were ui.debug() just because commandserver.log() was noop if no client
connected.
Yuya Nishihara <yuya@tcha.org> [Sun, 18 Nov 2018 19:47:04 +0900] rev 40845
commandserver: add config knob for various logging options
The default rotating options are copied from the blackbox extension.
Yuya Nishihara <yuya@tcha.org> [Sat, 10 Nov 2018 19:27:47 +0900] rev 40844
commandserver: expand log path for convenience
This allows us to set the log path relative to $XDG_RUNTIME_DIR, for instance.
[cmdserver]
log = $XDG_RUNTIME_DIR/chg/server.log
Yuya Nishihara <yuya@tcha.org> [Sat, 10 Nov 2018 19:09:37 +0900] rev 40843
commandserver: switch logging facility to ui.log() interface
The "pager subcommand" message is removed since ui isn't accessible there.
I think that's okay as cmdtable[cmd]() will call attachio() and some debug
message will be printed.
Yuya Nishihara <yuya@tcha.org> [Sat, 10 Nov 2018 19:00:17 +0900] rev 40842
commandserver: install logger to record server events through canonical API
The global commandserver.log() will be replaced with this.
Yuya Nishihara <yuya@tcha.org> [Sat, 10 Nov 2018 18:19:34 +0900] rev 40841
commandserver: enable logging when server process started
This allows us to keep track of server events before client connects to
the server.
Tests will be added later. Currently there's no log() call to check if
things are working well.
Yuya Nishihara <yuya@tcha.org> [Sat, 10 Nov 2018 18:16:33 +0900] rev 40840
test-commandserver: change way of triggering early crash
Future patches will move the logging facility out of the server class, so
cmdserver.log can't be (ab)used for this purpose. Instead, let's hook the
factory function to raise exception.
Yuya Nishihara <yuya@tcha.org> [Sun, 18 Nov 2018 18:58:06 +0900] rev 40839
loggingutil: add basic logger backends
These classes will be used in command server. They are similar to
the blackboxlogger, but it can't be factored out since the blackbox is so
tightly coupled with a repo object.
Augie Fackler <augie@google.com> [Tue, 04 Dec 2018 17:13:01 -0500] rev 40838
merge with stable
Georges Racinet <gracinet@anybox.fr> [Thu, 29 Nov 2018 09:13:13 +0000] rev 40837
rust: peek_mut optim for lazy ancestors
This is one of the two optimizations that are also
present in the Python code: replacing pairs of pop/push
on the BinaryHeap by single updates, hence having it
under the hood maintain its consistency (sift) only once.
On Mozilla central, the measured gain (see details below)
is around 7%.
Creating the PeekMut object by calling peek_mut() right away
instead of peek() first is less efficient (gain is only 4%, stats
not included).
Our interpretation is that its creation has a cost which is vasted
in the cases where it ends by droping the value (Peekmut::pop()
just does self.heap.pop() anyway). On the other hand, the immutable
peek() is very fast: it's just taking a reference in the
underlying vector.
The Python version still has another optimization:
if parent(current) == current-1, then the heap doesn't need
to maintain its consistency, since we already know that
it's bigger than all the others in the heap.
Rust's BinaryHeap doesn't allow us to mutate its biggest
element with no housekeeping, but we tried it anyway, with a
copy of the BinaryHeap implementation with a dedicaded added
method: it's not worth the technical debt in our opinion
(we measured only a further 1.6% improvement).
One possible explanation would be that the sift is really fast
anyway in that case, whereas it's not in the case of Python,
because it's at least partly done in slow Python code.
Still it's possible that replacing BinaryHeap by something more
dedicated to discrete ordered types could be faster.
Measurements on mozilla-central:
Three runs of 'hg perfancestors' on the parent changeset:
Moyenne des médianes: 0.100587
! wall 0.100062 comb 0.100000 user 0.100000 sys 0.000000 (best of 98)
! wall 0.135804 comb 0.130000 user 0.130000 sys 0.000000 (max of 98)
! wall 0.102864 comb 0.102755 user 0.099286 sys 0.003469 (avg of 98)
! wall 0.101486 comb 0.110000 user 0.110000 sys 0.000000 (median of 98)
! wall 0.096804 comb 0.090000 user 0.090000 sys 0.000000 (best of 100)
! wall 0.132235 comb 0.130000 user 0.120000 sys 0.010000 (max of 100)
! wall 0.100258 comb 0.100300 user 0.096000 sys 0.004300 (avg of 100)
! wall 0.098384 comb 0.100000 user 0.100000 sys 0.000000 (median of 100)
! wall 0.099925 comb 0.100000 user 0.100000 sys 0.000000 (best of 98)
! wall 0.133518 comb 0.140000 user 0.130000 sys 0.010000 (max of 98)
! wall 0.102381 comb 0.102449 user 0.098265 sys 0.004184 (avg of 98)
! wall 0.101891 comb 0.090000 user 0.090000 sys 0.000000 (median of 98)
Mean of the medians: 0.100587
On the present changeset:
! wall 0.091344 comb 0.090000 user 0.090000 sys 0.000000 (best of 100)
! wall 0.122728 comb 0.120000 user 0.110000 sys 0.010000 (max of 100)
! wall 0.093268 comb 0.093300 user 0.089300 sys 0.004000 (avg of 100)
! wall 0.092567 comb 0.100000 user 0.090000 sys 0.010000 (median of 100)
! wall 0.093294 comb 0.080000 user 0.080000 sys 0.000000 (best of 100)
! wall 0.144887 comb 0.150000 user 0.140000 sys 0.010000 (max of 100)
! wall 0.097708 comb 0.097700 user 0.093400 sys 0.004300 (avg of 100)
! wall 0.094980 comb 0.100000 user 0.090000 sys 0.010000 (median of 100)
! wall 0.091262 comb 0.090000 user 0.080000 sys 0.010000 (best of 100)
! wall 0.123772 comb 0.130000 user 0.120000 sys 0.010000 (max of 100)
! wall 0.093188 comb 0.093200 user 0.089300 sys 0.003900 (avg of 100)
! wall 0.092364 comb 0.100000 user 0.090000 sys 0.010000 (median of 100)
Mean of the medians is 0.0933
Differential Revision: https://phab.mercurial-scm.org/D5358
Augie Fackler <augie@google.com> [Mon, 03 Dec 2018 18:07:09 -0500] rev 40836
fuzz: grep away HAVE_GETC_UNLOCKED in pyconfig.h to avoid msan badness
Per discussion with Greg Smith and the patches on
https://bugs.python.org/issue35214. This, combined with the previous
patch, fixes msan builds on oss-fuzz.
Differential Revision: https://phab.mercurial-scm.org/D5363
Augie Fackler <augie@google.com> [Tue, 13 Nov 2018 09:19:05 -0500] rev 40835
fuzz: more correctly specify CFLAGS and LDFLAGS when building Python
Gets us closer to a working msan build alongside our asan build.
Differential Revision: https://phab.mercurial-scm.org/D5362
Matt Harbison <matt_harbison@yahoo.com> [Tue, 04 Dec 2018 00:19:33 -0500] rev 40834
tests: stabilize test-blackbox.t on Windows
I didn't look into why the error is more detailed, but that seems like it's a
good thing (other than for recording tests).
Matt Harbison <matt_harbison@yahoo.com> [Tue, 04 Dec 2018 00:16:12 -0500] rev 40833
tests: stabilize for recent wcache changes
This goes with 47e3f554df35::d5622dfe4ba3. I'm not sure if it was really
expected that there would be no wcache directory if neither execbit nor symlink
is supported.
Matt Harbison <matt_harbison@yahoo.com> [Mon, 03 Dec 2018 12:48:42 -0500] rev 40832
extdiff: avoid double backslashes in the displayed tool path on Windows
This shows the tool path in the help, and changed in 67b180c0e263. uirepr()
already does the same thing, but that undoes the mangling in its call to repr().
Boris Feld <boris.feld@octobus.net> [Wed, 28 Nov 2018 05:06:58 +0100] rev 40831
contrib: add a helper script that help to build interesting repositories
The script is dedicated to building a couple of repositories that should be
interesting to run discovery from one another. It seems a common enough need
to contribute it upstream.
Pulkit Goyal <pulkit@yandex-team.ru> [Mon, 03 Dec 2018 19:42:46 +0300] rev 40830
py3: listify filter() to call len() on it
Differential Revision: https://phab.mercurial-scm.org/D5354
Yuya Nishihara <yuya@tcha.org> [Sun, 18 Nov 2018 18:35:31 +0900] rev 40829
loggingutil: document openlogfile()
This function will be used later for command-server logging.
Yuya Nishihara <yuya@tcha.org> [Sun, 18 Nov 2018 18:25:37 +0900] rev 40828
loggingutil: extract openlogfile() and proxylogger to new module
This module isn't placed under the "utils" package since it needs "ui" to
process things. It's called "loggingutil", not "logutil" because the word
"log" is too obscure in our codebase.
Yuya Nishihara <yuya@tcha.org> [Sun, 18 Nov 2018 18:21:39 +0900] rev 40827
blackbox: pass in options to _openlogfile() as arguments
This prepares for extracting utility function from the blackbox module.
Yuya Nishihara <yuya@tcha.org> [Sat, 17 Nov 2018 22:10:27 +0900] rev 40826
blackbox: just try writing to repo.vfs and update lastlogger on success
This is simpler and more robust. Before, an empty ".hg" directory would be
created if it's removed after checking vfs.isdir('.').
Yuya Nishihara <yuya@tcha.org> [Tue, 20 Nov 2018 22:31:12 +0900] rev 40825
vfs: add option to not create parent directories implicitly
In blackbox, we don't want to create a ".hg" directory by mistake. This
provides a race-safe option to achieve that.
Boris Feld <boris.feld@octobus.net> [Thu, 15 Nov 2018 02:55:33 +0100] rev 40824
repo: add a `wcachevfs` to access the `.hg/wcache/` directory
This wvfs will allow us to migrate various cache to the new `wcache` directory.
Helping with cache issues with "share".
Boris Feld <boris.feld@octobus.net> [Thu, 15 Nov 2018 02:46:31 +0100] rev 40823
cache: create `wcache` directory at init time
The cache directory will be needed very quickly, so it seems simpler to create
it early to make sure it has the same owner and permission than the other
directory in the repository.
Boris Feld <boris.feld@octobus.net> [Thu, 15 Nov 2018 02:38:55 +0100] rev 40822
cache: create `cache` directory at init time
The cache directory will be needed very quickly, so it seems simpler to create
it early to make sure it has the same owner and permission than the other
directory in the repository.
Boris Feld <boris.feld@octobus.net> [Thu, 15 Nov 2018 17:08:23 +0100] rev 40821
check-exec: write file in 'wcache' instead of 'cache'
Some cache are relevant or affected by the working copy used. So the `.hg/cache`
directory is not the best place for them because multiple shared repository can
end up fighting over them.
To address this issue, we introduce a new 'wcache' directory to host this kind
of cache.
The first user are the `checkisexec` type file. These files describe property of
the working copy and fit the use-case well.
Boris Feld <boris.feld@octobus.net> [Fri, 23 Nov 2018 06:09:44 +0100] rev 40820
mmapindex: set default to 1MB
mmapping index is more efficient if we only need a small part of it.
The 1MB value has been picked arbitrarily, a lower value might be better.
On a large repository with a 60MB index, we see the following performance
gain:
hg perfindex
before: ! wall 0.032023 comb 0.040000 user 0.000000 sys 0.040000 (best of 100)
after: ! wall 0.000196 comb 0.000000 user 0.000000 sys 0.000000 (best of 1060)
The speed boost benefit all cases, including the one where the full index
needs to be parsed.
hg perfindex --rev 0
before: ! wall 0.040673 comb 0.030000 user 0.000000 sys 0.030000 (best of 100)
after ! wall 0.010713 comb 0.020000 user 0.010000 sys 0.010000 (best of 212)
This gain reflect in higher level operation:
hg perfbookmarks --clear-revlogs
before: ! wall 0.161339 comb 0.160000 user 0.130000 sys 0.030000 (best of 56)
after: ! wall 0.123228 comb 0.120000 user 0.120000 sys 0.000000 (best of 68)
Boris Feld <boris.feld@octobus.net> [Fri, 23 Nov 2018 06:07:33 +0100] rev 40819
mmapindex: move the 'mmapindexthreshold' option out of experimental
The option is useful and should be advertised more. We move it out of
experimental as a first step. The `storage` section is selected as this is
related to how the storage is accessed. A new 'performance' section might be
more appropriate.
We move from 'mmapindexthreshold` to `mmap-threshold` as non-index item are
also suitable for mmap (eg: the rev-branch-cache).
If relevant, we can introduce sub-option `mmap-threshold.revlog-index` later.
Boris Feld <boris.feld@octobus.net> [Sat, 01 Dec 2018 15:57:27 +0100] rev 40818
perf: add a --rev attribute to perfindex
This allow for benchmarking the time necessary to look for other version than
the tip.
Boris Feld <boris.feld@octobus.net> [Fri, 23 Nov 2018 06:03:38 +0100] rev 40817
perf: update perfindex to be more realistic
The previous code was creating a revlog manually, we now use the actual
`localrepo` method to create it.
We have to jump though extra hops to work around the impact of filecache.
Martin von Zweigbergk <martinvonz@google.com> [Sun, 02 Dec 2018 13:09:46 -0800] rev 40816
match: drop unnecessary wrapping of regex in group
It seems the regexes have been wrapped in an unnamed group since
b6c42714d900 (Add locate command., 2005-07-05). In that commit, the
grouping was needed because there was a "head" ('^') added before the
group and a "tail" (os.sep) added after it. It seems the head was
moved inside the group in 1c0c413cccdd (Get add and locate to use new
repo and dirstate walk code., 2005-07-18) and the tail was moved
inside the group in 89985a1b3427 (Clean up walk and changes code to
use normalised names properly., 2005-07-31), So it seems to me that
we've carried around the unnecessary group for 13 years. This patch
removes it.
Differential Revision: https://phab.mercurial-scm.org/D5352
Martin von Zweigbergk <martinvonz@google.com> [Sun, 02 Dec 2018 13:45:20 -0800] rev 40815
match: use _BASE_SIZE instead of magic value 4
Differential Revision: https://phab.mercurial-scm.org/D5351
Martin von Zweigbergk <martinvonz@google.com> [Sun, 02 Dec 2018 13:44:49 -0800] rev 40814
match: make "groupsize" include the trailing "|"
I think this is a little easier to follow and it will simplify later
patches too.
Differential Revision: https://phab.mercurial-scm.org/D5350
Martin von Zweigbergk <martinvonz@google.com> [Sun, 02 Dec 2018 13:09:43 -0800] rev 40813
match: fix an unaligned (but harmless) indent
Differential Revision: https://phab.mercurial-scm.org/D5349
Boris Feld <boris.feld@octobus.net> [Thu, 22 Nov 2018 17:41:10 +0100] rev 40812
match: raise an Abort error instead of OverflowError
This case of OverflowError (one single pattern being too large) has never been
properly caught in the past.
Boris Feld <boris.feld@octobus.net> [Thu, 22 Nov 2018 21:02:02 +0100] rev 40811
match: avoid translating glob to matcher multiple times for large sets
For hgignore with many globs, the resulting regexp might not fit under the 20K
length limit. So the patterns need to be broken up in smaller pieces.
Before this change, the logic was re-starting the full process from scratch
for each smaller pieces, including the translation of globs into regexp.
Effectively doing the work over and over.
If the 20K limit is reached, we are likely in a case where there is many such
glob, so exporting them is especially expensive and we should be careful not
to do that work more than once.
To work around this, we now translate glob to regexp once and for all. Then,
we assemble the resulting individual regexp into valid blocks.
This raises a very significant performance win for large `.hgignore file`:
Before: ! wall 0.153153 comb 0.150000 user 0.150000 sys 0.000000 (median of 66)
After: ! wall 0.059793 comb 0.060000 user 0.060000 sys 0.000000 (median of 100)
Boris Feld <boris.feld@octobus.net> [Thu, 22 Nov 2018 17:25:49 +0100] rev 40810
match: extract function that group regexps
Boris Feld <boris.feld@octobus.net> [Thu, 22 Nov 2018 17:16:05 +0100] rev 40809
match: test for overflow error in pattern
If a single pattern is too large to handle, we raise an exception. This case is
now doctested.
Boris Feld <boris.feld@octobus.net> [Thu, 22 Nov 2018 17:20:32 +0100] rev 40808
match: extract a literal constant into a symbolic one
Matt Harbison <matt_harbison@yahoo.com> [Sat, 01 Dec 2018 21:42:48 -0500] rev 40807
tests: apply binary mode to output in seq.py
I noticed this when playing with running tests using WSL, and iterating over the
output yielded '0\r', '1\r',... Most of the other *.py tools do this, and `seq`
on MSYS lacks '\r' in the output, so this is more consistent.
Boris Feld <boris.feld@octobus.net> [Fri, 23 Nov 2018 01:09:37 +0100] rev 40806
perf: add a `--clear-caches` to `perfbranchmapupdate`
This flag will help to measure the time we spend loading various cache that
support the branchmap update.
Example for an 500 000 revisions repository:
hg perfbranchmapupdate --base 'not tip' --target 'tip'
! wall 0.000860 comb 0.000000 user 0.000000 sys 0.000000 (best of 336)
hg perfbranchmapupdate --base 'not tip' --target 'tip' --clear-caches
! wall 0.029494 comb 0.030000 user 0.030000 sys 0.000000 (best of 100)
Boris Feld <boris.feld@octobus.net> [Wed, 21 Nov 2018 21:11:47 +0000] rev 40805
perf: start from an existing branchmap if possible
If the --base set if a superset of one of the cached branchmap, we should use as
a starting point. This greatly help the overall runtime of
`hg perfbranchmapupdate`
For example, for a repository with about 500 000 revisions, using this trick
make the command runtime move from about 200 second to about 10 seconds. A 20x
gain.
Boris Feld <boris.feld@octobus.net> [Wed, 21 Nov 2018 20:35:22 +0000] rev 40804
perf: rely on repoview for perfbranchmapupdate
Using 'repoview' matching the base and target subset make the benchmark more
realistic. It also unlocks optimization to make the command initialization
faster.
Boris Feld <boris.feld@octobus.net> [Wed, 21 Nov 2018 22:56:06 +0100] rev 40803
perf: pre-indent some code in `perfbranchmapupdate`
This make the next patch easier to read.
Boris Feld <boris.feld@octobus.net> [Wed, 21 Nov 2018 12:02:25 +0000] rev 40802
perf: add a `perfbranchmapupdate` command
This command benchmark the time necessary to update the branchmap between two
sets of revisions. This changeset introduce a first version, doing nothing fancy
regarding cache or other internal details.
Anton Shestakov <av6@dwimlabs.net> [Mon, 05 Nov 2018 13:52:19 +0800] rev 40801
push: config option to control behavior when pushing to a publishing server
Pushing to a publishing server by mistake can lead to a difficult situation to
solve because evolution doesn't work on public changesets. This new
experimental config tries to help avoiding unintentionally (or at least being
aware of) pushing to publishing remotes.
`hg push --publish` can be used to make push succeed even when auto-publish is
set to 'abort'.
Pulkit Goyal <pulkit@yandex-team.ru> [Fri, 30 Nov 2018 17:42:55 +0300] rev 40800
narrowcommands: remove an unrequired `repo.narrowpats` call
We call that few lines above and do nothing significant in between which can
change the narrowpats. So let's use values returned by that call.
Differential Revision: https://phab.mercurial-scm.org/D5348
Augie Fackler <augie@google.com> [Thu, 29 Nov 2018 16:44:01 -0500] rev 40799
manifest: reject lines shorter than 42 bytes, not 22
Yuya correctly spotted during the review of f27f8e9ef1e73 that we're
dealing with hexlified hashes here, and so it should be 42 bytes not
22.
Differential Revision: https://phab.mercurial-scm.org/D5347
Yuya Nishihara <yuya@tcha.org> [Sun, 11 Nov 2018 20:05:38 +0900] rev 40798
blackbox: initialize logger with repo instance
The blackboxlogger is unusable without a repo. Let's simply initialize it
with a repo instance.
Yuya Nishihara <yuya@tcha.org> [Sat, 17 Nov 2018 20:56:25 +0900] rev 40797
blackbox: do not nullify repo to deactivate the logger on failure
The _repo will be a mandatory attribute. Instead, make the logger to not
track any events.
Yuya Nishihara <yuya@tcha.org> [Sun, 11 Nov 2018 20:02:34 +0900] rev 40796
blackbox: extract global last logger to proxylogger class
So the blackboxlogger can be instantiated with a repo.
Yuya Nishihara <yuya@tcha.org> [Sun, 11 Nov 2018 19:36:21 +0900] rev 40795
ui: pass in bytes opts dict to logger.log()
This is the convention of the Mercurial API.
Yuya Nishihara <yuya@tcha.org> [Sun, 11 Nov 2018 19:35:33 +0900] rev 40794
ui: pass in formatted message to logger.log()
This makes sure that all logger instances will handle the message arguments
properly.