Augie Fackler <augie@google.com> [Mon, 30 Jul 2018 10:42:37 -0400] rev 38835
linelog: add a Python implementation of the linelog datastructure
This datastructure was originally developed by Jun Wu at Facebook,
inspired by SCCS weaves. It's useful as a cache for blame information,
but also is the magic that makes `hg absorb` easy to implement. In
service of importing the code to Mercurial, I wanted to actually
/understand/ it, and once I did I decided to take a run at
implementing it.
The help/internals/linelog.txt document is the README from Jun Wu's
implementaiton. It all applies to our linelog implementation.
Differential Revision: https://phab.mercurial-scm.org/D3990
Gregory Szorc <gregory.szorc@gmail.com> [Sat, 28 Jul 2018 11:40:31 -0700] rev 38834
changegroup: move file matcher from narrow extension
Sparse changegroup generation requires the use of a matcher to
filter which files are relevant.
This commit moves the file matcher from the narrow extension to core
and updates the narrow extension to use it.
I'm not sure why the narrow extension was storing the matcher as
a callable that resolved to a matcher. So I changed it to be a
simple matcher instance.
In addition, code from narrow to intersect the matcher with the local
narrow spec is now performed automatically when the changegroup
packer is created.
If a matcher is not passed into getbundler() an alwaysmatcher() is
assumed. This ensures that a matcher is always defined for all
operations.
Differential Revision: https://phab.mercurial-scm.org/D4011
Kyle Lippincott <spectral@google.com> [Thu, 26 Jul 2018 17:11:03 -0700] rev 38833
resolve: add option to warn/abort on -m with unresolved conflict markers
When a user is dropped out of Mercurial to a terminal to resolve files, we emit
messages like:
conflicts while merging file1! (edit, then use 'hg resolve --mark')
conflicts while merging file2! (edit, then use 'hg resolve --mark')
We don't mention a file name in the hint, so some users might do something like
`$EDITOR file1; hg resolve --mark`, see that it says "(no more unresolved
files)" and forget to deal with file2 before running the next command.
Even if we did mention a file name in the hint, it's too easy to forget it
(maybe the merge spans a couple days or something). This option lets us inform
the user that they might have missed something.
In the scenario above, the output would be something like:
warning: the following files still have conflict markers:
file2
(no more unresolved files)
Differential Revision: https://phab.mercurial-scm.org/D4035
Gregory Szorc <gregory.szorc@gmail.com> [Mon, 02 Jul 2018 18:39:48 -0700] rev 38832
exchange: refactor control flow of _getbundlechangegrouppart()
The use of early return makes the control flow of this function much
easier to reason about IMO.
Differential Revision: https://phab.mercurial-scm.org/D4010
Gregory Szorc <gregory.szorc@gmail.com> [Mon, 02 Jul 2018 18:32:20 -0700] rev 38831
exchange: move _computeellipsis() from narrow
This is also referenced as part of the narrow changegroup code and
therefore needs to move to core before we can integrate the narrow
changegroup code into core.
Differential Revision: https://phab.mercurial-scm.org/D4009
Gregory Szorc <gregory.szorc@gmail.com> [Mon, 02 Jul 2018 18:24:26 -0700] rev 38830
exchange: move narrow acl functionality into core
This function is called by the custom changegroup generation code
in the narrow extension. I want to move that changegroup code into
core. That means we need to move this function.
The code is kinda hacky in that assumes existence of REMOTE_USER,
which is only present on authenticated HTTP requests. I've added
a comment indicating that.
Differential Revision: https://phab.mercurial-scm.org/D4008
Gregory Szorc <gregory.szorc@gmail.com> [Sat, 28 Jul 2018 10:41:23 -0700] rev 38829
exchange: move disabling of rev-branch-cache bundle part out of narrow
I'm attempting to refactor changegroup code in order to better
support alternate storage backends. The narrow extension is
performing a lot of monkeypatching to this code and it is making
it difficult to reason about how everything works. I'm reasonably
certain I would be unable to abstract storage without requiring
extensive rework of narrow. I believe it is less effort to move
narrow code into core so it can be accounted for when changegroup
code is refactored. So I'll be doing that.
The first part of this is integrating the disabling of the
cache:rev-branch-cache bundle2 part into core. This doesn't seem
like it is related to changegroup, but narrow's modifications to
changegroup are invasive and also require taking its code for
bundle generation and exchange into core in order for the changegroup
code to work.
Differential Revision: https://phab.mercurial-scm.org/D4007
Martin von Zweigbergk <martinvonz@google.com> [Tue, 24 Jul 2018 10:47:42 -0700] rev 38828
dispatch: show a short error message when invalid global option given
Similar reasoning as the previous patch.
Differential Revision: https://phab.mercurial-scm.org/D4025
Martin von Zweigbergk <martinvonz@google.com> [Tue, 24 Jul 2018 10:22:07 -0700] rev 38827
dispatch: don't show list of commands on bogus command
If a command is ambiguous, you get this:
$ hg ve
hg: command 've' is ambiguous:
verify version
[255]
If you typo a command, you get this:
$ hg comit
hg: unknown command 'comit'
(did you mean one of commit, incoming, mycommit?)
[255]
But if you completely mistype a command so it no longer looks like any
existing commands, you get a full list of commands. That might be
useful the first time you use Mercurial, but after that it's probably
more annoying than help, especially if you have the pager enabled and
have a short terminal. Let's instead give a short hint telling the
user to run `hg help` for more help.
Differential Revision: https://phab.mercurial-scm.org/D4024
Martin von Zweigbergk <martinvonz@google.com> [Tue, 24 Jul 2018 22:51:11 -0700] rev 38826
histedit: avoid repeating name of state file in a few places
We can depend on the state object instead.
Differential Revision: https://phab.mercurial-scm.org/D4006
Gregory Szorc <gregory.szorc@gmail.com> [Wed, 01 Aug 2018 13:10:07 -0700] rev 38825
py3: stop rewriting xrange() to pycompat.xrange()
We now require the use of pycompat.xrange() in source.
One less feature in the module importer gets us one step closer
to Python 3 native source code.
Differential Revision: https://phab.mercurial-scm.org/D4034
Gregory Szorc <gregory.szorc@gmail.com> [Wed, 01 Aug 2018 13:08:00 -0700] rev 38824
check-code: ban use of bare xrange()
We want everyone to use pycompat.xrange().
Differential Revision: https://phab.mercurial-scm.org/D4033
Gregory Szorc <gregory.szorc@gmail.com> [Wed, 01 Aug 2018 13:00:45 -0700] rev 38823
global: use pycompat.xrange()
On Python 3, our module importer automatically rewrites xrange()
to pycompat.xrange().
We want to move away from the custom importer on Python 3.
This commit converts all instances of xrange() to use
pycompat.xrange().
Differential Revision: https://phab.mercurial-scm.org/D4032
Gregory Szorc <gregory.szorc@gmail.com> [Wed, 01 Aug 2018 12:57:15 -0700] rev 38822
pycompat: add xrange alias for Python 2
We assign xrange on Python 3 but not Python 2. Adding the missing
symbol on Python 2 allows us to use `pycompat.xrange` to get a
generator range function.
Differential Revision: https://phab.mercurial-scm.org/D4031
Martijn Pieters <mj@zopatista.com> [Tue, 31 Jul 2018 19:37:54 +0200] rev 38821
manifest: persist the manifestfulltext cache
Reconstructing the manifest from the revlog takes time, so much so that there
already is a LRU cache to avoid having to load a manifest multiple times.
This patch persists that LRU cache in the .hg/cache directory, so we can re-use
this cache across hg commands. Commit benchmark (run on Macos 10.13 on a
2017-model Macbook Pro with Core i7 2.9GHz and flash drive), testing without
and with patch run 5 times, baseline is r2a227782e754:
* committing to an existing file, against the mozilla-central repository.
Baseline real time average 1.9692, with patch 1.3786.
A new debugcommand "hg debugmanifestfulltextcache" lets you inspect the cache,
clear it, or add specific manifest nodeids to it. When calling
repo.updatecaches(), the manifest(s) for the working copy parents are added to
the cache.
The hg perfmanifest command has an additional --clear-disk switch to clear this
cache when testing manifest loading performance.
Using this command to test performance on the firefox repository for revision
f947d902ed91, whose manifest has a delta chain length of 60540, we see:
$ hg perfmanifest f947d902ed91 --clear-disk
! wall 0.972253 comb 0.970000 user 0.850000 sys 0.120000 (best of 10)
$ hg debugmanifestfulltextcache -a `hg log --debug -r f947d902ed91 | grep manifest | cut -d: -f3`
Cache contains 1 manifest entries, in order of most to least recent:
id: 0294517df4aad07c70701db43bc7ff24c3ce7dbc, size 25.6 MB
Total cache data size 25.6 MB, on-disk 0 bytes
$ hg perfmanifest f947d902ed91
! wall 0.036748 comb 0.040000 user 0.020000 sys 0.020000 (best of 100)
Worst-case scenario: a manifest text loaded from a single delta; in the firefox
repository manifest node 9a1246ff762e is the chain base for the manifest
attached to revision f947d902ed91. Loading this from a full cache file is just
as fast as without the cache; the extra node ids ensure a big full cache:
$ for node in 9a1246ff762e 1a1922c14a3e 54a31d11a36a 0294517df4aa; do
> hgd debugmanifestfulltextcache -a $node > /dev/null
> done
$ hgd perfmanifest -m 9a1246ff762e
! wall 0.077513 comb 0.080000 user 0.030000 sys 0.050000 (best of 100)
$ hgd perfmanifest -m 9a1246ff762e --clear-disk
! wall 0.078547 comb 0.080000 user 0.070000 sys 0.010000 (best of 100)
Martijn Pieters <mj@zopatista.com> [Tue, 31 Jul 2018 19:37:48 +0200] rev 38820
debug: allow specifying a manifest node rather than a revision
Martin von Zweigbergk <martinvonz@google.com> [Fri, 20 Jul 2018 11:37:31 -0700] rev 38819
perfmoonwalk: make work with filtered repo
This also fixes the out-of-range "len(repo)" that weirdly works (and
gets the nullid). I suspect it wasn't intentional to include the
timing of reading the null revision.
Differential Revision: https://phab.mercurial-scm.org/D4017
Martin von Zweigbergk <martinvonz@google.com> [Fri, 20 Jul 2018 11:17:33 -0700] rev 38818
revlog: don't include trailing nullrev in revlog.revs(stop=len(revlog))
This was an odd side effect of the nullid entry that's in the
index. The existing callers (mostly repair.py) seem to have handled it
fine. It doesn't seem intentional, and it's pretty surprising, so
let's remove that surprise.
Differential Revision: https://phab.mercurial-scm.org/D4015
Augie Fackler <augie@google.com> [Mon, 16 Jul 2018 14:15:29 -0400] rev 38817
patchbomb: ensure all headers and values given to email mod are native strings
This lets test-patch-bookmark.t only fail with some harmless header
output changes on Python 3, so I think patchbomb might be basically
useful on Python 3 now.
Differential Revision: https://phab.mercurial-scm.org/D3952
Martin von Zweigbergk <martinvonz@google.com> [Mon, 30 Jul 2018 14:37:36 -0700] rev 38816
context: delete an obsolete comment
Obsolete since 91618801d5c3 (context: raise ProgrammingError on
repo['my-tag'], 2018-07-06).
Differential Revision: https://phab.mercurial-scm.org/D4002
Martin von Zweigbergk <martinvonz@google.com> [Fri, 27 Jul 2018 14:48:58 -0700] rev 38815
templatekw: use ctx1.status(ctx2) instead of repo.status(ctx1, ctx2)
Differential Revision: https://phab.mercurial-scm.org/D4001
Martin von Zweigbergk <martinvonz@google.com> [Fri, 27 Jul 2018 14:48:34 -0700] rev 38814
patch: use ctx1.status(ctx2) instead of repo.status(ctx1, ctx2)
Differential Revision: https://phab.mercurial-scm.org/D4000
Martin von Zweigbergk <martinvonz@google.com> [Fri, 27 Jul 2018 14:48:06 -0700] rev 38813
fileset: use ctx1.status(ctx2) instead of repo.status(ctx1, ctx2)
Differential Revision: https://phab.mercurial-scm.org/D3999
Martin von Zweigbergk <martinvonz@google.com> [Fri, 27 Jul 2018 14:47:38 -0700] rev 38812
uncommit: use ctx1.status(ctx2) instead of repo.status(ctx1, ctx2)
Differential Revision: https://phab.mercurial-scm.org/D3998
Martin von Zweigbergk <martinvonz@google.com> [Fri, 27 Jul 2018 14:46:59 -0700] rev 38811
fix: use ctx1.status(ctx2) instead of repo.status(ctx1, ctx2)
Differential Revision: https://phab.mercurial-scm.org/D3997
Martin von Zweigbergk <martinvonz@google.com> [Fri, 27 Jul 2018 14:49:45 -0700] rev 38810
amend: use ctx1.status(ctx2) instead of repo.status(ctx1, ctx2)
Differential Revision: https://phab.mercurial-scm.org/D3996
Rodrigo Damazio Bovendorp <rdamazio@google.com> [Thu, 19 Jul 2018 23:22:05 -0700] rev 38809
dispatch: making all hg abortions be output with a specific label
This allows abortions to be highlighted specially and separately
from warnings - for instance, red is a reasonable color for when hg
aborts, but is overly dramatic for most warnings produced elsewhere.
Differential Revision: https://phab.mercurial-scm.org/D3967
Augie Fackler <augie@google.com> [Thu, 19 Jul 2018 15:21:28 -0400] rev 38808
ui: fix implicit unicode-to-bytes conversion introduced in 9df29b7c62cf
This is harmless, unless you try and run hg with HGUNICODEPEDANTRY
enabled. It's technically wrong, so let's go ahead and fix it.
Differential Revision: https://phab.mercurial-scm.org/D3989
Augie Fackler <augie@google.com> [Wed, 01 Aug 2018 10:23:57 -0400] rev 38807
merge with stable
Matt Harbison <matt_harbison@yahoo.com> [Wed, 10 Oct 2018 12:25:28 -0400] rev 38806
lfs: avoid a potential variable reference before assignment error in cmdserver
A coworker hit this once yesterday when pulling in thg (a retry worked), and
then I hit it with strip after a pull. I had a difficult time recreating a test
for this (at least one of the tricks was to not use '-R', which seems to cause
reposetup() to be called for each command), so I'm not sure how large of a
window there actually is for this. Calling reposetup() *after* the requirement
is added will skip the hook entirely.
The other issue I had was adding a couple `ui.status()` lines around the check
that installs the hook. On Windows, the cmdserver process ballooned to 1.6GB
and hung. Changing that to `ui.warn()` avoided the hang. It also hung on
macOS, but without the large memory usage.
Augie Fackler <raf@durin42.com> [Mon, 01 Oct 2018 16:07:38 -0400] rev 38805
Added signature for changeset 5405cb1a7901
Augie Fackler <raf@durin42.com> [Mon, 01 Oct 2018 16:07:36 -0400] rev 38804
Added tag 4.7.2 for changeset 5405cb1a7901
Yuya Nishihara <yuya@tcha.org> [Wed, 05 Sep 2018 21:23:29 +0900] rev 38803
manifest: fix out-of-bounds read of corrupted manifest entry
Spotted by ASAN.
Yuya Nishihara <yuya@tcha.org> [Wed, 05 Sep 2018 21:32:45 +0900] rev 38802
manifest: rewrite pathlen() to not cross entry boundary
Even though the entire manifest data should be terminated by '\0', it seems
not nice to scan '\0' over the entry terminator, '\n'.
Yuya Nishihara <yuya@tcha.org> [Wed, 26 Sep 2018 21:24:14 +0900] rev 38801
chgserver: do not send system() back to client if stdio redirected (issue5992)
As the chg client doesn't know server-side stdio redirection, the server
shouldn't upcall on "runsystem" request if the stdio streams are redirected.
This patch teaches ui to remember the redirection flag, which is updated by
the caller right now. Future patches (for default) will add ui methods to
manage this flag internally.
Yuya Nishihara <yuya@tcha.org> [Wed, 26 Sep 2018 21:21:05 +0900] rev 38800
chgserver: update comment describing when to fall back to core _runsystem()
The condition "output needs to be captured" was moved to the core ui, and
removed at fbce78c58f1e "chg: refactor ui.system() to be partly overridden."
The next patch will add one more "if" to address the issue 5992.
Yuya Nishihara <yuya@tcha.org> [Wed, 26 Sep 2018 20:53:59 +0900] rev 38799
procutil: compare fd number to see if stdio protection is needed (issue5992)
When I wrote this function for commandserver at 69f86b937035, testing object
identity was suffice, and I was sloppy enough not to compare fileno() values.
However, it doesn't work in chg session because chgserver reopens stdio to
apply new buffering mode.
This patch partially fixes the issue 5992. Still we have another problem in
chgui._runsystem().
Yuya Nishihara <yuya@tcha.org> [Tue, 25 Sep 2018 23:06:02 +0900] rev 38798
test-ssh: show that stdio redirection doesn't work with chg
Running tests with --chg doesn't mean all hg invocations are replaced by
chg. This patch explicitly adds the test for "chg serve --stdio", which does
weird stdio dance.
Valentin Gatien-Baron <vgatien-baron@janestreet.com> [Tue, 25 Sep 2018 16:32:38 -0400] rev 38797
revset: make heads(commonancestors(x + x^)) be x^, not x
Differential Revision: https://phab.mercurial-scm.org/D4742
Valentin Gatien-Baron <vgatien-baron@janestreet.com> [Tue, 25 Sep 2018 16:29:39 -0400] rev 38796
revset: add tests of heads(commonancestors(..))
The second-to-last one shows the same bug as commonancestors(..): the
result should be 8, not 9.
Differential Revision: https://phab.mercurial-scm.org/D4741
Valentin Gatien-Baron <vgatien-baron@janestreet.com> [Tue, 25 Sep 2018 16:18:43 -0400] rev 38795
revset: reword commonancestor()'s help
The new version seems a bit more consistent with other doc comments,
and feels clearer to me (doesn't explain "commonancestors(set)" as
"common ancestors of set").
Differential Revision: https://phab.mercurial-scm.org/D4740
Valentin Gatien-Baron <vgatien-baron@janestreet.com> [Tue, 25 Sep 2018 16:14:57 -0400] rev 38794
revset: remove part of the commonancestors() comment
The reason is that:
- it shows up in "hg help revset", but it doesn't look like
documentation targeted at users
- it doesn't make sense to me: it doesn't say what happens with < 2
revisions, and is not quite right because my understanding is that
this revset was created precisely because "::x and ::y" was not
quite the same (when x and y don't evaluate to singletons).
Differential Revision: https://phab.mercurial-scm.org/D4739
Valentin Gatien-Baron <vgatien-baron@janestreet.com> [Tue, 25 Sep 2018 16:05:21 -0400] rev 38793
revset: make commonancestors(x + x^) be ::(x^), not ::x
Differential Revision: https://phab.mercurial-scm.org/D4738
Valentin Gatien-Baron <vgatien-baron@janestreet.com> [Tue, 25 Sep 2018 15:27:41 -0400] rev 38792
revset: add test demonstrating a bug with commonancestor()
Specifically, 9 is clearly not in "::8 and ::9".
Differential Revision: https://phab.mercurial-scm.org/D4737
Valentin Gatien-Baron <vgatien-baron@janestreet.com> [Tue, 25 Sep 2018 16:03:14 -0400] rev 38791
revset: fix commonancestor test so it demonstrates correct behavior
The problem is that hg log -r 'head()' is every changeset in the
repository, because in this test repository, every changeset has a
different branch. The author probably assumed all commits were on the
default branch, and that they were getting topological heads, 7 and 9.
As a result, this test was showing that the common ancestors of
0:9 are 0+1+2+4, which is not correct (next commit will test this).
Differential Revision: https://phab.mercurial-scm.org/D4736
Matt Harbison <matt_harbison@yahoo.com> [Tue, 11 Sep 2018 13:52:17 -0400] rev 38790
subrepo: mask out passwords embedded in the messages displaying a URL
I noticed the password in maintenance logs for the "no changes since last push"
and "pushing to" messages when pushing with an explicit path. But the test case
here with :pushurl was also affected. I didn't see that cloning or pulling
subrepos on demand had this problem, but it seems safer to just mask that too.
There's a bit of a disconnect here, because it looks like clone is slicing off
the password (makes sense not to store it in the hgrc in cleartext). But not
shearing it off of an explicit path causes the subrepo not to realize that it
already pushed the latest stuff. This is the easiest fix, however.
Augie Fackler <raf@durin42.com> [Wed, 05 Sep 2018 16:39:47 -0400] rev 38789
buildrpm: remove TODO that looks done to me
Differential Revision: https://phab.mercurial-scm.org/D4494
Augie Fackler <raf@durin42.com> [Wed, 05 Sep 2018 16:29:31 -0400] rev 38788
buildrpm: fix embarassing bug in shell variable expansions
`$distance_$node` looks up and concatenates `$distance_` and
`$node`. `$distance_` is empty, so we were getting the node without
the distance. Using the curly braces makes our intent explicit and
produces better-versioned RPMS.
Differential Revision: https://phab.mercurial-scm.org/D4493
Yuya Nishihara <yuya@tcha.org> [Wed, 05 Sep 2018 21:49:44 +0900] rev 38787
manifest: fix leak on error return from lazymanifest_filtercopy()
Spotted by ASAN.
free(copy->lines) and Py_DECREF(copy->pydata) are replaced by Py_XDECREF(copy),
which should call lazymanifest_dealloc(). Freeing half-initialized copy->lines
is safe since copy->numlines holds a valid value.
Yuya Nishihara <yuya@tcha.org> [Wed, 05 Sep 2018 21:46:53 +0900] rev 38786
manifest: incref/decref copy->pydata to clarify 'copy' holds a reference
Yuya Nishihara <yuya@tcha.org> [Wed, 05 Sep 2018 20:52:22 +0900] rev 38785
dirstate: use tuple interface to fix leak in pack_dirstate()
Spotted by ASAN.
Unlike PyTuple_GET_ITEM(), PySequence_ITEM() returns a new reference. This
bug could be fixed by inserting Py_CLEAR() and Py_XDECREF() appropriately,
but I think requiring a tuple object is simpler and less error-prone.
The cext version is jumped to 10 since 6..9 are used in the default branch.
We'll need to bump it again at merge.
Yuya Nishihara <yuya@tcha.org> [Wed, 05 Sep 2018 22:10:41 +0900] rev 38784
xdiff: fix leak in hunk_consumer()
Spotted by ASAN.
Since PyList_Append() does not "steal" a reference, Py_DECREF() is always
required. Perhaps, this is the largest leak in this series.
Yuya Nishihara <yuya@tcha.org> [Wed, 05 Sep 2018 20:57:38 +0900] rev 38783
base85: fix leak on error return from b85decode()
Spotted by ASAN.
We don't need to initialize 'out' to NULL, but I decided to do that for
clarity.
Augie Fackler <raf@durin42.com> [Tue, 04 Sep 2018 11:59:12 -0400] rev 38782
Added signature for changeset ede3bf31fe63
Augie Fackler <raf@durin42.com> [Tue, 04 Sep 2018 11:59:09 -0400] rev 38781
Added tag 4.7.1 for changeset ede3bf31fe63
Yuya Nishihara <yuya@tcha.org> [Fri, 31 Aug 2018 21:44:24 +0900] rev 38780
hgweb: load revcount + 1 entries to fill nextentry in log page (issue5972)
"revcount + 1" is moved to the call site to make it clearer.
Gregory Szorc <gregory.szorc@gmail.com> [Fri, 24 Aug 2018 18:21:55 -0700] rev 38779
scmutil: avoid quadratic membership testing (issue5969)
tr.changes['revs'] is an xrange, which has an O(n) __contains__
implementation. The `rev not in newrevs` lookup a few lines below
will therefore be O(n^2) if all incoming changesets are public.
This issue isn't present on @ because 45e05d39d9ce introduced
a custom type implementing an xrange primitive with O(1) contains
and switched tr.changes['revs'] to be an instance of that type.
We work around the problem on the stable branch by casting the
xrange to a set. This is a bit hacky because it requires allocating
memory to hold each integer in the range. But we are already
holding the full set of pulled revision numbers in memory
multiple times (such as in `tr.changes['phases']`). So this is
a relatively minor problem.
This issue has been present since the phases reporting code was
introduced in the 4.7 cycle by eb9835014d20.
This change should be reverted/ignored when stable is merged into
default.
On the mozilla-unified repository with 483492 changesets, `hg clone`
time improves substantially:
before: 1843.700s user; 29.810s sys
after: 461.170s user; 29.360s sys
Boris Feld <boris.feld@octobus.net> [Sat, 18 Aug 2018 10:24:57 +0200] rev 38778
phases: drop dead code in `newheads`
There are new code earlier in the function achieving the same result.
Spotted by Yuya Nishihara.
Boris Feld <boris.feld@octobus.net> [Sat, 18 Aug 2018 10:24:16 +0200] rev 38777
phases: fix `nullid` reference in newheads
Spotted by Yuya Nishihara.
Augie Fackler <augie@google.com> [Mon, 20 Aug 2018 16:33:48 -0400] rev 38776
beautifygraph: don't warn about busted terminal if HGPLAIN is set
This was breaking some automation for users that discovered the
extension and turned it on.
Differential Revision: https://phab.mercurial-scm.org/D4338
Yuya Nishihara <yuya@tcha.org> [Sun, 19 Aug 2018 13:27:02 +0900] rev 38775
merge: do not delete untracked files silently (issue5962)
37450a122128, 2a774cae3a03, and 656ac240f392 weren't enough to prevent data
loss. No unknown "files" weren't deleted before 7a8a16f8ea22, "context: also
consider path conflicts when clearing unknown files."
Yuya Nishihara <yuya@tcha.org> [Sat, 18 Aug 2018 10:33:03 +0900] rev 38774
perf: fix typo in perfphasesremote()
Boris Feld <boris.feld@octobus.net> [Fri, 17 Aug 2018 20:35:52 +0200] rev 38773
remotephase: avoid full changelog iteration (issue5964)
Changeset 88efb7d6bcb6 introduced a performance regression by triggering a
full ancestors walk.
This changeset reworks this logic so that we no longer walk down the full
changelog. The motivation for 88efb7d6bcb6, issue5939, is still fixed.
mercurial compared to a draft repository
----------------------------------------
8eeed92475d5: 0.012637 seconds
88efb7d6bcb6: 0.202699 seconds (x16)
46da52f4b820: 0.215551 seconds (+6%)
this code: 0.008397 seconds (-33% from base)
The payload size reduction we see in `test-bookmarks-pushpull.t` comes from a
more aggressive filter of nullid and is harmless.
Boris Feld <boris.feld@octobus.net> [Fri, 17 Aug 2018 16:00:32 -0700] rev 38772
remotephase: fast path newheads computation in simple case (issue5964)
Changeset 88efb7d6bcb6 fixed the logic of `phases.newheads` but greatly
regressed its performance (up to many order of magnitude). The first step to
fix the regression is to exit early when there is no work to do. If there are
no heads to filter or not roots to filter them, we don't have to do any work.
This fixes the regression when talking to an all public changeset. The
performance is even better than before.
pypy, compared to an all public repo
------------------------------------
8eeed92475d5: 0.005758 seconds
88efb7d6bcb6: 0.602517 seconds (x104)
this code: 0.001508 seconds (-74% from base)
mercurial compared to an all public repo
----------------------------------------
8eeed92475d5: 0.000577 seconds
88efb7d6bcb6: 0.185316 seconds (x321)
this code: 0.000150 seconds (-74% from base)
The performance of newheads, when actual computations are required, is fixed
in the next changeset.
Boris Feld <boris.feld@octobus.net> [Fri, 17 Aug 2018 17:51:06 +0200] rev 38771
perf: add a perfphasesremote command
This command measure the time we spend analysing remote phase during push and
pull and display some information relevant to this computation.
The `test-contrib-perf.t` expected output has to be updated but I do need
these module for this perf command.
Boris Feld <boris.feld@octobus.net> [Wed, 15 Aug 2018 14:43:40 +0200] rev 38770
sparse-revlog: fix delta validity computation
When considering the validity of a delta with sparse-revlog, we check the size
of the largest read. To do so, we use some regular logic with the extra delta
information. Some of this logic was not handling this extra delta properly,
confusing it with "nullrev". This confusion with nullrev lead to wrong results
for this computation but preventing a crash.
Changeset 781b2720d2ac on default revealed this error, crashing. This
changeset fixes the logic on stable so that the computation is correct (and
the crash is averted).
The fix is made on stable as this will impact 4.7 clients interacting with
sparse-revlog repositories (eg: created by later version).
Matt Harbison <matt_harbison@yahoo.com> [Tue, 14 Aug 2018 14:00:35 -0400] rev 38769
convert: don't drop missing or corrupt tag entries
Cleaning up the tags file could be a useful feature in some cases, so maybe
there should be a switch for this. However, the default hg -> hg convert tries
to maintain identical hashes (thus convert.hg.saverev is off by default, but is
on by default for other source types). It looks like _rewritesubstate() has a
`continue` in it, and therefore a similar problem.
I ran into this conversion divergence when a coworker "merged" two repositories
by copy/pasting all of the files from the source repo and massaging the code,
and forgetting to revert the .hg* files. That silently emptied the .hgtags file
after the conversion. (This isn't the manifest node bug Yuya has been helping
with- this occurred well after the bzr -> hg conversion and wasn't a merge
commit, which made it extra puzzling. That bug is still an issue.)
Anton Shestakov <av6@dwimlabs.net> [Thu, 09 Aug 2018 13:04:52 +0800] rev 38768
hgweb: catch ParseError that's raised by revset.match()
Some queries, like the demonstrated "first(::)", fail earlier than we call
mfunc(), and that results in a "500 Internal Server Error". To prevent it,
revset.match() also needs to be in a try-except block.
Augie Fackler <augie@google.com> [Fri, 03 Aug 2018 15:09:19 -0400] rev 38767
tests: update test expectations in pre-2.7.9 branch of this test
As far as I can tell this is the only spot that got overlooked. Rough
test methodology (run inside an hg working copy):
docker run --rm -v `pwd`:/hg -it debian:wheezy bash
apt-get update
apt-get install python python-dev build-essential unzip mercurial \
wget libbz2-dev
make testpy-2.7.8
You could try and use the system Python, but it's 2.7.3 and has lots
of issues for hg these days that are not worth fixing.
Differential Revision: https://phab.mercurial-scm.org/D4070
Gregory Szorc <gregory.szorc@gmail.com> [Wed, 01 Aug 2018 12:24:52 -0700] rev 38766
packaging: always execute builddeb from source root
Without this, dockerdeb and the make targets that invoke it may
fail depending on the pwd at the time of invocation.
Augie Fackler <raf@durin42.com> [Wed, 01 Aug 2018 10:20:19 -0400] rev 38765
Added signature for changeset 33ac6a72308a
Augie Fackler <raf@durin42.com> [Wed, 01 Aug 2018 10:20:18 -0400] rev 38764
Added tag 4.7 for changeset 33ac6a72308a
Boris Feld <boris.feld@octobus.net> [Wed, 01 Aug 2018 10:23:48 +0200] rev 38763
revlog: fix descendant deprecated method
Fix the descendant deprecated method introduced earlier in this cycle.
This was caught by Yuya, thank you.
Gregory Szorc <gregory.szorc@gmail.com> [Tue, 31 Jul 2018 13:46:57 -0700] rev 38762
ui: remove commands.status.terse=u from ui.tweakdefaults
commands.status.terse=u can add significant overhead when operating
on large repositories. Using the Firefox repository:
HGRCPATH= hg --time status
time: real 1.340 secs (user 0.960+0.000 sys 0.380+0.000)
HGRCPATH= hg --time --config commands.status.terse=u status
time: real 2.420 secs (user 2.070+0.000 sys 0.360+0.000)
HGRCPATH= hg --time --config extensions.fsmonitor= status
time: real 0.080 secs (user 0.050+0.010 sys 0.040+0.000)
HGRCPATH= ~/src/hg/hg --time --config extensions.fsmonitor= --config commands.status.terse=u status
time: real 2.470 secs (user 2.080+0.000 sys 0.390+0.000)
The performance regression - especially when fsmonitor is being
used - is too much to stomach for the 4.7 release. We've decided
to remove commands.status.terse=u from ui.tweakdefaults until we
can improve its performance, hopefully in the 4.8 cycle.
This commit effectively backs out 6acf41bb8d40.
Pulkit Goyal <pulkit@yandex-team.ru> [Wed, 25 Jul 2018 21:19:06 +0300] rev 38761
context: safegaurd against 'lx' being passed as file flag in manifest
Subversion can have a file as executable link. When using hgsubversion, we will
have both islink and isexec True. This will lead to _flags being set to `lx`.
However, manifest expects flag to be one-byte so it will crash if 'lx' is
passed. Also it's impossible to have an executable link.
This patch will safegaurd us from having 'lx' being a possible value.
This was authored by Ivan Lezhankin from Yandex.
Differential Revision: https://phab.mercurial-scm.org/D3985
Boris Feld <boris.feld@octobus.net> [Tue, 31 Jul 2018 13:53:06 -0700] rev 38760
localrepo: unconditionally enable general delta with sparse revlogs
This come as an extra security, better safe than sorry.
Boris Feld <boris.feld@octobus.net> [Mon, 30 Jul 2018 15:53:47 +0200] rev 38759
sparse-revlog: also use sparse-revlog config as a general delta trigger
Sparse revlog rely on general delta, so we should make sure it is used.
Yuya Nishihara <yuya@tcha.org> [Sun, 29 Jul 2018 16:25:51 +0900] rev 38758
fileset: suppress EACCES while reading arbitrary paths via filectx API
On Windows, EACCES is raised in place of EISDIR. This patch simply adds
EACCES to the list of errors to be ignored since I think it's okay for
filesets to treat inaccessible working-copy files as not existing.
Pulkit Goyal <pulkit@yandex-team.ru> [Fri, 27 Jul 2018 14:56:10 +0300] rev 38757
merge: do the trivial resolution after updating sparse checkout
In merge, we do trivial resolution for files which were deleted on one side and
changed on other. When sparse extension in involved that file might not be
present in wdir and trivial resolution can lead to file not found error. This
patch make sure we updates the sparse checkout before doing the trivial
resolution.
This fixes the test failure demonstrated in previous patch.
Differential Revision: https://phab.mercurial-scm.org/D3984
Pulkit Goyal <pulkit@yandex-team.ru> [Fri, 27 Jul 2018 14:53:28 +0300] rev 38756
sparse: add test showing `hg merge` is broken while using sparse extension
This patch adds a test to demonstrate that `hg merge` is broken in some cases
while using sparse extension.
The case is when you have a file which is:
* modified between current wdir parent and merge base
* deleted between merge base and merge destination
* excluded from sparse checkout
Doing `hg merge` results in file not found error. Next patch will fix this.
Differential Revision: https://phab.mercurial-scm.org/D3983
Martin von Zweigbergk <martinvonz@google.com> [Mon, 30 Jul 2018 14:36:42 -0700] rev 38755
context: use hex nodeid in error about filtered node
I went a little too far in my cleanup in 9231148ea599 (context:
convert to hex for error message only for 20-byte changeid,
2018-04-06). I missed that the case where a binary nodeid refers to a
filtered node.
Differential Revision: https://phab.mercurial-scm.org/D3987
Boris Feld <boris.feld@octobus.net> [Tue, 31 Jul 2018 10:55:14 +0200] rev 38754
doc: fix underline length for config title (issue5949)
This was an error in the initial commit.
Thanks go to Marcel Svitalský for reporting the bug.
Boris Feld <boris.feld@octobus.net> [Mon, 30 Jul 2018 15:36:04 +0200] rev 38753
clone: process 'lookup' return as an arbitrary symbol
In theory, checkout is expected to be a node here because it was returned by
peer.lookup.
In practice, multiple important extensions (like hg-git, hg-subversion) use
peers not backed by a mercurial repository where lookup cannot return a node.
Allowing arbitrary symbols is necessary to make these extensions working with
4.7.
We should probably introduce a new API in Core to have these extensions to
work without abusing the lookup API. In the meantime, a small change to
restore compatibility in 4.7 seems in order.
Augie Fackler <augie@google.com> [Mon, 30 Jul 2018 10:18:29 -0400] rev 38752
tests: use inline Python instead of sed to add trailing whitespace
The sed invocation was failing on OS X and FreeBSD. I'm far too lazy
to diagnose that, so just use some inline Python to fix the build.
Augie Fackler <augie@google.com> [Mon, 30 Jul 2018 09:50:32 -0400] rev 38751
context: add missing b prefix
This fixes
mercurial/context.py:593: SyntaxError: cannot mix bytes and nonbytes literals (context.py, line 593)
in Python 3.
Yuya Nishihara <yuya@tcha.org> [Sat, 28 Jul 2018 16:36:35 +0900] rev 38750
doctest: convert matcher root to native path
Otherwise it wouldn't be caught by a fast path of pathutil.canonpath(), and
fall back to file identity checks.
Yuya Nishihara <yuya@tcha.org> [Mon, 30 Jul 2018 21:26:55 +0900] rev 38749
test-fileset: make con.xml in output conditionally available
Yuya Nishihara <yuya@tcha.org> [Sat, 28 Jul 2018 15:52:03 +0900] rev 38748
test-obsmarker-template: run mkcommit in subshell to isolate envvars
I don't know if it is a feature or a bug, but it appears that environment
variables passed to a shell function persist on /bin/sh.
Yuya Nishihara <yuya@tcha.org> [Sat, 28 Jul 2018 15:49:51 +0900] rev 38747
test-obsmarker-template: add missing HGENCODING=latin-1
It just worked by accident on some Linux sh.
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 26 Jul 2018 10:06:46 -0700] rev 38746
gitweb: add link to graph
error.tmpl wasn't consistent with other templates in gitweb in
that it was missing a "graph" link. This commit makes it consistent.
Cédric Krier <ced@b2ck.com> [Wed, 25 Jul 2018 10:05:24 +0200] rev 38745
hgweb: strip trailing '/' in apppath before appending '/static/' (issue5943)
Differential Revision: https://phab.mercurial-scm.org/D3978
Boris Feld <boris.feld@octobus.net> [Fri, 20 Jul 2018 09:08:20 +0200] rev 38744
config: rename `revlog` section into `storage`
The idea was suggested by Gregory Szorc on IRC. It is more generic and seems
better. It is probably best to rename the section before it ever makes into an
official (non-rc) release.
The only config option currently in this section have been prefixed with
`revlog` to clarify it applies to `revlog` related storage.
Augie Fackler <raf@durin42.com> [Thu, 19 Jul 2018 14:17:26 -0400] rev 38743
Added signature for changeset e90130af47ce
Augie Fackler <raf@durin42.com> [Thu, 19 Jul 2018 14:17:25 -0400] rev 38742
Added tag 4.7rc0 for changeset e90130af47ce
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 19 Jul 2018 11:10:48 -0700] rev 38741
configitems: restore alias for format.aggressivemergedeltas
913ca175c4ae broke BC by dropping support for reading
format.aggressivemergedeltas. Let's restore it.
Differential Revision: https://phab.mercurial-scm.org/D3966
Yuya Nishihara <yuya@tcha.org> [Thu, 19 Jul 2018 21:36:24 +0900] rev 38740
worker: call selector.close() to release polling resources
Augie Fackler <augie@google.com> [Thu, 19 Jul 2018 13:55:54 -0400] rev 38739
release: merge default into stable for 4.7 release freeze
Sushil khanchi <sushilkhanchi97@gmail.com> [Thu, 19 Jul 2018 22:30:37 +0530] rev 38738
histedit: drop --no-backup option
Dropping this option because now we have a better option
than passing --no-backup flag every time, now user can
set a config in hgrc:
[ui]
history-editing-backup = False
This config aims to operate on every history editing command and
it is still work in progress. As yuya suggessted it probably to
late to add full support this config, so making this as
an experimental config.
Differential Revision: https://phab.mercurial-scm.org/D3965
Boris Feld <boris.feld@octobus.net> [Thu, 19 Jul 2018 10:35:29 +0200] rev 38737
aggressivemergedelta: document rename and move to `revlog` section
The config does not follow our naming guideline and "Aggressive" is probably a
word to keep away from users.
The option does not truly fit in the `format` section. It can be turned on and
off for existing repository without much consequence regarding compatibility.
A new `revlog` option is created to control behavior related to revlog writing
and reading. We can see multiple other config options that could be migrated
there.
* format.maxchainlen
* experimental.mmapindexthreshold
* experimental.sparse-read.density-threshold (in an updated form)
* experimental.sparse-read.min-gap-size (in an updated form)
In addition, we can foresee at least a couple of sparse-revlog related option
coming too (to reduce delta chain length and increase snapshot reuse)
These two extra options might fit there too. Unless we want to create a
section dedicated to caches and performance.
* format.chunkcachesize
* format.manifestcachesize
For now, we only migrate `optimize-delta-parent-choice` since it is getting
out of experimental. It is too close to the release to move the other one. In
addition, we still lack proper the prioritization of alias that would help
renaming them without bad consequence for users.
(Not fully happy about the `revlog` name but could not find better).
Boris Feld <boris.feld@octobus.net> [Thu, 19 Jul 2018 10:06:58 +0200] rev 38736
aggressivemergedeltas: rename variable internally
The "aggressivemergedeltas" name is not great. First, it is quite long,
second, we would rather have less "Aggressive" names within the project. We
are about to rename the config option, so it seems the appropriate time to
rename the internal variable.
Boris Feld <boris.feld@octobus.net> [Thu, 19 Jul 2018 09:57:42 +0200] rev 38735
config: document the purpose of the `format` config section
The config section now have a top level documentation to clarify its intend and
usage. In particular, user are now explain when the option are taken in account
and how to convert repository.
There are an handful of experimental options in this section that does not match
its definition. They should be relocated to other section before getting out of
experimental. (see next changeset for one example).
Rodrigo Damazio <rdamazio@google.com> [Wed, 18 Jul 2018 18:36:39 -0700] rev 38734
macosx: fixing macOS version generation after db9d1dd01bf0
With the Python3 change, the string is now something like
version = b'4.6.2+848-88be288e8ac1'
where it was previously just:
version = '4.6.2+848-88be288e8ac1'
Differential Revision: https://phab.mercurial-scm.org/D3964
Sushil khanchi <sushilkhanchi97@gmail.com> [Tue, 10 Jul 2018 17:01:06 +0530] rev 38733
histedit: add history-editing-backup config option
Instead of passing --no-backup option every time you don't
want to store backup, now you can set config option:
[ui]
history-editing-backup = False
This option aims to operate on every history editing command.
Differential Revision: https://phab.mercurial-scm.org/D3901
Gregory Szorc <gregory.szorc@gmail.com> [Wed, 18 Jul 2018 09:49:34 -0700] rev 38732
merge: mark file gets as not thread safe (issue5933)
In default installs, this has the effect of disabling the thread-based
worker on Windows when manifesting files in the working directory. My
measurements have shown that with revlog-based repositories, Mercurial
spends a lot of CPU time in revlog code resolving file data. This ends
up incurring a lot of context switching across threads and slows down
`hg update` operations when going from an empty working directory to
the tip of the repo.
On mozilla-unified (246,351 files) on an i7-6700K (4+4 CPUs):
before: 487s wall
after: 360s wall (equivalent to worker.enabled=false)
cpus=2: 379s wall
Even with only 2 threads, the thread pool is still slower.
The introduction of the thread-based worker (02b36e860e0b) states that
it resulted in a "~50%" speedup for `hg sparse --enable-profile` and
`hg sparse --disable-profile`. This disagrees with my measurement
above. I theorize a few reasons for this:
1) Removal of files from the working directory is I/O - not CPU - bound
and should benefit from a thread pool (unless I/O is insanely fast
and the GIL release is near instantaneous). So tests like `hg sparse
--enable-profile` may exercise deletion throughput and aren't good
benchmarks for worker tasks that are CPU heavy.
2) The patch was authored by someone at Facebook. The results were
likely measured against a repository using remotefilelog. And I
believe that revision retrieval during working directory updates with
remotefilelog will often use a remote store, thus being I/O and not
CPU bound. This probably resulted in an overstated performance gain.
Since there appears to be a need to enable the thread-based worker with
some stores, I've made the flagging of file gets as thread safe
configurable. I've made it experimental because I don't want to formalize
a boolean flag for this option and because this attribute is best
captured against the store implementation. But we don't have a proper
store API for this yet. I'd rather cross this bridge later.
It is possible there are revlog-based repositories that do benefit from
a thread-based worker. I didn't do very comprehensive testing. If there
are, we may want to devise a more proper algorithm for whether to use
the thread-based worker, including possibly config options to limit the
number of threads to use. But until I see evidence that justifies
complexity, simplicity wins.
Differential Revision: https://phab.mercurial-scm.org/D3963
Gregory Szorc <gregory.szorc@gmail.com> [Wed, 18 Jul 2018 09:46:45 -0700] rev 38731
worker: ability to disable thread unsafe tasks
The worker on Windows is implemented using a thread pool. If worker
tasks are not thread safe, badness can occur. In addition, if tasks
are executing CPU bound code and holding onto the GIL, there will be
non-substantial overhead in Python context switching between active
threads. This can result in significant slowdowns of tasks.
This commit teaches the code for determining whether to use a worker
to take thread safety into account. Effectively, thread unsafe tasks
don't use the thread-based worker on Windows.
Differential Revision: https://phab.mercurial-scm.org/D3962
Gregory Szorc <gregory.szorc@gmail.com> [Tue, 17 Jul 2018 16:57:27 -0700] rev 38730
worker: rename variable to reflect constant
Differential Revision: https://phab.mercurial-scm.org/D3961
Danny Hooper <hooper@google.com> [Mon, 16 Jul 2018 17:52:17 -0700] rev 38729
worker: use one pipe per posix worker and select() in parent process
This allows us to pass results larger than PIPE_BUF through the pipes without
interleaving them. This is necessary now because "hg fix" sends file contents
as the result from workers.
Differential Revision: https://phab.mercurial-scm.org/D3960
Joerg Sonnenberger <joerg@bec.de> [Sun, 28 Jan 2018 13:20:52 +0100] rev 38728
tests: refactor common bundle2 capabilities
Differential Revision: https://phab.mercurial-scm.org/D1945
Boris Feld <boris.feld@octobus.net> [Mon, 16 Jul 2018 18:02:30 +0200] rev 38727
debug: move extensions debug behind a dedicated flag
Since b86664c81833, we process the `--debug` flag earlier. This is overall
good and useful, but has at least one negative side effect.
Previously the debug message we report when trying to import extensions were
issued before we processed the `--debug` flag. Now they happen after.
Before:
$ ./hg id --debug
21f507b8de2f9c1606e9aeb5ec7d2a6dedb7a4a7 tip
After:
$ ./hg id --debug ☿ (revset-bench)
could not import hgext.evolve (No module named evolve): trying hgext3rd.evolve
could not import hgext.mercurial_keyring (No module named mercurial_keyring): trying hgext3rd.mercurial_keyring
could not import hgext3rd.mercurial_keyring (No module named mercurial_keyring): trying mercurial_keyring
could not import hgext.hggit (No module named hggit): trying hgext3rd.hggit
could not import hgext3rd.hggit (No module named hggit): trying hggit
21f507b8de2f9c1606e9aeb5ec7d2a6dedb7a4a7 tip
(This get worse if --traceback is used).
To work around this, we move this extensions related debug message behind a
new flag 'devel.debug.extensions' and restore the previous output.
I'm not fully happy about using the 'devel' section for a flag that can be
used by legitimate users to debug extensions issues. However, it fits well
next to other `devel.devel.*` options and is mostly used by extensions author
anyway.
We might move it to another, more appropriate section in the future (using
alias).
Kyle Lippincott <spectral@google.com> [Mon, 16 Jul 2018 11:38:56 -0700] rev 38726
curses: use "text" interface if TERM=dumb
Differential Revision: https://phab.mercurial-scm.org/D3948
Matt Harbison <matt_harbison@yahoo.com> [Mon, 16 Jul 2018 00:32:33 -0400] rev 38725
windows: expand '~/' and '~\' to %USERPROFILE% when translating to cmd.exe
It's convenient to be able to reference hooks in a portable location on any
platform.
Matt Harbison <matt_harbison@yahoo.com> [Sun, 15 Jul 2018 23:58:39 -0400] rev 38724
windows: replace single quote with double quote when translating to cmd.exe
Since cmd.exe doesn't understand single quotes, single quotes to prevent $var
expansion is basically unusable without this. Single quote isn't allowed in a
path name, so it seems unlikely to come up otherwise.