Boris Feld <boris.feld@octobus.net> [Tue, 13 Nov 2018 15:06:29 +0100] rev 40684
doctest: add a `issnapshot` method to _testrevlog
We'll need it soon.
Boris Feld <boris.feld@octobus.net> [Tue, 13 Nov 2018 14:41:04 +0100] rev 40683
tests: add `revlogutils.deltas` module to doctests
The doctest in these module have been from `mercurial.revlog` but the module was
not added to the doctests.
Spotted by Yuya Nishihara.
Yuya Nishihara <yuya@tcha.org> [Thu, 15 Nov 2018 20:20:31 +0900] rev 40682
merge with stable
joco <joco@google.com> [Mon, 05 Nov 2018 22:58:19 +0100] rev 40681
mergetools: adjust Beyond Compare config on Mac/Linux
Set the labels of the Linux and Mac versions of Beyond Compare from
Mercurial's builtin variables, same as the Windows version.
Differential Revision: https://phab.mercurial-scm.org/D5255
Anton Shestakov <av6@dwimlabs.net> [Wed, 14 Nov 2018 15:05:38 +0800] rev 40680
rewriteutil: move publicrevs closer to where it's used
Martin von Zweigbergk <martinvonz@google.com> [Wed, 14 Nov 2018 11:30:46 -0800] rev 40679
requires: use atomictemp=True when writing .hg/requires
We use an unusual file system at Google that allows writes (and
renames) but not deletions (for certain paths). That causes problems
when writing the requires files without atomictemp=True. There doesn't
appear to be any real drawbacks to using atomictemp, so I'm hoping we
can just change it in core.
Differential Revision: https://phab.mercurial-scm.org/D5274
Yuya Nishihara <yuya@tcha.org> [Sun, 11 Nov 2018 16:47:28 +0900] rev 40678
blackbox: extract _log() function which is called after lastui is resolved
This makes sure that self is the solo ui instance used in _log().
Yuya Nishihara <yuya@tcha.org> [Sun, 11 Nov 2018 16:44:30 +0900] rev 40677
blackbox: inline temporary variables which are referenced only once
Yuya Nishihara <yuya@tcha.org> [Sun, 11 Nov 2018 16:43:29 +0900] rev 40676
blackbox: simply update global lastui variable at once
Yuya Nishihara <yuya@tcha.org> [Sun, 11 Nov 2018 16:38:43 +0900] rev 40675
blackbox: consolidate conditions for early return
Just pick the lastui only if it is usable.
Yuya Nishihara <yuya@tcha.org> [Sun, 11 Nov 2018 16:34:49 +0900] rev 40674
blackbox: remove redundant check for unassigned repo
Since ui._bbvfs is looked through ui._bbrepo, the repo instance should
exist if ui._bbvfs isn't None.
Augie Fackler <augie@google.com> [Wed, 14 Nov 2018 10:15:28 -0500] rev 40673
tests: fix bytes/str issue I introduced when adding this test
# skip-blame just b prefixes for py3
Differential Revision: https://phab.mercurial-scm.org/D5271
Kyle Lippincott <spectral@google.com> [Tue, 13 Nov 2018 17:14:47 -0800] rev 40672
shelve: use matcher to restrict prefetch to just the modified files
Shelve currently operates by:
- make a temp commit
- identify all the bases necessary to shelve, put them in the bundle
- use exportfile to export the temp commit to the bundle ('file' here means
"export to this fd", not "export this file")
- remove the temp commit
exportfile calls prefetchfiles, and prefetchfiles uses a matcher to restrict
what files it's going to prefetch; if it's not provided, it's alwaysmatcher.
This means that `hg shelve` in a remotefilelog repo can possibly download the
file contents of everything in the repository, even when it doesn't need to. It
luckily is restricted to the narrowspec (if there is one), but this is still a
lot of downloading that's just unnecessary, especially if there's a "smart"
VCS-aware filesystem involved.
exportfile is called with exactly one revision to emit, so we're just
restricting it to prefetching the files from that revision. The base revisions
having separate files should not be a concern since they're handled already;
example:
commit 10 is draft and modifies foo/a.txt and foo/b.txt
commit 11 is draft and modifies foo/a.txt
my working directory that I'm shelving modifies foo/b.txt
By the time we get to exportfile, commit 10 and 11 are already handled, so the
matcher only specifying foo/b.txt does not cause any problems. I verified this
by doing an `hg unbundle` on the bundle that shelve produces, and getting the
full contents of those commits back out, instead of just the files that were
modified in the shelve.
Differential Revision: https://phab.mercurial-scm.org/D5268
Gregory Szorc <gregory.szorc@gmail.com> [Tue, 13 Nov 2018 12:32:05 -0800] rev 40671
revlog: automatically read from opened file handles
The revlog reading code commonly opens a new file handle for
reading on demand. There is support for passing a file handle
to revlog.revision(). But it is marked as an internal argument.
When revlogs are written, we write() data as it is available. But
we don't flush() data until all revisions are written.
Putting these two traits together, it is possible for an in-process
revlog reader during active writes to trigger the opening of a new
file handle on a file with unflushed writes. The reader won't have
access to all "available" revlog data (as it hasn't been flushed).
And with the introduction of the previous patch, this can lead to
the revlog raising an error due to a partial read.
I witnessed this behavior when applying changegroup data (via
`hg pull`) before issue6006 was fixed via different means. Having
this and the previous patch in play would have helped cause errors
earlier rather than manifesting as hash verification failures.
While this has been a long-standing issue, I believe the relatively
new delta computation code has tickled it into being more common.
This is because the new delta computation code will compute deltas
in more scenarios. This can lead to revlog reading. While the delta
computation code is probably supposed to reuse file handles, it
appears it isn't doing so in all circumstances.
But the issue runs deeper than that. Theoretically, any code can
access revision data during revlog writes. It appears we were just
getting lucky that it wasn't. (The "add revision callback" passed to
addgroup() provides an avenue to do this.)
If I changed the revlog's behavior to not cache the full revision
text or to clear caches after revision insertion during addgroup(),
I was able to produce crashes 100% of the time when writing changelog
revisions. This is because changelog's add revision callback attempts
to resolve the revision data to access the changed files list. And
without the revision's fulltext being cached, we performed a revlog
read, which required opening a new file handle. This attempted to read
unflushed data, leading to a partial read and a crash.
This commit teaches the revlog to store the file handles used for
writing multiple revisions during addgroup(). It also teaches the
code for resolving a file handle when reading to use these handles,
if available. This ensures that *any* reads (regardless of their
source) use the active writing file handles, if available. These
file handles have access to the unflushed data because they wrote it.
This allows reads to complete without issue.
Differential Revision: https://phab.mercurial-scm.org/D5267
Gregory Szorc <gregory.szorc@gmail.com> [Tue, 13 Nov 2018 12:30:59 -0800] rev 40670
revlog: detect incomplete revlog reads
_readsegment() is supposed to return N bytes of revlog revision
data starting at a file offset. Surprisingly, its behavior before
this patch never verified that it actually read and returned N
bytes! Instead, it would perform the read(), then return whatever
data was available. And even more surprisingly, nothing in the
call chain appears to have been validating that it received all
the data it was expecting.
This behavior could lead to partial or incomplete revision chunks
being operated on. This could result in e.g. cached deltas being
applied against incomplete base revisions. The delta application
process would happily perform this operation. Only hash
verification would detect the corruption and save us.
This commit changes the behavior of raw revlog reading to validate
that we actually read() the number of bytes that were requested.
We will raise a more specific error faster, rather than possibly
have it go undetected or manifest later in the call stack, at
delta application or hash verification.
Differential Revision: https://phab.mercurial-scm.org/D5266
Gregory Szorc <gregory.szorc@gmail.com> [Tue, 30 Oct 2018 16:50:05 -0700] rev 40669
revlog: use single file handle when de-inlining revlog
_getsegmentforrevs() will eventually call into _datareadfp() to
resolve a file handle to read revision data. If no file handle
is passed into _getsegmentforrevs(), it opens a new one.
Explicit is better than implicit.
This commit changes _enforceinlinesize() to open a file handle
explicitly when converting inline revlogs to split revlogs and
to pass this file handle into _getsegmentforrevs().
I haven't measured, but this change should improve performance,
as we no longer reopen the revlog for reading for every revision
in the revlog when it is converted from inline to split. Instead,
we open it at most once and use it for the duration of the
operation. That being said, I /think/ the chunk cache may mitigate
the number of file opens required.
Differential Revision: https://phab.mercurial-scm.org/D5265
Pulkit Goyal <pulkit@yandex-team.ru> [Tue, 13 Nov 2018 18:44:09 +0300] rev 40668
store: raise ProgrammingError if unable to decode a storage path
Right now, the function magically return False which is dangerous, so let's
raise ProgrammingError.
Suggested by Augie in D5139.
Differential Revision: https://phab.mercurial-scm.org/D5264
Matt Harbison <matt_harbison@yahoo.com> [Tue, 13 Nov 2018 23:54:23 -0500] rev 40667
tests: document a known failing interaction between narrow and lfs
This is one of the two remaining aborts I found looking into issue5794. I've
got no idea what's wrong with the hook, since the changes there fixed the other
two problems noted in that bug report. It seems like it might go away when the
narrow issue is fixed, but let's make sure this doesn't get lost.
The stacktrace for the hook seems to indicate that the missing file *is* in ctx:
remote: Traceback (most recent call last):
remote: File "c:\Users\Matt\projects\hg\hgext\lfs\__init__.py", line 253, in checkrequireslfs
remote: if any(f in ctx and match(f) and ctx[f].islfs() for f in ctx.files()):
remote: File "c:\Users\Matt\projects\hg\hgext\lfs\__init__.py", line 253, in <genexpr>
remote: if any(f in ctx and match(f) and ctx[f].islfs() for f in ctx.files()):
remote: File "c:\Users\Matt\projects\hg\hgext\lfs\wrapper.py", line 191, in filectxislfs
remote: return _islfs(self.filelog(), self.filenode())
remote: File "c:\Users\Matt\projects\hg\mercurial\context.py", line 631, in filenode
remote: return self._filenode
remote: File "c:\Users\Matt\projects\hg\mercurial\util.py", line 1528, in __get__
remote: result = self.func(obj)
remote: File "c:\Users\Matt\projects\hg\mercurial\context.py", line 579, in _filenode
remote: return self._filelog.lookup(self._fileid)
remote: File "c:\Users\Matt\projects\hg\mercurial\filelog.py", line 68, in lookup
remote: self._revlog.indexfile)
remote: File "c:\Users\Matt\projects\hg\mercurial\utils\storageutil.py", line 218, in fileidlookup
remote: raise error.LookupError(fileid, identifier, _('no match found'))
remote: LookupError: data/inside2/f.i@f59b4e021835: no match found
Yuya Nishihara <yuya@tcha.org> [Sun, 11 Nov 2018 12:55:58 +0900] rev 40666
logtoprocess: drop support for ui.log() call with invalid msg arguments (BC)
Before, the logtoprocess extension put a formatted message into $MSG1, and
its arguments to $MSG2... If the specified arguments couldn't be formatted
because of a caller bug, an unformatted message was passed in to $MSG1
instead of exploding. This behavior doesn't make sense.
Since I'm planning to formalize the ui.log() interface such that we'll no
longer have to extend the ui class, I want to remove any features not
conforming to the ui.log() API. So this patch removes the support for
ill-formed arguments, and $MSG{n} (where n > 1) parameters which seems
useless as long as the message can be formatted. The $MSG1 variable isn't
renamed for the maximum compatibility.
In future patches, a formatted msg will be passed to a processlogger object,
instead of overriding the ui.log() function.
.. bc::
The logtoprocess extension no longer supports invalid ``ui.log()``
arguments. A log message is always formatted and passed in to the
``$MSG1`` environment variable.
Yuya Nishihara <yuya@tcha.org> [Sun, 11 Nov 2018 12:35:38 +0900] rev 40665
py3: byte-stringify inline extension in test-logtoprocess.t
Yuya Nishihara <yuya@tcha.org> [Sun, 11 Nov 2018 12:33:14 +0900] rev 40664
logtoprocess: rewrite dict building in py3-compatible way
Yuya Nishihara <yuya@tcha.org> [Sun, 11 Nov 2018 12:27:23 +0900] rev 40663
logtoprocess: leverage procutil.shellenviron() to stringify variables (BC)
This should make the extension more Py3 friendly. The environment variables
of the main process are copied to the dict by shellenviron().
.. bc::
Boolean options passed to the logtoprocess extension are now formatted
as ``0`` or ``1`` instead of ``None``, ``False``, or ``True``.
Yuya Nishihara <yuya@tcha.org> [Tue, 13 Nov 2018 22:15:30 +0900] rev 40662
templater: check invalid use of list expression properly (issue5920)
The error message is still cryptic, but it should be better.
Pulkit Goyal <pulkit@yandex-team.ru> [Tue, 13 Nov 2018 18:08:55 +0300] rev 40661
py3: add b suffix to make sure file is opened in bytes mode
Differential Revision: https://phab.mercurial-scm.org/D5263
Pulkit Goyal <pulkit@yandex-team.ru> [Tue, 13 Nov 2018 18:08:17 +0300] rev 40660
py3: don't use dict.iterkeys()
dict.iterkeys() is not present on Python 3.
Differential Revision: https://phab.mercurial-scm.org/D5262
Pulkit Goyal <pulkit@yandex-team.ru> [Tue, 13 Nov 2018 18:07:21 +0300] rev 40659
py3: use node.hex(h.digest()) instead of h.hexdigest()
hashlib.sha1.hexdigest() returns str on Python 3.
Differential Revision: https://phab.mercurial-scm.org/D5261
Pulkit Goyal <pulkit@yandex-team.ru> [Tue, 13 Nov 2018 17:41:54 +0300] rev 40658
py3: make second argument of fdopen() a str
# skip-blame as just r'' prefixes
Differential Revision: https://phab.mercurial-scm.org/D5260
Pulkit Goyal <pulkit@yandex-team.ru> [Tue, 13 Nov 2018 17:41:26 +0300] rev 40657
py3: fix keyword arguments handling in hgext/remotefilelog/
Keys of kwargs on Python 3 should be strings. This patch fixes them by appending
r'' prefixes, and using pycompat.byteskwargs() and pycompat.strkwargs().
Differential Revision: https://phab.mercurial-scm.org/D5259
Pulkit Goyal <pulkit@yandex-team.ru> [Sat, 13 Oct 2018 05:09:18 +0300] rev 40656
py3: add __pycache__ to .hgignore
Something changed with Python 3 just like million on things. Maybe they should
named Python 3 as anaconda or cobra instead of just increasing the version
number.
This makes test-custom-filters.t pass on Python 3.
Differential Revision: https://phab.mercurial-scm.org/D5039
Boris Feld <boris.feld@octobus.net> [Thu, 08 Nov 2018 16:07:16 +0100] rev 40655
sparse-revlog: stop using a heap to track selected gap
Same logic as for 'gapsheap', we don't actually need a heap.
Boris Feld <boris.feld@octobus.net> [Thu, 08 Nov 2018 16:01:30 +0100] rev 40654
sparse-revlog: stop using a heap to track gaps
The heap doesn't bring any performance advantage as we can simply sort the
final list.
Moreover, the lesser complexity helps a lot when we later implement it in C.
Boris Feld <boris.feld@octobus.net> [Thu, 08 Nov 2018 15:29:58 +0100] rev 40653
sparse-revlog: fast-path before computing payload size
In this fast-path case, we do not need to compute the full delta chain
payload. Since it comes with a significant cost, we avoid doing so if
possible.
Boris Feld <boris.feld@octobus.net> [Tue, 16 Oct 2018 14:46:28 +0200] rev 40652
sparse-revlog: drop unused deltainfo parameter from segmentspan
We no longer need to deal with slicing logic including uncommitted revision.
We drop the associated code.
Boris Feld <boris.feld@octobus.net> [Tue, 16 Oct 2018 14:43:51 +0200] rev 40651
sparse-revlog: drop unused deltainfo parameter from _slicechunktodensity
We no longer need to deal with slicing logic including uncommitted revision. We
drop the associated code.
Boris Feld <boris.feld@octobus.net> [Mon, 15 Oct 2018 15:45:08 +0200] rev 40650
sparse-revlog: skip the span check in the sparse-revlog case
This significantly improves the performance on unbundling on smaller
repositories.
Mercurial: unbundling 1K revisions
no-sparse-revlog: 500 ms
sparse-revlog-before: 689 ms
sparse-revlog-after: 484 ms
Pypy: unbundling 1K revisions
no-sparse-revlog: 1.242 s
sparse-revlog-before: 1.135 s
sparse-revlog-after: 0.860 s
NetBeans: unbundling 1K revisions
no-sparse-revlog: 1.386 s
sparse-revlog-before: 2.368 s
sparse-revlog-after: 1.191 s
Mozilla: unbundling 1K revisions
no-sparse-revlog: 3.103 s
sparse-revlog-before: 3.367 s
sparse-revlog-after: 3.093 s
Augie Fackler <augie@google.com> [Wed, 17 Oct 2018 17:15:42 -0400] rev 40649
histedit: import chistedit curses UI from hg-experimental
I don't tend to like curses interfaces, but this gets enough use at
work that it seems like it's worth bringing into core. This is a
minimal import from hg-experimental revision 4c7f33bf5f00, in that
I've done the smallest amount of code movement and editing in order to
import the functionality.
.. feature::
`hg histedit` will now present a curses UI if curses is available
and `ui.interface` or `ui.interface.histedit` is set to `curses`.
Differential Revision: https://phab.mercurial-scm.org/D5146
Augie Fackler <augie@google.com> [Mon, 12 Nov 2018 20:32:58 -0500] rev 40648
manifest: perform cheap checks before potentially allocating memory
Differential Revision: https://phab.mercurial-scm.org/D5258
Augie Fackler <augie@google.com> [Mon, 12 Nov 2018 20:31:57 -0500] rev 40647
manifest: also reject obviously-too-short lines when parsing lines
Differential Revision: https://phab.mercurial-scm.org/D5257
Augie Fackler <augie@google.com> [Mon, 12 Nov 2018 20:35:22 -0500] rev 40646
manifest: make sure there's a filename before bothering to look for newline
There's no valid manifest that would have no characters before the NUL byte on
a line, and this fixes some erratic timeouts in the fuzzer.
Differential Revision: https://phab.mercurial-scm.org/D5256
Augie Fackler <augie@google.com> [Tue, 06 Nov 2018 11:12:56 -0500] rev 40645
revlog: replace PyInt_AS_LONG with a more portable helper function
PyInt_AS_LONG disappears on Python, and our previous #define was
producing some problems on Python 3. Let's give up and make an inline
helper function that makes this more sane.
Differential Revision: https://phab.mercurial-scm.org/D5235
Yuya Nishihara <yuya@tcha.org> [Mon, 12 Nov 2018 22:51:36 +0900] rev 40644
help: document weird behavior of uisetup()
While refactoring ui.log() API, it was really annoying that uisetup() is
actually an extsetup() the phase 1. Let's document that. I'm planning to
add another uisetup() which is called per ui instance, though I can't think
of a good name for it.
Yuya Nishihara <yuya@tcha.org> [Mon, 12 Nov 2018 22:26:24 +0900] rev 40643
help: merge section about uisetup() and extsetup()
They are technically the same callback, called only once per process.
The section name "ui setup" is confusing, so shouldn't be used.
Yuya Nishihara <yuya@tcha.org> [Mon, 12 Nov 2018 21:28:54 +0900] rev 40642
help: add internals.extensions topic
I think it's better to include the API overview in core as we now have the
internals section in our help system.
Retrieved from the wiki, and formatted as reST. Several wiki links are
removed since they are invalid in the help. The sections about example
extension are removed at all as they seemed too verbose.
https://www.mercurial-scm.org/wiki/WritingExtensions
Yuya Nishihara <yuya@tcha.org> [Sun, 04 Nov 2018 12:51:14 +0900] rev 40641
commandserver: send raw progress information to message channel
This is pretty basic implementation to support GUI progress bar.
Yuya Nishihara <yuya@tcha.org> [Sun, 04 Nov 2018 12:38:49 +0900] rev 40640
commandserver: make getpass() request distinct from normal prompt
Otherwise, GUI clients would have to parse the prompt text.
Yuya Nishihara <yuya@tcha.org> [Sun, 04 Nov 2018 12:17:20 +0900] rev 40639
commandserver: attach prompt default and choices to message
These attributes are important to provide a GUI prompt to user.
Yuya Nishihara <yuya@tcha.org> [Thu, 08 Nov 2018 22:25:04 +0900] rev 40638
test-commandserver: clean up quoting and location of dbgui extension
This helps embedding '$' in the script.
Yuya Nishihara <yuya@tcha.org> [Sun, 18 Jan 2015 17:55:28 +0900] rev 40637
ui: extract helpers to write message with type or label
This provides a 'type' attribute to command-server clients, which seems
more solid than relying on 'ui.<type>' labels. In future patches,
type='progress' will be added to send raw progress information.
Yuya Nishihara <yuya@tcha.org> [Sun, 18 Jan 2015 18:49:59 +0900] rev 40636
commandserver: add experimental option to use separate message channel
This is loosely based on the idea of the TortoiseHg's pipeui extension,
which attaches ui.label to message text so the command-server client can
capture prompt text, for example.
https://bitbucket.org/tortoisehg/thg/src/4.7.2/tortoisehg/util/pipeui.py
I was thinking that this functionality could be generalized to templating,
but changed mind as doing template stuff would be unnecessarily complex.
It's merely a status message, a simple serialization option should suffice.
Since this slightly changes the command-server protocol, it's gated by a
config knob. If the config is enabled, and if it's supported by the server,
"message-encoding: <name>" is advertised so the client can stop parsing
'o'/'e' channel data and read encoded messages from the 'm' channel. As we
might add new message encodings in future releases, client can specify a list
of encoding names in preferred order.
This patch includes 'cbor' encoding as example. Perhaps, 'json' should be
supported as well.
Yuya Nishihara <yuya@tcha.org> [Wed, 07 Nov 2018 22:37:51 +0900] rev 40635
commandserver: fix reference before assignment error in pipeservice cleanup
Spotted by the next patch.
Yuya Nishihara <yuya@tcha.org> [Sun, 18 Jan 2015 18:43:23 +0900] rev 40634
dispatch: pass around ui.fmsg channel
This will be set by the command server. See the next patch.
Yuya Nishihara <yuya@tcha.org> [Sat, 10 Nov 2018 12:01:21 +0900] rev 40633
test-subrepo: document why share magically works even if subrepos are missing
I was confused how it's working while reviewing fb490d798be0, "share: reload
repo after adjusting it in postshare()."
Augie Fackler <raf@durin42.com> [Sun, 11 Nov 2018 14:13:08 -0500] rev 40632
packaging: add Fedora 29 target
Differential Revision: https://phab.mercurial-scm.org/D5254
Yuya Nishihara <yuya@tcha.org> [Sat, 10 Nov 2018 11:52:32 +0900] rev 40631
store: pass in decoded filename to narrow matcher
Yuya Nishihara <yuya@tcha.org> [Sat, 10 Nov 2018 11:47:36 +0900] rev 40630
test-narrow-clone-stream: include no usefncache case which is broken
Since 9aeb9e2d28a7, encoded filenames are filtered by the narrow matcher,
which is clearly wrong.
Yuya Nishihara <yuya@tcha.org> [Sat, 10 Nov 2018 11:36:22 +0900] rev 40629
test-narrow-clone-stream: include uppercase letter in filename
This makes encoded filenames differ from the original names.
Matt Harbison <matt_harbison@yahoo.com> [Sat, 10 Nov 2018 13:37:18 -0500] rev 40628
phase: use the `phases.cmdphasenames` constant to walk available command flags
This reverts 1ea6772fb415.
Matt Harbison <matt_harbison@yahoo.com> [Fri, 09 Nov 2018 23:19:53 -0500] rev 40627
phases: add a list of names usable by the phase command
This is useful in TortoiseHg for example, which has until recently just added
all names in the `phasenames` list to the context menu that changes the phase.
The ones not supported by the command would just error out.
Boris Feld <boris.feld@octobus.net> [Wed, 07 Nov 2018 19:59:38 +0100] rev 40626
transaction: display data about why the transaction failed to rollback
We saw more of these a while back. Having more data available would be nice.
Yuya Nishihara <yuya@tcha.org> [Sat, 10 Nov 2018 12:07:26 +0900] rev 40625
merge with stable
Martin von Zweigbergk <martinvonz@google.com> [Fri, 09 Nov 2018 10:47:24 -0800] rev 40624
remotefilelog: avoid accessing repo instance after dispatch
Upstream commit c5e6c1ba1c79 (hg: don't reuse repo instance after
unshare(), 2018-09-12) poisoned the repo instance after
unshare(). That made `hg unshare` fail with remotefilelog because we
tried to close the fileserverclient after dispatch by accessing it via
the repo. This patch fixes that by storing the reference to the
fileserverclient at the beginning of dispatch.
An analogous patch was sent for remotefilelog version in FB's
hg-experimental as D5246.
Differential Revision: https://phab.mercurial-scm.org/D5253
Martin von Zweigbergk <martinvonz@google.com> [Fri, 09 Nov 2018 11:28:05 -0800] rev 40623
share: avoid a now-unnecessary reload of the repo
hg.share() now returns an already-reloaded repo, so let's just use
that. (This would have failed test-subrepo-recursion.t without the
previous patch.)
Differential Revision: https://phab.mercurial-scm.org/D5252
Martin von Zweigbergk <martinvonz@google.com> [Fri, 09 Nov 2018 10:46:02 -0800] rev 40622
share: reload repo after adjusting it in postshare()
When sharing a repo that's using remotefilelog, the update that happens
at the end of the `hg share` call does not see the remote repo path
that's copied in hg.postshare(). This patch reloads the repo after
hg.postshare() to address that.
This changes a subrepo test case. Note that `hg share -U; hg co tip`
worked there before, so I don't see see why `hg share` should fail. I
also don't know what a "locally referenced subrepo". So maybe this is
fixing a bug? Hopefully it's not breaking something someone actually
cares about at least. Maybe someone who knows and cares about subrepos
can review this.
Differential Revision: https://phab.mercurial-scm.org/D5251
Martin von Zweigbergk <martinvonz@google.com> [Fri, 09 Nov 2018 09:36:16 -0800] rev 40621
tests: don't load strip extension via mq extension for remotefilelog tests
The proper way to get the `hg strip` command has been via the "strip"
extension since 2013.
Differential Revision: https://phab.mercurial-scm.org/D5250
Martin von Zweigbergk <martinvonz@google.com> [Fri, 09 Nov 2018 09:35:05 -0800] rev 40620
remotefilelog: remove unused configs for {data,history}pack version
Differential Revision: https://phab.mercurial-scm.org/D5249
Martin von Zweigbergk <martinvonz@google.com> [Fri, 09 Nov 2018 09:27:37 -0800] rev 40619
tests: remove obsolete PYTHONPATH override from remotefilelog tests
They were added in
https://bitbucket.org/facebook/hg-experimental/commits/fdcad37a6a68cfbaf5920f3eeaa0cc9cae42bd26,
which says this:
When remotefilelog moved from its own repo, the tests needed to be updated to
adjust the PYTHONPATH to ensure the in-repo remotefilelog was loaded instead of
the system one.
This meant any local runs of remotefilelog tests would've been using the system
remotefilelog unless the user had manually set the PYTHONPATH themselves.
That doesn't seem relevant with remotefilelog in core.
Differential Revision: https://phab.mercurial-scm.org/D5248
Martin von Zweigbergk <martinvonz@google.com> [Fri, 09 Nov 2018 08:19:08 -0800] rev 40618
tests: drop obsolete "hginit" alias in remotefilelog tests
Differential Revision: https://phab.mercurial-scm.org/D5247
Danny Hooper <hooper@google.com> [Thu, 08 Nov 2018 12:35:26 -0800] rev 40617
fix: add extra field to fixed revisions to avoid creating obsolescence cycles
The extra field prevents sequential invocations of fix from producing the same
hash twice. Previously, this could cause problems because it would create an
obsolescence cycle instead of the expected new successor.
This change also adds an explicit check for whether a new revision should be
committed. Until now, the code relied on memctx.commit() to quietly do nothing
if the node already exists. Because of the new extra field, this no longer
covers the case where we don't want to replace an unchanged node.
Differential Revision: https://phab.mercurial-scm.org/D5245
Danny Hooper <hooper@google.com> [Thu, 08 Nov 2018 12:29:56 -0800] rev 40616
cleanup: use revision numbers instead of hashes in test output
This allows changes to the hashes produced by fix to not needlessly modify this
area of the test.
Differential Revision: https://phab.mercurial-scm.org/D5244
Kyle Lippincott <spectral@google.com> [Wed, 07 Nov 2018 15:42:56 -0800] rev 40615
tests: make the commands.resolve.confirm tests not pollute the test-wide hgrc
I want to add tests after these tests that create a new repo and use it, and was
confused for a bit as to why they were seeing different behavior than I
expected.
Differential Revision: https://phab.mercurial-scm.org/D5242
Kyle Lippincott <spectral@google.com> [Wed, 07 Nov 2018 15:41:18 -0800] rev 40614
tests: fix a couple typos in test-resolve.t comments and add a comment
Differential Revision: https://phab.mercurial-scm.org/D5241
Danny Hooper <hooper@google.com> [Tue, 06 Nov 2018 15:50:41 -0800] rev 40613
fix: add suboption for configuring execution order of tools
This could be accomplished by using wrapper scripts, but that would diminish
the usefulness of the incremental formatting logic. Configuring execution order
along with other things in the hgrc is probably more convenient anyway.
This change highlights some awkwardness with suboptions and default values,
which should be addressed separately.
Differential Revision: https://phab.mercurial-scm.org/D5237
Augie Fackler <augie@google.com> [Wed, 07 Nov 2018 14:21:39 -0500] rev 40612
tests: fix up some import statements caught by Python 3
I'm curious how the import checker manages to be so much more
pedantic in Python 3, but not enough to bother exploring.
Differential Revision: https://phab.mercurial-scm.org/D5240
Martin von Zweigbergk <martinvonz@google.com> [Wed, 07 Nov 2018 10:29:38 -0800] rev 40611
tests: replace `tac` reimplementation by `sort -r`
`sort -r` is better code "formatter" than `tac` since it's
stable. It's also portable so we don't need to reimplement it in
Python.
Differential Revision: https://phab.mercurial-scm.org/D5239
Augie Fackler <augie@google.com> [Wed, 07 Nov 2018 12:15:36 -0500] rev 40610
tests: work around `tac` not being portable
Introduce a tac.py helper and use it. Sigh.
Differential Revision: https://phab.mercurial-scm.org/D5238
Augie Fackler <augie@google.com> [Tue, 06 Nov 2018 11:22:16 -0500] rev 40609
revlog: give formatting to clang-format
This has two lines that are poorly formatted, both right after
PyObject_HEAD macros. It's possible that in the future there will be a
feature to tell clang-format that a token includes an end-of-line
character (in this case the ;), but for now I'm willing to live with
two poorly-formatted lines in a struct definition in exchange for not
thinking about how this file is formatted.
Differential Revision: https://phab.mercurial-scm.org/D5234
Augie Fackler <augie@google.com> [Tue, 06 Nov 2018 11:19:35 -0500] rev 40608
revlog: add blank line in comment to help clang-format
Differential Revision: https://phab.mercurial-scm.org/D5233
Augie Fackler <augie@google.com> [Tue, 06 Nov 2018 11:18:12 -0500] rev 40607
revlog: add a comment to help clang-format produce less-awful results
Differential Revision: https://phab.mercurial-scm.org/D5232
Augie Fackler <augie@google.com> [Tue, 06 Nov 2018 11:52:41 -0500] rev 40606
py3: ratchet caught two more passing tests
Differential Revision: https://phab.mercurial-scm.org/D5236
Boris Feld <boris.feld@octobus.net> [Mon, 05 Nov 2018 15:57:31 +0100] rev 40605
perf: add a lazydeltabase option to perfrevlogwrite
The option controls if we'll test the provided delta first. This is the current
default behavior for unbundle.
Boris Feld <boris.feld@octobus.net> [Mon, 05 Nov 2018 15:24:09 +0100] rev 40604
perf: add `storage` as possible source for perfrevlogwrite
This source will use the stored delta.
Boris Feld <boris.feld@octobus.net> [Mon, 05 Nov 2018 15:19:44 +0100] rev 40603
perf: add `parent-smallest` as possible source for perfrevlogwrite
This source will use the smallest of the possible diff against parent.
Boris Feld <boris.feld@octobus.net> [Mon, 05 Nov 2018 15:15:18 +0100] rev 40602
perf: add `parent-2` as possible source for perfrevlogwrite
This source will use a diff against p2 if it exists and fall back to p1
otherwise.
Boris Feld <boris.feld@octobus.net> [Mon, 05 Nov 2018 15:15:02 +0100] rev 40601
perf: add `parent-1` as possible source for perfrevlogwrite
This source will use a diff against p1 in all case.
Boris Feld <boris.feld@octobus.net> [Fri, 19 Oct 2018 17:23:29 +0200] rev 40600
perf: add the notion of "source" to perfrevlogwrite
We want to test performance associated witch various way to add a new revision.
They will be specified using this new argument.
Boris Feld <boris.feld@octobus.net> [Tue, 06 Nov 2018 00:57:34 +0100] rev 40599
perf: only display the total time for perfrevlogwrite if quiet
This provide a simple way to get an overview of the total performance.
Boris Feld <boris.feld@octobus.net> [Wed, 03 Oct 2018 11:04:57 +0200] rev 40598
perf: offer full details in perfrevlogwrite
This will be useful for people who want to study the timing pattern more
closely.
Boris Feld <boris.feld@octobus.net> [Wed, 03 Oct 2018 10:53:29 +0200] rev 40597
perf: introduce a perfrevlogwrite command
The command record times taken by adding many revisions to a revlog. Timing
each addition, individually. The "added revision" are recreations of the
original ones.
To time each addition individually, we have to handle the timing and the
reporting ourselves.
This command is introduced to track the impact of sparse-revlog format on
delta computations at initial storage time. It starts with the full text, a
situation similar to the "commit". Additions from an existing delta are better
timed with bundles.
The complaints from `check-perf-code.py` are not relevant. We are accessing
and "revlog" opener, not a repository opener.
Augie Fackler <augie@google.com> [Tue, 06 Nov 2018 10:41:00 -0500] rev 40596
tests: fix config knob in test-narrow-clone-stream.t
Two patches landed in parallel and had a semantic conflict. This
resolves the mess and leaves us with passing tests.
Differential Revision: https://phab.mercurial-scm.org/D5231
Augie Fackler <augie@google.com> [Tue, 06 Nov 2018 10:26:33 -0500] rev 40595
remotefilelog: fix various whitespace issues in docstring
Differential Revision: https://phab.mercurial-scm.org/D5230
Yuya Nishihara <yuya@tcha.org> [Sat, 03 Nov 2018 19:42:50 +0900] rev 40594
ui: add config knob to redirect status messages to stderr (API)
This option can be used to isolate structured output from status messages.
For now, "stdio" (stdout/err pair) and "stderr" are supported. In future
patches, I'll add the "channel" option which will send status messages to
a separate command-server channel with some metadata attached, maybe in
CBOR encoding.
This is a part of the generic templating plan:
https://www.mercurial-scm.org/wiki/GenericTemplatingPlan#Sanity_check_output
.. api::
Status messages may be sent to a dedicated stream depending on
configuration. Don't use ``ui.status()``, etc. as a shorthand for
conditional writes. Use ``ui.write()`` for data output.
Yuya Nishihara <yuya@tcha.org> [Sat, 03 Nov 2018 20:53:31 +0900] rev 40593
ui: hide fin/fout/ferr attributes behind @property functions
This allows keeping references to fout/ferr/fin which are updated when these
properties are changed. See the next patch.
Yuya Nishihara <yuya@tcha.org> [Sun, 18 Jan 2015 17:42:53 +0900] rev 40592
ui: label prompt and echo messages
I'm going to add a dedicated command-server channel for status messages,
which carries metadata alongside a message text. 'ui.*' label provides a
hint how message text should be processed.
Yuya Nishihara <yuya@tcha.org> [Sat, 03 Nov 2018 18:17:30 +0900] rev 40591
ui: add inner function to select write destination
I'm going to add a config knob to redirect any status messages to stderr.
This function helps to switch underlying file objects.
# no-check-commit because of existing write_err() function
Yuya Nishihara <yuya@tcha.org> [Sat, 03 Nov 2018 18:04:22 +0900] rev 40590
ui: remove _write() and _write_err() functions
Yuya Nishihara <yuya@tcha.org> [Sat, 03 Nov 2018 18:03:09 +0900] rev 40589
ui: move pre/post processes from low-level write()s to _writenobuf()
This helps adding a dedicated stream for status/error messages. I don't
want to add _write*() function per stream.
Yuya Nishihara <yuya@tcha.org> [Sat, 03 Nov 2018 17:56:17 +0900] rev 40588
ui: pass in file object to _writenobuf()
See the subsequent patches for why. The "if" block in _writenobuf() will
be removed soon.
Boris Feld <boris.feld@octobus.net> [Mon, 05 Nov 2018 17:24:39 +0100] rev 40587
perf: fix perfrevlogrevisions --reverse
Currently, 'endrev' equals `len(revlog)`, a revision that does not exist.
When asking for the reverse order, the arguments passed to xrange are
`xrange(len(revlog), startrev)` which then crash.
We need to offset 'endrev' by one so we don't crash anymore. Also, we offset
'startrev' to ensure we get the same number of revisions with and without the
`--reverse` option.
Differential Revision: https://phab.mercurial-scm.org/D5228
Boris Feld <boris.feld@octobus.net> [Tue, 06 Nov 2018 11:54:15 +0100] rev 40586
procutil: import concerns about creationflags on Windows from D1701
I don't have the need anymore for the change in D1701 nor the time to
investigate the changes on all supported Windows platforms.
I import the stuff I learned on D1701 in the `runbgcommand` so the next people
working on it can starts from there.
Differential Revision: https://phab.mercurial-scm.org/D5229
Gregory Szorc <gregory.szorc@gmail.com> [Mon, 05 Nov 2018 14:14:32 -0800] rev 40585
localrepo: extract loading of hgrc files to standalone function
Various 3rd party extensions supplement where per-repo config data
lives. Looking at their sources, they resort to unorthodox means to
inject the config data. And the way they do it is susceptible to
corner cases. e.g. not processing automatic extension loads,
not reacting to new or disabled extensions in configs, etc.
This commit extracts the core logic of loading hgrc files into
a standalone function so there is a clear function that can be
monkeypatched to inject per-repo config data at repository open
time.
Differential Revision: https://phab.mercurial-scm.org/D5221
Martin von Zweigbergk <martinvonz@google.com> [Mon, 05 Nov 2018 09:09:48 -0800] rev 40584
revsets: make bookmark/named('re:nonexistent') not abort (issue6018) (BC)
Foozy documented the differences between revsets branch(), tag(),
bookmark(), and named() in eeb5d5ab14a6 (revset: raise RepoLookupError
to make present() predicate continue the query, 2015-01-31). He seemed
to want tag() to change behavior to not error out on non-matching
regular expressions. I think it's instead bookmark() and named() that
should not error out. So that's what this patch does.
Differential Revision: https://phab.mercurial-scm.org/D5220
Danny Hooper <hooper@google.com> [Mon, 05 Nov 2018 16:05:45 -0800] rev 40583
fix: rename :fileset subconfig to :pattern
This name was always inaccurate, since the config accepts any pattern.
Hopefully so few people use this right now that it won't matter, but there will
now be a warning if the old config name is used.
Differential Revision: https://phab.mercurial-scm.org/D5226
Danny Hooper <hooper@google.com> [Wed, 31 Oct 2018 13:11:51 -0700] rev 40582
fix: add a config to abort when a fixer tool fails
This allows users to stop and address tool failures before proceeding, instead
of the default behavior of continuing to apply any tools that didn't fail. For
example, a code formatting tool could fail if you have syntax errors, and you
might want your repo to stay in its current state while you fix the syntax
error before re-running 'hg fix'. It's conceivable that this would even be
necessary for the correctness of some fixer tools across a chain of revisions.
Differential Revision: https://phab.mercurial-scm.org/D5200
Boris Feld <boris.feld@octobus.net> [Tue, 06 Nov 2018 11:05:13 +0100] rev 40581
perf: measure slicing time in perfrevlogrevision
Slicing a sparse delta chain can be expensive. We now benchmark the associated
time.
Boris Feld <boris.feld@octobus.net> [Tue, 06 Nov 2018 11:04:23 +0100] rev 40580
perf: teach perfrevlogrevision about sparse reading
Before this change, chunks were always read in a single block. Even in the
sparse-read/sparse-revlog case. This gave a false view of the performance and
could lead to memory consumption issue.
Boris Feld <boris.feld@octobus.net> [Tue, 06 Nov 2018 11:13:31 +0100] rev 40579
perf: use the same timer for all section of perfrevlogrevision
Otherwise the -T json output is invalid.
Boris Feld <boris.feld@octobus.net> [Tue, 22 May 2018 15:26:17 +0200] rev 40578
obsutil: clarify the access to "repo"
We use the variable multiple times and we might use it even more in the
future. We use a temporary variable instead.
rdamazio@google.com [Mon, 05 Nov 2018 19:52:42 -0800] rev 40577
pycompat: adding Linux detection and fixing Mac
Python 3 recommends detecting OSs with the prefix of the platform, but we were
comparing the full string for macOS. We also didn't have Linux detection, which
is convenient for extensions to use (rather than have some OSs detected by hg
and some by the extension).
Reference:
https://docs.python.org/3/library/sys.html#sys.platform
Differential Revision: https://phab.mercurial-scm.org/D5227
Augie Fackler <augie@google.com> [Mon, 05 Nov 2018 17:48:23 -0500] rev 40576
remotefilelog: fix various minor py3 problems
# skip-blame b prefixes and pycompat.long, nothing remotely interesting
Differential Revision: https://phab.mercurial-scm.org/D5223
Augie Fackler <augie@google.com> [Mon, 05 Nov 2018 17:37:37 -0500] rev 40575
remotefilelog: rip out a missed mention of lz4
Differential Revision: https://phab.mercurial-scm.org/D5222
Yuya Nishihara <yuya@tcha.org> [Sat, 03 Nov 2018 17:47:23 +0900] rev 40574
ui: wrap whole _write() block with timeblockedsection
I think the cost of color labeling is negligible compared to the I/O
syscalls. Let's simply wrap the whole write() function so that we can
eliminate _write() and _write_err() in later changeset.
Yuya Nishihara <yuya@tcha.org> [Sat, 03 Nov 2018 17:43:57 +0900] rev 40573
ui: indent _writenobuf() to prepare moving bits from _write() functions
Yuya Nishihara <yuya@tcha.org> [Sat, 03 Nov 2018 17:42:05 +0900] rev 40572
ui: simply concatenate messages before applying color labels
This should be cheaper in space than applying labels for each message.
Yuya Nishihara <yuya@tcha.org> [Sat, 03 Nov 2018 17:36:10 +0900] rev 40571
ui: simplify interface of low-level write() functions
_write() and _write_err() will be replaced with fout.write() and ferr.write()
respectively. This is the first step.
Yuya Nishihara <yuya@tcha.org> [Sat, 03 Nov 2018 17:32:35 +0900] rev 40570
ui: factor out function that writes data to fout/ferr with labeling
I'm thinking of adding an option to send status messages to stderr (or a
dedicated command-server channel) so that structured output (e.g. JSON)
would never be interleaved with non-formatter output. A unified write()
interface helps to do that.
Yuya Nishihara <yuya@tcha.org> [Sat, 03 Nov 2018 16:04:16 +0900] rev 40569
ui: consolidate places where _progclear() is called
The progress bar has to be cleared when we start writing some data to the
output stream. Let's make it always triggered immediately before switching
by _colormode, so that we can easily factor out helper functions.
Yuya Nishihara <yuya@tcha.org> [Sun, 04 Nov 2018 20:44:26 +0900] rev 40568
templater: compute revset lazily
This speeds up e.g. "{ifcontains(rev, revset('::.'), ...)}" in common cases
where 'rev' is near the working parent.
The templater API is ugly, but it helps here. 'f' can be either a generator
or a function returning a generator.
Pulkit Goyal <pulkit@yandex-team.ru> [Wed, 24 Oct 2018 18:48:43 +0300] rev 40567
remotefilelog: drop compat code for "getbundle_shallow" wireprotocol command
Doing some annotate on hgexperimental shows that getbundle_shallow used to exist
in 2013 or before. We don't have any pre-2013 remotefilelog users except Fb
themselves and I doubt they are going to use in-core remotefilelog. So it's safe
to remove this.
Differential Revision: https://phab.mercurial-scm.org/D5193
Pulkit Goyal <pulkit@yandex-team.ru> [Wed, 24 Oct 2018 17:54:34 +0300] rev 40566
tests: remove lz4 as dependency while running tests
One of the previous patch authored by Augie rips out the lz4 dependency and
things should work without it.
Now there are just 2-3 tests failing because of same change in emitrevisions()
API.
Differential Revision: https://phab.mercurial-scm.org/D5192
Pulkit Goyal <pulkit@yandex-team.ru> [Wed, 24 Oct 2018 17:51:49 +0300] rev 40565
remotefilelogserver: add a matcher argument to _walkstreamfiles()
Implementing narrow stream clones in core, I added an optional matcher argument.
The function in remotefilelogserver.py does not know about that argument and
does not accept that and hence some tests fails.
Differential Revision: https://phab.mercurial-scm.org/D5191