Matt Harbison <matt_harbison@yahoo.com> [Fri, 09 Nov 2018 23:19:53 -0500] rev 40627
phases: add a list of names usable by the phase command
This is useful in TortoiseHg for example, which has until recently just added
all names in the `phasenames` list to the context menu that changes the phase.
The ones not supported by the command would just error out.
Boris Feld <boris.feld@octobus.net> [Wed, 07 Nov 2018 19:59:38 +0100] rev 40626
transaction: display data about why the transaction failed to rollback
We saw more of these a while back. Having more data available would be nice.
Yuya Nishihara <yuya@tcha.org> [Sat, 10 Nov 2018 12:07:26 +0900] rev 40625
merge with stable
Martin von Zweigbergk <martinvonz@google.com> [Fri, 09 Nov 2018 10:47:24 -0800] rev 40624
remotefilelog: avoid accessing repo instance after dispatch
Upstream commit c5e6c1ba1c79 (hg: don't reuse repo instance after
unshare(), 2018-09-12) poisoned the repo instance after
unshare(). That made `hg unshare` fail with remotefilelog because we
tried to close the fileserverclient after dispatch by accessing it via
the repo. This patch fixes that by storing the reference to the
fileserverclient at the beginning of dispatch.
An analogous patch was sent for remotefilelog version in FB's
hg-experimental as D5246.
Differential Revision: https://phab.mercurial-scm.org/D5253
Martin von Zweigbergk <martinvonz@google.com> [Fri, 09 Nov 2018 11:28:05 -0800] rev 40623
share: avoid a now-unnecessary reload of the repo
hg.share() now returns an already-reloaded repo, so let's just use
that. (This would have failed test-subrepo-recursion.t without the
previous patch.)
Differential Revision: https://phab.mercurial-scm.org/D5252
Martin von Zweigbergk <martinvonz@google.com> [Fri, 09 Nov 2018 10:46:02 -0800] rev 40622
share: reload repo after adjusting it in postshare()
When sharing a repo that's using remotefilelog, the update that happens
at the end of the `hg share` call does not see the remote repo path
that's copied in hg.postshare(). This patch reloads the repo after
hg.postshare() to address that.
This changes a subrepo test case. Note that `hg share -U; hg co tip`
worked there before, so I don't see see why `hg share` should fail. I
also don't know what a "locally referenced subrepo". So maybe this is
fixing a bug? Hopefully it's not breaking something someone actually
cares about at least. Maybe someone who knows and cares about subrepos
can review this.
Differential Revision: https://phab.mercurial-scm.org/D5251
Martin von Zweigbergk <martinvonz@google.com> [Fri, 09 Nov 2018 09:36:16 -0800] rev 40621
tests: don't load strip extension via mq extension for remotefilelog tests
The proper way to get the `hg strip` command has been via the "strip"
extension since 2013.
Differential Revision: https://phab.mercurial-scm.org/D5250
Martin von Zweigbergk <martinvonz@google.com> [Fri, 09 Nov 2018 09:35:05 -0800] rev 40620
remotefilelog: remove unused configs for {data,history}pack version
Differential Revision: https://phab.mercurial-scm.org/D5249
Martin von Zweigbergk <martinvonz@google.com> [Fri, 09 Nov 2018 09:27:37 -0800] rev 40619
tests: remove obsolete PYTHONPATH override from remotefilelog tests
They were added in
https://bitbucket.org/facebook/hg-experimental/commits/fdcad37a6a68cfbaf5920f3eeaa0cc9cae42bd26,
which says this:
When remotefilelog moved from its own repo, the tests needed to be updated to
adjust the PYTHONPATH to ensure the in-repo remotefilelog was loaded instead of
the system one.
This meant any local runs of remotefilelog tests would've been using the system
remotefilelog unless the user had manually set the PYTHONPATH themselves.
That doesn't seem relevant with remotefilelog in core.
Differential Revision: https://phab.mercurial-scm.org/D5248
Martin von Zweigbergk <martinvonz@google.com> [Fri, 09 Nov 2018 08:19:08 -0800] rev 40618
tests: drop obsolete "hginit" alias in remotefilelog tests
Differential Revision: https://phab.mercurial-scm.org/D5247
Danny Hooper <hooper@google.com> [Thu, 08 Nov 2018 12:35:26 -0800] rev 40617
fix: add extra field to fixed revisions to avoid creating obsolescence cycles
The extra field prevents sequential invocations of fix from producing the same
hash twice. Previously, this could cause problems because it would create an
obsolescence cycle instead of the expected new successor.
This change also adds an explicit check for whether a new revision should be
committed. Until now, the code relied on memctx.commit() to quietly do nothing
if the node already exists. Because of the new extra field, this no longer
covers the case where we don't want to replace an unchanged node.
Differential Revision: https://phab.mercurial-scm.org/D5245
Danny Hooper <hooper@google.com> [Thu, 08 Nov 2018 12:29:56 -0800] rev 40616
cleanup: use revision numbers instead of hashes in test output
This allows changes to the hashes produced by fix to not needlessly modify this
area of the test.
Differential Revision: https://phab.mercurial-scm.org/D5244
Kyle Lippincott <spectral@google.com> [Wed, 07 Nov 2018 15:42:56 -0800] rev 40615
tests: make the commands.resolve.confirm tests not pollute the test-wide hgrc
I want to add tests after these tests that create a new repo and use it, and was
confused for a bit as to why they were seeing different behavior than I
expected.
Differential Revision: https://phab.mercurial-scm.org/D5242
Kyle Lippincott <spectral@google.com> [Wed, 07 Nov 2018 15:41:18 -0800] rev 40614
tests: fix a couple typos in test-resolve.t comments and add a comment
Differential Revision: https://phab.mercurial-scm.org/D5241
Danny Hooper <hooper@google.com> [Tue, 06 Nov 2018 15:50:41 -0800] rev 40613
fix: add suboption for configuring execution order of tools
This could be accomplished by using wrapper scripts, but that would diminish
the usefulness of the incremental formatting logic. Configuring execution order
along with other things in the hgrc is probably more convenient anyway.
This change highlights some awkwardness with suboptions and default values,
which should be addressed separately.
Differential Revision: https://phab.mercurial-scm.org/D5237
Augie Fackler <augie@google.com> [Wed, 07 Nov 2018 14:21:39 -0500] rev 40612
tests: fix up some import statements caught by Python 3
I'm curious how the import checker manages to be so much more
pedantic in Python 3, but not enough to bother exploring.
Differential Revision: https://phab.mercurial-scm.org/D5240
Martin von Zweigbergk <martinvonz@google.com> [Wed, 07 Nov 2018 10:29:38 -0800] rev 40611
tests: replace `tac` reimplementation by `sort -r`
`sort -r` is better code "formatter" than `tac` since it's
stable. It's also portable so we don't need to reimplement it in
Python.
Differential Revision: https://phab.mercurial-scm.org/D5239
Augie Fackler <augie@google.com> [Wed, 07 Nov 2018 12:15:36 -0500] rev 40610
tests: work around `tac` not being portable
Introduce a tac.py helper and use it. Sigh.
Differential Revision: https://phab.mercurial-scm.org/D5238
Augie Fackler <augie@google.com> [Tue, 06 Nov 2018 11:22:16 -0500] rev 40609
revlog: give formatting to clang-format
This has two lines that are poorly formatted, both right after
PyObject_HEAD macros. It's possible that in the future there will be a
feature to tell clang-format that a token includes an end-of-line
character (in this case the ;), but for now I'm willing to live with
two poorly-formatted lines in a struct definition in exchange for not
thinking about how this file is formatted.
Differential Revision: https://phab.mercurial-scm.org/D5234
Augie Fackler <augie@google.com> [Tue, 06 Nov 2018 11:19:35 -0500] rev 40608
revlog: add blank line in comment to help clang-format
Differential Revision: https://phab.mercurial-scm.org/D5233
Augie Fackler <augie@google.com> [Tue, 06 Nov 2018 11:18:12 -0500] rev 40607
revlog: add a comment to help clang-format produce less-awful results
Differential Revision: https://phab.mercurial-scm.org/D5232
Augie Fackler <augie@google.com> [Tue, 06 Nov 2018 11:52:41 -0500] rev 40606
py3: ratchet caught two more passing tests
Differential Revision: https://phab.mercurial-scm.org/D5236
Boris Feld <boris.feld@octobus.net> [Mon, 05 Nov 2018 15:57:31 +0100] rev 40605
perf: add a lazydeltabase option to perfrevlogwrite
The option controls if we'll test the provided delta first. This is the current
default behavior for unbundle.
Boris Feld <boris.feld@octobus.net> [Mon, 05 Nov 2018 15:24:09 +0100] rev 40604
perf: add `storage` as possible source for perfrevlogwrite
This source will use the stored delta.
Boris Feld <boris.feld@octobus.net> [Mon, 05 Nov 2018 15:19:44 +0100] rev 40603
perf: add `parent-smallest` as possible source for perfrevlogwrite
This source will use the smallest of the possible diff against parent.
Boris Feld <boris.feld@octobus.net> [Mon, 05 Nov 2018 15:15:18 +0100] rev 40602
perf: add `parent-2` as possible source for perfrevlogwrite
This source will use a diff against p2 if it exists and fall back to p1
otherwise.
Boris Feld <boris.feld@octobus.net> [Mon, 05 Nov 2018 15:15:02 +0100] rev 40601
perf: add `parent-1` as possible source for perfrevlogwrite
This source will use a diff against p1 in all case.
Boris Feld <boris.feld@octobus.net> [Fri, 19 Oct 2018 17:23:29 +0200] rev 40600
perf: add the notion of "source" to perfrevlogwrite
We want to test performance associated witch various way to add a new revision.
They will be specified using this new argument.
Boris Feld <boris.feld@octobus.net> [Tue, 06 Nov 2018 00:57:34 +0100] rev 40599
perf: only display the total time for perfrevlogwrite if quiet
This provide a simple way to get an overview of the total performance.
Boris Feld <boris.feld@octobus.net> [Wed, 03 Oct 2018 11:04:57 +0200] rev 40598
perf: offer full details in perfrevlogwrite
This will be useful for people who want to study the timing pattern more
closely.
Boris Feld <boris.feld@octobus.net> [Wed, 03 Oct 2018 10:53:29 +0200] rev 40597
perf: introduce a perfrevlogwrite command
The command record times taken by adding many revisions to a revlog. Timing
each addition, individually. The "added revision" are recreations of the
original ones.
To time each addition individually, we have to handle the timing and the
reporting ourselves.
This command is introduced to track the impact of sparse-revlog format on
delta computations at initial storage time. It starts with the full text, a
situation similar to the "commit". Additions from an existing delta are better
timed with bundles.
The complaints from `check-perf-code.py` are not relevant. We are accessing
and "revlog" opener, not a repository opener.
Augie Fackler <augie@google.com> [Tue, 06 Nov 2018 10:41:00 -0500] rev 40596
tests: fix config knob in test-narrow-clone-stream.t
Two patches landed in parallel and had a semantic conflict. This
resolves the mess and leaves us with passing tests.
Differential Revision: https://phab.mercurial-scm.org/D5231
Augie Fackler <augie@google.com> [Tue, 06 Nov 2018 10:26:33 -0500] rev 40595
remotefilelog: fix various whitespace issues in docstring
Differential Revision: https://phab.mercurial-scm.org/D5230
Yuya Nishihara <yuya@tcha.org> [Sat, 03 Nov 2018 19:42:50 +0900] rev 40594
ui: add config knob to redirect status messages to stderr (API)
This option can be used to isolate structured output from status messages.
For now, "stdio" (stdout/err pair) and "stderr" are supported. In future
patches, I'll add the "channel" option which will send status messages to
a separate command-server channel with some metadata attached, maybe in
CBOR encoding.
This is a part of the generic templating plan:
https://www.mercurial-scm.org/wiki/GenericTemplatingPlan#Sanity_check_output
.. api::
Status messages may be sent to a dedicated stream depending on
configuration. Don't use ``ui.status()``, etc. as a shorthand for
conditional writes. Use ``ui.write()`` for data output.
Yuya Nishihara <yuya@tcha.org> [Sat, 03 Nov 2018 20:53:31 +0900] rev 40593
ui: hide fin/fout/ferr attributes behind @property functions
This allows keeping references to fout/ferr/fin which are updated when these
properties are changed. See the next patch.
Yuya Nishihara <yuya@tcha.org> [Sun, 18 Jan 2015 17:42:53 +0900] rev 40592
ui: label prompt and echo messages
I'm going to add a dedicated command-server channel for status messages,
which carries metadata alongside a message text. 'ui.*' label provides a
hint how message text should be processed.
Yuya Nishihara <yuya@tcha.org> [Sat, 03 Nov 2018 18:17:30 +0900] rev 40591
ui: add inner function to select write destination
I'm going to add a config knob to redirect any status messages to stderr.
This function helps to switch underlying file objects.
# no-check-commit because of existing write_err() function
Yuya Nishihara <yuya@tcha.org> [Sat, 03 Nov 2018 18:04:22 +0900] rev 40590
ui: remove _write() and _write_err() functions
Yuya Nishihara <yuya@tcha.org> [Sat, 03 Nov 2018 18:03:09 +0900] rev 40589
ui: move pre/post processes from low-level write()s to _writenobuf()
This helps adding a dedicated stream for status/error messages. I don't
want to add _write*() function per stream.
Yuya Nishihara <yuya@tcha.org> [Sat, 03 Nov 2018 17:56:17 +0900] rev 40588
ui: pass in file object to _writenobuf()
See the subsequent patches for why. The "if" block in _writenobuf() will
be removed soon.
Boris Feld <boris.feld@octobus.net> [Mon, 05 Nov 2018 17:24:39 +0100] rev 40587
perf: fix perfrevlogrevisions --reverse
Currently, 'endrev' equals `len(revlog)`, a revision that does not exist.
When asking for the reverse order, the arguments passed to xrange are
`xrange(len(revlog), startrev)` which then crash.
We need to offset 'endrev' by one so we don't crash anymore. Also, we offset
'startrev' to ensure we get the same number of revisions with and without the
`--reverse` option.
Differential Revision: https://phab.mercurial-scm.org/D5228
Boris Feld <boris.feld@octobus.net> [Tue, 06 Nov 2018 11:54:15 +0100] rev 40586
procutil: import concerns about creationflags on Windows from D1701
I don't have the need anymore for the change in D1701 nor the time to
investigate the changes on all supported Windows platforms.
I import the stuff I learned on D1701 in the `runbgcommand` so the next people
working on it can starts from there.
Differential Revision: https://phab.mercurial-scm.org/D5229
Gregory Szorc <gregory.szorc@gmail.com> [Mon, 05 Nov 2018 14:14:32 -0800] rev 40585
localrepo: extract loading of hgrc files to standalone function
Various 3rd party extensions supplement where per-repo config data
lives. Looking at their sources, they resort to unorthodox means to
inject the config data. And the way they do it is susceptible to
corner cases. e.g. not processing automatic extension loads,
not reacting to new or disabled extensions in configs, etc.
This commit extracts the core logic of loading hgrc files into
a standalone function so there is a clear function that can be
monkeypatched to inject per-repo config data at repository open
time.
Differential Revision: https://phab.mercurial-scm.org/D5221
Martin von Zweigbergk <martinvonz@google.com> [Mon, 05 Nov 2018 09:09:48 -0800] rev 40584
revsets: make bookmark/named('re:nonexistent') not abort (issue6018) (BC)
Foozy documented the differences between revsets branch(), tag(),
bookmark(), and named() in eeb5d5ab14a6 (revset: raise RepoLookupError
to make present() predicate continue the query, 2015-01-31). He seemed
to want tag() to change behavior to not error out on non-matching
regular expressions. I think it's instead bookmark() and named() that
should not error out. So that's what this patch does.
Differential Revision: https://phab.mercurial-scm.org/D5220
Danny Hooper <hooper@google.com> [Mon, 05 Nov 2018 16:05:45 -0800] rev 40583
fix: rename :fileset subconfig to :pattern
This name was always inaccurate, since the config accepts any pattern.
Hopefully so few people use this right now that it won't matter, but there will
now be a warning if the old config name is used.
Differential Revision: https://phab.mercurial-scm.org/D5226
Danny Hooper <hooper@google.com> [Wed, 31 Oct 2018 13:11:51 -0700] rev 40582
fix: add a config to abort when a fixer tool fails
This allows users to stop and address tool failures before proceeding, instead
of the default behavior of continuing to apply any tools that didn't fail. For
example, a code formatting tool could fail if you have syntax errors, and you
might want your repo to stay in its current state while you fix the syntax
error before re-running 'hg fix'. It's conceivable that this would even be
necessary for the correctness of some fixer tools across a chain of revisions.
Differential Revision: https://phab.mercurial-scm.org/D5200
Boris Feld <boris.feld@octobus.net> [Tue, 06 Nov 2018 11:05:13 +0100] rev 40581
perf: measure slicing time in perfrevlogrevision
Slicing a sparse delta chain can be expensive. We now benchmark the associated
time.
Boris Feld <boris.feld@octobus.net> [Tue, 06 Nov 2018 11:04:23 +0100] rev 40580
perf: teach perfrevlogrevision about sparse reading
Before this change, chunks were always read in a single block. Even in the
sparse-read/sparse-revlog case. This gave a false view of the performance and
could lead to memory consumption issue.
Boris Feld <boris.feld@octobus.net> [Tue, 06 Nov 2018 11:13:31 +0100] rev 40579
perf: use the same timer for all section of perfrevlogrevision
Otherwise the -T json output is invalid.
Boris Feld <boris.feld@octobus.net> [Tue, 22 May 2018 15:26:17 +0200] rev 40578
obsutil: clarify the access to "repo"
We use the variable multiple times and we might use it even more in the
future. We use a temporary variable instead.
rdamazio@google.com [Mon, 05 Nov 2018 19:52:42 -0800] rev 40577
pycompat: adding Linux detection and fixing Mac
Python 3 recommends detecting OSs with the prefix of the platform, but we were
comparing the full string for macOS. We also didn't have Linux detection, which
is convenient for extensions to use (rather than have some OSs detected by hg
and some by the extension).
Reference:
https://docs.python.org/3/library/sys.html#sys.platform
Differential Revision: https://phab.mercurial-scm.org/D5227
Augie Fackler <augie@google.com> [Mon, 05 Nov 2018 17:48:23 -0500] rev 40576
remotefilelog: fix various minor py3 problems
# skip-blame b prefixes and pycompat.long, nothing remotely interesting
Differential Revision: https://phab.mercurial-scm.org/D5223
Augie Fackler <augie@google.com> [Mon, 05 Nov 2018 17:37:37 -0500] rev 40575
remotefilelog: rip out a missed mention of lz4
Differential Revision: https://phab.mercurial-scm.org/D5222
Yuya Nishihara <yuya@tcha.org> [Sat, 03 Nov 2018 17:47:23 +0900] rev 40574
ui: wrap whole _write() block with timeblockedsection
I think the cost of color labeling is negligible compared to the I/O
syscalls. Let's simply wrap the whole write() function so that we can
eliminate _write() and _write_err() in later changeset.
Yuya Nishihara <yuya@tcha.org> [Sat, 03 Nov 2018 17:43:57 +0900] rev 40573
ui: indent _writenobuf() to prepare moving bits from _write() functions
Yuya Nishihara <yuya@tcha.org> [Sat, 03 Nov 2018 17:42:05 +0900] rev 40572
ui: simply concatenate messages before applying color labels
This should be cheaper in space than applying labels for each message.
Yuya Nishihara <yuya@tcha.org> [Sat, 03 Nov 2018 17:36:10 +0900] rev 40571
ui: simplify interface of low-level write() functions
_write() and _write_err() will be replaced with fout.write() and ferr.write()
respectively. This is the first step.
Yuya Nishihara <yuya@tcha.org> [Sat, 03 Nov 2018 17:32:35 +0900] rev 40570
ui: factor out function that writes data to fout/ferr with labeling
I'm thinking of adding an option to send status messages to stderr (or a
dedicated command-server channel) so that structured output (e.g. JSON)
would never be interleaved with non-formatter output. A unified write()
interface helps to do that.
Yuya Nishihara <yuya@tcha.org> [Sat, 03 Nov 2018 16:04:16 +0900] rev 40569
ui: consolidate places where _progclear() is called
The progress bar has to be cleared when we start writing some data to the
output stream. Let's make it always triggered immediately before switching
by _colormode, so that we can easily factor out helper functions.
Yuya Nishihara <yuya@tcha.org> [Sun, 04 Nov 2018 20:44:26 +0900] rev 40568
templater: compute revset lazily
This speeds up e.g. "{ifcontains(rev, revset('::.'), ...)}" in common cases
where 'rev' is near the working parent.
The templater API is ugly, but it helps here. 'f' can be either a generator
or a function returning a generator.
Pulkit Goyal <pulkit@yandex-team.ru> [Wed, 24 Oct 2018 18:48:43 +0300] rev 40567
remotefilelog: drop compat code for "getbundle_shallow" wireprotocol command
Doing some annotate on hgexperimental shows that getbundle_shallow used to exist
in 2013 or before. We don't have any pre-2013 remotefilelog users except Fb
themselves and I doubt they are going to use in-core remotefilelog. So it's safe
to remove this.
Differential Revision: https://phab.mercurial-scm.org/D5193
Pulkit Goyal <pulkit@yandex-team.ru> [Wed, 24 Oct 2018 17:54:34 +0300] rev 40566
tests: remove lz4 as dependency while running tests
One of the previous patch authored by Augie rips out the lz4 dependency and
things should work without it.
Now there are just 2-3 tests failing because of same change in emitrevisions()
API.
Differential Revision: https://phab.mercurial-scm.org/D5192
Pulkit Goyal <pulkit@yandex-team.ru> [Wed, 24 Oct 2018 17:51:49 +0300] rev 40565
remotefilelogserver: add a matcher argument to _walkstreamfiles()
Implementing narrow stream clones in core, I added an optional matcher argument.
The function in remotefilelogserver.py does not know about that argument and
does not accept that and hence some tests fails.
Differential Revision: https://phab.mercurial-scm.org/D5191
Pulkit Goyal <pulkit@yandex-team.ru> [Mon, 05 Nov 2018 11:52:10 -0800] rev 40564
shallowutil: introduce a helper function isenabled()
This patch introduces a inenabled() function which will check whether
remotefilelog is enabled or not. The function is then also used at all the
places where check whether remotefilelog is enabled or not. The new function
makes code easy to read without need to understand what is the constant involved
and why we are checking repo.requirements.
Differential Revision: https://phab.mercurial-scm.org/D5190
Pulkit Goyal <pulkit@yandex-team.ru> [Wed, 24 Oct 2018 16:59:05 +0300] rev 40563
remotefilelog: remove some BC code related to streamclone
Since remotefilelog is now moved to core and we have
streamclone._walkstreamfiles() in core, we don't need to have the backward
compatibility code.
People with old mercurial version should use remotefilelog from hg-experimental
repo as IMO remotefilelog will go under a good refactoring and old clients will
break.
# no-check-commit foo_bar function name
Differential Revision: https://phab.mercurial-scm.org/D5189
Augie Fackler <augie@google.com> [Wed, 17 Oct 2018 09:12:29 -0400] rev 40562
remotefilelog: add some docstring
Differential Revision: https://phab.mercurial-scm.org/D5131
Augie Fackler <augie@google.com> [Thu, 04 Oct 2018 10:03:10 -0400] rev 40561
remotefilelog: rename wireproto methods and remaining capabilities
This is a mess, in part because there should be more constants
throughout. I know we typically do exp- instead of the x_ business in
this change, but I also had to use this in some function names, so I
figured until I can break that coupling I'd go with this. If it's too
unpleasant during review, let me know and I can probably clean it up
some more.
# no-check-commit due to new foo_bar naming - too hard to avoid right now :(
Differential Revision: https://phab.mercurial-scm.org/D5129
Augie Fackler <augie@google.com> [Tue, 16 Oct 2018 17:02:48 -0400] rev 40560
remotefilelog: consolidate and rename on-disk store requirement
The value of this constant appeared in too many places. While we're
here, rename it to be more consistent with our naming conventions for
experimental functionality.
Differential Revision: https://phab.mercurial-scm.org/D5128
Augie Fackler <augie@google.com> [Tue, 16 Oct 2018 17:30:47 -0400] rev 40559
remotefilelog: consolidate and rename bundle2 capability
Bonus: we catch a spot where the shallowrepo requirement was being
misused as the bundle2 capability.
Differential Revision: https://phab.mercurial-scm.org/D5127
Augie Fackler <augie@google.com> [Tue, 16 Oct 2018 16:38:40 -0400] rev 40558
remotefilelog: rename capability for legacy ssh file fetching method
I think I want to delete this, but for now we'll just rename this so
it's easy to tell apart from other strings that have the same value.
Differential Revision: https://phab.mercurial-scm.org/D5126
Augie Fackler <augie@google.com> [Thu, 04 Oct 2018 00:11:37 -0400] rev 40557
remotefilelog: rip out lz4 support
Some methods are left teased out so it's easier to build a migration
extension from v1 packs to v2. I also anticipate those methods will be
of use if we get around to adding configurable compression engines
before we end up jettisoning packs.
I think it would make sense to register one-shot `compress` and
`decompress` methods on our compression engines in util.py, but
indygreg mentioned that might not be straightforward.
Differential Revision: https://phab.mercurial-scm.org/D4944
Augie Fackler <augie@google.com> [Wed, 03 Oct 2018 23:25:31 -0400] rev 40556
remotefilelog: bump pack file formats to version 2
Drop support for format 0 and 1 packs. Version 2 will be different in
that it'll use zlib compression instead of lz4, as the latter isn't
really suitable for core.
If I can make it work, I'll make compression engines pluggable, but
the logic for handling these files is spread out enough it's a little
tricky to get that right.
Test changes are only due to pack files being renamed. The contents of
the packs are the same.
Differential Revision: https://phab.mercurial-scm.org/D4943
Augie Fackler <augie@google.com> [Wed, 03 Oct 2018 23:47:43 -0400] rev 40555
remotefilelog: avoid hard-coding pack names in tests
I'm about to alter the pack format, which alters the content-addressed
name of the pack file. In order to reduce my own confusion I found it
helpful to use an ls | head pipeline since we always want to inspect
the newest pack.
Differential Revision: https://phab.mercurial-scm.org/D4942
Augie Fackler <augie@google.com> [Tue, 16 Oct 2018 14:04:41 -0400] rev 40554
remotefilelog: remove now-unused wirepack code
Differential Revision: https://phab.mercurial-scm.org/D5124
Augie Fackler <augie@google.com> [Tue, 16 Oct 2018 14:04:11 -0400] rev 40553
remotefilelogserver: remove pack-serving functionality
Differential Revision: https://phab.mercurial-scm.org/D5123
Augie Fackler <augie@google.com> [Tue, 16 Oct 2018 13:17:16 -0400] rev 40552
remotefilelog: remove support for fetching packs
We've never used this at Google, and it's strictly an optimization
that indygreg and I are confident we can handle more elegantly
later. Let's rip it out to keep the surface area of the imported
remotefilelog small.
Some tests have changes because the tests for regular packs and
repacking were mixed with tests of fetching packs. I'm pretty sure
I've done the right thing here.
Differential Revision: https://phab.mercurial-scm.org/D5122
Augie Fackler <augie@google.com> [Wed, 03 Oct 2018 17:20:44 -0400] rev 40551
remotefilelog: jettison example cache client
We can always reconstruct it from history if it becomes interesting.
Differential Revision: https://phab.mercurial-scm.org/D4941
Augie Fackler <augie@google.com> [Wed, 03 Oct 2018 17:18:20 -0400] rev 40550
remotefilelog: prune obsolete method
Differential Revision: https://phab.mercurial-scm.org/D4940
Augie Fackler <augie@google.com> [Wed, 03 Oct 2018 14:01:22 -0400] rev 40549
logtoprocess: use new runbgcommand from procutil
Differential Revision: https://phab.mercurial-scm.org/D4939
Augie Fackler <augie@google.com> [Tue, 16 Oct 2018 14:42:57 -0400] rev 40548
procutil: port over windows encoding fixes from logtoprocess
Differential Revision: https://phab.mercurial-scm.org/D5121
Augie Fackler <augie@google.com> [Wed, 03 Oct 2018 14:01:04 -0400] rev 40547
remotefilelog: transplant runbgcommand to procutil
While cleaning up the deprecated runshellcommand I noticed a
near-clone of this in logtoprocess, so I'm standardizing on what
appears to be the newer one by moving it to procutil.
Differential Revision: https://phab.mercurial-scm.org/D4938
Augie Fackler <augie@google.com> [Wed, 03 Oct 2018 13:54:45 -0400] rev 40546
remotefilelog: remove function that was described as deprecated
Differential Revision: https://phab.mercurial-scm.org/D4937
Augie Fackler <augie@google.com> [Thu, 27 Sep 2018 13:03:19 -0400] rev 40545
remotefilelog: import pruned-down remotefilelog extension from hg-experimental
This is remotefilelog as of my recent patches for compatibility with
current tip of hg, minus support for old versions of Mercurial and
some FB-specific features like their treemanifest extension and
fetching linkrev data from a patched phabricator. The file extutil.py
moved from hgext3rd to remotefilelog.
This is not yet ready to be landed, consider it a preview for
now. Planned changes include:
* replace lz4 with zstd
* rename some capabilities, requirements and wireproto commands to mark
them as experimental
* consolidate bits of shallowutil with related functions (eg readfile)
I'm certainly open to other (small) changes, but my rough mission is
to land this largely as-is so we can use it as a model of the
functionality we need going forward for lazy-fetching of file contents
from a server.
# no-check-commit because of a few foo_bar functions
Differential Revision: https://phab.mercurial-scm.org/D4782
Pulkit Goyal <pulkit@yandex-team.ru> [Wed, 17 Oct 2018 17:42:32 +0300] rev 40544
store: introduce _matchtrackedpath() and use it to filter store files
This patch introduces a function to filter store files on the basis of the path
which they are tracking.
The function assumes that the entries can be of two types, 'meta/*' and 'data/*'
which means it will just work on revlog based storage and not with another
storage ways.
For the 'data/*' entries, we remove the 'data/' part and '.i/.d' part from the
beginning and the end then pass that to matcher.
For the 'meta/*' entries, we remove the 'meta/' and '/00manifest.(i/d)' part from
beginning and end then call matcher.visitdir() with it to make sure all the
parent directories are also downloaded.
Since the storage filtering for narrow stream clones is implemented with this
patch, we remove the un-implemented error message, add some more tests and add
the treemanifest case to tests too.
The tests demonstrate that it works correctly.
After this patch, we have now narrow stream clones working. Narrow stream clones
are a very important feature for large repositories who have good internet
connection because they use streamclones for cloning and if they do normal
narrow clone, that takes more time then a full streamclone. Also narrow-stream
clone will drastically speed up clone timings.
Differential Revision: https://phab.mercurial-scm.org/D5139
Pulkit Goyal <pulkit@yandex-team.ru> [Wed, 17 Oct 2018 17:24:55 +0300] rev 40543
narrow: only send includepats and excludepats if they are not empty
If we send an empty includepats or excludepats argument to getbundle, it's
translated to `['']` on the server which causes problems because even though
it's empty, bool of that value if True and we end up creating differencematcher
with narrowspec.match() which results in unexpected behavior.
Differential Revision: https://phab.mercurial-scm.org/D5138
Pulkit Goyal <pulkit@yandex-team.ru> [Tue, 16 Oct 2018 17:53:26 +0300] rev 40542
exchange: pass includepats and excludepats as arguments to getbundle()
This will help in implementing narrow stream clones. Also narrow extension used
to add these arguments, now we add them by default if they are not empty.
Since reading includepats and excludepats on the server only works when narrow
is enabled, we check if narrow if enabled or not before passing them.
Differential Revision: https://phab.mercurial-scm.org/D5119
Kyle Lippincott <spectral@google.com> [Thu, 01 Nov 2018 16:51:21 -0700] rev 40541
catapult: add a bit more documentation on how to use catapult tracing
Differential Revision: https://phab.mercurial-scm.org/D5217
Kyle Lippincott <spectral@google.com> [Thu, 01 Nov 2018 16:44:02 -0700] rev 40540
catapult: introduce HGTESTCATAPULTSERVERPIPE to control run-tests' tracing
If unset, it will inherit its value from HGCATAPULTSERVERPIPE (which hg itself
also respects). By setting only HGTESTCATAPULTSERVERPIPE, we can get per-command
breakdowns of the test runtimes for the whole test suite without overloading the
trace file with the contents of the tracing from hg (such as demandimport, etc.)
Differential Revision: https://phab.mercurial-scm.org/D5216
Kyle Lippincott <spectral@google.com> [Thu, 01 Nov 2018 16:44:37 -0700] rev 40539
catapult: rename 'active' to 'activetrace'; this isn't storing a boolean state
(This is an attempt to make it more obvious that the indentation is correct, to
prevent a repeat of 97f52862b1bd)
Differential Revision: https://phab.mercurial-scm.org/D5215
Kyle Lippincott <spectral@google.com> [Thu, 01 Nov 2018 16:44:32 -0700] rev 40538
catapult: fix broken run-tests catapult tracing
Commit 97f52862b1bd thought this block was misindented, but it was not; the
indentation after that commit means we never log a catapult trace for individual
commands. Also, we've never emitted the "end" trace line when the test was
finished, so the trace viewer claims that these "did not finish", fixing that as
well.
Differential Revision: https://phab.mercurial-scm.org/D5214
Sangeet Kumar Mishra <mail2sangeetmishra@gmail.com> [Sat, 03 Nov 2018 23:24:15 +0530] rev 40537
test: fix self._testdir to use the right mercurial library during testing
Currently if you run tests from some other directory other than the
`../tests/`, you will get a warning stating
```
warning: Testing with unexpected mercurial lib: mercurial
(expected /tmp/hgtests.xxxxxx/install/lib/python/mercurial)
```
This is because the current directory being added to the 'PATH', if the
`self._testdir != runtestdir`, owing to this line
```
if self._testdir != runtestdir:
path = [self._testdir] + path
```
Also say you ran the tests from the hg base directory,
because directory is being added in the PATH (see the above snippet, at
that stage the `self._testdir` has the value as `cwd`, owing to a faulty
initialization). And since the current directory already has the 'hg',
that is used in place of the hg that is installed for the testing purposes
in `/tmp/hgtests.xxxxxx/...`.
Differential Revision: https://phab.mercurial-scm.org/D5199
Matt Harbison <matt_harbison@yahoo.com> [Sat, 03 Nov 2018 20:50:04 -0400] rev 40536
tests: conditonalize a difference in test-merge-tools.t on Windows
Augie Fackler <augie@google.com> [Thu, 01 Nov 2018 17:12:22 -0400] rev 40535
statprof: clean up unicode/bytes a little
I'm not really sure how this worked before, but something perturbed it
and what I've got in this change I believe is a little tidier. This
fixes test-profile.t on Python 3.
Differential Revision: https://phab.mercurial-scm.org/D5210
Augie Fackler <augie@google.com> [Thu, 01 Nov 2018 17:11:31 -0400] rev 40534
statprof: add a couple of asserts to avoid storing unicodes
I don't feel strongly about this change, so it'd be fine to drop it
during review, but it was _extremely_ helpful in drafting the next
change.
Differential Revision: https://phab.mercurial-scm.org/D5209
Augie Fackler <augie@google.com> [Thu, 01 Nov 2018 17:09:55 -0400] rev 40533
tests: add some helpful `|| cat` bits to test-profile.t
This way if the hg invocation crashes, you get to see a stacktrace
without having to edit the test.
Differential Revision: https://phab.mercurial-scm.org/D5208
Augie Fackler <augie@google.com> [Thu, 01 Nov 2018 16:13:01 -0400] rev 40532
tests: skip wireproto clientreactor tests on Python 3.6.0-3.6.3 inclusive
See comment next to the if statement for sad details.
Differential Revision: https://phab.mercurial-scm.org/D5207