Gregory Szorc <gregory.szorc@gmail.com> [Thu, 20 Sep 2018 17:47:34 -0700] rev 39860
lfs: access revlog directly
LFS is monkeypatching filelog.filelog and is then accessing
various filelog attributes in the monkeypatched function. This is all
fine.
But some of the attributes being accessed by LFS are revlog centric
and shouldn't be exposed on the file storage interface.
This commit changes the monkeypatched functions to access proxied
attributes on self._revlog instead of self.
This should be safe to do because non-revlog repositories should not
be using filelog instances: instead they should have a separate class
to represent file storage. So it is reasonable for LFS to assume the
_revlog attribute exists and points to a revlog.
Differential Revision: https://phab.mercurial-scm.org/D4714
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 20 Sep 2018 15:30:00 -0700] rev 39859
largefiles: automatically load largefiles extension when required (BC)
This is very similar to what we just did for LFS but for largefiles.
See recent commit messages for the rationale here.
Differential Revision: https://phab.mercurial-scm.org/D4713
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 20 Sep 2018 15:18:13 -0700] rev 39858
lfs: don't add extension to hgrc after clone or share (BC)
Now that repository loading in core supports automatically loading
the lfs extension when the "lfs" requirement is present, we no
longer need to update the .hg/hgrc of newly-created repos to load
the lfs extension!
I'm marking this as BC because it is a change in behavior. But users
should not notice unless they create an LFS repo with new Mercurial
and then attempt to use it with an old versions that doesn't support
automatic extension loading.
Differential Revision: https://phab.mercurial-scm.org/D4712
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 20 Sep 2018 15:06:43 -0700] rev 39857
localrepo: automatically load lfs extension when required (BC)
If an unrecognized requirement is present (possibly due to an unloaded
extension), the user will get an error message telling them to go to
https://mercurial-scm.org/wiki/MissingRequirement for more info.
And some requirements clearly map to known extensions shipped by
Mercurial.
This commit teaches repository loading to automatically map
requirements to extensions. We implement support for loading the
lfs extension when the "lfs" requirement is present.
This behavior feels more user-friendly to me and I'm having trouble
coming up with a compelling reason to not do it. The strongest
argument I have against is that - strictly speaking - requirements
are general repository features and there could be N providers of that
feature. e.g. in the case of LFS, there could be another extension
implementing LFS support. And the user would want to use this
non-official extension rather than the built-in one. The way this
patch implements things, the non-official extension could be
missing and Mercurial would load the official lfs extension, leading
to unexpected behavior. But this feels like a highly marginal use
case to me and doesn't outweigh the user benefit of "it just works."
If someone really wanted to e.g. use a custom LFS extension, they
could prevent the built-in one from being loaded by either defining
"extensions.lfs=/path/to/custom/extension" or "extensions.lfs=!",
as the automatic extension loading only occurs if there is no config
entry for that extension.
Differential Revision: https://phab.mercurial-scm.org/D4711
Gregory Szorc <gregory.szorc@gmail.com> [Wed, 19 Sep 2018 13:48:59 -0700] rev 39856
lfs: add repository feature denoting the use of LFS
Whether LFS is enabled seems like a useful feature to expose. This
will also facilitate some future work around LFS feature compatibility.
Differential Revision: https://phab.mercurial-scm.org/D4710
Gregory Szorc <gregory.szorc@gmail.com> [Wed, 19 Sep 2018 14:36:57 -0700] rev 39855
localrepo: define "features" on repository instances (API)
There are a handful of attributes/methods on repository instances that
describe the behavior of the repository. Furthermore, there is
an unbound set of repository descriptors that we may wish to expose.
For example, an extension may wish to add a descriptor and have
monkeypatched functions look for the presence of an attribute before
taking actions.
This commit introduces a "features" mechanism to allow repositories
to self-advertise an arbitrary set of strings that describe repository
behavior or capabilities.
We implement basic support for advertising a few features to give an
idea of what I want to use this for.
Differential Revision: https://phab.mercurial-scm.org/D4709
Gregory Szorc <gregory.szorc@gmail.com> [Wed, 19 Sep 2018 17:27:37 -0700] rev 39854
localrepo: support writing shared file (API)
Now that we can create a shared repository via creation options, we
can handle other special actions related to share at repo creation time
as well.
One of the things we do after creating a shared repository is write
out a .hg/shared file containing the list of additional things to
share. Of which only "bookmarks" is supported.
We add a creation option to hold the set of additional items to
share. If items are defined, we write out the .hg/shared file at
repo creation time.
As part of this, we no longer hold the repo lock when writing the
file. I'm pretty sure we don't care about the tiny race condition
window. I'm also pretty sure the reason we used the lock was because
the vfs auditor on the repo instance complained otherwise. Since the
repo creation code doesn't have an audited vfs, we don't need to
appease it.
Because we no longer need to tell the post share hook what items
are shared, the "bookmarks" argument to that function has been
dropped, incurring an API change.
Differential Revision: https://phab.mercurial-scm.org/D4708
Gregory Szorc <gregory.szorc@gmail.com> [Wed, 19 Sep 2018 17:05:59 -0700] rev 39853
localrepo: support shared repo creation
Previously, hg.share() had its own logic for creating a new
repository on the filesystem.
With the recent introduction of the createopts dict for passing
options to influence repository creation, it is now possible
to consolidate the repo creation code for both the normal and
shared use cases.
This commit teaches the repo creation code in localrepo to
recognize when we're creating a shared repo and to act
appropriately.
Meaningful behavior should be identical. However, there are a
few subtle changes:
* The .hg/requires file is written out in sorted order (rather
than having share-related requirements appended at end).
* The .hg directory is created with notindexed=True when a shared
repo is being created.
Differential Revision: https://phab.mercurial-scm.org/D4707
Gregory Szorc <gregory.szorc@gmail.com> [Wed, 19 Sep 2018 16:51:57 -0700] rev 39852
localrepo: validate directories before creating any
There is no meaningful change in behavior because wdir would
already exist in the case where we raised RepoError. But I think
the code is easier to read if we do all validation first then
take actions with side-effects.
Differential Revision: https://phab.mercurial-scm.org/D4706
Gregory Szorc <gregory.szorc@gmail.com> [Wed, 19 Sep 2018 16:15:22 -0700] rev 39851
localrepo: add missing join()
Differential Revision: https://phab.mercurial-scm.org/D4705
Gregory Szorc <gregory.szorc@gmail.com> [Wed, 19 Sep 2018 11:38:05 -0700] rev 39850
revlog: use proper version comparison during verify
Verify appears to want to compare the changelog's revlog version
number with the version number of filelogs and error if they are
different. But what it was actually doing was comparing the full
32-bit header integer, which contains 2 shorts: 1 for the revlog
version number and 1 for feature flags.
This commit tweaks the verification code so it only looks at the
version number component of the header and emits a warning if they
differ.
The new code is more robust because it accounts for future revlog
version numbers without them needing to be special cased.
Differential Revision: https://phab.mercurial-scm.org/D4704
Gregory Szorc <gregory.szorc@gmail.com> [Wed, 19 Sep 2018 11:22:56 -0700] rev 39849
filelog: stop proxying checksize() (API)
This was only used by verify code. And the check using it is now
implemented as part of verifyintegrity(). The method is unused
and is revlog-centric, which means it isn't appropriate for the file
storage interface. So remove it.
Differential Revision: https://phab.mercurial-scm.org/D4703
Gregory Szorc <gregory.szorc@gmail.com> [Wed, 19 Sep 2018 11:20:02 -0700] rev 39848
filelog: remove version attribute (API)
This was only used by verify code. The check it was used for is now
implemented as part of the verifyintegrity() implementation. The
attribute is now unused, is revlog-specific, and isn't appropriate
to be exposing on the file storage interface. So drop it.
Differential Revision: https://phab.mercurial-scm.org/D4702
Gregory Szorc <gregory.szorc@gmail.com> [Wed, 19 Sep 2018 11:17:28 -0700] rev 39847
verify: start to abstract file verification
Currently, the file storage interface has a handful of attributes
that are exclusively or near-exclusively used by repo verification
code. In order to support verification on non-revlog/alternate
storage backends, we'll need to abstract verification so it can
be performed in a storage-agnostic way.
This commit starts that process.
We establish a new verifyintegrity() method on revlogs and expose
it to the file storage interface. Most of verify.verifier.checklog()
has been ported to this new method.
We need a way to represent verification problems. So we invent an
interface to represent a verification problem, invent a revlog type
to implement that interface, and use it.
The arguments to verifyintegrity() will almost certainly change in
the future, once more functionality is ported from the verify code.
And the "revlogv1" version check is very hacky. (The code in verify
is actually buggy because it is comparing the full 32-bit header
integer instead of just the revlog version short. I'll likely fix
this in a subsequent commit.)
Differential Revision: https://phab.mercurial-scm.org/D4701
Gregory Szorc <gregory.szorc@gmail.com> [Mon, 24 Sep 2018 08:58:57 -0700] rev 39846
unionrepo: remove _constructmanifest()
It is unused after a previous refactor. Spotted in D4641.
Differential Revision: https://phab.mercurial-scm.org/D4700
Gregory Szorc <gregory.szorc@gmail.com> [Wed, 26 Sep 2018 08:46:56 -0700] rev 39845
merge with stable
Martin von Zweigbergk <martinvonz@google.com> [Tue, 25 Sep 2018 08:53:20 -0700] rev 39844
encoding: remove unnecessary lambdas from _encodingfixers
They have been unnecessary since 1a3a08b5d4d5 (encoding: remove
workaround for locale.getpreferredencoding(), 2017-05-13). Also rename
the variable since "fixer" sounds like a function.
Differential Revision: https://phab.mercurial-scm.org/D4743
Gregory Szorc <gregory.szorc@gmail.com> [Tue, 25 Sep 2018 18:59:04 -0700] rev 39843
py3: cast exception to bytes
In order to appease Python 3.
Differential Revision: https://phab.mercurial-scm.org/D4733
Gregory Szorc <gregory.szorc@gmail.com> [Tue, 25 Sep 2018 09:11:56 -0700] rev 39842
py3: cast exception to bytes
This was raising a TypeError on Python 3 and preventing most (all?)
wireprotov2 commands from working.
Differential Revision: https://phab.mercurial-scm.org/D4731
Matt Harbison <matt_harbison@yahoo.com> [Tue, 25 Sep 2018 23:25:36 -0400] rev 39841
py3: remove a couple of superfluous calls to pycompat.rapply()
These places can only be strings.
Matt Harbison <matt_harbison@yahoo.com> [Tue, 25 Sep 2018 22:11:17 -0400] rev 39840
py3: byteify an inline python test extension
# skip-blame for b'' prefixing
Matt Harbison <matt_harbison@yahoo.com> [Tue, 25 Sep 2018 21:39:42 -0400] rev 39839
py3: conditionalize access to socketserver.ForkingMixIn
This is no longer exported on platforms that don't support forking, as of 3.6.
https://github.com/python/cpython/commit/aadff9bea61a2fc9f4cf0f213f0ee50fc54d6574
Matt Harbison <matt_harbison@yahoo.com> [Tue, 25 Sep 2018 22:46:18 -0400] rev 39838
convert: fix a file descriptor leak
test-check-code flagged this after I changed this line for something unrelated.
Yuya Nishihara <yuya@tcha.org> [Wed, 26 Sep 2018 20:33:09 +0900] rev 39837
merge with stable
Yuya Nishihara <yuya@tcha.org> [Tue, 25 Sep 2018 22:19:40 +0900] rev 39836
revlog: catch more specific exception in shortest()
Since revlog._partialmatch() catches RevlogError coming from cext and
re-raises AmbiguousPrefixLookupError, catching RevlogError here seems
less correct.
Differential Revision: https://phab.mercurial-scm.org/D4735
Matt Harbison <matt_harbison@yahoo.com> [Mon, 24 Sep 2018 22:32:30 -0400] rev 39835
py3: update missing module list in test-check-py3-compat.t for Windows
Gregory Szorc <gregory.szorc@gmail.com> [Mon, 24 Sep 2018 20:31:42 -0700] rev 39834
py3: add b'' prefixes to wire protocol test
Otherwise the CBOR encoder fails on Python 3 due to lacking support
for encoding str/unicode.
# skip-blame just some b'' prefixes
Differential Revision: https://phab.mercurial-scm.org/D4734
Gregory Szorc <gregory.szorc@gmail.com> [Mon, 24 Sep 2018 20:17:42 -0700] rev 39833
py3: use pycompat.strkwargs()
Otherwise we get an error attempting to dispatch a command with
arguments because we're passing a dict with bytes keys.
Differential Revision: https://phab.mercurial-scm.org/D4732
Gregory Szorc <gregory.szorc@gmail.com> [Mon, 24 Sep 2018 20:10:01 -0700] rev 39832
py3: ensure _start_response() is called with system string
This was preventing HTTP 500's from being sent in Python 3.
Differential Revision: https://phab.mercurial-scm.org/D4730
Matt Harbison <matt_harbison@yahoo.com> [Sun, 23 Sep 2018 00:47:04 -0400] rev 39831
py3: convert arguments, cwd and env to native strings when spawning subprocess
This keeps Windows happy.
Matt Harbison <matt_harbison@yahoo.com> [Fri, 21 Sep 2018 21:14:27 -0400] rev 39830
py3: apply byteskwargs to contrib/perf
Matt Harbison <matt_harbison@yahoo.com> [Fri, 21 Sep 2018 20:28:00 -0400] rev 39829
py3: un-byteify strings around os.system() and os.devnull in contrib/perf