Pulkit Goyal <7895pulkit@gmail.com> [Sat, 31 Mar 2018 14:49:16 +0530] rev 37238
bunlde2: add 'source' as an optional argument to processbundle()
This will help us to pass the source variable to bundleoperation class.
Differential Revision: https://phab.mercurial-scm.org/D2994
Pulkit Goyal <7895pulkit@gmail.com> [Sat, 31 Mar 2018 14:47:43 +0530] rev 37237
bundle2: add 'source' atrribute to bundleoperation class
This will help us in easily finding out which command leads to the current
operation without hacking on the transaction.
Differential Revision: https://phab.mercurial-scm.org/D2993
Pulkit Goyal <7895pulkit@gmail.com> [Sat, 31 Mar 2018 10:13:05 -0700] rev 37236
infinitepush: don't force ipv6 while connecting to mysql server
Facebook internally enforces this but looks like we can't force this for pur
users.
Differential Revision: https://phab.mercurial-scm.org/D2992
Pulkit Goyal <7895pulkit@gmail.com> [Sat, 31 Mar 2018 14:33:37 +0530] rev 37235
infinitepush: drop the default value of config options which are registered
Differential Revision: https://phab.mercurial-scm.org/D2991
Pulkit Goyal <7895pulkit@gmail.com> [Sat, 31 Mar 2018 14:31:29 +0530] rev 37234
infinitepush: replace `remotenames.hoist` with `remotenames.hoistedpeer`
The remotenames.hoist config option was renamed to remotenames.hoistedpeer while
moving to core as an extension. Let's start using the config option provided by
the in-core extension.
Differential Revision: https://phab.mercurial-scm.org/D2990
Yuya Nishihara <yuya@tcha.org> [Sun, 18 Mar 2018 16:57:07 +0900] rev 37233
templater: drop global exception catcher from runfilter() (API)
Now all built-in template filters declare input data types or handle type
errors by themselves.
.. api::
Template filters should declare input data type and/or catch
AttributeError, ValueError, TypeError, etc. as needed. See the doc of
``registrar.templatefilters`` for details.
Yuya Nishihara <yuya@tcha.org> [Sun, 18 Mar 2018 16:53:08 +0900] rev 37232
templatefilters: raise ProgrammingError if unencodable type passed to json()
This shouldn't happen for any template data types (though I know it does
because of some templater bugs.) Let's clarify it is a bug.
Yuya Nishihara <yuya@tcha.org> [Sun, 18 Mar 2018 16:47:44 +0900] rev 37231
templatefilters: handle TypeError by count()
Prepares for removing the weird exception catcher from runfilter().
Yuya Nishihara <yuya@tcha.org> [Sun, 18 Mar 2018 17:02:39 +0900] rev 37230
keyword: declare input type of date filters as date
Dropped redundant dateutil.parsedate(), which is now handled by the
templater.
Yuya Nishihara <yuya@tcha.org> [Sun, 18 Mar 2018 16:12:44 +0900] rev 37229
templatefilters: declare input type as date where appropriate
I'm not sure if the templateutil.date type can be a thing. Currently it's
just a constant.
Yuya Nishihara <yuya@tcha.org> [Tue, 20 Mar 2018 22:57:36 +0900] rev 37228
templater: attach hint to input-type error of runfilter()
Tests will be added by the next patch.
Yuya Nishihara <yuya@tcha.org> [Sun, 18 Mar 2018 15:58:22 +0900] rev 37227
templatefuncs: use evaldate() where seems appropriate
This means date("today") is allowed.
Also fixes evaldate() to forcibly use the custom error message if specified.
Yuya Nishihara <yuya@tcha.org> [Sun, 18 Mar 2018 15:55:31 +0900] rev 37226
templater: factor out function that parses argument as date tuple
Yuya Nishihara <yuya@tcha.org> [Sun, 18 Mar 2018 15:42:28 +0900] rev 37225
templatefilters: declare input type as bytes where appropriate
Some test outputs changed since input is now coerced to a byte string. I
think that's okay. Maybe {date} should have some readable representation?
Yuya Nishihara <yuya@tcha.org> [Sun, 18 Mar 2018 15:14:58 +0900] rev 37224
templatefilters: allow declaration of input data type
Currently filters take an unwrapped value, which should have no hybrid magic
but actually it does because stringify() relies on it. The 'intype' allows
us to pre-process the magic by .e.g. evalstring() keeping filter functions
as simple as they are now.
stringify() is ported as an example. More follow.
Yuya Nishihara <yuya@tcha.org> [Mon, 19 Mar 2018 20:39:06 +0900] rev 37223
hgweb: use registrar to declare "websub" template filter
Prepares for adding 'intype' attribute.
Yuya Nishihara <yuya@tcha.org> [Sun, 25 Mar 2018 12:15:33 +0900] rev 37222
procutil: redirect ui.fout to stderr while stdio is protected
The new behavior seems slightly nicer as we can at least read the output.
And this is similar to what the sshserver is doing, so we can probably
reuse protectstdio() instead of the weird hook.redirect(True) hack.
Yuya Nishihara <yuya@tcha.org> [Sun, 25 Mar 2018 12:07:18 +0900] rev 37221
procutil: unroll uin/uout loop in protectstdio()
I'll change uout to be redirected to stderr.
Martin von Zweigbergk <martinvonz@google.com> [Fri, 30 Mar 2018 22:46:22 -0700] rev 37220
context: drop support for changeid of type long (API?)
I don't see a reason to support type long. It's pretty much the same
type as int. There was some discussion about it on the mailing list
around the time of ff2f90503d64 (context: work around `long` not
existing on Python 3, 2017-03-11), but I couldn't find a good reason
to keep it. There was some mention of hgtk doing "repo[long(rev)]",
but that was in 2012.
Differential Revision: https://phab.mercurial-scm.org/D2989
Matt Harbison <matt_harbison@yahoo.com> [Sat, 31 Mar 2018 00:02:31 -0400] rev 37219
lfs: drop a duplicate blob verification method
Matt Harbison <matt_harbison@yahoo.com> [Fri, 30 Mar 2018 23:32:06 -0400] rev 37218
server: minor code cleanup
Suggested by Yuya after 77f9e95fe3c4, this is mostly using named values for
stdio descriptors. The lockfile is also opened in binary mode when reading back
content from the child.
Matt Harbison <matt_harbison@yahoo.com> [Fri, 30 Mar 2018 23:21:46 -0400] rev 37217
server: refactor 'daemon_postexec' instructions into a dictionary
Gregory Szorc <gregory.szorc@gmail.com> [Wed, 28 Mar 2018 13:37:19 -0700] rev 37216
cbor: import CBORDecoder and CBOREncoder
And format the imports so it is cleaner.
Differential Revision: https://phab.mercurial-scm.org/D2978
Yuya Nishihara <yuya@tcha.org> [Sat, 31 Mar 2018 10:28:18 +0900] rev 37215
py3: fix fix doctests to be bytes-safe
Matt Harbison <matt_harbison@yahoo.com> [Wed, 28 Mar 2018 00:11:09 -0400] rev 37214
server: add an error feedback mechanism for when the daemon fails to launch
There's a recurring problem on Windows where `hg serve -d` will randomly fail to
spawn a detached process. The reason for the failure is completely hidden, and
it takes hours to get a single failure on my laptop. All this does is redirect
stdout/stderr of the child to a file until the lock file is freed, and then the
parent dumps it out if it fails to spawn.
I chose to put the output into the lock file because that is always cleaned up.
There's no way to report errors after that anyway. On Windows, killdaemons.py
is roughly `kill -9`, so this ensures that junk won't pile up.
This may end up being a case of EADDRINUSE. At least that's what I saw spit out
a few times (among other odd errors and missing output on Windows). But I also
managed to get the same thing on Fedora 26 by running test-hgwebdir.t with
--loop -j10 for several hours. Running `netstat` immediately after killing that
run printed a wall of sockets in the TIME_WAIT state, which were gone a couple
seconds later. I couldn't match up ports that failed, because --loop doesn't
print out the message about the port that was used. So maybe the fix is to
rotate the use of HGPORT[12] in the tests. But, let's collect some more data
first.
Matt Harbison <matt_harbison@yahoo.com> [Fri, 30 Mar 2018 20:53:36 -0400] rev 37213
tests: conditionalize test-stream-bundle-v2 for Windows
Connor Sheehan <sheehan@mozilla.com> [Mon, 19 Mar 2018 11:16:21 -0400] rev 37212
templatefuncs: add mailmap template function
This commit adds a template function to support the .mailmap file
in Mercurial repositories. The .mailmap file comes from git, and
can be used to map new emails and names for old commits. The general
use case is that someone may change their name or author commits
under different emails and aliases, which would make these
commits appear as though they came from different persons. The
file allows you to specify the correct name that should be used
in place of the author field specified in the commit.
The mailmap file has 4 possible formats used to map old "commit"
names to new "proper" names:
1. <proper@email.com> <commit@email.com>
2. Proper Name <commit@email.com>
3. Proper Name <proper@email.com> <commit@email.com>
4. Proper Name <proper@email.com> Commit Name <commit@email.com>
Essentially there is a commit email present in each mailmap entry,
that maps to either an updated name, email, or both. The final
possible format allows commits authored by a person who used
both an old name and an old email to map to a new name and email.
To parse the file, we split by spaces and build a name out
of every element that does not start with "<". Once we find an element
that does start with "<" we concatenate all the name elements that preceded
and add that as a parsed name. We then add the email as the first
parsed email. We repeat the process until the end of the line, or
a comment is found. We will be left with all parsed names in a list,
and all parsed emails in a list, with the 0 index being the proper
values and the 1 index being the commit values (if they were specified
in the entry).
The commit values are added as the keys to a dict, and with the proper
fields as the values. The mapname function takes the mapping object and
the commit author field and attempts to look for a corresponding entry.
To do so we try (commit name, commit email) first, and if no results are
returned then (None, commit email) is also looked up. This is due to
format 4 from above, where someone may have a mailmap entry with both
name and email, and if they don't it is possible they have an entry that
uses only the commit email.
Differential Revision: https://phab.mercurial-scm.org/D2904
Kyle Lippincott <spectral@google.com> [Fri, 30 Mar 2018 12:16:46 -0700] rev 37211
extdiff: document that it copies modified files back to working directory
Differential Revision: https://phab.mercurial-scm.org/D2976
Gregory Szorc <gregory.szorc@gmail.com> [Fri, 30 Mar 2018 13:39:06 -0700] rev 37210
zope: force module import by importing symbols
Previously, we tried to import a module and handle the ImportError.
Our lazy module importer doesn't verify the module exists and returns
a dummy object representing the module. Only once we attempt to load
a symbol in the module does the ImportError get raises.
This means that simple imports inside `try..except ImportError` don't
detect missing modules.
This commit changes imports in zope.interface to access symbols, thus
forcing module load and triggering ImportError.
This fixes zope.interface for pure builds.
Differential Revision: https://phab.mercurial-scm.org/D2980
Danny Hooper <hooper@google.com> [Fri, 30 Mar 2018 13:17:20 -0700] rev 37209
fix: remove unused imports
Differential Revision: https://phab.mercurial-scm.org/D2977
Pulkit Goyal <7895pulkit@gmail.com> [Tue, 27 Mar 2018 17:26:43 +0530] rev 37208
infinitepush: introduce server option to route every push to bundlestore
This patch introduces a new config option for server
`infinitepush.pushtobundlestore` which if sets to True, the server will route
each incoming push to the bundlestore and store all the parts i.e. changegroups,
phases, obsmarkers in the bundlestore and won't be applied to the revlog.
This config option does not need any client side wrapping and does not need any
custom bundle2 part or stream level parameter to decide where the push should
go.
This is very useful for Mozilla CI use case where they have a central server
that recieves pushes to trigger code-reviews, trigger a test run of CI, run
static analysis etc. The server using the new config option can stash standalone
bundles to the bundlestore and server can get access to individual revisions on
demand.
A new test file which has related tests are added for the config option.
Differential Revision: https://phab.mercurial-scm.org/D2958
Pulkit Goyal <7895pulkit@gmail.com> [Tue, 27 Mar 2018 17:19:02 +0530] rev 37207
infinitepush: don't wrap bundle2.processparts while calling `hg unbundle`
This patch adds dirty logic to check whether we are processing `hg unbundle`
instead of an `hg incoming` to prevent the wrapping of bundle2.processparts
function.
Differential Revision: https://phab.mercurial-scm.org/D2957
Pulkit Goyal <7895pulkit@gmail.com> [Tue, 27 Mar 2018 14:15:42 +0530] rev 37206
inifnitepush: use utils.{stringutil|procutil}.* instead of util.*
Recently in core, util.py is splitted into various modules in mercurial/utils/.
Differential Revision: https://phab.mercurial-scm.org/D2956
Pulkit Goyal <7895pulkit@gmail.com> [Fri, 30 Mar 2018 12:50:06 -0700] rev 37205
infinitepush: delete the non-forward-move flag for hg push
Differential Revision: https://phab.mercurial-scm.org/D2955
Pulkit Goyal <7895pulkit@gmail.com> [Wed, 21 Mar 2018 19:23:10 +0530] rev 37204
infinitepush: drop the wrapping of update command on client side
The extension wraps update command to client side to add functionality to pull
and checkout to a revision if it does not present locally. There is a twist to
that, only changesets which can resolve to remotenames can be pulled using this.
Differential Revision: https://phab.mercurial-scm.org/D2954
Pulkit Goyal <7895pulkit@gmail.com> [Fri, 30 Mar 2018 12:49:50 -0700] rev 37203
infinitepush: delete infinitepushcommands.py and related tests
This patch drops infinitepushcommands.py and the tests which were related to the
command `debugfillinfinitepushmetadata` introduced in the commit.
The patch also drops a config option which was related to
debuginfinitepushmetadata command.
Differential Revision: https://phab.mercurial-scm.org/D2953
Pulkit Goyal <7895pulkit@gmail.com> [Tue, 27 Mar 2018 18:04:05 +0530] rev 37202
infinitepush: delete infinitepush.fillmetadatabranchpattern config option
This patch deletes the config option infinitepush.fillmetadatabranchpattern
which if set to true sets a background process which will save metadata in
infinitepush index.
This series is meant to have a state where we can use it for CI purposes.
Differential Revision: https://phab.mercurial-scm.org/D2952
Pulkit Goyal <7895pulkit@gmail.com> [Fri, 09 Feb 2018 15:49:46 +0530] rev 37201
infinitepush: drop the `--to` flag to push and use `-B` instead
The extension added a `--to` flag to specify the bookmark to which revs should
be pushed. This patch deletes that flag and instead uses the `-B` flag. After
this patch, bookmark passed as `-B` is parsed and if it matches the infinitepush
bookmark pattern, we consider that push as infinitepush.
This is still not the best of what we can do. Later patches in the series will
drop the use of `-B` flag and will instead handle things at bookmark bundle2
part. Plugging these logic to bookmark bundle2 part will also get rid of the
scratchbranchparttype bundle2 part.
Differential Revision: https://phab.mercurial-scm.org/D2108
Pulkit Goyal <7895pulkit@gmail.com> [Fri, 09 Feb 2018 14:16:03 +0530] rev 37200
infinitepush: drop the `--create` flag to push command
The extension added a `--create` flag for creating a new bookmark on the
bundlestore. This patch changes the bahviour to create a bookmark if it does not
exists and removes the requirement of the `--create` and then drop the logic
around the `--create` flag.
Tests are changed to drop the usage of `--create` flag.
Differential Revision: https://phab.mercurial-scm.org/D2107
Pulkit Goyal <7895pulkit@gmail.com> [Fri, 09 Feb 2018 14:02:03 +0530] rev 37199
infinitepush: drop logic related to treemanifest extension
treemanifest is another facebook's extension which is not in core. Let's drop
the logic related to infinitepush extension which is now ported to core.
Differential Revision: https://phab.mercurial-scm.org/D2106
Pulkit Goyal <7895pulkit@gmail.com> [Fri, 09 Feb 2018 13:56:09 +0530] rev 37198
infinitepush: drop hack related to --to, --create flags of remotenames-ext
The remotenames extension has --to, --create. --allow-anon flags which won't be
ported to core. This patch drops the hack which fixes the interaction with
remotenames-ext during push.
In upcoming patches, --to and --create extensions will be removed from
infinitepush also.
Differential Revision: https://phab.mercurial-scm.org/D2105
Pulkit Goyal <7895pulkit@gmail.com> [Fri, 09 Feb 2018 13:50:16 +0530] rev 37197
infinitepush: drop error handling while pushing to svn server
This is something which is internal to Facebook and we don't want in core.
Differential Revision: https://phab.mercurial-scm.org/D2104
Pulkit Goyal <7895pulkit@gmail.com> [Fri, 09 Feb 2018 13:36:50 +0530] rev 37196
infinitepush: drop `--list-remote`, `--remote-path` flags to bookmark cmd
The extension adds these two flags to list remote bookmarks using the `hg
bookmark` command. These are not required in core currently and needs some
discussion before getting them in, so let's drop them for now. This makes us
deleting the wrapping of `hg bookmark` command on client side.
The end goal here is to have minimal or no wrapping at client side.
Differential Revision: https://phab.mercurial-scm.org/D2103
Pulkit Goyal <7895pulkit@gmail.com> [Thu, 08 Feb 2018 17:27:47 +0530] rev 37195
infinitepush: drop the scratchbookmarksparttype bundle2 part
The scratchbookmarksparttype bundle2 part was used in backupcommands.py which we
deleted in an earlier changeset. We don't need this part anymore. Moreover we
now have bookmarks bundle2 part in core which we can use. This patch also drops
the related encoding and decoding functions.
Differential Revision: https://phab.mercurial-scm.org/D2102
Pulkit Goyal <7895pulkit@gmail.com> [Thu, 08 Feb 2018 17:18:28 +0530] rev 37194
infinitepush: remove wrapping around bundle2._addpartsfromopts()
This wrapping around bundle2._addpartsfromopts() was added by me while hacking
on an alternate way to mark a push as infinitepush or not. However the wrapping
is wrong as the push command does not go through the code path which was wrapped
on the client side.
Differential Revision: https://phab.mercurial-scm.org/D2101
Pulkit Goyal <7895pulkit@gmail.com> [Thu, 08 Feb 2018 17:12:53 +0530] rev 37193
infinitepush: delete the bundle2 part ordering related to pushrebase
pushrebase is another Facebook extension which does not live in core yet. So
let's remove the logic which make sures infinitepush part is moved before the
pushrebase one.
Differential Revision: https://phab.mercurial-scm.org/D2100
Pulkit Goyal <7895pulkit@gmail.com> [Thu, 08 Feb 2018 17:09:24 +0530] rev 37192
infinitepush: delete the hack to load the extension after remotenames
The remotenames extension which circumvents the default push implementation
lives outside the core, so let's delete logic related to loading of extension on
the basis of remotenames ext.
Differential Revision: https://phab.mercurial-scm.org/D2099
Pulkit Goyal <7895pulkit@gmail.com> [Sat, 03 Feb 2018 17:59:49 +0530] rev 37191
infinitepush: drop the logic related to inhibit extension
The inhibit extension lives in hg-experimental and is not a core extension.
Differential Revision: https://phab.mercurial-scm.org/D2098
Pulkit Goyal <7895pulkit@gmail.com> [Sat, 03 Feb 2018 17:54:55 +0530] rev 37190
infinitepush: remove backupcommands.py
This file contains command pushbackup and pullbackup which are used to store
things to infinitepush.
This is an advanced functionality which we don't require as of now. Also this
uses shareutil.py from fb-hgext/ which is not there in core. Therefore this
patch deletes the file and the config option which belongs to the backup thing.
If we need this functionality in future, we can always move this file back.
Differential Revision: https://phab.mercurial-scm.org/D2097
Pulkit Goyal <7895pulkit@gmail.com> [Fri, 09 Feb 2018 13:39:15 +0530] rev 37189
infinitepush: move the extension to core from fb-hgext
This patch moves the infinitepush extension from fb-hgext to core. The
extension is used to store incoming bundles during a push in bundlestore rather
than applying them to the revlog.
The extension was copied from the repository revision at
f27f094e91553d3cae5167c0b1c42ae940f888d5 and following changes were made:
* added `from __future__ import absolute_import` where missing
* fixed module imports to follow the core style
* minor fixes for test-check-code.t
* registered the configs
* adding the testedwith value to match core's convention
* removed double newlines to make test-check-commit.t happy
* added one line doc about extension and marked it as experimental
Only one test file test-infinitepush-bundlestore.t is moved to core and
following changes are made to file:
* remove dependency of library.sh
* split the tests into two tests i.e. test-infinitepush.t and
test-infinitepush-bundlestore.t
* removed testing related to other facebook's extensions pushrebase, inhibit,
fbamend
library-infinitepush.sh is also copied from fb-hgext from the same revision and
following changes are made:
* change the path to infinitepush extension as it's in core with this patch
* removed sql handling from the file as we are not testing that initially
Currently at this revision, test-check-module-imports.t does not pass as there
is import of a module from fb/hgext in one the of the file which will be removed
in the next patch.
This extension right now has a lot of things which we don't require in core like
`--to`, `--create` flags to `hg bookmark`, logic related to remotenames
extension and another facebook's extensions, custom bundle2parts which can be
prevented by using bookmarks bundle part and also logic related to sql store
which is probably we don't want initially.
The next patches in this series will remove all the unwanted and unrequired
things from the extension and will make this a nice one.
The end goal is to have a very lighweight extension with no or very less
wrapping on the client side.
Differential Revision: https://phab.mercurial-scm.org/D2096
Martin von Zweigbergk <martinvonz@google.com> [Fri, 30 Mar 2018 11:15:58 -0700] rev 37188
narrow: use repo.local() instead of isinstance()
Differential Revision: https://phab.mercurial-scm.org/D2973
Gregory Szorc <gregory.szorc@gmail.com> [Fri, 30 Mar 2018 11:33:21 -0700] rev 37187
tests: add zope.interface to clang-format ignore list
Differential Revision: https://phab.mercurial-scm.org/D2975
Gregory Szorc <gregory.szorc@gmail.com> [Fri, 30 Mar 2018 11:35:17 -0700] rev 37186
contrib: rename clang-format-blacklist to clang-format-ignorelist
"blacklist" is racially charged. Let's rename it to something
that isn't.
Differential Revision: https://phab.mercurial-scm.org/D2974
Danny Hooper <hooper@google.com> [Sat, 03 Mar 2018 14:08:44 -0800] rev 37185
fix: new extension for automatically modifying file contents
This change implements most of the corresponding proposal as discussed at the
4.4 and 4.6 sprints: https://www.mercurial-scm.org/wiki/AutomaticFormattingPlan
This change notably does not include parallel execution of the formatter/fixer
tools. It does allow for implementing that without affecting other areas of the
code.
I believe the test coverage to be good, but this is a hotbed of corner cases.
Differential Revision: https://phab.mercurial-scm.org/D2897
Gregory Szorc <gregory.szorc@gmail.com> [Fri, 30 Mar 2018 11:11:42 -0700] rev 37184
tests: ignore zope packages when running pyflakes
Differential Revision: https://phab.mercurial-scm.org/D2972
Gregory Szorc <gregory.szorc@gmail.com> [Wed, 21 Mar 2018 19:48:36 -0700] rev 37183
repository: define interface for local repositories
Per discussions on the mailing list and at the 4.4 and 4.6 sprints,
we want to start defining interfaces for local repository primitives
so that we a) have a better idea of what the formal interface for
various types is b) can more easily introduce alternate implementations
of various components (e.g. in Rust).
We have previously implemented interfaces that declare the peer and
wire protocol APIs using the abc module.
This commit introduces a monolithic interface for the localrepository
class. It uses zope.interface - not abc - for defining and declaring
the interface.
The newly defined "completelocalrepository" interface is objectively
horrible. It is based on what is actually in localrepository and
doesn't represent a reasonable interface definition IMO. There's lots
of... unwanted garbage in the interface. In other words, it reflects
the horrible state of the localrepository "god object." But this is
fine: a goal of this commit is to get the interface defined so that
we have an interface. Future commits can refactor the interface
into sub-interfaces, remove unwanted public attributes, etc.
I attempted to define reasonable docstrings for the various interface
members. But there are so many of them and I didn't know what some are
used for. So I was lazy in a number of places and didn't write
docstrings or detailed usage docs.
Also, the members of the interface are defined in the order they are
declared in localrepo.py. This revealed that the grouping of things
in localrepo.py is... odd.
The localrepository class now declares that it implements our newly
defined interface. Unlike abc, zope.interface doesn't check interface
conformance at type creation time (abc uses __metaclass__ magic to
validate interface conformance when a type is created - usually at
module import time). It does provide some functions for validating
class and object conformance with declared interfaces. We add these
checks to test-check-interfaces.py. We /could/ validate at run-time.
But we hold off for now. (I'm a bit scared of doing that because of
the various ways extensions monkeypatch repo instances.)
After this commit, test-check-interfaces.py will fail if the set of
public attributes on the localrepository class or instances change
without corresponding updates to the interface. This is by design.
Differential Revision: https://phab.mercurial-scm.org/D2933
Gregory Szorc <gregory.szorc@gmail.com> [Wed, 21 Mar 2018 18:32:47 -0700] rev 37182
setup: register zope.interface packages and compile C extension
With this change, we should be able to use zope.interface in core!
Differential Revision: https://phab.mercurial-scm.org/D2932
Gregory Szorc <gregory.szorc@gmail.com> [Wed, 21 Mar 2018 18:13:40 -0700] rev 37181
thirdparty: allow zope.interface.advice to be lazily imported
The symbol from this module is only used in functions. Let's access
that symbol through its imported module so importing
zope.interface.advice can be lazy.
Differential Revision: https://phab.mercurial-scm.org/D2931
Gregory Szorc <gregory.szorc@gmail.com> [Wed, 21 Mar 2018 19:52:30 -0700] rev 37180
thirdparty: port zope.interface to relative imports
By using relative imports, we're guaranteed to get modules
vendored with Mercurial rather than other random modules
that might be in sys.path.
My editor strips trailing whitespace on save. So some minor
source code cleanup was also performed as part of this commit.
# no-check-commit because some modified lines have double newlines
Differential Revision: https://phab.mercurial-scm.org/D2930
Gregory Szorc <gregory.szorc@gmail.com> [Wed, 21 Mar 2018 19:49:07 -0700] rev 37179
thirdparty: don't make zope a namespace package
There are a gazillion zope.* packages in the wild. So zope/__init__.py
needs to be a namespace package. But in Mercurial, we have 1 zope
package. And even if we had multiple packages, they'd all be in
thirdparty/zope/. So we don't need a namespace package.
Differential Revision: https://phab.mercurial-scm.org/D2929
Gregory Szorc <gregory.szorc@gmail.com> [Wed, 21 Mar 2018 19:48:50 -0700] rev 37178
thirdparty: vendor zope.interface 4.4.3
I've been trying to formalize interfaces for various components
of Mercurial. So far, we've been using the "abc" package. This
package is "good enough" for a lot of tasks. But it quickly
falls over. For example, if you declare an @abc.abstractproperty,
you must implement that attribute with a @property or the class
compile time checking performed by abc will complain. This often
forces you to implement dumb @property wrappers to return a
_ prefixed attribute of the sane name. That's ugly.
I've also wanted to implement automated checking that classes
conform to various interfaces and don't expose other "public"
attributes.
After doing a bit of research and asking around, the general
consensus seems to be that zope.interface is the best package for
doing interface-based programming in Python. It has built-in
support for verifying classes and objects conform to interfaces.
It allows an interface's properties to be defined during __init__.
There's even an "adapter registry" that allow you to register
interfaces and look up which classes implement them. That could
potentially be useful for places where our custom registry.py
modules currently facilitates central registrations, but at a
type level. Imagine extensions providing alternate implementations
of things like the local repository interface to allow opening
repositories with custom requirements.
Anyway, this commit vendors zope.interface 4.4.3. The contents of
the source tarball have been copied into mercurial/thirdparty/zope/
without modifications.
Test modules have been removed because they are not interesting
to us.
The LICENSE.txt file has been copied so it lives next to the
source.
The Python modules don't use relative imports. zope/__init__.py
defines a namespace package. So we'll need to modify the source
code before this package is usable inside Mercurial. This will
be done in subsequent commits.
# no-check-commit for various style failures
Differential Revision: https://phab.mercurial-scm.org/D2928
Martin von Zweigbergk <martinvonz@google.com> [Thu, 29 Mar 2018 23:05:41 -0700] rev 37177
context: set repo property in basectx
It seems like a good practice to call the super constructor. Let's
start by passing the repo along to basectx so it can assign it to a
private attribute. We should perhaps pass the rev and node along as
well, but that requires more work before it can be done.
Differential Revision: https://phab.mercurial-scm.org/D2970
Martin von Zweigbergk <martinvonz@google.com> [Thu, 29 Mar 2018 22:51:45 -0700] rev 37176
context: move reuse of context object to repo.__getitem__ (API)
As an example of how weird the basectx.__new__ is: whenever you create
a workingctx, basectx.__new__ gets called first. Since our __new__ has
a "changeid" argument as second parameter, when create the
workingctx(repo, text="blah"), the text gets bound to
"changeid". Since a string isn't a basectx, our __new__ ends up not
doing anything funny, but that's still very confusing code.
Another case is metadataonlyctx.__new__(), which I think exists in
order to prevent metadataonlyctx.__init__'s third argument
(originalctx) from being interpreted as a changeid in
basectx.__new__(), thereby getting reused.
Let's move this to repo.__getitem__ instead, where it will be pretty
obvious what the code does.
After this patch, changectx(ctx) will be an error (it will fail when
trying to see if it's a 20-byte string).
Differential Revision: https://phab.mercurial-scm.org/D2969
Martin von Zweigbergk <martinvonz@google.com> [Thu, 29 Mar 2018 22:22:51 -0700] rev 37175
memctx: create parent contexts using "repo[p]" syntax
I want to reduce dependence on basectx.__new__() and move that code
over to repo.__getitem__().
Differential Revision: https://phab.mercurial-scm.org/D2968
Martin von Zweigbergk <martinvonz@google.com> [Thu, 29 Mar 2018 21:29:15 -0700] rev 37174
context: avoid using a context object as a changeid
I find it misleading to pass changeid=changectx. It currently works to
do that because there's weird (IMO) handling of it in
basectx.__new__. I'm planning on removing that code. Passing changeid
as "changeid" and context as "context" makes it more readable.
Note that the documentation of filectx.__init__ doesn't even seem to
be aware that a changeid can be a context ("changeset revision, node,
or tag").
Differential Revision: https://phab.mercurial-scm.org/D2967
Martin von Zweigbergk <martinvonz@google.com> [Thu, 29 Mar 2018 21:15:40 -0700] rev 37173
context: remove unwanted assignments in basectx.__new__() (API)
The two subclasses in core apparently didn't trust __new__() to do the
job anyway (they both reassigned all the properties after).
Differential Revision: https://phab.mercurial-scm.org/D2966
Boris Feld <boris.feld@octobus.net> [Wed, 31 Jan 2018 17:15:44 +0100] rev 37172
streamclonebundle: make sure we accept new stream clone bundle spec
When asked specifically, the code do a sanity check on the clone bundle to
ensure it's a stream bundle. As we introduced a new stream bundle spec, update
the logic to support it.
With this final changeset, we can now announce safely a stream v2 clone bundle
and old clients would not crash trying to process it.
This changeset address the last comment from Gregory Szorc on the stream v2
bundle series.
Differential Revision: https://phab.mercurial-scm.org/D1957
Boris Feld <boris.feld@octobus.net> [Wed, 31 Jan 2018 11:44:33 +0100] rev 37171
streamclonebundle: add a test for stream clone bundle v2
Differential Revision: https://phab.mercurial-scm.org/D1956
Boris Feld <boris.feld@octobus.net> [Wed, 31 Jan 2018 11:10:55 +0100] rev 37170
bundlespec: add support for some variants
This way the stream v2 bundle spec can disable the changegroup part while
enabling the stream v2 part.
Differential Revision: https://phab.mercurial-scm.org/D1955
Boris Feld <boris.feld@octobus.net> [Wed, 31 Jan 2018 11:09:20 +0100] rev 37169
bundle: add the possibility to bundle a stream v2 part
Differential Revision: https://phab.mercurial-scm.org/D1954
Boris Feld <boris.feld@octobus.net> [Wed, 31 Jan 2018 10:55:15 +0100] rev 37168
streambundlev2: add a new test-file
Add the new test file in a separate changeset before supporting the new format
so we better see the differences.
Differential Revision: https://phab.mercurial-scm.org/D1953
Boris Feld <boris.feld@octobus.net> [Fri, 30 Mar 2018 12:43:57 +0200] rev 37167
bundlespec: move computing the bundle contentops in parsebundlespec
We will introduce a new bundlespec for stream bundle which will influence the
contentops.
Differential Revision: https://phab.mercurial-scm.org/D1952
Boris Feld <boris.feld@octobus.net> [Fri, 30 Mar 2018 12:43:08 +0200] rev 37166
bundlespec: introduce an attr-based class for bundlespec
We will add support of contentops in the next patch, introduce a class instead
of returning a 4-items tuple.
Differential Revision: https://phab.mercurial-scm.org/D2971
Yuya Nishihara <yuya@tcha.org> [Fri, 23 Mar 2018 20:43:55 +0900] rev 37165
templater: factor out unwrapastype() from evalastype()
So ParseError of unwrapastype() can be caught reliably.
Yuya Nishihara <yuya@tcha.org> [Fri, 23 Mar 2018 20:34:12 +0900] rev 37164
templater: extract unwrapinteger() function from evalinteger()
Yuya Nishihara <yuya@tcha.org> [Fri, 23 Mar 2018 21:03:21 +0900] rev 37163
templater: extract type conversion from evalfuncarg()
Needed by the subsequent patches.
Yuya Nishihara <yuya@tcha.org> [Fri, 23 Mar 2018 20:23:55 +0900] rev 37162
templater: drop bool support from evalastype()
Future patches will split evalastype() into two functions, evalrawexp()
and unwrapastype(), so we can catch the exception of type conversion.
# evaluating function may bubble up inner ParseError
thing = evalrawexp(context, mapping, arg)
try:
return unwrapastype(context, thing)
except ParseError:
# add hint and reraise
However, evalboolean() can't be factored out in this way since it has to
process boolean-like symbols as non keyword. Fortunately, it's unlikely
that we'll need a general type converter supporting bool, so this patch
drops it from the table.
Yuya Nishihara <yuya@tcha.org> [Sat, 17 Mar 2018 20:13:06 +0900] rev 37161
templater: do not use stringify() to concatenate flattened template output
Yuya Nishihara <yuya@tcha.org> [Sat, 17 Mar 2018 20:04:20 +0900] rev 37160
templateutil: reimplement stringify() using flatten()
Yuya Nishihara <yuya@tcha.org> [Sat, 17 Mar 2018 20:00:54 +0900] rev 37159
templateutil: move flatten() from templater
It's the same kind of utility as stringify().
Connor Sheehan <sheehan@mozilla.com> [Tue, 27 Mar 2018 11:01:13 -0400] rev 37158
stringutil: move person function from templatefilters
Move the person function from template filters to the stringutil
module, so it can be reused in the mailmap template function.
Differential Revision: https://phab.mercurial-scm.org/D2960
Connor Sheehan <sheehan@mozilla.com> [Thu, 22 Mar 2018 09:48:22 -0400] rev 37157
stringutil: add isauthorwellformed function
The regular expression for this function formerly lived at
https://hg.mozilla.org/hgcustom/version-control-tools/file/tip/hghooks/mozhghooks/author_format.py#l13
Differential Revision: https://phab.mercurial-scm.org/D2959
Matt Harbison <matt_harbison@yahoo.com> [Sat, 17 Mar 2018 02:37:46 -0400] rev 37156
test-lfs-test-server: add a testcase for `hg serve`
I haven't figured out yet how to make the authentication checks work for a
specific list of users, so the 'web.allow-push' list is wildcarded. (It appears
that the client doesn't react to a 401 by sending authentication data, which may
be caused in part by not having all of the headers in httpbasicauthhandler's
http_error_auth_reqed(), compared to a run of test-http.t. But in any case, we
should probably have a separate set of tests for various authentication
scenarios. As it is, without the wildcard, no push access is granted.)
There are several deviations from the `lfs-test-server` case:
- `hg serve` emits a Server header. I think Gregory indicated that this isn't
easily suppressed.
- `hg serve` names the "basic" transfer handler in the Batch API response. Not
having to specify it was for backwards compatability, so this seems like the
right thing to do. (`lfs-test-server` doesn't name it, whether it was
explicitly requested by the client or not.)
- PUT status for a newly created file is 201, per RFC-2616 [1]. The Basic
Transfer API [2] shows an example upload transcript with a 200 response. It
doesn't make much sense to re-upload a file (unless it is corrupt) in an
example, but I wouldn't be surprised if some other implementations also
expect 200 because of this. But the RFC says MUST use 201 for creation.
- The Content-Type for the file transfers is "application/octet-stream", like
the sample transcript (though I don't see it explicitly called out in the
text elsewhere). Using "text/plain" seems clearly wrong.
- `lfs-test-server` isn't removing the action property and sending back an
error code like the spec calls out when a file is missing or corrupt. Doing
so on the `hg serve` side reveals a bug in our client code when handling the
response- it indicates the remote file is missing instead of corrupt around
line 452.
I'll probably glob over the Content-Length differences once this settles down.
Prior to the recent hgweb refactoring, the Batch API response was using chunked
encodings instead.
Back to the RFC, I have no idea if the python framework handles the "MUST NOT
ignore any Content-* (e.g. Content-Range) headers that it does not understand or
implement and MUST return a 501" for a PUT request.
[1] https://www.w3.org/Protocols/rfc2616/rfc2616-sec9.html#sec9.6
[2] https://github.com/git-lfs/git-lfs/blob/master/docs/api/basic-transfers.md#uploads
Martin von Zweigbergk <martinvonz@google.com> [Thu, 29 Mar 2018 09:23:39 -0700] rev 37155
push: use "repo['.']" instead of old form "repo['']"
Note that this does not conflict with the commit message of my
previous patch: I found this after writing the previous patch
(besides, I very easily forget things).
Differential Revision: https://phab.mercurial-scm.org/D2965
Martin von Zweigbergk <martinvonz@google.com> [Thu, 29 Mar 2018 09:34:17 -0700] rev 37154
context: change default changeid from old form '' to '.'
I don't think I've seen repo[''] in the codebase at least in the last
two years.
Differential Revision: https://phab.mercurial-scm.org/D2964
Sushil khanchi <sushilkhanchi97@gmail.com> [Wed, 21 Mar 2018 23:36:11 +0530] rev 37153
remove: add dry-run functionality
Matt Harbison <matt_harbison@yahoo.com> [Sat, 17 Mar 2018 01:53:44 -0400] rev 37152
lfs: add support for serving blob files
Matt Harbison <matt_harbison@yahoo.com> [Sat, 17 Mar 2018 01:47:57 -0400] rev 37151
lfs: add server side support for the Batch API
Matt Harbison <matt_harbison@yahoo.com> [Sat, 17 Mar 2018 01:23:01 -0400] rev 37150
lfs: add basic routing for the server side wire protocol processing
The recent hgweb refactoring yielded a clean point to wrap a function that could
handle this, so I moved the routing for this out of the core. While not an hg
wire protocol, this seems logically close enough. For now, these handlers do
nothing other than check permissions.
The protocol requires support for PUT requests, so that has been added to the
core, and funnels into the same handler as GET and POST. The permission
checking code was assuming that anything not checking 'pull' or None ops should
be using POST. But that breaks the upload check if it checks 'push'. So I
invented a new 'upload' permission, and used it to avoid the mandate to POST. A
function wrap point could be added, but security code should probably stay
grouped together. Given that anything not 'pull' or None was requiring POST,
the comment on hgweb.common.permhooks is probably wrong- there is no 'read'.
The rationale for the URIs is that the spec for the Batch API[1] defines the URL
as the LFS server url + '/objects/batch'. The default git URLs are:
Git remote: https://git-server.com/foo/bar
LFS server: https://git-server.com/foo/bar.git/info/lfs
Batch API: https://git-server.com/foo/bar.git/info/lfs/objects/batch
'.git/' seems like it's not something a user would normally track. If we adhere
to how git defines the URLs, then the hg-git extension should be able to talk to
a git based server without any additional work.
The URI for the transfer requests starts with '.hg/' to ensure that there are no
conflicts with tracked files. Since these are handed out by the Batch API, we
can change this at any point in the future. (Specifically, it might be a good
idea to use something under the proposed /api/ namespace.) In any case, no
files are stored at these locations in the repository directory.
I started a new module for this because it seems like a good idea to keep all of
the security sensitive server side code together. There's also an issue with
`hg verify` in that it will want to download *all* blobs in order to run.
Sadly, there's no way in the protocol to ask the server to verify the content of
a blob it may have. (The verify action is for storing files on a 3rd party
server, and then informing the LFS server when that completes.) So we may end
up implementing a custom transfer adapter that simply indicates if the blobs are
valid, and fall back to basic transfers for non-hg servers. In other words,
this code is likely to get bigger before this is made non-experimental.
[1] https://github.com/git-lfs/git-lfs/blob/master/docs/api/batch.md
Matt Harbison <matt_harbison@yahoo.com> [Wed, 14 Mar 2018 23:34:08 -0400] rev 37149
test-lfs: drop trailing ', ' item separators from debug JSON output
The trailing space looks weird when conditionalizing the line. The commas
shouldn't be necessary because of the indenting. The `lfs-test-server` isn't
sending all of the same items (notably, the "transfer" attribute is missing), so
having the commas means more lines need to be conditionalized.
Matt Harbison <matt_harbison@yahoo.com> [Thu, 22 Feb 2018 23:33:52 -0500] rev 37148
lfs: add a blob verification method to the local store
A corrupt blob can be signaled through the Batch API response, without actually
transferring the file. A true/false indicator is slightly easier than
immediately catching an exception.
Matt Harbison <matt_harbison@yahoo.com> [Wed, 28 Mar 2018 22:38:01 -0400] rev 37147
tests: conditionalize printed environment variable output in test-alias
Martin von Zweigbergk <martinvonz@google.com> [Wed, 28 Mar 2018 15:31:18 -0700] rev 37146
debugsetparents: avoid using "r1/r2" variable names for nodeids
Differential Revision: https://phab.mercurial-scm.org/D2963
Martin von Zweigbergk <martinvonz@google.com> [Wed, 28 Mar 2018 15:29:49 -0700] rev 37145
tag: avoid using "r" variable name for nodeid
Differential Revision: https://phab.mercurial-scm.org/D2962
Martin von Zweigbergk <martinvonz@google.com> [Wed, 28 Mar 2018 15:30:09 -0700] rev 37144
locate: avoid using "rev" variable name for nodeid
Also, drop silly "ctx = repo[ctx.node()]".
Differential Revision: https://phab.mercurial-scm.org/D2961
Yuya Nishihara <yuya@tcha.org> [Wed, 28 Mar 2018 22:06:47 +0900] rev 37143
py3: bytes/unicode dance on __doc__ of cmdalias