sidedata: adjust string for python3 This was pointed out by Gregory Szorc: Python 3 will require a r'' because of source transformation. Differential Revision: https://phab.mercurial-scm.org/D6968

py3: fix phabricator's use of json.loads() for py3.5 Missed this in c340a8ac7ef3 since `loads()` takes bytes from 3.6 onwards. Differential Revision: https://phab.mercurial-scm.org/D6961

run-tests: make coverage work out of tree Currently coverage fails when run on an out-of-tree extension since run-tests.py tries to load sitecustomize.py from self._testdir, which is the dir for the extension's tests. Differential Revision: https://phab.mercurial-scm.org/D6960

unfinishedstate: suggested `hg update .` (including `.`) to complete update `hg update` can update to a different and undesired commit. For users who have commands.update.requiredest=yes, it's even an error to run just `hg update. Differential Revision: https://phab.mercurial-scm.org/D6956

setup: build extensions in parallel by default The build_ext distutils command in Python 3.5+ has a "parallel" option that controls whether to build extensions in parallel. It is disabled by default (None) and can be set to an integer value for number of cores or True to indicate use all available CPU cores. This commit changes our build_ext command override to set "parallel" to True unless a value has been provided by the caller. On my machine, this makes `python setup.py build_ext` 1-4s faster. It is worth noting that at this time, each individual source file constituting the extension is still built serially. For Mercurial, this means that we can't build faster than the slowest-to-build extension, which is the zstd extension by a long shot. This means that setup.py is still not very efficient at utilizing multiple cores. But we're better than before. Differential Revision: https://phab.mercurial-scm.org/D6923 # no-check-commit because of foo_bar naming

merge with stable

sidedata: check that the sidedata safely roundtrip We actually use the data we stored during the test. It is a way to make sure their content is not corrupted. Differential Revision: https://phab.mercurial-scm.org/D6898

sidedata: add a debugsidedata command The command provide basic capabilities to inspect revision side data. Differential Revision: https://phab.mercurial-scm.org/D6897

sidedata: test we can successfully write sidedata For now we just write them and rejoice on the lack of crashes. Differential Revision: https://phab.mercurial-scm.org/D6896

sidedata: make sure we don't use the flag if there are not sidedata Ensuring this at this level seems safer than relying on caller doing the right thing. G: changed mercurial/revlog.py Differential Revision: https://phab.mercurial-scm.org/D6895

revlog: add the appropriate flag is sidedata are passed to `addrevision` If we need to store sidedata, we need the flag to be set and the associated processing to be called. Differential Revision: https://phab.mercurial-scm.org/D6894

sidedata: register the flag processors if the repository allows for it Differential Revision: https://phab.mercurial-scm.org/D6893

sidedata: add a function for _processflagsraw usage Entry with side data cannot directly validate their rawtext Differential Revision: https://phab.mercurial-scm.org/D6892

sidedata: add a function to write sidedata into a raw text Differential Revision: https://phab.mercurial-scm.org/D6891

sidedata: add a function to read sidedata from revlog raw text This implement the "reading" part of a `sidedata` flag processor. Differential Revision: https://phab.mercurial-scm.org/D6890

sidedata: add a new module with basic documentation For now the storage strategy is very simple. We can augment it in the future if needed. Code to actually support what is described will be introduced in later changesets. Differential Revision: https://phab.mercurial-scm.org/D6889

sidedata: add a new revision flag constant for side data Small step first. Differential Revision: https://phab.mercurial-scm.org/D6888

upgrade: detect the side-data format variants Note that for now we cannot upgrade/downgrade to it. Differential Revision: https://phab.mercurial-scm.org/D6887

sidedata: introduce a new requirement to protect the feature The feature will be usable only within repository supported by version of Mercurial with the support. Since the support is not fully implemented yet, the requirement has a temporary, experimental name. Differential Revision: https://phab.mercurial-scm.org/D6886

Added signature for changeset 181e52f2b62f

Added tag 5.1.2 for changeset 181e52f2b62f

copies: move file input processsing early If we are to store the same kind of data outside of extra, we need to explicitly prepare them before that. On the long run, other storage (eg: sidedata) might use a different encoding to store this information, since the constraint from extra does not apply to it. Differential Revision: https://phab.mercurial-scm.org/D6937

copies: prepare changelog for more copies storage mode We want to store copies information in sidedata, the logic will be quite similar to the one used for extra, but sightly different (not in extra, no hash impact). We start with small refactoring to make our work easier and the coming changesets cleaners. Differential Revision: https://phab.mercurial-scm.org/D6936

vfs: give all vfs an options attribute by default Multiple code path assume vfs have an options attribute, including the vfs module itself. So for consistency we explicitly add one to all vfs. This will prevent various crash in the next changesets. Differential Revision: https://phab.mercurial-scm.org/D6935

revlog: explicitly set revlogv0 in vfs options Relying on having an attribute or not for something so fundamental seems too fragile to me. (And indeed I had issue with that later in this series). So we explicitly record the fact the repository use revlog-v0. Differential Revision: https://phab.mercurial-scm.org/D6934

context: clarify the various mode in the _copies property cache The previous code was compact but a bit dense. The new proposed code deal with each mode separately, there are some duplicated lines, but the meaning of each mode stand out. One of the benefit it to make it simpler to add further mode in the future. Differential Revision: https://phab.mercurial-scm.org/D6933

context: clarify the various mode in the filesremoved method The previous code was compact but a bit dense. The new proposed code deal with each mode separately, there are some duplicated lines, but the meaning of each mode stand out. One of the benefit it to make it simpler to add further mode in the future. Differential Revision: https://phab.mercurial-scm.org/D6932

context: clarify the various mode in the filesadded method The previous code was compact but a bit dense. The new proposed code deal with each mode separately, there are some duplicated lines, but the meaning of each mode stand out. One of the benefit it to make it simpler to add further mode in the future. Differential Revision: https://phab.mercurial-scm.org/D6931

copies: expand the logic of usechangesetcentricalgo Using intermediate variable is clearer and will make is simple to expand the logic. Differential Revision: https://phab.mercurial-scm.org/D6930

patchbomb: protect email addresses from shell When patchbomb sends email via a sendmail-like program it invokes this using procutil.popen which passes the string to a shell to be parsed. To protect any special characters in the email addresses on the command line from being interpretered by the shell they must be quoted.

automation: support and use Debian Buster by default While Debian Buster (Debian 10) was released in July, the AWS AMIs were not published until mid September. This commit teaches the automation system to create AMIs for Debian Buster. Since Debian Buster is the new stable Debian release, we make it the default distribution for automation. Differential Revision: https://phab.mercurial-scm.org/D6917

automation: use latest AMIs This cuts down on the overhead to create a new AMI, as we don't have to install as many package upgrades. Differential Revision: https://phab.mercurial-scm.org/D6916

automation: increase size of Linux AMI build volume I ran out of space attempting to build a few AMIs. Let's give the environment an extra 2 GB to utilize. Differential Revision: https://phab.mercurial-scm.org/D6915

automation: install awscli and python3-boto3 packages This provides the `aws` command-line utility as well as a Python library for interacting with AWS. It is useful to have these packages installed in AMIs so you can easily interact with other AWS services. Differential Revision: https://phab.mercurial-scm.org/D6914

automation: upgrade packages in Linux environment Let's stay modern. Differential Revision: https://phab.mercurial-scm.org/D6913

automation: remove Ubuntu 18.10 It reached end of life on July 18, 2019. I don't think we should care about supporting it. Differential Revision: https://phab.mercurial-scm.org/D6912

automation: always install docker-ce Docker has published packages for Ubuntu Disco and we no longer need this one-off. Differential Revision: https://phab.mercurial-scm.org/D6911

automation: use LSB_RELEASE instead of DEBIAN_VERSION This should be more robust since I believe the minor version can change mid release. Differential Revision: https://phab.mercurial-scm.org/D6910

tests: use silenttestrunner in test-simplemerge.py The time monkeypatching doesn't appear to work reliably in Python 3, possibly due to unittest using a different time function. This makes the test intermittent due to the execution time not always being `0.00s`. We have our own wrapper around unittest for more deterministic test output. So let's use it. As a bonus, all test output disappeared, so we can remove the .out file! Differential Revision: https://phab.mercurial-scm.org/D6921

contrib: use pycompat.xrange in bdiff-torture.py For Python 3 compatibility. Differential Revision: https://phab.mercurial-scm.org/D6920

interfaces: use triple quotes for Attribute value Otherwise the source transformer converts it to bytes and test-check-interfaces.py complains on Python 3. Differential Revision: https://phab.mercurial-scm.org/D6919

tests: suppress output from write() Otherwise it is printed and the test fails with tons of output on Python 3. Differential Revision: https://phab.mercurial-scm.org/D6918

tests: look for additional pip warning variant The wording has been tweaked in new versions of pip. Differential Revision: https://phab.mercurial-scm.org/D6909

tests: use test harness `hg` in test-hghave.t This prevents the sub test harness from rebuilding Mercurial. It shaves ~30s from the test wall time on my machine. Differential Revision: https://phab.mercurial-scm.org/D6908

synthrepo: use pycompat.xrange This silences some test-check-pyflakes.t warnings on Python 3. Differential Revision: https://phab.mercurial-scm.org/D6907

perf: fix perfhelper-pathcopies without --stats Some conditional were missing.

upgrade: also register copied `.d` files to fncache Oops, we forgot to register data file when blanky copying revlog. Now that we are actually copying these `.d` files, we should also register them. Differential Revision: https://phab.mercurial-scm.org/D6905

upgrade: fix handling of `.d` data file The test did not used any `.d` file. So of course, their handling was broken (for vfs encoding reason again). We fix the issue and update the test. Note that some fncache issue remains with `.d` file. It will be taken care of in the next changesets. Differential Revision: https://phab.mercurial-scm.org/D6904

upgrade: also copy data file Differential Revision: https://phab.mercurial-scm.org/D6903

upgrade: create the correct destination directory for copies revlogs The encoding in vfs mess up with the directory we are trying to create manually. We could duplicate the advanced logic some more, but that seems ill fated. Instead, we let the vfs deal with directory creation. We update the test to contains a name affected by encoding. Differential Revision: https://phab.mercurial-scm.org/D6902

upgrade: fix DELTAREUSEFULLADD implementation in revlog.clone If we do a full addition, we need to start from the full text. Differential Revision: https://phab.mercurial-scm.org/D6901

upgrade: document DELTAREUSEFULLADD in revlog.clone The documentation of the `debugupgraderepo` command was updated when `re-delta-fulladd` was introduced. However, the docstring of the `revlog.clone` method was not. Differential Revision: https://phab.mercurial-scm.org/D6900

upgrade: move most of revlog.clone method into a _clone method The content of the clone method now focus on parameters validation and processing. The `_clone` method focus on the actual cloning logic. Splitting the method out save some indentation and clarify each method code since it a focussed on one goal. Differential Revision: https://phab.mercurial-scm.org/D6899

flagprocessors: remove flagprocessorsmixin It became an empty shell. Differential Revision: https://phab.mercurial-scm.org/D6823

flagprocessors: move _flagserrorclass attribute on revlog & co This is a small duplication, and the last bit we need to get rid of the mixin. Honestly, I am not fan of that class attribute and it mostly exist to accomodate The simple-storage whose usage of flag processors is dumbious and that is currently dead code anyway. However I don't want to be pulled into futher unrelated cleaning so it is a small price to pay. Differential Revision: https://phab.mercurial-scm.org/D6822

flagprocessors: directly duplicate the deprecated layer back into revlog The code duplication benign and will get removed in a couple of month anyway. Differential Revision: https://phab.mercurial-scm.org/D6821

flagprocessors: make `processflagsraw` a module level function One more steps toward removing the mixin. Differential Revision: https://phab.mercurial-scm.org/D6820

flagprocessors: make `processflagsread` a module level function One more steps toward removing the mixin. Differential Revision: https://phab.mercurial-scm.org/D6819

flagprocessors: make `processflagswrite` a module level function One more step towards removing the mixin. Differential Revision: https://phab.mercurial-scm.org/D6818

flagprocessors: make `_processflagsfunc` a module level function This is the first step toward removing the flag processing mixin. Differential Revision: https://phab.mercurial-scm.org/D6817

flagprocessors: writetransform function take side data as parameter (API) If we want some flag processors to be able to store sidedata it needs to be actually fed that data. Differential Revision: https://phab.mercurial-scm.org/D6816

flagprocessors: add a `sidedata` parameters to _processflagswrite To read sidedata using flagprocessors, we need flag processors to store them. So we pass this information to the flag processing layer. Differential Revision: https://phab.mercurial-scm.org/D6815

revlog: add a `sidedata` parameters to addrevision If we want to eventually store sidedata we need to be able to pass them along. Differential Revision: https://phab.mercurial-scm.org/D6814

flagprocessors: have the read transform function return side data (API) This makes it possible for flag processors to -read- flag data. Differential Revision: https://phab.mercurial-scm.org/D6813