sidedata: add a debugsidedata command The command provide basic capabilities to inspect revision side data. Differential Revision: https://phab.mercurial-scm.org/D6897

sidedata: test we can successfully write sidedata For now we just write them and rejoice on the lack of crashes. Differential Revision: https://phab.mercurial-scm.org/D6896

sidedata: make sure we don't use the flag if there are not sidedata Ensuring this at this level seems safer than relying on caller doing the right thing. G: changed mercurial/revlog.py Differential Revision: https://phab.mercurial-scm.org/D6895

revlog: add the appropriate flag is sidedata are passed to `addrevision` If we need to store sidedata, we need the flag to be set and the associated processing to be called. Differential Revision: https://phab.mercurial-scm.org/D6894

sidedata: register the flag processors if the repository allows for it Differential Revision: https://phab.mercurial-scm.org/D6893

sidedata: add a function for _processflagsraw usage Entry with side data cannot directly validate their rawtext Differential Revision: https://phab.mercurial-scm.org/D6892

sidedata: add a function to write sidedata into a raw text Differential Revision: https://phab.mercurial-scm.org/D6891

sidedata: add a function to read sidedata from revlog raw text This implement the "reading" part of a `sidedata` flag processor. Differential Revision: https://phab.mercurial-scm.org/D6890

sidedata: add a new module with basic documentation For now the storage strategy is very simple. We can augment it in the future if needed. Code to actually support what is described will be introduced in later changesets. Differential Revision: https://phab.mercurial-scm.org/D6889

sidedata: add a new revision flag constant for side data Small step first. Differential Revision: https://phab.mercurial-scm.org/D6888

upgrade: detect the side-data format variants Note that for now we cannot upgrade/downgrade to it. Differential Revision: https://phab.mercurial-scm.org/D6887

sidedata: introduce a new requirement to protect the feature The feature will be usable only within repository supported by version of Mercurial with the support. Since the support is not fully implemented yet, the requirement has a temporary, experimental name. Differential Revision: https://phab.mercurial-scm.org/D6886

copies: move file input processsing early If we are to store the same kind of data outside of extra, we need to explicitly prepare them before that. On the long run, other storage (eg: sidedata) might use a different encoding to store this information, since the constraint from extra does not apply to it. Differential Revision: https://phab.mercurial-scm.org/D6937

copies: prepare changelog for more copies storage mode We want to store copies information in sidedata, the logic will be quite similar to the one used for extra, but sightly different (not in extra, no hash impact). We start with small refactoring to make our work easier and the coming changesets cleaners. Differential Revision: https://phab.mercurial-scm.org/D6936

vfs: give all vfs an options attribute by default Multiple code path assume vfs have an options attribute, including the vfs module itself. So for consistency we explicitly add one to all vfs. This will prevent various crash in the next changesets. Differential Revision: https://phab.mercurial-scm.org/D6935

revlog: explicitly set revlogv0 in vfs options Relying on having an attribute or not for something so fundamental seems too fragile to me. (And indeed I had issue with that later in this series). So we explicitly record the fact the repository use revlog-v0. Differential Revision: https://phab.mercurial-scm.org/D6934

context: clarify the various mode in the _copies property cache The previous code was compact but a bit dense. The new proposed code deal with each mode separately, there are some duplicated lines, but the meaning of each mode stand out. One of the benefit it to make it simpler to add further mode in the future. Differential Revision: https://phab.mercurial-scm.org/D6933

context: clarify the various mode in the filesremoved method The previous code was compact but a bit dense. The new proposed code deal with each mode separately, there are some duplicated lines, but the meaning of each mode stand out. One of the benefit it to make it simpler to add further mode in the future. Differential Revision: https://phab.mercurial-scm.org/D6932

context: clarify the various mode in the filesadded method The previous code was compact but a bit dense. The new proposed code deal with each mode separately, there are some duplicated lines, but the meaning of each mode stand out. One of the benefit it to make it simpler to add further mode in the future. Differential Revision: https://phab.mercurial-scm.org/D6931

copies: expand the logic of usechangesetcentricalgo Using intermediate variable is clearer and will make is simple to expand the logic. Differential Revision: https://phab.mercurial-scm.org/D6930

patchbomb: protect email addresses from shell When patchbomb sends email via a sendmail-like program it invokes this using procutil.popen which passes the string to a shell to be parsed. To protect any special characters in the email addresses on the command line from being interpretered by the shell they must be quoted.

automation: support and use Debian Buster by default While Debian Buster (Debian 10) was released in July, the AWS AMIs were not published until mid September. This commit teaches the automation system to create AMIs for Debian Buster. Since Debian Buster is the new stable Debian release, we make it the default distribution for automation. Differential Revision: https://phab.mercurial-scm.org/D6917

automation: use latest AMIs This cuts down on the overhead to create a new AMI, as we don't have to install as many package upgrades. Differential Revision: https://phab.mercurial-scm.org/D6916

automation: increase size of Linux AMI build volume I ran out of space attempting to build a few AMIs. Let's give the environment an extra 2 GB to utilize. Differential Revision: https://phab.mercurial-scm.org/D6915

automation: install awscli and python3-boto3 packages This provides the `aws` command-line utility as well as a Python library for interacting with AWS. It is useful to have these packages installed in AMIs so you can easily interact with other AWS services. Differential Revision: https://phab.mercurial-scm.org/D6914

automation: upgrade packages in Linux environment Let's stay modern. Differential Revision: https://phab.mercurial-scm.org/D6913

automation: remove Ubuntu 18.10 It reached end of life on July 18, 2019. I don't think we should care about supporting it. Differential Revision: https://phab.mercurial-scm.org/D6912

automation: always install docker-ce Docker has published packages for Ubuntu Disco and we no longer need this one-off. Differential Revision: https://phab.mercurial-scm.org/D6911

automation: use LSB_RELEASE instead of DEBIAN_VERSION This should be more robust since I believe the minor version can change mid release. Differential Revision: https://phab.mercurial-scm.org/D6910

tests: use silenttestrunner in test-simplemerge.py The time monkeypatching doesn't appear to work reliably in Python 3, possibly due to unittest using a different time function. This makes the test intermittent due to the execution time not always being `0.00s`. We have our own wrapper around unittest for more deterministic test output. So let's use it. As a bonus, all test output disappeared, so we can remove the .out file! Differential Revision: https://phab.mercurial-scm.org/D6921

contrib: use pycompat.xrange in bdiff-torture.py For Python 3 compatibility. Differential Revision: https://phab.mercurial-scm.org/D6920

interfaces: use triple quotes for Attribute value Otherwise the source transformer converts it to bytes and test-check-interfaces.py complains on Python 3. Differential Revision: https://phab.mercurial-scm.org/D6919

tests: suppress output from write() Otherwise it is printed and the test fails with tons of output on Python 3. Differential Revision: https://phab.mercurial-scm.org/D6918

tests: look for additional pip warning variant The wording has been tweaked in new versions of pip. Differential Revision: https://phab.mercurial-scm.org/D6909

tests: use test harness `hg` in test-hghave.t This prevents the sub test harness from rebuilding Mercurial. It shaves ~30s from the test wall time on my machine. Differential Revision: https://phab.mercurial-scm.org/D6908

synthrepo: use pycompat.xrange This silences some test-check-pyflakes.t warnings on Python 3. Differential Revision: https://phab.mercurial-scm.org/D6907

perf: fix perfhelper-pathcopies without --stats Some conditional were missing.

upgrade: also register copied `.d` files to fncache Oops, we forgot to register data file when blanky copying revlog. Now that we are actually copying these `.d` files, we should also register them. Differential Revision: https://phab.mercurial-scm.org/D6905

upgrade: fix handling of `.d` data file The test did not used any `.d` file. So of course, their handling was broken (for vfs encoding reason again). We fix the issue and update the test. Note that some fncache issue remains with `.d` file. It will be taken care of in the next changesets. Differential Revision: https://phab.mercurial-scm.org/D6904

upgrade: also copy data file Differential Revision: https://phab.mercurial-scm.org/D6903

upgrade: create the correct destination directory for copies revlogs The encoding in vfs mess up with the directory we are trying to create manually. We could duplicate the advanced logic some more, but that seems ill fated. Instead, we let the vfs deal with directory creation. We update the test to contains a name affected by encoding. Differential Revision: https://phab.mercurial-scm.org/D6902

upgrade: fix DELTAREUSEFULLADD implementation in revlog.clone If we do a full addition, we need to start from the full text. Differential Revision: https://phab.mercurial-scm.org/D6901

upgrade: document DELTAREUSEFULLADD in revlog.clone The documentation of the `debugupgraderepo` command was updated when `re-delta-fulladd` was introduced. However, the docstring of the `revlog.clone` method was not. Differential Revision: https://phab.mercurial-scm.org/D6900

upgrade: move most of revlog.clone method into a _clone method The content of the clone method now focus on parameters validation and processing. The `_clone` method focus on the actual cloning logic. Splitting the method out save some indentation and clarify each method code since it a focussed on one goal. Differential Revision: https://phab.mercurial-scm.org/D6899

flagprocessors: remove flagprocessorsmixin It became an empty shell. Differential Revision: https://phab.mercurial-scm.org/D6823

flagprocessors: move _flagserrorclass attribute on revlog & co This is a small duplication, and the last bit we need to get rid of the mixin. Honestly, I am not fan of that class attribute and it mostly exist to accomodate The simple-storage whose usage of flag processors is dumbious and that is currently dead code anyway. However I don't want to be pulled into futher unrelated cleaning so it is a small price to pay. Differential Revision: https://phab.mercurial-scm.org/D6822

flagprocessors: directly duplicate the deprecated layer back into revlog The code duplication benign and will get removed in a couple of month anyway. Differential Revision: https://phab.mercurial-scm.org/D6821

flagprocessors: make `processflagsraw` a module level function One more steps toward removing the mixin. Differential Revision: https://phab.mercurial-scm.org/D6820

flagprocessors: make `processflagsread` a module level function One more steps toward removing the mixin. Differential Revision: https://phab.mercurial-scm.org/D6819

flagprocessors: make `processflagswrite` a module level function One more step towards removing the mixin. Differential Revision: https://phab.mercurial-scm.org/D6818

flagprocessors: make `_processflagsfunc` a module level function This is the first step toward removing the flag processing mixin. Differential Revision: https://phab.mercurial-scm.org/D6817

flagprocessors: writetransform function take side data as parameter (API) If we want some flag processors to be able to store sidedata it needs to be actually fed that data. Differential Revision: https://phab.mercurial-scm.org/D6816

flagprocessors: add a `sidedata` parameters to _processflagswrite To read sidedata using flagprocessors, we need flag processors to store them. So we pass this information to the flag processing layer. Differential Revision: https://phab.mercurial-scm.org/D6815

revlog: add a `sidedata` parameters to addrevision If we want to eventually store sidedata we need to be able to pass them along. Differential Revision: https://phab.mercurial-scm.org/D6814

flagprocessors: have the read transform function return side data (API) This makes it possible for flag processors to -read- flag data. Differential Revision: https://phab.mercurial-scm.org/D6813

flagprocessors: return flagdata in the main processing function This function input and return are becoming stranger and stranger bnut I don't have a good plan to make is saner without problematic code duplication, so it will be this way to now. Differential Revision: https://phab.mercurial-scm.org/D6812

flagprocessors: return sidedata map in `_processflagsread` Right now, flag processors does not return sidedata, by they will. So, we prepare the caller to receive it. Differential Revision: https://phab.mercurial-scm.org/D6811

revlog: use the new sidedata map return in the sidedata method So far things, seems logical. Differential Revision: https://phab.mercurial-scm.org/D6810

revlog: return sidedata map from `_revisiondata` Nothing extra any side data yet. However, it will happens in the future. So we better prepare the callers of the `_revisiondata` to deal with it. Differential Revision: https://phab.mercurial-scm.org/D6809

revlog: introduce a `sidedata` method The method give access to extra information related to the revision. Such data will not be part of the hash be strongly related to the revision. Having them stored at the revlog level helps the storage consistency story and simplify various things. Example of data we could store there: - copy tracing related informations - graph structure related information (useful for discovery) - unresolved conflict data The full implementation will be introduced gradually in the coming changesets. Differential Revision: https://phab.mercurial-scm.org/D6808

update: clarify calculateupdate() call sites by specifying argument names merge.calculateupdate() takes a lot of parameters and I get confused all the time which is which. See also b14fdf1fb615 (update: clarify update() call sites by specifying argument names, 2017-02-09). Differential Revision: https://phab.mercurial-scm.org/D6883

bookmarks: remove changectx() method from bmstore (API) All the callsites of this method have access to the repo, and I'd rather not have to duplicate this across alternative bmstore implementations. Besides, it feels like a bit of a layering violation. .. api:: `mercurial.bookmarks.bmstore` no longer has a convenience method for looking up changectx instances from a bookmark name. Use `repo[repo.bookmarks[name]]` intead of `repo.bookmarks.changectx(name)`. Differential Revision: https://phab.mercurial-scm.org/D6884

histedit: sniff-test for untracked file conflicts before prompting for rules This bug is as old as histedit, which is more than 10 years! I'm a little sad about the extra calculations here that we're just going to throw out, but I don't see any better way to look for untracked file conflicts and I want the bug fixed. Differential Revision: https://phab.mercurial-scm.org/D6882

histedit: demonstrate breakage when `update` to a revision breaks I'm honestly impressed that nobody has hit this in the over a decade that histedit has existed, but here we are. Differential Revision: https://phab.mercurial-scm.org/D6881