packaging: add Cosmic and Disco, remove Trusty and Artful - Trusty was publicly supported until 2019-04-30 - Artful was publicly supported until 2018-07-19 - Cosmic was publicly supported until 2019-07-18 - Disco will be publicly supported until 2020-01 Cosmic is officially out-of-date, but since it still may be in use, and because we didn't add it when it first came out, I think it would be nice to support it until the next time somebody decides to update this list of Ubuntu releases. Differential Revision: https://phab.mercurial-scm.org/D6761

narrow: fix typo "respositories" Differential Revision: https://phab.mercurial-scm.org/D6758

merge with stable

makefile: run Rust tests if cargo is installed While no particular minimum toolchain version is targeted as of yet, this serves as a first step to make more people/machines run the Rust tests.

merge: hint about using `hg resolve` for resolving conflicts This was suggested by one of our users at Google. Makes sense to me. Differential Revision: https://phab.mercurial-scm.org/D6755

rust-dirstate: remove test case for DirsMultiset::new(Manifest, Some) It's no longer possible.

rust-dirstate: split DirsMultiset constructor per input type Since skip_state only applies to dirstate, it doesn't make sense to unify these constructors and dispatch by enum.

rust-dirstate: remove excessive clone() of parameter and return value I think pass-by-ref is preferred in general.

rust-dirstate: handle invalid length of p1/p2 parameters It uses match syntax since map_err() failed to deduce the argument type.

rust: simply use TryInto to convert slice to array Since our rust module depends on TryInto, there's no point to avoid using it. While rewriting copy_into_array(), I noticed CPython interface doesn't check the length of the p1/p2 values, which is marked as TODO.

rust-dirstate: use PARENT_SIZE constant where appropriate

rust-dirstate: rename NULL_REVISION to NULL_ID which isn't a revision number

rust-dirstate: remove repetition in array literal

rust-dirstate: remove too abstracted way of getting &[u8]

rust-dirstate: remove unneeded "ref" At 7cae6bc29ff9, .to_owned() was rewritten as .to_owned().to_vec(), which is no longer needed since the filename is a single reference.

rust-parsers: fix unboxing of PyInt on Python 3 Broken at 7cae6bc29ff9.

revlog: split `rawtext` retrieval out of _revisiondata This part is reasonably independent. Having it on its own clarify the code flow and will help code that inherit from revlog to overwrite specific area only.

revlog: avoid caching raw text too early in _revisiondata Without this change, we could cache the rawtext without considering for it validating the cache or not. If the exception raised by the invalid hash were to be caught and the same revision accessed again, the invalid rawtext would be returned.

revlog: add some documentation to `_revisiondata` code

revlog: move `nullid` early return sooner in `_revisiondata` Let us deal with the special case before we start dealing with more generic case.

revlog: stop calling `basetext` `rawtext` in _revisiondata If the cache entry is used as a base test for delta, it is not the rawtext we need. We update the variable name to clarify this.

revlog: assign rawtext earlier in `_revisiondata` Assigning the revision earlier make the code easier to read.

revlog: drop silly `raw` parameter to `rawdata` function This is a leftover from `revision` and does not make sense for `rawdata`

perf: don't depend on pycompat for older Mercurial versions We already define local _sysstr() and _xrange() for compatibility, so we should use those. Also create a similar local _bytestr() corresponding to pycompat.bytestr(). Differential Revision: https://phab.mercurial-scm.org/D6745

perf: don't try to call `util.queue` on Mercurial version before it existed Differential Revision: https://phab.mercurial-scm.org/D6744

perf: handle NameError for `pycompat.foo` when pycompat wasn't imported On old Mercurial versions, we won't have a pycompat variable defined, and then `pycompat.foo` will raise a NameError. Differential Revision: https://phab.mercurial-scm.org/D6743

rawdata: update callers in shallowbundle We update callers incrementally because this help bisecting failures. This was useful during development, so we expect it might be useful again in the future.

rawdata: update callers in storageutils We update callers incrementally because this help bisecting failures. This was useful during development, so we expect it might be useful again in the future.

rawdata: update callers in delta utils We update callers incrementally because this help bisecting failures. This was useful during development, so we expect it might be useful again in the future.

rawdata: update callers in repository We update callers incrementally because this help bisecting failures. This was useful during development, so we expect it might be useful again in the future.

rawdata: update callers in testing/storage.py We update callers incrementally because this help bisecting failures. This was useful during development, so we expect it might be useful again in the future.

rawdata: update callers in test-revlog-raw We update callers incrementally because this help bisecting failures. This was useful during development, so we expect it might be useful again in the future.

rawdata: update callers in lfs' tests We update callers incrementally because this help bisecting failures. This was useful during development, so we expect it might be useful again in the future.

rawdata: update callers in lfs' wrapper We update callers incrementally because this help bisecting failures. This was useful during development, so we expect it might be useful again in the future.

rawdata: update caller in wireprotov2server We update callers incrementally because this help bisecting failures. This was useful during development, so we expect it might be useful again in the future.

rawdata: update callers in debugcommands We update callers incrementally because this help bisecting failures. This was useful during development, so we expect it might be useful again in the future.

rawdata: update callers in sqlitestore We update callers incrementally because this help bisecting failures. This was useful during development, so we expect it might be useful again in the future.

rawdata: update caller in remotefilelog We update callers incrementally because this help bisecting failures. This was useful during development, so we expect it might be useful again in the future.

rawdata: update callers in bundlerepo We update callers incrementally because this help bisecting failures. This was useful during development, so we expect it might be useful again in the future.

rawdata: update callers in context We update callers incrementally because this help bisecting failures. This was useful during development, so we expect it might be useful again in the future.

rawdata: update caller in revlog We update callers incrementally because this help bisecting failures. This was useful during development, so we expect it might be useful again in the future.

setup: fix a sorting issue I noticed in package names Differential Revision: https://phab.mercurial-scm.org/D6733

py3: do not convert rust module/attribute names to bytes policy.import*() functions expect system strings.

transplant: unnest --stop case It should be aligned with --continue.

rust-discovery: use while loop instead of match + break This looks slightly nicer.

rust-discovery: remove useless extern crate

transplant: added support for --stop flag This adds fuctionality for `--stop` flag to `transplant`. A new method `stop` is added to `transplanter` class containing logic to abort transplant. Tests are updated as shown. Differential Revision: https://phab.mercurial-scm.org/D6695

unshelve: abort on using --keep and --interactive together I am working on making interactive mode support `--keep` flag. Until we support the usage of `--interactive` and `--keep` together, let us abort on it. Differential Revision: https://phab.mercurial-scm.org/D6699

config: add experimental argument to the config registrar Until now, there are almost 28 config items which are considered as `experimental` but, not present in the `experimental` section of the registrar. This patch adds an `experimental` argument to the config registrar to mark such config items. Differential Revision: https://phab.mercurial-scm.org/D6728 Differential Revision: https://phab.mercurial-scm.org/D6746

tests: split joint repo/changelog fake into one for each type I'm just not comfortable with fakes that get overloaded for multiple types: too often it gets out of hand and becomes difficult to trace. Differential Revision: https://phab.mercurial-scm.org/D6725

fix: pass line ranges as value instead of callback The callback no longer takes any arguments from the inner function, so we might as well call it sooner and pass the value instead. Note the value still needs to be recomputed every iteration to account for the previous iteration's changes to the file content. Differential Revision: https://phab.mercurial-scm.org/D6727

fix: correctly parse the :metadata subconfig It's being handled as a string instead of a bool, though the thruthiness of the string makes the feature still essentially work. Added a regression test. Differential Revision: https://phab.mercurial-scm.org/D6726

fix: allow tools to use :linerange, but also run if a file is unchanged The definition of "unchanged" here is subtle, because pure deletion diff hunks are ignored. That means this is different from using the --whole flag. This change allows you to configure, for example, a code formatter that: 1. Formats specific line ranges if specified via flags 2. Does not format the entire file when there are no line ranges provided 3. Performs some other kind of formatting regardless of provided line ranges This sounds a little far fetched, but it is meant to address a specific corner case encountered in Google's use of the fix extension. The default behavior is kept because it exists to prevent mistakes that could erase uncommitted changes. Differential Revision: https://phab.mercurial-scm.org/D6723

rust-dirstate: call rust dirstatemap from Python Since Rust-backed Python classes cannot be used as baseclasses (for rust-cpython anyway), we use composition rather than inheritance. This also allows us to keep the IO operations in the Python side, removing (for now) the need to rewrite VFS in Rust, which would be a heavy undertaking. Differential Revision: https://phab.mercurial-scm.org/D6634

rust-dirstate: rust-cpython bridge for dirstatemap This change also showcases the limitations of the `py_shared_ref!` macro. See the previous commit 'rust-dirstate: rust implementation of dirstatemap` for an explanation for the TODOs in the code. Differential Revision: https://phab.mercurial-scm.org/D6633

rust-dirstate: rust implementation of dirstatemap The `dirstatemap` is one of the last building blocks needed to get to a `dirstate.walk` Rust implementation. Disclaimer: This change is part of a big (10) series of patches, all of which started as one big changeset that took a long time to write. This `dirstatemap` implementation is a compromise in terms of complexity both for me and for the reviewers. I chose to submit this patch right now because while it is not perfect, it works and is simple enough (IMHO) to be reviewed. The Python implementation uses a lot of lazy propertycaches, breaks encapsulation and is used as an iterator in a lot of places, all of which dictated the somewhat unidiomatic patterns in this change. Like written in the comments, rewriting this struct to use the typestate pattern might be a good idea, but this is a good first step. Differential Revision: https://phab.mercurial-scm.org/D6632

rust-cpython: add macro for sharing references Following an experiment done by Georges Racinet, we now have a working way of sharing references between Python and Rust. This is needed in many points of the codebase, for example every time we need to expose an iterator to a Rust-backed Python class. In a few words, references are (unsafely) marked as `'static` and coupled with manual reference counting; we are doing manual borrow-checking. This changes introduces two declarative macro to help reduce boilerplate. While it is better than not using macros, they are not perfect. They need to: - Integrate with the garbage collector for container types (not needed as of yet), as stated in the docstring - Allow for leaking multiple attributes at the same time - Inject the `py_shared_state` data attribute in `py_class`-generated structs - Automatically namespace the functions and attributes they generate For at least the last two points, we will need to write a procedural macro instead of a declarative one. While this reference-sharing mechanism is being ironed out I thought it best not to implement it yet. Lastly, and implementation detail renders our Rust-backed Python iterators too strict to be proper drop-in replacements, as will be illustrated in a future patch: if the data structure referenced by a non-depleted iterator is mutated, an `AlreadyBorrowed` exception is raised, whereas Python would allow it, only to raise a `RuntimeError` if `next` is called on said iterator. This will have to be addressed at some point. Differential Revision: https://phab.mercurial-scm.org/D6631

rust-docstrings: add missing module docstrings Differential Revision: https://phab.mercurial-scm.org/D6630

rust-dirstate: improve API of `DirsMultiset` - Use opaque `Iterator` type instead of implementation-specific one from `HashMap` - Make `DirsMultiset` behave like a set both in Rust and from Python Differential Revision: https://phab.mercurial-scm.org/D6690

rust-dirstate: use EntryState enum instead of literals This improves code readability quite a bit, while also adding a layer of safety because we're checking the state byte against the enum. Differential Revision: https://phab.mercurial-scm.org/D6629

rust-parsers: switch to parse/pack_dirstate to mutate-on-loop Both `parse_dirstate` and `pack_dirstate` can operate directly on the data they're passed, which prevents the creation of intermediate data structures, simplifies the function signatures and reduces boilerplate. They are exposed directly to the Python for now, but a later patch will make use of them inside `hg-core`. Differential Revision: https://phab.mercurial-scm.org/D6628

rust-parsers: move parser bindings to their own file and Python module This tidies up the Rust side while simplifying the Python side. Differential Revision: https://phab.mercurial-scm.org/D6627

rust-dirstate: create dirstate submodule in hg-cpython This module will soon hold multiple files, this change is to make the review process easier. Differential Revision: https://phab.mercurial-scm.org/D6626

rust-discovery: using from Python code As previously done in other topics, the Rust version is used if it's been built. The version fully in Rust of the partialdiscovery class has the performance advantage over the Python version (actually using the Rust MissingAncestor) if the undecided set is big enough. Otherwise no sampling occurs, and the discovery is reasonably fast anyway. Note: it's hard to predict the size of the initial undecided set, it can depend on the kind of topological changes between the local and remote graphs. The point of the Rust version is to make the bad cases acceptable. More specifically, the performance advantages are: - faster sampling, especially takefullsample() - much faster addmissings() in almost all cases (see commit message in grandparent of the present changeset) - no conversion cost of the undecided set at the interface between Rust and Python == Measurements with big undecided sets For an extreme example, discovery between mozilla-try and mozilla-unified (over one million undecided revisions, same case as in dbd0fcca6dfc), we get roughly a x2.5/x3 better performance: Growing sample size (5% starting with 200): time goes down from 210 to 72 seconds. Constant sample size of 200: time down from 1853 to 659 seconds. With a sample size computed from number of roots and heads of the undecided set (`respectsize` is `False`), here are perfdiscovery results: Before ! wall 9.358729 comb 9.360000 user 9.310000 sys 0.050000 (median of 50) After ! wall 3.793819 comb 3.790000 user 3.750000 sys 0.040000 (median of 50) In that later case, the sample sizes are routinely in the hundreds of thousands of revisions. While still faster, the Rust iteration in addmissings has less of an advantage than with smaller sample sizes, but one sees addcommons becoming faster, probably a consequence of not having to copy big sets back and forth. This example is not a goal in itself, but it showcases several different areas in which the process can become slow, due to different factors, and how this full Rust version can help. == Measurements with small undecided sets In cases the undecided set is small enough than no sampling occurs, the Rust version has a disadvantage at init if `targetheads` is really big (some time is lost in the translation to Rust data structures), and that is compensated by the faster `addmissings()`. On a private repository with over one million commits, we still get a minor improvement, of 6.8%: Before ! wall 0.593585 comb 0.590000 user 0.550000 sys 0.040000 (median of 50) After ! wall 0.553035 comb 0.550000 user 0.520000 sys 0.030000 (median of 50) What's interesting in that case is the first addinfo() at 180ms for Rust and 233ms for Python+C, mostly due to add_missings and the children cache computation being done in less than 0.2ms on the Rust side vs over 40ms on the Python side. The worst case we have on hand is with mozilla-try, prepared with discovery-helper.sh for 10 heads and depth 10, time goes up 2.2% on the median. In this case `targetheads` is really huge with 165842 server heads. Before ! wall 0.823884 comb 0.810000 user 0.790000 sys 0.020000 (median of 50) After ! wall 0.842607 comb 0.840000 user 0.800000 sys 0.040000 (median of 50) If that would be considered a problem, more adjustments can be made, which are prematurate at this stage: cooking special variants of methods of the inner MissingAncestors object, retrieving local heads directly from Rust to avoid the cost of conversion. Effort would probably be better spent at this point improving the surroundings if needed. Here's another data point with a smaller repository, pypy, where performance is almost identical Before ! wall 0.015121 comb 0.030000 user 0.020000 sys 0.010000 (median of 186) After ! wall 0.015009 comb 0.010000 user 0.010000 sys 0.000000 (median of 184) Differential Revision: https://phab.mercurial-scm.org/D6430

rust-discovery: optimization of add commons/missings for empty arguments These two cases have to be catched early for different reasons. In the case of add_missing_revisions, we don't want to trigger the computation of the undecided set (and the children cache) too early: the later the better. In the case of add_common_revisions, the inner `MissingAncestors` object wouldn't know that all ancestors of its bases have already been removed from the undecided. In principle, that would in itself be a lead for further improvement: this remove_ancestors_from could be more incremental, but the current performance seems to be good enough. Differential Revision: https://phab.mercurial-scm.org/D6429

rust-discovery: using the children cache in add_missing The DAG range computation often needs to get back to very old revisions, and turns out to be disproportionately long, given that the end goal is to remove the descendents of the given missing revisons from the undecided set. The fast iteration capabilities available in the Rust case make it possible to avoid the DAG range entirely, at the cost of precomputing the children cache, and to simply iterate on children of the given missing revisions. This is a case where staying on the same side of the interface between the two languages has clear benefits. On discoveries with initial undecided sets small enough to bypass sampling entirely, the total cost of computing the children cache and the subsequent iteration becomes better than the Python + C counterpart, which relies on reachableroots2. For example, on a repo with more than one million revisions with an initial undecided set of 11 elements, we get these figures: Rust version with simple iteration addcommons: 57.287us first undecided computation: 184.278334ms first children cache computation: 131.056us addmissings iteration: 42.766us first addinfo total: 185.24 ms Python + C version first addcommons: 0.29 ms addcommons 0.21 ms first undecided computation 191.35 ms addmissings 45.75 ms first addinfo total: 237.77 ms On discoveries with large undecided sets, the initial price paid makes the first addinfo slower than the Python + C version, but that's more than compensated by the gain in sampling and subsequent iterations. Here's an extreme example with an undecided set of a million revisions: Rust version: first undecided computation: 293.842629ms first children cache computation: 407.911297ms addmissings iteration: 34.312869ms first addinfo total: 776.02 ms taking initial sample query 2: sampling time: 1318.38 ms query 2; still undecided: 1005013, sample size is: 200 addmissings: 143.062us Python + C version: first undecided computation 298.13 ms addmissings 80.13 ms first addinfo total: 399.62 ms taking initial sample query 2: sampling time: 3957.23 ms query 2; still undecided: 1005013, sample size is: 200 addmissings 52.88 ms Differential Revision: https://phab.mercurial-scm.org/D6428

discovery: new devel.discovery.randomize option By default, this is True, but setting it to False is a uniform way to kill all randomness in integration tests such as test-setdiscovery.t By "uniform" we mean that it can be passed to implementations in other languages, for which the monkey-patching of random.sample would be irrelevant. In the above mentioned test file, we use it right away, replacing the adhoc extension that had the same purpose, and to derandomize a case with many round-trips, that we'll need to behave uniformly in the Rust version. Differential Revision: https://phab.mercurial-scm.org/D6427

rust-discovery: optionally don't randomize at all, for tests As seen from Python, this is a new `randomize` kwarg in init of the discovery object. It replaces random picking by some arbitrary yet deterministic strategy. This is the same as what test-setdiscovery.t does, with the added benefit to be usable both in Python and Rust implementations. Differential Revision: https://phab.mercurial-scm.org/D6426

rust-discovery: exposing sampling to python Differential Revision: https://phab.mercurial-scm.org/D6425

tests: use `tr -d` and not `tr --delete` as the latter is absent on BSD tr(1) Differential Revision: https://phab.mercurial-scm.org/D6729

rust-discovery: takefullsample() core implementation take_full_sample() browses the undecided set in both directions: from its roots as well as from its heads. Following what's done on the Python side, we alter update_sample() signature to take a closure returning an iterator: either ParentsIterator or an iterator over the children found in `children_cache`. These constructs should probably be split off in a separate module. This is a first concrete example where a more abstract graph notion (probably a trait) would be useful, as this is nothing but an operation on the reversed DAG. A similar motivation in the context of the discovery process would be to replace the call to dagops::range in `add_missing_revisions()` with a simple iteration over descendents, again an operation on the reversed graph. Differential Revision: https://phab.mercurial-scm.org/D6424

rust-discovery: core implementation for take_quick_sample() This makes in particular `rand` no longer a testing dependency. We keep a seedable random generator on the `PartialDiscovery` object itself, to avoid lengthy initialization. In take_quick_sample() itself, we had to avoid keeping the reference to `self.undecided` to cope with the mutable reference introduced by the the call to `limit_sample`, but it's still manageable without resorting to inner mutability. Sampling being prone to be improved in the mid-term future, testing is minimal, amounting to checking which code path got executed. Differential Revision: https://phab.mercurial-scm.org/D6423

rust-discovery: read the index from a repo passed at init This makes the API of the Rust PartialDiscovery object now the same (or rather a subset) of the Python object, hence easier to control through module policy down the road. Differential Revision: https://phab.mercurial-scm.org/D6517

rust-discovery: accept the new 'respectsize' init arg At this stage, we don't do anything about it: it will be meaningful in sampling methods that aren't implemented yet. Differential Revision: https://phab.mercurial-scm.org/D6516

merge with stable

unshelve: forget unknown files after a partial unshelve This is a follow-up patch to 6957f7b93e03. This allows hg to forget unknown files after a partial unshelve. Differential Revision: https://phab.mercurial-scm.org/D6724

flagutil: move addflagprocessor to the new module (API)

flagutil: move insertflagprocessor to the new module (API)

flagutil: move REVIDX_KNOWN_FLAGS source of truth in flagutil (API) Since REVIDX_KNOWN_FLAGS is "not really a constant" (extension can update it) and python integer,... it needs to be the responsability of a single module and always accessed through the module. We update all the user and move the source of truth in flagutil.

flagutil: move the `flagprocessors` mapping in the new module This module is meant to host most of the flag processing logic. We start with the mapping between flag and processors.

flagutil: create a `mercurial.revlogutils.flagutil` module The flagprocessings logic is duplicated in 2 extra places, and usually in a less robust flavor. This is a maintenance nightmare that I would like to see cleaned up. To do so I am creating a `flagutil` module to move flag processings related code and make it easily reusable by other code.

rawdata: register the method for `ifiledata` The interface have a `revision(..., raw=False)` method so it should get a `rawdata` one. I am not sure why nothing complained about the lack of it earlier.

rawdata: implement the method for `unionrepo` too This is required for all implementations.

rawdata: implement the method for `remotefilelog` too This is needed for all storage implementations.

rawdata: implement `rawdata` for `simplestore` too This is needed for all implementation.

rawdata: forward `rawdata` call on `manifestlog` This needs to be sent to the underlying `revlog` too.

rawdata: implement `rawdata` for `sqlitestore` too This is a different store, it needs it declared.

rawdata: add the method to bundlerevlog The bundlerepo logic has its own `revision` method on its own `revlog` object. We need to "implement" `rawdata` there too.

rawdata: forward the method call on `filelog` object We have a new method, we need to expose it.

rawdata: introduce a `rawdata` method on revlog This method aims at replacing `revision(..., raw=True)` call. The purpose of data returned without and without raw are different enough that having two different method would make sense. This split is motivated by other work aiming at storing data on the side of the main revision of a revlog. Having a cleaner API makes it simpler to add this work. The series following this first changesets is organised as follow: 1) add `rawdata` method everywhere it is useful 2) update all caller 3) implement all `rawdata` method without using `revision` 4) deprecate the `rawdata` parameter of `revision`

revlog: split a `_revisiondata` method to file `revision` job We are about to introduce more public method to access revision data (eg: `rawdata`). revset subclass tend to recursively call `revision` which will create all kind of issue with the coming series. To avoid them we introduce an explicit difference between the internal call and the public all. This will be useful for later work anyway (so the subclass issue is just moving it earlier in the series). I am not sure if the subclass are actually doing something sensible. However, I am certain I don't want to be rabbit holed into figuring it out right now.

continue: added support for transplant This creates a seperate function `continuetransplant()` containing logic for resuming transplant from interrupted state. `continuetransplant()` is then registered as `continuefunc` for state detection API. Results are shown in tests. Differential Revision: https://phab.mercurial-scm.org/D6689

merge with stable

fncache: make debugrebuildfncache not fail on broken fncache The code reading the fncache changed in 5.0, to complain if the file is not \n terminated. This makes apparent the fact that the fncache gets corrupted. Make it possible to recover, instead of having `hg debugrebuildfncache` failing by saying `(run hg debugrebuildfncache)`. The corruption itself is most likely due to hg not using fsync in general, and so various bad things can happen. Here, the reported problems happened when running out of disk space. So I suspect that because the fncache is much bigger than the average commit/pull, when running out of disk space, the bulk of the pull may succeed, but the new fncache may get half-written and still renamed into place. Differential Revision: https://phab.mercurial-scm.org/D6722

fncache: show that debugrebuildfncache is partly broken Differential Revision: https://phab.mercurial-scm.org/D6721

test: further fixes to matching for run-tests.py bug The fix in bac24a8a095a did not fix the full issue, because the extra number also eat some of the separator space. Since we are already matching arbitrary number of space, we easily fix the matching.

branchmap: explicitly warm+write all subsets of the branchmap caches 'full' claims it will warm all of the caches that are known about, but this was not the case - it did not actually warm the branchmap caches for subsets that we haven't requested, or for subsets that are still considered "valid". By explicitly writing them to disk, we can force the subsets for ex: "served" to be written ("immutable" and "base"), making it cheaper to calculate "served" the next time it needs to be updated. Differential Revision: https://phab.mercurial-scm.org/D6710

changectx: extract explicit computechangesetfilesremoved method from context Right now, the logic around changeset centric removed files data are buried into the "changectx" code. We extract this code in a dedicated method (in the scmutil module) for clarity. This clarity will help to explicitly compute and caches these data in the future.

changectx: extract explicit computechangesetfilesadded method from context Right now, the logic around changeset centric added files data are buried into the "changectx" code. We extract this code in a dedicated method (in the scmutil module) for clarity. This clarity will help to explicitly compute and caches these data in the future.

demandimport: explicitly declare `_session` at the module level The `_session` module level variable is set within a function using the `global` keyword. This confuses my `test-check-pyflakes.t`. Explicitly declaring the variable at the top level solves the issue (and seems absolutely reasonable).

tests: give more room for slowness in test-run-tests.t The test expected any run-test.py run to end in less than 10 seconds. On slower loaded CI machine, this gets slower than that. We give a bit more room to the regexp.

copies: extract an explicit `computechangesetcopie` method from context Right now, the logic around changeset centric copies data are buried into the "changectx" code. We extract this code in a dedicated method (in the copies module) for clarity. This clarity will help to explicitly compute and caches these data in the future.

config: fix fm.data() handling of defaultvalue This is a follow-up patch to rHG51a2e3102db2. This moves `fm.data()` out of the if block in `commands.config()`. Differential Revision: https://phab.mercurial-scm.org/D6720

config: remove pycompat.bytestr() for defaultvalue This is a follow-up patch to 51a2e3102db2. This removes `pycompat.bytestr` to preserve `None` in `commands.config()`. Differential Revision: https://phab.mercurial-scm.org/D6712

unshelve: clear shelvedstate and _finishunshelve() on partial unshelve On a partial unshelve, `shelvedstate` was not cleared and `_finishunshelve()` was not called. Ideally, these should be called on this case. This patch makes `shelvedstate` to delete after a successful partial unshelve and calls `_finishunshelve()` in the same case. Differential Revision: https://phab.mercurial-scm.org/D6708

unshelve: delete shelvedstate after a successful unshelve --continue `unshelve --continue` was preventing the deletion of `shelvedstate` on a partial `unshelve`. Ideally, `shelvedstate` should be deleted after a successful `unshelve`. Now, the behavior of `unshelve --continue` will be as follows in interactive mode: 1] The user tried to `unshelve` changes interactively but, ran into conflicts. 2] They resolved the conflicts and triggered `unshelve --continue` but, unshelved changes partially. 3] Now, on trying to do `unshelve --continue` again will abort as the last `unshelve` was successful and we are deleting the `shelvedstate`. 4] If they want to unshelve the remaining shelved change, they need to trigger `unshelve` without `--continue`. Differential Revision: https://phab.mercurial-scm.org/D6694

unshelve: handle stripping changesets on interactive mode On interactive mode, changesets on `nodestoremove` should be stripped regardless of the shelve is partial or not. This patch modifies `unshelvecontinue()` to do that. Differential Revision: https://phab.mercurial-scm.org/D6686

byteify-strings: add --version argument This is indispensable for automated tools to detect changes in behavior.

byteify-strings: add space in special comments to silence flake8 error This is done soon enough that nobody has had the time to use this feature yet.

revset: drop argument when it's None getstack's definition is `getstack(repo, rev=None)`, so providing None explicitly is unnecessary. Moreover, when x is not None, it's definitely not a revision but a part of a parsed tree of revset arguments. Differential Revision: https://phab.mercurial-scm.org/D6707

stack: remove unnecessary reverse() predicate Stack already sorts revisions, so no need to do it twice. This change was a part of D2400, which didn't land for other reasons. See also D2399, where this change was suggested. Differential Revision: https://phab.mercurial-scm.org/D6706

automation: increase root volume size on Linux It is close to full in the AMI. I actually ran out of space running tests without increasing the volume size! Differential Revision: https://phab.mercurial-scm.org/D6716

automation: install Rust in Linux environment This will install Rust 1.31.1, 1.34.2, and whatever stable is at the time the install runs. We install 1.31.1 as our minimum supported Rust version (I think that's what we're currently targeting) and 1.34 because that's what Debian 10 is shipping. Differential Revision: https://phab.mercurial-scm.org/D6715

automation: update packages in requirements files We like keeping up to date. The content of the autogenerated files changed slightly because I used a newer version of `pip-compile` than what was used previously. Differential Revision: https://phab.mercurial-scm.org/D6714

automation: install latest Python versions This required bumping the pyenv commit so the new versions are available. Differential Revision: https://phab.mercurial-scm.org/D6713

upgrade: introduce the internal code for revlog cloning selection For now we still clone every single revlogs but all the selection mechanism is now in place in the lower layer. The next changesets will introduce the user interface part of the selection.

relnotes: copy "next" to "5.1" and clear "next" To avoid merge conflicts, we want to avoid modifying the file on multiple branches in parallel. This patch is therefore meant to be applied to the stable branch and then quickly be merged to default (at least before edits are made to relnotes/next there). Another option would have been to copy the file on the stable branch and to clear it on the default branch. However, that still results in conflicts if the copy is edited on the stable branch (Mercurial would try to apply the changes from the default branch to it). We could also delete the file in one commit and recreate it in another commit. However, Mercurial is quite inconsistent in what it considers a break in history (see test-copies-unrelated.t), so I'd like to avoid that. Differential Revision: https://phab.mercurial-scm.org/D6705

upgrade: introduce a _copyrevlog method This function copies a revlog from the old store to the new one, without re-adding all deltas manually. This will eventually save a lot of time when some revlog does not needs any conversions. Code actually using this will be introduced in later changesets.

upgrade: rename `_copyrevlogs` to `_clonerevlogs` The underlying revlog method is named `clone`, keeping the naming consistent seems clearer. This is motivated to clarify the difference with an (upcoming) function that simply copy revlog files as is.

upgrade: walk the source store file only once I don't expect this to have a significant performance impact, but it seems simpler and saner to do the operation only once and to keep the result around.