shallowutil: narrow scope of try/except block This will make this code easier to understand in the future. Differential Revision: https://phab.mercurial-scm.org/D11007

remotefilelog: stop using RuntimeError for control flow We introduce a new exception to handle the various failure categories, rather than relying on RuntimeError. Differential Revision: https://phab.mercurial-scm.org/D11006

remotefilelog: tweak corrupt cache test to grep more flexibly I'm about to fix the abuse of RuntimeError here, which breaks the test if I don't do this. Differential Revision: https://phab.mercurial-scm.org/D11005

dirstatenonnormalcheck: fix some bytes formating on python3 Passing any object to `%s` no longer works, we need to explicitely convert the representation to bytes. Differential Revision: https://phab.mercurial-scm.org/D10993

dirstate-item: use the properties in `contrib/dirstatenonnormalcheck.py` Differential Revision: https://phab.mercurial-scm.org/D10991

dirstate-item: use the properties in largefiles I am now sure what large file is doing with then, but at least it is no longer using the tuple API to access data. Differential Revision: https://phab.mercurial-scm.org/D10990

dirstate-item: use the `state` property in debugpathcomplete Ideally we would use narrower property, but this is a good start. Differential Revision: https://phab.mercurial-scm.org/D10989

dirstate-item: use the properties in fakedirstatewritetime Differential Revision: https://phab.mercurial-scm.org/D10987

dirstate-item: use the properties in dirstatemap Differential Revision: https://phab.mercurial-scm.org/D10986

dirstate-item: use the properties in pathutil Differential Revision: https://phab.mercurial-scm.org/D10985

dirstate-item: use need_delay when packing dirstate Differential Revision: https://phab.mercurial-scm.org/D10984

dirstate-item: rename the class to DirstateItem The object is no longer a tuple, so it seems clearer to rename it (and its associated method) Differential Revision: https://phab.mercurial-scm.org/D10982

dirstate-entry: add a `mtime` property This is clearer than "tuple" indexing. Differential Revision: https://phab.mercurial-scm.org/D10981

dirstate-entry: add a `size` property This is clearer than "tuple" indexing. Differential Revision: https://phab.mercurial-scm.org/D10980

dirstate-entry: add a `mode` property This is clearer than "tuple" indexing. Differential Revision: https://phab.mercurial-scm.org/D10979

dirstate: no longer pass `oldstate` to the `dropfile` The `oldstate` value come literally from `_map` so we don't need to pass tha information along. Differential Revision: https://phab.mercurial-scm.org/D10978

dirstate: drop last explicite `state` usage in status Given that the previous conditional we can safely use `tracked` here. Differential Revision: https://phab.mercurial-scm.org/D10976

dirstate: reorder "state" checking conditional This is easier to read and will make this simpler to update in the next changeset. Differential Revision: https://phab.mercurial-scm.org/D10975

dirstate-entry: add a `need_delay` method This abstract the internal processing need for entry that would have an ambiguous mtime (If I understand things correctly). Differential Revision: https://phab.mercurial-scm.org/D10974

dirstate-entry: add a `tracked` property This abstract the individual `state` value and has a clear semantic. Differential Revision: https://phab.mercurial-scm.org/D10973

dirstate: drop `state` to `_addpath` All its value are inferred within the dirstatemap now. Differential Revision: https://phab.mercurial-scm.org/D10972

dirstate: infer the 'n' state from `from_p2` This flag is only used with 'n' so lets set the state based on that parameter in this case. Differential Revision: https://phab.mercurial-scm.org/D10971

dirstate: infer the 'n' state from `possibly_dirty` This flag is only used with 'n' so lets set the state based on that parameter in this case. Differential Revision: https://phab.mercurial-scm.org/D10970

dirstate: use a `merged` parameter to _addpath Differential Revision: https://phab.mercurial-scm.org/D10969

dirstate-entry: add a `added` property Lets use more semantic property instead of the `state` implementation details. Differential Revision: https://phab.mercurial-scm.org/D10968

dirstate: use a `added` parameter to _addpath My next target are the explicit `state` byte, so let start with abstracting their initial storage. Differential Revision: https://phab.mercurial-scm.org/D10967

dirstate: no longer pass the `oldstate` value to the dirstatemap The dirstatemap already have this information. Differential Revision: https://phab.mercurial-scm.org/D10966

dirstate: stop using `oldstate` in `dirstate._addpath` We don't need to pass the oldstate around, so lets use the new property instead. Differential Revision: https://phab.mercurial-scm.org/D10965

dirstate: drop the now unused magic constants for the dirstate module We no longer need them for the dirstate logic. We only need them in the dirstate map (and parsers) logic. This smell like progress. Differential Revision: https://phab.mercurial-scm.org/D10964

dirstate: move the handling of special case within the dirstatemap The dirstatemap is as well, if not better, suited to decided how to encode the various case. So we move the associated code in the dirstatemap `addfile` method. This means the dirstate no longer need to know about the various magic value used in the dirstate V1 format. The pain of the messy API start to be quite palpable in Rust, we should clean this up once the current large refactoring is dealt with. Differential Revision: https://phab.mercurial-scm.org/D10963

rust: bump rust-cpython version for 0.6.0 That function as a nice `is_none` method on `PyObject` that we will put to use. Differential Revision: https://phab.mercurial-scm.org/D10962

rust-dirstate: make the MTIME_UNSET public We will use it in the cpython layer soon. Differential Revision: https://phab.mercurial-scm.org/D10961

rust-dirstatemap: expand the wrapping code a bit This is easier to read. Differential Revision: https://phab.mercurial-scm.org/D10960

dirstate-entry: restrict `from_p2` property to tracked file The other case is now handled in `from_p2_removed`, and should ideally be moved to a lower level. So we can drop the `state == 'n'` narrowing and move it in the property itself. Differential Revision: https://phab.mercurial-scm.org/D10959

dirstate-entry: `merged_removed` and `from_p2_removed` properties Lets start to define and use more semantic property. These two might be a bit too low level and could be shaved off later, however this seems an improvement for now. Differential Revision: https://phab.mercurial-scm.org/D10958

dirstate-entry: add a `removed` property Lets start to define and use more semantic property. Differential Revision: https://phab.mercurial-scm.org/D10957

dirstate-entry: add a `from_p2` property Lets start to define and use more semantic property. Differential Revision: https://phab.mercurial-scm.org/D10956

dirstate-entry: add a `merged` property Lets start to define and use more semantic property. Differential Revision: https://phab.mercurial-scm.org/D10955

dirstate-entry: add a `state` property (and use it) This replace the [0] access. Ultimately is we should probably get ride of this in its current form. However this is a good transitional solution to move away for tuple indexing for now. Differential Revision: https://phab.mercurial-scm.org/D10954

dirstate: move most of the `remove` logic with dirstatemap `removefile` This code deal with special logic to preserving "merged" and "from_p2" information when removing a file. These are implementation details that are more suitable for the dirstatemap layer. Since the dirstatemap layer alreaday have most of the information necessary to do so, the move is easy. This move helps us to encapsulate more implementation details within the dirstatemap and its entry. Easing the use of a different storage for dirstate v2. Differential Revision: https://phab.mercurial-scm.org/D10953

dirstate: add a `in_merge` property This factor the "p2 is not null" check and is fairly simpler to read. Differential Revision: https://phab.mercurial-scm.org/D10952

dirstate-entry: introduce dedicated accessors for v1 serialization In the spirit of changing the content and storage of the dirstate entry, we add new method that the code doing v1 serialisation can use. Adding such method to the C object is quite trivial. Differential Revision: https://phab.mercurial-scm.org/D10951

dirstate-entry: goes through the `dirstatetuple` constructor in all cases We need to make sure we build an object. Differential Revision: https://phab.mercurial-scm.org/D10950

dirstate-entry: turn dirstate tuple into a real object (like in C) With dirstate V2, the stored information and actual format will change. This mean we need to start an a better abstraction for a dirstate entry that a tuple directly accessed. By chance, the C code is already doing this and pretend to be a tuple. So it should be fairly easy. We start with turning the tuple into an object, we will slowly migrate the dirstate code to no longer use the tuple directly in later changesets. Differential Revision: https://phab.mercurial-scm.org/D10949

dirstate: split dirstatemap in its own file The dirstate file is large enough and the dirstatemap is quite insulated logic already. Differential Revision: https://phab.mercurial-scm.org/D10934

run-tests: stop writing a `python3` symlink pointing to python2 Having `python3` actually pointing to `python2` is bad. So we stop doing so. In addition we need to re-introduce a `python` executable since some of the script really need to be able to say "current python" in their shbang. For example, `hghave` is one of such script. The faulty changes where introduced by c102b704edb5. Differential Revision: https://phab.mercurial-scm.org/D10943

check-code: stop forbidding return code result There is no explication of what is the intend of that check and what is the alternative. I suspect this comes from the transition to the "unified test" format circa 2010. With the non zero return explicitly listed in the output explicit $? checking became Redundant. However there is valid use case for checking $? so I am dropping this check. Differential Revision: https://phab.mercurial-scm.org/D10994

run-tests: avoid an early return To fix the "python3 pointing to python2" we will also need to create a "python" pointer. So we will need to create multiple pointer. So we need to stop using early return. We replace the early return with a loop and a continue, since the next changeset will introduce that loop anyway. Differential Revision: https://phab.mercurial-scm.org/D10942

run-tests: clarify pyexename assignement That assignement is wrong, we clarify that it is wrong before starting to fix it. This will make the fix clearer. Differential Revision: https://phab.mercurial-scm.org/D10941

run-tests: factor appdata out This is a gratuitous readability change. Differential Revision: https://phab.mercurial-scm.org/D10940

run-tests: unroll a non-so-one-liner This is shorter and clearer. Differential Revision: https://phab.mercurial-scm.org/D10939

run-tests: unroll a non-so-one-liner This is shorter and clearer. Differential Revision: https://phab.mercurial-scm.org/D10938

testing: fix _timeout_factor With `--debug`, `run-tests.py` set the timeout to 0... that breaks the logic in `mercurial.testing`. Differential Revision: https://phab.mercurial-scm.org/D10937

exchange: use "served" repo filter to guess what the server will publish Previously, the second push in the added test would say "push would publish 2 changesets" because _checkpublish() was considering secret changesets to be "pushable" when push command had a --rev argument. Without --rev argument to push command, we already filter the repo properly. Differential Revision: https://phab.mercurial-scm.org/D10948

rust: Document the DirstateMapMethods trait Differential Revision: https://phab.mercurial-scm.org/D10919

dirstate: document the dirstatetuple content We are about to modify this, so let us document the existing code. Differential Revision: https://phab.mercurial-scm.org/D10935

dirstate: explicitely deal with the `added` case in `_addpath` This special case is now directly handled, this clarify the possible value for the other options. Differential Revision: https://phab.mercurial-scm.org/D10933

dirstate: add an explicit `possibly_dirty` parameter to `_addpath` This lets says what we mean instead of using magic value. The lower level can then decide how to express that. Differential Revision: https://phab.mercurial-scm.org/D10932

dirstate: add an explicit `from_p2` parameter to `_addpath` This lets says what we mean instead of using magic value. The lower level can then decide how to express that. Differential Revision: https://phab.mercurial-scm.org/D10931

dirstate: add default value to _addpath We want to remove the magic value usage from the higher level, so lets stop passing them explicitely when possible. Differential Revision: https://phab.mercurial-scm.org/D10930

dirstate: move the _rangemask filtering closer to its storage The goal is to get ready to move these kind of processing at a lower level. We start with move move _rangemask filtering close to where it is sent to the lower level to make the future move trivial. Differential Revision: https://phab.mercurial-scm.org/D10929

dirstate: introduce a symbolic constant for the AMBIGUOUS_TIME marker This is going to be clearer and easier to track than -1. Ultimately I would like to get ride of this special value everywhere but in the lower level, however we need to clarify the API first. This changeset is part of such clarification. Differential Revision: https://phab.mercurial-scm.org/D10928

dirstate: introduce a symbolic constant for the NONNORMAL marker This is going to be clearer and easier to track than -1. Ultimately I would like to get ride of this special value everywhere but in the lower level, however we need to clarify the API first. This changeset is part of such clarification. Differential Revision: https://phab.mercurial-scm.org/D10927

dirstate: introduce a symbolic constant for the FROM_P2 marker This is going to be clearer and easier to track than -2. Ultimately I would like to get ride of this special value everywhere but in the lower level, however we need to clarify the API first. This changeset is part of such clarification. Differential Revision: https://phab.mercurial-scm.org/D10926

dirstate: split a not-so-one-liner This is shorter and simpler to read. Differential Revision: https://phab.mercurial-scm.org/D10925

dirstate: split a not-so-one-liner This is simpler to read. Differential Revision: https://phab.mercurial-scm.org/D10924

dirstate: split a not-so-one-liner This is shorter and simpler to read. Differential Revision: https://phab.mercurial-scm.org/D10923

dirstate-tree: Keep a counter of descendant nodes that have an entry … and change the `DirstateMap::has_dir` method to be based on this counter being non-zero instead of the mere presence of a node. A node with zero descendent with an entry currently should be removed from the tree, but soon we’ll make the dirstate track additional nodes. (Specifically, for non-ignored directories in order to keep track of their mtime and optimize status by doing fewer `read_dir` calls.) Differential Revision: https://phab.mercurial-scm.org/D10922

dirstate: Removed unused instances of `DirsMultiset` … in Rust-backed dirstatemap. The Python class `dirstatemap` had cached properties `_dirs` and `_alldirs` that were not used for `hastrackeddir` and `hasdir` since they were redundant with corresponding fields for the Rust `DirstateMap` struct. `dirfoldmap` is modified to reuse instead the directory iterator introduced in 3b9914b28133c0918186b6e8b9e4f1916e21338d. Differential Revision: https://phab.mercurial-scm.org/D10921

dirstate-v2: Use 32-bit integers instead of 64-bit for offsets This saves 12 bytes per node. (Nodes representing files or directories.) These are offsets to other parts of the file. This would only be a limitation for a `.hg/dirstate` file larger than 4 GiB, which would only happen for a repository with dozens of millions of files and directories. Differential Revision: https://phab.mercurial-scm.org/D10920

status: Extend read_dir caching to directories with ignored files See code comments Differential Revision: https://phab.mercurial-scm.org/D10909

dirstate-v2: Drop cached read_dir results after .hgignore changes Soon we’ll want the status algorithm to be able to skip `std::fs::read_dir` in more cases, notabling when listing unknown files but not ignored files. When ignore patterns change (which we detect by their hash, added to the dirstate-v2 format in a previous changeset), a formerly-ignored file could become unknown without changing its parent directory’s modification time. Therefore we remove any directory mtime from the dirstate, effictively invalidating the existing caches. Differential Revision: https://phab.mercurial-scm.org/D10907

censor: split the core of the logic into its own function We now have a "generic" rewrite function (only able to do censoring for now) and a thin wrapper that implement the `censor` API with it. We are now ready to start incorporating strip specific changes. Differential Revision: https://phab.mercurial-scm.org/D10904

censor: migrate the logic to a set of `censor_revs` Instead of considering a special unique censored revision within the code, we consider a set of revision (currently always of size 1). This make the main code less censor-centric and prepare for the usage of a similar approach for stripping changesets. Differential Revision: https://phab.mercurial-scm.org/D10903

censor: rename `rl` to `revlog` in the main function Now that the bulk of the code moved to smaller function we are less restricted on line length and we can use more explicite naming. Differential Revision: https://phab.mercurial-scm.org/D10902

censor: extract the part about creating and opening new files in a function The v2_censor function is huge, now that its content has settled a bit it is a good time to split individual part inside dedicated function. The last part is the file copying and opening logic. It now have its own function. Differential Revision: https://phab.mercurial-scm.org/D10901

censor: extract the part about recomputing delta in a function The v2_censor function is huge, now that its content has settled a bit it is a good time to split individual part inside dedicated function. It is now the turn of the logic that recompute new delta to replace the one based on the revision that are going away. Differential Revision: https://phab.mercurial-scm.org/D10900

censor: extract the part about writing the other revision in a function The v2_censor function is huge, now that its content has settled a bit it is a good time to split individual part inside dedicated function. We continue with a small function that add a non-censored revision back to the revlog. Differential Revision: https://phab.mercurial-scm.org/D10899

censor: extract the part about writing the censored revision in a function The v2_censor function is huge, now that its content has settled a bit it is a good time to split individual part inside dedicated function. We start with a small function that process the censored revision. Differential Revision: https://phab.mercurial-scm.org/D10898

censor: put the tuple of open files in an explicit variable This will be simpler to pass these file around in future changesets. Differential Revision: https://phab.mercurial-scm.org/D10897

revlog: rewrite `censors.py` to `rewrite.py` The logic we use for censoring revision will be mostly common with the one we needs for stripping. So we rename the module to `rewrite` to better match its future content. Differential Revision: https://phab.mercurial-scm.org/D10896

copies: Keep changelog sidedata file open during copy tracing Instead of having a callback that opens and closes that file many times, a add and use a context manager method on the `revlog` class that keeps files open for its duration. Differential Revision: https://phab.mercurial-scm.org/D10888

purge: restore the `testedwith` declaration Otherwise, this extension gets highlighted as a 3rd party extension when an exception occurs. Differential Revision: https://phab.mercurial-scm.org/D10895

rust: remove dead utils module This was introduced a while back while not being imported anywhere in the module system. How pertinent this Facebook module was to the codebase was debatable at the time, nowadays with the Rust VFS layer having laid its foundations, it is clear that we should drop it. Differential Revision: https://phab.mercurial-scm.org/D10890

rebase: use str-keyed opts in remaining places Differential Revision: https://phab.mercurial-scm.org/D10892

rebase: keep str-keyed opts long enough to make `action` a str This is just another little py3 cleanup. Differential Revision: https://phab.mercurial-scm.org/D10891

benchmarks: restore `output` variable lost in D10884 Noticed by pyflakes (which I didn't have installed for Python 3.9 when I sent D10884). Differential Revision: https://phab.mercurial-scm.org/D10894

censor: implement censoring for revlogv2 It is a bit verbose and rough, but it works. Most of that logic can be common for `stripping`, so we can expect more refactoring of that code to accommodate both needs. However I wanted to keep this changesets "simple enough" and before moving forward. We also need to properly delete the older index/data/sidedata file, but this has implication for streaming clone and transaction, so this will come later. Differential Revision: https://phab.mercurial-scm.org/D10869

deltas: at a `target_rev` parameter to finddeltainfo Otherwise, recomputing a delta for a revision might result in a delta against a later revision or a full snapshot thinking we are appending a new revision. We will make use of this during censoring (and later, stripping). Differential Revision: https://phab.mercurial-scm.org/D10887

revlog: list older-but-still-around file in `files` These older files will eventually be removed, but we have to take them in account until they are. So we update `files` to keep fncache happy. Differential Revision: https://phab.mercurial-scm.org/D10868

revlog: add docket method to request new content files If we want to write content into new files, we need to be able to ask for them. Differential Revision: https://phab.mercurial-scm.org/D10867

revlog: add a way to keep track of older uids in the docket When the revlog content is rewritten, we will use new files, to avoid truncating the previous ones. We need some way to keep track of the older files before we clean them up. Differential Revision: https://phab.mercurial-scm.org/D10866

revlog: factor the logic to determine the delta compression out This make the logic clearly isolated and documented and it will help use to reuse it during censors/strip. Differential Revision: https://phab.mercurial-scm.org/D10889

nodemap: fix some comment formatting Black seems unhappy about it. Differential Revision: https://phab.mercurial-scm.org/D10865

revlogv2: fix `hg verify` with revlog v2 We added a bunch of new files but we never teached some internal about it. This is now fixed. Differential Revision: https://phab.mercurial-scm.org/D10858

revlogv2: also test that local clone works This is now the case so lets add a test. Differential Revision: https://phab.mercurial-scm.org/D10857