Tue, 15 Jun 2021 09:06:12 +0200 packaging: disable rust extensions again on CentOS stable
Mathias De Mare <mathias.de_mare@nokia.com> [Tue, 15 Jun 2021 09:06:12 +0200] rev 47344
packaging: disable rust extensions again on CentOS Backed out changeset eccbfa7e19c0 We're seeing (very rarely) crashes of 'hg purge' on some of our machines (see https://bz.mercurial-scm.org/show_bug.cgi?id=6509 ). Unfortunately, I haven't been able to find out much more about what is going wrong. To avoid further impact on our users and CI, I would prefer to disable the rust extensions for now. Differential Revision: https://phab.mercurial-scm.org/D10877
Sun, 06 Jun 2021 01:24:30 +0200 cext: fix memory leak in phases computation stable
Georges Racinet <georges.racinet@octobus.net> [Sun, 06 Jun 2021 01:24:30 +0200] rev 47343
cext: fix memory leak in phases computation Without this a buffer whose size in bytes is the number of changesets in the repository is leaked each time the repository is opened and changeset phases are computed. Impact: the current code in hgwebdir creates a new `localrepository` instance for each HTTP request. Since any pull or push is made of several requests, a team of 100 people can easily produce thousands of such requests per day. Being a low-level malloc, this leak can't be seen with the gc module and tools relying on that, but was spotted by valgrind immediately. Reproduction ------------ for i in range(cl_args.iterations): repo = hg.repository(baseui, repo_path) rev = repo.revs(rev).first() ctx = repo[rev] del ctx del repo # avoid any pollution by other type of leak # (that should be fixed in 5.8) repoview._filteredrepotypes.clear() gc.collect() Measurements ------------ Resident Set Size (RSS), taken on a clone of mozilla-central for performance analysis (440 000 changesets). before: 5.8+hg19.5ac0f2a8ba72 1000 iterations: 1606MB 5.8+hg19.5ac0f2a8ba72 10000 iterations: 5723MB after: 5.8+hg20.e2084d39e145 1000 iterations: 555MB 5.8+hg20.e2084d39e145 10000 iterations: 555MB (double checked, not a copy/paste error) (e2084d39e14 is the present changeset, before amendment of the message to add the measurements)
Sun, 30 May 2021 22:12:48 +0200 revlogv2: make sure bundling pick a compatible bundle format
Pierre-Yves David <pierre-yves.david@octobus.net> [Sun, 30 May 2021 22:12:48 +0200] rev 47342
revlogv2: make sure bundling pick a compatible bundle format Before this change, revlog-v2 repository where bundled using the incompatible "v1" format. Differential Revision: https://phab.mercurial-scm.org/D10802
Sun, 30 May 2021 20:42:51 +0200 censor: do not process sidedata of censored revision while bundling
Pierre-Yves David <pierre-yves.david@octobus.net> [Sun, 30 May 2021 20:42:51 +0200] rev 47341
censor: do not process sidedata of censored revision while bundling The revision is censored, we should ignore it. Differential Revision: https://phab.mercurial-scm.org/D10801
Fri, 28 May 2021 20:00:27 +0200 changegroup: fix deltachunk API to be consistent from one class to another
Pierre-Yves David <pierre-yves.david@octobus.net> [Fri, 28 May 2021 20:00:27 +0200] rev 47340
changegroup: fix deltachunk API to be consistent from one class to another Depending of the subclass the 8th index of `chunkdata` items was either a sidedata dict of a proto_flags integer. We have not fixed the inconsistency and we already return fixed "delta" items from `deltaiter`. Differential Revision: https://phab.mercurial-scm.org/D10778
Thu, 27 May 2021 12:10:59 -0400 fuzz: add hg to sys.path when constructing mpatch corpus
Augie Fackler <augie@google.com> [Thu, 27 May 2021 12:10:59 -0400] rev 47339
fuzz: add hg to sys.path when constructing mpatch corpus Differential Revision: https://phab.mercurial-scm.org/D10777
Wed, 19 May 2021 16:18:16 +0200 dirstate-tree: Skip readdir() in `hg status -mard`
Simon Sapin <simon.sapin@octobus.net> [Wed, 19 May 2021 16:18:16 +0200] rev 47338
dirstate-tree: Skip readdir() in `hg status -mard` When running the status algorithm in a mode where we don’t list unknown or ignored files, all we care about are files that are listed in the dirstate. We can there for skip making expensive calls to readdir() to list the contents of filesystem directories, and instead only run stat() to get the filesystem state of files listed in the dirstate. (This state may be an error for files that don’t exist anymore on the filesystem.) On 16 CPU threads, this reduces the time spent in the `status()` function for `hg status -mard` on an old snapshot of mozilla-central from ~70ms to ~50ms. Differential Revision: https://phab.mercurial-scm.org/D10752
Wed, 19 May 2021 13:15:00 +0200 dirstate-v2: Parse the dirstate lazily, with copy-on-write nodes
Simon Sapin <simon.sapin@octobus.net> [Wed, 19 May 2021 13:15:00 +0200] rev 47337
dirstate-v2: Parse the dirstate lazily, with copy-on-write nodes TODO: more description Differential Revision: https://phab.mercurial-scm.org/D10751
Wed, 19 May 2021 13:15:00 +0200 dirstate-v2: Make the dirstate bytes buffer available in more places
Simon Sapin <simon.sapin@octobus.net> [Wed, 19 May 2021 13:15:00 +0200] rev 47336
dirstate-v2: Make the dirstate bytes buffer available in more places Differential Revision: https://phab.mercurial-scm.org/D10750
Wed, 19 May 2021 13:15:00 +0200 dirstate-v2: Make more APIs fallible, returning Result
Simon Sapin <simon.sapin@octobus.net> [Wed, 19 May 2021 13:15:00 +0200] rev 47335
dirstate-v2: Make more APIs fallible, returning Result When parsing becomes lazy, parse error will potentially happen in more places. This propagates such errors to callers. Differential Revision: https://phab.mercurial-scm.org/D10749
Wed, 19 May 2021 13:15:00 +0200 dirstate-v2: Add a zero-size error type for dirstate v2 parse errors
Simon Sapin <simon.sapin@octobus.net> [Wed, 19 May 2021 13:15:00 +0200] rev 47334
dirstate-v2: Add a zero-size error type for dirstate v2 parse errors This error should only happen if Mercurial is buggy or the file is corrupted. It indicates for example that: * A part of the file refers to another part, and the byte offset or item count would cause reading out of bounds, beyond the end of the file. * The byte for an entry state has an invalid value When parsing becomes lazy, many more functions will return a `Result` with this error. Making it zero-size reduces the work that the `?` operator needs to do to pass around the error value. Differential Revision: https://phab.mercurial-scm.org/D10748
Wed, 19 May 2021 13:15:00 +0200 dirstate-tree: Add `NodeRef` and `ChildNodesRef` enums
Simon Sapin <simon.sapin@octobus.net> [Wed, 19 May 2021 13:15:00 +0200] rev 47333
dirstate-tree: Add `NodeRef` and `ChildNodesRef` enums They are used instead of `&Node` and `&ChildNodes` respectively. The `ChildNodes` type alias also becomes a similar enum. For now they only have one variant each, to be extended later. Adding enums now forces various use sites go through new methods instead of manipulating the underlying data structure directly. Differential Revision: https://phab.mercurial-scm.org/D10747
Wed, 19 May 2021 13:15:00 +0200 rust: Return owned instead of borrowed DirstateEntry in DirstateMap APIs
Simon Sapin <simon.sapin@octobus.net> [Wed, 19 May 2021 13:15:00 +0200] rev 47332
rust: Return owned instead of borrowed DirstateEntry in DirstateMap APIs This will enable the tree-based DirstateMap to not always have an actual DirstateEntry in memory for all nodes, but construct it on demand. Differential Revision: https://phab.mercurial-scm.org/D10746
Wed, 19 May 2021 13:15:00 +0200 dirstate-tree: Downgrade `&mut Node` to `&Node` in status and serialization
Simon Sapin <simon.sapin@octobus.net> [Wed, 19 May 2021 13:15:00 +0200] rev 47331
dirstate-tree: Downgrade `&mut Node` to `&Node` in status and serialization Mutable access is not used, and upcoming changes will make it more costly (with copy-on-write nodes that can be read from disk representation) Differential Revision: https://phab.mercurial-scm.org/D10745
Wed, 19 May 2021 13:15:00 +0200 dirstate-tree: Remove DirstateMap::iter_node_data_mut
Simon Sapin <simon.sapin@octobus.net> [Wed, 19 May 2021 13:15:00 +0200] rev 47330
dirstate-tree: Remove DirstateMap::iter_node_data_mut In an upcoming changeset we want DirstateMap to be able to work directly with nodes in their "on disk" representation, without always allocating corresponding in-memory data structures. Nodes would have two possible representations: one immutable "on disk" refering to the bytes buffer of the contents of the .hg/dirstate file, and one mutable with HashMap like the curren data structure. These nodes would have copy-on-write semantics: when an immutable node would need to be mutated, instead we allocate new mutable node for it and its ancestors. A mutable iterator of the entire tree would still be possible, but it would become much more expensive since we’d need to allocate mutable nodes for everything. Instead, remove this iterator. It was only used to clear ambiguous mtimes while serializing the `DirstateMap`. Instead clearing and serialization are now two separate passes. Clearing first uses an immutable iterator to collect the paths of nodes that need to be cleared, then accesses only those nodes mutably. Differential Revision: https://phab.mercurial-scm.org/D10744
Fri, 28 May 2021 17:33:20 -0400 merge with stable
Matt Harbison <matt_harbison@yahoo.com> [Fri, 28 May 2021 17:33:20 -0400] rev 47329
merge with stable
Wed, 26 May 2021 21:46:45 +0200 revlog: close the index file handle after the data one
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 26 May 2021 21:46:45 +0200] rev 47328
revlog: close the index file handle after the data one This make sure the data file is flushed before the index. preventing the index to reference unflushed data. Differential Revision: https://phab.mercurial-scm.org/D10776
Wed, 26 May 2021 21:35:51 +0200 revlog: simplify the try nesting in the `_writing` context
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 26 May 2021 21:35:51 +0200] rev 47327
revlog: simplify the try nesting in the `_writing` context Lets use a single try, with conditional cleanup. This make is easier to add a file handle dedicated to sidedata. Differential Revision: https://phab.mercurial-scm.org/D10775
Thu, 20 May 2021 21:54:21 +0200 revlogv2: add a `get_data` helper to grab the next piece of docket
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 20 May 2021 21:54:21 +0200] rev 47326
revlogv2: add a `get_data` helper to grab the next piece of docket This make the processing more compact but abstracting repetitive processing away. Differential Revision: https://phab.mercurial-scm.org/D10774
Thu, 20 May 2021 21:48:53 +0200 revlogv2: simplify and clarify the processing of each entry
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 20 May 2021 21:48:53 +0200] rev 47325
revlogv2: simplify and clarify the processing of each entry As we add more entries and some of them has non trivial processing it seems useful to make the processing leaner and clearly separated to simplify futures patches. Differential Revision: https://phab.mercurial-scm.org/D10773
Wed, 19 May 2021 16:55:36 +0200 revlogv2: use a unique filename for data
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 19 May 2021 16:55:36 +0200] rev 47324
revlogv2: use a unique filename for data Having a unique data will allow for ambiguity less rewriting of revlog content, something useful to clarify handling of some operation like censoring or stripping. Differential Revision: https://phab.mercurial-scm.org/D10772
Tue, 18 May 2021 15:07:17 +0200 revlogv2: use a unique filename for index
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 18 May 2021 15:07:17 +0200] rev 47323
revlogv2: use a unique filename for index Having a unique index will allow for ambiguity less rewriting of revlog content, something useful to clarify handling of some operation like censoring or stripping. Differential Revision: https://phab.mercurial-scm.org/D10771
Sat, 22 May 2021 17:35:54 +0200 rhg: A missing .hg/dirstate file is not an error
Simon Sapin <simon.sapin@octobus.net> [Sat, 22 May 2021 17:35:54 +0200] rev 47322
rhg: A missing .hg/dirstate file is not an error Instead treat it as like an empty file Differential Revision: https://phab.mercurial-scm.org/D10766
Sat, 22 May 2021 17:32:09 +0200 rhg: Sort `rhg status` output correctly
Simon Sapin <simon.sapin@octobus.net> [Sat, 22 May 2021 17:32:09 +0200] rev 47321
rhg: Sort `rhg status` output correctly * The relative order of states is: modified, added, removed, deleted, unknown, ignored, clean * Files in the same state should be sorted by name, regardless of whether or not the were in "unsure" state based on metadata alone. Differential Revision: https://phab.mercurial-scm.org/D10765
Wed, 19 May 2021 18:35:43 +0200 dirstate-v2: Add `hg debugupgraderepo` command support
Simon Sapin <simon.sapin@octobus.net> [Wed, 19 May 2021 18:35:43 +0200] rev 47320
dirstate-v2: Add `hg debugupgraderepo` command support This command changes changes the file formats used inside an existing repository to what they would be in a new repository with the current config. For example: hg debugupgraderepo --config format.exp-dirstate-v2=1 --run hg debugupgraderepo --config format.exp-dirstate-v2=0 --run If a repository has a dirstate in v1 format, the first command would upgrade it to dirstate-v2. Conversely, if a repository has a dirstate in v2 format, the second command would downgrade it to v1. (Both may also run some unrelated upgrades.) Since `format.exp-dirstate-v2` is currently disabled by default, not specifying it in `--config` or any configuration file would result in the second command. Differential Revision: https://phab.mercurial-scm.org/D10769
Fri, 21 May 2021 17:12:47 +0200 upgrade: Use `improvement` subclasses everywhere, not instances
Simon Sapin <simon.sapin@octobus.net> [Fri, 21 May 2021 17:12:47 +0200] rev 47319
upgrade: Use `improvement` subclasses everywhere, not instances This changes the source definition of optimizations to match that of formats: a subclass with a decorator, instead of an instance passed to a function call. Not having any instance removes the confusion between class attributes and instance attributes, which were used interchangeably. Differential Revision: https://phab.mercurial-scm.org/D10768
Thu, 20 May 2021 14:20:39 -0400 tests: monkeypatch `util.get_password()` to avoid deadlocks on Windows stable
Matt Harbison <matt_harbison@yahoo.com> [Thu, 20 May 2021 14:20:39 -0400] rev 47318
tests: monkeypatch `util.get_password()` to avoid deadlocks on Windows This should have been part of 5b3513177f2b. Differential Revision: https://phab.mercurial-scm.org/D10759
Tue, 04 May 2021 10:46:50 +0200 rust-status: fix ignore and include not composing (issue6514) stable
Raphaël Gomès <rgomes@octobus.net> [Tue, 04 May 2021 10:46:50 +0200] rev 47317
rust-status: fix ignore and include not composing (issue6514) While the fix is pretty simple, the overall dispatch logic has become kind of ugly. Thankfully we're currently upstreaming a better algorithm, this code is temporary anyway. Differential Revision: https://phab.mercurial-scm.org/D10639
(0) -30000 -10000 -3000 -1000 -300 -100 -50 -28 +28 +50 +100 +300 +1000 +3000 tip