Simon Sapin <simon.sapin@octobus.net> [Tue, 06 Apr 2021 15:49:01 +0200] rev 47111
rust: Add doc-comments to DirstateStatus fields
Differential Revision: https://phab.mercurial-scm.org/D10495
Simon Sapin <simon.sapin@octobus.net> [Tue, 06 Apr 2021 15:14:19 +0200] rev 47110
rust: Move "lookup" a.k.a. "unsure" paths into `DirstateStatus` struct
Instead of having `status()` returning a tuple of those paths and
`DirstateStatus`.
Differential Revision: https://phab.mercurial-scm.org/D10494
Simon Sapin <simon.sapin@octobus.net> [Tue, 13 Apr 2021 17:02:58 +0200] rev 47109
rust: Remove DirstateMap::file_fold_map
This was a HashMap constructed on demand and then cached in the DirstateMap
struct to avoid reconstructing at the next access. However the only use is
in Python bindings converting it to a PyDict. That method in turn is wrapped
in a @cachedproperty in Python code.
This was two redudant layers of caching. This changeset removes the Rust-level
one to keep the Python dict cache, and have bindings create a PyDict by
iterating.
Differential Revision: https://phab.mercurial-scm.org/D10493
Simon Sapin <simon.sapin@octobus.net> [Fri, 09 Apr 2021 13:13:19 +0200] rev 47108
dirstate-tree: Add "non normal" and "from other parent" sets
Unlike the other DirstateMap implementation, these sets are not materialized
separately in memory. Instead we traverse the main tree.
Differential Revision: https://phab.mercurial-scm.org/D10492
Simon Sapin <simon.sapin@octobus.net> [Fri, 09 Apr 2021 12:55:35 +0200] rev 47107
dirstate-tree: Add add_file, remove_file, and drop_file
Again, various counters need to be kept up to date.
Differential Revision: https://phab.mercurial-scm.org/D10491
Simon Sapin <simon.sapin@octobus.net> [Mon, 12 Apr 2021 19:46:24 +0200] rev 47106
dirstate-tree: Add has_dir and has_tracked_dir
A node without a `DirstateMap` entry represents a directory.
Only some values of `EntryState` represent tracked files.
A directory is considered "tracked" if it contains any descendant file that
is tracked. To avoid a sub-tree traversal in `has_tracked_dir` we add a
counter for this. A boolean flag would become insufficent when we implement
remove_file and drop_file.
`add_file_node` is more general than needed here, in anticipation of adding
the `add_file` and `remove_file` methods.
Differential Revision: https://phab.mercurial-scm.org/D10490
Simon Sapin <simon.sapin@octobus.net> [Mon, 12 Apr 2021 18:42:51 +0200] rev 47105
dirstate-tree: Add clear_ambiguous_times in the new DirstateMap
Also drive-by refactor it in the other DirstateMap
Differential Revision: https://phab.mercurial-scm.org/D10489
Simon Sapin <simon.sapin@octobus.net> [Mon, 12 Apr 2021 17:53:37 +0200] rev 47104
dirstate-tree: Add copy_map_insert and copy_map_remove
Differential Revision: https://phab.mercurial-scm.org/D10488
Simon Sapin <simon.sapin@octobus.net> [Mon, 12 Apr 2021 17:29:55 +0200] rev 47103
dirstate-tree: Maintain a counter of DirstateEntry’s and copy sources
This allows implementing __len__ for DirstateMap and CopyMap efficiently,
without traversing the tree.
Differential Revision: https://phab.mercurial-scm.org/D10487
Simon Sapin <simon.sapin@octobus.net> [Mon, 12 Apr 2021 14:21:47 +0200] rev 47102
dirstate-tree: Serialize to disk
The existing `pack_dirstate` function relies on implementation details
of `DirstateMap`, so extract some parts of it as separate functions
for us in the tree-based `DirstateMap`.
The `bytes-cast` crate is updated to a version that has an `as_bytes` method,
not just `from_bytes`:
https://docs.rs/bytes-cast/0.2.0/bytes_cast/trait.BytesCast.html#method.as_bytes
Drive-by refactor `clear_ambiguous_times` which does part of the same thing.
Differential Revision: https://phab.mercurial-scm.org/D10486
Simon Sapin <simon.sapin@octobus.net> [Mon, 12 Apr 2021 14:43:45 +0200] rev 47101
rust: Add a Timestamp struct instead of abusing Duration
`SystemTime` would be the standard library type semantically appropriate
instead of `Duration`.
But since the value is coming from Python as a plain integer and used in
dirstate packing code as an integer, let’s make a type that contains a single
integer instead of using one with sub-second precision.
Differential Revision: https://phab.mercurial-scm.org/D10485
Simon Sapin <simon.sapin@octobus.net> [Tue, 06 Apr 2021 21:07:12 +0200] rev 47100
dirstate-tree: Add tree traversal/iteration
Like Python’s, Rust’s iterators are "external" in that they are driven
by a caller who calls a `next` method. This is as opposed to "internal"
iterators who drive themselves and call a callback for each item.
Writing an internal iterator traversing a tree is easy with recursion,
but internal iterators cannot rely on the call stack in that way,
they must save in an explicit object all state that they need to be
preserved across two `next` calls.
This algorithm uses a `Vec` as a stack that contains what would be
local variables on the call stack if we could use recursion.
Differential Revision: https://phab.mercurial-scm.org/D10370
Simon Sapin <simon.sapin@octobus.net> [Tue, 06 Apr 2021 14:35:39 +0200] rev 47099
dirstate-tree: Add map `get` and `contains_key` methods
Differential Revision: https://phab.mercurial-scm.org/D10369
Simon Sapin <simon.sapin@octobus.net> [Tue, 06 Apr 2021 14:29:05 +0200] rev 47098
dirstate-tree: Add parsing only dirstate parents from disk
Differential Revision: https://phab.mercurial-scm.org/D10368
Simon Sapin <simon.sapin@octobus.net> [Wed, 31 Mar 2021 18:59:49 +0200] rev 47097
dirstate-tree: Implement DirstateMap::read
This reads the on-disk dirstate in its current format (a flat sequence of
entries) and creates a tree in memory.
Differential Revision: https://phab.mercurial-scm.org/D10367
Simon Sapin <simon.sapin@octobus.net> [Thu, 08 Apr 2021 20:12:24 +0200] rev 47096
dirstate-tree: Add `WithBasename` wrapper for `HgPath`
In the tree-shaped dirstate we want to have nodes representing files or
directories, where directory nodes contain a map associating "base" names
to child nodes for child files and directories.
Many dirstate operations expect a full path from the repository root, but
re-concatenating string from nested map keys all the time might be expensive.
Instead, `WithBasename` stores a full path for these operations but
behaves as its base name (last path component) for equality and comparison.
Additionally `inclusive_ancestors` provides the successive map keys
that are needed when inserting a new dirstate node at a given full path.
Differential Revision: https://phab.mercurial-scm.org/D10365
Simon Sapin <simon.sapin@octobus.net> [Tue, 30 Mar 2021 09:56:04 +0200] rev 47095
dirstate-tree: Empty shell for a second Rust DirstateMap implementation
For background see description of the previous changeset
"Make Rust DirstateMap bindings go through a trait object".
Add an empty shell for a opt-in second Rust implementation of the
`DirstateMap` type and the `status` function. For now all methods panic.
This can be seen in "action" with:
./hg status --config experimental.dirstate-tree.in-memory=1
Differential Revision: https://phab.mercurial-scm.org/D10364
Simon Sapin <simon.sapin@octobus.net> [Thu, 08 Apr 2021 14:58:44 +0200] rev 47094
dirstate-tree: Abstract "non-normal" and "other parent" sets
Instead of exposing `HashSet`s directly, have slightly higher-level
methods for the operations that Python bindings need on them.
Differential Revision: https://phab.mercurial-scm.org/D10363
Simon Sapin <simon.sapin@octobus.net> [Tue, 30 Mar 2021 14:15:23 +0200] rev 47093
dirstate-tree: Make Rust DirstateMap bindings go through a trait object
This changeset starts a series that adds an experiment to make status faster
by changing the dirstate (first only in memory and later also on disk) to
be shaped as a tree matching the directory structure, instead of the current
flat collection of entries. The status algorithm can then traverse this tree
dirstate at the same time as it traverses the filesystem.
We (Octobus) have made prototypes that show promising results but are prone
to bitrot. We would like to start upstreaming some experimental Rust code that
goes in this direction, but to avoid disrupting users it should only be
enabled by some run-time opt-in while keeping the existing dirstate structure
and status algorithm as-is.
The `DirstateMap` type and `status` function look like the appropriate
boundary. This adds a new trait that abstracts everything Python bindings need
and makes those bindings go through a `dyn` trait object. Later we’ll have two
implementations of this trait, and the same bindings can use either.
Differential Revision: https://phab.mercurial-scm.org/D10362
Kévin Lévesque <klevesque@innovmetric.com> [Wed, 05 May 2021 18:26:04 -0400] rev 47092
remotefilelog: use the correct capability when using getfilestype threaded
The functon was overlooked when the capability was renamed
Differential Revision: https://phab.mercurial-scm.org/D10673
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 19 Apr 2021 11:22:24 +0200] rev 47091
test-copies: test that copies' sidedata can get computed during push
If the source of the push does not have the necessary sidedata but the server
needs them, the client should compute them on the fly while pushing.
Differential Revision: https://phab.mercurial-scm.org/D10350
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 19 Apr 2021 11:22:24 +0200] rev 47090
test-copies: test that copies' sidedata can get computed during pull
If the source does not have the data, the pulling client should compute the necessary side-data while pulling.
Differential Revision: https://phab.mercurial-scm.org/D10349
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 19 Apr 2021 11:22:24 +0200] rev 47089
test-copies: test that copies' sidedata does not get corrupted during push
This is an important usecase.
Differential Revision: https://phab.mercurial-scm.org/D10348
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 19 Apr 2021 11:22:24 +0200] rev 47088
test-copies: test that copies' sidedata does not get corrupted during pull
This is an important usecase.
Differential Revision: https://phab.mercurial-scm.org/D10347
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 19 Apr 2021 11:22:24 +0200] rev 47087
test-copies: simplify some conditional output
Now that all computation using sidedata give the same result, we can
simplify conditional
Differential Revision: https://phab.mercurial-scm.org/D10346
Raphaël Gomès <rgomes@octobus.net> [Mon, 19 Apr 2021 11:22:24 +0200] rev 47086
sidedata: move documentation about sidedata helpers to sidedata module
Differential Revision: https://phab.mercurial-scm.org/D10361
Raphaël Gomès <rgomes@octobus.net> [Mon, 19 Apr 2021 11:22:24 +0200] rev 47085
sidedata: move sidedata-related utils to the dedicated module
Differential Revision: https://phab.mercurial-scm.org/D10360
Raphaël Gomès <rgomes@octobus.net> [Mon, 19 Apr 2021 11:22:24 +0200] rev 47084
sidedata: replace sidedata upgrade mechanism with the new one
Note: this is split into a separate change (like some other patches in this
series) because it's not easy to have all patches work 100% and this seemed
easier for reviewers.
When cloning or upgrading a repo, we may need to compute (or remove) sidedata.
This is the same mechanism that is used in exchange, so we re-use the new
system to simplify the code and fix the remaining issues (correctly dropping
flags and handling partial removal, etc.).
This also highlighted an issue with `test-copies-in-changeset.t` that kept
sidedata categories that are not relevant anymore. They should probably be
dropped entirely, but that would be for another patch.
Differential Revision: https://phab.mercurial-scm.org/D10359
Raphaël Gomès <rgomes@octobus.net> [Mon, 19 Apr 2021 11:22:21 +0200] rev 47083
sidedata: add a way of replacing an existing sidedata computer
This will be useful in a future patch to replace a sequential computer with
a parallel computer. We only allow for explicit replacement, to force the users
to think about overriding computers.
Differential Revision: https://phab.mercurial-scm.org/D10358
Raphaël Gomès <rgomes@octobus.net> [Thu, 08 Apr 2021 16:30:10 +0200] rev 47082
bundle2: remove restriction around sidedata
We are now capable of generating the missing sidedata on-the-fly.
Differential Revision: https://phab.mercurial-scm.org/D10345
Matt Harbison <matt_harbison@yahoo.com> [Mon, 12 Apr 2021 20:58:19 -0400] rev 47081
heptapod-ci: enable pytype checking
The trigger is manual for now, until we get a better idea about stability,
resource usage, etc.
Differential Revision: https://phab.mercurial-scm.org/D10462
Matt Harbison <matt_harbison@yahoo.com> [Thu, 06 May 2021 18:40:23 -0400] rev 47080
hghave: fix the definition of `python3` to work on Windows
Both py2 and py3 executables are named `python.exe`, and may or may not be on
PATH. So use the dispatcher executable that comes with py3 to fetch the version
of the latest py3 executable. This allows at least one relnotes test to run on
Windows.
Differential Revision: https://phab.mercurial-scm.org/D10694
Matt Harbison <matt_harbison@yahoo.com> [Wed, 12 May 2021 12:41:52 -0400] rev 47079
util: avoid echoing the password to the console on Windows py3 (
issue6446)
The `getpass.getpass()` implementation on Windows first checks if `sys.stdin`
and `sys.__stdin__` are the same object. It's not on py3 because the former is
replaced in dispatch.py with something that doesn't normalize '\n' to '\r\n'.
When they aren't the same object, it simply calls `sys.stdin.readline()` instead
of the mscvrt functions that read the input characters before they are echoed.
This simply copies the `getpass.win_getpass()` implementation without the stdin
check, and byteifies around the edges. I'm not sure if there's a reasonable
replacement for the check that we could implement. When echoing input into the
hg command, the `ui.interactive()` check causes `ui.getpass()` to bail before
getting here. If the proper config switches are used to bypass that and call
this, the process stalls until '\n' is input into the console. So there could
be a deadlock here when run by another command if the wrong config settings are
applied.
Differential Revision: https://phab.mercurial-scm.org/D10708
Raphaël Gomès <rgomes@octobus.net> [Thu, 08 Apr 2021 16:55:17 +0200] rev 47078
sidedata: enable sidedata computers to optionally rewrite flags
Sidedata computers may want to influence the flags of the revision they touch.
For example, the computer for changelog-based copytracing can add a flag to
signify that this revision might affect copytracing, inversely removing said
flag if the information is no longer applicable.
See inline documentation in `storageutil` for more details.
Differential Revision: https://phab.mercurial-scm.org/D10344
Raphaël Gomès <rgomes@octobus.net> [Sat, 10 Apr 2021 11:27:40 +0200] rev 47077
cg4: introduce protocol flag to signify the presence of sidedata
We need a way of signaling whether the current revision has sidedata or not,
and re-using the revision flags would waste potential revlog flags and mix two
normally independent layers.
In this change, we add a single byte at the start of the ch4 delta header to
set potential protocol flags. We also reclaim the revlog flag for sidedata,
since it is no longer used, in its place now lives the (also experimental)
copytracing flag.
When generating deltas, apply the `CG_FLAG_SIDEDATA` flag if there is sidedata.
When applying the deltas, if said flag is present, the next chunk contains the
sidedata.
Differential Revision: https://phab.mercurial-scm.org/D10343
Raphaël Gomès <rgomes@octobus.net> [Thu, 08 Apr 2021 16:34:11 +0200] rev 47076
changegroup: don't limit cgv4 to revlogv2 repos
To help the transition from revlogv1 to revlogv2, we need to be able to enable
cgv4 for revlogv1 repos, so that revlogv2 clients can handle adding/removing
sidedata over the wire.
Differential Revision: https://phab.mercurial-scm.org/D10342
Raphaël Gomès <rgomes@octobus.net> [Thu, 08 Apr 2021 16:39:39 +0200] rev 47075
sidedata: gate sidedata functionality to revlogv2 in more places
Since revlogv1 is not capable of storing sidedata, we prevent sidedata
mechanisms around the revlog layer from doing anything. We however keep the
ones that allow a revlogv1 repo to generate sidedata on-the-fly on a push, the
pull case simply does not add the sidedata to the revlog.
Differential Revision: https://phab.mercurial-scm.org/D10341
Raphaël Gomès <rgomes@octobus.net> [Tue, 30 Mar 2021 17:03:02 +0200] rev 47074
sidedata: register copies sidedata computer regardless of the revlog version
Repositories should not gate their sidedata computers based on any requirement,
only their wanted sidedata. A repository might need to generate sidedata wanted
by the peer that it itself does not want.
Differential Revision: https://phab.mercurial-scm.org/D10340
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 08 Apr 2021 19:00:21 +0200] rev 47073
revlog: replace the old `revlog_kind` approach with the new `target` one
The new `target` attribute supersedes the previous one.
Differential Revision: https://phab.mercurial-scm.org/D10353
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 06 Apr 2021 05:20:24 +0200] rev 47072
revlog: introduce an explicit tracking of what the revlog is about
Since the dawn of time, people have been forced to rely to lossy introspection
of the index filename to determine what the purpose and role of the revlog they
encounter is. This is hacky, error prone, inflexible, abstraction-leaky,
<insert-your-own-complaints-here>.
In
f63299ee7e4d Raphaël introduced a new attribute to track this information:
`revlog_kind`. However it is initialized in an odd place and various instances
end up not having it set. In addition is only tracking some of the information
we end up having to introspect in various pieces of code.
So we add a new attribute that holds more data and is more strictly enforced.
This work is done in collaboration with Raphaël.
The `revlog_kind` one will be removed/adapted in the next changeset. We expect
to be able to clean up various existing piece of code and to simplify coming
work around the newer revlog format.
Differential Revision: https://phab.mercurial-scm.org/D10352
Martin von Zweigbergk <martinvonz@google.com> [Tue, 04 May 2021 08:54:28 -0700] rev 47071
config: add --source option to include source of value
Showing the source of each config option is quite useful and not
something the user should have to reach for the `--debug` flag for.
I updates documentation and tests, except for one place in
`test-hgrc.t` where I thought the test might have been intended to
also test that `--debug` results in `ui.quiet` etc being test.
Differential Revision: https://phab.mercurial-scm.org/D10668
Martin von Zweigbergk <martinvonz@google.com> [Tue, 04 May 2021 10:49:32 -0700] rev 47070
rewriteutil: say how many commits would become orphan if commit is rewritten
This copies the message from the evolve extension, but modifies it a
bit to work with the grammar (in particular with the use of "change
branch of" as `action`). I don't know why it doesn't use the same
`_formatrevs()` as for public commmits.
Differential Revision: https://phab.mercurial-scm.org/D10671
Martin von Zweigbergk <martinvonz@google.com> [Tue, 04 May 2021 10:16:34 -0700] rev 47069
rewriteutil: give examples of public changesets that can't be rewritten
This patch copies the feature from the evolve extension.
Differential Revision: https://phab.mercurial-scm.org/D10670
Nate Skulic <nate.skulic@gmail.com> [Mon, 10 May 2021 00:54:08 +0000] rev 47068
convert: update p4 for Python 3
Differential Revision: https://phab.mercurial-scm.org/D10703
Matt Harbison <matt_harbison@yahoo.com> [Sun, 02 May 2021 16:56:20 -0400] rev 47067
tests: change the fixer commands to use the buffer attribute on stdio objects
Otherwise `\r` was getting injected into the fixed lines and throwing off the
commit hashes on Windows when the fixer is invoked with py3.
Differential Revision: https://phab.mercurial-scm.org/D10637
Matt Harbison <matt_harbison@yahoo.com> [Sat, 01 May 2021 16:13:53 -0400] rev 47066
tests: stabilize test-persistent-nodemap.t on Windows
Several issues here:
- Hooks can't invoke shell scripts on Windows, so use `sh` to launch
- `dd` in MSYS only recognizes `status=noxfer`
- The `PATH` updating triggered a massive slowdown, but is no longer needed
I have no idea why, but removing the `PATH` update substantially increased the
speed of the test. It was running finishing at ~4:30 with `--debug` and ~14:50
without it, but now completes in ~2:20 on my Windows laptop.
Differential Revision: https://phab.mercurial-scm.org/D10636
Matt Harbison <matt_harbison@yahoo.com> [Sun, 02 May 2021 19:54:08 -0400] rev 47065
tests: invoke some shell scripts through the shell interpreter for Windows
Otherwise, Windows was prompting what program to use to open the file (or just
opening it if there was a file association configured).
Differential Revision: https://phab.mercurial-scm.org/D10635
Matt Harbison <matt_harbison@yahoo.com> [Sat, 01 May 2021 13:04:56 -0400] rev 47064
tests: run python script through quoted interpreter instead of directly
This helps Windows when python is installed to %PROGRAMFILES%.
Differential Revision: https://phab.mercurial-scm.org/D10634
Matt Harbison <matt_harbison@yahoo.com> [Sat, 01 May 2021 00:41:43 -0400] rev 47063
tests: ensure `$PYTHON` is quoted for Windows
Global installs of python3 go into "Program Files", and tons of tests fail with
mysterious errors if this isn't quoted. Most of this is a followup to
0826d684a1b5, but a some of these were existing issues. Shebang lines are
ignored because quoting breaks direct execution- these will need to be launched
indirectly with the quoted `$PYTHON` command.
Differential Revision: https://phab.mercurial-scm.org/D10633
Kyle Lippincott <spectral@google.com> [Fri, 30 Apr 2021 16:13:02 -0700] rev 47062
black: make codebase compatible with black v21.4b2 and v20.8b1
I don't know what exact version of black made it care about these whitespace
differences, but this is the version I got when I just installed it with
`pip3 install black`.
I'm intentionally not increasing the version of black required, as I don't want
to force everyone to upgrade their version of black, and these fixes are
backwards compatible with black v20.8b1. If there are more issues in the future
and this becomes a maintenance burden I may do so in a future change.
Tested with both versions of black (I got the older version via
`pip3 install black==20.8b1`)
Differential Revision: https://phab.mercurial-scm.org/D10539
Augie Fackler <augie@google.com> [Tue, 04 May 2021 13:28:42 -0400] rev 47061
debugcommands: fix some plural-agreements I noticed
Differential Revision: https://phab.mercurial-scm.org/D10669
Augie Fackler <augie@google.com> [Mon, 03 May 2021 15:14:09 -0400] rev 47060
merge: with stable
Pulkit Goyal <7895pulkit@gmail.com> [Mon, 03 May 2021 22:53:02 +0530] rev 47059
Added signature for changeset
067f2c53fb24
Pulkit Goyal <7895pulkit@gmail.com> [Mon, 03 May 2021 22:52:56 +0530] rev 47058
Added tag 5.8 for changeset
067f2c53fb24
Raphaël Gomès <rgomes@octobus.net> [Mon, 03 May 2021 18:55:19 +0200] rev 47057
branching: merge stable into default
Matt Harbison <matt_harbison@yahoo.com> [Sat, 01 May 2021 00:28:39 -0400] rev 47056
phabricator: adapt to the new `urlutil.url()` API
This avoids a bunch of deprecation warnings in the tests.
Differential Revision: https://phab.mercurial-scm.org/D10541
Matt Harbison <matt_harbison@yahoo.com> [Fri, 30 Apr 2021 17:36:09 -0400] rev 47055
extensions: ignore exceptions from an extension's `getversion()` method
This method is usually called when there's a stacktrace being generated, or with
`hg version -v`. Raising another exception risks mangling the bug report info.
I hit this issue when trying to add the method to the keyring extension to
report the version of the extension and the underlying module, and ran into
demandimport issues prior to py3.8. It seems like a wise thing to do anyway,
though unfortunately there's no convenient `ui` object around to issue a
warning. Use 'unknown' to signal that it tried to report a version and failed,
unlike the default case of printing nothing.
Differential Revision: https://phab.mercurial-scm.org/D10540
Matt Harbison <matt_harbison@yahoo.com> [Wed, 28 Apr 2021 17:05:32 -0400] rev 47054
git: ensure all dirstate state values are bytes
I'm not sure how this particular git status occurs, but after the fallout of
issue 6510 and getting into the issue 6511 state where `git status` shows the
files as modified in both the "to be committed" and "not staged" lists,
`hg diff` was crashing in `workingctx.__contains__()`.
Differential Revision: https://phab.mercurial-scm.org/D10532
Matt Harbison <matt_harbison@yahoo.com> [Wed, 28 Apr 2021 10:29:45 -0400] rev 47053
tests: synchronize the git and Mercurial username
The problem with the default name of "test" set by the test runner is the
stringutil methods are unable to split out separate user and email addresses
that git wants. This means the username is recorded in git as "test <test>".
Amending a commit with that user ends up trying to use "<test>" as the person
field for the new commit, and the git library complains about the angle
brackets. We should probably abort with a clearer message any time this bad
form is used with the git extension.
One of the commit dates is tweaked to recreate the ambiguous hash prefix from
before.
Differential Revision: https://phab.mercurial-scm.org/D10531
Matt Harbison <matt_harbison@yahoo.com> [Tue, 27 Apr 2021 19:38:19 -0400] rev 47052
git: initialize `extra` to have at least the branch name for nullid
Otherwise, this crashes trying to convert to local encoding:
...
File "/mnt/c/Users/Matt/hg/mercurial/logcmdutil.py", line 333, in _show
branch = ctx.branch()
File "/mnt/c/Users/Matt/hg/mercurial/context.py", line 675, in branch
return encoding.tolocal(self._changeset.extra.get(b"branch"))
File "/mnt/c/Users/Matt/hg/mercurial/encoding.py", line 181, in tolocal
if isasciistr(s):
TypeError: a bytes-like object is required, not 'NoneType'
This was originally reported to the thg bug tracker.
https://foss.heptapod.net/mercurial/tortoisehg/thg/-/issues/5629
Differential Revision: https://phab.mercurial-scm.org/D10528
Matt Harbison <matt_harbison@yahoo.com> [Tue, 27 Apr 2021 18:39:59 -0400] rev 47051
git: consistently use str for parents when rebuilding the index database
The tests show no changes, but when these values are overwritten shortly after
when a git commit object is available, that uses str. It seems better to use
that for consistency.
It does materially affect the database though, because the old value stored was
`X'3030..3030'` and is now '00..00' when the changelog table is dumped in
sqlite3. There is one query that specifies the parents, but it passes the non
null hashes as str, so it worked as expected. That likely explains the lack of
test changes.
Differential Revision: https://phab.mercurial-scm.org/D10527
Matt Harbison <matt_harbison@yahoo.com> [Tue, 27 Apr 2021 17:54:08 -0400] rev 47050
git: pass the correct type to the sqlite3 query for baselog.hasnode()
It looks like this function is mostly used in exchange (which isn't supported
now) and histedit (which crashes). But I did verify in a REPL that passing
bytes to `db.execute()` fails to find the row, whereas passing str works.
Differential Revision: https://phab.mercurial-scm.org/D10526
Matt Harbison <matt_harbison@yahoo.com> [Tue, 27 Apr 2021 17:11:55 -0400] rev 47049
git: use the correct type for stopping changelog.revs()
The `tip` function returns a binary node, but the database is expecting an int.
Differential Revision: https://phab.mercurial-scm.org/D10525
Matt Harbison <matt_harbison@yahoo.com> [Tue, 27 Apr 2021 16:58:59 -0400] rev 47048
git: use the proper filenode for deleted files in changelog.changelogrevision
Inline printing shows it still doesn't find the removed file in the test, but it
is at least a str instead of bytes like the immediate query before it that does
find files. Looking at the database, it doesn't look like the remove was
recorded. Additionally, `hg log -r 'removes("re:.*")'` stacktraces.
Differential Revision: https://phab.mercurial-scm.org/D10524
Matt Harbison <matt_harbison@yahoo.com> [Tue, 27 Apr 2021 12:59:17 -0400] rev 47047
tests: add coverage for git.changelog.headrevs(...)
Differential Revision: https://phab.mercurial-scm.org/D10523
Matt Harbison <matt_harbison@yahoo.com> [Tue, 27 Apr 2021 12:31:30 -0400] rev 47046
git: make changelog.tiprev() return int instead of tuple (
issue6510)
Differential Revision: https://phab.mercurial-scm.org/D10522
Matt Harbison <matt_harbison@yahoo.com> [Tue, 27 Apr 2021 00:26:12 -0400] rev 47045
git: fix partial node matching
Looks like everything is meant to be a str here.
Differential Revision: https://phab.mercurial-scm.org/D10521
Matt Harbison <matt_harbison@yahoo.com> [Tue, 27 Apr 2021 00:23:05 -0400] rev 47044
tests: stabilize test-git-interop.t for some versions of git
Not sure why this differs on my system, but it's likely due to running 2.17.1 on
Ubuntu 18.04.
Differential Revision: https://phab.mercurial-scm.org/D10520
Joerg Sonnenberger <joerg@bec.de> [Fri, 30 Apr 2021 02:11:58 +0200] rev 47043
manifests: push down expected node length into the parser
This strictly enforces the node length in the manifest lines according
to what the repository expects. One test case moves large hash testing
into the non-treemanifest part as treemanifests don't provide an
interface for overriding just the node length for now.
Differential Revision: https://phab.mercurial-scm.org/D10533
Joerg Sonnenberger <joerg@bec.de> [Fri, 30 Apr 2021 03:19:45 +0200] rev 47042
core: don't hard-code node length
Differential Revision: https://phab.mercurial-scm.org/D10536
Joerg Sonnenberger <joerg@bec.de> [Fri, 30 Apr 2021 03:09:16 +0200] rev 47041
core: don't hard-code hex node lengths
Differential Revision: https://phab.mercurial-scm.org/D10535
Joerg Sonnenberger <joerg@bec.de> [Fri, 30 Apr 2021 02:25:10 +0200] rev 47040
tests: bump default timeout to 360s
A number of tests hit or almost hit the default limit even on modern
hardware. While the tests are ideally split into smaller pieces, that's
non-trivial work. HyperThreading and similar technologies can trigger
this often, even without any other load on the machine.
Differential Revision: https://phab.mercurial-scm.org/D10534
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 02:33:00 +0200] rev 47039
enforcesinglehead-test: add the expected node output next to the error
this make it simpler to validate that the test is correct.
Differential Revision: https://phab.mercurial-scm.org/D10545
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 08 Apr 2021 00:34:16 +0200] rev 47038
revlog: code for `revlogv0` in its own module
This code is mostly unused compatiblity code. Yet it take a prohiminent place in
the `revlog.py` module. That module is already quite big, so we move all that
code in a dedicated module.
Differential Revision: https://phab.mercurial-scm.org/D10511
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 01 Apr 2021 11:31:54 +0200] rev 47037
revlog: have an explicit "pack_header" method
Having to pass the version header when retrieving the binary version of every
single entry is a bit silly. So we extract that special logic in its own method.
This also prepare the move to newer revlog format, not storing the header within
an actual entry…
Differential Revision: https://phab.mercurial-scm.org/D10510
Pierre-Yves David <pierre-yves.david@octobus.net> [Sat, 01 May 2021 14:47:39 +0200] rev 47036
revlog: remove the revlogio class
The class only contains a single `parseindex` method. Lets just make it a
function and remove the `revlogio` class. It served us well but became thinner
and thinner overtime.
Differential Revision: https://phab.mercurial-scm.org/D10509
Pierre-Yves David <pierre-yves.david@octobus.net> [Sat, 01 May 2021 14:47:33 +0200] rev 47035
revlog: fix some comment style
They displease check-code.
Differential Revision: https://phab.mercurial-scm.org/D10542
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 08 Apr 2021 00:01:11 +0200] rev 47034
revlog: add a `entry_binary` method on index
The revlog index is already responsible for unpacking the binary entry, it would be
simpler to make it responsible for packing them. In practice the C version of
the index is already doing this internally.
We introduce a "entry_binary" method that return the binary version of an
existing revision. The method currently need to also take the revlog header to
deal with the "first revision" special case. We will introduce further refactor
in a later changeset to split that logic out.
Differential Revision: https://phab.mercurial-scm.org/D10508
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 15 Apr 2021 12:08:34 +0200] rev 47033
template: make an explicit closure for formatting entry in peerurls
This is about to be become significantly more complicated as `ui.path[x]` will
become a list.
Differential Revision: https://phab.mercurial-scm.org/D10443
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 15 Apr 2021 11:50:08 +0200] rev 47032
template: use `list_paths` in `peerurls`
Using common code will make it simpler to update the logic behind the path
definition and storage.
Differential Revision: https://phab.mercurial-scm.org/D10442
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 15 Apr 2021 11:48:29 +0200] rev 47031
paths: use `list_paths` in `hg paths`
Using common code will make it simpler to update the logic behind the path
definition and storage.
Differential Revision: https://phab.mercurial-scm.org/D10441
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 15 Apr 2021 11:46:31 +0200] rev 47030
urlutil: introduce a new `list_paths` function
This function will be useful for command and template that wants to display path
related information.
Differential Revision: https://phab.mercurial-scm.org/D10440
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 15 Apr 2021 10:05:51 +0200] rev 47029
urlutil: deprecate `getpath`
There as no remaining user of that function.
Differential Revision: https://phab.mercurial-scm.org/D10439
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 15 Apr 2021 10:01:44 +0200] rev 47028
urlutil: inline the relevant part of `getpath` in `get_push_paths`
The part that `get_push_paths` needs is quite simple, inclining will help us
to deprecated `getpath`.
Differential Revision: https://phab.mercurial-scm.org/D10438
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 15 Apr 2021 09:50:56 +0200] rev 47027
url_util: introduce a `try_path` function
That function… try a build a path, returning None on failure. This helps us to simplify various part of the existing code.
Differential Revision: https://phab.mercurial-scm.org/D10437
Martin von Zweigbergk <martinvonz@google.com> [Tue, 20 Apr 2021 11:22:35 -0700] rev 47026
narrow: add more status messages when narrowing
Each of the steps I added status messages for in this patch frequently
take minutes or tens of minutes for our internal users.
It would be nice to also have a progress bar but that will have to
come later.
Differential Revision: https://phab.mercurial-scm.org/D10503
Martin von Zweigbergk <martinvonz@google.com> [Tue, 20 Apr 2021 10:24:03 -0700] rev 47025
narrow: add progress-reporting when looking for local changes in `hg tracked`
Looking for local changes (changes not on the given remote) can take a
long time, so we should have progress-reporting for it.
Differential Revision: https://phab.mercurial-scm.org/D10501
Kyle Lippincott <spectral@google.com> [Fri, 16 Apr 2021 16:21:26 -0700] rev 47024
chg: pass --no-profile to disable profiling when starting hg serve
If profiling is enabled via global/user config (as far as I can tell, this
doesn't affect use of the --profile flag, but it probably does affect --config
profiling.enabled=1), then the profiling data can be *cumulative* for the
lifetime of the chg process.
This leads to some "interesting" results where hg claims the walltime is
something like 200s on a command that took only a second or two to run. Worse,
however, is that with at least some profilers (such as the default "stat"
profiler), this can cause a large slowdown while generating the profiler output.
Differential Revision: https://phab.mercurial-scm.org/D10470
Kyle Lippincott <spectral@google.com> [Fri, 16 Apr 2021 15:31:05 -0700] rev 47023
profiling: add --no-profile to disable profiling enabled via config
Differential Revision: https://phab.mercurial-scm.org/D10469
Kyle Lippincott <spectral@google.com> [Fri, 16 Apr 2021 18:56:26 -0700] rev 47022
tests: fix test-chg to ignore a warning about being unable to set locale
This is apparently coming from bash when bash is providing the sh that we're
using to execute the .sh file generated by run-tests for this test.
Bash on my machine:
```
$ sh --version
GNU bash, version 5.1.4(1)-release (x86_64-pc-linux-gnu)
Copyright (C) 2020 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software; you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
$ sh
sh-5.1$ LC_CTYPE=unsupported_value echo hi
sh: warning: setlocale: LC_CTYPE: cannot change locale (unsupported_value): No such file or directory
hi
```
Differential Revision: https://phab.mercurial-scm.org/D10468
Martin von Zweigbergk <martinvonz@google.com> [Tue, 27 Apr 2021 14:36:52 -0700] rev 47021
branch: delete obsolete message about changing branch of obsolete commit
We now rely on `rewriteutil.precheck()` to check for divergence, so we
don't need the extra check in `cmdutil.changebranch()`. The former
check is a little less strict in that it allows you to rewrite a
commit without non-obsolete successors.
Differential Revision: https://phab.mercurial-scm.org/D10519
Martin von Zweigbergk <martinvonz@google.com> [Tue, 23 Feb 2021 10:28:42 -0800] rev 47020
rewriteutil: check for divergence
This code is adapted from the code in the evolve extension. It seems
to be equivalent as far as the evolve extension's test suite can tell
(the only impact when making their `precheck()` delegate to our
version is that error messages are less detailed).
I had to change the error message to work with "change branch of"
being inserted as the action.
Differential Revision: https://phab.mercurial-scm.org/D10518
Martin von Zweigbergk <martinvonz@google.com> [Wed, 28 Apr 2021 08:48:10 -0700] rev 47019
rewriteutil: adapt "cannot %s while merging" to work with "change branch of"
`rewriteutil.precheck()` creates error messages by inserting a given
verb into a sentence. The `hg branch -r` command passes in "change
branch of" as the verb. That doesn't work well with "cannot %s while
merging" (making it "cannot change branch of while merging"). Let's
insert a "changeset" there to make it work better.
Building sentences like this seems obviously bad for i18n, but fixing
that is out of scope for this series, IMO.
Differential Revision: https://phab.mercurial-scm.org/D10530
Martin von Zweigbergk <martinvonz@google.com> [Thu, 11 Feb 2021 15:11:10 -0800] rev 47018
rewriteutil: point to help about instability when rewriting creates orphan
This replicates the message from the evolve extension.
Differential Revision: https://phab.mercurial-scm.org/D10517
Martin von Zweigbergk <martinvonz@google.com> [Tue, 27 Apr 2021 14:59:45 -0700] rev 47017
help: add topic about evolution, based on text from evolve extension
I've taken the text produced by `hg help evolution` when the evolve
extension is enabled and made that available by the same command with
just hg core. Changes I've made:
* Added "(EXPERIMENTAL)" to the title. (That doesn't hide the topic
from `hg help`, though.)
* Replaced old-style `experimental.evolution=<names>` config by
new-style `experimental.evolution.<name>=true`.
* Replaces a "obsolete markers" by "obsolescence markers".
* Removed most content from "Current feature status".
When the evolve extension is enabled, its help text takes precedence.
Differential Revision: https://phab.mercurial-scm.org/D10516
Martin von Zweigbergk <martinvonz@google.com> [Thu, 11 Feb 2021 15:02:57 -0800] rev 47016
rewriteutil: add devel warning if precheck is called with contexts
I'm trying to upstream parts from the evolve extension. This check
exists there.
Differential Revision: https://phab.mercurial-scm.org/D10515
Martin von Zweigbergk <martinvonz@google.com> [Tue, 27 Apr 2021 11:02:41 -0700] rev 47015
rewriteutil: replace "null changeset" by "the null revision" in error message
The evolve extension uses "the null revision" and we seem to use that
term much more frequently in core too.
Differential Revision: https://phab.mercurial-scm.org/D10514
Martin von Zweigbergk <martinvonz@google.com> [Tue, 20 Apr 2021 08:28:11 -0700] rev 47014
relnotes: copy contents of "next" to "5.8" and clear "next"
The same procedure as every year^Wcycle.
Unlike previous cycles, I haven't actually recorded the copy because
we don't want to propagate further changes in "next" (from the stable
branch) to "5.8".
Differential Revision: https://phab.mercurial-scm.org/D10500
Kyle Lippincott <spectral@google.com> [Tue, 20 Apr 2021 13:01:47 -0700] rev 47013
dirstateguard: use mktemp-like functionality to generate the backup filenames
Previously these were generated with names like:
`dirstate.backup.commit.<memory address of dirstateguard>`
This could cause problems if two hg commands ran at the same time that used the
same memory address, (which is apparently not uncommon if chg is involved), as
memory addresses are not unique across processes.
This issue was reported in the post-review comments on
http://phab.mercurial-scm.org/D9952.
Differential Revision: https://phab.mercurial-scm.org/D10504
Joerg Sonnenberger <joerg@bec.de> [Mon, 29 Mar 2021 01:52:06 +0200] rev 47012
node: replace nullid and friends with nodeconstants class
The introduction of 256bit hashes require changes to nullid and other
constant magic values. Start pushing them down from repository and
revlog where sensible.
Differential Revision: https://phab.mercurial-scm.org/D9465
Georges Racinet <georges.racinet@octobus.net> [Sat, 24 Apr 2021 16:30:05 +0200] rev 47011
repoview: separate concerns in _filteredrepotypes comment
The cited issue in Python bugtracker is closed, but hasn't been
fixed. We've been able to use the attached example and reproduce
it with Python 3.9.
The point where it turns from needless stress on the GC to
the an actual leak is when one factors in the fact that the GC
was before Python 3.4 unable to collect some types (see PEP 442).
Note that even with Python 2.7, the simple example of cycles
due to __mro__ are collectable. This was seen again with the
example attached on the CPython issue.
Georges Racinet <georges.racinet@octobus.net> [Fri, 23 Apr 2021 18:30:53 +0200] rev 47010
repoview: fix memory leak of filtered repo classes
The leak occurs in long-running server processes with
extensions, and is measured at 110kB per request.
Before this change, the contents of the `_filteredrepotypes`
cache are not properly garbage collected, despite it begin
a `WeakKeyDictionary`.
Extensions have a tendency to generate a new repository class
for each `localrepo` instantiation. Server processes based
on `hgwebdir_mod` will instantiate a new `localrepo` for each
HTTP request that involves a repository.
As a result, with a testing process that repeatedly opens a
repository with several extensions activated
(`topic` notably among them), we see a steady increase in
resident memory of 110kB per repository instantiation before this
change. This is also true, if we call `gc.collect()` at each
instantiation, like `hgwebdir_mod` does, or not.
The cause of the leak is that the *values* aren't weak references.
This change uses `weakref.ref` for the values, and this makes
in our measurements the resident size increase drop to 5kB per
repository instantiation, with no explicit call of `gc.collect()`
at all.
There is currently no reason to believe that this remaining leak
of 5kB is related to or even due to Mercurial core.
We've also seen evidence that `ui.ui` instances weren't properly
garbage collected before the change (with the change, they are).
This could explain why the figures are relatively high.
In theory, the collection of weak references could lead to
much more misses in the cache, so we measured the impact on
the original case that was motivation for introducing that cache
in
7e89bd0cfb86 (see also
issue5043): `hg convert` of the
mozilla-central repository. The bad news here is that there is a
major memory leak there, both with and without the present changeset.
There were no more cache misses, and we could see no
more memory leak with this change: the resident size after importing
roughly 100000 changesets was at 12.4GB before, and 12.5GB after.
The small increase is mentioned for completeness only, and we
believe that it should be ignored, at least as long as the main
leak isn't fixed. At less than 1% of the main leak, even finding out
whether it is merely noise would be wasteful.
Original context where this was spotted and first mitigated:
https://foss.heptapod.net/heptapod/heptapod/-/issues/466
The leak reduction was also obtained in Heptapod inner HTTP server,
which amounts to the same as `hgwebdir_mod` for these questions.
The measurements done with Python 3.9, similar figures seen with 3.8.
More work on our side would be needed to give measurements with 2.7,
because of testing server process does not support it.
Georges Racinet <georges.racinet@octobus.net> [Sat, 24 Apr 2021 15:46:39 +0200] rev 47009
repoview: style change in newtype() cache handling
This way of writing it does not change the logic at all,
but is more fit for the change we want to make in the
next changeset.
If anything, that's one dict lookup less in the hot path,
but that should be non measurable.
Joerg Sonnenberger <joerg@bec.de> [Thu, 22 Apr 2021 02:57:30 +0200] rev 47008
tests: don't hard-code /bin/bash
From Thomas Klausner in pkgsrc.
Differential Revision: https://phab.mercurial-scm.org/D10507
Raphaël Gomès <rgomes@octobus.net> [Wed, 21 Apr 2021 17:59:14 +0200] rev 47007
lint: fix failing code check
8d2b62d716b0 introduced code that `test-check-code.t` wasn't happy about, this
fixes it.
Differential Revision: https://phab.mercurial-scm.org/D10506
Matt Harbison <matt_harbison@yahoo.com> [Wed, 14 Apr 2021 09:49:36 -0400] rev 47006
typing: enable pytype processing on mercurial/upgrade_utils/actions.py
This was the suggested workaround for the pytype bug that caused this file to be
processed forever.
Differential Revision: https://phab.mercurial-scm.org/D10460
Matt Harbison <matt_harbison@yahoo.com> [Sun, 18 Apr 2021 00:56:09 -0400] rev 47005
incoming: use bytes for an error message
Caught by pytype.
Differential Revision: https://phab.mercurial-scm.org/D10461
Augie Fackler <augie@google.com> [Wed, 21 Apr 2021 10:58:42 -0400] rev 47004
merge: stable heads
I forgot to pull before rolling rc1, so we just have a couple of
patches that missed the rc1 train. Mea culpa.
Augie Fackler <raf@durin42.com> [Wed, 21 Apr 2021 10:49:29 -0400] rev 47003
Added signature for changeset
8d2b62d716b0
Augie Fackler <raf@durin42.com> [Wed, 21 Apr 2021 10:49:28 -0400] rev 47002
Added tag 5.8rc1 for changeset
8d2b62d716b0
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 20 Apr 2021 16:31:13 +0200] rev 47001
relnote: mention a nodemap bug fixes
The fix was in
a3720569a43f.
Differential Revision: https://phab.mercurial-scm.org/D10499
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 20 Apr 2021 16:29:05 +0200] rev 47000
relnote: document the use of persistent-nodemap by default for Rust build
Differential Revision: https://phab.mercurial-scm.org/D10498