Mitchell Plamann <mplamann@janestreet.com> [Wed, 15 Apr 2020 16:43:05 -0400] rev 44878
dirstate: force _checkexec to return a bool
posix.checkexec can return True, False, or None. The rust status
implementation expects a boolean, so make sure _checkexec returns a boolean.
Differential Revision: https://phab.mercurial-scm.org/D8432
Kyle Lippincott <spectral@google.com> [Tue, 21 Apr 2020 13:37:45 -0700] rev 44877
locking: wait for locks in `hg cp` and `hg mv`
I traced this back to revision 1822 (64df422), and there's no explanation why we
would prefer to error out instead of waiting for the locks.
Differential Revision: https://phab.mercurial-scm.org/D8464
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 14 Apr 2020 05:37:54 +0200] rev 44876
nodemap: teach `hg debugformat` about the persistent nodemap option
We have a new requirement, we should display it.
Differential Revision: https://phab.mercurial-scm.org/D8430
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 15 Apr 2020 18:58:35 +0200] rev 44875
upgrade: support the --quiet flag
The command is currently very verbose with a various bit of output being time
sensitive or randomized. The make invocation bulky and hard to match in the
test. We move various message from `ui.write` to `ui.status` in order for the
`--quiet` flag to have effect and helps the situation.
As a side benefit, we can replace the various redirection to > /dev/null with the --quiet flag.
Differential Revision: https://phab.mercurial-scm.org/D8429
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 15 Apr 2020 19:20:15 +0200] rev 44874
upgrade: clearly list optimisations
This makes the command operation clearer. And this will be necessary to have
a short version of this information with the next changesets, teaching `hg
debugupgraderepo` about `--quiet`.
Differential Revision: https://phab.mercurial-scm.org/D8428
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 14 Apr 2020 04:23:20 +0200] rev 44873
nodemap: move the mode option to storage.revlog.nodemap.mode
The option stay experimental as long as the main feature is.
Differential Revision: https://phab.mercurial-scm.org/D8422
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 14 Apr 2020 03:20:21 +0200] rev 44872
nodemap: move the option for mmap usage to storage.revlog.nodemap.mmap
The option is stay experimental as long as the main feature is.
Differential Revision: https://phab.mercurial-scm.org/D8421
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 14 Apr 2020 04:08:46 +0200] rev 44871
nodemap: move and update the commend about persistence being experimental
The comment was at the wrong place (on the developer option instead of the
activation switch). So we move it at the right location and update it.
Differential Revision: https://phab.mercurial-scm.org/D8420
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 14 Apr 2020 03:18:14 +0200] rev 44870
nodemap: move the main switch to the `format` section
The config to enable persistent nodemap is now `format.use-persistent-nodemap`.
However the option remain marked as experimental because it only improve
performance for people using the rust extensions.
Differential Revision: https://phab.mercurial-scm.org/D8419
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 14 Apr 2020 03:27:04 +0200] rev 44869
nodemap: drop the 'exp-' prefix for internal opener option
The feature is now in a descent shape and we can consider having it "less"
experimental.
We won't be able to make it "totally" non-experimental, because its benefit
rely on rust, which is totally experimental.
Differential Revision: https://phab.mercurial-scm.org/D8418
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 14 Apr 2020 03:16:23 +0200] rev 44868
nodemap: gate the feature behind a new requirement
Now that the feature is working smoothly, a question was still open, should we
gate the feature behind a new requirement or just treat it as a cache to be
warmed by those who can and ignored by other.
The advantage of using the cache approach is a transparent upgrade/downgrade
story, making the feature easier to move to. However having out of date cache
can come with a significant performance hit for process who expect an up to
date cache but found none. In this case the file needs to be stored under
`.hg/cache`.
The "requirement" approach guarantee that the persistent nodemap is up to date.
However, it comes with a less flexible activation story since an explicite
upgrade is required. In this case the file can be stored in `.hg/store`.
This wiki page is relevant to this questions:
https://www.mercurial-scm.org/wiki/ComputedIndexPlan
So which one should we take? Another element came into plan, the persistent
nodemap use the `add` method of the transaction, it is used to keep track of a
file content before a transaction in case we need to rollback it back. It turns
out that the `transaction.add` API does not support file stored anywhere than
`.hg/store`. Making it support file stored elsewhere is possible, require a
change in on disk transaction format. Updating on disk file requires…
introducing a new requirements.
As a result, we pick the second option "gating the persistent nodemap behind a
new requirements".
Differential Revision: https://phab.mercurial-scm.org/D8417
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 14 Apr 2020 03:05:54 +0200] rev 44867
nodemap: move on disk file to version 1
The current format contains the information we need, lets freeze it before the
release.
Differential Revision: https://phab.mercurial-scm.org/D8416
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 14 Apr 2020 03:01:52 +0200] rev 44866
nodemap: add a new mode value, "strict"
When "strict" is set, operation will refuse to use the slow path and abort if
they would. This is useful for testing, benchmarking and server operation.
Differential Revision: https://phab.mercurial-scm.org/D8415
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 14 Apr 2020 02:45:05 +0200] rev 44865
nodemap: add a new mode option, with an optional "warn" value
When "warn" is set, user will get notified when the slow code, used for
compatibility is used. This can help people to detect situation were using that
feature will give them a slowdown instead of a speedup.
Differential Revision: https://phab.mercurial-scm.org/D8414
Pierre-Yves David <pierre-yves.david@octobus.net> [Sun, 05 Apr 2020 18:32:46 +0200] rev 44864
nodemap: also warm manifest nodemap with other caches
The `hg debugupdatecache` command now also warm the persistent nodemap for the
manifest (when applicable).
Differential Revision: https://phab.mercurial-scm.org/D8411
Pierre-Yves David <pierre-yves.david@octobus.net> [Sun, 05 Apr 2020 13:12:05 +0200] rev 44863
nodemap: also use persistent nodemap for manifest
The manifest as a different usage pattern than the changelog. First, while the
lookup in changelog are not garanteed to match, the lookup in the manifest
nodemap come from changelog and will exist in the manifest. In addition, looking
up a manifest almost always result in unpacking a manifest an operation that
rarely come cheap.
Nevertheless, using a persistent nodemap provide a significant gain for some
operations.
For our measurementw, we use `hg cat --rev REV FILE` on the our reference
mozilla-try. On this repository the persistent nodemap cache is about 29 MB in
side for a total store side of 11,988 MB
File with large history (file: b2g/config/gaia.json, revision: 195a1146daa0)
no optimisation: 0.358s
using mmap for index: 0.297s (-0.061s)
persistent nodemap for changelog only: 0.275s (-0.024s)
persistent nodemap for manifest too: 0.258s (-0.017s)
File with small history (file: .hgignore, revision: 195a1146daa0)
no optimisation: 0.377s
using mmap for index: 0.296s (-0.061s)
persistent nodemap for changelog only: 0.274s (-0.022s)
persistent nodemap for manifest too: 0.257s (-0.017s)
Same file but using a revision (8ba995b74e18) with a smaller manifest (3944829
bytes vs 10 bytes)
no optimisation: 0.192s (-0.185s)
using mmap for index: 0.131s (-0.061s)
persistent nodemap for changelog only: 0.106s (-0.025s)
persistent nodemap for manifest too: 0.087s (-0.019s)
Differential Revision: https://phab.mercurial-scm.org/D8410
Pierre-Yves David <pierre-yves.david@octobus.net> [Sun, 05 Apr 2020 13:49:27 +0200] rev 44862
nodemap: create files in the repository used in the test
We need a manifest with more content to test persistent nodemap for manifest.
This change the repository content and affect all the hashes.
Differential Revision: https://phab.mercurial-scm.org/D8409
Raphaël Gomès <rgomes@octobus.net> [Thu, 07 May 2020 10:10:13 +0200] rev 44861
rust-matchers: add timing tracing to regex compilation
This might be useful to diagnose later performance issues or just to show
the difference between engines.
Differential Revision: https://phab.mercurial-scm.org/D8498
Augie Fackler <augie@google.com> [Mon, 04 May 2020 10:06:53 -0400] rev 44860
merge with stable
Martin von Zweigbergk <martinvonz@google.com> [Fri, 01 May 2020 08:07:25 -0700] rev 44859
merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 05 Mar 2020 17:55:05 +0100] rev 44858
copies: fix the changeset based algorithm regarding merge
In 99ebde4fec99, we changed the list of files stored into the `files` field.
This lead to the changeset centric copy algorithm to break in various merge
situation involving merge. Older information could reach the merge through
`p1`, and while information from `p2` was strictly fresher, it would get
overwritten anyway.
We update the situation with more details about which revision introduces rename
information. This help use making the right decision in case of merge.
We are now running a more comprehensive suite of test with include this kind of
situation. The behavior differ slightly from the filelog based in a couple of
instance. There is mostly two distinct cases:
1) there are conflicting rename information in a merge (different rename history
on each side). In this case the filelog based implementation arbitrarily pick a
side based on the file-revision-number. So it depends on a local factor. The
changeset centric algorithm will use a deterministic approach, by picking the
information coming from the first parent of the merge. This is stable across
different clone.
2) rename information related to file that exist in both source and destination.
The filelog based implementation do not even try to detect these, however the
changeset centric one get them for "free" (it is simpler to detect them than
not).
The new implementation focus on correctness. Performance improvement will come
later.
Differential Revision: https://phab.mercurial-scm.org/D8244
Augie Fackler <augie@google.com> [Fri, 24 Apr 2020 15:06:42 -0400] rev 44857
merge with stable
Yuya Nishihara <yuya@tcha.org> [Sat, 11 Apr 2020 17:43:29 +0900] rev 44856
rust-chg: clean up excessive indents
Differential Revision: https://phab.mercurial-scm.org/D8450
Yuya Nishihara <yuya@tcha.org> [Sat, 11 Apr 2020 02:51:03 +0900] rev 44855
rust-chg: do not terminate tokio runtime until pager exits
We no longer need to spawn a task just to keep the pager handle. The pager
handle can be held by ChgUiHandler since the handler itself is not consumed
and recreated across async calls.
Differential Revision: https://phab.mercurial-scm.org/D8449
Yuya Nishihara <yuya@tcha.org> [Sat, 11 Apr 2020 02:21:06 +0900] rev 44854
rust-chg: modernize entry function
Finally the entire build passes.
There's a bug that run() no longer waits for the spawned pager, which will
be fixed by the next patch.
Differential Revision: https://phab.mercurial-scm.org/D8448
Yuya Nishihara <yuya@tcha.org> [Sat, 11 Apr 2020 00:47:32 +0900] rev 44853
rust-chg: reimplement locator by using async/await and tokio-0.2
connect_spawned() is rewritten from scratch by using std::process. Before,
it would select completion of either connection or server process. New code
could be implemented as such, but it's much simpler to occasionally run
try_wait() to detect server death.
Differential Revision: https://phab.mercurial-scm.org/D8447
Yuya Nishihara <yuya@tcha.org> [Fri, 10 Apr 2020 23:26:36 +0900] rev 44852
rust-chg: reimplement ChgClientExt as ChgClient wrapper
ChgClient is no longer an extension trait because:
a. Client object is not consumed and recreated in future-0.3 world, which
unblocks writing a simple wrapper struct.
b. async fn isn't allowed in trait.
Overall, the API should become simpler.
Differential Revision: https://phab.mercurial-scm.org/D8446
Yuya Nishihara <yuya@tcha.org> [Fri, 10 Apr 2020 22:44:51 +0900] rev 44851
rust-chg: reimplement run_command operation as async function
The crafted state machine is no longer needed thanks to async/await.
The state machine is basically rewritten as follows:
- Ready(..) -> return ..
- PollAgain(..) -> run .. and await
- Err(..) -> return Err(..)
Differential Revision: https://phab.mercurial-scm.org/D8445
Yuya Nishihara <yuya@tcha.org> [Fri, 10 Apr 2020 22:23:10 +0900] rev 44850
rust-chg: reimplement uihandler by using async-trait and tokio-0.2
We no longer have to consume self and arguments.
Differential Revision: https://phab.mercurial-scm.org/D8444
Yuya Nishihara <yuya@tcha.org> [Fri, 10 Apr 2020 23:08:57 +0900] rev 44849
rust-chg: have attach_io() simply take reference of AsRawFd object
We no longer have to deal with the restriction of the Future type.
Before, these file objects couldn't be references and that's the only
reason why we had to make stderr an Option<T>.
This fixes future type deduction issue of stderr = None, where rustc would
complain that T of Option<T> couldn't be deduced.
Differential Revision: https://phab.mercurial-scm.org/D8443
Yuya Nishihara <yuya@tcha.org> [Fri, 10 Apr 2020 22:07:11 +0900] rev 44848
rust-chg: reimplement attach_io operation as async function
In short, MessageLoop<Connection> was redesigned as Protocol<Connection>,
and the protocol methods no longer consume self.
API changes are briefly documented in the following page:
https://docs.rs/tokio-hglib/0.3.0/tokio_hglib/struct.Protocol.html
Differential Revision: https://phab.mercurial-scm.org/D8442
Yuya Nishihara <yuya@tcha.org> [Fri, 10 Apr 2020 21:54:03 +0900] rev 44847
rust-chg: upgrade to futures-0.3 based libraries
And do some trivial fixes:
- BytesMut::put_u32_be() -> put_u32()
- tokio_process -> tokio::process, CommandExt -> Command,
spawn_async() -> spawn(), stdin() -> stdin
- tokio_timer::sleep() -> tokio::time::delay_for()
Differential Revision: https://phab.mercurial-scm.org/D8441