Martin von Zweigbergk <martinvonz@google.com> [Tue, 22 Jun 2021 08:24:27 -0700] rev 47458
benchmarks: restore `output` variable lost in D10884
Noticed by pyflakes (which I didn't have installed for Python 3.9 when
I sent D10884).
Differential Revision: https://phab.mercurial-scm.org/D10894
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 07 Jun 2021 11:59:27 +0200] rev 47457
censor: implement censoring for revlogv2
It is a bit verbose and rough, but it works. Most of that logic can be common
for `stripping`, so we can expect more refactoring of that code to accommodate
both needs. However I wanted to keep this changesets "simple enough" and before
moving forward.
We also need to properly delete the older index/data/sidedata file, but this has
implication for streaming clone and transaction, so this will come later.
Differential Revision: https://phab.mercurial-scm.org/D10869
Pierre-Yves David <pierre-yves.david@octobus.net> [Sun, 20 Jun 2021 23:05:58 +0200] rev 47456
deltas: at a `target_rev` parameter to finddeltainfo
Otherwise, recomputing a delta for a revision might result in a delta against a
later revision or a full snapshot thinking we are appending a new revision.
We will make use of this during censoring (and later, stripping).
Differential Revision: https://phab.mercurial-scm.org/D10887
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 09 Jun 2021 19:47:29 +0200] rev 47455
revlog: list older-but-still-around file in `files`
These older files will eventually be removed, but we have to take them in account until they are. So we update `files` to keep fncache happy.
Differential Revision: https://phab.mercurial-scm.org/D10868
Pierre-Yves David <pierre-yves.david@octobus.net> [Sat, 29 May 2021 00:11:12 +0200] rev 47454
revlog: add docket method to request new content files
If we want to write content into new files, we need to be able to ask for them.
Differential Revision: https://phab.mercurial-scm.org/D10867
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 20 May 2021 21:47:09 +0200] rev 47453
revlog: add a way to keep track of older uids in the docket
When the revlog content is rewritten, we will use new files, to avoid truncating
the previous ones. We need some way to keep track of the older files before we
clean them up.
Differential Revision: https://phab.mercurial-scm.org/D10866
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 21 Jun 2021 15:00:53 +0200] rev 47452
revlog: factor the logic to determine the delta compression out
This make the logic clearly isolated and documented and it will help use to
reuse it during censors/strip.
Differential Revision: https://phab.mercurial-scm.org/D10889
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 10 Jun 2021 00:16:54 +0200] rev 47451
nodemap: fix some comment formatting
Black seems unhappy about it.
Differential Revision: https://phab.mercurial-scm.org/D10865
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 09 Jun 2021 12:19:42 +0200] rev 47450
revlogv2: fix `hg verify` with revlog v2
We added a bunch of new files but we never teached some internal about it. This
is now fixed.
Differential Revision: https://phab.mercurial-scm.org/D10858
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 08 Jun 2021 02:34:59 +0200] rev 47449
revlogv2: also test that local clone works
This is now the case so lets add a test.
Differential Revision: https://phab.mercurial-scm.org/D10857
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 08 Jun 2021 03:56:33 +0200] rev 47448
clone: also report the bookmark file as copied
This is a small UI adjustement, but this is easy enough to do.
Differential Revision: https://phab.mercurial-scm.org/D10856
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 08 Jun 2021 02:06:45 +0200] rev 47447
clone: reuse the stream clone logic for local clone
Streaming clone and local (non `--pull`) clone do mostly the same thing,
however they were using different logic to do so. This means the logic
frequently went out of sync and that new case had to be dealt with twice.
This is fragile and anoying. So we replace this with a re-use of the logic we
use for streaming clone.
I can see various test changes:
- a more precise progress output,
- armless fncache loading during clone,
- fncache is no longer hardlinked (since we write it by hand).
I am not reinstalling the `reposimplestore` specific output, as far as I
understand this variant have been broken for years.
Differential Revision: https://phab.mercurial-scm.org/D10855
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 09 Jun 2021 15:33:58 +0200] rev 47446
copyfiles: add a way to relax the file system checking for hardlink
This is critical for transaction file, less for hardlink/copy clone as we are
about to do. Since `pure` build does not have a `getfstype` implementation this
would disable hardlink clone for all pure build. So we add a parameter to
control that extra check.
Differential Revision: https://phab.mercurial-scm.org/D10854
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 08 Jun 2021 02:31:17 +0200] rev 47445
copyfile: add a option callback for failed hardlinking
Local clone, adjust its UI depending on the success of using hardlinking, so we
add a small callback making it possible for `copyfile` to signal if the
requested hardlinking failed.
Differential Revision: https://phab.mercurial-scm.org/D10853
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 08 Jun 2021 02:06:02 +0200] rev 47444
streamingclone: extract the scanning part from the generation part
We will reuse the scanning part for local clone, so we need it in a dedicated
function.
Differential Revision: https://phab.mercurial-scm.org/D10852
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 08 Jun 2021 02:05:05 +0200] rev 47443
vfs: add a `register_file` method on the vfs class
This is used by the fncache vfs to register new file. Until now, `fncache` have
been doing this "automatically" by monitoring write pattern. However this is
fragile and when we copy files in place by other means, we need something more
robuts. So we add an explicit method to do so.
Differential Revision: https://phab.mercurial-scm.org/D10851
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 09 Jun 2021 01:10:34 +0200] rev 47442
clone: use "official" API to create local clone destination
This make sure we have a properly created, fully functional repository early.
This will be useful to simply the hardlink/copy phases of the local clone to
make it share more of its logic with the similar "stream" cloning.
This has a minor impact of the test and the resulting repository has is better initialized (eg: the `wcache` directory is pre-created.)
Differential Revision: https://phab.mercurial-scm.org/D10850
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 09 Jun 2021 01:10:26 +0200] rev 47441
localrepo: introduce a clone_requirements function
This function take a source repository and return a relevant set of
requirements that should be used by a copy clone.
This will help make the creation of the destination repository during copy
clone simpler.
Differential Revision: https://phab.mercurial-scm.org/D10849
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 07 Jun 2021 20:40:43 +0200] rev 47440
createrepository: allow to directly pass the target requirements
This is useful when doing a local clone that copies store contents, it will
requires the destination to use the very same store requirements so directly
providing them will be simpler and safer
Differential Revision: https://phab.mercurial-scm.org/D10848
Martin von Zweigbergk <martinvonz@google.com> [Fri, 18 Jun 2021 16:03:42 -0700] rev 47439
narrowbundle: use new context manager for silencing the ui
Same reasoning as the previous change. This affects a few tests
because of the hack from
d7304434390f (changegroup: move message about
added changes to transaction summary, 2019-09-08).
Differential Revision: https://phab.mercurial-scm.org/D10886
Martin von Zweigbergk <martinvonz@google.com> [Fri, 18 Jun 2021 16:00:58 -0700] rev 47438
debugbackupbundle: use new context manager for silencing the ui
A difference between setting `ui.quiet` and using `ui.silent()` is
that the latter also silences `ui.write()` calls. That's practically
always what one wants, including here, I think.
Differential Revision: https://phab.mercurial-scm.org/D10885
Martin von Zweigbergk <martinvonz@google.com> [Fri, 18 Jun 2021 15:48:51 -0700] rev 47437
ui: add a context manager for silencing the ui (pushbuffer+popbuffer)
We often silence the ui by calling `ui.pushbuffer()` followed (a later
in the code) by `ui.popbuffer()`. These places can be identified by
the fact that they ignore the output returned from
`ui.popbuffer()`. Let's create a context manager for these cases, to
avoid repetition, and to avoid accidentally leaving the ui silent on
exceptions. I deliberately called the new function `silent()` instead
of `buffered()`, because it's just an implementation detail that it
uses `pushbuffer()` and `popbuffer()`. We could later optimize it to
not buffer the output.
Differential Revision: https://phab.mercurial-scm.org/D10884
Martin von Zweigbergk <martinvonz@google.com> [Fri, 18 Jun 2021 14:17:43 -0700] rev 47436
shelve: don't swallow errors that happen when reverting the working copy
Same reasoning as the parent commit. The code in question here comes
from
fb8065de47b0 (unshelve: silence internal revert, 2014-08-15),
which says that it was to prepare for some upcoming revert changes. I
suppose those changes have happened by now but we fixed something
about it, because I don't see any unwanted changes in tests (no
changes at all in tests, in fact).
Differential Revision: https://phab.mercurial-scm.org/D10883
Martin von Zweigbergk <martinvonz@google.com> [Fri, 18 Jun 2021 13:55:45 -0700] rev 47435
histedit: don't swallow errors that happen when updating the working copy
Commit
34165875fa5d (histedit: limit updated and merging output to
important updates, 2016-02-03) added a call to
`ui.pushbuffer(labeled=True, errors=True)`. I don't know if there was
a reason for the `errors=True` (nor the `labeled=True` bit, which I
removed in the previous commit). It seems bad to just discard the
errors, so let's not do that. By not doing that, we can also remove a
workaround for the lack of errors from `test-removeemptydirs.t`.
Differential Revision: https://phab.mercurial-scm.org/D10882
Martin von Zweigbergk <martinvonz@google.com> [Fri, 18 Jun 2021 13:44:05 -0700] rev 47434
histedit: don't record labels for output that we will discard
Differential Revision: https://phab.mercurial-scm.org/D10881
Martin von Zweigbergk <martinvonz@google.com> [Thu, 10 Jun 2021 15:56:55 -0700] rev 47433
dog-raft: work with str-keyed opts throughout
Differential Revision: https://phab.mercurial-scm.org/D10864
Martin von Zweigbergk <martinvonz@google.com> [Thu, 10 Jun 2021 15:45:22 -0700] rev 47432
cmdutil: make resolvecommitoptions() work on str-keyed opts
As with `checknotesize()`, I also changed to snake_case while at it,
to help extensions a little.
Differential Revision: https://phab.mercurial-scm.org/D10863
Martin von Zweigbergk <martinvonz@google.com> [Thu, 10 Jun 2021 14:55:10 -0700] rev 47431
cmdutil: make amend() take str-keyed opts
I couldn't think of a reasonable change to the signature here, so
unfortunately it keeps the existing one.
Differential Revision: https://phab.mercurial-scm.org/D10862
Martin von Zweigbergk <martinvonz@google.com> [Thu, 10 Jun 2021 14:47:14 -0700] rev 47430
commit: keep opts dict str-keyed a bit longer
Differential Revision: https://phab.mercurial-scm.org/D10861
Martin von Zweigbergk <martinvonz@google.com> [Thu, 10 Jun 2021 14:36:26 -0700] rev 47429
dog-raft: keep opts dict str-keyed a bit longer
Differential Revision: https://phab.mercurial-scm.org/D10860
Martin von Zweigbergk <martinvonz@google.com> [Thu, 10 Jun 2021 14:25:16 -0700] rev 47428
cmdutil: make checknotesize() work on str-keyed opts
Since this patch changes the behavior of the function, I consider it a
bonus if the signature also changes so that extensions can detect
which version they're dealing with (for the record, I don't have any
such extensions). Therefore, I also dropped an unused `ui` argument
and renamed the function to use snake_case.
Differential Revision: https://phab.mercurial-scm.org/D10859
Martin von Zweigbergk <martinvonz@google.com> [Thu, 10 Jun 2021 10:53:43 -0700] rev 47427
amend: work mostly with str-type **opts for simplicity
Differential Revision: https://phab.mercurial-scm.org/D10880
Simon Sapin <simon.sapin@octobus.net> [Tue, 15 Jun 2021 12:34:14 +0200] rev 47426
revlog: use file read caching for sidedata
The previous changeset extracted this caching logic from the revlog class into
a new class. Adding a new instance of that class allows using the same logic
for side data.
Differential Revision: https://phab.mercurial-scm.org/D10879
Simon Sapin <simon.sapin@octobus.net> [Tue, 08 Jun 2021 19:55:00 +0200] rev 47425
revlog: Extract low-level random-access file read caching logic
The `revlog` class does many things, among which fulfilling requests for
arbitrary byte slices from the revlog "data file" by reading a larger chunk
and caching it in memory, in order to reduce the number of system calls.
This extracts that logic into a new class, so that it may later also be used
for the side-data file (with another instance of that class).
The copyright notice of the new file does not include a date or author name
since such information tend not to be kept up-to-date:
https://www.linuxfoundation.org/en/blog/copyright-notices-in-open-source-software-projects/
Differential Revision: https://phab.mercurial-scm.org/D10878
Matt Harbison <matt_harbison@yahoo.com> [Fri, 11 Jun 2021 23:39:31 -0400] rev 47424
typing: disable warnings for a couple of Windows specific attributes
Differential Revision: https://phab.mercurial-scm.org/D10870
Matt Harbison <matt_harbison@yahoo.com> [Sat, 12 Jun 2021 14:50:33 -0400] rev 47423
typing: suppress a false error in mercurial/revlogutils/docket.py on py2
`ord()` wants bytes or str on py3, so I'm guessing it got confused by passing a
single byte instead of a one byte string. But this seems to work on 2.7.18
anyway.
Differential Revision: https://phab.mercurial-scm.org/D10876
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 09 Jun 2021 00:59:44 +0200] rev 47422
transaction: simplify `undo.backupfiles` file creation with a variable
This make the code shorter and clearer.
(This is an opportunity improvement while looking at something next to that.)
Differential Revision: https://phab.mercurial-scm.org/D10846
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 09 Jun 2021 00:59:04 +0200] rev 47421
transaction: narrow the error filtering when failing to rename undo file
Having inconsistent "undo" files can be quite serious so we narrow the error
filtering to the intent that the comment explain.
(This is an opportunity improvement while looking at something next to that.)
Differential Revision: https://phab.mercurial-scm.org/D10845
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 09 Jun 2021 01:13:09 +0200] rev 47420
transaction: clarify a conditionnal about version check
Let us move the short branch early.
Differential Revision: https://phab.mercurial-scm.org/D10844
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 09 Jun 2021 01:14:18 +0200] rev 47419
transaction: extract message about different version in a constants
This make the code a bit simpler and clearer.
(This is an opportunity improvement while looking at something next to that.)
Differential Revision: https://phab.mercurial-scm.org/D10843
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 09 Jun 2021 01:12:03 +0200] rev 47418
transaction: explain why some recovery failed
Right now we issue a message about failing to recover some file, but not why.
It seems useful to add some information about that.
(This is an opportunity improvement while looking at something next to that.)
Differential Revision: https://phab.mercurial-scm.org/D10842
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 07 Jun 2021 21:09:31 +0200] rev 47417
copyfiles: deal with existing file when hardlinking
If the hardlinking fails, we fallback to `shutil.copy`, but do not consider
future hardlinking doomed.
This is a slight improvement from the current situation, we still avoid
hardliking in a case we might be able to do it. However this does not have an
impact of the rest of the operation.
(This is an opportunity improvement while looking at something next to that.)
Differential Revision: https://phab.mercurial-scm.org/D10841
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 08 Jun 2021 03:40:36 +0200] rev 47416
clone: cleanup the "cleanup dir" logic used during local clone
With the previous code, there was a small windows were we could delete too much
data. This should no longer be the case.
(This is an opportunity improvement while looking at something next to that.)
Differential Revision: https://phab.mercurial-scm.org/D10840
Pierre-Yves David <pierre-yves.david@octobus.net> [Fri, 09 Apr 2021 17:04:39 +0200] rev 47415
config: add an experimental option to list all known config
That option is not ready for prime-time, hence the `exp-` prefix. However, this
is a good base to start going toward completion. This is also quite useful for
developer on its own for now.
Differential Revision: https://phab.mercurial-scm.org/D10356
Matt Harbison <matt_harbison@yahoo.com> [Sat, 12 Jun 2021 00:51:34 -0400] rev 47414
revlog: byteify a few error strings
Differential Revision: https://phab.mercurial-scm.org/D10875
Matt Harbison <matt_harbison@yahoo.com> [Sat, 12 Jun 2021 00:50:03 -0400] rev 47413
revlog: fix a typo closing the wrong file
Caught by pytype as possibly being None, but the file is closed in the previous
conditional.
Differential Revision: https://phab.mercurial-scm.org/D10874
Matt Harbison <matt_harbison@yahoo.com> [Fri, 11 Jun 2021 23:57:14 -0400] rev 47412
py3: byteify the version string passed to the deprecation warning method
The other callers were already correct.
Differential Revision: https://phab.mercurial-scm.org/D10873
Matt Harbison <matt_harbison@yahoo.com> [Fri, 11 Jun 2021 23:51:27 -0400] rev 47411
urlutil: byteify several localized messages
Flagged by pytype.
Differential Revision: https://phab.mercurial-scm.org/D10872
Matt Harbison <matt_harbison@yahoo.com> [Fri, 11 Jun 2021 23:50:29 -0400] rev 47410
urlutil: use the deprecation warning function on the `ui` object
PyCharm flagged this as not existing, and I suspect a copy/paste error.
Differential Revision: https://phab.mercurial-scm.org/D10871
Simon Sapin <simon.sapin@octobus.net> [Wed, 02 Jun 2021 11:25:18 +0200] rev 47409
dirstate-v2: Store a hash of ignore patterns (.hgignore)
Later, this help extend `read_dir` caching to directories that contain ignored
files (but no unknown files). Such cache must be invalidated when ignore patterns
change since a formerly-ignored file might become unknown.
This helps the default configuration of `hg status` where unknown files must
be listed, but ignored files are not.
Differential Revision: https://phab.mercurial-scm.org/D10836
Pulkit Goyal <7895pulkit@gmail.com> [Mon, 07 Jun 2021 17:29:32 +0530] rev 47408
rhg: propogate error coming from HgError::Abort to CommandError
Differential Revision: https://phab.mercurial-scm.org/D10839
Pulkit Goyal <7895pulkit@gmail.com> [Mon, 07 Jun 2021 17:27:49 +0530] rev 47407
rhg: add exit code to HgError::Abort()
My previous attempts to have rhg end with correct exit code was more of bug
hunting. I found cases which were failing and fixed them. But as one might
expect, more tests started failing.
Let's add exit code `HgError::Abort()` and make it users explicitly tell what
exit code they want.
Differential Revision: https://phab.mercurial-scm.org/D10838
Pulkit Goyal <7895pulkit@gmail.com> [Mon, 07 Jun 2021 17:19:46 +0530] rev 47406
rhg: split non_repo_config and `--config` loading in different functions
This will help us in better handling of error caused when trying to
load `--config` values.
Differential Revision: https://phab.mercurial-scm.org/D10837
Pulkit Goyal <7895pulkit@gmail.com> [Mon, 24 May 2021 16:27:54 +0530] rev 47405
rhg: look for repository in ancestors also instead of cwd only
Last patch introduced config reading at startup to parse value of `--repository`
flag. However, that patch only tried to check for current repository at current
working directory and not it's ancestors. This patch fixes that.
Differential Revision: https://phab.mercurial-scm.org/D10767
Pulkit Goyal <7895pulkit@gmail.com> [Sun, 11 Apr 2021 00:50:10 +0530] rev 47404
rhg: read [paths] for `--repository` value
hg parses `-R` and `--repository` CLI arguments "early" in order to know which
local repository to load config from. (Config can then affect whether or how to
fall back.)
The value of of those arguments can be not only a filesystem path, but also an
alias configured in the `[paths]` section. This part was missing in rhg and
this patch implements that.
The current patch still lacks functionality to read config of current repository
if we are not at root of repo. That will be fixed in upcoming patches.
A new crate `home` is added to get path of home directory.
Differential Revision: https://phab.mercurial-scm.org/D10296
Augie Fackler <augie@google.com> [Mon, 07 Jun 2021 17:10:35 -0400] rev 47403
merge: with stable
Pierre-Yves David <pierre-yves.david@octobus.net> [Sun, 30 May 2021 18:09:29 +0200] rev 47402
revlog: allow to pass an existing docket to `_loadindex()`
This will be when switching undelying files, for examples during censors and strip operation with rvlog-v2.
Differential Revision: https://phab.mercurial-scm.org/D10800
Pierre-Yves David <pierre-yves.david@octobus.net> [Sun, 30 May 2021 18:10:15 +0200] rev 47401
revlog: add a ways to blacklist some revision when searching for a delta
This will be useful to recompute appropriate deltas one the fly during
censor/strip operation with revlog-v2.
Differential Revision: https://phab.mercurial-scm.org/D10799
Pierre-Yves David <pierre-yves.david@octobus.net> [Sun, 30 May 2021 18:08:52 +0200] rev 47400
util: add `nb_bytes` argument to `copyfile` to partially copy a file
When set, this allow to copy only the first `nb_bytes` of a file. This will be
useful for censor/strip operation with revlogv2.
Differential Revision: https://phab.mercurial-scm.org/D10798
Pierre-Yves David <pierre-yves.david@octobus.net> [Sun, 30 May 2021 16:20:36 +0200] rev 47399
revlog: move `revisioninfo` in `revlogutils`
We will need it in other utility module. So lets extract it from `revlog.py`,
the module is too large already anyway.
Differential Revision: https://phab.mercurial-scm.org/D10797