contrib: have byteify-strings explode if run in Python 2 Differential Revision: https://phab.mercurial-scm.org/D6341

repair: reword comment about bookmarks logic Again, this will help auto-formatting shortly. Differential Revision: https://phab.mercurial-scm.org/D6340

monotone: fix a bogus _() wrapper that was caught when formatting code There was a spurious space after `debug`, which hid the _() inside ui.debug() from check-code. Sigh. While here, wrap things more concisely. Differential Revision: https://phab.mercurial-scm.org/D6338

commit: add ability to print file status after each successful invocation When commands.commit.post-status is enabled, `hg commit` will effectively run `hg status -mardu` after committing. It can help catch mistakes like not committing all needed files or not adding unknown files that should've been part of the just created commit.

tests: flatten repo structure in test-commit.t Let's move to parent directory before `hg init` repos, since they don't need to be nested. It makes amend/strip messages that include full path to the backup bundle shorter, for instance.

lfs: add a TODO file This is a cleaned up and reorganized list of items I sent out about a year ago. But tracking this in the repo (like the narrow extension) gives more visibility in case anyone wants to help out.

copies: make "limit" argument to _tracefile() mandatory We always pass a limit. I think the fact that it was optional was also the reason we checked ">=limit" before we used it. So now we can remove that condition too. Differential Revision: https://phab.mercurial-scm.org/D6335

localrepo: don't use defaults arguments that will never be overridden The commithook() callback will be called when the lock is released. lock.release() calls the callback without arguments, so it was quite confusing to me that this function declared extra arguments. We can just close on the variables in the outer scope instead. Differential Revision: https://phab.mercurial-scm.org/D6336

tags: avoid double-reversing a list Differential Revision: https://phab.mercurial-scm.org/D6337

updatecaches: also warm hgtagsfnodescache Now that a full update of this cache run in a reasonable amount of time, we can warm everything when during a full update.

hgtagsfnodescache: inherit fnode from parent when possible If a changeset does not update the content of `.hgtags`, it means it will use the same file-node (for `.hgtags`) as its parents. In this case we can directly reuse the parent's file-node. We use this property when updating the `hgtagsfnodescache` taking a faster path if we already have a cached value for the parents of the node we are looking at. Doing so provides a large performance boost when looking at a lot of fnodes, especially on repository with very large manifest: timing for `tagsmod.fnoderevs(ui, repo, repo.changelog.revs())` mercurial: (41907 revisions, 1923 files) before: 6.9 seconds after: 2.7 seconds (-54%) pypy: (96266 revisions, 5198 files) before: 80 seconds after: 20 seconds (-75%) mozilla-central: (463411 revisions, 272080 files) before: 7166.4 seconds after: 47.8 seconds (-99%, x150 speedup) On a copy of mozilla-try with about 35K heads ans 1.7M changesets, this moves the computation from many hours to a couple of minutes, making it more interesting to do a full warm up of this cache before computing tags (from a cold cache). There seems to be other performance low hanging fruits, like avoiding the use of changectx or a more revision centric logic. However, the new code is fast enough for my needs right now.

hgtagsfnodescache: handle nullid lookup The null revision is empty, so it `.hgtags` content is `nullid` in regards with the `hgtagsfnodescache`. Dealing with `nullid` will help with the next changeset. Before this change, feeding `nullid` to `hgtagsfnodescache.getfnode` would return a wrong result (fnode for tip).

help: register the 'gpg' command category and give it a description help.py expects extensions to register their command category in the CATEGORY_ORDER and CATEGORY_NAMES variables. Once gendoc.py orders commands by category, in the next patch, it'll assume this registration (and raise an exception on encountering any unregistered categories). Luckily, gpg is the only bundled extension with an unregistered custom category, so let's fix it. Differential Revision: https://phab.mercurial-scm.org/D6324

histedit: Speed up scrolling in patch view mode Store patchcontents into the mode state, avoiding the expensive call to ui for computing the patchcontents. Before this change in large repos histedit patch view mode can be very irresponsive.

histedit: Show file names in multiple line format

repoview: introduce a `experimental.extra-filter-revs` config The option define revisions to additionally filter out of all repository "view". The end goal is to provide and easy to way to serve multiple subset of the same repository using multiple "shares". The simplest use case of this feature is to have one view serving the public changesets and one view also serving the draft. This is currently achievable using the new `server.view` option introduced recently by Joerg Sonnenberger. However, more advanced use cases need more advanced definitions. For example some needs a view dedicated to some release branches, or view that hides security fixes to be released. Joerg Sonnenberger and I discussed this topic at the recent mini-sprint and the both of us have seen real life use cases for this. (This series got written during the same mini-sprint). The feature is fully functional, and use similar cache-fallback mechanism to ensure decent performance. However,there remaining room to ensure each share caches and hooks collaborate with each others. This will come at a later time once users start to actually test this feature on real usecase.

copies: filter out copies from non-existent source later in _chain() _changesetforwardcopies() repeatedly calls _chain(). That is very expensive because _chain() does lookups in the manifest. I hope to split up the function in two parts: 1) simple chaining, not considering end points, and 2) filter out files that don't exist in the end points (and ping-pong copies/renames). This patches gets us closer to that by moving the check for non-existent source later in the function. Now there are no more checks for "src" and "dst" in the first loop; all the filtering of invalid copies is done in the second loop. The code also looks much more consistent now. No measureable impact on `hg debugpathcopies 4.0 4.8`. That shouldn't be surprising since the only case we're doing more checks now is in case of chained copies/renames, which are quire rare in practice. Differential Revision: https://phab.mercurial-scm.org/D6277

copies: clarify mutually exclusive cases in _chain() with a s/if/elif/ If the 'b' dict has a rename from 'x' to 'y', it shouldn't be possible for 'x' to be both (a key) in 'a' and in 'src'. That would mean that 'x' is a file in the source commit and also a rename destination in the intermediate commit. But we currently don't allow renaming files onto existing files, so that shouldn't happen. So let's clarify that by using an "elif" instead of an "if". And if we did allow renaming files onto existing files, we should prefer to use the rename destination in the intermediate commit as source anyway. Differential Revision: https://phab.mercurial-scm.org/D6276

copies: delete a redundant cleanup step in _chain() The check is redundant since d5edb5d3a337 (copies: filter out copies when target is not in destination manifest, 2019-02-14). To test that hypothesis, I made this change in the commit that commit, but all tests still passed. I think the case was necessary before then, we just didn't have tests for it. Differential Revision: https://phab.mercurial-scm.org/D6275

copies: document cases in _chain() Differential Revision: https://phab.mercurial-scm.org/D6274

copies: ignore heuristics copytracing when using changeset-centric algos Differential Revision: https://phab.mercurial-scm.org/D6269

copies: move check for experimental.copytrace==<falsy> earlier I'm going to ignore experimental.copytrace when changeset-centric algorithms are required. This little refactoring makes that easier to add. Differential Revision: https://phab.mercurial-scm.org/D6268

copies: replace .items() by .values() where appropriate As pointed out by Pierre-Yves. Differential Revision: https://phab.mercurial-scm.org/D6266

copies: inline _computenonoverlap() in mergecopies() We now call pathcopies() from the base to each of the commits, and that calls _computeforwardmissing(), which does file prefetching (in the remotefilelog override). So the call to _computenonoverlap() is now pointless (the sets of files from _computenonoverlap() are subsets of the sets of files from _computeforwardmissing()). This somehow also fixes a broken remotefilelog test. Differential Revision: https://phab.mercurial-scm.org/D6256

copies: calculate mergecopies() based on pathcopies() When copies are stored in changesets, we need a changeset-centric version of mergecopies() just like we have a changeset-centric version of pathcopies(). I think the natural way of thinking about mergecopies() is in terms of pathcopies() from the base to each of the commits. So if we can rewrite mergecopies() based on two such pathcopies() calls, we'll get the changeset-centric version for free. That's what this patch does. A nice bonus is that it ends up being a lot simpler. mergecopies() has accumulated a lot of technical debt over time. One good example is the code for dealing with grafts (the "partial/incomplete/dirty" stuff). Since pathcopies() already deals with backwards renames and ping-pong renames, we get that for free. I've run tests with hard-coded debug logging for "fullcopy" and while I haven't looked at every difference it produces, all the ones I have looked at seemed reasonable to me. I'm a little surprised that no more tests fail when run with '--extra-config-opt experimental.copies.read-from=compatibility' compared to before this patch. This patch also fixes the broken cases in test-annotate.t and test-fastannotate.t. It also enables the part of test-copies.t that was previously disabled exactly because mergecopies() needed to get a changeset-centric version. One drawback of the rewritten code is that we may now make remotefilelog prefetch more files. We used to prefetch files that were unique to either side of the merge compared to the other. We now prefetch files that are unique to either side of the merge compared to the base. This means that if you added the same file to each side, we would not prefetch it before, but we would now. Such cases are probably quite rare, but one likely scenario where they happen is when moving from a commit to its successor (or the other way around). The user will probably already have the files in the cache in such cases, so it's probably not a big deal. Some timings for calculating mergecopies between two revisions (revisions shown on each line, all using the common ancestor as base): In the hg repo: 4.8 4.9: 0.21s -> 0.21s 4.0 4.8: 0.35s -> 0.63s In and old copy of the mozilla-unified repo: FIREFOX_BETA_60_BASE^ FIREFOX_BETA_60_BASE: 0.82s -> 0.82s FIREFOX_NIGHTLY_59_END FIREFOX_BETA_60_BASE: 2.5s -> 2.6s FIREFOX_BETA_59_END FIREFOX_BETA_60_BASE: 3.9s -> 4.1s FIREFOX_AURORA_50_BASE FIREFOX_BETA_60_BASE: 31s -> 33s So it's measurably slower in most cases. The most significant difference is in the hg repo between revisions 4.0 and 4.8. In that case it seems to come from the fact that pathcopies() uses fctx.isintroducedafter() (in _tracefile), while the old mergecopies() used fctx.linkrev() (in _checkcopies()). That results in a single call to filectx._adjustlinkrev(), which is responsible for the entire difference in time (in my repo). So we pay a performance penalty but we get more correct code (see change in test-mv-cp-st-diff.t). Deleting the "== f.filenode()" in _tracefile() recovers the lost performance in the hg repo. There were are few other optimizations in _checkcopies() that I could not measure any impact from. One was from the "seen" set. Another was from a "continue" when the file was not in the destination manifest (corresponding to "am" in _tracefile). Also note that merge copies are not calculated when updating with a clean working copy, which is probably the most common case. I therefore think the much simpler code is worth the slowdown. Differential Revision: https://phab.mercurial-scm.org/D6255

tests: add test where copy source is deleted and added back This shows another difference between pathcopies() and mergecopies(): mergecopies() considers files that have been deleted and then added back as different files, but pathcopies() does not. Differential Revision: https://phab.mercurial-scm.org/D6330

merge with stable

obsolete: drop the legacy `_enabled` variable Evolve 8.5.0 stopped setting this, and it would have been easier to figure out why TortoiseHg stopped allowing amends if it would have crashed on the missing variable.

discovery: only calculate closed branches if required The number of new closed branches is required for printing in error message. So let's only calculate them if we need to print error about new branches. Differential Revision: https://phab.mercurial-scm.org/D6314

branchcache: store the maximum tip in a variable inside for loop Instead of assigning self.tiprev multiple times in the for loop, and calling cl.node() on it, let's store that in a temporary variable and assign it in the end of loop. Differential Revision: https://phab.mercurial-scm.org/D6311

tests: demonstrate that rename is followed to wrong parent from merge This test case shows another way that copies are handled differently between `hg st` (pathcopies()) and `hg co -m` (mergecopies). The reason is that pathcopies() calls _tracefiles(), which checks that the file nodeid of an ancestor matches the file nodeid in the base commit. mergecopies() should probably be doing the same. Differential Revision: https://phab.mercurial-scm.org/D6323

test: demonstrate failure to follow rename with shadowed linkrev This shows a difference in handling of copies between `hg st` (pathcopies()) and `hg co -m`. The issue here is that mergecopies() uses the unadjusted linkrev() for determining when to stop walking ancestors. Differential Revision: https://phab.mercurial-scm.org/D6322

tests: slightly modify a linkrev test to prepare for expanding it The test case checks that the copy tracing code doesn't get confused by linkrevs when walking a file's ancestors. This patch chnages the test slightly so a second commit is grafted, thus producing a second "bad" linkrev. I'll use this in the next patch to demonstrate a bug. Differential Revision: https://phab.mercurial-scm.org/D6321

copies: process files in deterministic order for stable tests I also fixed a typo while at it. Differential Revision: https://phab.mercurial-scm.org/D6320

py3: properly reject non-encoded strings given to hgweb

py3: handle meta-path finders that only use pre-python3.4 API

remotefilelog: add missing argument to hg.verify wrapper Differential Revision: https://phab.mercurial-scm.org/D6313

revsetbenchmark: track some simple use of "only" The only revset is quite useful and has various possible optimisation. tracking its timing seems useful.

push: added clear warning message when pushing closed branches(issue6080) Differential Revision: https://phab.mercurial-scm.org/D6038

branch: abort if closing branch from a non-branchhead cset This patch make sure that we abort if the user is trying to close a branch from a cset which is not a branch head. Changes in test file reflect the fixed behaviour. Differential Revision: https://phab.mercurial-scm.org/D6282

branch: add tests which shows branch can be closed from a non-branchhead cset This patch shows that we can close a branch even from a cset which is not a branch head. It was supposed to abort this operation. Next patch will be fixing the issue. Differential Revision: https://phab.mercurial-scm.org/D6281

phabricator: read more metadata from local:commits local:commits metadata can contain branch info, and 'rev' has been superseded by 'commit', see: https://github.com/phacility/arcanist/blob/83661809e532c3fe444a8bf7c7d6936e6377691b/src/repository/api/ArcanistMercurialAPI.php#L281 Differential Revision: https://phab.mercurial-scm.org/D6300

phabricator: don't assume the existence of properties of local:commits Not all the properties are guaranteed to be there, so if we don't check first we could die with a KeyError. Differential Revision: https://phab.mercurial-scm.org/D6299

phabricator: include branch in the diffproperty metadata This does not make Phabricator display the branch in web UI anywhere as that still need us to use creatediff API for that. However a future patch will make phabread use this to include the branch in its `hg import`-able output. Differential Revision: https://phab.mercurial-scm.org/D6297

tests: demonstrate `hg log -r . <file>` linkrev bug Differential Revision: https://phab.mercurial-scm.org/D6309

unionrepo: sync with repository API Differential Revision: https://phab.mercurial-scm.org/D6289

match: remove unused match.__iter__ implementation (API) Differential Revision: https://phab.mercurial-scm.org/D6305

fix: allow fixer tools to return metadata in addition to the file content With this change, fixer tools can be configured to output a JSON object that will be parsed and passed to hooks that can be used to print summaries of what code was formatted or perform other post-fixing work. The motivation for this change is to allow parallel executions of a "meta-formatter" tool to report back statistics, which are then aggregated and processed after all formatting has completed. Providing an extensible mechanism inside fix.py is far simpler, and more portable, than trying to make a tool like this communicate through some other channel. Differential Revision: https://phab.mercurial-scm.org/D6167

merge with stable

phabricator: set local:commits time metadata as an int, not a string Same as arcanist does Differential Revision: https://phab.mercurial-scm.org/D6296

phabricator: use templatefilters.json in writediffproperties Instead of json.dumps, since it makes the code simpler and more readable. This would have been the better option for 8fd19a7b4ed6 but I wasn't aware of it at the time. Differential Revision: https://phab.mercurial-scm.org/D6295

commands: use byteskwargs() in verify() Otherwise Python 3 complains about the missing key. Differential Revision: https://phab.mercurial-scm.org/D6294

match: use raw strings to avoid illegal baskslash escape Python 3.8 was complaining about the invalid escape sequences. Let's use raw strings to avoid the warning and double baskslashes. Differential Revision: https://phab.mercurial-scm.org/D6293

revbranchcache: use context manager in _writerevs() to write to file The other _writenames() is a bit complicated to use context manager. Differential Revision: https://phab.mercurial-scm.org/D6292

revbranchcache: factor logic to write names and revs in separate functions Before this patch, the write function was so populated with upto 4 level of indentation, it was hard to understand what's going on. Differential Revision: https://phab.mercurial-scm.org/D6291

tests: make log style a little easier to read in test-copytrace-heuristics.t Revision numbers are much shorter and easier to read (especially compared to the full nodeids that were used here), so I switched to that. That's also what almost all the commands used (e.g. `hg rebase -s . -d 1`). I updated the two instances that used nodeids. I also made some other little cleanups to the log templates. Differential Revision: https://phab.mercurial-scm.org/D6279

tests: avoid cryptic nodeids in tests/test-rename-merge1.t These two nodeids had not been part of any output before, so one can't know which revision they refer to without adding something like `hg log` before them. It turned out that '.^' was equivalent for both of them, so that's what I replaced them with. Differential Revision: https://phab.mercurial-scm.org/D6280

tests: defines aliases for `hg log` calls in test-copytrace-heuristics.t This also makes the test cases more consistent since a few had missed the ":" in "changeset:" that the others used. Differential Revision: https://phab.mercurial-scm.org/D6278

rust-discovery: implementing and exposing stats() This time, it's simple enough that we can do it in all layers in one shot. Differential Revision: https://phab.mercurial-scm.org/D6233

rust-discovery: cpython bindings for the core logic As previously done with the ancestors submodule, testing for the bindings is provided from Python on a trivial case. Differential Revision: https://phab.mercurial-scm.org/D6232