rust-filepatterns: use bytes instead of String In my initial patch, I introduced an unnecessary hard constraint on UTF-8 filenames and patterns which I forgot to remove. Although the performance penalty for using String might be negligible, we don't want to break compatibility with non-UTF-8 encodings for no reason. Moreover, this change allows for a cleaner Rust core API. This patch introduces a new utils module that is used with this fix. Finally, PatternError was not put inside the Python module generated by Rust, which would have raised a NameError. Differential Revision: https://phab.mercurial-scm.org/D6485

doc: fix description of "predecessors" to match reality Differential Revision: https://phab.mercurial-scm.org/D6467

phabricator: make `hg debugcallconduit` work outside a hg repo I am trying to write some automations around phabricator and having debugcallconduit work outside a hg repo will be nice! Marking command as optionalrepo instead of norepo because we might to load repo/.hg/hgrc. Differential Revision: https://phab.mercurial-scm.org/D6499

phabricator: pass ui instead of repo to callconduit This will help us make `hg debugcallconduit` work outside a hg repo as next patch will mark that command as no repo. Differential Revision: https://phab.mercurial-scm.org/D6498

phabricator: pass ui into readurltoken instead of passing repo The goal of this series is to make `hg debugcallconduit` work outside of a hg repo. This patch, removes requirement of repo object from readurltoken as we only need ui there. It also updates the callers to pass in ui instead of repo. Differential Revision: https://phab.mercurial-scm.org/D6497

py3: add test-contrib-emacs.t to passing tests list I installed emacs on the server running buildbot and the test started passing on Python 3. Lets add it to the list of passing test. Differential Revision: https://phab.mercurial-scm.org/D6500

phabricator: add commenting to phabsend for new/updated Diffs Especially useful when sending updates to existing Revisions so one can specify the sort of changes e.g. "Address review comments" or "Rebase to tip" If the diff content hasn't changed then it only needs a metadata update which doesn't show in the Phabricator updates UI, so don't add a comment that will. Differential Revision: https://phab.mercurial-scm.org/D6496

py3: fix test-bookmarks-corner-case.t For some reasons, the output of print was not going through. Replaced that ui.status(). Differential Revision: https://phab.mercurial-scm.org/D6481

py3: fix test-fix-metadata.t # skip-blame as just b'' prefixes Differential Revision: https://phab.mercurial-scm.org/D6480

py3: add b'' prefix at one place in run-tests.py #skip-blame because just b'' prefix Differential Revision: https://phab.mercurial-scm.org/D6482

copies: separate added/removed files by newline instead of null This makes it more consistent with how we encode copies (newline-separated lists of null-separated pairs). This perhaps makes {extras} a little less readable (?) despite avoiding the escaping. I don't know how I feel about this patch. I'm okay with it being queued or dropped. Differential Revision: https://phab.mercurial-scm.org/D6486

copies: also encode p[12]copies destination as index into "files" list This is mostly for consistency with the filesaddes/filesremoved fields, but it should also save a bit of space. Differential Revision: https://phab.mercurial-scm.org/D6431

discovery: be more conservative when adjusting the sample size Since 5b34972a0094, the discovery will increase the sample size when it detect a "complex" undecided set. However this detection focussed on the number of roots only, this could regress discovery performance when the undecided set has many roots that eventually get merged into a few heads. To prevent such misbehavior, we adjust the logic to take in account both heads and roots. The sample size will be increased only if both are especially large. Performance testing on the same case as 5b34972a0094, does not show a significant difference.

rust-dirstate: create dirstate submodule This change is here to facilitate a future patch that is written in a different file. I expect this module to grow a few different files. Differential Revision: https://phab.mercurial-scm.org/D6389

profiling: show actual time spent in hotpath display To get, for instance: ... \ 6.6% 4.08s lock.py: __exit__ line 1566: ... | 6.5% 4.01s exchange.py: close line 1191: ... | 6.5% 4.01s transaction.py: _active line 1443: ... | 6.5% 4.01s transaction.py: close line 47: ... | 6.2% 3.84s scmutil.py: wrapped line 529: ... | 6.2% 3.81s localrepo.py: wrapper line 2114: ... | 6.2% 3.81s localrepo.py: updatecaches line 177: ... ... instead of: ... \ 6.6% lock.py: __exit__ line 1566: ... | 6.5% exchange.py: close line 1191: ... | 6.5% transaction.py: _active line 1443: ... | 6.5% transaction.py: close line 47: ... | 6.2% scmutil.py: wrapped line 529: ... | 6.2% localrepo.py: wrapper line 2114: ... | 6.2% localrepo.py: updatecaches line 177: ... ... I find that if it's not displayed, I frequently end up estimating the numbers by hand. Differential Revision: https://phab.mercurial-scm.org/D6477

merge with stable

merge with stable

root: add template variables pointing to repository directories These paths are useful for GUI applications to detect changes. A GUI process typically monitors .hg and .hg/store directories so that it will be notified on lock/wlock deletion. Alternatively, maybe we can add debugpaths command if we don't want to extend the root command. I'm not sure which will be nicer.

root: add support for -Tformatter option It's useless right now, but it should just work and I want to add a few more fields.

narrow: pass the bundle to bundle2.widen_bundle() instead of generating there This will make the code in narrowwirepeer.py more better for further refactoring. Differential Revision: https://phab.mercurial-scm.org/D6438

narrow: refactor code around widening complicated by previous patch Previous patch while adding support for using narrow_widen wireproto command, complicated the code a bit. This patch refactors that. Differential Revision: https://phab.mercurial-scm.org/D6437

narrow: use narrow_widen wireproto command to widen in case of ellipses Few releases ago, we introduce narrow_widen wireproto command to be used to widen narrow repositories. Before this patch, that was used in non-ellipses cases only. In ellipses cases, we still do exchange.pull() which can pull more data than required. After this patch, the client will first check whether server supports doing ellipses widening using wireproto command or not by checking server's wireproto capability. If the server is upto date and support latest ellipses capability, we call the wireproto command. Otherwise we fallback to exchange.pull() like before. The compat code make sure that things works even if one of the client or server is old. The initial version of this patch does not had this compat code. It's added to help Google release things smoothly internally. I plan to drop the compat code before the upcoming major release. Due to change to wireproto command, the code looks a bit dirty, next patches will clean that up. Differential Revision: https://phab.mercurial-scm.org/D6436

merge: correct argument name in docstring Differential Revision: https://phab.mercurial-scm.org/D6476

narrowspec: replace one recursion-avoidance hack with another When updating the working copy narrowspec, we call context.walk() in order to find which files to update the working copy with. context.walk() calls repo.narrowmatch(). In order to avoid infinite recursion in this case, we have a hack that assigns the new values for repo.narrowpats and repo._narrowmatch. However, doing that of course breaks future invalidation of those properties (they're @storecache'd). Let's instead avoid the infinite recursion by setting a flag on the repo instance when we're updating the working copy. Differential Revision: https://phab.mercurial-scm.org/D6468

merge: simplify initialization of "pas" Differential Revision: https://phab.mercurial-scm.org/D6472

merge: reorder some initialization to make more sense This puts the closely related definitions of "pl", "p1", "p2", "pas" close together, and moves the definition of "overwrite" away and closer to where it's first used. Differential Revision: https://phab.mercurial-scm.org/D6471

rust-dirstate: architecture independence fix Apparently, c_char is u8 on ppc64le and i8 on amd64 Differential Revision: https://phab.mercurial-scm.org/D6473

context: get filesadded() and filesremoved() from changeset if configured This adds the read side for getting the sets of added and removed files from the changeset extras. I timed this command on the hg repo: hg log -T '{rev}\n {files}\n %:{file_mods}\n +{file_adds}\n -{file_dels}\n' It took 1m21s before and 6.4s after. I also used that command to check that the result didn't change compared to calculating the values from the manifests on the fly (it didn't change). In the mozilla-unified repo, the same command run on FIREFOX_BETA_58_END::FIREFOX_BETA_59_END went from 29s to 0.67s. Differential Revision: https://phab.mercurial-scm.org/D6417

changelog: optionally store added and removed files in changeset extras As mentioned in an earlier patch, copies._chain() is used a lot in the changeset-centric version of pathcopies(). It is expensive because it needs to look at the manifest in order to filter out copies whose target file has since been removed. I want to store the sets of added and removed files in the changeset in order to speed that up. This patch does the writing part of that. It could easily be a separate config, but it's currently tied to experimental.copies.write-to since that's the only real use case (it will also make the {file_*} template keywords faster, but I doubt that anyone cares enough about those to write extra metadata for them). The new information is stored in the changeset extras. Since they're always subsets of the changeset's "files" list, they're stored as indexes into that list. I've stored the indexes as stringified ints separated by NUL bytes. The size of 00changelog.d for the hg repo increased in size by 0.28% percent (compared to the size with only copy information in the changesets, which in turn is 0.17% larger than without copy information). We could store only the delta between the indexes and we could store them in binary, but the chosen format is more readable. We could also have implemented this as a cache outside the changelog. One advantage of doing it that way is that we would get the speedups from the {file_*} template keywords also on old repos. Another advantage is that it we can rewrite the cache if we find a bug in how we calculate the set of files. A disadvantage is that it would be more complex. Another is that it would surely use more space. We already write the copy information to the changeset extras, so it seems like a small step to also write these file sets. Differential Revision: https://phab.mercurial-scm.org/D6416

templatekw: make {file_*} compare to both merge parents (issue4292) This redefines the {file_adds}, {file_dels}, {file_mods} template keywords by getting the lists from the recently introduced context methods instead of getting them from status compared to p1. As mentioned before, these are better defined on merge commits. The total number of files from the three lists now always add up to the number of files in {files}. I timed this command: hg log -r 4.0::5.0 -T '{rev}\n {file_mods}\n {file_adds}\n {file_dels}\n' It went from 7.6s to 5.6s with this patch. So it's actually faster than before. Note that the "files:" field in the bazaar test log output was using "{file_mods}" (not "{files}" as one might think based on the label). Differential Revision: https://phab.mercurial-scm.org/D6369