tests: clean up built binaries after running test-fuzz-targets.t Most users won't notice a change here because they won't have the fuzzer infra, but by good fortune my workstation has the required bits and keeps leaving the fuzzer binaries around. Differential Revision: https://phab.mercurial-scm.org/D6862

fastannotate: remove support for flock() locking We've seen enough weirdness in CI with flock for remotefilelog that I'm now of the opinion we should just stop using flock() everywhere until someone has a concrete need for the extra performance *and* a way to only use it when safe (even if that's just default-to-off.) Differential Revision: https://phab.mercurial-scm.org/D6861

remotefilelog: remove dead code for using flock() for locking Differential Revision: https://phab.mercurial-scm.org/D6860

narrow: add option for automatically removing unused includes It's been a somewhat common request among our users to have Mercurial automatically pick includes to remove. This patch adds an option for that: `hg tracked --auto-remove-includes`. I'm not sure if this is the right name and semantics for it. Perhaps the feature should also add excludes of large subdirectories even if other files in the include are needed? Narrow clones are experimental, so we can change the name and/or semantics later if necessary. Differential Revision: https://phab.mercurial-scm.org/D6848

narrow: don't hexify paths and double-hexify known nodes on wire (BC) It isn't obvious, but wireprototypes.encodelist() is meant only for binary nodeids. So when we used it for encoding hex nodeids and paths, the encoded result was surprising and hard to read. This patch changes the encoding to make the list of paths a comma-separated list and the list of common nodes to be a encodelist()-encoded list of binary nodeids (so the result is just singly-hexified nodeids). This is clearly a breaking change, but the feature is experimental and we're not aware of anyone running a server using this command yet. Differential Revision: https://phab.mercurial-scm.org/D6851

remotefilelog: replace repack lock to solve race condition 2c74337e6483 reduced the probability of race-conditions when starting background repack and prefetch and we saw the difference in our CI instance with all failures disappearing except one where one call to waitonrepack seems to returns too early. I'm not sure what exactly goes wrong but I realized that while the prefetch operation uses a standard Mercurial lock, the repack operation is using a custom lock based on `fcntl.flock` on available platforms. As `extutil.flock` fallback on traditional Mercurial locks on other platforms and the tests are stable on my laptop, our CI environment and GCC112, I'm sending this patch to standardize the behavior across environments. Differential Revision: https://phab.mercurial-scm.org/D6844

perf: add a --stats argument to perfhelper-pathcopies The arguments will display some statisting about the distribution of the value we measure.

perf: add a --stats argument to perfhelper-mergecopies The arguments will display some statistics about the distribution of the value we measure.

archive: add XZ support if built with Python 3

rust-cpython: add sanity check to PySharedState::decrease_leak_count() If decrease_leak_count() were called unnecessarily, there must be a serious bug. It's better to not silently ignore such cases.

tests: stabilize test-fix.t on Windows `pwd` prints /tmp/... style paths, not C:\... needed for $TESTTMP to be substituted. In the final test, for whatever reason, Windows was missing EOL in the files and printing: [wdir] changedlines: printf: warning: ignoring excess arguments, starting with 'printf' even though it was trying to run: printf "Line ranges:\n"; printf "2 through 2\n"; I tried wrapping both :command and :linerange in `sh -c "..."`, and while that fixed the missing EOL, it missed the "2 through 2" output. Differential Revision: https://phab.mercurial-scm.org/D6852

zstandard: vendor python-zstandard 0.12 The upstream source distribution from PyPI was extracted. Unwanted files were removed. The clang-format ignore list was updated to reflect the new source of files. test-repo-compengines.t was updated to reflect a change in behavior of the zstd library. The project contains a vendored copy of zstandard 1.4.3. The old version was 1.3.8. This should result in some minor performance wins. # no-check-commit because 3rd party code has different style guidelines Differential Revision: https://phab.mercurial-scm.org/D6858

uncommit: enable support for adding a note This comes from the evolve extension's version of uncommit. The logic was already in place, and appears to be the last of the trivial things that can be enabled. Should these note options (including on amend) be marked advanced to keep the help text clutter level down? Differential Revision: https://phab.mercurial-scm.org/D6857

amend: enable support for using the secret phase This comes from the evolve extension's version of amend. The logic was already in place, and appears to be the last of the trivial things that can be enabled. Differential Revision: https://phab.mercurial-scm.org/D6856

amend: enable support for closing the branch This comes from the evolve extension's version of amend. The logic was already in place. Differential Revision: https://phab.mercurial-scm.org/D6855

amend: prevent '\n' in the note string This comes from the evolve function. I'm not sure why this check was missing in core, since it was present when the length check was added to evolve. I didn't flag this as BC because 530b7361e3a9 mentioned this argument wasn't added to the release notes due to no display capability, and that hasn't changed AFAIK. Differential Revision: https://phab.mercurial-scm.org/D6854

amend: add option to update to the current user This is also from the evolve extension's version of amend. A side effect of this refactoring is for uncommit to support `rewrite.update-timestamp`. Differential Revision: https://phab.mercurial-scm.org/D6853

bundle2: fix an off-by-one in debug message of number of parts Differential Revision: https://phab.mercurial-scm.org/D6850

tests: move a config write to top of file since it applies to all tests I'm about to add another test that depends on this config. Differential Revision: https://phab.mercurial-scm.org/D6849

idirstate: group private methods and attrs that are in the interface This makes it a little more obvious at a glance what work is left. Fortunately there's not a whole lot left. I suspect the ignore logic is going to be the tricky bit. Differential Revision: https://phab.mercurial-scm.org/D6839

idirstate: remove now non-public _map attribute Differential Revision: https://phab.mercurial-scm.org/D6838

interfaces: introduce an interface for dirstate implementations As usual with adding interface definitions, this describes the way things are, not the way we'd like things to be. There are some clear problems in the interface right now (eg ._map leaks in a few places), but I have plans to clean those up. There are also many missing docstrings, but again, we'll make a second pass to clean that up. Differential Revision: https://phab.mercurial-scm.org/D6836

cleanup: fix leakage of dirstate._map to client code We already had proper accessors for most of the behavior of dirstate._map that callers cared about exposed in the actual dirstate class as public methods. Sigh. There are two remaining privacy violations in the codebase after this change: 1) In the perf extension, which I suspect has to stick around because it's really testing the dirstate implementation directly 2) In largefiles, where we deal with standins and mutating status. Looking at this, I _strongly_ suspect a formal dirstate interface would allow this to work via composition instead of inheritance and monkeypatching. Fortunately, such wins are a part of my motivation for this work. I anticipate we'll come back to this in due time. Differential Revision: https://phab.mercurial-scm.org/D6837

exchange: convert bookmark nodes from hex to bin ASAP Differential Revision: https://phab.mercurial-scm.org/D6831

exchange: avoid unnecessary conversion of bookmark nodes to hex (API) Differential Revision: https://phab.mercurial-scm.org/D6830

highlight: fix encoding issues to enable Py3 compatibility This commit fixes various encoding issues with the `highlight` extension to enable compatibility with Python 3. Python `.encode()` and `.decode()` requires the target encoding to be passed as a `str`, so the value of `mercurial.encoding.encoding` must be converted before passing to the function. Pygments also assumes the `str` type for values it works with, so we must perform conversions before and after receiving values from its APIs. After applying this patch, `test-highlight.t` passes under Python 3. We add it to `python3-whitelist` as well. Tested with Pygments 2.4.2. Differential Revision: https://phab.mercurial-scm.org/D6832

hgweb: add a `message` attribute to `hgweb.common.ErrorResponse` This fixes a Python 3 bug where hgweb assumes an Exception subclass will have a `.message` attribute after running `Exception.__init__`.[1] The Python 3 way to get this info would be `e.args[0]`, but adding a new named attribute is more ergonomic in my view. [1] https://www.mercurial-scm.org/repo/hg-committed/file/6ccf539aec71/mercurial/hgweb/hgwebdir_mod.py#l459 Differential Revision: https://phab.mercurial-scm.org/D6840

uncommit: make -D/--date and -U/--user mutually exclusive This is how amend and graft work (but not MQ). I'm not sure why this didn't work for me when I first tried it. Differential Revision: https://phab.mercurial-scm.org/D6842

uncommit: drop the hyphen from --current-user and --current-date I didn't pay enough attention to these long forms- graft, amend and MQ already use the old style naming. It's probably more important to be consistent than modern. The hypenated style came from evolve. Yuya mentioned this naming discrepancy in 4145fd3569c3, but it didn't attract any discussion[1]. There's also a bit of inconsistency in that the default parameter for `currentdate` is `False` for graft, and `None` for the rest. [1] https://www.mercurial-scm.org/pipermail/mercurial-devel/2019-January/126767.html Differential Revision: https://phab.mercurial-scm.org/D6841

hgweb: fix websub regex flag syntax on Python 3 The `websub` config section for hgweb is broken under Python 3 when using regex flags syntax (ie the optional `i` in the example from `hg help config.websub`: patternname = s/SEARCH_REGEX/REPLACE_EXPRESSION/[i] Flags are pulled out of the specified byte-string using a regular expression, and uppercased. The flags are then iterated over and passed to the `re` module using `re.__dict__[item]`, to get the object attribute of the same name from the `re` module. So on Python 2 if the `il` flags are passed, this transition looks like: `'il'` -> `'IL'` -> `'I'` -> `re.__dict__['I']` -> `re.I` However on Python 3, these are bytes objects. When we iterate over a bytes object in Python 3, instead of getting the individual characters in the string as string objects of length one, we get the integer \ value corresponding to that byte. So the same transition looks like: `b'il'` -> `b'IL'` -> `73` -> `re.__dict__[73]` -> `KeyError` This commit fixes the type mismatch by converting the bytes to a system string before iterating over each element to pass to `re`. The transition will now look like: `b'il'` -> `u'IL'` -> `u'I'` -> `re.__dict__[u'I']` -> `re.I` In addition we expand `test-websub.t` to cover the regex flag case (for both the `websub` section and `interhg`). Differential Revision: https://phab.mercurial-scm.org/D6788