revert: no longer mark --interactive as experimental We seem to have settled down on behavior changes here (nothing matching revset `keyword(revert) and keyword(interactive)` since 4.2 was released), so let's go ahead and advertise this excellent feature. .. feature:: revert --interactive The revert command now accepts the flag --interactive to allow reverting only some of the changes to the specified files.

ui: add configlist doctest to document a bit more of the whitespace behavior I'm surprised this wasn't explicitly tested, so I added a test.

merge stable into default

test-run-tests: stabilize the test (issue5735) Previously there is a race condition because things happen in this order: 1. Check shouldStop 2. If shouldStop is false, print the diff 3. Call fail() -> set shouldStop The check and set should really happen in a same critical section. This patch adds a lock to address the issue. Test Plan: Run `run-tests.py -l test-run-tests.t` 20 times on gcc112 and the race condition does not reproduce. Differential Revision: https://phab.mercurial-scm.org/D1830

run-tests: avoid set PYTHONUSERBASE environment variable to None According to python documentation, get_config_var can return None and it was during running mercurial tests under a virtual environment with the local built mercurial installed as a python package with `pip install -e .`. On the other hand, if mercurial is installed in the user library folders with `pip install --local -e .` the issue doesn't reproduce. The fallback is that it will just set an empty string and everything should work as before, according to https://www.python.org/dev/peps/pep-0370/: The path to the user base directory can be overwritten with the environment variable PYTHONUSERBASE. The default location is used when PYTHONUSERBASE is not set or empty.

templatefilters: fix doc of basename() os.path.basename() does NOT strip trailing slashes. > Note that the result of this function is different from the Unix basename > program; where basename for '/foo/bar/' returns 'bar', the basename() > function returns an empty string (''). https://docs.python.org/2.7/library/os.path.html#os.path.basename

identify: document -r. explicitly how to disable wdir scanning (issue5622)

diff: disable diff.noprefix option for diffstat (issue5759) We could use patch.diffhunks() instead of patch.diff() to get filenames without parsing patch content, but that isn't always possible because we sometimes feed raw patch data to patch.diffstat().

hgweb: disable diff.noprefix option for diffstat Copied from cf1e15f91c90.

debugssl: convert port number to int (issue5757) It doesn't use util.getport(), which may resolve service name to port number.

histedit: preserve active branch while histediting The branch information was properly preserved in the changeset, but the "active" branch of the working copy could be lost (the branch of the base being used). Histedit used to behave properly in this regard but the case was not tested and regressed 4 years ago in ab2362e1672e.

templater: fix "one arguments"

tests: write and use a custom helper script to avoid find's -printf -printf on find is a GNU-ism and will be banned in an upcoming check-code change. Differential Revision: https://phab.mercurial-scm.org/D1597

convert: restore the ability to use bzr < 2.6.0 (issue5733) This effectively conditionalizes a234b32b744a. Some Linux distributions (like CentOS 7) use really old versions, and the change referenced was causing exceptions to be thrown. Even though the deprecation warning says 'since 2.5.0', it wasn't marked as such in 2.5.1, but is by 2.6.0. This was tested with 2.4.2 and 2.6.0 with PYTHONWARNINGS=::DeprecationWarning, and both paths were exercized.

Added signature for changeset a92b9f8e11ba

Added tag 4.4.2 for changeset a92b9f8e11ba

merge: check created file dirs for path conflicts only once (issue5716) In large repositories, updates involving the creation of many files check the same directories repeatedly in the wctx manifest. Move these checks out to a separate loop to avoid repeated checks hitting the manifest. Differential Revision: https://phab.mercurial-scm.org/D1226

merge: cache unknown dir checks (issue5716) As mentioned in D1222, the recent pathconflicts change regresses update performance in large repositories when many files are being updated. To mitigate this, we introduce two caches of directories that have already found to be either: - unknown directories, but which are not aliased by files and so don't need to be checked if they are files again; and - missing directores, which cannot cause path conflicts, and cannot contain a file that causes a path conflict. When checking the paths of a file, testing against this caches means we can skip tests that involve touching the filesystem. Differential Revision: https://phab.mercurial-scm.org/D1224

dispatch: add HGPLAIN=+strictflags to restrict early parsing of global options If this feature is enabled, early options are parsed using the global options table. As the parser stops processing options when non/unknown option is encountered, it won't mistakenly take an option value as a new early option. Still "--" can be injected to terminate the parsing (e.g. "hg -R -- log"), I think it's unlikely to lead to an RCE. To minimize a risk of this change, new fancyopts.earlygetopt() path is enabled only when +strictflags is set. Also the strict parser doesn't support '--repo', a short for '--repository' yet. This limitation will be removed later. As this feature is backward incompatible, I decided to add a new opt-in mechanism to HGPLAIN. I'm not pretty sure if this is the right choice, but I'm thinking of adding +feature/-feature syntax to HGPLAIN. Alternatively, we could add a new environment variable. Any bikeshedding is welcome. Note that HGPLAIN=+strictflags doesn't work correctly in chg session since command arguments are pre-processed in C. This wouldn't be easily fixed.

fancyopts: add early-options parser compatible with getopt() The next patch will add a flag for strict parsing of early options, where we'll have to parse all early options at once instead of processing them one-by-one by dispatch._earlygetopt(). That's why I decided to hook fancyopts(). All dispatch._early*opt() functions is planned to be replaced with this function. But in this stable series, only the strict mode will be handled by fancyopts.earlygetopt().

largefiles: pay attention to dropped standin files when updating largefiles Previously, the largefile for a dropped standin would be deleted here, and then restored from the cache. This had the effect of clobbering uncommitted changes if a revert caused the file to be forgotten, which is not what happens with a normal file. Now the removal and update is skipped for dropped largefiles, and the corresponding standin is deleted from disk. This was noticed when working on issue5738 because the forgotten standin files were left behind, and that changes the behavior of the next rename to that directory. My first attempt was to cleanup the standins before calling this. That failed, because this function deletes the largefile if the corresponding standin is missing. This function is called by the revert command, merge (and therefore update), and patch, via the scmutil.marktouched() override. So it should be pretty narrow in scope. I didn't mark issue5738 as fixed because the move related issues can still happen if the main tree and the .hglf subtree get out of sync somehow. I don't see an easy fix for that, but that should be an edge case. If whoever queues this thinks it is good enough to close out the bug and can cram it into the summary, go for it.

test-largefiles: demonstrate problems with renaming and reverting a directory These things were uncovered looking at issue5738. First, if the destination directory exists under .hglf, the source is moved under the destination instead of renaming the last component for `hg mv srcdir dstdir`. This is extra confusing, because it occurs even if the user visible destination (i.e. the path _not_ under .hglf) does not exist. Additionally, when a largefile is forgotten via revert, any modifications end up getting clobbered. For normal files, the forgotten file is left unchanged, as shown by test-import.t. The forget command on a largefile will correctly leave the file unmodified.

tests: add globs for Windows

cat: record the current behavior of wildcard matches in subrepos Mercurial subrepos support any match patterns.

merge with i18n

i18n-pt_BR: synchronized with cabc840ffdee

convert: avoid wrong lfconvert defaults by moving configitems to core The `hg lfconvert --to-normal` command uses the convert extension internally to work its magic, but that produced devel-warn messages if the convert extension wasn't loaded by the user. The test in fcd2f9b06629 (modified here) wasn't showing the warnings because the convert extension was loaded via $HGRCPATH. Most of the config options default to None/False, but 'hg.usebranchnames' and 'hg.tagsbranch' are supposed to default to True and 'default' respectively. The first iteration of this was to ui.setconfig() inside lfconvert, to force the convert extension to load. But there really is no precedent for doing this, and check-config complained that 'extensions.convert' isn't documented. Yuya suggested this alternative. This partially backs out 0d5a1175d0f9.

tr-summary: keep a weakref to the unfiltered repository Repoview can have a different life cycle, causing issue in some corner cases. The particular instance that revealed this comes from localpeer. The localpeer hold a reference to the unfiltered repository, but calling 'local()' will create an on-demand 'visible' repoview. That repoview can be garbaged collected any time. Here is a simplified step by step reproduction:: 1) tr = peer.local().transaction('foo') 2) tr.close() After (1), the repoview object is garbage collected, so weakref used in (2) point to nothing. Thanks to Sean Farley for helping raising and debugging this issue.

dispatch: verify result of early command parsing Before, early options were stripped from args, and because of this, some kind of parsing errors weren't reported. For example, $ hg ci -m -Ra file would execute "hg ci -m file" in repository "a". This patch fixes the issue by parsing early options again by real getopt-based parser, and verifying the results. If the early parsing appears wrong, hg just aborts. The current error message seems not nice, and should be improved, maybe in V2 or follow-up. Note that this isn't a security feature because we can still do anything by using shell aliases.

dispatch: convert non-list option parsed by _earlygetopt() to string So we can easily compare it with the corresponding getopt() result. There's a minor behavior change. Before, "hg --cwd ''" failed with ENOENT. But with this patch, an empty cwd is silently ignored. "hg -R ''" has always worked as such, so -R has no BC.

dispatch: add option to not strip command args parsed by _earlygetopt() This allows us to parse the original args later by full-blown getopt() in order to verify the result of the faulty early parsing. Still we need the 'strip=True' behavior for shell aliases. Note that this series is RFC because it seems to change too much to be included in stable release.

dispatch: fix early parsing of short option with value like -R=foo Before, -R=foo was parsed as '-R' 'foo', which disagrees with the standard getopt behavior.

dispatch: abort if early boolean options can't be parsed Perhaps we'll need to restrict the parsing rules of --debugger and --profile, where this patch will help us know why the --debugger option doesn't work. I have another series to extend this feature to --config/--cwd/-R, but even with that, shell aliases can be used to get around the restriction.

dispatch: stop parsing of early boolean option at "--"

dispatch: extract stub function to peek boolean command option We should at least stop parsing at "--". The 'name' argument is passed for future extension.

dispatch: do not drop unpaired argument at _earlygetopt() Before, "hg log -R" just worked.

amend: update .hgsubstate before committing a memctx (issue5677) This is a minimal copy of localrepo.commit(). As the current amend() function heavily depends on the wctx API, it wasn't easy to port it to use a separate status tuple. So for now, wctx._status is updated in-place.

subrepo: extract preprocess of repo.commit() to free function No code change other than extracting a function. Maybe we should stop mutating the status argument, but that's out of the scope of stable changes.

tests: demonstrate that .hgsubstate isn't updated on amend Loosely based on test-amend.t and test-subrepo.t

amend: do not drop missing files (issue5732) samefile() can be wrong since wctx.manifest() does not include missing files, whereas missing files should be preserved on commit.

amend: do not take untracked files as modified or clean (issue5732) fctx.exists() shouldn't be used here as it checks if a file physically exists, which may disagree with the dirstate.

tests: add more complete test for status changes on amend This demonstrates that missing/untracked files are handled incorrectly. The correct outputs are suppressed by (false !), and wrong outputs are added with (true !) instead. The setup code is copied from test-status-rev.t.

scmutil: don't try to delete origbackup symlinks to directories (issue5731) When origbackuppath is set, when looking to see if a file we are backing up conflicts with a directory in the origbackuppath, we incorrectly match on symlinks to directories. This means we try to call vfs.rmtree on the symlink, which fails. Differential Revision: https://phab.mercurial-scm.org/D1311

tests: add a test demonstrating issue5731 If origbackups are in use, a symlink to a valid directory is backed up, and an update is made that attempts to backup a file or link over that symlink, we abort with a bad error message instead of successfully updating. Differential Revision: https://phab.mercurial-scm.org/D1310

Added signature for changeset cabc840ffdee

Added tag 4.4.1 for changeset cabc840ffdee

stable: merge with security patches

subrepo: use per-type config options to enable subrepos We change subrepos.allowed from a list of allowed subrepo types to a combination of a master switch and per-type boolean flag. If the master switch is set, subrepos can be disabled wholesale. If subrepos are globally enabled, then per-type options are consulted. Mercurial repos are enabled by default. Everything else is disabled by default.

config: add some more documentation around why svn and git subrepos are off

subrepo: disable git and svn subrepos by default (BC) (SEC) We have a security issue with git subrepos. I'm not sure if svn subrepo is vulnerable, but it seems not 100% safe to allow writing arbitrary data into a metadata directory. So for now, only hg subrepo is enabled by default. Maybe we should improve the help to describe why git/svn subrepos are disabled.

subrepo: extend config option to disable subrepos by type (SEC) This allows us to minimize the behavior change introduced by the next patch. I have no idea which config style is preferred in UX POV, but I decided to get things done. a) list: 'allowed = hg, git, svn' b) sub option: 'allowed.hg = True' or 'allowed:hg = True' c) per-type action: 'hg = allow', 'git = abort'

subrepo: add config option to reject any subrepo operations (SEC) This is an alternative workaround for the issue5730. Perhaps this is the simplest way of disabling subrepo operations. It does nothing clever, but just aborts if Mercurial starts accessing to a subrepo. I think Greg's patch is more useful since it allows us to at least check out the parent repository. However, that would be confusing if the default is flipped to checkout=False and subrepos are silently ignored. I don't like the config name 'allowed', but I couldn't get any better name.

subrepo: disallow symlink traversal across subrepo mount point (SEC) It wasn't easy to extend the pathauditor to check symlink traversal across subrepos because pathauditor._checkfs() rejects a directory having ".hg" directory. That's why I added the explicit islink() check. No idea if this patch is necessary after we've fixed the issue5730 by splitting submerge() into planning and execution phases.

tests: show symlink traversal across subrepo mount point (SEC) Also adds a couple of tests where the auditor does work as expected.

share: move config item declarations into core These config items control share behavior that is implemented in core. Since the functionality is implemented in core, extensions may leverage it. Mozilla has one such extension. And, it needs to access share.pool. Before this patch, a devel warning regarding accessing an unregistered config option would be issued unless the share extension were loaded. Moving the registration of the config options to core fixes this.

morestatus: don't crash with different drive letters for repo.root and CWD Previously, if there were unresolved files and the CWD drive was different from the repo drive, `hg status -v` would page the normal status, followed by the exception header. A stacktrace was waiting when the pager exited. The underlying cause was the same as f445b10dc7fb. Unfortunately, I don't see any reasonable way to write a test this [1]. [1] https://www.mercurial-scm.org/pipermail/mercurial-devel/2017-November/107401.html

pathutil: add doctests for canonpath() This is a followup to f445b10dc7fb. Since there's no way to ensure that more drive letters than C: exist, this seems like the only way to test it. This is enough to catch the f445b10dc7fb scenario, as well as CWD outside of the repo when the path isn't prefixed with path/to/repo.

share: handle --relative shares to a different drive letter gracefully This had the same problem as f445b10dc7fb. Banning os.path.relpath() is tempting, but the hint it provides is useful here.

pathutil: use util.pathto() to calculate relative cwd in canonpath() os.path.relpath() exploded if the 'root' and 'cwd' directories had different drive letters. I noticed this in TortoiseHg when typing a fileset into the filter, and it kept complaining until the closing '()' was typed. This was reproducible on the command line with: $ cd /d $ hg -R /c/Users/Matt/Projects/hg files 'set:e' Traceback (most recent call last): ... File "mercurial\pathutil.pyc", line 182, in canonpath File "ntpath.pyc", line 529, in relpath ValueError: path is on drive c:, start on drive d:

Added signature for changeset 0ccb43d4cf01