dispatch: add option to not strip command args parsed by _earlygetopt() This allows us to parse the original args later by full-blown getopt() in order to verify the result of the faulty early parsing. Still we need the 'strip=True' behavior for shell aliases. Note that this series is RFC because it seems to change too much to be included in stable release.

dispatch: fix early parsing of short option with value like -R=foo Before, -R=foo was parsed as '-R' 'foo', which disagrees with the standard getopt behavior.

dispatch: abort if early boolean options can't be parsed Perhaps we'll need to restrict the parsing rules of --debugger and --profile, where this patch will help us know why the --debugger option doesn't work. I have another series to extend this feature to --config/--cwd/-R, but even with that, shell aliases can be used to get around the restriction.

dispatch: stop parsing of early boolean option at "--"

dispatch: extract stub function to peek boolean command option We should at least stop parsing at "--". The 'name' argument is passed for future extension.

dispatch: do not drop unpaired argument at _earlygetopt() Before, "hg log -R" just worked.

amend: update .hgsubstate before committing a memctx (issue5677) This is a minimal copy of localrepo.commit(). As the current amend() function heavily depends on the wctx API, it wasn't easy to port it to use a separate status tuple. So for now, wctx._status is updated in-place.

subrepo: extract preprocess of repo.commit() to free function No code change other than extracting a function. Maybe we should stop mutating the status argument, but that's out of the scope of stable changes.

tests: demonstrate that .hgsubstate isn't updated on amend Loosely based on test-amend.t and test-subrepo.t

amend: do not drop missing files (issue5732) samefile() can be wrong since wctx.manifest() does not include missing files, whereas missing files should be preserved on commit.

amend: do not take untracked files as modified or clean (issue5732) fctx.exists() shouldn't be used here as it checks if a file physically exists, which may disagree with the dirstate.

tests: add more complete test for status changes on amend This demonstrates that missing/untracked files are handled incorrectly. The correct outputs are suppressed by (false !), and wrong outputs are added with (true !) instead. The setup code is copied from test-status-rev.t.

scmutil: don't try to delete origbackup symlinks to directories (issue5731) When origbackuppath is set, when looking to see if a file we are backing up conflicts with a directory in the origbackuppath, we incorrectly match on symlinks to directories. This means we try to call vfs.rmtree on the symlink, which fails. Differential Revision: https://phab.mercurial-scm.org/D1311

tests: add a test demonstrating issue5731 If origbackups are in use, a symlink to a valid directory is backed up, and an update is made that attempts to backup a file or link over that symlink, we abort with a bad error message instead of successfully updating. Differential Revision: https://phab.mercurial-scm.org/D1310

Added signature for changeset cabc840ffdee

Added tag 4.4.1 for changeset cabc840ffdee

stable: merge with security patches

subrepo: use per-type config options to enable subrepos We change subrepos.allowed from a list of allowed subrepo types to a combination of a master switch and per-type boolean flag. If the master switch is set, subrepos can be disabled wholesale. If subrepos are globally enabled, then per-type options are consulted. Mercurial repos are enabled by default. Everything else is disabled by default.

config: add some more documentation around why svn and git subrepos are off

subrepo: disable git and svn subrepos by default (BC) (SEC) We have a security issue with git subrepos. I'm not sure if svn subrepo is vulnerable, but it seems not 100% safe to allow writing arbitrary data into a metadata directory. So for now, only hg subrepo is enabled by default. Maybe we should improve the help to describe why git/svn subrepos are disabled.

subrepo: extend config option to disable subrepos by type (SEC) This allows us to minimize the behavior change introduced by the next patch. I have no idea which config style is preferred in UX POV, but I decided to get things done. a) list: 'allowed = hg, git, svn' b) sub option: 'allowed.hg = True' or 'allowed:hg = True' c) per-type action: 'hg = allow', 'git = abort'

subrepo: add config option to reject any subrepo operations (SEC) This is an alternative workaround for the issue5730. Perhaps this is the simplest way of disabling subrepo operations. It does nothing clever, but just aborts if Mercurial starts accessing to a subrepo. I think Greg's patch is more useful since it allows us to at least check out the parent repository. However, that would be confusing if the default is flipped to checkout=False and subrepos are silently ignored. I don't like the config name 'allowed', but I couldn't get any better name.

subrepo: disallow symlink traversal across subrepo mount point (SEC) It wasn't easy to extend the pathauditor to check symlink traversal across subrepos because pathauditor._checkfs() rejects a directory having ".hg" directory. That's why I added the explicit islink() check. No idea if this patch is necessary after we've fixed the issue5730 by splitting submerge() into planning and execution phases.

tests: show symlink traversal across subrepo mount point (SEC) Also adds a couple of tests where the auditor does work as expected.

share: move config item declarations into core These config items control share behavior that is implemented in core. Since the functionality is implemented in core, extensions may leverage it. Mozilla has one such extension. And, it needs to access share.pool. Before this patch, a devel warning regarding accessing an unregistered config option would be issued unless the share extension were loaded. Moving the registration of the config options to core fixes this.

morestatus: don't crash with different drive letters for repo.root and CWD Previously, if there were unresolved files and the CWD drive was different from the repo drive, `hg status -v` would page the normal status, followed by the exception header. A stacktrace was waiting when the pager exited. The underlying cause was the same as f445b10dc7fb. Unfortunately, I don't see any reasonable way to write a test this [1]. [1] https://www.mercurial-scm.org/pipermail/mercurial-devel/2017-November/107401.html

pathutil: add doctests for canonpath() This is a followup to f445b10dc7fb. Since there's no way to ensure that more drive letters than C: exist, this seems like the only way to test it. This is enough to catch the f445b10dc7fb scenario, as well as CWD outside of the repo when the path isn't prefixed with path/to/repo.

share: handle --relative shares to a different drive letter gracefully This had the same problem as f445b10dc7fb. Banning os.path.relpath() is tempting, but the hint it provides is useful here.

pathutil: use util.pathto() to calculate relative cwd in canonpath() os.path.relpath() exploded if the 'root' and 'cwd' directories had different drive letters. I noticed this in TortoiseHg when typing a fileset into the filter, and it kept complaining until the closing '()' was typed. This was reproducible on the command line with: $ cd /d $ hg -R /c/Users/Matt/Projects/hg files 'set:e' Traceback (most recent call last): ... File "mercurial\pathutil.pyc", line 182, in canonpath File "ntpath.pyc", line 529, in relpath ValueError: path is on drive c:, start on drive d:

Added signature for changeset 0ccb43d4cf01

Added tag 4.4 for changeset 0ccb43d4cf01

test-dispatch: stabilize the test When cwd is removed and `hg` is executed, some shells may run `getcwd` before forking and executing, some may not do it, some may print a different error message. The test should be shell-independent so let's just avoid checking the error message. Differential Revision: https://phab.mercurial-scm.org/D1282

internals: update test-help.t for config registrar copy-edit

internals: copy-edit "register" -> "registrar" in configitem docs

merge with i18n

i18n-pt_BR: synchronized with cab34bda259e

help: minor copy editing for grammar

configitems: relax warning about unwanted default value The original condition was a bit harsh for extension authors since third-party extensions need to preserve compatibility with older Mercurial versions, where no defaults would be loaded from the configtable. So let's silence the warning if the given default value matches, which should be harmless.

filemerge: pass a default value to _toolstr (issue5718) After a refactoring, _toolstr stopped having default="" as one of it's args, therefore when called without a default it returns None and not "". This causes concatenation to fail.

children: fix the log expansion of `hg children` in doc `hg log -r children()` returns `hg: parse error: missing argument`. Differential Revision: https://phab.mercurial-scm.org/D1269

test-static-http: flush access log per request It appears that stderr is fully buffered on Windows. # no-check-commit because of log_message() function

statichttprepo: do not use platform path separator to build a URL It wouldn't work between Windows client and Unix server.

merge: disable path conflict checking by default (issue5716) We shouldn't ship a severe perf regression in hg update for 4.4. Differential Revision: https://phab.mercurial-scm.org/D1223

merge: add a config option to disable path conflict checking We've found a severe perf regression in `hg update` caused by the path conflict checking code. The next patch will disable this by default. Differential Revision: https://phab.mercurial-scm.org/D1222

dirstate: clean up when restoring identical backups When a dirstate backup is restored, it is possible that no actual changes to the dirstate have been made. In this case, the backup is still a hardlink to the original dirstate. Unfortunately, `os.rename` silently fails (nothing happens, and no error occurs) when `src` and `dst` are hardlinks to the same file. As a result, the backup is left lying around. Over time, these files accumulate. When restoring dirstate backups, check if the backup and the dirstate are the same file, and if so, just delete the backup. Differential Revision: https://phab.mercurial-scm.org/D1201

tests: add a test demonstrating failure to clean up dirstate backups Differential Revision: https://phab.mercurial-scm.org/D1200

tests: adjust hooks for Windows I'm not sure why these weren't working on Windows. The failures were generally in the style of: - remote: phase-move: cb9a9f314b8b07ba71012fcdbc544b5a4d82ff5b: 1 -> 0 + remote: "phase-move: $HG_NODE: $HG_OLDPHASE -> $HG_PHASE" and - abort: pretxnclose-bookmark.force-forward hook exited with status 1 - [255] + abort: pretxnclose-bookmark.force-public hook exited with status 255 + [255] These failures originated in ee5f0d047b41::f6d17075608f.

test-log-linerange: open binary file in binary mode The '\n' characters were being translated on Windows, throwing off hg and git hashes, as well as the file content in the diff.

test-arbitraryfilectx: stabilize for Windows Previously, the second last test (context.arbitraryfilectx(..)) returned True on Windows. I changed the repo setup sequence to import a patch, so that way the repo would have a proper symlink. That made the last test fail, since it is comparing files in wdir(), one of which is not the expected symlink. Apparently the (feature !) line matching doesn't work well with (no-eol), so I had to conditionalize the test instead of the output.

dirstate: remove excess attribute lookups for dirstate.status (issue5714) A recent refactor added a layer of abstraction to the dirstate which makes doing things like 'foo in dirstate' now require some extra Python attribute lookups. This is causing a 100ms slow down in hg status for mozilla-central. The fix is to hoist the inner dict's functions onto the main class once the lazy loading it complete, as well as store the actual functions before doing the status loop (as is done for other such functions). In my testing, it seems to address the performance regression, but we'll need to see the perf run results to know for sure. Differential Revision: https://phab.mercurial-scm.org/D1257

dirstate: avoid reading the map when possible (issue5713) (issue5717) Before the recent refactor, we would not load the entire map until it was accessed. As part of the refactor, that got lost and even just trying to load the dirstate parents would load the whole map. This caused a perf regression (issue5713) and a regression with static http serving (issue5717). Making it lazy loaded again fixes both. Differential Revision: https://phab.mercurial-scm.org/D1253

dirstate: move clear onto dirstatemap class A future diff will move the lazy loading aspect of dirstate to the dirstatemap class. This means it requires a slightly different strategy of clearing than just reinstantiating the object (since just reinstantiating the object will lazily load the on disk data again later instead of remaining permanently empty). So let's give it it's own clear function. Differential Revision: https://phab.mercurial-scm.org/D1252

internal-doc: document the config register mechanism This explains the various usage and feature of the config register introduced in Mercurial 4.3 and 4.4.

help: clarify the pre-txnclose-phase documentation Gregory Szorc requested some clarification.

internal-doc: document the 'phases' parameters to 'getbundle' The getbundle wireprotocol method has some extended documentation. We update it with the next parameters introduced for binary phases.

status: update the help to indicate that clean files are not normally tersed The same applies to '?' if --quiet is used (or any of the other states if some of -marduic is specified), but I couldn't figure out how to express that clearly.

terse: split on repo separator instead of os.sep (issue5715) The paths being processed are from scmutil.status, and therefore normalized to '/' separators.

makefile: put format-c into .PHONY targets

statichttprepo: prevent loading dirstate over HTTP on node lookup (issue5717) This seems a bit hacky, but works well. There should be no reason that static-http repo had to load dirstate. Initially I tried to proxy os.stat() call through vfs so that statichttpvfs could hook it, but there wasn't a good error value which the statichttpvfs could return to get around the util.filestat issue.

test-static-http: show all files accessed over HTTP This provides some confidence on files that should be loadable over HTTP. Hopefully it will prevent future bugs.