run-tests: pass --with-hg to run-tests.py command used by bisect This makes `run-tests.py -l test-run-tests.t` 23 seconds faster on my laptop. Inside the test, `$ rt --known-good-rev=0 test-bisect.t` took 24.9 seconds before, and 1.2 seconds after. Differential Revision: https://phab.mercurial-scm.org/D576

import-checker: allow relative import a module being checked This would make the checker more friendly for 3rd-party code. For example, In remotefilelog/x.py, it may have: from . import shallowutils That could trigger "relative import of stdlib module" if "remotefilelog" was installed in the system. If the module being checked conflicts with the system module, it makes sense to not treat that module as system module. This patch makes it so. Differential Revision: https://phab.mercurial-scm.org/D552

merge: move some of the logic in batchget() to workingfilectx We will use this logic in two places with in-memory merge. Differential Revision: https://phab.mercurial-scm.org/D444

filemerge: add _restorebackup Differential Revision: https://phab.mercurial-scm.org/D404

filemerge: reduce creation of tempfiles until needed This restricts the creation of temporary files to just `_xmerge`, when we call an external tool. Differential Revision: https://phab.mercurial-scm.org/D403

filemerge: add `_workingpath` This reduces any reliance on `a`. Differential Revision: https://phab.mercurial-scm.org/D401

filemerge: move a util copy call to filectx.write This way a future in-memory-merge context can intercept them. Differential Revision: https://phab.mercurial-scm.org/D400

filemerge: eliminate most uses of tempfiles Emphasize that they're unused so we can more easily remove them later. Differential Revision: https://phab.mercurial-scm.org/D399

filemerge: extract _maketemp and _makebackup These functions will be modified by in-memory merge, so let's extract them first and add some comments. This also shortens `_filemerge` a bit. Differential Revision: https://phab.mercurial-scm.org/D388

encoding: check overflow while calculating size of JSON escape buffer The minimum input size to exploit is ~682MB (= INT_MAX / len('\\u0000') * 2) on 32bit system, which isn't easy to achieve using Python str in 2GB process address space, but probably doable.

editor: use an unambiguous path suffix for editor files Changes the API of `ui.edit()` to take an optional `action` argument, which is used when constructing the suffix of the temp file. Previously, it was possible to set the suffix by specifying a `suffix` to the optional `extra` dict that was passed to `ui.edit()`, but the goal is to drop support for `extra.suffix` and make `action` a required argument. To this end, `ui.edit()` now yields a `develwarn()` if `action` is not set or if `extra.suffix` is set. I updated all calls to `ui.edit()` I could find in `hg-crew` to specify the appropriate `action`. This means that when creating a commit, instead of the path to the editor file being something like: `/tmp/hg-editor-XXXXXX.txt` it is now something like: `/tmp/hg-editor-XXXXXX.commit.hg.txt` Some editors (such as Atom) make it possible to statically define a [TextMate] grammar for files with a particular suffix. For example, because Git reliably uses `.git/COMMIT_EDITMSG` and `.git/MERGE_MSG` as the paths for commit-type messages, it is trivial to define a grammar that is applied when files of either name are opened in Atom: https://github.com/atom/language-git/blob/v0.19.1/grammars/git%20commit%20message.cson#L4-L5 Because Hg historically used a generic `.txt` suffix, it was much harder to disambiguate whether a file was an arbitrary text file as opposed to one created for the specific purpose of authoring an Hg commit message. This also makes it easier to add special support for `histedit`, as it has its own suffix that is distinct from a commit: `/tmp/hg-histedit-XXXXXX.histedit.hg.txt` Test Plan: Added an integration test: `test-editor-filename.t`. Manually tested: ran `hg ci --amend` for this change and saw that it used `/tmp/hg-editor-ZZjcz0.commit.hg.txt` as the path instead of `/tmp/hg-editor-ZZjcz0.txt` as the path. Verified `make tests` passes. Differential Revision: https://phab.mercurial-scm.org/D464

revlog: move check for wdir from changelog to revlog Yuya said he preferred this (to keep them in one place, I think). Differential Revision: https://phab.mercurial-scm.org/D569

debugssl: allow a URL to be specified without a local repository This was the original intent, but I bungled the logic. Otherwise if there is a certificate chain issue, the repository can't be cloned in order for there to be a repo object. I think I missed this case because I was inside of a Mercurial clone as I was originally developing and testing this.

revlog: use pycompat.bytestr() to reliably have a %s-able value

debugcommands: stabilize output of debugbundle by having a custom repr We handle all dict-like things the same, and don't worry about it actually being a repr.

python3: whitelist another 5 passing tests found with the ratchet script

python3: replace sorted(<dict>.iterkeys()) with sorted(<dict>)

python3: wrap all uses of <exception>.strerror with strtolocal Our string literals are bytes, and we mostly want to %-format a strerror into a one of those literals, so this fixes a ton of issues.

pager: do not start pager if `ui` has been `pushbuffer`-ed The `pushbuffer`, `popbuffer` APIs are intended to capture internal output. They will prevent `ui.write` from writing to the actual `ui.fout`. So a pager won't receive the output and do the right thing. In general, it does not make sense to start a pager if ui is in the "pushbuffer" mode. Differential Revision: https://phab.mercurial-scm.org/D574

revset: do not flip "and" arguments when optimizing Rewrite `flipand(y, x)` to `andsmally(x, y)` so the AST order is unchanged, which could be more friendly to developers. Differential Revision: https://phab.mercurial-scm.org/D579

revset: make match function follow given subset if specified (API) This should be sensible default since mfunc(subset) is roughly equivalent to 'subset & mfunc'. The order argument is still there so we can specify 'anyorder' if the order doesn't really matter.

revset: move order argument to run-time match function We no longer need the order flag to build a parsed tree.

revset: fix example describing how ordering is determined It was 'X & !Y' before.

revset: move order constants from revsetlang Thanks to the recent refactor, the ordering rule is fully processed at runtime.

tag: use filtered repo when creating new tags (issue5539) When pruning a changeset that added a tag and then adding another tag, the "pruned" tag gets restored. This is because the tag creation step (tags._tag() call in tags.tag()) is currently done on the unfiltered repo. This behavior has been there from 7977d35df13b which backs out b08af8f0ac01 with no clear reason but caution on unthought situations at that time. In this changeset, we pass the filtered repo to tags._tag(), preventing "pruned" tags to reappear. This somehow restores b08af8f0ac01, though now we arguably have a valid use case for.

extensions: add wrappedfunction() context manager Several extensions exist that temporarily want to wrap a function (at least narrowhg, any many of the extensions in hg-experimental). That's why we have the unwrapfunction() that was introduced in 19578bb84731 (extensions: add unwrapfunction to undo wrapfunction, 2016-08-10). This patch adds a simple wrappedfunction() that returns a context manager. Differential Revision: https://phab.mercurial-scm.org/D472

mdiff: add a --ignore-space-at-eol option Add an option that only ignores whitespaces at EOL. The name of the option is the same as Git. .. feature:: Added `--ignore-space-at-eol` diff option to ignore whitespace differences at line endings. Differential Revision: https://phab.mercurial-scm.org/D422

revset: improve documentation about ordering handling The old documentation is a bit confusing. Namely, it's unclear whether `define` means "I should ALWAYS define a new order", or "I should SOMETIMES define a new order", and if it's the latter, what's the difference between `define` and `any`? This patch clarifies that and adds more examples. Differential Revision: https://phab.mercurial-scm.org/D523

revset: remove order information from tree (API) Keeping `order` in tree makes AST operation harder. And there could be invalid cases if trees could be generated and compounded freely, like: SetA(order=define) & SetB(order=define) ^^^^^^ couldn't be satisfied This patch changes the code to calculate order on the fly, during tree traversal. Optimization of reordering `and` arguments is preserved by introducing a new internal operation `flipand`. .. api:: revset.stringset() now takes 'order' as the last argument. Differential Revision: https://phab.mercurial-scm.org/D451

revset: drop optimization about reordering "or" set elements The reordering optimization is more important for "and" than "or", given the implementation details about "addset" and "filteredset" - reordering "or" may help "__contains__" test but not iteration, reordering "and" could help both. We are going to simplify the tree to remove ordering information. Removing "or" reordering optimization would make things simpler. This effectively reverts c63cb2d10d6d. It tracks back to the "orset" function added by the initial commit of revset (c9ce8ecd6). In the future, we might consider optimization at runtime (ex. do reordering and rewrites inside "orset"). Differential Revision: https://phab.mercurial-scm.org/D561

rebase: use _ctxdesc in one more place This simplifies the code. Differential Revision: https://phab.mercurial-scm.org/D566

rebase: move working parent and bookmark for obsoleted revs (BC) Previously, obsoleted revs with successors in destination are completely ignored. That caused some inconvenience when working copy is obsoleted. Most commands avoid working copy being obsoleted, but `hg pull` is an exception. This patch makes rebase able to move bookmarks or working parent for those obsoleted revs. It does so by keeping the obsoleted revs in `state` and marking them as "skipped, rebased to desired destination" during run-time. This reverts part of the behavior change of 3b7cb3d17137 and D24. Differential Revision: https://phab.mercurial-scm.org/D527

rebase: remove an unnecessary adjustdest in clearrebased `rev` being "skipped" could currently be caused by moving `rev` does not create a new commit. In this case, `state[rev]` is already changed to `p1`, and is a sane destination for bookmark or working parent movement. Therefore an additional destination adjustment is unnecessary. Differential Revision: https://phab.mercurial-scm.org/D565

rebase: sort destmap topologically Previously rebase source and destination could not overlap. But with the multi-destination support, source and destination could reasonably partially overlap. That requires another topological sort on `{sourcerev: destrev}` graph (destmap). This patch implements that. If a revision's destination is itself, the error message gets changed from "source is ancestor of destination" to "source and destination form a cycle". Not marking as BC since automation should depend on exit code, not error message. Differential Revision: https://phab.mercurial-scm.org/D470

rebase: initial support for multiple destinations This patch defines `SRC` (a single source revision) and `ALLSRC` (all source revisions) to be valid names in `--dest` revset if `--src` or `--rev` is used. So destination could be defined differently according to source revisions. The names are capitalized to make it clear they are "dynamically defined", distinguishable from normal revsets (Thanks Augie for the suggestion). This is useful, for example, `-r 'orphan()' -d 'calc-dest(SRC)'` to solve instability, which seems to be a highly wanted feature. The feature is not completed, namely if `-d` overlaps with `-r`, things could go wrong. A later patch will handle that case. The feature is also gated by `experimental.rebase.multidest` config option which is default off. Differential Revision: https://phab.mercurial-scm.org/D469

rebase: change internal format to support destination map A later patch will add multiple destination support. This patch changes internal state and the rebase state file format to support that. But the external interface still only supports single destination. A test was added to make sure rebase still supports legacy state file. The new state file is incompatible with old clients. We had done similar state file format change before: 5eac7ab, 92409f8, and 72412af. The state file is transient, so the impact of incompatibility is limited. Besides, the old client won't support multiple destinations anyway so it does not really make sense to make the file format compatible with them. Differential Revision: https://phab.mercurial-scm.org/D348

rebase: rewrite _computeobsoletenotrebased The old code stores successors of all related nodes together, which works fine if destination is unique. A future patch would make destination non-unique so let's change the implementation to test successors for rebaseset separately. Differential Revision: https://phab.mercurial-scm.org/D347

util: use ~ as a suffix for a temp file in the same directory as a source file Tools like Buck have patterns to ignore the creation of files (in the working copy) that match certain patterns: https://github.com/facebook/buck/blob/39278a4f0701c5239eae148968dc1ed4cc8661f7/src/com/facebook/buck/cli/Main.java#L259-L299 When Buck sees a new source file (as reported by Watchman), it has to invalidate a number of caches associated with the directory that contains the file. Using a standard suffix, such as `~`, would make it easier for Buck and others to filter out these types of file creation events. The other uses of `tempfile.mkstemp()` in Hg do not appear to be problematic because they (generally speaking) do not specify the `dir` parameter, so the new file is created in the system-appropriate temp directory, which is outside the working copy. Test Plan: `make tests` Differential Revision: https://phab.mercurial-scm.org/D468

morestatus: simplify check for unresolved merge conflicts Differential Revision: https://phab.mercurial-scm.org/D546

tests: rename test-terse-status.t to test-status-terse.t When looking for status tests, most people would probably look for "test-status*", so it would be nice if they could find it there. This also let's them run (most) status tests with "run-tests.py test-status*". Differential Revision: https://phab.mercurial-scm.org/D547

metadataonlyctx: don't crash when reusing the manifest with deletions This was originally fixed by Mateusz Kwapich for the `metaedit` command in fb-hgext with a test for the `metaedit` command. It didn't get upstreamed because `metaedit` was not in core. This patch fixes the crash and adds a test about `metadataonlyctx` to avoid future regressions. Differential Revision: https://phab.mercurial-scm.org/D550

context: make parents and text optional in metadataonlyctx The metadataonlyctx is to copy an existing context with some minor metadata changes. If the caller only wants to change "extra", or "user", ideally it does not have to read and pass "parents" and "text" information. This patch makes "parents" and "text" optionally to convenient callers. Differential Revision: https://phab.mercurial-scm.org/D548

test: add more obsmarker tests for pruning scenarios The obsfate output in cases of pruning is not ideal right now, add some tests so have these scenarios around.

template: better prune support in obsfate successorssets don't returns good results for pruned commit, add a workaround for simple cases. A proper fix would require a large rework of successorssets algorithm, I will send a separate series for this refactoring.

template: compute dates in obsfatedate Extract the dates from obsmarkers. Compute the min and max date from the obsmarker range list.

template: compute user in obsfateusers Extract, deduplicate users informations from obs markers in order to display them. Print all users for the moment, we might want to display users only in verbose mode later.

template: compute verb in obsfateverb Add a template function obsfateverb which use the markers information to compute a better obsfate verb. The current logic behind the obsfate verb is simple for the moment: - If the successorsets is empty, the changeset has been pruned, for example: Obsfate: pruned - If the successorsets length is 1, the changeset has been rewritten without divergence, for example: Obsfate: rewritten as 2:337fec4d2edc, 3:f257fde29c7a - If the successorsets length is more than 1, the changeset has diverged, for example: Obsfate: split as 2:337fec4d2edc, 3:f257fde29c7a As the divergence might occurs on a subset of successors, we might see some successors twice: Obsfate: split as 9:0b997eb7ceee, 5:dd800401bd8c, 10:eceed8f98ffc; split as 8:b18bc8331526, 5:dd800401bd8c, 10:eceed8f98ffc

contrib: add test-check-module-imports.t to the Python 3 whitelist This has the benefit of also catching most (if not all!) old-style print statements and except statements.

tests: update test-obsolete to pass our import checker

tests: update test-largefiles-cache to pass our import checker

tests: update test-inherit-mode to pass our import checker

tests: update test-relink to pass our import checker

tests: update test-strip to pass our import checker

tests: update test-share to pass our import checker

tests: update test-requires to pass our import checker

tests: update test-progress to pass our import checker

tests: update test-patchbomb to pass our import checker

merge with stable

dirstate: perform transactions with _copymap using single call, where possible This replaces patterns such as this: ``` if f in self._copymap: del self._copymap[f] ``` with this: ``` self._copymap.pop(f, None) ``` Although eliminating the extra lookup/call may be a negligible performance win in the standard dirstate, alternative implementations, such as [sqldirstate](https://bitbucket.org/facebook/hg-experimental/src/default/sqldirstate/) may see a bigger win where each of these calls results in an RPC, so the savings is greater. Test Plan: `make tests` Differential Revision: https://phab.mercurial-scm.org/D493

pull: do not prompt "hg update" if update.requiredest is set Previously, after pull, we show: (run 'hg update' to get a working copy) unconditionally. People might run `hg update` and get an exception if `update.requiredest` is set, and get a bit frustrated. This patch changes the code to not prompt `hg update` in that case. Differential Revision: https://phab.mercurial-scm.org/D516