tests: make chunk header of external diff glob-ed for portability Before this patch, some tests using external "diff" command via extdiff extension fail on Solaris, because system standard "diff" (= /usr/bin/diff) on Solaris always formats chunk header in the style below: @@ -X.x +Y.y @@ even though "diff" on Linux sometimes omits ".x" and/or ".y" in it. This patch makes chunk header of external diff glob-ed for portability of tests, and adds check-code.py rules to detect such diff output in tests. This patch also changes "hg diff" output in test-subrepo-git to simplify detection rules, even though it is certainly portable because these lines are generated by "git" command. This patch is a part of making tests using external "diff" portable, and tests below aren't yet portable even after this patch. test-largefiles-update.t test-subrepo-deep-nested-change.t

tests: make timezone in diff output glob-ed for portability Before this patch, some tests using external "diff" command via extdiff extension fail on Solaris, because system standard "diff" (= /usr/bin/diff) on Solaris doesn't display timezone for timestamp of each files in diff output. This patch makes timezone in external diff output glob-ed for portability of tests, and adds check-code.py a rule to detect such

tests: omit -p for external diff via extdiff extension for portability Before this patch, some tests using external "diff" command via extdiff extension fail on Solaris, because "-p" (show which C function each change is in) option isn't supported by system standard "diff" on Solaris, even though extdiff passes it to external "diff" by default. Fortunately, this non-portable option isn't important for (current, at least) tests using external "diff" command via extdiff extension. This patch omits "-p" for external "diff" command via extdiff extension for portability of tests, and adds check-code.py a rule to detect invocation of "diff" with "-p". Newly added check-code.py rule examines only lines generated by external "diff" with "-r", because strict examination might misidentify "hg diff -p" or other complicated lines consisting of "diff" string as wrong one. This patch is a part of making tests using external "diff" portable, and tests below aren't yet portable even after this patch. test-graft.t test-largefiles-update.t test-subrepo-deep-nested-change.t

update: check command line before modifying repo A failed command should not have any effect on the repo.

treemanifest: fix debugrebuildfncache When I taught debugrebuildfncache about dirlogs in fb92927f9775 (treemanifests: fix streaming clone, 2016-02-04), I added a last-minute "if 'treemanifest' in repo" guard. That should have been checking for "... in repo.requirements". Fix that and add tests for it.

update: warn about other topological head in pull and unbundle Other commands have a '--update' triggering a bare update. We now issue the message introduced into the previous changeset for these too.

update: warn about other topological heads on bare update A concern around the user experience of Mercurial is user getting stuck on there own topological branch forever. For example, someone pulling another topological branch, missing that message in pull asking them to merge and getting stuck on there own local branch. The current way to "address" this concern was for bare 'hg update' to target the tipmost (also latest pulled) changesets and complain when the update was not linear. That way, failure to merge newly pulled changesets would result in some kind of failure. Yet the failure was quite obscure, not working in all cases (eg: commit right after pull) and the behavior was very impractical in the common case (eg: issue4673). To be able to change that behavior, we need to provide other ways to alert a user stucks on one of many topological head. We do so with an extra message after bare update: 1 other heads for branch "default" Bookmark get its own special version: 1 other divergent bookmarks for "foobar" There is significant room to improve the message itself, and we should augment it with hint about how to see theses other heads or handle the situation (see in-line comment). But having "a" message is already a significant improvement compared to the existing situation. Once we have it we can iterate on a better version of it. As having such message is an important step toward changing the default destination for update and other nicety, I would like to move forward quickly on getting such message. This was discussed during London - October 2015 Sprint.

tests: mock getpid to reduce glob usage With util.getpid, it is now possible to define fixed pids. Future iterations can define a map of pids on a locked first come first serve basis to create a more realistic harness, but for now this is good enough. This applies to blackbox, but could apply to other tests as well.

util: enable getpid to be replaced This will enable tests to write stable process ids.

blackbox: refactor use of vfs as _bbvfs

blackbox: flush output file descriptor Without this, when there are multiple ui views, each blackbox will have its own file handle, and the logging will be in a really bad order. Also, because of the way blackbox works, it never closes its file handles, which means the last output before exit is often lost.

tests: change blackbox test to work cross platform While it is not easy to make a file 000 on Windows, you can emulate most of the behaviors by replacing the file with a directory. Also corrects test description to properly indicate that failing to read from the log is fatal.

merge: document checkignored and checkunknown configs again These options were undocumented for 3.7 because of an issue found during the freeze (see rev 7cb7264cfd52). This issue has now been fixed, so we can document these options again.

rebase: respect checkunknown and checkignored in more cases checkunknown and checkignored are currently respected for updates and regular merges, but not for certain kinds of rebases. To be precise, they aren't respected for rebases when: (1) we're rebasing while currently on the destination commit, and (2) an untracked or ignored file F is currently in the working copy, and (3) the same file F is in a source commit, and (4) F has different contents in the source commit. This happens because rebases set force to True when calling merge.update. Setting force to True makes a lot of sense in general, but it turns out the force option is overloaded: there's a deprecated '--force' option in merge that allows you to merge in outstanding changes, including changes in untracked files. We use the 'mergeforce' parameter to tell those two cases apart. I think the behavior during rebases when checkunknown is 'abort' (the default) is wrong -- we should abort on or overwrite differing untracked files, not try to merge them in. However that currently breaks rebases by aborting in the middle -- we need better handling for that case before we can change the default.

test-merge-force: add tests for merge.checkunknown=warn In an upcoming patch we're going to change the behavior of some merges with merge.checkunknown=warn or ignore -- ensure that the behavior of the deprecated 'merge --force' remains the same.

merge: tell _checkunknownfiles about whether this was merge --force In an upcoming patch we'll have different behavior here for when 'merge --force' is used as opposed to when other kinds of force operations are performed, like rebases.

merge: add missing doc for 'labels' parameter

merge: move abort/warn checks up to the top level of _checkunknownfiles In upcoming patches, we're also going to do these checks when force is True.

match: rename "narrowmatcher" to "subdirmatcher" (API) I keep mistaking "narrowmatcher" for narrowhg's narrowmatcher. "subdirmatcher" seems more to the point anyway.

tests: relax test-devel-warnings to reduce false positives This test is interested in warning output, so glob away line numbers and hashes as they aren't relevant to its core.

merge with stable

chgserver: create new process group after fork (issue5051) This is to make SIGTSTP work. Before the patch, the server process group is considered "orphaned" and will ignore SIGTSTP, SIGTTIN, SIGTTOU, according to POSIX. See the comment above `will_become_orphaned_pgrp` in `kernel/exit.c` from Linux 4.3 for details. SIGTSTP is important if chgserver runs some ncurses commend like `commit -i`. Ncurses has its own SIGTSTP handler which will do the following: 1. Clean the screen 2. Stop itself by resending SIGTSTP to itself 3. Restore the screen If SIGTSTP is ignored, step 2 will be a noop, which means the process cannot be suspended properly. In order to make things work, chg client needs to forward SIGTSTP and SIGCONT to server as well.

check-commit: check for double-addition of blank lines Previously, we were only checking for a blank line being added next to an existing one. Now we also check for two being added at the same time.

check-commit: scan for multiple instances of error patterns

merge: add file ancestor linknode to mergestate During a merge, each file has a current commitnode+filenode, an other commitnode+filenode, and an ancestor commitnode+filenode. The ancestor commitnode is not stored though, and we rely on the ability for the filectx() to look up the commitnode by using the filenode's linkrev. In alternative backends (like remotefilelog), linkrevs may have restriction that prevent arbitrary linkrev look up given a filenode. This patch accounts for that by storing the ancestor commitnode in the merge state so that it is available later at resolve time. This results in some test changes because the ancestor commitnode we're using at resolve time changes slightly. Before, we used the linkrev commit, which is the earliest commit that introduced that particular filenode (which may not be the latest common ancestor of the commits being merged). Now we use the latest common ancestor of the merged commits as the commitnode. This is fine though, because that commit contains the same filenode as the linkrev'd commit.

merge: add debugmergestate support for _stateextras Now that we can store extras for each file, we need to have support for showing it in debugmergestate (the tests depend on this).

merge: add state extras merge state data In future commits we will want to store more data related to each file in the merge state. This patch adds an optional record for storing a dictionary of extras for each file.

revset: use manifest.matches in _follow revset The old _follow revset iterated over every file in the commit and checked if it matched. For repos with large manifests, this could take 500ms. By switching to use manifest.matches() we can take advantage of the fastpaths built in to manifest.py that allows iterating over only the files in the matcher when it's a simple matcher. This brings the time spent down from 500ms to 0ms during simple operations like 'hg log -f file.txt'.

treemanifests: fix streaming clone Similar to the previous patch, the .hg/store/meta/ directory does not get copied when when using "hg clone --uncompressed". Fix by including "meta/" in store.datafiles(). This seems safe to do, as there are only a few users of this method. "hg manifest" already filters the paths by "data/" prefix. The calls from largefiles also seem safe. The use in verify needs updating to prevent it from mistaking dirlogs for orphaned filelogs. That change is included in this patch. Since the dirlogs will now be in the fncache when using fncachestore, let's also update debugrebuildfncache(). That will also allow any existing treemanifest repos to get their dirlogs into the fncache. Also update test-treemanifest.t to use an a directory name that requires dot-encoding and uppercase-encoding so we test that the path encoding works.

treemanifests: fix local clone When doing a local clone with treemanifests, the .hg/store/meta/ directory currently does not get copied. To fix it, all we need to do is to add it to the list of directories to copy.

tests: simplify treemanifest test by backing up entire .hg/store

histedit: limit updated and merging output to important updates Output is retained when: * There's a conflict * User asked to edit * User aborts otherwise, output is suppressed

tests: fix rebase-abort directory nesting Without this change, some test repositories were nesting needlessly

rebase: document that tool does not apply to deleted files

rebase: mention help merge-tools in help

copies: optimize forward copy detection logic for rebases Forward copy detection (i.e. detecting what files have been moved/copied in commit X since ancestor Y) previously required diff'ing the manifests of both X and Y. This was expensive since it required reading both entire manifests and doing a set difference (they weren't already in a set because of the lazymanifest work). This cost almost 1 second on very large repositories, and happens N times for a rebase of N commits. This patch optimizes it for the case of rebase. In a rebase, we are comparing a commit against it's immediate parent, and therefore we can know what files changed by looking at ctx.files(). This lets us drastically decrease the size of the set comparison, and makes it O(# of changes) instead of O(size of manifest). This makes it take 1ms instead of 1000ms.

tests: pass settings of hypothesis by with statement given(..., settings=) is no longer available in Hypothesis 2.0.0. https://github.com/DRMacIver/hypothesis/commit/7712c01

tests: adjust for code move in Hypothesis 2.0.0 It appears that the Settings class was renamed to settings, and because of this, the settings module was renamed to configuration. https://github.com/DRMacIver/hypothesis/commit/a0e663b

test-highlight: add normalization rule for Pygments 2.1 It appears that several classes have 'h', '1', '2' suffixes on Pygments 2.1, which is the current version on Debian sid.

test-highlight: factor out function that normalizes pygments output

test-highlight: unify normalization rule of pygments output We had two variants for unknown reason, s/mf/mi/g and s/mi/mf/g. This patch unifies them to s/mf/mi/g so that we can introduce a utility function.

tests: remove protected file forcibly for portability Before this patch, test-command-template.t is timed out on Solaris, because "rm" on permission denied file implies prompting "override protection 0 (yes/no)?" and blocks execution of test script.

tests: dump journal file by python script instead of sed for portability Before this patch, test-repair-strip.t fails on Solaris, because of "sed" on it doesn't work as expected if input contains null ('\0') character.

tests: execute check-config.py without xargs Before this patch, test-check-config.t fails on Solaris, because "xargs" doesn't invoke check-config.py with all filenames at once. "xargs" may invoke specified command multiple times with part of arguments given from stdin: according to "xargs(1)" man page, this dividing arguments is system-dependent. For portability of test-check-config.t, this patch adds "xargs" like mode to check-config.py and executes it in test-check-config.t without "xargs".

tests: avoid nested quoting on command line for portability Before this patch, test-revset.t fails on Solaris using ksh as default "sh", because nested quoting below isn't acceptable for it. +-----------------------------+ inner quoting "`python -c "print '|'.join(['0:1'] * 500)"`" +-------------------------------------------+ outer quoting This patch does below for portability. - omit outer quoting This should be safe, because generated string contains no white space character. - use '+' instead of '|' (for safety) '|' has special meaning for many shell, but '+' isn't (at least, for ordinary ones).

version: verbose list internal and external extension source (issue4731)

check-code: allow "grep pattern filename-containing-dash-a" We're trying to forbid "grep -a" and unintentionally complained even if the "-a" was part of the filename. Requiring a space before "-a" to match is probably good enough.

rebase: update working directory when aborting (issue5084)