blackbox: refactor use of vfs as _bbvfs

blackbox: flush output file descriptor Without this, when there are multiple ui views, each blackbox will have its own file handle, and the logging will be in a really bad order. Also, because of the way blackbox works, it never closes its file handles, which means the last output before exit is often lost.

tests: change blackbox test to work cross platform While it is not easy to make a file 000 on Windows, you can emulate most of the behaviors by replacing the file with a directory. Also corrects test description to properly indicate that failing to read from the log is fatal.

merge: document checkignored and checkunknown configs again These options were undocumented for 3.7 because of an issue found during the freeze (see rev 7cb7264cfd52). This issue has now been fixed, so we can document these options again.

rebase: respect checkunknown and checkignored in more cases checkunknown and checkignored are currently respected for updates and regular merges, but not for certain kinds of rebases. To be precise, they aren't respected for rebases when: (1) we're rebasing while currently on the destination commit, and (2) an untracked or ignored file F is currently in the working copy, and (3) the same file F is in a source commit, and (4) F has different contents in the source commit. This happens because rebases set force to True when calling merge.update. Setting force to True makes a lot of sense in general, but it turns out the force option is overloaded: there's a deprecated '--force' option in merge that allows you to merge in outstanding changes, including changes in untracked files. We use the 'mergeforce' parameter to tell those two cases apart. I think the behavior during rebases when checkunknown is 'abort' (the default) is wrong -- we should abort on or overwrite differing untracked files, not try to merge them in. However that currently breaks rebases by aborting in the middle -- we need better handling for that case before we can change the default.

test-merge-force: add tests for merge.checkunknown=warn In an upcoming patch we're going to change the behavior of some merges with merge.checkunknown=warn or ignore -- ensure that the behavior of the deprecated 'merge --force' remains the same.

merge: tell _checkunknownfiles about whether this was merge --force In an upcoming patch we'll have different behavior here for when 'merge --force' is used as opposed to when other kinds of force operations are performed, like rebases.

merge: add missing doc for 'labels' parameter

merge: move abort/warn checks up to the top level of _checkunknownfiles In upcoming patches, we're also going to do these checks when force is True.

match: rename "narrowmatcher" to "subdirmatcher" (API) I keep mistaking "narrowmatcher" for narrowhg's narrowmatcher. "subdirmatcher" seems more to the point anyway.

tests: relax test-devel-warnings to reduce false positives This test is interested in warning output, so glob away line numbers and hashes as they aren't relevant to its core.

merge with stable

chgserver: create new process group after fork (issue5051) This is to make SIGTSTP work. Before the patch, the server process group is considered "orphaned" and will ignore SIGTSTP, SIGTTIN, SIGTTOU, according to POSIX. See the comment above `will_become_orphaned_pgrp` in `kernel/exit.c` from Linux 4.3 for details. SIGTSTP is important if chgserver runs some ncurses commend like `commit -i`. Ncurses has its own SIGTSTP handler which will do the following: 1. Clean the screen 2. Stop itself by resending SIGTSTP to itself 3. Restore the screen If SIGTSTP is ignored, step 2 will be a noop, which means the process cannot be suspended properly. In order to make things work, chg client needs to forward SIGTSTP and SIGCONT to server as well.

check-commit: check for double-addition of blank lines Previously, we were only checking for a blank line being added next to an existing one. Now we also check for two being added at the same time.

check-commit: scan for multiple instances of error patterns

merge: add file ancestor linknode to mergestate During a merge, each file has a current commitnode+filenode, an other commitnode+filenode, and an ancestor commitnode+filenode. The ancestor commitnode is not stored though, and we rely on the ability for the filectx() to look up the commitnode by using the filenode's linkrev. In alternative backends (like remotefilelog), linkrevs may have restriction that prevent arbitrary linkrev look up given a filenode. This patch accounts for that by storing the ancestor commitnode in the merge state so that it is available later at resolve time. This results in some test changes because the ancestor commitnode we're using at resolve time changes slightly. Before, we used the linkrev commit, which is the earliest commit that introduced that particular filenode (which may not be the latest common ancestor of the commits being merged). Now we use the latest common ancestor of the merged commits as the commitnode. This is fine though, because that commit contains the same filenode as the linkrev'd commit.

merge: add debugmergestate support for _stateextras Now that we can store extras for each file, we need to have support for showing it in debugmergestate (the tests depend on this).

merge: add state extras merge state data In future commits we will want to store more data related to each file in the merge state. This patch adds an optional record for storing a dictionary of extras for each file.

revset: use manifest.matches in _follow revset The old _follow revset iterated over every file in the commit and checked if it matched. For repos with large manifests, this could take 500ms. By switching to use manifest.matches() we can take advantage of the fastpaths built in to manifest.py that allows iterating over only the files in the matcher when it's a simple matcher. This brings the time spent down from 500ms to 0ms during simple operations like 'hg log -f file.txt'.

treemanifests: fix streaming clone Similar to the previous patch, the .hg/store/meta/ directory does not get copied when when using "hg clone --uncompressed". Fix by including "meta/" in store.datafiles(). This seems safe to do, as there are only a few users of this method. "hg manifest" already filters the paths by "data/" prefix. The calls from largefiles also seem safe. The use in verify needs updating to prevent it from mistaking dirlogs for orphaned filelogs. That change is included in this patch. Since the dirlogs will now be in the fncache when using fncachestore, let's also update debugrebuildfncache(). That will also allow any existing treemanifest repos to get their dirlogs into the fncache. Also update test-treemanifest.t to use an a directory name that requires dot-encoding and uppercase-encoding so we test that the path encoding works.

treemanifests: fix local clone When doing a local clone with treemanifests, the .hg/store/meta/ directory currently does not get copied. To fix it, all we need to do is to add it to the list of directories to copy.

tests: simplify treemanifest test by backing up entire .hg/store

histedit: limit updated and merging output to important updates Output is retained when: * There's a conflict * User asked to edit * User aborts otherwise, output is suppressed

tests: fix rebase-abort directory nesting Without this change, some test repositories were nesting needlessly

rebase: document that tool does not apply to deleted files

rebase: mention help merge-tools in help

copies: optimize forward copy detection logic for rebases Forward copy detection (i.e. detecting what files have been moved/copied in commit X since ancestor Y) previously required diff'ing the manifests of both X and Y. This was expensive since it required reading both entire manifests and doing a set difference (they weren't already in a set because of the lazymanifest work). This cost almost 1 second on very large repositories, and happens N times for a rebase of N commits. This patch optimizes it for the case of rebase. In a rebase, we are comparing a commit against it's immediate parent, and therefore we can know what files changed by looking at ctx.files(). This lets us drastically decrease the size of the set comparison, and makes it O(# of changes) instead of O(size of manifest). This makes it take 1ms instead of 1000ms.

tests: pass settings of hypothesis by with statement given(..., settings=) is no longer available in Hypothesis 2.0.0. https://github.com/DRMacIver/hypothesis/commit/7712c01

tests: adjust for code move in Hypothesis 2.0.0 It appears that the Settings class was renamed to settings, and because of this, the settings module was renamed to configuration. https://github.com/DRMacIver/hypothesis/commit/a0e663b

test-highlight: add normalization rule for Pygments 2.1 It appears that several classes have 'h', '1', '2' suffixes on Pygments 2.1, which is the current version on Debian sid.

test-highlight: factor out function that normalizes pygments output

test-highlight: unify normalization rule of pygments output We had two variants for unknown reason, s/mf/mi/g and s/mi/mf/g. This patch unifies them to s/mf/mi/g so that we can introduce a utility function.