run-tests: allow specifying an output dir to write .errs to I'm trying to use run-tests.py on a read-only file system. This series allows that to happen.

run-tests: add information about skipped tests to XUnit output The XUnit spec supports skipped tests.

run-tests: wrap failures in an XUnit 'failure' element This is closer to what most XUnit consumers can understand.

discovery: log discovery result in non-trivial cases We log the discovery summary, the number of roundtrips and the elapsed time. This is useful to understand where slow push might come from when lloking at the blackbox.

discovery: include timing in the debug output Having such date easily available is useful. It also prepare the inclusion of some discovery related data in blackbox.

tests: remove sys.executable from "required tools" In practice this doesn't appear to have been true for some time - we reference Python using the $PYTHON variable in all the tests now (which we have to for PyPy and Python 3), and I've been using ~/.../python.exe to test with tip of the cpython 3.6 release branch while working on manifest tests in Python 3 and everything seems to be just fine. The only real observable difference from this change is that I stop getting a warning about python.exe not being a thing on $PATH, which seems like an improvement.

perfbranchmap: add an option to purge the revbranch cache The perf extension needs to be able to reflect this reality too. (eg: 4s vs 60s on a Million-ish revisions repository).

push: add a way to allow concurrent pushes on unrelated heads Client has a mechanism for the server to check that nothing changed server side since the client prepared a push. That check is wide and any head changed on the server will lead to an aborted push. We introduce a way for the client to send a less strict checking. That logic will check that no heads impacted by the push have been affected. If other unrelated heads (including named branches heads) have been affected, the push will proceed. This is very helpful for repositories with high developers traffic on different heads, a common setup. That behavior is currently controlled by an experimental option. The config should live in the "server" section but bike-shedding of the name will happen in the next changesets. Servers advertise this capability through a new bundle2 capability 'checkeads', using the value 'related'. The 'test-push-race.t' is updated to check that new capabilities on the documented cases.

headsummary: expose the 'discardedheads' set in the headssummary That information will be useful to detect push race on related part of the history. See next changeset for details.

checkheads: perform obsolescence post processing directly in _headssummary The goal is to have the function directly return something meaningful and useful for the whole pull. Note: we skip adding post-processing in '_oldheadssummary' because if a client is too old for branchmap it will be too old for obsolescence too.

headssummary: directly feed the function with the 'pushop' object Our goal is to be able to perform the post processing directly into the '_headssummary' function. However before this patch the '_headsummary' function only had access to repo, remote, outgoing while the '_postprocessobsolete' function takes a 'pushop' object. Experience shows that having the 'pushop' object helps extensions so we update '_headssummary' to take a pushop object as argument.

checkheads: gather the postprocessing with other obsolescence specific code We extract this function from the loop and gather it with the rest of the obsolescence specific code. That will help to clarify the move of the whole logic inside the "heads summary" computation.

run-tests: add a way to list tests, with JSON and XUnit support Some test runners are interested in listing tests, so they can do their own filtering on top (usually based on attributes like historically observed runtime). Add support for that.

run-tests: install hg after computing tests to run We're going to add a way to list tests, and we don't need to install hg for that.

run-tests: make time field optional for xunit report We're going to use XUnit to list tests, and we don't have a time field in that case.

run-tests: factor out json write code into another method We're going to use this code to output a JSON-formatted listing of tests.

run-tests: factor out xunit write code into another method We're going to use this code to output an XUnit-formatted listing of tests.

revset: lookup descendents for negative arguments to ancestor operator Negative offsets to the `~` operator now search for descendents. The search is aborted when a node has more than one child as we do not have a definition for 'nth child'. Optionally we can introduce such a notion and take the nth child ordered by rev number. The current revset language does provides a short operator for ancestor lookup but not for descendents. This gives user a simple revset to move to the previous changeset, e.g. `hg up '.~1'` but not to the 'next' changeset. With this change userse can now use `.~-1` as a shortcut to move to the next changeset. This fits better into allowing users to specify revisions via revsets and avoiding the need for special `hg next` and `hg prev` operations. The alternative to negative offsets is adding a new operator. We do not have many operators in ascii left that do not require bash escaping (',', '_', and '/' come to mind). If we decide that we should add a more convenient short operator such as ('/', e.g. './1') we can later add it and allow ascendents lookup via negative numbers.

update: show the commit to which we updated in case of multiple heads (BC) Currently when we have multiple heads on the same branch, update tells us that there some more heads for the current branch but does not tells us the head to which the repository has been updated to. It makes more sense showing the head we updated to and then telling there are some more heads.

revlog: skeleton support for version 2 revlogs There are a number of improvements we want to make to revlogs that will require a new version - version 2. It is unclear what the full set of improvements will be or when we'll be done with them. What I do know is that the process will likely take longer than a single release, will require input from various stakeholders to evaluate changes, and will have many contentious debates and bikeshedding. It is unrealistic to develop revlog version 2 up front: there are just too many uncertainties that we won't know until things are implemented and experiments are run. Some changes will also be invasive and prone to bit rot, so sitting on dozens of patches is not practical. This commit introduces skeleton support for version 2 revlogs in a way that is flexible and not bound by backwards compatibility concerns. An experimental repo requirement for denoting revlog v2 has been added. The requirement string has a sub-version component to it. This will allow us to declare multiple requirements in the course of developing revlog v2. Whenever we change the in-development revlog v2 format, we can tweak the string, creating a new requirement and locking out old clients. This will allow us to make as many backwards incompatible changes and experiments to revlog v2 as we want. In other words, we can land code and make meaningful progress towards revlog v2 while still maintaining extreme format flexibility up until the point we freeze the format and remove the experimental labels. To enable the new repo requirement, you must supply an experimental and undocumented config option. But not just any boolean flag will do: you need to explicitly use a value that no sane person should ever type. This is an additional guard against enabling revlog v2 on an installation it shouldn't be enabled on. The specific scenario I'm trying to prevent is say a user with a 4.4 client with a frozen format enabling the option but then downgrading to 4.3 and accidentally creating repos with an outdated and unsupported repo format. Requiring a "challenge" string should prevent this. Because the format is not yet finalized and I don't want to take any chances, revlog v2's version is currently 0xDEAD. I figure squatting on a value we're likely never to use as an actual revlog version to mean "internal testing only" is acceptable. And "dead" is easily recognized as something meaningful. There is a bunch of cleanup that is needed before work on revlog v2 begins in earnest. I plan on doing that work once this patch is accepted and we're comfortable with the idea of starting down this path.

check-code: ban grep's context flags (-A/-B/-C) since they're not on Solaris

patchbomb: avoid -r and -B options at the same time

debugbundle: add --part-type flag to emit only named part types This removes the need in the tests for grep -A, which is not supported on Solaris.

test-obsolete-bundle-strip: add globs for Windows

obsstore: move header encoding to a separate function This patch moves encodeheader from encodemarkers. So markers and header could be encoded separately.

obsstore: move _version to a propertycache This makes sure _version is correct even if "_all" is not called.

obsstore: separate marker parsing from obsstore reading This allows us to get raw obsstore content without parsing any markers. Reading obsstore is much cheaper than parsing markers.

obsstore: move marker version reading to a separate function This allows us to read marker version without reading markers.

obsstore: minor optimization for the obsolete revset Use local variables in a loop.

dispatch: do not close stdout and stderr, just flush() instead Since 3a4c0905f357 "util: always force line buffered stdout when stdout is a tty", we have two file objects attached to the same STDOUT_FILENO. If one is closed, the underlying file descriptor is also closed, and writing to the other file object would crash the Python interpreter in a hard way, at least on Windows. So, it seems safer to not close the standard streams. This also matches the behavior of the default sys.stdout/stderr.close(), which never close the FILE* streams in C layer. https://hg.python.org/cpython/file/v2.7.13/Python/sysmodule.c#l1401

windows: do not close stdout on flush() failure It's been there since e817c68edfed (2007-02-19), but seems wrong since any I/O operations to a closed file would raise ValueError, not IOError. We should keep the file object open even if the underlying file descriptor is half dead.

test-obsolete-bundle-strip: do not include \n in filename (issue5586)