tests: work around flaky test failure on FreeBSD and relatives test-http-bad-server has a flaky output case that reproduces fairly often (20% or more of runs on my Mac) on BSD-family OSes. So that tests can be green for the 4.2 release, let's more or less give up on this output, and plan to pursue it in the future.

context: start walking from "introrev" in blockancestors() Previously, calling blockancestors() with a fctx not touching file would sometimes yield this filectx first, instead of the first "block ancestor", because when compared to its parent it may have changes in specified line range despite not touching the file at all. Fixing this by starting the algorithm from the "base" filectx obtained using fctx.introrev() (as done in annotate()). In tests, add a changeset not touching file we want to follow lines of to cover this case. Do this in test-annotate.t for followlines revset tests and in test-hgweb-filelog.t for /log/<rev>/<file>?linerange=<from>:<to> tests.

sshpeer: try harder to snag stderr when stdout closes unexpectedly Resolves test failures on FreeBSD, but I'm not happy about the fix. A previous version of this also wrapped readline by putting the hack in the _call method on doublepipe. That was confusing for readers and wasn't necessary - just doing this on read() is sufficient to fix the bugs I'm observing. We can always come back and do readline later if needed.

test-worker: disable tests of forked workers on Windows The number of the "Traceback" lines differs on Windows because the main process does not raise SystemExit.

test-profile: allow negative time in JSON output (issue5542) I don't know why it can be a negative number, but that shouldn't be important here.

show: add basic labels to work template `hg show work` is much more usable if output is colored. This patch implements coloring via label() in a very hacky way. In a default Mercurial install, you'll see yellow node labels for all phases. Branches and bookmarks use the same formatting as the commit message. So this change doesn't help much in a default install. But if you have a custom colors defined for these things, output is much more readable. The implementation obviously needs some work. But for a minor change on a feature that isn't convered by BC, this seems like a clear win for the feature in 4.2.

show: rename "underway" to "work" Durham and I both like this better than "underway." We can add aliases and bikeshed on the name during the 4.3 cycle, as this whole extension is highly experimental.

histedit: make check for unresolved conflicts explicit (issue5545) Previously, we'd rely on the implicit check that `localrepo.commit` did. The problem is that that check only happened when the working copy was dirty. With a "clean" working copy but unresolved conflicts we'd get into a broken state. To fix that, do what rebase does and check for unresolved conflicts at the start of histedit --continue.

Added signature for changeset 616e788321cc

Added tag 4.2-rc for changeset 616e788321cc

freeze: merge default into stable for 4.2 code freeze

Merge stable with security patch.

Added signature for changeset 77eaf9539499

Added tag 4.1.3 for changeset 77eaf9539499

dispatch: protect against malicious 'hg serve --stdio' invocations (sec) Some shared-ssh installations assume that 'hg serve --stdio' is a safe command to run for minimally trusted users. Unfortunately, the messy implementation of argument parsing here meant that trying to access a repo named '--debugger' would give the user a pdb prompt, thereby sidestepping any hoped-for sandboxing. Serving repositories over HTTP(S) is unaffected. We're not currently hardening any subcommands other than 'serve'. If your service exposes other commands to users with arbitrary repository names, it is imperative that you defend against repository names of '--debugger' and anything starting with '--config'. The read-only mode of hg-ssh stopped working because it provided its hook configuration to "hg serve --stdio" via --config parameter. This is banned for security reasons now. This patch switches it to directly call ui.setconfig(). If your custom hosting infrastructure relies on passing --config to "hg serve --stdio", you'll need to find a different way to get that configuration into Mercurial, either by using ui.setconfig() as hg-ssh does in this patch, or by placing an hgrc file someplace where Mercurial will read it. mitrandir@fb.com provided some extra fixes for the dispatch code and for hg-ssh in places that I overlooked.

progress: retry ferr.flush() and .write() on EINTR (issue5532) See the inline comment how this could mitigate the issue. I couldn't reproduce the exact problem on my Linux machine, but there are at least two people who got EINTR in progress.py, and it seems file_write() of Python 2 is fundamentally broken [1]. Let's make something in on 4.2. [1]: https://hg.python.org/cpython/file/v2.7.13/Objects/fileobject.c#l1850

progress: extract stubs to restart ferr.flush() and .write() on EINTR

filemerge: optionally strip quotes from merge marker template (BC) For consistency with the other template options. Quotes are necessary if you want to preserve leading/trailing whitespace, which would be stripped by config parser.

commit: optionally strip quotes from commit template (BC) For consistency with the other template options.

graphlog: optionally strip quotes from graphnode template (BC) For consistency with the other template options.

dispatch: ignore further SIGPIPE while handling KeyboardInterrupt I got the following error by running "hg log" and quitting the pager immediately. Any output here may trigger another SIGPIPE, so only thing we can do is to swallow the exception and exit with an error status. Traceback (most recent call last): File "./hg", line 45, in <module> mercurial.dispatch.run() File "mercurial/dispatch.py", line 83, in run status = (dispatch(req) or 0) & 255 File "mercurial/dispatch.py", line 167, in dispatch req.ui.warn(_("interrupted!\n")) File "mercurial/ui.py", line 1224, in warn self.write_err(*msg, **opts) File "mercurial/ui.py", line 790, in write_err self._write_err(*msgs, **opts) File "mercurial/ui.py", line 798, in _write_err self.ferr.write(a) File "mercurial/ui.py", line 129, in _catchterm raise error.SignalInterrupt mercurial.error.SignalInterrupt Perhaps this wasn't visible before de5c9d0e02ea because the original stderr handle was restored very late.

worker: print traceback for uncaught exception unconditionally This is what a Python interpreter would do if there were no os._exit().

worker: propagate exit code to main process Follows up 86cd09bc13ba.

dispatch: print traceback in scmutil.callcatch() if --traceback specified Otherwise, traceback wouldn't be printed for a known exception occurred in worker processes.

dispatch: mark callcatch() as a private function

templatefilters: fix crash by string formatting of '{x|splitlines}' Before, it crashed because mapping['templ'] was missing. As it didn't support the legacy list template from the beginning, we can simply use hybridlist().

templatekw: factor out showdict() helper Make it less cryptic for common cases.

templatekw: have showlist() take mapping dict with no **kwargs expansion (API) See the previous commit for why. splitlines() does not pass a mapping dict, which would probably mean the legacy template didn't work from the beginning.

templatekw: change _showlist() to take mapping dict with no **kwargs expansion There was a risk that a template keyword could conflict with an argument name (e.g. 'name', 'values', 'plural', etc.) Let's make it less magical.

templatekw: rename 'args' to 'mapping' in showlist() The name 'args' provides no information. Call it 'mapping' as in templater.py.

templatekw: eliminate unnecessary temporary variable 'names' from _showlist() Replace 'names' with the optional argument 'plural'.

color: update the help with the new default The default is now "auto" we update the help to match reality.

upgrade: register all format variants in a list Now that all known format variants exists outside of the function, we can gather them in a lists. This build a single entry point other code can use (current target: extensions). The repository upgrade code is updated to simply use entries from this list. As a side effect this will also allow extensions to register their own format variants, to do this "properly" we should introduce a "registrar" for this category of object. However I prefer to keep this series simple, and that will be adventure for future time.

upgrade: move descriptions and selection logic in individual classes Our goal here is to get top level definition for all the format variants. Having them defined outside of the function enabled other users of that logic. They are two keys components of a format variant: 1) the name and various descriptions of its effect, 2) the code that checks if the repo is using this variant and if the config enables it. That second items make us pick a class-based approach, since different variants requires different code (even if in practice, many can reuse the same logic). Each variants define its own class that is then used like a singleton. The class-based approach also clarify the definitions part a bit since each are simple assignment in an indented block. The 'fromdefault' and 'fromconfig' are respectively replaced by a class attribute and a method to be called at the one place where "fromconfig" matters. Overall, they are many viable approach for this, but this is the one I picked.

upgrade: introduce a 'formatvariant' class The 'deficiency' type has multiple specificities. We create a dedicated class to host them. More logic will be added incrementally in future changesets.

upgrade: implement '__hash__' on 'improvement' class The pythonomicon request its implementation.

upgrade: implement '__ne__' on 'improvement' class The pythonomicon request its implementation.

color: also enable by default on windows I've not found anything related to color + windows on the bug tracker. So I'm suggesting we get bolder and turn it on for windows too in the release candidate. We can always backout this changeset if we find serious issue on windows.

color: turn on by default (but for windows) Color support is all in core for a couple of months. I've browsed the bug tracker without finding any blocker bug. So I'm moving forward and enable color on by default before '4.2-rc'. In the worse case, having it on in the release candidate will help us to find blocker bug and we can turn it off for the final release. I remember people talking about issue with Windows during the freeze so I'm keeping it off by default on that OS. We could do various cleaning of the color used and the label issued. However the label are probably already in our backward compatibility envelope since the color extensions has been around since for ever and I do not think the color choice themself should be considered BC. So I think we should rather gives color to all user sooner than later. A couple of test needs to be updated to avoid having color related control code spoil the tested output.

pager: stop using the color extension in tests The feature is in core so let us use the core config knob directly.

bundle2: ignore errors seeking a bundle after an exception (issue4784) Many have seen a "stream ended unexpectedly" error. This message is raised from changegroup.readexactly() when a read(n) operation fails to return exactly N bytes. I believe most occurrences of this error in the wild stem from the code changed in this patch. Before, if bundle2's part applicator raised an Exception when processing/applying parts, the exception handler would attempt to iterate the remaining parts. If I/O during this iteration failed, it would likely raise the "stream ended unexpectedly" exception. The problem with this approach is that if we already encountered an I/O error iterating the bundle2 data during application, then any further I/O would almost certainly fail. If the original stream were closed, changegroup.readexactly() would obtain an empty string, triggering "stream ended unexpectedly" with "got 0." This is the error message that users would see. What's worse is that the original I/O related exception would be lost since a new exception would be raised. This made debugging the actual I/O failure effectively impossible. This patch changes the exception handler for bundle2 application to ignore errors when seeking the underlying stream. When the underlying error is I/O related, the seek should fail fast and the original exception will be re-raised. The output changes in test-http-bad-server.t demonstrate this. When the underlying error is not I/O related and the stream can be seeked, the old behavior is preserved.

error: rename RichIOError to PeerTransportError This is a more descriptive name. RichIOError was introduced just hours ago, so it doesn't need to be marked as BC.

httppeer: don't send empty Vary request header As part of writing test-http-bad-server.t, I noticed that some requests include an empty Vary HTTP request header. The Vary HTTP request header indicates which headers should be taken into account when determining if a cached response can be used. It also accepts the special value of "*". The previous code unconditionally added a Vary header. This could lead to an empty header value. While I don't believe this violates the HTTP spec, this is weird and just wastes bytes. So this patch changes behavior to only send a Vary header when it has a value. Some low-level wire protocol byte reporting tests changed. In some cases, the exact point of data termination changed. However, the behavior being tested - that clients react when the connection is closed in the middle of an HTTP request line or header - remains unchanged.

badserverext: explicitly flush each log write No idea why, but without this, there was no content in the error logs that are `cat`d in the test on Windows.

test-http-bad-server: conditionalize error output for Windows

test-tag: make hook runnable on Windows Simply setting the hook to *.sh causes Windows to open a dialog asking how to open *.sh files.

obsolescence: add test for the "branch replacement" logic during push, case D7 Mercurial checks for the introduction of new heads on push. Evolution comes into play to detect if existing branches on the server are being replaced by some of the new one we push. This changeset adds test for the improved "branch replacement" logic introduce in an earlier commits. This tests initially lived in the evolve extensions. Since we now have the code handling this logic in core, it make sense to have the tests in core too. See inline documentation for details about the test case added in this changeset.

obsolescence: add test for the "branch replacement" logic during push, case D5 Mercurial checks for the introduction of new heads on push. Evolution comes into play to detect if existing branches on the server are being replaced by some of the new one we push. This changeset adds test for the improved "branch replacement" logic introduce in an earlier commits. This tests initially lived in the evolve extensions. Since we now have the code handling this logic in core, it make sense to have the tests in core too. See inline documentation for details about the test case added in this changeset.

obsolescence: add test for the "branch replacement" logic during push, case D3 Mercurial checks for the introduction of new heads on push. Evolution comes into play to detect if existing branches on the server are being replaced by some of the new one we push. This changeset adds test for the improved "branch replacement" logic introduce in an earlier commits. This tests initially lived in the evolve extensions. Since we now have the code handling this logic in core, it make sense to have the tests in core too. See inline documentation for details about the test case added in this changeset.

obsolescence: add test for the "branch replacement" logic during push, case B8 Mercurial checks for the introduction of new heads on push. Evolution comes into play to detect if existing branches on the server are being replaced by some of the new one we push. This changeset adds test for the improved "branch replacement" logic introduce in an earlier commits. This tests initially lived in the evolve extensions. Since we now have the code handling this logic in core, it make sense to have the tests in core too. See inline documentation for details about the test case added in this changeset.

obsolescence: add test for the "branch replacement" logic during push, case B7 Mercurial checks for the introduction of new heads on push. Evolution comes into play to detect if existing branches on the server are being replaced by some of the new one we push. This changeset adds test for the improved "branch replacement" logic introduce in an earlier commits. This tests initially lived in the evolve extensions. Since we now have the code handling this logic in core, it make sense to have the tests in core too. See inline documentation for details about the test case added in this changeset.

obsolescence: add test for the "branch replacement" logic during push, case B6 Mercurial checks for the introduction of new heads on push. Evolution comes into play to detect if existing branches on the server are being replaced by some of the new one we push. This changeset adds test for the improved "branch replacement" logic introduce in an earlier commits. This tests initially lived in the evolve extensions. Since we now have the code handling this logic in core, it make sense to have the tests in core too. See inline documentation for details about the test case added in this changeset.

obsolescence: add test for the "branch replacement" logic during push, case B5 Mercurial checks for the introduction of new heads on push. Evolution comes into play to detect if existing branches on the server are being replaced by some of the new one we push. This changeset adds test for the improved "branch replacement" logic introduce in an earlier commits. This tests initially lived in the evolve extensions. Since we now have the code handling this logic in core, it make sense to have the tests in core too. See inline documentation for details about the test case added in this changeset.

obsolescence: add test for the "branch replacement" logic during push, case B4 Mercurial checks for the introduction of new heads on push. Evolution comes into play to detect if existing branches on the server are being replaced by some of the new one we push. This changeset adds test for the improved "branch replacement" logic introduce in an earlier commits. This tests initially lived in the evolve extensions. Since we now have the code handling this logic in core, it make sense to have the tests in core too. See inline documentation for details about the test case added in this changeset.

obsolescence: add test for the "branch replacement" logic during push, case B2 Mercurial checks for the introduction of new heads on push. Evolution comes into play to detect if existing branches on the server are being replaced by some of the new one we push. This changeset adds test for the improved "branch replacement" logic introduce in an earlier commits. This tests initially lived in the evolve extensions. Since we now have the code handling this logic in core, it make sense to have the tests in core too. See inline documentation for details about the test case added in this changeset.

checkheads: upgrade the obsolescence postprocessing logic (issue4354) The previous logic had many shortcoming (eg: looking at the head only, not handling prune, etc...), the new logic use a more robust approach: For each head, we check if after the push all changesets exclusive to this heads will be obsolete. If they are, the branch considered be "replaced". To check if a changeset will be obsolete, we simply checks: * the changeset phase * the existence of a marker relevant to the "pushed set" that affects the changesets.. This fixes two major issues of the previous algorithm: * branch partially rewritten (eg: head but not root) are no longer detected as replaced, * Prune are now properly handled. (This implementation was introduction in the evolve extension, version 6.0.0.) This new algorithm has an extended number of tests, a basic one is provided in this patch. The others will be introduced in their own changeset for clarity. In addition, we stop trying to process heads unknown locally, we do not have enough data to take an informed decision so we should stop pretending we do. This reflect a test that is now update.

test-http: add a (glob) for Windows I'm not sure why the glob was ineffective with $TESTTMP here, but replacing it with '*' works.

testlib: move the prune alias into a shell script for Windows Cramming all of this directly into an alias doesn't play nicely on Windows. Various test-exchange-obsmarkers-case-XX.t were failing with: abort: cannot select revision when creating marker It turned out that inside debugcommands.debugobsolete(), the following differed from Linux (where they were empty, at least in the case I debugged): 'rev': ['.`;'], 'template': "'{node}\\n'"

hidden: extract the code generating "filtered rev" error for wrapping The goal is to help experimentation in extensions (ie: evolve) around more advance messages.

serve: add support for Mercurial subrepositories I've been using `hg serve --web-conf ...` with a simple '/=projects/**' [paths] configuration for awhile without issue. Let's ditch the need for the manual configuration in this case, and limit the repos served to the actual subrepos. This doesn't attempt to handle the case where a new subrepo appears while the server is running. That could probably be handled with a hook if somebody wants it. But it's such a rare case, it probably doesn't matter for the temporary serves. The main repo is served at '/', just like a repository without subrepos. I'm not sure why the duplicate 'adding ...' lines appear on Linux. They don't appear on Windows (see 594dd384803c), so they are optional. Subrepositories that are configured with '../path' or absolute paths are not cloneable from the server. (They aren't cloneable locally either, unless they also exist at their configured source, perhaps via the share extension.) They are still served, so that they can be browsed, or cloned individually. If we care about that cloning someday, we can probably just add the extra entries to the webconf dictionary. Even if the entries use '../' to escape the root, only the related subrepositories would end up in the dictionary.

hgwebdir: allow a repository to be hosted at "/" This can be useful in general, but will also be useful for hosting subrepos, with the main repo at /.

httppeer: eliminate decompressresponse() proxy Now that the response instance itself is wrapped with error handling, we no longer need this code. This code became dead with the previous patch because the added code catches HTTPException and re-raises as something else.

httppeer: wrap HTTPResponse.read() globally There were a handful of places in the code where HTTPResponse.read() was called with no explicit error handling or with inconsistent error handling. In order to eliminate this class of bug, we globally swap out HTTPResponse.read() with a unified error handler. I initially attempted to fix all call sites. However, after going down that rabbit hole, I figured it was best to just change read() to do what we want. This appears to be a worthwhile change, as the tests demonstrate many of our uncaught exceptions go away. To better represent this class of failure, we introduce a new error type. The main benefit over IOError is it can hold a hint. I'm receptive to tweaking its name or inheritance.

tests: add tests for poorly behaving HTTP server I've spent several hours over the past few weeks investigating networking failures involving hg.mozilla.org. As part of this, it has become clear that the Mercurial client's error handling when it encounters network failures is far from robust. To prove this is true, I've devised a battery of tests simulating various network failures, notably premature connection closes. To achieve this, I've implemented an extension that monkeypatches the built-in HTTP server and hooks in at the socket level and allows various events to occur based on config options. For example, you can refuse to accept() a client socket or you can close() the socket after N bytes have been sent or received. The latter effectively simulates an unexpected connection drop (and these occur all the time in the real world). The new test file launches servers exhibiting various "bad" behaviors and points a client at them. As the many TODO comments in the test call attention to, Mercurial often displays unhelpful errors when network-related failures occur. This makes it difficult for users to understand what's going on and difficult for server administrators to pinpoint root causes without packet tracing. Upcoming patches will attempt to fix these error handling deficiencies.

phases: emit phases to pushkey protocol in deterministic order An upcoming test will report exact bytes sent over the wire protocol. Without this change, the ordering of phases listkey data is non-deterministic.

keepalive: send HTTP request headers in a deterministic order An upcoming patch will add low-level testing of the bytes being sent over the wire. As part of developing that test, I discovered that the order of headers in HTTP requests wasn't deterministic. This patch makes the order deterministic to make things easier to test.

revset: properly parse "descend" argument of followlines() We parse "descend" symbol as a Boolean using getboolean (prior extraction by getargsdict already checked that it is a symbol). In tests, check for error cases and vary Boolean values here and there.

revsetlang: add a getboolean helper function This will be used to parse followlines's "descend" argument.

track-tags: write all tag changes to a file The tag changes information we compute is now written to disk. This gives hooks full access to that data. The format picked for that file uses a 2 characters prefix for the action: -R: tag removed +A: tag added -M: tag moved (old value) +M: tag moved (new value) This format allows hooks to easily select the line that matters to them without having to post process the file too much. Here is a couple of examples: * to select all newly tagged changeset, match "^+", * to detect tag move, match "^.M", * to detect tag deletion, match "-R". Once again we rely on the fact the tag tests run through all possible situations to test this change.

track-tags: compute the actual differences between tags pre/post transaction We now compute the proper actuall differences between tags before and after the transaction. This catch a couple of false positives in the tests. The compute the full difference since we are about to make this data available to hooks in the next changeset.

track-tags: introduce first bits of tags tracking during transaction This changeset introduces detection of tags changes during transaction. When this happens a 'tag_moved=1' argument is set for hooks, similar to what we do for bookmarks and phases. This code is disabled by default as there are still various performance concerns. Some require a smarter use of our existing tag caches and some other require rework around the transaction logic to skip execution when unneeded. These performance improvements have been delayed, I would like to be able to experiment and stabilize the feature behavior first. Later changesets will push the concept further and provide a way for hooks to know what are the actual changes introduced by the transaction. Similar work is needed for the other families of changes (bookmark, phase, obsolescence, etc). Upgrade of the transaction logic will likely be performed at the same time. The current code can report some false positive when .hgtags file changes but resulting tags are unchanged. This will be fixed in the next changeset. For testing, we simply globally enable a hook in the tag test as all the possible tag update cases should exist there. A couple of them show the false positive mentioned above. See in code documentation for more details.

tags: introduce a function to return a valid fnodes list from revs This will get used to compare tags between two set of revisions during a transaction (pre and post heads). The end goal is to be able to track tags movement in transaction hooks.

context: possibly yield initial fctx in blockdescendants() If initial 'fctx' has changes in line range with respect to its parents, we yield it first. This makes 'followlines(..., descend=True)' consistent with 'descendants()' revset which yields the starting revision. We reuse one iteration of blockancestors() which does exactly what we want. In test-annotate.t, adjust 'startrev' in one case to cover the situation where the starting revision does not touch specified line range.

context: add an assertion checking linerange consistency in blockdescendants() If this assertion fails, this indicates a flaw in the algorithm. So fail fast instead of possibly producing wrong results. Also extend the target line range in test to catch a merge changeset with all its parents.

windows: add win32com.shell to demandimport ignore list Module 'appdirs' tries to import win32com.shell (and catch ImportError as an indication of failure) to check whether some further functionality should be implemented one or another way [1]. Of course, demandimport lets it down, so if we want appdirs to work we have to add it to demandimport's ignore list. The reason we want appdirs to work is becuase it is used by setuptools [2] to determine egg cache location. Only fairly recent versions of setuptools depend on this so people don't see this often. [1] https://github.com/ActiveState/appdirs/blob/master/appdirs.py#L560 [2] https://github.com/pypa/setuptools/blob/aae0a928119d2a178882c32bded02270e30d0273/pkg_resources/__init__.py#L1369

obsolescence: add test for the "branch replacement" logic during push, case D6 Mercurial checks for the introduction of new heads on push. Evolution comes into play to detect if existing branches on the server are being replaced by some of the new one we push. The current code for this logic is very basic (eg: issue4354) and was poorly tested. We have a better implementation coming in the evolve extension fixing these issues and with more serious tests coverage. In the process of upstreaming this improved logic, we start with adding the test case that are already passing with the current implementation. Once they are all in, we'll upstream the better implementation and the extra test cases. See inline documentation for details about the test case added in this changeset.

obsolescence: add test dor the "branch replacement" logic during push, case D4 Mercurial checks for the introduction of new heads on push. Evolution comes into play to detect if existing branches on the server are being replaced by some of the new one we push. The current code for this logic is very basic (eg: issue4354) and was poorly tested. We have a better implementation coming in the evolve extension fixing these issues and with more serious tests coverage. In the process of upstreaming this improved logic, we start with adding the test case that are already passing with the current implementation. Once they are all in, we'll upstream the better implementation and the extra test cases. See inline documentation for details about the test case added in this changeset.

obsolescence: add test for the "branch replacement" logic during push, case D2 Mercurial checks for the introduction of new heads on push. Evolution comes into play to detect if existing branches on the server are being replaced by some of the new one we push. The current code for this logic is very basic (eg: issue4354) and was poorly tested. We have a better implementation coming in the evolve extension fixing these issues and with more serious tests coverage. In the process of upstreaming this improved logic, we start with adding the test case that are already passing with the current implementation. Once they are all in, we'll upstream the better implementation and the extra test cases. See inline documentation for details about the test case added in this changeset.

obsolescence: add test for the "branch replacement" logic during push, case D1 Mercurial checks for the introduction of new heads on push. Evolution comes into play to detect if existing branches on the server are being replaced by some of the new one we push. The current code for this logic is very basic (eg: issue4354) and was poorly tested. We have a better implementation coming in the evolve extension fixing these issues and with more serious tests coverage. In the process of upstreaming this improved logic, we start with adding the test case that are already passing with the current implementation. Once they are all in, we'll upstream the better implementation and the extra test cases. See inline documentation for details about the test case added in this changeset.

obsolescence: add test for the "branch replacement" logic during push, case C4 Mercurial checks for the introduction of new heads on push. Evolution comes into play to detect if existing branches on the server are being replaced by some of the new one we push. The current code for this logic is very basic (eg: issue4354) and was poorly tested. We have a better implementation coming in the evolve extension fixing these issues and with more serious tests coverage. In the process of upstreaming this improved logic, we start with adding the test case that are already passing with the current implementation. Once they are all in, we'll upstream the better implementation and the extra test cases. See inline documentation for details about the test case added in this changeset.

obsolescence: add test for the "branch replacement" logic during push, case C3 Mercurial checks for the introduction of new heads on push. Evolution comes into play to detect if existing branches on the server are being replaced by some of the new one we push. The current code for this logic is very basic (eg: issue4354) and was poorly tested. We have a better implementation coming in the evolve extension fixing these issues and with more serious tests coverage. In the process of upstreaming this improved logic, we start with adding the test case that are already passing with the current implementation. Once they are all in, we'll upstream the better implementation and the extra test cases. See inline documentation for details about the test case added in this changeset.

obsolescence: add test for the "branch replacement" logic during push, case C2 Mercurial checks for the introduction of new heads on push. Evolution comes into play to detect if existing branches on the server are being replaced by some of the new one we push. The current code for this logic is very basic (eg: issue4354) and was poorly tested. We have a better implementation coming in the evolve extension fixing these issues and with more serious tests coverage. In the process of upstreaming this improved logic, we start with adding the test case that are already passing with the current implementation. Once they are all in, we'll upstream the better implementation and the extra test cases. See inline documentation for details about the test case added in this changeset.

obsolescence: add test for the "branch replacement" logic during push, case B3 Mercurial checks for the introduction of new heads on push. Evolution comes into play to detect if existing branches on the server are being replaced by some of the new one we push. The current code for this logic is very basic (eg: issue4354) and was poorly tested. We have a better implementation coming in the evolve extension fixing these issues and with more serious tests coverage. In the process of upstreaming this improved logic, we start with adding the test case that are already passing with the current implementation. Once they are all in, we'll upstream the better implementation and the extra test cases. See inline documentation for details about the test case added in this changeset.

obsolescence: add test for the "branch replacement" logic during push, case A8 Mercurial checks for the introduction of new heads on push. Evolution comes into play to detect if existing branches on the server are being replaced by some of the new one we push. The current code for this logic is very basic (eg: issue4354) and was poorly tested. We have a better implementation coming in the evolve extension fixing these issues and with more serious tests coverage. In the process of upstreaming this improved logic, we start with adding the test case that are already passing with the current implementation. Once they are all in, we'll upstream the better implementation and the extra test cases. See inline documentation for details about the test case added in this changeset.

obsolescence: add test for the "branch replacement" logic during push, case A7 Mercurial checks for the introduction of new heads on push. Evolution comes into play to detect if existing branches on the server are being replaced by some of the new one we push. The current code for this logic is very basic (eg: issue4354) and was poorly tested. We have a better implementation coming in the evolve extension fixing these issues and with more serious tests coverage. In the process of upstreaming this improved logic, we start with adding the test case that are already passing with the current implementation. Once they are all in, we'll upstream the better implementation and the extra test cases. See inline documentation for details about the test case added in this changeset.

obsolescence: add test for the "branch replacement" logic during push, case A6 Mercurial checks for the introduction of new heads on push. Evolution comes into play to detect if existing branches on the server are being replaced by some of the new one we push. The current code for this logic is very basic (eg: issue4354) and was poorly tested. We have a better implementation coming in the evolve extension fixing these issues and with more serious tests coverage. In the process of upstreaming this improved logic, we start with adding the test case that are already passing with the current implementation. Once they are all in, we'll upstream the better implementation and the extra test cases. See inline documentation for details about the test case added in this changeset.

obsolescence: add test for the "branch replacement" logic during push, case A5 Mercurial checks for the introduction of new heads on push. Evolution comes into play to detect if existing branches on the server are being replaced by some of the new one we push. The current code for this logic is very basic (eg: issue4354) and was poorly tested. We have a better implementation coming in the evolve extension fixing these issues and with more serious tests coverage. In the process of upstreaming this improved logic, we start with adding the test case that are already passing with the current implementation. Once they are all in, we'll upstream the better implementation and the extra test cases. See inline documentation for details about the test case added in this changeset.

obsolescence: add test for the "branch replacement" logic during push, case A4 Mercurial checks for the introduction of new heads on push. Evolution comes into play to detect if existing branches on the server are being replaced by some of the new one we push. The current code for this logic is very basic (eg: issue4354) and was poorly tested. We have a better implementation coming in the evolve extension fixing these issues and with more serious tests coverage. In the process of upstreaming this improved logic, we start with adding the test case that are already passing with the current implementation. Once they are all in, we'll upstream the better implementation and the extra test cases. See inline documentation for details about the test case added in this changeset.

obsolescence: add test for the "branch replacement" logic during push, case A3 Mercurial checks for the introduction of new heads on push. Evolution comes into play to detect if existing branches on the server are being replaced by some of the new one we push. The current code for this logic is very basic (eg: issue4354) and was poorly tested. We have a better implementation coming in the evolve extension fixing these issues and with more serious tests coverage. In the process of upstreaming this improved logic, we start with adding the test case that are already passing with the current implementation. Once they are all in, we'll upstream the better implementation and the extra test case. See inline documentation for details about the test case added in this changeset.

obsolescence: add test for the "branch replacement" logic during push, case A2 Mercurial checks for the introduction of new heads on push. Evolution comes into play to detect if existing branches on the server are being replaced by some of the new one we push. The current code for this logic is very basic (eg: issue4354) and was poorly tested. We have a better implementation coming in the evolve extension fixing these issues and with more serious tests coverage. In the process of upstreaming this improved logic, we start with adding the test case that are already passing with the current implementation. Once they are all in, we'll upstream the better implementation and the extra test case. See inline documentation for details about the test case added in this changeset.

obsolescence: add test for the "branch replacement" logic during push, case A1 Mercurial checks for the introduction of new heads on push. Evolution comes into play to detect if existing branches on the server are being replaced by some of the new one we push. The current code for this logic is very basic (eg: issue4354) and was poorly tested. We have a better implementation coming in the evolve extension fixing these issues and with more serious tests coverage. In the process of upstreaming this improved logic, we start with adding the test case that are already passing with the current implementation. Once they are all in, we'll upstream the better implementation and the extra test case. See inline documentation for details about the test case added in this changeset.

obsolescence: add test utility for the "branch replacement" logic during push Mercurial checks for the introduction of new heads on push. Evolution comes into play to detect if existing branches on the server are being replaced by some of the new one we push. The current code for this logic is very basic (eg: issue4354) and was poorly tested. We have a better implementation coming in the evolve extension fixing these issues and with more serious tests coverage. In the process of upstreaming this improved logic, we start with adding the test case that are already passing with the current implementation. Once they are all in, we'll upstream the better implementation and the extra test case. This changeset introduce the common setup script used by these tests.

obsolescence: add test case D-4 for obsolescence markers exchange About 3 years ago, in August 2014, the logic to select what markers to select on push was ported from the evolve extension to Mercurial core. However, for some unclear reasons, the tests for that logic were not ported alongside. I realised it a couple of weeks ago while working on another push related issue. I've made a clean up pass on the tests and they are now ready to integrate the core test suite. This series of changesets do not change any logic. I just adds test for logic that has been around for about 10 versions of Mercurial. They are a patch for each test case. It makes it easier to review and postpone one with documentation issues without rejecting the wholes series. This patch introduce case D-4: unknown changeset in between known on Each test case comes it in own test file. It help parallelism and does not introduce a significant overhead from having a single unified giant test file. Here are timing to support this claim. # Multiple test files version: # run-tests.py --local -j 1 test-exchange-*.t 53.40s user 6.82s system 85% cpu 1:10.76 total 52.79s user 6.97s system 85% cpu 1:09.97 total 52.94s user 6.82s system 85% cpu 1:09.69 total # Single test file version: # run-tests.py --local -j 1 test-exchange-obsmarkers.t 52.97s user 6.85s system 85% cpu 1:10.10 total 52.64s user 6.79s system 85% cpu 1:09.63 total 53.70s user 7.00s system 85% cpu 1:11.17 total

obsolescence: add test case D-3 for obsolescence markers exchange About 3 years ago, in August 2014, the logic to select what markers to select on push was ported from the evolve extension to Mercurial core. However, for some unclear reasons, the tests for that logic were not ported alongside. I realised it a couple of weeks ago while working on another push related issue. I've made a clean up pass on the tests and they are now ready to integrate the core test suite. This series of changesets do not change any logic. I just adds test for logic that has been around for about 10 versions of Mercurial. They are a patch for each test case. It makes it easier to review and postpone one with documentation issues without rejecting the wholes series. This patch introduce case D3: missing prune target (prune not in "pushed set") Each test case comes it in own test file. It help parallelism and does not introduce a significant overhead from having a single unified giant test file. Here are timing to support this claim. # Multiple test files version: # run-tests.py --local -j 1 test-exchange-*.t 53.40s user 6.82s system 85% cpu 1:10.76 total 52.79s user 6.97s system 85% cpu 1:09.97 total 52.94s user 6.82s system 85% cpu 1:09.69 total # Single test file version: # run-tests.py --local -j 1 test-exchange-obsmarkers.t 52.97s user 6.85s system 85% cpu 1:10.10 total 52.64s user 6.79s system 85% cpu 1:09.63 total 53.70s user 7.00s system 85% cpu 1:11.17 total

obsolescence: add test case D-2 for obsolescence markers exchange About 3 years ago, in August 2014, the logic to select what markers to select on push was ported from the evolve extension to Mercurial core. However, for some unclear reasons, the tests for that logic were not ported alongside. I realised it a couple of weeks ago while working on another push related issue. I've made a clean up pass on the tests and they are now ready to integrate the core test suite. This series of changesets do not change any logic. I just adds test for logic that has been around for about 10 versions of Mercurial. They are a patch for each test case. It makes it easier to review and postpone one with documentation issues without rejecting the wholes series. This patch introduce D-2: missing prune target (prune in "pushed set") Each test case comes it in own test file. It help parallelism and does not introduce a significant overhead from having a single unified giant test file. Here are timing to support this claim. # Multiple test files version: # run-tests.py --local -j 1 test-exchange-*.t 53.40s user 6.82s system 85% cpu 1:10.76 total 52.79s user 6.97s system 85% cpu 1:09.97 total 52.94s user 6.82s system 85% cpu 1:09.69 total # Single test file version: # run-tests.py --local -j 1 test-exchange-obsmarkers.t 52.97s user 6.85s system 85% cpu 1:10.10 total 52.64s user 6.79s system 85% cpu 1:09.63 total 53.70s user 7.00s system 85% cpu 1:11.17 total

obsolescence: add test case D-1 for obsolescence markers exchange About 3 years ago, in August 2014, the logic to select what markers to select on push was ported from the evolve extension to Mercurial core. However, for some unclear reasons, the tests for that logic were not ported alongside. I realised it a couple of weeks ago while working on another push related issue. I've made a clean up pass on the tests and they are now ready to integrate the core test suite. This series of changesets do not change any logic. I just adds test for logic that has been around for about 10 versions of Mercurial. They are a patch for each test case. It makes it easier to review and postpone one with documentation issues without rejecting the wholes series. This patch introduce case D-1: Pruned changeset based on missing precursor of something not present Each test case comes it in own test file. It help parallelism and does not introduce a significant overhead from having a single unified giant test file. Here are timing to support this claim. # Multiple test files version: # run-tests.py --local -j 1 test-exchange-*.t 53.40s user 6.82s system 85% cpu 1:10.76 total 52.79s user 6.97s system 85% cpu 1:09.97 total 52.94s user 6.82s system 85% cpu 1:09.69 total # Single test file version: # run-tests.py --local -j 1 test-exchange-obsmarkers.t 52.97s user 6.85s system 85% cpu 1:10.10 total 52.64s user 6.79s system 85% cpu 1:09.63 total 53.70s user 7.00s system 85% cpu 1:11.17 total

obsolescence: add test case C-4 for obsolescence markers exchange About 3 years ago, in August 2014, the logic to select what markers to select on push was ported from the evolve extension to Mercurial core. However, for some unclear reasons, the tests for that logic were not ported alongside. I realised it a couple of weeks ago while working on another push related issue. I've made a clean up pass on the tests and they are now ready to integrate the core test suite. This series of changesets do not change any logic. I just adds test for logic that has been around for about 10 versions of Mercurial. They are a patch for each test case. It makes it easier to review and postpone one with documentation issues without rejecting the wholes series. This patch introduce C.4: multiple successors, one is pruned Each test case comes it in own test file. It help parallelism and does not introduce a significant overhead from having a single unified giant test file. Here are timing to support this claim. # Multiple test files version: # run-tests.py --local -j 1 test-exchange-*.t 53.40s user 6.82s system 85% cpu 1:10.76 total 52.79s user 6.97s system 85% cpu 1:09.97 total 52.94s user 6.82s system 85% cpu 1:09.69 total # Single test file version: # run-tests.py --local -j 1 test-exchange-obsmarkers.t 52.97s user 6.85s system 85% cpu 1:10.10 total 52.64s user 6.79s system 85% cpu 1:09.63 total 53.70s user 7.00s system 85% cpu 1:11.17 total

obsolescence: add test case C-3 for obsolescence markers exchange About 3 years ago, in August 2014, the logic to select what markers to select on push was ported from the evolve extension to Mercurial core. However, for some unclear reasons, the tests for that logic were not ported alongside. I realised it a couple of weeks ago while working on another push related issue. I've made a clean up pass on the tests and they are now ready to integrate the core test suite. This series of changesets do not change any logic. I just adds test for logic that has been around for about 10 versions of Mercurial. They are a patch for each test case. It makes it easier to review and postpone one with documentation issues without rejecting the wholes series. This patch introduce case C-3: Pruned changeset on precursors of another pruned one Each test case comes it in own test file. It help parallelism and does not introduce a significant overhead from having a single unified giant test file. Here are timing to support this claim. # Multiple test files version: # run-tests.py --local -j 1 test-exchange-*.t 53.40s user 6.82s system 85% cpu 1:10.76 total 52.79s user 6.97s system 85% cpu 1:09.97 total 52.94s user 6.82s system 85% cpu 1:09.69 total # Single test file version: # run-tests.py --local -j 1 test-exchange-obsmarkers.t 52.97s user 6.85s system 85% cpu 1:10.10 total 52.64s user 6.79s system 85% cpu 1:09.63 total 53.70s user 7.00s system 85% cpu 1:11.17 total

obsolescence: add test case C-2 for obsolescence markers exchange About 3 years ago, in August 2014, the logic to select what markers to select on push was ported from the evolve extension to Mercurial core. However, for some unclear reasons, the tests for that logic were not ported alongside. I realised it a couple of weeks ago while working on another push related issue. I've made a clean up pass on the tests and they are now ready to integrate the core test suite. This series of changesets do not change any logic. I just adds test for logic that has been around for about 10 versions of Mercurial. They are a patch for each test case. It makes it easier to review and postpone one with documentation issues without rejecting the wholes series. This patch introduce C-2: Pruned changeset on precursors Each test case comes it in own test file. It help parallelism and does not introduce a significant overhead from having a single unified giant test file. Here are timing to support this claim. # Multiple test files version: # run-tests.py --local -j 1 test-exchange-*.t 53.40s user 6.82s system 85% cpu 1:10.76 total 52.79s user 6.97s system 85% cpu 1:09.97 total 52.94s user 6.82s system 85% cpu 1:09.69 total # Single test file version: # run-tests.py --local -j 1 test-exchange-obsmarkers.t 52.97s user 6.85s system 85% cpu 1:10.10 total 52.64s user 6.79s system 85% cpu 1:09.63 total 53.70s user 7.00s system 85% cpu 1:11.17 total

obsolescence: add test case C-1 for obsolescence markers exchange About 3 years ago, in August 2014, the logic to select what markers to select on push was ported from the evolve extension to Mercurial core. However, for some unclear reasons, the tests for that logic were not ported alongside. I realised it a couple of weeks ago while working on another push related issue. I've made a clean up pass on the tests and they are now ready to integrate the core test suite. This series of changesets do not change any logic. I just adds test for logic that has been around for about 10 versions of Mercurial. They are a patch for each test case. It makes it easier to review and postpone one with documentation issues without rejecting the wholes series. This patch introduce C-1: Multiple pruned changeset atop each other Each test case comes it in own test file. It help parallelism and does not introduce a significant overhead from having a single unified giant test file. Here are timing to support this claim. # Multiple test files version: # run-tests.py --local -j 1 test-exchange-*.t 53.40s user 6.82s system 85% cpu 1:10.76 total 52.79s user 6.97s system 85% cpu 1:09.97 total 52.94s user 6.82s system 85% cpu 1:09.69 total # Single test file version: # run-tests.py --local -j 1 test-exchange-obsmarkers.t 52.97s user 6.85s system 85% cpu 1:10.10 total 52.64s user 6.79s system 85% cpu 1:09.63 total 53.70s user 7.00s system 85% cpu 1:11.17 total

stdio: add Linux-specific tests for error checking

stdio: raise StdioError if something goes wrong in ui.flush The prior code used to ignore all errors, which was intended to deal with a decade-old problem with writing to broken pipes on Windows. However, that code inadvertantly went a lot further, making it impossible to detect *all* I/O errors on stdio ... but only sometimes. What actually happened was that if Mercurial wrote less than a stdio buffer's worth of output (the overwhelmingly common case for most commands), any error that occurred would get swallowed here. But if the buffering strategy changed, an unhandled IOError could be raised from any number of other locations. Because we now have a top-level StdioError handler, and ui._write and ui._write_err (and now flush!) will raise that exception, we have one rational place to detect and handle these errors.

stdio: raise StdioError if something goes wrong in ui._write_err The prior code used to ignore certain classes of error, which was not the right thing to do.

stdio: raise StdioError if something goes wrong in ui._write

stdio: catch StdioError in dispatch.run and clean up appropriately We attempt to report what went wrong, and more importantly exit the program with an error code. (The exception we catch is not yet raised anywhere in the code.)

stdio: add machinery to identify failed stdout/stderr writes Mercurial currently fails to notice failures to write to stdout or stderr. A correctly functioning command line tool should detect this and exit with an error code. To achieve this, we need a little extra plumbing, which we start adding here.

atexit: switch to home-grown implementation

atexit: test failing handlers

ui: add special-purpose atexit functionality In spite of its longstanding use, Python's built-in atexit code is not suitable for Mercurial's purposes, for several reasons: * Handlers run after application code has finished. * Because of this, the code that runs handlers swallows exceptions (since there's no possible stacktrace to associate errors with). If we're lucky, we'll get something spat out to stderr (if stderr still works), which of course isn't any use in a big deployment where it's important that exceptions get logged and aggregated. * Mercurial's current atexit handlers make unfortunate assumptions about process state (specifically stdio) that, coupled with the above problems, make it impossible to deal with certain categories of error (try "hg status > /dev/full" on a Linux box). * In Python 3, the atexit implementation is completely hidden, so we can't hijack the platform's atexit code to run handlers at a time of our choosing. As a result, here's a perfectly cromulent atexit-like implementation over which we have control. This lets us decide exactly when the handlers run (after each request has completed), and control what the process state is when that occurs (and afterwards).

context: follow all branches in blockdescendants() In the initial implementation of blockdescendants (and thus followlines(..., descend=True) revset), only the first branch encountered in descending direction was followed. Update the algorithm so that all children of a revision ('x' in code) are considered. Accordingly, we need to prevent a child revision to be yielded multiple times when it gets visited through different path, so we skip 'i' when this occurs. Finally, since we now consider all parents of a possible child touching a given line range, we take care of yielding the child if it has a diff in specified line range with at least one of its parent (same logic as blockancestors()).

pager: set some environment variables if they're not set Git did this already [1] [2]. We want this behavior too [3]. This provides a better default user experience (like, supporting colors) if users have things like "PAGER=less" set, which is not uncommon. The environment variables are provided by a method so extensions can override them on demand. [1]: https://github.com/git/git/blob/6a5ff7acb5965718cc7016c0ab6c601454fd7cde/pager.c#L87 [2]: https://github.com/git/git/blob/6a5ff7acb5965718cc7016c0ab6c601454fd7cde/Makefile#L1545 [3]: https://www.mercurial-scm.org/pipermail/mercurial-devel/2017-March/094780.html

sshpeer: fix docstring typo

util: pass sysstrs to warnings.filterwarnings Un-breaks the Python 3 build.

vfs: deprecate all old classes in scmutil Now that all vfs class moved to the vfs module, we can deprecate the old one.

util: add a way to issue deprecation warning without a UI object Our current deprecation warning mechanism relies on ui object. They are case where we cannot have access to the UI object. On a general basis we avoid using the python mechanism for deprecation warning because up to Python 2.6 it is exposing warning to unsuspecting user who cannot do anything to deal with them. So we build a "safe" strategy to hide this warnings behind a flag in an environment variable. The test runner set this flag so that tests show these warning. This will help us marker API as deprecated for extensions to update their code.

gitweb: plug followlines UI in filerevision view Mostly copy CSS rules from style-paper.css into style-gitweb.css. The only modification is addition of !important on "background-color" rule for "pre.sourcelines > span.followlines-selected" selector as the background color is otherwise overriden by "pre.sourcelines.stripes > :nth-child(4n+4)" rule.

gitweb: handle "patch" query parameter in filelog view As for paper style, in f36dc643ffdc, we display "diff" data as an additional row in the table of revision entries for the gitweb template. Also, as these additional diff rows have a white background, they may be confused with log entry rows ("age", "author", "description", "links") of even parity (parity0 also have a white background). So we disable parity colors for log entry rows when diff is displayed and fix the color to the "dark" parity (i.e. parity1 #f6f6f0) so that it's always distinguishable from

gitweb: add information about "linerange" filtering in filelog view As for paper style, in 5e6d44511317, we display a "(following lines <fromline>:<toline> <a href='...'>back to filelog</a>)" message alongside the file name when "linerange" query parameter is present.

util: fix human-readable printing of negative byte counts Apply the same human-readable printing rules to negative byte counts as to positive ones. Fixes output of debugupgraderepo.

show: make template option actually show up in help Previously, the --template/-T option didn't show up in help because it's marked as experimental. It's not really experimental for show, and its quite important for show's funcationality, so let's make sure it always shows up.

show: implement underway view This is the beginning of a wip/smartlog view. It is basically a manually constructed (read: fast) revset function to collect "relevant" changesets combined with a custom template and a graph displayer. It obviously needs a lot of work. I'd like to get *something* usable in 4.2 so `hg show` has some value to end-users. Let the bikeshedding begin.

show: fix formatting of multiple commands Because we're formatting to RST, short lines wrap and there needs to be an extra line break between paragraphs to prevent that. In addition, the indentation in the old code was a bit off. Refactor the code to a function (so we don't leak variables outside the module) and modify it so it renders more correctly.

pycompat: import correct cookie module on Python 3 http.cookielib doesn't exist. http.cookiejar does and it contains the symbols we need. This fixes test failures on Python 3.

chg: respect environment variables for pager Previously chg runs the pager command without respecting its environment variables being told to use. This patch makes it so.

hgweb: add a link to followlines in descending direction We change the content of the followlines popup to display two links inviting to follow the history of selected lines in ascending (as before) and descending directions. The popup now renders as: follow history of lines <fromline>:<toline>: <a href=...>ascending</a> / <a href=...>descending</a>

hgweb: handle a "descend" query parameter in filelog command When this "descend" query parameter is present along with "linerange" parameter, we get revisions following line range in descending order. The parameter has no effect without "linerange".

revset: add a 'descend' argument to followlines to return descendants This is useful to follow changes in a block of lines forward in the history (for instance, when one wants to find out how a function evolved from a point in history). We added a 'descend' parameter to followlines(), which defaults to False. If True, followlines() returns descendants of startrev. Because context.blockdescendants() does not follow renames, these are not followed by the revset either, so history will end when a rename occurs (as can be seen in tests).

context: add a blockdescendants function This is symmetrical with blockancestors() and yields descendants of a filectx with changes in the given line range. The noticeable difference is that the algorithm does not follow renames (probably because filelog.descendants() does not), so we are missing branches with renames.

url: support auth.cookiesfile for adding cookies to HTTP requests Mercurial can't currently send cookies as part of HTTP requests. Some authentication systems use cookies. So, it seems like adding support for sending cookies seems like a useful feature. This patch implements support for reading cookies from a file and automatically sending them as part of the request. We rely on the "cookiejar" Python module to do the heavy lifting of parsing cookies files. We currently only support the Mozilla (really Netscape-era) cookie format. There is another format supported by cookielib and we may want to consider using that, especially since the Netscape cookie parser can't parse ports. It wasn't immediately obvious to me what the format of the other parser is, so I didn't know how to test it. I /think/ it might be literal "Cookie" header values, but I'm not sure. If it is more robust than the Netscape format, we may want to just support it.

httpconnection: allow a global auth.cookiefile config entry This foreshadows support for defining a cookies file.

util: make cookielib module available In preparation for supporting sending cookies on HTTP requests.

crecord: avoid setting non-existing SIGTSTP signal on windows (issue5512) Windows do not have a SIGTSTP so we avoid setting the handler if the signal is unknown.

crecord: ensure we reinstall the SIGTSTP handler Previous, exceptions would prevent the reinstallation of the signal.

crecord: avoid setting non-existing signal SIGWINCH on windows Windows do not have a SIGWINCH so we avoid setting the handler if the signal is unknown.

crecord: ensure we reinstall the SIGWINCH handler Previous, exception in _main(...) would prevent the reinstallation of the signal.

crecord: extract most of 'main' into a sub function There are some setup and cleanup necessary around the main code, that setup/cleanup code needs multiple adjustments so we extract the core code into its own function first for clarity.

templater: add shorthand for building a dict like {"key": key} Like field init shorthand of Rust. This is convenient for building a JSON object from selected keywords. This means dict() won't support Python-like dict(iterable) syntax because it's ambiguous. Perhaps it could be implemented as 'mapdict(xs % (k, v))'.

templater: find keyword name more thoroughly on filtering error Before, it could spill an internal representation of compiled template such as [(<function runsymbol at 0x....>, 'extras'), ...]. Show less cryptic message if no symbol found. New findsymbolicname() function will be also used by dict() constructor.

templater: add dict() constructor It's troublesome to build JSON by template, so let's add programmatic way.

templatekw: add public function to wrap a dict by _hybrid object

templatekw: add public function to wrap a list by _hybrid object

templatekw: add default implementation of _hybrid.gen This is convenient for new template keyword, which doesn't need to support the legacy list hack (provided by _showlist()), but still wants to have a string representation.

parser: preserve order of keyword arguments This helps building dict(key1=value1, ...) in deterministic way.

parser: extend buildargsdict() to support arbitrary number of **kwargs Prepares for adding dict(key1=value1, ...) template function. More tests will be added later.

parser: verify excessive number of args excluding kwargs in buildargsdict() This makes the next patch slightly simpler. We don't need to check the excessive number of keyword arguments since unknown and duplicated kwargs are rejected.

obsolescence: add test case B-7 for obsolescence markers exchange About 3 years ago, in August 2014, the logic to select what markers to select on push was ported from the evolve extension to Mercurial core. However, for some unclear reasons, the tests for that logic were not ported alongside. I realised it a couple of weeks ago while working on another push related issue. I've made a clean up pass on the tests and they are now ready to integrate the core test suite. This series of changesets do not change any logic. I just adds test for logic that has been around for about 10 versions of Mercurial. They are a patch for each test case. It makes it easier to review and postpone one with documentation issues without rejecting the wholes series. This patch introduce case B-7: Prune above non-targeted common changeset Each test case comes it in own test file. It help parallelism and does not introduce a significant overhead from having a single unified giant test file. Here are timing to support this claim. # Multiple test files version: # run-tests.py --local -j 1 test-exchange-*.t 53.40s user 6.82s system 85% cpu 1:10.76 total 52.79s user 6.97s system 85% cpu 1:09.97 total 52.94s user 6.82s system 85% cpu 1:09.69 total # Single test file version: # run-tests.py --local -j 1 test-exchange-obsmarkers.t 52.97s user 6.85s system 85% cpu 1:10.10 total 52.64s user 6.79s system 85% cpu 1:09.63 total 53.70s user 7.00s system 85% cpu 1:11.17 total

obsolescence: add test case B-6 for obsolescence markers exchange About 3 years ago, in August 2014, the logic to select what markers to select on push was ported from the evolve extension to Mercurial core. However, for some unclear reasons, the tests for that logic were not ported alongside. I realised it a couple of weeks ago while working on another push related issue. I've made a clean up pass on the tests and they are now ready to integrate the core test suite. This series of changesets do not change any logic. I just adds test for logic that has been around for about 10 versions of Mercurial. They are a patch for each test case. It makes it easier to review and postpone one with documentation issues without rejecting the wholes series. This patch introduce case B6: Pruned changeset with precursors not in pushed set Each test case comes it in own test file. It help parallelism and does not introduce a significant overhead from having a single unified giant test file. Here are timing to support this claim. # Multiple test files version: # run-tests.py --local -j 1 test-exchange-*.t 53.40s user 6.82s system 85% cpu 1:10.76 total 52.79s user 6.97s system 85% cpu 1:09.97 total 52.94s user 6.82s system 85% cpu 1:09.69 total # Single test file version: # run-tests.py --local -j 1 test-exchange-obsmarkers.t 52.97s user 6.85s system 85% cpu 1:10.10 total 52.64s user 6.79s system 85% cpu 1:09.63 total 53.70s user 7.00s system 85% cpu 1:11.17 total

obsolescence: add test case B-5 for obsolescence markers exchange About 3 years ago, in August 2014, the logic to select what markers to select on push was ported from the evolve extension to Mercurial core. However, for some unclear reasons, the tests for that logic were not ported alongside. I realised it a couple of weeks ago while working on another push related issue. I've made a clean up pass on the tests and they are now ready to integrate the core test suite. This series of changesets do not change any logic. I just adds test for logic that has been around for about 10 versions of Mercurial. They are a patch for each test case. It makes it easier to review and postpone one with documentation issues without rejecting the wholes series. This patch introduce B-5: Push of a children of changeset which successors is pruned Each test case comes it in own test file. It help parallelism and does not introduce a significant overhead from having a single unified giant test file. Here are timing to support this claim. # Multiple test files version: # run-tests.py --local -j 1 test-exchange-*.t 53.40s user 6.82s system 85% cpu 1:10.76 total 52.79s user 6.97s system 85% cpu 1:09.97 total 52.94s user 6.82s system 85% cpu 1:09.69 total # Single test file version: # run-tests.py --local -j 1 test-exchange-obsmarkers.t 52.97s user 6.85s system 85% cpu 1:10.10 total 52.64s user 6.79s system 85% cpu 1:09.63 total 53.70s user 7.00s system 85% cpu 1:11.17 total

obsolescence: add test case B-4 for obsolescence markers exchange About 3 years ago, in August 2014, the logic to select what markers to select on push was ported from the evolve extension to Mercurial core. However, for some unclear reasons, the tests for that logic were not ported alongside. I realised it a couple of weeks ago while working on another push related issue. I've made a clean up pass on the tests and they are now ready to integrate the core test suite. This series of changesets do not change any logic. I just adds test for logic that has been around for about 10 versions of Mercurial. They are a patch for each test case. It makes it easier to review and postpone one with documentation issues without rejecting the wholes series. This patch introduce case B4: Pruned changeset on common part of history Each test case comes it in own test file. It help parallelism and does not introduce a significant overhead from having a single unified giant test file. Here are timing to support this claim. # Multiple test files version: # run-tests.py --local -j 1 test-exchange-*.t 53.40s user 6.82s system 85% cpu 1:10.76 total 52.79s user 6.97s system 85% cpu 1:09.97 total 52.94s user 6.82s system 85% cpu 1:09.69 total # Single test file version: # run-tests.py --local -j 1 test-exchange-obsmarkers.t 52.97s user 6.85s system 85% cpu 1:10.10 total 52.64s user 6.79s system 85% cpu 1:09.63 total 53.70s user 7.00s system 85% cpu 1:11.17 total

obsolescence: add test case B-3 for obsolescence markers exchange About 3 years ago, in August 2014, the logic to select what markers to select on push was ported from the evolve extension to Mercurial core. However, for some unclear reasons, the tests for that logic were not ported alongside. I realised it a couple of weeks ago while working on another push related issue. I've made a clean up pass on the tests and they are now ready to integrate the core test suite. This series of changesets do not change any logic. I just adds test for logic that has been around for about 10 versions of Mercurial. They are a patch for each test case. It makes it easier to review and postpone one with documentation issues without rejecting the wholes series. This patch introduce case B3: Pruned changeset on non-pushed part of the history Each test case comes it in own test file. It help parallelism and does not introduce a significant overhead from having a single unified giant test file. Here are timing to support this claim. # Multiple test files version: # run-tests.py --local -j 1 test-exchange-*.t 53.40s user 6.82s system 85% cpu 1:10.76 total 52.79s user 6.97s system 85% cpu 1:09.97 total 52.94s user 6.82s system 85% cpu 1:09.69 total # Single test file version: # run-tests.py --local -j 1 test-exchange-obsmarkers.t 52.97s user 6.85s system 85% cpu 1:10.10 total 52.64s user 6.79s system 85% cpu 1:09.63 total 53.70s user 7.00s system 85% cpu 1:11.17 total

obsolescence: add test case B-2 for obsolescence markers exchange About 3 years ago, in August 2014, the logic to select what markers to select on push was ported from the evolve extension to Mercurial core. However, for some unclear reasons, the tests for that logic were not ported alongside. I realised it a couple of weeks ago while working on another push related issue. I've made a clean up pass on the tests and they are now ready to integrate the core test suite. This series of changesets do not change any logic. I just adds test for logic that has been around for about 10 versions of Mercurial. They are a patch for each test case. It makes it easier to review and postpone one with documentation issues without rejecting the wholes series. This patch introduce case B-2: Prune on targeted common changeset Each test case comes it in own test file. It help parallelism and does not introduce a significant overhead from having a single unified giant test file. Here are timing to support this claim. # Multiple test files version: # run-tests.py --local -j 1 test-exchange-*.t 53.40s user 6.82s system 85% cpu 1:10.76 total 52.79s user 6.97s system 85% cpu 1:09.97 total 52.94s user 6.82s system 85% cpu 1:09.69 total # Single test file version: # run-tests.py --local -j 1 test-exchange-obsmarkers.t 52.97s user 6.85s system 85% cpu 1:10.10 total 52.64s user 6.79s system 85% cpu 1:09.63 total 53.70s user 7.00s system 85% cpu 1:11.17 total

obsolescence: add test case B-1 for obsolescence markers exchange About 3 years ago, in August 2014, the logic to select what markers to select on push was ported from the evolve extension to Mercurial core. However, for some unclear reasons, the tests for that logic were not ported alongside. I realised it a couple of weeks ago while working on another push related issue. I've made a clean up pass on the tests and they are now ready to integrate the core test suite. This series of changesets do not change any logic. I just adds test for logic that has been around for about 10 versions of Mercurial. They are a patch for each test case. It makes it easier to review and postpone one with documentation issues without rejecting the wholes series. This patch introduce case B-1: Prune on non-targeted common changeset Each test case comes it in own test file. It help parallelism and does not introduce a significant overhead from having a single unified giant test file. Here are timing to support this claim. # Multiple test files version: # run-tests.py --local -j 1 test-exchange-*.t 53.40s user 6.82s system 85% cpu 1:10.76 total 52.79s user 6.97s system 85% cpu 1:09.97 total 52.94s user 6.82s system 85% cpu 1:09.69 total # Single test file version: # run-tests.py --local -j 1 test-exchange-obsmarkers.t 52.97s user 6.85s system 85% cpu 1:10.10 total 52.64s user 6.79s system 85% cpu 1:09.63 total 53.70s user 7.00s system 85% cpu 1:11.17 total

obsolescence: add test case A-7 for obsolescence markers exchange About 3 years ago, in August 2014, the logic to select what markers to select on push was ported from the evolve extension to Mercurial core. However, for some unclear reasons, the tests for that logic were not ported alongside. I realised it a couple of weeks ago while working on another push related issue. I've made a clean up pass on the tests and they are now ready to integrate the core test suite. This series of changesets do not change any logic. I just adds test for logic that has been around for about 10 versions of Mercurial. They are a patch for each test case. It makes it easier to review and postpone one with documentation issues without rejecting the wholes series. This patch introduce case A-7: non targeted common changeset Each test case comes it in own test file. It help parallelism and does not introduce a significant overhead from having a single unified giant test file. Here are timing to support this claim. # Multiple test files version: # run-tests.py --local -j 1 test-exchange-*.t 53.40s user 6.82s system 85% cpu 1:10.76 total 52.79s user 6.97s system 85% cpu 1:09.97 total 52.94s user 6.82s system 85% cpu 1:09.69 total # Single test file version: # run-tests.py --local -j 1 test-exchange-obsmarkers.t 52.97s user 6.85s system 85% cpu 1:10.10 total 52.64s user 6.79s system 85% cpu 1:09.63 total 53.70s user 7.00s system 85% cpu 1:11.17 total

obsolescence: add test case A-6 for obsolescence markers exchange About 3 years ago, in August 2014, the logic to select what markers to select on push was ported from the evolve extension to Mercurial core. However, for some unclear reasons, the tests for that logic were not ported alongside. I realised it a couple of weeks ago while working on another push related issue. I've made a clean up pass on the tests and they are now ready to integrate the core test suite. This series of changesets do not change any logic. I just adds test for logic that has been around for about 10 versions of Mercurial. They are a patch for each test case. It makes it easier to review and postpone one with documentation issues without rejecting the wholes series. This patch introduce case A6: new markers between changesets already known on both side Each test case comes it in own test file. It help parallelism and does not introduce a significant overhead from having a single unified giant test file. Here are timing to support this claim. # Multiple test files version: # run-tests.py --local -j 1 test-exchange-*.t 53.40s user 6.82s system 85% cpu 1:10.76 total 52.79s user 6.97s system 85% cpu 1:09.97 total 52.94s user 6.82s system 85% cpu 1:09.69 total # Single test file version: # run-tests.py --local -j 1 test-exchange-obsmarkers.t 52.97s user 6.85s system 85% cpu 1:10.10 total 52.64s user 6.79s system 85% cpu 1:09.63 total 53.70s user 7.00s system 85% cpu 1:11.17 total

obsolescence: add test case A-5 for obsolescence markers exchange About 3 years ago, in August 2014, the logic to select what markers to select on push was ported from the evolve extension to Mercurial core. However, for some unclear reasons, the tests for that logic were not ported alongside. I realised it a couple of weeks ago while working on another push related issue. I've made a clean up pass on the tests and they are now ready to integrate the core test suite. This series of changesets do not change any logic. I just adds test for logic that has been around for about 10 versions of Mercurial. They are a patch for each test case. It makes it easier to review and postpone one with documentation issues without rejecting the wholes series. This patch introduce case A-5: partial reordering Each test case comes it in own test file. It help parallelism and does not introduce a significant overhead from having a single unified giant test file. Here are timing to support this claim. # Multiple test files version: # run-tests.py --local -j 1 test-exchange-*.t 53.40s user 6.82s system 85% cpu 1:10.76 total 52.79s user 6.97s system 85% cpu 1:09.97 total 52.94s user 6.82s system 85% cpu 1:09.69 total # Single test file version: # run-tests.py --local -j 1 test-exchange-obsmarkers.t 52.97s user 6.85s system 85% cpu 1:10.10 total 52.64s user 6.79s system 85% cpu 1:09.63 total 53.70s user 7.00s system 85% cpu 1:11.17 total

obsolescence: add test case A-4 for obsolescence markers exchange About 3 years ago, in August 2014, the logic to select what markers to select on push was ported from the evolve extension to Mercurial core. However, for some unclear reasons, the tests for that logic were not ported alongside. I realised it a couple of weeks ago while working on another push related issue. I've made a clean up pass on the tests and they are now ready to integrate the core test suite. This series of changesets do not change any logic. I just adds test for logic that has been around for about 10 versions of Mercurial. They are a patch for each test case. It makes it easier to review and postpone one with documentation issues without rejecting the wholes series. This patch introduce case A-4: Push in the middle of the obsolescence chain Each test case comes it in own test file. It help parallelism and does not introduce a significant overhead from having a single unified giant test file. Here are timing to support this claim. # Multiple test files version: # run-tests.py --local -j 1 test-exchange-*.t 53.40s user 6.82s system 85% cpu 1:10.76 total 52.79s user 6.97s system 85% cpu 1:09.97 total 52.94s user 6.82s system 85% cpu 1:09.69 total # Single test file version: # run-tests.py --local -j 1 test-exchange-obsmarkers.t 52.97s user 6.85s system 85% cpu 1:10.10 total 52.64s user 6.79s system 85% cpu 1:09.63 total 53.70s user 7.00s system 85% cpu 1:11.17 total

obsolescence: add test case A-3 for obsolescence markers exchange About 3 years ago, in August 2014, the logic to select what markers to select on push was ported from the evolve extension to Mercurial core. However, for some unclear reasons, the tests for that logic were not ported alongside. I realised it a couple of weeks ago while working on another push related issue. I've made a clean up pass on the tests and they are now ready to integrate the core test suite. This series of changesets do not change any logic. I just adds test for logic that has been around for about 10 versions of Mercurial. They are a patch for each test case. It makes it easier to review and postpone one with documentation issues without rejecting the wholes series. This patch introduce case A3: new branchs created, one pushed Each test case comes it in own test file. It help parallelism and does not introduce a significant overhead from having a single unified giant test file. Here are timing to support this claim. # Multiple test files version: # run-tests.py --local -j 1 test-exchange-*.t 53.40s user 6.82s system 85% cpu 1:10.76 total 52.79s user 6.97s system 85% cpu 1:09.97 total 52.94s user 6.82s system 85% cpu 1:09.69 total # Single test file version: # run-tests.py --local -j 1 test-exchange-obsmarkers.t 52.97s user 6.85s system 85% cpu 1:10.10 total 52.64s user 6.79s system 85% cpu 1:09.63 total 53.70s user 7.00s system 85% cpu 1:11.17 total

obsolescence: add test case A-2 for obsolescence markers exchange About 3 years ago, in August 2014, the logic to select what markers to select on push was ported from the evolve extension to Mercurial core. However, for some unclear reasons, the tests for that logic were not ported alongside. I realised it a couple of weeks ago while working on another push related issue. I've made a clean up pass on the tests and they are now ready to integrate the core test suite. This series of changesets do not change any logic. I just adds test for logic that has been around for about 10 versions of Mercurial. They are a patch for each test case. It makes it easier to review and postpone one with documentation issues without rejecting the wholes series. This patch introduce case A-2: Two heads, only one of them pushed Each test case comes it in own test file. It help parallelism and does not introduce a significant overhead from having a single unified giant test file. Here are timing to support this claim. # Multiple test files version: # run-tests.py --local -j 1 test-exchange-*.t 53.40s user 6.82s system 85% cpu 1:10.76 total 52.79s user 6.97s system 85% cpu 1:09.97 total 52.94s user 6.82s system 85% cpu 1:09.69 total # Single test file version: # run-tests.py --local -j 1 test-exchange-obsmarkers.t 52.97s user 6.85s system 85% cpu 1:10.10 total 52.64s user 6.79s system 85% cpu 1:09.63 total 53.70s user 7.00s system 85% cpu 1:11.17 total

obsolescence: add test case A-1 for obsolescence markers exchange About 3 years ago, in August 2014, the logic to select what markers to select on push was ported from the evolve extension to Mercurial core. However, for some unclear reasons, the tests for that logic were not ported alongside. I realised it a couple of weeks ago while working on another push related issue. I've made a clean up pass on the tests and they are now ready to integrate the core test suite. This series of changesets do not change any logic. I just adds test for logic that has been around for about 10 versions of Mercurial. They are a patch for each test case. It makes it easier to review and postpone one with documentation issues without rejecting the wholes series. This patch introduce case A-1: pushing a single head Each test case comes it in own test file. It help parallelism and does not introduce a significant overhead from having a single unified giant test file. Here are timing to support this claim. # Multiple test files version: # run-tests.py --local -j 1 test-exchange-*.t 53.40s user 6.82s system 85% cpu 1:10.76 total 52.79s user 6.97s system 85% cpu 1:09.97 total 52.94s user 6.82s system 85% cpu 1:09.69 total # Single test file version: # run-tests.py --local -j 1 test-exchange-obsmarkers.t 52.97s user 6.85s system 85% cpu 1:10.10 total 52.64s user 6.79s system 85% cpu 1:09.63 total 53.70s user 7.00s system 85% cpu 1:11.17 total

obsolescence: add setup script for obsolescence markers exchange tests About 3 years ago, in August 2014, the logic to select what markers to select on push was ported from the evolve extension to Mercurial core. However, for some unclear reasons, the tests for that logic were not ported alongside. I realised it a couple of weeks ago while working on another push related issue. I've made a clean up pass on the tests and they are now ready to integrate the core test suite. This series of changesets do not change any logic. I just adds test for logic that has been around for about 10 versions of Mercurial. They are a patch for each test case. It makes it easier to review and postpone one with documentation issues without rejecting the wholes series. This patch introduce the common script that setup the basic environment for the test cases. Once this script is in. We can accept the other patches in any order. Each test case comes it in own test file. It help parallelism and does not introduce a significant overhead from having a single unified giant test file. Here are timing to support this claim. # Multiple test files version: # run-tests.py --local -j 1 test-exchange-*.t 53.40s user 6.82s system 85% cpu 1:10.76 total 52.79s user 6.97s system 85% cpu 1:09.97 total 52.94s user 6.82s system 85% cpu 1:09.69 total # Single test file version: # run-tests.py --local -j 1 test-exchange-obsmarkers.t 52.97s user 6.85s system 85% cpu 1:10.10 total 52.64s user 6.79s system 85% cpu 1:09.63 total 53.70s user 7.00s system 85% cpu 1:11.17 total

upgrade: simplify the "origin" dispatch in dry run We could compute the final set we need directly.

upgrade: use 'improvement' object for action too This simplify multiple pieces of code. For now we restrict this upgrade to the top level function to keep this patch simple.

upgrade: implement equality for 'improvement' object Through the code, we use a mix of 'improvement' object and string. Having a single type would be simpler. For this we need the object to be comparable.

upgrade: simplify some of the initial dispatch for dry run Since we already have the list of deficiencies, we can use it directly.

upgrade: simplify 'determineactions' Since we only takes 'deficiencies', we can simplify the function and clarify its arguments.

upgrade: filter optimizations outside of 'determineactions' This sounds like higher level logic to process arguments. Moving it out of 'determineactions' will allow passing only deficiencies to the function. Then, in a future changeset, we will remove dispatch on "improvement type" within the function. See next changeset for details.

upgrade: directly iterate over optimisations Since we already have the list of optimisations independent from the deficiencies, we can use it directly. (we make a dual assignement in this changeset to simplify the next one)

upgrade: simplify optimisations validation Since we fetch optimizations distinctly from the deficiencies, we can simplify some code.

upgrade: split finding deficiencies from finding optimisations Our ultimate goal is to make it easier to get a diagnostic of the repository format. A first important and step for that is to separate part related to repository format from the optimisation. We start by having two different functions returning the two categories of possible "improvement".

upgrade: update the copyright statement

upgrade: update the header comment

upgrade: import 'localrepo' globally The in-function imports mention a cycle that seems to no longer be relevant. As a result, we just import it globally.

largefiles: set the extension as enabled locally after a share requiring it This has been done for clone since e1dbe0b215ae, so it makes sense here for the same reasons.

windows: add context manager support to mixedfilemodewrapper I stumbled into this in the next patch. The difference between getting a context manager capable object or not from vfs classes was as subtle as adding a '+' to the file mode.

chg: always wait for pager Previously, when runcommand raises, chg aborts with, and does not wait for pager. The call stack is like: hgc_runcommand -> handleresponse -> readchannel -> debugmsg("failed to read channel") -> exit(255) That means, chg returns to the shell, then both the pager and the shell will read from the terminal at the same time, causing problems. This patch fixes that by using "atexit" to register the pager cleanup function so chg will always wait for pager even if runcommand raises.

bundle2: move 'seek' and 'tell' methods off the unpackermixin class These methods are unrelated to unpacking. They are used internally by the 'unbundlepart' class only. So me move them there as private methods. In the same go, we clarify their internal role in the their docstring.

shelve: rename nodestoprune to nodestoremove As per feedback from the community.

templater: port pad() to take keyword arguments This is another example where keyword arguments can be actually useful.

templater: add support for keyword arguments Unlike revset, function arguments are pre-processed in templater. That's why we need to define argspec per function. An argspec field looks somewhat redundant in @templatefunc definition as a name field contains human-readable list of arguments. I'll make function doc be built from argspec later. Ported separate() function as an example.

templater: add parsing rule for key-value pair Based on the revset implementation, 70a2082f855a. This patch also adjusts the test as '=' is now a valid token.

templater: adjust binding strengths to make room for key-value operator Changed as follows: - template ops (%, |): +10 - arithmetic ops: +1 (but "negate" should be greater than "%")

templater: sort token table by binding strength Just for readability.

templater: make _hybrid provide more list/dict-like methods So the JSON filter works.

templater: hide private variable of _hybrid

templater: remove __iter__() from _hybrid, resolve it explicitly The goal is to fix "{hybrid_obj|json}" output. A _hybrid object must act as a list or a dict as well as a generator of legacy template strings. Before, _hybrid.__iter__() was assigned for legacy template, which conflicted with list.__iter__() API. This patch drops _hybrid.__iter__() and makes stringify/flatten functions unwrap a generator instead.

templater: rewrite doc of _hybrid class as docstring

sortdict: fix .pop() to return a value My future patch will need it.

checkcode: only match pushd/popd as word otherwise it partially match common words like "pushdestination".

treemanifest: add walksubtrees api Adds a new function to treemanifest that allows walking over the directories in the tree. Currently it only accepts a matcher to prune the walk, but in the future it will also accept a list of trees and will only walk over subtrees that differ from the versions in the list. This will be useful for identifying what parts of the tree are new to this revision, which is useful when deciding the minimal set of trees to send to a client given that they have a certain tree already. Since this is intended for an extension to use, the only current consumer is a test. In the future this function may be useful for implementing other algorithms like diff and changegroup generation.

merge with stable

upgrade: drop the prefix to the '_finishdatamigration' function Now that we are in the 'upgrade' module we can simplify the name.

upgrade: drop the prefix to the '_filterstorefile' function Now that we are in the 'upgrade' module we can simplify the name.

upgrade: drop the prefix to the 'determineactions' function Now that we are in the 'upgrade' module we can simplify the name.

upgrade: drop the prefix to the 'findimprovements' function Now that we are in the 'upgrade' module we can simplify the name.

upgrade: drop the prefix to the 'supporteddestrequirements' function Now that we are in the 'upgrade' module we can simplify the name.

upgrade: drop the prefix to the 'allowednewrequirements' function Now that we are in the 'upgrade' module we can simplify the name.

upgrade: drop the prefix to the 'improvement' class Now that we are in the 'upgrade' module we can simplify the name.

upgrade: drop the prefix to the 'supportremovedrequirements' function Now that we are in the 'upgrade' module we can simplify the name.

upgrade: drop the prefix to the 'blocksourcerequirements' function Now that we are in the 'upgrade' module we can simplify the name.

upgrade: drop the prefix to the 'requiredsourcerequirements' function Now that we are in the 'upgrade' module we can simplify the name.

upgrade: extract code in its own module Given about 2/3 or 'mercurial.repair' is now about repository upgrade, I think it is fair to move it into its own module. An expected benefit is the ability to drop the 'upgrade' prefix of many functions. This will be done in coming changesets.

bundle2: move the 'close' method off the unpackermixin This method is unrelated to unpacking and only used by the unbundle20 class. We move it there for clarity.

bundle2: clarify the docstring of unpackermixin methods The unpackermixin is a utility used to implement the bundle2 protocol. It should not be used when writing part handlers. We update the docstring to clarify this.

bundle2: add documention to 'part.addparams' There are some non-obvious limitations on the parameters of this method. Add some documentation where people will likely look to understand how to use this API.

test-patchbomb: disable sendmail tests on Windows These tests were failing, and there isn't a trivial way to execute a script on Windows [1]. [1] https://www.mercurial-scm.org/pipermail/mercurial-devel/2017-April/096497.html

show: fix corrupt json output with no bookmarks

show: tweak plain abort language for clarity

tests: print Unix style paths in *.py tests These tests don't support (glob). I didn't audit all tests, but these ones were failing.

revlog: make "size" diverge from "rawsize" Previously, revlog.size equals to revlog.rawsize. However, the flag processor framework could make a difference - "size" could mean the length of len(revision(raw=False)), while "rawsize" means len(revision(raw=True)). This patch makes it so. This corrects "hg status" output when flag processor is involved. The call stack looks like: basectx.status -> workingctx._buildstatus -> workingctx._dirstatestatus -> workingctx._checklookup -> filectx.cmp -> filelog.cmp -> filelog.size -> revlog.size

test-flagprocessor: add a case about hg status This shows how "hg status" is wrong - nothing changed but the file is labeled as "M".

test-http-proxy: add the Windows variant of "connection refused" The full error is "No connection could be made because the target machine actively refused it".

test-fileset: eliminate a duplicate test that was conditionalized for output

test-fileset: glob away hash differences There are various files committed above that can't be used on Windows because of the name, or being a symlink.

test-bundle: glob away a URL protocol separator MSYS thinks the ':' is a Unix path separator, and replaces it with ';'.

templatekw: clarify the result of {latesttag} when no tag exists My initial expectation was that the list would be empty, and therefore detectable with {if()}. The map for {latesttag()} is populated with real values in this case (except {tag}), so it probably doesn't make any sense to change this to an empty list.

hgweb: position the "followlines" box close to latest cursor position

hgweb: add a floating tooltip to invite on followlines action In followlines.js, we create a <div id="followlines-tooltip"> element to draw attention of users on "followlines" feature. The element shows up on hover of source lines after one second and follows the cursor. After first click (start line selection), the text changes and indicates that next click will terminate selection.

zstd: vendor python-zstandard 0.8.1 This contains a fix for compilation on BSDs. #no-check-commit

fsmonitor: match watchman and filesystem encoding watchman's paths encoding can differ from filesystem encoding. For example, on Windows, it's always utf-8. Before this patch, on Windows, mismatch in path comparison between fsmonitor state and osutil.statfiles would yield a clean status for added/modified files. In addition to status reporting wrong results, this leads to files being discarded from changesets while doing history editing operations such as rebase. Benchmark: There is a little overhead at module import: python -m timeit "import hgext.fsmonitor" Windows before patch: 1000000 loops, best of 3: 0.563 usec per loop Windows after patch: 1000000 loops, best of 3: 0.583 usec per loop Linx before patch: 1000000 loops, best of 3: 0.579 usec per loop Linux after patch: 1000000 loops, best of 3: 0.588 usec per loop 10000 calls to _watchmantofsencoding: python -m timeit -s "from hgext.fsmonitor import _watchmantofsencoding, _fixencoding" "fname = '/path/to/file'" "for i in range(10000):" " if _fixencoding: fname = _watchmantofsencoding(fname)" Windows (_fixencoding is True): 100 loops, best of 3: 19.5 msec per loop Linux (_fixencoding is False): 100 loops, best of 3: 3.08 msec per loop

pull: abort pull --update if config requires destination (issue5528)

tests: move update requiredest test to own test file More tests for this flag are coming in upcoming patches.

py3: add pycompat.unicode and add it to importer On python 3, builtins.unicode does not exist.

py3: add a bytes version of urllib.parse.urlencode() to pycompat.py urllib.parse.urlencode() returns unicodes on Python 3. This commit adds a method which will take its output and encode it to bytes so that we can use bytes consistently.

py3: replace str() with bytes()

metadataonlyctx: replace "changeset()[0]" to "manifestnode()" As Yuya pointed out [1], "changeset()[0]" could be simplified to "manifestnode()". I didn't notice that method earlier. It should definitely be used - it's easier to read, and faster. [1]: https://www.mercurial-scm.org/pipermail/mercurial-devel/2017-March/095716.html

test-check-pylint: match its output "pylint --version" shows: pylint 2.0.0, astroid 1.5.0 Python 2.7.13 (default, Dec 21 2016, 07:16:46) [GCC 6.2.1 20160830] I got "Your code has been rated at 10.00/10" every time and didn't know how to turn it off. Therefore the fix.

test-flagprocessor: remove unnecessary greps The "2>&1 | egrep ..." code is used for removing uninteresting parts from tracebacks. Now the test does not dump tracebacks, they can be removed.

bundlerepo: use raw revision in revdiff() This is similar to "revlog: use raw revisions in revdiff". revdiff() generates raw text used in revlog directly. This makes test-flagprocessor.t happy.

bundlerepo: fix raw handling in revision() Similar to fixes in revlog.py, this patch uses "rawtext" to explicitly label contents expected to be raw, and makes sure content stored in _cache is raw text. Now test-flagprocessor.t points us to another issue.

bundlerepo: build revlog index with flags This fixes bundlerevlog.flags(rev) for any revisions provided by the bundle. Now test-flagprocessor.t points us to another issue.

bundlerepo: make baserevision return raw text "baserevision" returns the text that will be used to apply deltas. Since deltas are against raw texts, "baserevision" should return raw text. Now test-flagprocessor.t points us to a new error.

test-flagprocessor: add tests about bundlerepo This shows flag processor is broken with a bundle repo. The test creates non-liner history to exercise code path where the deltaparent cannot be reused.

test-flagprocessor: use changegroup3 in bundle2 This will force "hg bundle" to use changegroup3 in the test. It is important since only changegroup3 preserves revlog flags.

bundle: allow bundle command to use changegroup3 in tests Since bundle2 writes changegroup version, we can just reuse the bundle2 format for changegroup3. This won't enable the bundle command to write changegroup3 in the wild, since exchange.parsebundlespec only returns changegroup2. It unlocks tests to override exchange.parsebundlespec and get "hg bundle" write changegroup3.

tests: add per-line output conditionals for Windows

run-tests: support per-line conditional output in tests Duplicating entire tests just because the output is different is both error prone and can make the tests harder to read. This harnesses the existing '(?)' infrastructure, both to improve readability, and because it seemed like the path of least resistance. The form is: $ test_cmd output (hghave-feature !) # required if hghave.has_feature(), else optional out2 (no-hghave-feature2 !) # req if not hghave.has_feature2(), else optional I originally extended the '(?)' syntax. For example, this: 2 r4/.hg/cache/checkisexec (execbit ?) pretty naturally reads as "checkisexec, if execbit". In some ways though, this inverts the meaning of '?'. For '(?)', the line is purely optional. In the example, it is mandatory iff execbit. Otherwise, it is carried forward as optional, to preserve the test output. I tried it the other way, (listing 'no-exec' in the example), but that is too confusing to read. Kostia suggested using '!', and that seems fine.

test-run-tests: pad the failure test to preserve the run order Test size seems to dictate the order in which the tests are run, and the next patch will add to test-success.t. Similar to c0cecc153d25.

run-tests: prevent a (glob) declaration from reordering (?) lines Previously, if a series of optional output lines marked with '(?)' had a (glob) in one of the first lines, the output would be reordered such that it came last if none of the lines were output. The (re) declaration wasn't affected, which was helpful in figuring this out. There were no tests for '(re) (?)' so add that to make sure everything plays nice.

py3: use pycompat.byteskwargs() to convert opts to bytes We have converted opts to unicodes before passing them.

test-check-code: do not use xargs We have too many files, and passing them via arguments could cause strange errors on some platforms [1]. Since check-code.py can now take "-" and read file names from stdin, use it instead of xargs to avoid the argv size limit. [1]: https://www.mercurial-scm.org/pipermail/mercurial-devel/2017-April/096346.html

check-code: use "-" to specify a list of files from stdin This will be used by the next patch.

perf: add historical portability for util.timer util.timer has been introduced in ae5d60bb and used in perf.py since 22fbca1d. For historical portability, forcibly define util.timer in perf.py

diff: add --binary option for git mode diffs This patch adds --binary option to `hg diff` and `hg export` to allow more control about when binary diffs are displayed in Git mode as well as some tests to verify it behaves correctly (issue5510).

patch: make diff in git mode respect --text option (issue5510) This changeset makes patch respect -a/--text option in --git mode by aligning its behaviour with git itself.

py3: have registrar process docstrings in bytes Mixing bytes and unicode creates a mess. Do things in bytes as possible. New sysbytes() helper only takes care of ASCII characters, but avoids raising nasty unicode exception. This is the same design principle as sysstr().

localrepo: fix deprecation version for 'repo._link' The patch lingered for a while and nobody noticed when it was resubmitted.

localrepo: fix deprecation version for 'repo.join' The patch lingered for a while and nobody noticed when it was resubmitted.

tests: make zstd-related output optional Caught by the buildbot with --pure, fix suggested by indygreg.

check-code: update test IP address enforcement checks Instead of mentioning 127.0.0.1, we should use $LOCALIP. Anytime $LOCALIP appears in output, we should make sure we use (glob) on that line of output so that weird environments that do remapping jiggery pokery (such as our FreeBSD buildbot that's in a jail) don't get spurious test failures.

tests: fix missing (glob) annotations in test-push-http.t

tests: fix missing (glob) annotations in test-push-http-bundle1.t

tests: fix missing (glob) annotations in test-https.t

tests: fix missing (glob) annotations in test-bundle2-exchange.t

util: fix %-formatting on docstring by moving a closing parenthesis We have to do the % formatting over the sysstr, since the things we're going to splat into it are themselves sysstrs. This is probably technically wrong-ish, since bt is probably actually a bytestr here, but this fixes the immediate issue, which was that hg was broken on Python 3.

revset: stop supporting plain list as input set (API) There was no deprecwarn(), but this is the same kind of API compatibility as the one removed by the previous patch.

revset: stop supporting predicate that returns plain list (API) It's said to be removed after 3.9.

mdiff: add a hunkinrange helper function This factors out hunk filtering logic by line range that is similar in mdiff.blocksinrange() and hgweb.webutil.diffs().

templater: provide loop counter as "index" keyword This was originally written for JSON templating where we would have to be careful to not add extra comma, but seems generally useful. Inner loop started by % operator has its own counter.

templater: rename variable "i" to "v" in runmap() I want to reuse "i" for index.

formatter: reorder code that builds template mapping This makes the future patch slightly simpler.

revlog: avoid applying delta chain on cache hit Previously, revlog.revision(raw=False) may try to apply the delta chain on _cache hit. That happens if flags are non-empty. This patch makes rawtext reused so delta chain application is avoided. "_cache" and "rev" are moved a bit to avoid unnecessary assignments.

revlog: indent block to make review easier

revlog: avoid calculating "flags" twice in revision() This is more consistent with other code in "revision()" - prefer performance to code length.

revlog: use raw revision for rawsize When writing the revlog-ng index, the third field is len(rawtext). See revlog._addrevision: textlen = len(rawtext) .... e = (offset_type(offset, flags), l, textlen, base, link, p1r, p2r, node) self.index.insert(-1, e) Therefore, revlog.index[rev][2] returned by revlog.rawsize should be len(rawtext), where "rawtext" is revlog.revision(raw=True). Unfortunately it's hard to add a test for this code path because "if l >= 0" catches most cases.

revsetlang: enable optimization of 'x + y' expression It's been disabled since 4d1e56b29a91, but it can be enabled now as the ordering requirement is resolved at analyze().

zstd: vendor python-zstandard 0.8.0 Commit 81e1f5bbf1fc54808649562d3ed829730765c540 from https://github.com/indygreg/python-zstandard is imported without modifications (other than removing unwanted files). Updates relevant to Mercurial include: * Support for multi-threaded compression (we can use this for bundle and wire protocol compression). * APIs for batch compression and decompression operations using multiple threads and optimal memory allocation mechanism. (Can be useful for revlog perf improvements.) * A ``BufferWithSegments`` type that models a single memory buffer containing N discrete items of known lengths. This type can be used for very efficient 0-copy data operations. # no-check-commit

commands: update help for "unbundle" Similar to the recent change to "bundle," this command no longer just deals with "changegroup" data.

commands: update help for "bundle" We now have a dedicated help topic to describe bundle specification strings. Let's update `hg bundle`'s documentation to reflect its existence. While I was hear, I also tweaked some wording which I felt was out of date and needed tweaking. Specifically, `hg bundle` no longer just deals with "changegroup" data: it can also generate files that have non-changegroup data.

help: document bundle specifications I softly formalized the concept of a "bundle specification" a while ago when I was working on clone bundles and stream clone bundles and wanted a more robust way to define what exactly is in a bundle file. The concept has existed for a while. Since it is part of the clone bundles feature and exposed to the user via the "-t" argument to `hg bundle`, it is something we need to support for the long haul. After the 4.1 release, I heard a few people comment that they didn't realize you could generate zstd bundles with `hg bundle`. I'm partially to blame for not documenting it in bundle's docstring. Additionally, I added a hacky, experimental feature for controlling the compression level of bundles in 76104a4899ad. As the commit message says, I went with a quick and dirty solution out of time constraints. Furthermore, I wanted to eventually store this configuration in the "bundlespec" so it could be made more flexible. Given: a) bundlespecs are here to stay b) we don't have great documentation over what they are, despite being a user-facing feature c) the list of available compression engines and their behavior isn't exposed d) we need an extensible place to modify behavior of compression engines I want to move forward with formalizing bundlespecs as a user-facing feature. This commit does that by introducing a "bundlespec" help page. Leaning on the just-added compression engine documentation and API, the topic also conveniently lists available compression engines and details about them. This makes features like zstd bundle compression more discoverable. e.g. you can now `hg help -k zstd` and it lists the "bundlespec" topic.

util: document bundle compression An upcoming patch will add support for documenting bundle specifications in more detail. As part of this, we'd like to enumerate available bundle compression formats. In order to do this, we need to provide the help mechanism a dict of names and objects with docstrings. This patch adds docstrings to compengine.bundletype and adds a function for retrieving a dict of them. The code is not yet used.

tests: store ETag when using --headeronly Previously, --headeronly would prevent --twice from working because the ETag wasn't stored when --headeronly was used. This feels like a bug. That feeling is reaffirmed by the fact that this change doesn't regress any tests.

hgweb: extract path traversal checking into standalone function A common exploit in web applications that access paths is to insert path separator strings like ".." to try to get the server to serve up files it shouldn't. We have code for detecting this in staticfile(). A subsequent commit will need to perform this test as well. Since this is security code, let's factor the check so we don't have to reinvent the wheel.

hgweb: use context manager for file I/O

tags: rename "head" to "node" where we don't care Followup to 5eb4d206202b (tags: extract fnode retrieval into its own function, 2017-03-28) in which the "for head in head" became "for head in nodes".

manifest: update comment to be about bytearray Looks like a leftover from 2a18e9e6ca43 (py3: use bytearray() instead of array('c', ...) constructions, 2017-03-12).

check-code: fix "covert" typo

hgweb: rename linerangelog.js as followlines.js So that the file name matches both the feature name and user facing vocabulary (e.g. the revset function).

hgweb: rely on a specific class to change cursor type in followlines UI The previous CSS rule would also apply in pages where followlines UI was not available (e.g. "changeset" view at /rev/<node>/). We insert a "followlines-select" class in JavaScript on actually selectable lines and restrict the CSS selector to use it.

hgweb: use a function expression for the install listener of followlines UI We define the listener of document's "DOMContentLoaded" inline in registration and use a function expression (anonymous) with everything inside. This makes it clearer that this file is not a library of JavaScript functions but rather an executable script. (Most of changes consists of reindenting the "followlinesBox" function, so mostly white space changes.)

formatter: use templatefilters.json() Now _jsonifyobj() is identical to templatefilters.json(paranoid=False).

templatefilters: use list comprehension in json() Not important, but the code slightly looks better.

templatefilters: unroll handling of None/False/True It doesn't make sense to use a dict here.

templatefilters: drop callable support from json() This backs out ae5447de4c11. A callable should be evaluated beforehand by templater.runsymbol().

ui: use bytes IO and convert EOL manually in ui.editor() Text IO sucks on Python 3 as it must be a unicode stream. We could introduce a wrapper that converts unicode back to bytes, but it wouldn't be simple to handle offsets transparently from/to underlying IOBase API. Fortunately, we don't need to process huge text files, so let's stick to bytes IO and convert EOL in memory.

util: add helper to convert between LF and native EOL See the next patch for why.

util: extract pure tolf/tocrlf() functions from eol extension This can be used for EOL conversion of text files.

pycompat: provide bytes os.linesep

pycompat: introduce identity function as a compat stub I was sometimes too lazy to use 'str' instead of 'lambda a: a'. Let's add a named function for that purpose.

test-blackbox: glob away quoting differences on Windows Windows uses double quotes in these places.

test-subrepo: update output for Windows

test-serve: disable unfixable tests on Windows These tests would run if hghave.has_serve() were enabled on Windows. Windows has no issue allowing an unpriviledged process to open port 13, so it doesn't abort. The other tests are related to how MSYS tries to be helpful and converts Unix constructs to the Windows equivalent. There isn't any way to disable this behavior, though it supposedly doesn't happen if the exe is linked against the MSYS library.

test-serve: kill daemons before deleting the access and error logs On Windows, `rm` will fail with 'Permission denied' if another process has it open. It looks like the rollback test was missing the kill entirely.

test-serve: make the 'listening at *' lines optional The daemonized serve process doesn't print these lines out (see 448d0c452140). I was able to get it to with the following hack: diff --git a/mercurial/win32.py b/mercurial/win32.py --- a/mercurial/win32.py +++ b/mercurial/win32.py @@ -418,6 +418,11 @@ return str(ppid) def spawndetached(args): + + import subprocess + return subprocess.Popen(args, cwd=pycompat.getcwd(), env=encoding.environ, + creationflags=subprocess.CREATE_NEW_PROCESS_GROUP).pid + # No standard library function really spawns a fully detached # process under win32 because they allocate pipes or other objects # to handle standard streams communications. Passing these objects However, MSYS translates --prefixes starting with '/' to 'C:/MinGW/msys/1.0', which changes the output. The output isn't so important that I want to spend a bunch of time on this, and risk breaking some subtle behavior of `hg serve -d` with the more complicated code.

test-http: update output for Windows The http test simply wasn't updated in 161ab32b44a1 for Windows. It looks like the https test meant to glob away the error message in 3e2d8120528b, but forgot the '*', and was subsequently removed in 408f2202bd80.

tests: quote paths in shell script hooks Without the quoting, MSYS will remove the '\' directory separators, and the repo can't be opened.

tests: add globs for Windows

show: new extension for displaying various repository data Currently, Mercurial has a number of commands to show information. And, there are features coming down the pipe that will introduce more commands for showing information. Currently, when introducing a new class of data or a view that we wish to expose to the user, the strategy is to introduce a new command or overload an existing command, sometimes both. For example, there is a desire to formalize the wip/smartlog/underway/mine functionality that many have devised. There is also a desire to introduce a "topics" concept. Others would like views of "the current stack." In the current model, we'd need a new command for wip/smartlog/etc (that behaves a lot like a pre-defined alias of `hg log`). For topics, we'd likely overload `hg topic[s]` to both display and manipulate topics. Adding new commands for every pre-defined query doesn't scale well and pollutes `hg help`. Overloading commands to perform read-only and write operations is arguably an UX anti-pattern: while having all functionality for a given concept in one command is nice, having a single command doing multiple discrete operations is not. Furthermore, a user may be surprised that a command they thought was read-only actually changes something. We discussed this at the Mercurial 4.0 Sprint in Paris and decided that having a single command where we could hang pre-defined views of various data would be a good idea. Having such a command would: * Help prevent an explosion of new query-related commands * Create a clear separation between read and write operations (mitigates footguns) * Avoids overloading the meaning of commands that manipulate data (bookmark, tag, branch, etc) (while we can't take away the existing behavior for BC reasons, we now won't introduce this behavior on new commands) * Allows users to discover informational views more easily by aggregating them in a single location * Lowers the barrier to creating the new views (since the barrier to creating a top-level command is relatively high) So, this commit introduces the `hg show` command via the "show" extension. This command accepts a positional argument of the "view" to show. New views can be registered with a decorator. To prove it works, we implement the "bookmarks" view, which shows a table of bookmarks and their associated nodes. We introduce a new style to hold everything used by `hg show`. For our initial bookmarks view, the output varies from `hg bookmarks`: * Padding is performed in the template itself as opposed to Python * Revision integers are not shown * shortest() is used to display a 5 character node by default (as opposed to static 12 characters) I chose to implement the "bookmarks" view first because it is simple and shouldn't invite too much bikeshedding that detracts from the evaluation of `hg show` itself. But there is an important point to consider: we now have 2 ways to show a list of bookmarks. I'm not a fan of introducing multiple ways to do very similar things. So it might be worth discussing how we wish to tackle this issue for bookmarks, tags, branches, MQ series, etc. I also made the choice of explicitly declaring the default show template not part of the standard BC guarantees. History has shown that we make mistakes and poor choices with output formatting but can't fix these mistakes later because random tools are parsing output and we don't want to break these tools. Optimizing for human consumption is one of my goals for `hg show`. So, by not covering the formatting as part of BC, the barrier to future change is much lower and humans benefit. There are some improvements that can be made to formatting. For example, we don't yet use label() in the templates. We obviously want this for color. But I'm not sure if we should reuse the existing log.* labels or invent new ones. I figure we can punt that to a follow-up. At the aforementioned Sprint, we discussed and discarded various alternatives to `hg show`. We considered making `hg log <view>` perform this behavior. The main reason we can't do this is because a positional argument to `hg log` can be a file path and if there is a conflict between a path name and a view name, behavior is ambiguous. We could have introduced `hg log --view` or similar, but we felt that required too much typing (we don't want to require a command flag to show a view) and wasn't very discoverable. Furthermore, `hg log` is optimized for showing changelog data and there are things that `hg display` could display that aren't changelog centric. There were concerns about using "show" as the command name. Some users already have a "show" alias that is similar to `hg export`. There were also concerns that Git users adapted to `git show` would be confused by `hg show`'s different behavior. The main difference here is `git show` prints an `hg export` like view of the current commit by default and `hg show` requires an argument. `git show` can also display any Git object. `git show` does not support displaying more complex views: just single objects. If we implemented `hg show <hash>` or `hg show <identifier>`, `hg show` would be a superset of `git show`. Although, I'm hesitant to do that at this time because I view `hg show` as a higher-level querying command and there are namespace collisions between valid identifiers and registered views. There is also a prefix collision with `hg showconfig`, which is an alias of `hg config`. We also considered `hg view`, but that is already used by the "hgk" extension. `hg display` was also proposed at one point. It has a prefix collision with `hg diff`. General consensus was "show" or "view" are the best verbs. And since "view" was taken, "show" was chosen. There are a number of inline TODOs in this patch. Some of these represent decisions yet to be made. Others represent features requiring non-trivial complexity. Rather than bloat the patch or invite additional bikeshedding, I figured I'd document future enhancements via TODO so we can get a minimal implmentation landed. Something is better than nothing.

test-revlog-raw: remove duplicated option

test-revlog-raw: fix "genbits" implementation The "genbits" implementation is actually incorrect. This patch fixes it. A good "genbits" implementation should pass the below assertion: n = 3 # or other number l = list(genbits(n)) assert 2**(n*2) == len(set((l[i]<<n)+l[i+1] for i in range(len(l)-1))) An assertion is added to make sure "genbits" won't work unexpectedly.

verify: fix length check According to the document added above, we should check L1 == L2, and the only way to get L1 in all cases is to call "rawsize()", and the only way to get L2 is to call "revision(raw=True)". Therefore the fix. Meanwhile there are still a lot of things about flagprocessor broken in revlog.py. Tests will be added after revlog.py gets fixed.

verify: document corner cases It seems a good idea to list all kinds of "surprises" and expected behavior to make the upcoming changes easier to understand. Note: the comment added does not reflect the actual behavior of the current code.

hgweb: expose a followlines UI in filerevision view In filerevision view (/file/<rev>/<fname>) we add some event listeners on mouse clicks of <span> elements in the <pre class="sourcelines"> block. Those listeners will capture a range of lines selected between two mouse clicks and a box inviting to follow the history of selected lines will then show up. Selected lines (i.e. the block of lines) get a CSS class which make them highlighted. Selection can be cancelled (and restarted) by either clicking on the cancel ("x") button in the invite box or clicking on any other source line. Also clicking twice on the same line will abort the selection and reset event listeners to restart the process. As a first step, this action is only advertised by the "cursor: cell" CSS rule on source lines elements as any other mechanisms would make the code significantly more complicated. This might be improved later. All JavaScript code lives in a new "linerangelog.js" file, sourced in filerevision template (only in "paper" style for now).

shelve: move ui.quiet manipulations to configoverride

revlog: add a fast path for revision(raw=False) If cache hit and flags are empty, no flag processor runs and "text" equals to "rawtext". So we check flags, and return rawtext. This resolves performance issue introduced by a previous patch.

revlog: make _addrevision only accept rawtext All 3 users of _addrevision use raw: - addrevision: passing rawtext to _addrevision - addgroup: passing rawtext and raw=True to _addrevision - clone: passing rawtext to _addrevision There is no real user using _addrevision(raw=False). On the other hand, _addrevision is low-level code dealing with raw revlog deltas and rawtexts. It should not transform rawtext to non-raw text. This patch removes the "raw" parameter from "_addrevision", and does some rename and doc change to make it clear that "_addrevision" expects rawtext. Archeology shows 2df983125d37 added "raw" flag to "_addrevision", follow-ups e12c0fa1f65b and c1b7b2285522 seem to make the flag unnecessary. test-revlog-raw.py no longer complains.

revlog: use raw revisions in clone test-revlog-raw.py now shows "clone test passed", but there is more to fix.

revlog: use raw revisions in revdiff See the added comment. revdiff is meant to output the raw delta that will be written to revlog. It should use raw. test-revlog-raw.py now shows "addgroupcopy test passed", but there is more to fix.

revlog: use raw content when building delta Using external content provided by flagprocessor when building revlog delta is wrong, because deltas are applied to raw contents in revlog. This patch fixes the above issue by adding "raw=True". test-revlog-raw.py now shows "local test passed", but there is more to fix.

revlog: fix _cache usage in revision() As documented at revlog.__init__, revlog._cache stores raw text. The current read and write usage of "_cache" in revlog.revision lacks of raw=True check. This patch fixes that by adding check about raw, and storing rawtext explicitly in _cache. Note: it may slow down cache hit code path when raw=False and flags=0. That performance issue will be fixed in a later patch. test-revlog-raw now points us to a new problem.

revlog: rename some "text"s to "rawtext" This makes code easier to understand. "_addrevision" is left untouched - it will be changed in a later patch.

revlog: clarify flagprocessor documentation The words "text", "newtext", "bool" could be confusing. Use explicit "text" or "rawtext" and document more about the "bool".

revlog: add a stronger test for raw processing There are some issues about revlog raw processing (flag processor). The test is relatively strong covering many cases. It will verify fixes.

hook: add hook name information to external hook While we are here, we can also add the hook name information to external hook.

hook: provide hook type information to external hook The python hooks have access to the hook type information. There is not reason for external hook to not be aware of it too. For the record my use case is to make sure a hook script is configured for the right type.

hook: use 'htype' in 'hook' Same rational as for 'runhooks', we fix the naming in another function.

hook: use 'htype' in 'runhooks' Same rational as for '_pythonhook', 'htype' is more accurate and less error prone. We just fixed an error from the 'name'/'hname' confusion and this should prevent them in the future.

hook: fix name used in untrusted message The name used in the message we issue when a hook is untrusted was using "name" which is actually the hook type and not the name of the hook.

hook: use "htype" as variable name in _pythonhook We rename 'name' to 'htype' because it fits the variable content better. Multiple python hooks already use 'htype' as a name for the argument. This makes the difference with "hname" clearer and the code less error prone.

run-tests: auto-replace 'TXNID' output Hooks related to the transaction are aware of the transaction id. By definition this txn-id is unique and different for each transaction. As a result it can never be predicted in test and always needs matching. As a result, touching any like with this data is annoying. We solve the problem once and for all by installing an automatic replacement. In test, this will now show as: TXNID=TXN:$ID$

largefiles: use readasstandin() to read hex hash directly from filectx BTW, C implementation of hexdigest() for SHA-1/256/512 returns hex hash in lower case, and doctest in Python standard hashlib assumes that, too. But it isn't explicitly described in API document or so. Therefore, we can't assume that hexdigest() always returns hex hash in lower case, for any hash algorithms, on any Python runtimes and versions. From point of view of that, it is reasonable for portability that 40800668e019 applies lower() on hex hash in overridefilemerge(). But on the other hand, in largefiles extension, there are still many code paths comparing between hex hashes or storing hex hash into standin file, without lower(). Switching to hash algorithm other than SHA-1 may be good chance to clarify our policy about hexdigest()-ed hash value string. - assume that hexdigest() always returns hex hash in lower case, or - apply lower() on hex hash in appropriate layers to ensure lower-case-ness of it for portability

largefiles: remove unused readstandin() Now, there is no client of readstandin().

largefiles: make copytostore() accept only changectx as the 2nd argument (API) As the name describes, the 2nd argument 'revorctx' of copytostore() can accept non-changectx value, for historical reason, But, since 10561eb97c7f, copyalltostore(), the only one copytostore() client in Mercurial source tree, always passes changectx as 'revorctx'. Therefore, it is reasonable to make copytostore() accept only changectx as the 2nd argument, now.

largefiles: remove unused keyword argument of copytostore() (API) AFAIK, 'uploaded' argument of copytostore() (or copytocache(), before renaming at c65f5b6e26d4) has been never used both on caller and callee sides, since official release of bundled largefiles extension.

largefiles: add copytostore() fstandin argument to replace readstandin() (API) copyalltostore(), only one caller of copytostore(), already knows standin file name of the target largefile. Therefore, passing it to copytostore() is more efficient than calculating it in copytostore() or readstandin().

largefiles: replace readstandin() by readasstandin() These code paths already (or should, for efficiency at repetition) know the target changectx and path of standin file.

largefiles: introduce readasstandin() to read hex hash from given filectx This will be used to centralize and encapsulate the logic to read hash from given (filectx of) standin file. readstandin() isn't suitable for this purpose, because there are some code paths, which want to read hex hash directly from filectx.

rebase: abort hg pull --rebase if rebase.requiredest is set (issue5514) Previously, the pull would succeed, but the subsequent rebase would fail due to the rebase.requiredest flag. Now abort earlier with a more useful error message.

rebase: demonstrate behavior with requiredest and pull --rebase

rebase: allow destination-free continue and abort (issue5513)

rebase: test to show brokenness with requiredest As shown in issue5513, --continue is broken when destination is required. This adds a patch that demonstates this silly behavior, which will be fixed in a future patch.

rebase: move destination test to new test file We'll be adding a lot more tests here, so it makes sense to have this in its own file now.

templatefilter: add support for 'long' to json() When disabling the '#requires serve' check in test-hgwebdir.t and running it on Windows, several 500 errors popped up when querying '?style=json', with the following in the error log: File "...\\mercurial\\templater.py", line 393, in runfilter "keyword '%s'") % (filt.func_name, dt)) Abort: template filter 'json' is not compatible with keyword 'lastchange' The swallowed exception at that point was: File "...\\mercurial\\templatefilters.py", line 242, in json raise TypeError('cannot encode type %s' % obj.__class__.__name__) TypeError: cannot encode type long This corresponds to 'lastchange' being populated by hgweb.common.get_stat(), which uses os.stat().st_mtime. os.stat_float_times() is being disabled in util, so the type for the times is 'long' on Windows, and 'int' on Linux.

hgweb: prefix line id by ctx shortnode in filelog when patches are shown When "patch" query parameter is present in requests to filelog view, line ids in patches diff are no longer unique in the page since several patches are shown on the same page. We now prefix line id by changeset shortnode when several patches are displayed in the same page to have unique line ids overall.

test-check-code: prevent files being added to the root directory Adding new files in the root directory is probably a mistake, and is usually discouraged [1]. The test catches it to avoid mistakes like [2]. Modify the test if files need to be added in the root. [1]: https://www.mercurial-scm.org/pipermail/mercurial-devel/2016-July/086442.html [2]: https://www.mercurial-scm.org/pipermail/mercurial-devel/2017-March/095836.html

sslutil: clarify internal documentation I ran into this python issue with an incomplete certificate chain on Windows recently, and this is the clarification that came from that experimenting. The comment I left on the bug tracker [1] with a reference to the CPython code [2] indicates that the original problem I had is a different bug, but happened to be mentioned under issue20916 on the Python bug tracker. [1] https://bz.mercurial-scm.org/show_bug.cgi?id=5313#c7 [2] https://hg.python.org/cpython/file/v2.7.12/Modules/_ssl.c#l628

unionrepo: avoid unnecessary node -> rev conversion

bundlerepo: avoid unnecessary node -> rev conversion

revlog: avoid unnecessary node -> rev conversion

check-code: detect r.revision(r.node(rev)) revlog.revision takes either node or rev, but taking a rev is more efficient, because converting rev to node is just a seek and read. That's cheaper than converting node to rev, which may require O(n) walk in revlog index for the first times, and then triggering building the radix tree index. Even with the radix tree built, rev -> node is still faster than node -> rev because the radix tree requires more jumps in memory. So r.revision(r.node(rev)) should be changed to r.revision(rev). This patch adds a check-code rule to detect that.

hardlink: check directory's st_dev when copying files Previously, when copying a file, copyfiles will compare src's st_dev with dirname(dst)'s st_dev, to decide whether to enable hardlink or not. That could have issues on Linux's overlayfs, where stating directories could result in different st_dev from st_dev of stating files, even if both the directories and the files exist in the overlay's upperdir. This patch fixes it by checking dirname(src) instead. It's more consistent because we are checking directories for both src and dest. That fixes test-hardlinks.t running on common Docker setups.

hardlink: duplicate hardlink detection for copying files and directories A later patch will change one of them so they diverge.

hardlink: extract topic text logic of copyfiles The topic text shows whether it's "linking" or "copying", based on "hardlink" value. The function is extracted so a later patch can reuse it.

tests: add check to make sure summary works on Python 2

color: replace str() with pycompat.bytestr()

diff: slice over bytes to make sure conditions work normally Both of this are part of generating `hg diff` on python 3.

minirst: remove redundant _admonitions set As Yuya pointed out during a review a month ago, _admonitions and _admonitiontitles are largely redundant. With the last commit, they are exactly redundant. So, remove _admonitions and use _admonitiontitles.keys() instead.

minirst: remove "admonition" from _admonitions The "admonition" rst primitive is split into "specific" admonitions ("attention," "caution," etc) and the "generic" admonition ("admonition"). For more, see http://docutils.sourceforge.net/docs/ref/rst/directives.html#admonitions The _admonitions set and keys of the _admonitiontitles dict overlap exactly except _admonitions has an "admonition" entry. Nowhere in Mercurial is the "admonition" admonition directive used. Even if it were, it doesn't have a title, so it wouldn't be rendered correctly. So, let's remove "admonition" from the set of recognized admonition directives.

minirst: reindent _admonitiontitles I don't like the verical indent. While I was here, I cleaned up some whitespace and added a trailing comma on the last element.

tags: extract filenode filtering into its own function We'll also need to reuse this logic so we extract it into its own function. We document some of the logic in the process.

tags: extract tags computation from fnodes into its own function I'm about to introduce code that needs to perform such computation on "arbitrary" nodes. The logic is extracted into its own function for reuse.

tags: only return 'alltags' in 'findglobaltags' This is minor update along the way. We simplify the 'findglobaltags' function to only return the tags. Since no existing data is reused, we know that all tags returned are global and we can let the caller get that information if it cares about it.

tags: make argument 'tagtype' optional in '_updatetags' This is the next step from the previous changesets, we are now ready to use this function in a simpler way.

tags: reorder argument of '_updatetags' We move all arguments related to tagtype to the end, together. This will allow us to make these arguments optional and reuse of this logic for callers that do not care about the tag types.

tags: do not feed dictionaries to 'findglobaltags' The code asserts that these dictionary are empty. So we can be more explicit and have the function return the dictionaries directly.

tags: extract fnode retrieval into its own function My main goal here is to be able to reuse this logic easily. As a side effect this important logic is now insulated and the code is clearer.

hgweb: fix diff hunks filtering by line range in webutil.diffs() The previous clause for filter out a diff hunk was too restrictive. We need to consider the following cases (assuming linerange=(lb, ub) and the @s2,l2 hunkrange): <-(s2)--------(s2+l2)-> <-(lb)---(ub)-> <-(lb)---(ub)-> <-(lb)---(ub)-> previously on the first and last situations were considered. In test-hgweb-filelog.t, add a couple of lines at the beginning of file "b" so that the line range we will follow does not start at the beginning of file. This covers the change in aforementioned diff hunk filter clause.

summary: display obsolete state of parents Extend the "parent: " lines in summary to display "(obsolete)" when the parent is obsolete.

templates: add "changeset.obsolete" label in command line style Following respective change in cmdutil.changeset_printer.

templates: shorten definition of changeset labels in command-line style We'll add more labels and the line is already quite long, so let's define a variable to hold all evolution "troubles" labels.

templates: use separate() to build changeset labels in command-line style

templatekw: add an "obsolete" keyword Definition is the same as the one in evolve extension.

cmdutil: add a "changeset.obsolete" label in changeset_printer Until now there were no label to highlight obsolete changesets in log output, only evolution troubles (unstable, bumped, divergent) are supported. We add a "changeset.obsolete" label on changeset entries produced by changeset_printer so that obsolete changesets can be highlighted in log output. This is useful because, unless using a graph log where obsolete changesets have a 'x' marker, there's no way to identify obsolete changesets. And even in graph mode, when working directory's parent is obsolete, we get a '@' marker and we do not see it as obsolete.

fileset: perform membership test against set for status queries Previously, fileset functions operating on status items performed membership tests against a list of items. When there are thousands of items having a specific status, that test can be extremely slow. Changing the membership test to a set makes this operation substantially faster. On the mozilla-central repo: $ hg files -r d14cac631ecc 'set:added()' before: 28.120s after: 0.860s $ hg status --change d14cac631ecc --added 0.690s

worker: flush ui buffers before running the worker a91c6275 introduces flushing ui buffers after a worker finished. If the ui was not flushed before the worker was started, fork will copy the existing buffers to the worker. This causes messages issued before the worker started to be written to the terminal for each worker. We are now flushing the ui before we start a worker and add an appropriate test which will fail before this patch.

chgserver: do not copy configs set by environment variables Config set by environment variables have a source like "$ENVNAME". They should not be copied because they will be recalculated by rcutil.rccomponents.

rcutil: extract duplicated logic to a lambda This simplifies the code a bit.

rcutil: unindent a block Since global _rccomponents is gone, the code could be simplified.

rcutil: do not cache rccomponents The function is only called once except for "hg debugconfig", where it is called twice. So there is no need to cache it. Caching it will cause issues with chgserver. Instead of dropping the cache in chgserver, it seems cleaner to just avoid the cache.

ui: rerun color.setup() once the pager has spawned to honor 'color.pagermode' Otherwise, ui.pageractive is False when color is setup in dispatch.py (without --pager=on), and this config option is ignored.

ui: defer setting pager related properties until the pager has spawned When --pager=on is given, dispatch.py spawns a pager before setting up color. If the pager failed to launch, ui.pageractive was left set to True, so color configured itself based on 'color.pagermode'. A typical MSYS setting would be 'color.mode=auto, color.pagermode=ansi'. In the failure case, this would print a warning, disable the pager, and then print the raw ANSI codes to the terminal. Care needs to be taken, because it appears that leaving ui.pageractive=True was the only thing that prevented an attempt at running the pager again from inside the command. This results in a double warning message, so pager is simply disabled on failure. The ui config settings didn't need to be moved to fix this, but it seemed like the right thing to do for consistency.

color: stop mutating the default effects map A future change will make color.setup() callable a second time when the pager is spawned, in order to honor the 'color.pagermode' setting. The problem was that when 'color.mode=auto' was resolved to 'win32' in the first pass, the default ANSI effects were overwritten, making it impossible to honor 'pagermode=ansi'. Also, the two separate maps didn't have the same keys. The symmetric difference is 'dim' and 'italic' (from ANSI), and 'bold_background' (from win32). Thus, the update left entries that didn't belong for the current mode. This bled through `hg debugcolor`, where the unsupported ANSI keys were listed in 'win32' mode. As an added bonus, this now correctly enables color with MSYS `less` for a command like this, where pager is forced on: $ hg log --config color.pagermode=ansi --pager=yes --color=auto Previously, the output was corrupted. The raw output, as seen through the ANSI blind `more.com` was: <-[-1;6mchangeset: 34840:3580d1197af9<-[-1m ... which MSYS `less -FRX` rendered as: 1;6mchangeset: 34840:3580d1197af91m ... (The two '<-' instances were actually an arrow character that TortoiseHg warned couldn't be encoded, and notepad++ translated to a single '?'.) Returning an empty map for 'ui._colormode == None' seems better that defaulting to '_effects' (since some keys are mode dependent), and is better than None, which blows up `hg debugcolor --color=never`.

pager: do not read from environment variable $PAGER is converted to the pager.pager config item. So it's no longer necessary to read $PAGER in ui.pager().

ui: simplify geteditor Now $EDITOR and $VISUAL will affect ui.editor directly. So it's no longer necessary to test them in ui.geteditor.

debugconfig: list environment variables in debug output Since we print "read config from" for config files, printing environment variables will make it more consistent.

rcutil: let environ override system configs (BC) This is BC because system configs won't be able to override $EDITOR, $PAGER. The new behavior is arguably more rational.

rcutil: add a method to convert environment variables to config items Handling config and environ priorities has been messy. Partially because we don't have config layers - you either get all configs (sys + user), or none. Ideally, environ like $EDITOR, $PAGER should be able to override the system configs "ui.editor", "pager.pager". This patch provides the ability to convert them into config items, so they can be inserted into the middle config layer between system rc and user rc.

rcutil: let rccomponents return different types of configs (API) The next patches will convert environ to raw config items, and insert the config items between systemrcpath and userrcpath. This patch teaches rccomponents to return the type information so the caller could distinguish between "path" and raw config "items".

rcutil: rename rcpath to rccomponents (API)

rcutil: extract rc directory listing logic The logic of listing a ".rc" directory is duplicated in two functions, extract it to a single function to make the code cleaner.

rcutil: split osrcpath to return default.d paths (API) After this change, there are 3 rcpath functions: - defaultrcpath - systemrcpath - userrcpath This will allow us to insert another config layer in the middle.

rcutil: move scmutil.*rcpath to rcutil (API) As discussed at [1], the logic around "actual config"s seem to be non-trivial enough that it's worth a new module. This patch creates the module and move "scmutil.*rcpath" functions there as the first step. More methods will be moved to the module in the future. The module is different from config.py because the latter only cares about data structure and parsing, and does not care about special case, or system config paths, or environment variables. [1]: https://www.mercurial-scm.org/pipermail/mercurial-devel/2017-March/095503.html

statfs: make getfstype() raise OSError It's better for getfstype() function to not suppress an error. Callers can handle it as necessary. Now "hg debugfsinfo" will report OSError.

statfs: rename pygetfstype to getfstype There's no ambiguity now.

statfs: refactor inner function as a mapping from statfs to string The major difference between BSD and Linux is how to get a fstype string. Let's split the longest part of getfstype() as a pure function.

statfs: simplify handling of return value Py_BuildValue() can translate NULL pointer to None.

hghave: use util.getfstype

runtests: change local IP glob pattern from "127.0.0.1" to "$LOCALIP" This is similar to what 348b2b9da703 does. Since 636cf3f7620d has changed "127.0.0.1" to "$LOCALIP". The glob pattern needs update accordingly. It is expected to fix tests running in some BSD jails.

tags: deprecated 'repo.tag' All user are gone. We can now celebrate the removal of some extra line from the 'localrepo' class.

drawdag: use 'tagsmod.tag' instead of 'repo.tag' The former is about to be deprecated.

tags: use the 'tag' function from the 'tags' module in the 'tag' command There is No need to go through the 'repo' object method anymore.

tags: move 'repo.tag' in the 'tags' module Similar logic, pretty much nobody use this method (that creates a tag) so we move it into the 'tags' module were it belong.

tags: move '_tags' from 'repo' to 'tags' module As far as I understand, that function do not needs to be on the local repository class, so we extract it in the 'tags' module were it will be nice and comfortable. We keep the '_' in the name since its only user will follow in the next changeset.

hgweb: filter diff hunks when 'linerange' and 'patch' are specified in filelog

hgweb: add a 'linerange' parameter to webutil.diffs() This is used to filter out hunks based on their range (with respect to 'node2' for patch.diffhunks() call, i.e. 'ctx' for webutil.diffs()). This is the simplest way to filter diff hunks, here done on server side. Later on, it might be interesting to perform this filtering on client side and expose a "toggle" action to alternate between full and filtered diff.

hgweb: handle a "linerange" request parameter in filelog command We now handle a "linerange" URL query parameter to filter filelog using a logic similar to followlines() revset. The URL syntax is: log/<rev>/<file>?linerange=<fromline>:<toline> As a result, filelog entries only consists of revision changing specified line range. The linerange information is propagated to "more"/"less" navigation links but not to numeric navigation links as this would apparently require a dedicated "revnav" class. Only update the "paper" template in this patch.

shelve: add logic to preserve active bookmarks This adds an explicit active-bookmark-handling logic to shelve. Traditional shelve handles it by transaction aborts, but it is a bit ugly and having an explicit functionality seems better.

metadataonlyctx: speed up sanity check Previously the sanity check will construct manifestctx for both p1 and p2. But it only needs the "manifest node" information, which could be read from changelog directly.

revset: factor out linerange processing into a utility function Similar processing will be done in hgweb.webutil in forthcoming changeset.

hgweb: add a "patch" query parameter to filelog command Add support for a "patch" query parameter in filelog web command similar to --patch option of `hg log` to display the diff of each changeset in the table of revisions. The diff text is displayed in a dedicated row of the table that follows the existing one for each entry and spans over all columns. Only update "paper" template in this patch.

hgweb: handle "parity" internally in webutil.diffs() There's apparently no reason to have the "parity" of diff blocks that webutil.diffs() generates coming from outside the function. So have it internally managed. We thus now pass a "web" object to webutil.diffs() to get access to both "repo" and "stripecount" attribute.

largefiles: add lfile argument to updatestandin() for efficiency (API) Before this patch, updatestandin() takes "standin" argument, and applies splitstandin() on it to pick out a path to largefile (aka "lfile" or so) from standin. But in fact, all callers already knows "lfile". In addition to it, many callers knows both "standin" and "lfile". Therefore, making updatestandin() take only one of "standin" or "lfile" is inefficient.

largefiles: use strip() instead of slicing to get rid of EOL of standin This slicing prevents from replacing SHA-1 by another (= longer hash value) in the future.

largefiles: rename local variable appropriately repo['.'] is called not as "working context" but as "parent context". In this code path, hash value of current content of file should be compared against hash value recorded in "parent context". Therefore, "wctx" may cause misunderstanding in this case.

largefiles: avoid redundant loop to eliminate None from list Before this patch, this code path contains two loops for m._files: one for replacement with standin, and another for elimination of None, which comes from previous replacement ("standin in wctx or lfdirstate[f] == 'r'" case in tostandin()). These two loops can be unified into simple one "for" loop.

largefiles: avoid meaningless changectx looking up Logically, "repo[ctx.node()]" should be equal to "ctx". In addition to it, this redundant code path is repeated "len(match.m_files)" times.

largefiles: avoid redundant changectx looking up at each repetitions These code paths look up changectx at each repetitions, even though the changectx key isn't changed while loop.

largefiles: omit updating newly added standin at linear merging Updating standin for newly added largefile is needed, only if same name largefile exists in destination context at linear merging. In such case, updated standin is used to detect divergence of largefile at overridefilemerge(). Otherwise, standin doesn't have any responsibility for its content (usually, it is empty).

largefiles: reuse hexsha1() to centralize hash calculation logic into it This patch also renames argument of hexsha1(), not only for readability ("data" isn't good name for file-like object), but also for reviewability (including hexsha1() code helps reviewers to confirm how these functions are similar). BTW, copyandhash() has also similar logic, but it can't reuse hexsha1(), because it writes read-in data into specified fileobj simultaneously.

py3: prove second commit works Finally it works.

py3: fix manifestdict.fastdelta() to be compatible with memoryview This doesn't look nice, but a straightforward way to support Python 3. bytes(m[start:end]) is needed because a memoryview doesn't support ordering operations. On Python 2, m[start:end] returns a bytes object even if m is a buffer, so calling bytes() should involve no additional copy. I'm tired of trying cleaner alternatives, including: a. extend memoryview to be compatible with buffer type => memoryview is not an acceptable base type b. wrap memoryview by buffer-like class => zlib complains it isn't bytes-like

crecord: use ProgrammingError

transaction: use ProgrammingError

bundle2: use ProgrammingError

merge: use ProgrammingError

repair: use ProgrammingError

py3: abuse r'' to preserve str-ness of literals passed to __setattr__()

py3: fix slicing of byte string in revlog.compress() I tried .startswith('\0'), but data wasn't always a bytes nor a bytearray.

py3: use bytes() to cast to immutable bytes in changelog.appender.write()

py3: use bytes() to cast to immutable bytes in pure.bdiff.bdiff()

bdiff: drop support for array.array argument from pure.bdiff.bdiff() Thanks to 2a18e9e6ca43, we no longer pass array.array('c') object to bdiff().

histedit: test that an aborted histedit can be rerun (with obsolete) In the future, this should help catching issue as the one introduced in 6f0b7475cf9a.

histedit: backout changeset 6f0b7475cf9a Before 6f0b7475cf9a, histedit (like rebase) was only creating markers on final success from the old-rewritten node to the newly created nodes (as of before 6f0b7475cf9a). In case of abort the aborted attempt were stripped to restore the repository in its state prior to the attempt. This use of strip was on purpose. Using markers in this case introduces various issues. The main one is that keeping the partial result of histedit as obsolete prevents us to recreates the same nodes in a second attempt. The same operation will lead to an identical results, using an identical node that already exists in the repository as obsolete. To conclude, we cannot and should not switch to obsolescence markers creation on histedit --abort and we backout 6f0b7475cf9a. A test to catch this class of issue will be introduced in the next changeset.

histedit: backout changeset 2b599f5468a4 Its parent is about to be backedout so this one needs to be removed too.

revsetlang: fix _quote on int on python3 Thanks to Yuya for spotting the need.

runtests: unset editor and pager related environment variables Those environment variables could affect some configuration and future tests. Drop them to avoid issues.

debugfsinfo: improve case-sensitive testing Previously the case-sensitive test was for the current directory, and is fragile with errors, and could remove a real file called ".debugfsinfo". This patch improves the case-sensitive testing so it test the given path using a unique temporary file, and does not crash on errors.

debugfsinfo: show fstype for given path

test-check-py3-commands: cleanup tests related to `hg status` We were testing hg status on Python 3 at two places in the test. Cleaned up one of them.

diff: use pycompat.{byteskwargs, strkwargs} to switch opts b/w bytes and str

patch: make regular expressions bytes by adding b''

dispatch: use pycompat.maplist() instead of map() to get a list

color: fix grammar in help text

statfs: detect more filesystems on Linux Previously, the code only has what manpager says. In <linux/magic.h>, there are more defined. This patch adds filesystems that appear in the current Arch Linux's /proc/filesystems (autofs, overlay, securityfs) and f2fs, which was seen in news.

repair: use context manager for lock management If repo.lock() raised inside of the try block, 'tr' would have been None in the finally block where it tries to release(). Modernize the syntax instead of just winching the lock out of the try block. I found several other instances of acquiring the lock inside of the 'try', but those finally blocks handle None references. I also started switching some trivial try/finally blocks to context managers, but didn't get them all because indenting over 3x for lock, wlock and transaction would have spilled over 80 characters. That got me wondering if there should be a repo.rwlock(), to handle locking and unlocking in the proper order. It also looks like py27 supports supports multiple context managers for a single 'with' statement. Should I hold off on the rest until py26 is dropped?

gitweb: use monospace font for commit messages Commit messages often contain vertically aligned text. The default paper style already uses monospace fonts for rendering commit messages. And, AFAICT, a number of Git servers also render commit messages with monospace. It seems like the reasonable thing to do. This commit converts all instances of the full commit message in the gitweb style to render with monospace.

pager: improve support for various flavors of `more` on Windows Hardcoding 'more' -> 'more.com' means that 'more.exe' from MSYS would need to be configured with its *.exe extension. This will resolve to either one, as cmd.exe would have done if the command ran through the shell. Something that's maybe problematic with this is it comes after 'pageractive' and various ui configs have been set by the calling method. But the other early exits in this method don't undo those changes either.

statfs: avoid static allocation Previously we have "static struct statfs" to return a string. That is not multiple-thread safe. This patch moves the allocation to the caller to address the problem.

statfs: change Linux feature detection Previously we check three things: "statfs" function, "linux/magic.h" and "sys/vfs.h" headers. But we didn't check "struct statfs" or the "f_type" field. That means if a system has "statfs" but "struct statfs" is not defined in the two header files we check, or defined without the "f_type" field, the compilation will fail. This patch combines the checks (2 headers + 1 function + 1 field) together and sets "HAVE_LINUX_STATFS". It makes setup.py faster (less checks), and more reliable (immutable to the issue above).

rebase: don't require destination if commands.rebase.requiredest=False

tests: add tests with commands.{update,rebase}.requiredest=False This shows how rebase is currently broken with commands.rebase.requiredest=False.

rebase: move state serialization to use unfiltered repo Now that rebasestate is serialized as part of the transaction, the repo state it sees is the version at the end of the transaction, which may have hidden nodes. Therefore, it's possible parts of the rebase commit set are no longer visible by the time the transaction is closing, which causes a filtered revision error in this code. I don't think state serialization should be blocked from accessing commits it knows exist, especially if all it's trying to do is get the hex of them, so let's use an unfiltered repo here. Unfortunately, the only known repro is with the fbamend Facebook extension, so I'm not sure how to repro it in core Mercurial for a test.

largefiles: avoid redundant standin() invocations There are some code paths, which apply standin() on same value multilpe times instead of using already standin()-ed value. "fstandin" is common name for "path to standin file" in lfutil.py, to avoid shadowing "standin()".

largefiles: replace hashrepofile by hashfile (API) There is only one user for the former, and repo.wjoin()-ed value is alread known by that user.

largefiles: call readstandin() with changectx itself instead of rev or node readstandin() takes "node" argument to get changectx by "repo[node]". There are some readstandin() invocations, which use ctx.node(), ctx.rev(), or '.' as "node" argument above, even though corresponded changectx object is already looked up on caller side. This patch calls readstandin() with already known changectx itself, to avoid meaningless re-construction of changectx (indirect case via copytostore() is also included). BTW, copytostore() uses "rev" argument only for readstandin() invocation. Therefore, this patch also renames it to "revorctx" to indicate that it can take not only revision ID or so but also changectx, for readability.

largefiles: omit redundant splitstandin() invocations There are 3 splitstandin() invocations in updatestandin() for same "standin" value.

largefiles: replace splitstandin() by isstandin() to omit str creation If splitstandin()-ed str itself isn't used, isstandin() should be used instead of it, to omit meaningless str creation.

largefiles: omit redundant isstandin() before splitstandin() There are many isstandin() invocations before splitstandin(). The former examines whether specified path starts with ".hglf/". The latter returns after ".hglf/" of specified path if it starts with that prefix, or returns None otherwise. Therefore, value returned by splitstandin() can be used for replacement of preceding isstandin(), and this replacement can omit redundant string comparison after isstandin().

misc: update descriptions about removed file for filectxfn Since 650b5b6e75ed, filectxfn for memctx should return None for removed file instead of raising IOError.

osx: always purge build/mercurial before starting build This prevents stray files from previous builds from polluting newer builds.

osx: use more paranoid quoting in test

osx: install completion scripts using install(1) to avoid umask badness This actually fixes some tests that were showing us umask badness that had been overlooked.

merge with stable

py3: prove `hg {add,addremove,commit} all work We can't do a second commit in a repo yet, because pure-Python bdiff is broken on Python 3. That is probably a good next step.

revsetlang: perform quoting using ui.escapestr instead of repr() This changes one of the doctest results, but I'm pretty sure on inspection that it's an equivalent result.

revsetlang: add docstring with some tests to _quote

revsetlang: move quoting function to not be a closure I'm about to change the implementation here and I'd like to add some doctests, which means this needs to not be hidden inside another function.

revsetlang: portably bytestring-ify another pair of int() calls

checkcode: enforce lowercase for extension docstring title This will ensure new extensions are consistent and `hg help -e` has a consistent output. I have to add a new group since the normal "pypats" will be filtered by "pyfilters", which will remove comments and docstrings.

logtoprocess: use lowercase for docstring title

journal: use lowercase for docstring title See the previous patch for why.

automv: use lowercase for docstring title The first line of docstring is used in `hg help -e`. Use lowercase to be consistent with most extensions.

util: enable hardlink for some BSD-family filesystems Since we can now detect filesystems on FreeBSD and OSX. Add their major filesystems (ufs, zfs for FreeBSD; hfs for OSX) to the hardlink whitelist.

osutil: report fstype for BSD and OSX

setup: use a more strict way to test BSD or OSX's statfs We want to use the `f_fstypename` field to get the filesystem type. Test it directly. The new macro HAVE_BSD_STATFS implys the old HAVE_SYS_MOUNT_H and HAVE_SYS_PARAM_H. So the latter ones are removed.

debugfsinfo: use util.getfstype This changes the behavior slightly. It now always prints fstype, regardless of whether osutil.getfstype exists.

util: use util.getfstype

util: add a getfstype method The util version is a thin wrapper of the osutil version, which is not always available.

pager: fix the invocation of `more` on Windows After 9335dc6b2a9c, with 'shell' being (mostly) set to False, invoking `more` no longer worked. Instead, a warning was printed and the pager was disabled. Invoking `more.com` works. Since a user may have configured 'pager.pager=more', do this substitution at the end. Surprisingly, `more` does allow for arguments, so those are preserved. This also allows `more` to work in MSYS. Setting 'shell=False' runs the executable via CreateProcess(), which has rather wonky rules for resolving an executable without an extension [1]. Resolving to *.com is not among them. Since 'shell=True' yields a cryptic error for a bad $PAGER, and a *.exe program will work without specifying the extension, sticking with current 'shell=False' seems like the right thing to do. I don't think there are any other *.com pagers out there, so this one special case seems OK. If somebody wants to do something crazy that requires cmd.exe, I was able to get normal paged output with 'pager.pager="cmd.exe /c more"'. I assume you can replace `more` with *.bat, *.vbs or various other creatures listed in $PATHEXT. [1] https://msdn.microsoft.com/en-us/library/windows/desktop/ms682425(v=vs.85).aspx

help: format ``commands`` heading correctly The number of dashes under it needs to match exactly for it to be rendered as a heading. Without this change, the dashes end up on the same line as "commands", and "hg help config.commands" does not work.

convert: fix the handling of empty changlist descriptions in P4 Empty changelist descriptions are valid in Perforce. If we encounter one of them we are currently running into an IndexError. In case of empty commit messages set the commit message to **empty changelist description**, which follows Perforce terminology.

status: support commands.status.relative config When the config is set to true, status output becomes relative to the working directory. This has bugged me since I started using hg and it turns it is sillily simple to support it (unless I missed something, of course). We could also add a --relative flag, but I would personally always want that on, and I haven't heard any use for having it sometimes on, so this patch only lets you enable it via config.

plain: ignore [commands] config We only have commands.{update,rebase}.requiredest so far. We should clearly ignore those two if HGPLAIN is in effect, and it seems like we should ignore any future config that will be added in [commands] since that is about changing the behavior of commands. Thanks to Yuya for suggesting to centralize the code in ui.py. While at it, remove the unnecessary False values passed to ui.configbool() for the aforementioned config options.

checkheads: extract obsolete post processing in its own function The checkheads function is long and complex, extract that logic in a subfunction is win in itself. As the comment in the code says, this postprocessing is currently very basic and either misbehave or fails to detect valid push in many cases. My deeper motive for this extraction is to be make it easier to provide extensive testing of this case and strategy to cover them. Final test and logic will makes it to core once done.

tests: make test-simplekeyvaluefile.py py2.6-compatible Python 2.6 unittest.TestCase does not have assertRaisesRegexp.

similar: use cheaper hash() function to test exact matches We just need a hash table {fctx.data(): fctx} which doesn't keep fctx.data() in memory. Let's simply use hash(fctx.data()) to put data out from memory, and manage collided fctx objects by list. This isn't significantly faster than using sha1, but is more correct as we know SHA-1 collision attack is getting practical. Benchmark with 50k added/removed files, on tmpfs: $ hg addremove --dry-run --time -q previous: real 12.420 secs (user 11.120+0.000 sys 1.280+0.000) this patch: real 12.350 secs (user 11.210+0.000 sys 1.140+0.000)

similar: take the first match instead of the last It seems more natural. This makes the next patch slightly cleaner.

similar: do not look up and create filectx more than once Benchmark with 50k added/removed files, on tmpfs: $ hg addremove --dry-run --time -q previous: real 16.070 secs (user 14.470+0.000 sys 1.580+0.000) this patch: real 12.420 secs (user 11.120+0.000 sys 1.280+0.000)

similar: use common names for changectx variables We generally use 'wctx' and 'pctx' for working context and its parent respectively.

similar: get rid of quadratic addedfiles.remove() Instead, build a set of files to be removed and recreate addedfiles only if necessary. Benchmark with 50k added/removed files, on tmpfs: $ hg addremove --dry-run --time -q original: real 16.550 secs (user 15.000+0.000 sys 1.540+0.000) previous: real 16.730 secs (user 15.280+0.000 sys 1.440+0.000) this patch: real 16.070 secs (user 14.470+0.000 sys 1.580+0.000)

similar: sort files not by object id but by path for stable result Perhaps the original implementation would want to sort added/removed files alphabetically, but actually it did sort fctx objects by memory location. This patch removes the use of set()s in order to preserve the order of added/removed files. addedfiles.remove() becomes quadratic, but its cost appears not dominant. Anyway, the quadratic behavior will be eliminated by the next patch. Benchmark with 50k added/removed files, on tmpfs: $ mkdir src $ for n in `seq 0 49`; do > mkdir `printf src/%02d $n` > done $ for n in `seq 0 49999`; do > f=`printf src/%02d/%05d $(($n/1000)) $n` > dd if=/dev/urandom of=$f bs=8k count=1 status=none > done $ hg ci -qAm 'add 50k files of random content' $ mv src dest $ hg addremove --dry-run --time -q original: real 16.550 secs (user 15.000+0.000 sys 1.540+0.000) this patch: real 16.730 secs (user 15.280+0.000 sys 1.440+0.000)

debugfsinfo: print fstype information Since we have osutil.getfstype, it'll be handy if "debugfsinfo" prints it.

util: enable hardlink for copyfile This patch removes the global variable "allowhardlinks" that disables hardlink in all cases, so hardlink gets enabled if the filesystem type is whitelisted. Third party extensions wanting to enable hardlink support unconditionally can replace "_hardlinkfswhitelist.__contains__".

hghave: add a check about whitelisted filesystem that supports hardlink This is needed for the test added by the next patch.

util: disable hardlink for copyfile if fstype is outside a whitelist Since osutil.getfstype is available, use it to detect filesystem types. The whitelist currently includes common local filesystems on Linux where they should have good hardlink support. We may add new filesystems for other platforms later.

revlog: use pycompat.maplist to eagerly evaluate map on Python 3 According to Pulkit, this should fix `hg status --all` on Python 3.

py3: stop exporting urlparse from pycompat and util (API) There are no consumers of this in tree. Functions formerly available on this object/module can now be accessed via {pycompat,util}.urlreq.

check-code: recommend util.urlreq when importing urlparse

tests: use urlreq in tinyproxy.py This is our last consumer of util.urlparse.

bugzilla: use util.urlreq.urlparse And stop saving a module variable because it shouldn't be necessary.

pycompat: define urlreq.urlparse and urlreq.unparse aliases Currently, we export urlparse via util.urlparse then call util.urlparse.urlparse() and util.urlparse.urlunparse() in a few places. This is the only url* module exported from pycompat, making it a one-off. So let's transition to urlreq to match everything else. Yes, we double import "urlparse" now on Python 2. This will be cleaned up in a subsequent patch. Also, the Python 3 functions trade in str/unicode not bytes. So we'll likely need to write a custom implementation that speaks bytes. But moving everyone to an abstracted API is a good first step.

pycompat: remove urlunquote alias It is duplicated by urlreq.unquote and is unused. Kill it. We retain the imports because it is re-exported via util.urlparse, which is used elsewhere. Since we no longer access attributes of urlparse at module load time, this change /should/ result in that module reverting to a lazy module.

util: use urlreq.unquote pycompat.urlreq.unquote and pycompat.urlunquote effectively alias the same thing. pycompat.urlunquote is only used once in the code base. So let's switch to urlreq.unquote. "Effectively" in the above paragraph is because pycompat.urlreq.unquote aliases urllib.unquote and pycompat.urlunquote aliases urlparse.unquote on Python 2. You might think one of urllib.unquote and urlparse.unquote is an alias to the other, but you would be incorrect. In fact, these functions are copies of each other. There is even a comment in the CPython source code saying to keep them in sync. You can't make this up.

pycompat: alias urlreq.unquote to unquote_to_bytes Previously, urlreq.unquote aliased to urllib.parse.unquote, which returned a str/unicode. We like bytes, so switch urlreq.unquote to dispatch to urllib.parse.unquote_to_bytes. This required a minor helper function to register an alias under a different name from which it points. If this turns into a common pattern, we could likely teach _registeralias to accept tuple values defining the mapping. Until then, I didn't feel like adding complexity to _registeralias.

revsetlang: portably turn int into bytestring

osutil: export a "getfstype" method This patch exports the "getfstype" method. So we can use it to enable hardlinks for known safe filesystems. The patch was tested manually via debugshell on a Linux system. "mercurial.osutil.getfstype" works as expected. It's hard to mount filesystem on user-space easily. I will add a test for real hardlink support to indirectly test this patch, after turning on real hardlinks support for certain whitelisted filesystems.

osutil: add a C function getting filesystem type Currently it only has Linux filesystems, according to my Linux manpage, built at 2016-03-15. The code uses "if" instead of "switch" because there could be some duplicated values.

setup: test some header files The next patch will use "statfs", which requires different header files on different platforms. Linux: sys/vfs.h or sys/statfs.h FreeBSD or OSX: sys/param.h and sys/mount.h Therefore test them so we can include the correct ones.

setup: detect statfs statfs is not defined by POSIX but is available in various systems to help decide filesystem type. Let's detect it and set the macro HAVE_STATFS.

setup: add a function to test header files

setup: split "hasfunction" to test arbitrary code The next patch wants to test include files.

rebase: add flag to require destination In some mercurial workflows, the default destination for rebase does not always work well and can lead to confusing behavior. With this flag enabled, every rebase command will require passing an explicit destination, eliminating this confusion.

update: add flag to require update destination In some mercurial workflows, the default destination for update does not always work well and can lead to confusing behavior. With this flag enabled, every update command will require passing an explicit destination, eliminating this confusion.

mq: reject new patch name containing leading/trailing whitespace We could create a patch of such name, but it wouldn't be processed properly by mq as parseseries() strips leading/trailing whitespace. The test of default message (added by b9a16ed5acec) is no longer be useful so removed. This issue was reported as: https://bitbucket.org/tortoisehg/thg/issues/4693/

shelve: rename stripnodes to nodestoprune Since we are introducing obs-based shelve, we are no longer stripping temporary nodes, we are obsoleting them. Therefore it looks like stipnodes would be a misleading name, while prune has a connotaion of "strip but with obsolescense", so nodestoprune seems like a good rename.

shelve: add an ability to write key-val data to a new type of shelve files Obsolescense-based shelve only needs metadata stored in .hg/shelved and if feels that this metadata should be stored in a simplekeyvaluefile format for potential extensibility purposes. I want to avoid storing it in an unstructured text file where order of lines determines their semantical meanings (as now happens in .hg/shelvedstate. .hg/rebasestate and I suspect other state files as well). Not included in this series, I have ~30 commits, doubling test-shelve.t in size and testing almost every tested shelve usecase for obs-shelve. Here's the series for the curious now: http://pastebin.com/tGJKx0vM I would like to send it to the mailing list and get accepted as well, but: 1. it's big, so should I send like 6 patches a time or so? 2. instead of having a commit per test case, it more like a commit per some amount of copy-pasted code. I tried to keep it meaningful and named commits somewhat properly, but it is far from this list standards IMO. Any advice on how to get it in without turning it into a 100 commits and spending many days writing descriptions? 3. it makes test-shelve.t run for twice as long (and it is already a slow test). Newest test-shelve.r runs for ~1 minute.

scmutil: add a simple key-value file helper The purpose of the added class is to serve purposes like save files of shelve or state files of shelve, rebase and histedit. Keys of these files can be alphanumeric and start with letters, while values must not contain newlines. In light of Mercurial's reluctancy to use Python's json module, this tries to provide a reasonable alternative for a non-nested named data. Comparing to current approach of storing state in plain text files, where semantic meaning of lines of text is only determined by their oreder, simple key-value file allows for reordering lines and thus helps handle optional values. Initial use-case I see for this is obs-shelve's shelve files. Later we can possibly migrate state files to this approach. The test is in a new file beause I did not figure out where to put it within existing test suite. If you give me a better idea, I will gladly follow it.

graphlog: draw multiple edges towards null node (issue5440) Before, edge (r, null) was processed only once by newparents. However what we really need is just stripping the edge (null, null).

merge with stable

localrepo: use tryunlink

vfs: use tryunlink

server: use tryunlink

dirstate: use tryunlink