manifest: add shallow option to treemanifestctx.readdelta and readfast The old manifest had different functions for performing shallow reads, shallow readdeltas, and shallow readfasts. Since a lot of the code is duplicate (and since those functions don't make sense on a normal manifestctx), let's unify them into flags on the existing readdelta and readfast functions. A future diff will change consumers of these functions to use the manifestctx versions and will delete the old apis.

manifest: change manifestlog mancache to be directory based In the last patch we added a get() function that allows fetching directory level treemanifestctxs. It didn't handle caching at directory level though, so we need to change our mancache to support multiple directories.

manifest: add manifestlog.get to obtain subdirectory instances Previously manifestlog only allowed obtaining root level manifests. Future patches will need direct access to subdirectory manifests as part of changegroup creation, so let's add a get() function that knows how to deal with subdirectories.

manifest: throw LookupError if node not in revlog When accessing a manifest via manifestlog[node], let's verify that the node actually exists and throw a LookupError if it doesn't. This matches the old read behavior, so we don't accidentally return invalid manifestctxs. We do this in manifestlog instead of in the manifestctx/treemanifestctx constructors because the treemanifest code currently relies on the fact that certain code paths can produce treemanifests without touching the revlogs (and it has tests that verify things work if certain revlogs are missing entirely, so they break if we add validation that tries to read them).

revlog: optimize _chunkraw when startrev==endrev In many cases, _chunkraw() is called with startrev==endrev. When this is true, we can avoid an extra index lookup and some other minor operations. On the mozilla-unified repo, `hg perfrevlogchunks -c` says this has the following impact: ! read w/ reused fd ! wall 0.371846 comb 0.370000 user 0.350000 sys 0.020000 (best of 27) ! wall 0.337930 comb 0.330000 user 0.300000 sys 0.030000 (best of 30) ! read batch w/ reused fd ! wall 0.014952 comb 0.020000 user 0.000000 sys 0.020000 (best of 197) ! wall 0.014866 comb 0.010000 user 0.000000 sys 0.010000 (best of 196) So, we've gone from ~25x slower than batch to ~22.5x slower. At this point, there's probably not much else we can do except implement an optimized function in the index itself, including in C.

revlog: inline start() and end() for perf reasons When I implemented `hg perfrevlogchunks`, one of the things that stood out was N * _chunk() calls was ~38x slower than 1 _chunks() call. Specifically, on the mozilla-unified repo: N*_chunk: 0.528997s 1*_chunks: 0.013735s This repo has 352,097 changesets. So the average time per changeset comes out to: N*_chunk: 1.502us 1*_chunks: 0.039us If you extrapolate these numbers to a repository with 1M changesets, that comes out to 1.502s versus 0.039s, which is significant. At these latencies, Python attribute lookups and function calls matter. So, this patch inlines some code to cut down on that overhead. The impact of this patch on N*_chunk() calls is clear: ! wall 0.528997 comb 0.520000 user 0.500000 sys 0.020000 (best of 19) ! wall 0.367723 comb 0.370000 user 0.350000 sys 0.020000 (best of 27) So, we go from ~38x slower to ~27x. A nice improvement. But there's still a long way to go. It's worth noting that functionality like revsets perform changelog lookups one revision at a time. So this code path is worth optimizing.

revlog: reorder index accessors to match data structure order Index entries are ordered tuples. We have accessors in the revlog class to map tuple offsets to names. To help reinforce the order, reorder the methods so they match the order of elements in the tuple. While I'm here, also sneak in some minimal documentation.

color: add the ability to display configured style to 'debugcolor' The 'hg debugcolor' command gains a '--style' flag to display all the configured labels and their styles. This have many benefits: * discovering documented label, * checking consistency between label's style, * showing the actual style of a label.

color: sort output of 'debugcolor' The previous ordering were provided by the set. The new output is more stable and rational. In addition we have some logic to keep the '_background' version together to help readability.

color: extract color and effect display from 'debugcolor' We are about to introduce a second mode for 'hg debugcolor' that would list the known label and their configuration, so we split the code related to color and effect out of the main function.

color: restore _style global after debugcolor ran Before this change, running 'debugcolor' would destroy all color style for the rest of the process life. We now properly backup and restore the variable content. Using a global variable is sketchy in general and could probably be removed. However, this is a quest for another adventure.

color: add basic documentation to 'debugcolor' This does not hurt.

tests: merge 'test-push-hook-lock.t' into 'test-push.t' That test file is very small and is merge with the new 'test-push.t'. No logic is changed. We don't register this as a copy because is actually a "ypoc" merging two file together without replacing the destination and Mercurial cannot express that.

tests: merge 'test-push-validation.t' into 'test-push.t' That test file is very small and is merge with the new 'test-push.t'. No logic is changed but repository name are update to avoid collision. We don't register this as a copy because is actually a "ypoc" merging two file together without replacing the destination and Mercurial cannot express that.

test: rename 'test-push-r.t' to 'test-push.t' We do not have a simple test for 'hg push' but we have multiple tiny tests for various aspect of it. We'll unify them into a single file, and we start with 'test-push-r.t'. The code is unchanged but we renamed the repository used to avoid collision with other tests we'll import in coming changesets. Test timing for the record: start end cuser csys real Test 1.850 2.640 0.650 0.090 0.790 test-push-validation.t 2.640 3.520 0.760 0.090 0.880 test-push-hook-lock.t 0.000 1.850 1.560 0.210 1.850 test-push-r.t

tests: simplify command script in 'test-push-r.t' I came across this code by chance. The script of this test is a bit messy with a lot of unnecessary intermediate commands. We simplify the script and unify repository access through '-R'. In the process the update after the unbundle is dropped as it does not add anything to the tests.

py3: use encoding.environ in ui.py Using source transformer we add b'' everywhere. So there are no chances that those bytes string will work with os.environ on Py3 as that returns a dict of unicodes. We are relying on the errors, even though no error is raised even in future, these pieces of codes will tend to do wrong things. if statements can result in wrong boolean and certain errors can be raised while using this piece of code. Let's not wait for them to happen, fix what is wrong. If this patch goes in, I will try to do it for all the cases. Leaving it as it is buggy.

py3: make scmposix.userrcpath() return bytes We are making sure that we deal with bytes as much we can. This is a part of fixing functions so that they return bytes if they have to. Used encoding.environ to return bytes. After this patch, scmposix.userrcpath() returns bytes and scmutil.osrcpath() will also return bytes if the platform is posix. Functions is scmposix returns bytes on Python 3 now.

adjustlinkrev: remove unnecessary parameters Since adjustlinkrev has "self", and is a method of a filectx object, it does not need path, filelog, filenode. They can be fetched from the "self" easily.

repair: make strip() return backup file path narrowhg wants to strip some commits and then re-apply them after applying another bundle. Having repair.strip() return the bundle path will be helpful for it.

rebase: check for conflicts before continuing When there are unresolved merge conflicts, there is no reason to make the user wait for rebase to process all of the already rebased commits just to complain that it cannot do anything. Abort early.

cmdutil: refactor checkunresolved localrepo.commit had code to check for unresolved merge conflicts, it would be helpful for at least rebase to be able to use that code without calling commit().

rebase: rename merge to mergemod

context: make sure __str__ works, also when there is no _changectx Before, it could crash when trying to print the wrong kind of object at the wrong time.

largefiles: clarify variable name holding file mode A follow-up to c01acee367ec. 'st' sounds like the whole stat result while 'mode' is a better name for the actual file mode.

changegroup: use changelogrevision() Using offsets for accessing changelog entries isn't very readable. As a bonus, changelog.changelogrevision() also accepts a revision, so we don't need to perform the inline node resolution either.

changegroup: cache changelog and manifestlog outside of loop History has taught us that repo.changelog can add significant overhead to loops. So cache the changelog instance outside of the loop to avoid the lookup. While we're here, do the same for manifestlog, since each loop would otherwise initialize a new manifestlog instance.

make: targets for building packages for ubuntu yakkety Ubuntu 16.10 Yakkety Yak is out. Let's support it.

util: put compression code next to each other ctxmanager was injecting itself between the compression and decompression code. Let's restore some order.

hgweb: make log streams compatible with command server Even though it would be useless to start a web server by a command server, it should be doable in principle. Also, we can't use sys.stdout/err directly on Python 3 because they are unicode streams.