py3: have pycompat.ospathsep and pycompat.ossep We needed bytes version of os.sep and os.pathsep in py3 as they return unicodes.

py3: add a bytes version of os.name os.name returns unicodes on py3. Most of our checks are like os.name == 'nt' Because of the transformer, on the right hand side we have b'nt'. The condition will never satisfy even if os.name returns 'nt' as that will be an unicode. We either need to encode every occurence of os.name or have a new variable which is much cleaner. Now we have pycompat.osname. There are around 53 occurences of os.name in the codebase which needs to be replaced by pycompat.osname to support Python 3.

py3: make util.datapath a bytes variable In this patch we make util.datapath a bytes variable, but we have to pass a unicode to gettext.translation otherwise it will cry. Used pycompat.fsdecode() to decode it back to unicode as it was converted to bytes using pycompat.fsencode().

py3: add os.fsdecode() as pycompat.fsdecode() We need to use os.fsdecode() but this was not present in Python 2. So added the function in pycompat.py

statprof: return state from stop() I don't like global variables. Have stop() return the captured state so callers can pass data to the display function.

hgweb: cache fctx.parents() in annotate command (issue5414) 9c37df347485 introduced a call to fctx.parents() for each line in annotate output. This function call isn't cheap, as it requires linkrev adjustment. Since multiple lines in annotate output tend to belong to the same file revision, a cache of fctx.parents() lookups for each input should be effective in the common case. So we implement one. Since the cache has to precompute parents so an aborted generator doesn't leave an incomplete cache, we could just return a list. However, we preserve the generator for backwards compatibility. The effect of this change when requesting /annotate/96ca0ecdcfa/ browser/locales/en-US/chrome/browser/downloads/downloads.dtd on the mozilla-aurora repo is significant: p1(9c37df347485) 5.5s 9c37df347485: 66.3s this patch: 10.8s We're still slower than before. But only by ~2x instead of ~12x. On the tip revisions of layout/base/nsCSSFrameConstructor.cpp file in the mozilla-unified repo, time went from 12.5s to 14.5s and back to 12.5s. I'm not sure why the mozilla-aurora repo is so slow. Looking at the code of basefilectx.parents(), there is room for further improvements. Notably, we still perform redundant calls to filelog.renamed() and basefilectx._parentfilectx(). And basefilectx.annotate() also makes similar calls, so there is potential for object reuse. However, introducing caches here are not appropriate for the stable branch.

hghave: check darcs version more strictly test-convert-darcs.t suddenly started failing on my Debian sid machine. The reason was Darcs was upgraded from 2.12.0 to 2.12.4 so the original pattern got to match the last two digits. Fix the pattern to match 2.2+.

tests: silence output of darcs command It appears darcs is more verbose by default these days. I got test failure with Darcs 2.12.4.

manifest: remove manifest.readshallowdelta This removes manifest.readshallowdelta and converts its one consumer to use manifestlog instead.

manifest: get rid of manifest.readshallowfast This removes manifest.readshallowfast and converts it's one user to use manifestlog instead.

manifest: add shallow option to treemanifestctx.readdelta and readfast The old manifest had different functions for performing shallow reads, shallow readdeltas, and shallow readfasts. Since a lot of the code is duplicate (and since those functions don't make sense on a normal manifestctx), let's unify them into flags on the existing readdelta and readfast functions. A future diff will change consumers of these functions to use the manifestctx versions and will delete the old apis.

manifest: change manifestlog mancache to be directory based In the last patch we added a get() function that allows fetching directory level treemanifestctxs. It didn't handle caching at directory level though, so we need to change our mancache to support multiple directories.

manifest: add manifestlog.get to obtain subdirectory instances Previously manifestlog only allowed obtaining root level manifests. Future patches will need direct access to subdirectory manifests as part of changegroup creation, so let's add a get() function that knows how to deal with subdirectories.

manifest: throw LookupError if node not in revlog When accessing a manifest via manifestlog[node], let's verify that the node actually exists and throw a LookupError if it doesn't. This matches the old read behavior, so we don't accidentally return invalid manifestctxs. We do this in manifestlog instead of in the manifestctx/treemanifestctx constructors because the treemanifest code currently relies on the fact that certain code paths can produce treemanifests without touching the revlogs (and it has tests that verify things work if certain revlogs are missing entirely, so they break if we add validation that tries to read them).

revlog: optimize _chunkraw when startrev==endrev In many cases, _chunkraw() is called with startrev==endrev. When this is true, we can avoid an extra index lookup and some other minor operations. On the mozilla-unified repo, `hg perfrevlogchunks -c` says this has the following impact: ! read w/ reused fd ! wall 0.371846 comb 0.370000 user 0.350000 sys 0.020000 (best of 27) ! wall 0.337930 comb 0.330000 user 0.300000 sys 0.030000 (best of 30) ! read batch w/ reused fd ! wall 0.014952 comb 0.020000 user 0.000000 sys 0.020000 (best of 197) ! wall 0.014866 comb 0.010000 user 0.000000 sys 0.010000 (best of 196) So, we've gone from ~25x slower than batch to ~22.5x slower. At this point, there's probably not much else we can do except implement an optimized function in the index itself, including in C.

revlog: inline start() and end() for perf reasons When I implemented `hg perfrevlogchunks`, one of the things that stood out was N * _chunk() calls was ~38x slower than 1 _chunks() call. Specifically, on the mozilla-unified repo: N*_chunk: 0.528997s 1*_chunks: 0.013735s This repo has 352,097 changesets. So the average time per changeset comes out to: N*_chunk: 1.502us 1*_chunks: 0.039us If you extrapolate these numbers to a repository with 1M changesets, that comes out to 1.502s versus 0.039s, which is significant. At these latencies, Python attribute lookups and function calls matter. So, this patch inlines some code to cut down on that overhead. The impact of this patch on N*_chunk() calls is clear: ! wall 0.528997 comb 0.520000 user 0.500000 sys 0.020000 (best of 19) ! wall 0.367723 comb 0.370000 user 0.350000 sys 0.020000 (best of 27) So, we go from ~38x slower to ~27x. A nice improvement. But there's still a long way to go. It's worth noting that functionality like revsets perform changelog lookups one revision at a time. So this code path is worth optimizing.

revlog: reorder index accessors to match data structure order Index entries are ordered tuples. We have accessors in the revlog class to map tuple offsets to names. To help reinforce the order, reorder the methods so they match the order of elements in the tuple. While I'm here, also sneak in some minimal documentation.

color: add the ability to display configured style to 'debugcolor' The 'hg debugcolor' command gains a '--style' flag to display all the configured labels and their styles. This have many benefits: * discovering documented label, * checking consistency between label's style, * showing the actual style of a label.

color: sort output of 'debugcolor' The previous ordering were provided by the set. The new output is more stable and rational. In addition we have some logic to keep the '_background' version together to help readability.

color: extract color and effect display from 'debugcolor' We are about to introduce a second mode for 'hg debugcolor' that would list the known label and their configuration, so we split the code related to color and effect out of the main function.

color: restore _style global after debugcolor ran Before this change, running 'debugcolor' would destroy all color style for the rest of the process life. We now properly backup and restore the variable content. Using a global variable is sketchy in general and could probably be removed. However, this is a quest for another adventure.

color: add basic documentation to 'debugcolor' This does not hurt.

tests: merge 'test-push-hook-lock.t' into 'test-push.t' That test file is very small and is merge with the new 'test-push.t'. No logic is changed. We don't register this as a copy because is actually a "ypoc" merging two file together without replacing the destination and Mercurial cannot express that.

tests: merge 'test-push-validation.t' into 'test-push.t' That test file is very small and is merge with the new 'test-push.t'. No logic is changed but repository name are update to avoid collision. We don't register this as a copy because is actually a "ypoc" merging two file together without replacing the destination and Mercurial cannot express that.

test: rename 'test-push-r.t' to 'test-push.t' We do not have a simple test for 'hg push' but we have multiple tiny tests for various aspect of it. We'll unify them into a single file, and we start with 'test-push-r.t'. The code is unchanged but we renamed the repository used to avoid collision with other tests we'll import in coming changesets. Test timing for the record: start end cuser csys real Test 1.850 2.640 0.650 0.090 0.790 test-push-validation.t 2.640 3.520 0.760 0.090 0.880 test-push-hook-lock.t 0.000 1.850 1.560 0.210 1.850 test-push-r.t

tests: simplify command script in 'test-push-r.t' I came across this code by chance. The script of this test is a bit messy with a lot of unnecessary intermediate commands. We simplify the script and unify repository access through '-R'. In the process the update after the unbundle is dropped as it does not add anything to the tests.

py3: use encoding.environ in ui.py Using source transformer we add b'' everywhere. So there are no chances that those bytes string will work with os.environ on Py3 as that returns a dict of unicodes. We are relying on the errors, even though no error is raised even in future, these pieces of codes will tend to do wrong things. if statements can result in wrong boolean and certain errors can be raised while using this piece of code. Let's not wait for them to happen, fix what is wrong. If this patch goes in, I will try to do it for all the cases. Leaving it as it is buggy.

py3: make scmposix.userrcpath() return bytes We are making sure that we deal with bytes as much we can. This is a part of fixing functions so that they return bytes if they have to. Used encoding.environ to return bytes. After this patch, scmposix.userrcpath() returns bytes and scmutil.osrcpath() will also return bytes if the platform is posix. Functions is scmposix returns bytes on Python 3 now.

adjustlinkrev: remove unnecessary parameters Since adjustlinkrev has "self", and is a method of a filectx object, it does not need path, filelog, filenode. They can be fetched from the "self" easily.

repair: make strip() return backup file path narrowhg wants to strip some commits and then re-apply them after applying another bundle. Having repair.strip() return the bundle path will be helpful for it.

rebase: check for conflicts before continuing When there are unresolved merge conflicts, there is no reason to make the user wait for rebase to process all of the already rebased commits just to complain that it cannot do anything. Abort early.

cmdutil: refactor checkunresolved localrepo.commit had code to check for unresolved merge conflicts, it would be helpful for at least rebase to be able to use that code without calling commit().