py3: have pycompat.ospathsep and pycompat.ossep We needed bytes version of os.sep and os.pathsep in py3 as they return unicodes.

py3: add a bytes version of os.name os.name returns unicodes on py3. Most of our checks are like os.name == 'nt' Because of the transformer, on the right hand side we have b'nt'. The condition will never satisfy even if os.name returns 'nt' as that will be an unicode. We either need to encode every occurence of os.name or have a new variable which is much cleaner. Now we have pycompat.osname. There are around 53 occurences of os.name in the codebase which needs to be replaced by pycompat.osname to support Python 3.

py3: make util.datapath a bytes variable In this patch we make util.datapath a bytes variable, but we have to pass a unicode to gettext.translation otherwise it will cry. Used pycompat.fsdecode() to decode it back to unicode as it was converted to bytes using pycompat.fsencode().

py3: add os.fsdecode() as pycompat.fsdecode() We need to use os.fsdecode() but this was not present in Python 2. So added the function in pycompat.py

statprof: return state from stop() I don't like global variables. Have stop() return the captured state so callers can pass data to the display function.

hgweb: cache fctx.parents() in annotate command (issue5414) 9c37df347485 introduced a call to fctx.parents() for each line in annotate output. This function call isn't cheap, as it requires linkrev adjustment. Since multiple lines in annotate output tend to belong to the same file revision, a cache of fctx.parents() lookups for each input should be effective in the common case. So we implement one. Since the cache has to precompute parents so an aborted generator doesn't leave an incomplete cache, we could just return a list. However, we preserve the generator for backwards compatibility. The effect of this change when requesting /annotate/96ca0ecdcfa/ browser/locales/en-US/chrome/browser/downloads/downloads.dtd on the mozilla-aurora repo is significant: p1(9c37df347485) 5.5s 9c37df347485: 66.3s this patch: 10.8s We're still slower than before. But only by ~2x instead of ~12x. On the tip revisions of layout/base/nsCSSFrameConstructor.cpp file in the mozilla-unified repo, time went from 12.5s to 14.5s and back to 12.5s. I'm not sure why the mozilla-aurora repo is so slow. Looking at the code of basefilectx.parents(), there is room for further improvements. Notably, we still perform redundant calls to filelog.renamed() and basefilectx._parentfilectx(). And basefilectx.annotate() also makes similar calls, so there is potential for object reuse. However, introducing caches here are not appropriate for the stable branch.

hghave: check darcs version more strictly test-convert-darcs.t suddenly started failing on my Debian sid machine. The reason was Darcs was upgraded from 2.12.0 to 2.12.4 so the original pattern got to match the last two digits. Fix the pattern to match 2.2+.

tests: silence output of darcs command It appears darcs is more verbose by default these days. I got test failure with Darcs 2.12.4.

manifest: remove manifest.readshallowdelta This removes manifest.readshallowdelta and converts its one consumer to use manifestlog instead.

manifest: get rid of manifest.readshallowfast This removes manifest.readshallowfast and converts it's one user to use manifestlog instead.

manifest: add shallow option to treemanifestctx.readdelta and readfast The old manifest had different functions for performing shallow reads, shallow readdeltas, and shallow readfasts. Since a lot of the code is duplicate (and since those functions don't make sense on a normal manifestctx), let's unify them into flags on the existing readdelta and readfast functions. A future diff will change consumers of these functions to use the manifestctx versions and will delete the old apis.

manifest: change manifestlog mancache to be directory based In the last patch we added a get() function that allows fetching directory level treemanifestctxs. It didn't handle caching at directory level though, so we need to change our mancache to support multiple directories.

manifest: add manifestlog.get to obtain subdirectory instances Previously manifestlog only allowed obtaining root level manifests. Future patches will need direct access to subdirectory manifests as part of changegroup creation, so let's add a get() function that knows how to deal with subdirectories.

manifest: throw LookupError if node not in revlog When accessing a manifest via manifestlog[node], let's verify that the node actually exists and throw a LookupError if it doesn't. This matches the old read behavior, so we don't accidentally return invalid manifestctxs. We do this in manifestlog instead of in the manifestctx/treemanifestctx constructors because the treemanifest code currently relies on the fact that certain code paths can produce treemanifests without touching the revlogs (and it has tests that verify things work if certain revlogs are missing entirely, so they break if we add validation that tries to read them).