Thu, 20 Oct 2016 21:57:32 +0900 scmutil: remove superfluous indent from termwidth()
Yuya Nishihara <yuya@tcha.org> [Thu, 20 Oct 2016 21:57:32 +0900] rev 30325
scmutil: remove superfluous indent from termwidth()
Thu, 20 Oct 2016 21:50:29 +0900 scmutil: narrow ImportError handling in termwidth()
Yuya Nishihara <yuya@tcha.org> [Thu, 20 Oct 2016 21:50:29 +0900] rev 30324
scmutil: narrow ImportError handling in termwidth() The array module must exist. It's sufficient to suppress the ImportError of termios. Also salvaged the comment why we have to handle AttributeError, from 7002bb17cc5e.
Thu, 20 Oct 2016 21:42:11 +0900 scmutil: make termwidth() obtain stdio from ui
Yuya Nishihara <yuya@tcha.org> [Thu, 20 Oct 2016 21:42:11 +0900] rev 30323
scmutil: make termwidth() obtain stdio from ui I'm getting rid of direct sys.stderr|out|in references so Py3 porting will be slightly easier.
Thu, 20 Oct 2016 21:38:44 +0900 scmutil: move util.termwidth()
Yuya Nishihara <yuya@tcha.org> [Thu, 20 Oct 2016 21:38:44 +0900] rev 30322
scmutil: move util.termwidth() I'm going to get rid of sys.stderr|out|in references from posix.termwidth(). In order to do that, termwidth() needs to take a ui, but functions in util.py shouldn't depend on a ui object. So moves termwidth() to scmutil.py.
Sun, 06 Nov 2016 00:37:50 -0700 bdiff: don't check border condition in loop
Gregory Szorc <gregory.szorc@gmail.com> [Sun, 06 Nov 2016 00:37:50 -0700] rev 30321
bdiff: don't check border condition in loop `plast = a + len - 1`. So, this "for" loop iterates from "a" to "plast", inclusive. So, `p == plast` can only be true on the final iteration of the loop. So checking for it on every loop iteration is wasteful. This patch simply decreases the upper bound of the loop by 1 and adds an explicit check after iteration for the `p == plast` case. We can't simply add 1 to the initial value for "i" because that doesn't do the correct thing on empty input strings. `perfbdiff -m 3041e4d59df2` on the Firefox repo becomes significantly faster: ! wall 0.072763 comb 0.070000 user 0.070000 sys 0.000000 (best of 100) ! wall 0.053221 comb 0.060000 user 0.060000 sys 0.000000 (best of 100) For the curious, this code has its origins in 8b067bde6679, which is the changeset that introduced bdiff.c in 2005. Also, GNU diffutils is able to perform a similar line-based diff in under 20ms. So there's likely more perf wins to be found in this code. One of them is the hashing algorithm. But it looks like mpm spent some time testing hash collisions in d0c48891dd4a. I'd like to do the same before switching away from lyhash, just to be on the safe side.
Sat, 05 Nov 2016 23:41:52 -0700 perf: add perfbdiff
Gregory Szorc <gregory.szorc@gmail.com> [Sat, 05 Nov 2016 23:41:52 -0700] rev 30320
perf: add perfbdiff bdiff shows up a lot in profiling. I think it would be useful to have a perf command that runs bdiff over and over so we can find hot spots.
Sun, 06 Nov 2016 06:54:31 +0530 help: show help for disabled extensions (issue5228)
Pulkit Goyal <7895pulkit@gmail.com> [Sun, 06 Nov 2016 06:54:31 +0530] rev 30319
help: show help for disabled extensions (issue5228) This patch does not exactly solve issue5228 but it results in a better condition on this issue. For disabled extensions, we used to parse the module and get the first occurrences of docstring and then return the first line of that as an introductory heading of extension. This is what we get today. This patch returns the whole docstring of the module as a help for extension, which is more informative. There are some modules which don't have much docstring at top level except the heading so those are unaffected by this change. To follow the existing trend of showing commands either we have to load the extension or have a very ugly parsing method which don't even assure correctness.
Sun, 06 Nov 2016 04:17:19 +0530 py3: make scmutil.rcpath() return bytes
Pulkit Goyal <7895pulkit@gmail.com> [Sun, 06 Nov 2016 04:17:19 +0530] rev 30318
py3: make scmutil.rcpath() return bytes This patch make sure scmutil.rcpath() returns bytes independent of which platform is used on Python 3. If we want to change type for windows we can just conditionalize the return variable.
Sun, 06 Nov 2016 04:10:33 +0530 py3: use pycompat.ossep at certain places
Pulkit Goyal <7895pulkit@gmail.com> [Sun, 06 Nov 2016 04:10:33 +0530] rev 30317
py3: use pycompat.ossep at certain places Certain instances of os.sep has been converted to pycompat.ossep where it was sure to use bytes only. There are more such instances which needs some more attention and will get surely.
Sun, 06 Nov 2016 03:44:44 +0530 py3: have pycompat.ospathsep and pycompat.ossep
Pulkit Goyal <7895pulkit@gmail.com> [Sun, 06 Nov 2016 03:44:44 +0530] rev 30316
py3: have pycompat.ospathsep and pycompat.ossep We needed bytes version of os.sep and os.pathsep in py3 as they return unicodes.
Sun, 06 Nov 2016 03:33:22 +0530 py3: add a bytes version of os.name
Pulkit Goyal <7895pulkit@gmail.com> [Sun, 06 Nov 2016 03:33:22 +0530] rev 30315
py3: add a bytes version of os.name os.name returns unicodes on py3. Most of our checks are like os.name == 'nt' Because of the transformer, on the right hand side we have b'nt'. The condition will never satisfy even if os.name returns 'nt' as that will be an unicode. We either need to encode every occurence of os.name or have a new variable which is much cleaner. Now we have pycompat.osname. There are around 53 occurences of os.name in the codebase which needs to be replaced by pycompat.osname to support Python 3.
Sun, 06 Nov 2016 12:18:23 +0900 py3: make util.datapath a bytes variable
Pulkit Goyal <7895pulkit@gmail.com> [Sun, 06 Nov 2016 12:18:23 +0900] rev 30314
py3: make util.datapath a bytes variable In this patch we make util.datapath a bytes variable, but we have to pass a unicode to gettext.translation otherwise it will cry. Used pycompat.fsdecode() to decode it back to unicode as it was converted to bytes using pycompat.fsencode().
Sun, 06 Nov 2016 03:12:40 +0530 py3: add os.fsdecode() as pycompat.fsdecode()
Pulkit Goyal <7895pulkit@gmail.com> [Sun, 06 Nov 2016 03:12:40 +0530] rev 30313
py3: add os.fsdecode() as pycompat.fsdecode() We need to use os.fsdecode() but this was not present in Python 2. So added the function in pycompat.py
Fri, 04 Nov 2016 20:22:37 -0700 statprof: return state from stop()
Gregory Szorc <gregory.szorc@gmail.com> [Fri, 04 Nov 2016 20:22:37 -0700] rev 30312
statprof: return state from stop() I don't like global variables. Have stop() return the captured state so callers can pass data to the display function.
Sat, 05 Nov 2016 13:20:53 +0900 hghave: check darcs version more strictly
Yuya Nishihara <yuya@tcha.org> [Sat, 05 Nov 2016 13:20:53 +0900] rev 30311
hghave: check darcs version more strictly test-convert-darcs.t suddenly started failing on my Debian sid machine. The reason was Darcs was upgraded from 2.12.0 to 2.12.4 so the original pattern got to match the last two digits. Fix the pattern to match 2.2+.
Sat, 05 Nov 2016 13:16:40 +0900 tests: silence output of darcs command
Yuya Nishihara <yuya@tcha.org> [Sat, 05 Nov 2016 13:16:40 +0900] rev 30310
tests: silence output of darcs command It appears darcs is more verbose by default these days. I got test failure with Darcs 2.12.4.
Wed, 02 Nov 2016 17:10:47 -0700 manifest: remove manifest.readshallowdelta
Durham Goode <durham@fb.com> [Wed, 02 Nov 2016 17:10:47 -0700] rev 30309
manifest: remove manifest.readshallowdelta This removes manifest.readshallowdelta and converts its one consumer to use manifestlog instead.
Wed, 02 Nov 2016 17:10:47 -0700 manifest: get rid of manifest.readshallowfast
Durham Goode <durham@fb.com> [Wed, 02 Nov 2016 17:10:47 -0700] rev 30308
manifest: get rid of manifest.readshallowfast This removes manifest.readshallowfast and converts it's one user to use manifestlog instead.
Wed, 02 Nov 2016 17:10:47 -0700 manifest: add shallow option to treemanifestctx.readdelta and readfast
Durham Goode <durham@fb.com> [Wed, 02 Nov 2016 17:10:47 -0700] rev 30307
manifest: add shallow option to treemanifestctx.readdelta and readfast The old manifest had different functions for performing shallow reads, shallow readdeltas, and shallow readfasts. Since a lot of the code is duplicate (and since those functions don't make sense on a normal manifestctx), let's unify them into flags on the existing readdelta and readfast functions. A future diff will change consumers of these functions to use the manifestctx versions and will delete the old apis.
Wed, 02 Nov 2016 17:10:47 -0700 manifest: change manifestlog mancache to be directory based
Durham Goode <durham@fb.com> [Wed, 02 Nov 2016 17:10:47 -0700] rev 30306
manifest: change manifestlog mancache to be directory based In the last patch we added a get() function that allows fetching directory level treemanifestctxs. It didn't handle caching at directory level though, so we need to change our mancache to support multiple directories.
Wed, 02 Nov 2016 17:24:06 -0700 manifest: add manifestlog.get to obtain subdirectory instances
Durham Goode <durham@fb.com> [Wed, 02 Nov 2016 17:24:06 -0700] rev 30305
manifest: add manifestlog.get to obtain subdirectory instances Previously manifestlog only allowed obtaining root level manifests. Future patches will need direct access to subdirectory manifests as part of changegroup creation, so let's add a get() function that knows how to deal with subdirectories.
Wed, 02 Nov 2016 17:33:31 -0700 manifest: throw LookupError if node not in revlog
Durham Goode <durham@fb.com> [Wed, 02 Nov 2016 17:33:31 -0700] rev 30304
manifest: throw LookupError if node not in revlog When accessing a manifest via manifestlog[node], let's verify that the node actually exists and throw a LookupError if it doesn't. This matches the old read behavior, so we don't accidentally return invalid manifestctxs. We do this in manifestlog instead of in the manifestctx/treemanifestctx constructors because the treemanifest code currently relies on the fact that certain code paths can produce treemanifests without touching the revlogs (and it has tests that verify things work if certain revlogs are missing entirely, so they break if we add validation that tries to read them).
Sun, 23 Oct 2016 10:40:33 -0700 revlog: optimize _chunkraw when startrev==endrev
Gregory Szorc <gregory.szorc@gmail.com> [Sun, 23 Oct 2016 10:40:33 -0700] rev 30303
revlog: optimize _chunkraw when startrev==endrev In many cases, _chunkraw() is called with startrev==endrev. When this is true, we can avoid an extra index lookup and some other minor operations. On the mozilla-unified repo, `hg perfrevlogchunks -c` says this has the following impact: ! read w/ reused fd ! wall 0.371846 comb 0.370000 user 0.350000 sys 0.020000 (best of 27) ! wall 0.337930 comb 0.330000 user 0.300000 sys 0.030000 (best of 30) ! read batch w/ reused fd ! wall 0.014952 comb 0.020000 user 0.000000 sys 0.020000 (best of 197) ! wall 0.014866 comb 0.010000 user 0.000000 sys 0.010000 (best of 196) So, we've gone from ~25x slower than batch to ~22.5x slower. At this point, there's probably not much else we can do except implement an optimized function in the index itself, including in C.
Sat, 22 Oct 2016 15:41:23 -0700 revlog: inline start() and end() for perf reasons
Gregory Szorc <gregory.szorc@gmail.com> [Sat, 22 Oct 2016 15:41:23 -0700] rev 30302
revlog: inline start() and end() for perf reasons When I implemented `hg perfrevlogchunks`, one of the things that stood out was N * _chunk() calls was ~38x slower than 1 _chunks() call. Specifically, on the mozilla-unified repo: N*_chunk: 0.528997s 1*_chunks: 0.013735s This repo has 352,097 changesets. So the average time per changeset comes out to: N*_chunk: 1.502us 1*_chunks: 0.039us If you extrapolate these numbers to a repository with 1M changesets, that comes out to 1.502s versus 0.039s, which is significant. At these latencies, Python attribute lookups and function calls matter. So, this patch inlines some code to cut down on that overhead. The impact of this patch on N*_chunk() calls is clear: ! wall 0.528997 comb 0.520000 user 0.500000 sys 0.020000 (best of 19) ! wall 0.367723 comb 0.370000 user 0.350000 sys 0.020000 (best of 27) So, we go from ~38x slower to ~27x. A nice improvement. But there's still a long way to go. It's worth noting that functionality like revsets perform changelog lookups one revision at a time. So this code path is worth optimizing.
(0) -30000 -10000 -3000 -1000 -300 -100 -50 -24 +24 +50 +100 +300 +1000 +3000 +10000 tip