checkheads: extract obsolete post processing in its own function The checkheads function is long and complex, extract that logic in a subfunction is win in itself. As the comment in the code says, this postprocessing is currently very basic and either misbehave or fails to detect valid push in many cases. My deeper motive for this extraction is to be make it easier to provide extensive testing of this case and strategy to cover them. Final test and logic will makes it to core once done.

tests: make test-simplekeyvaluefile.py py2.6-compatible Python 2.6 unittest.TestCase does not have assertRaisesRegexp.

similar: use cheaper hash() function to test exact matches We just need a hash table {fctx.data(): fctx} which doesn't keep fctx.data() in memory. Let's simply use hash(fctx.data()) to put data out from memory, and manage collided fctx objects by list. This isn't significantly faster than using sha1, but is more correct as we know SHA-1 collision attack is getting practical. Benchmark with 50k added/removed files, on tmpfs: $ hg addremove --dry-run --time -q previous: real 12.420 secs (user 11.120+0.000 sys 1.280+0.000) this patch: real 12.350 secs (user 11.210+0.000 sys 1.140+0.000)

similar: take the first match instead of the last It seems more natural. This makes the next patch slightly cleaner.

similar: do not look up and create filectx more than once Benchmark with 50k added/removed files, on tmpfs: $ hg addremove --dry-run --time -q previous: real 16.070 secs (user 14.470+0.000 sys 1.580+0.000) this patch: real 12.420 secs (user 11.120+0.000 sys 1.280+0.000)

similar: use common names for changectx variables We generally use 'wctx' and 'pctx' for working context and its parent respectively.

similar: get rid of quadratic addedfiles.remove() Instead, build a set of files to be removed and recreate addedfiles only if necessary. Benchmark with 50k added/removed files, on tmpfs: $ hg addremove --dry-run --time -q original: real 16.550 secs (user 15.000+0.000 sys 1.540+0.000) previous: real 16.730 secs (user 15.280+0.000 sys 1.440+0.000) this patch: real 16.070 secs (user 14.470+0.000 sys 1.580+0.000)

similar: sort files not by object id but by path for stable result Perhaps the original implementation would want to sort added/removed files alphabetically, but actually it did sort fctx objects by memory location. This patch removes the use of set()s in order to preserve the order of added/removed files. addedfiles.remove() becomes quadratic, but its cost appears not dominant. Anyway, the quadratic behavior will be eliminated by the next patch. Benchmark with 50k added/removed files, on tmpfs: $ mkdir src $ for n in `seq 0 49`; do > mkdir `printf src/%02d $n` > done $ for n in `seq 0 49999`; do > f=`printf src/%02d/%05d $(($n/1000)) $n` > dd if=/dev/urandom of=$f bs=8k count=1 status=none > done $ hg ci -qAm 'add 50k files of random content' $ mv src dest $ hg addremove --dry-run --time -q original: real 16.550 secs (user 15.000+0.000 sys 1.540+0.000) this patch: real 16.730 secs (user 15.280+0.000 sys 1.440+0.000)

debugfsinfo: print fstype information Since we have osutil.getfstype, it'll be handy if "debugfsinfo" prints it.

util: enable hardlink for copyfile This patch removes the global variable "allowhardlinks" that disables hardlink in all cases, so hardlink gets enabled if the filesystem type is whitelisted. Third party extensions wanting to enable hardlink support unconditionally can replace "_hardlinkfswhitelist.__contains__".

hghave: add a check about whitelisted filesystem that supports hardlink This is needed for the test added by the next patch.

util: disable hardlink for copyfile if fstype is outside a whitelist Since osutil.getfstype is available, use it to detect filesystem types. The whitelist currently includes common local filesystems on Linux where they should have good hardlink support. We may add new filesystems for other platforms later.

revlog: use pycompat.maplist to eagerly evaluate map on Python 3 According to Pulkit, this should fix `hg status --all` on Python 3.

py3: stop exporting urlparse from pycompat and util (API) There are no consumers of this in tree. Functions formerly available on this object/module can now be accessed via {pycompat,util}.urlreq.