parsers: remove unused dependency on util Parsers.py had a reference to util.sha1 which was unused. This commit removes this reference as well as the unused import of util to simplify the dependency graph. This is important for the next commit which actually relocates part of a module to eliminate a cycle.

changectx.walk: drop unnecessary call to match function If all the files in match.files() are in the context/manifest, we already know that the matcher will match each file.

merge with stable

largefiles: use the share source as the primary local store (issue4471) The benefit of retargeting the local store to the share source is that all shares will always have access to the largefiles any one of them commit, even if the user cache is deleted (which is documented to be OK to do). Further, any push into the source (and now any shares), will likewise make the largefile(s) visible to all related repositories. In order to maintain compatibility with existing repos, where the largefiles would be cached only in the local share, fallback to searching the local share if it isn't found at the share source. The unshare command should probably be taught to copy the source store into the store for the repo being unshared to complete the loop. This patch changes the test like this: @@ -159,6 +159,5 @@ $ hg share -q src share_dst --config extensions.share= $ hg -R share_dst update -r0 getting changed largefiles - large: largefile $HASH not available from file:///$TESTTMP\share_dst - 0 largefiles updated, 0 removed + 1 largefiles updated, 0 removed 1 files updated, 0 files merged, 0 files removed, 0 files unresolved The issue writeup mentions pushing a largefile from a remote repo to the main local repo, and the largefile is then not available in any shares. Since the push doesn't cache the largefile in $USERCACHE, the trashed $USERCACHE in this test is equivalent.

largefiles: use lfutil.findstorepath() when verifying a local repo

largefiles: introduce lfutil.findstorepath() The handful of direct uses of lfutil.storepath() merely need a single path to read from or write to the largefile, whether or not it exists. Most callers that care about the file existing call lfutil.findfile(), in order to fallback from the store to the user cache. localstore._verify() doesn't call lfutil.findfile(). This prevents redirecting the store to the share source because the largefiles for existing repos may not be in the source's store, so verification may fail. It can't be changed to call findfile(), because findfile() links the file from the usercache to the local store[1], and because it returns None instead of a path if the file doesn't exist. For now, this method is just a cover for lfutil.storepath(), but it will be filled out in an upcoming patch. [1] Maybe we shouldn't care? But on a filesystem that doesn't support hardlinks, then verify will take a lot longer, and start to consume disk space.

vfs: make it possible to pass multiple path elements to join os.path.join(), localrepo.join() and localrepo.wjoin() allow passing multiple path elements; vfs.join() should be as convenient.

largefiles: drop os.path reference in lfutil.storepath() localrepo.join() can concatenate multiple parts on its own.

histedit: fix preventing strips during histedit We were trying to prevent strips of important nodes during histedit, but the check was actually comparing the short hashes in the rules to the exact value the user typed in, so it only ever worked if the user typed a 12 character hash.

copies: pass changectx instead of manifest to _computenonoverlap The _computenonoverlap function takes two manifests to allow extensions to hook in and read the manifest nodes produced by the function. The remotefilelog extension actually needs the entire changectx instead (which includes the manifest) so it can prefetch the subset of files necessary for a sparse checkout (and the sparse checkout depends on which commit is being accessed, hence the need for the changectx). I have tests in the remotefilelog extension that cover this.

dirs._addpath: don't mutate Python strings after exposing them (issue4589) One of the rules of Python strings is that they're immutable. dirs._addpath breaks this assumption for performance, which is fine as long as it is done safely -- once a string is no longer internal-only it shouldn't be mutated. Unfortunately, we weren't being safe here -- we were mutating 'key' even after adding it to a dictionary. This only really affects other C code that reads strings, so it's somewhat hard to write a test for this without poking into the internal representation of the string via ctypes or similar. There is currently no C code that reads the output of the string, but there will likely be some soon as the bug indicates. There's no significant difference in performance.

parsers: check for memory allocation overflows more carefully

parsers.c: avoid implicit conversion loses integer precision warning This warning is raised by Apple LLVM version 6.0 (clang-600.0.57) (based on LLVM 3.5svn) and was introduced in 670aaee7931c

dirstate.walk: don't report same file stat multiple times dirstate.walk() generates pairs of filename and a stat-like object. After "hg mv foo Foo", it generates one pair for "foo" and one for "Foo", as it should. However, on case-insensitive file systems, when it tries to stat to get the disk state as well, it gets the same stat result for both names. This confuses at least scmutil._interestingfiles(), making it think that "foo" was forgotten rather than removed. That, in turn, makes "hg addremove" add "foo" back, resulting in both cases in the dirstate, as reported in issue4590. This change only takes care of the "if unknown" branch. A similar fix should perhaps be applied to the other branch.

repoview: avoid processing the same rev twice in _getstatichidden If a rev had multiple children, it would be added to the heap multiple times. We now ensure it is added only once.

repoview: skip public parent earlier in _getstatichidden Public changeset have nothing to offer regarding hidden changeset. Lets not add them to the heap at all.

repoview: directly skip public head in _getstatichidden Public heads have nothing to offer regarding hidden stuff, let's skip them.

repoview: simplify process in _getstatichidden Since all children are processed before their parents, we can apply the following algorithm: For each rev (descending order): * If I'm still hidden, no children will block me, * If I'm not hidden, I must remove my parent from the hidden set, This allows us to dynamically change the set of 'hidden' revisions, dropping the need for the 'actuallyhidden' dictionary and the 'blocked' boolean in the queue. As before, we start iterating from all heads and stop at the first public changesets. This ensures the hidden computation is 'O(not public())' instead of 'O(len(min(not public()):))'.

repoview: use a heap in _getstatichidden Since we want to process all non-public changesets from top to bottom, a heap seems more appropriate. This will ensure any revision is processed after all its children, opening the way to code simplification.

repoview: update documentation of _getstatichidden In 2f7cb6e6acdd, the function name, role and return was changed. But the documentation was not. This fixes it.

ssl: resolve symlink before checking for Apple python executable (issue4588) test-https.t was broken at 07fafcd4bc74 if /usr/bin/pythonX.Y is used on Mac OS X. If python executable is not named as "python", run-tests.py creates a symlink and hghave uses it. On the other hand, the installed hg executable knows the real path to the system Python. Therefore, there was an inconsistency that hghave said it was not an Apple python but hg knew it was.

test-subrepo-recursion: fix output on non-hardlink systems There's a slight bug present where a topic doesn't get closed when it should. This isn't a regression, so I've made a note in the test file.

test-subrepo-recursion: set progress.changedelay really high This avoids subtopics from showing up, which were introduced by my recent change that added progress bars to hardlink clones in some cases.

test-clone: fix test expectations on systems without hardlinks

dirstate: use parsers.make_file_foldmap when available This is a significant performance win on large repositories. perffilefoldmap: On Linux/gcc, on a test repo with over 500,000 files: before: wall 0.605021 comb 0.600000 user 0.560000 sys 0.040000 (best of 17) after: wall 0.280530 comb 0.280000 user 0.250000 sys 0.030000 (best of 35) On Mac OS X/clang, on a real-world repo with over 200,000 files: before: wall 0.281103 comb 0.280000 user 0.260000 sys 0.020000 (best of 34) after: wall 0.133622 comb 0.140000 user 0.120000 sys 0.020000 (best of 65) This visibly impacts status times on case-insensitive file systems. On the Mac OS X repo, status goes from 3.64 seconds to 3.50. With the third-party hgwatchman extension [1], 'hg status' on the same repo goes from 0.80 seconds to 0.65. [1] https://bitbucket.org/facebook/hgwatchman

parsers: add a C function to create a file foldmap This is a hot path on case-insensitive filesystems -- it's guaranteed to be called every time 'hg status' is run. This is significantly faster than the equivalent Python code: see the following patch for numbers.

util.h: define an enum for normcase specs These will be used in upcoming patches to efficiently create a dirstate foldmap.

perf: make measuring foldmap perf work again Rev 25c1d3ca5ff6 split the foldmap into two, but I forgot to update perf for the changes.

parsers._asciitransform: also accept a fallback function This function will be used in upcoming patches to provide a C implementation of the function to generate the foldmap.

util: add normcase spec and fallback These will be used in upcoming patches to efficiently create a dirstate foldmap.