merge with stable

largefiles: don't walk through all ignored files Problem: 'hg status' with largefiles enabled would walk through all the files that .hgignore said should be ignored. That made it slow if a lot of files were .hgignored or the cache was cold. It seems like there was a reason to this, but other improvements has rendered this unnecessary. Solution: .hgignore is now only ignored when that is requested (--ignore). This is a minimal 'stable' change. There is room for other improvement.

largefiles revert: update lfdirstate with result from first cleanliness check Largefiles revert do for some reason have two lfdirstates and lfdirstatestatus invocations in one function. The result from the first lfdirstate check was however not written back to the lfdirstate, and some files was thus checked twice.

largefiles status: update lfdirstate with result from cleanliness check Problem: 'hg status' kept checking largefiles with an unknown state until some other command wrote the updated dirstate. Solution: Add missing lfdirstate.write().

bundlerepo: don't return the peer without bundlerepo from getremotechanges Problem: getremotechanges would return the 'other' repo if nothing was incoming and there thus wasn't any bundle to base the repo on. The 'other' could be a http peer which only implement the functionality available over the http protocol. Transplant could thus fail with TypeError: argument of type 'httppeer' is not iterable Solution: Return the local repo instead of the remote peer if there is no reason to place a bundlerepo on top of the local repo.

strip: do not update branchcache during strip (issue3745) At this moment, the cache is invalid, and will be thrown away. Later the strip function will call the `localrepo.destroyed` method that will update the branchmap cache.

copies: do not track backward copies, only renames (issue3739) The inverse of a rename is a rename, but the inverse of a copy is not a copy. Presenting it as such -- in particular, stuffing it into the same dict as real copies -- causes bugs because other code starts believing the inverse copies are real. The only test whose output changes is test-mv-cp-st-diff.t. When a backwards status -C command is run where a copy is involved, the inverse copy (which was hitherto presented as a real copy) is no longer displayed. Keeping track of inverse copies is useful in some situations -- composability of diffs, for example, since adding "a" followed by an inverse copy "b" to "a" is equivalent to a rename "b" to "a". However, representing them would require a more complex data structure than the same dict in which real copies are also stored.

copies: make debug messages more sensible The -> in debug messages is currently overloaded to mean both source to dest and dest to source. To fix this, we add explicit labels and make the arrow direction consistent.

copies: separate moves via directory renames from explicit copies Currently the "copy" dict contains both explicit copies/moves made by a context and pending moves that need to happen because the other context moved the directory the file was in. For explicit copies, the dict stores a destination to source map, while for pending moves via directory renames, it stores a source to destination map. The merge code uses this fact in a non- obvious way to differentiate between these two cases. We make this explicit by storing these pending moves in a separate dict. The dict still has a source to destination map, but that is called out in the docstring.

merge with stable

branchmap: move validity logic in the object itself In several place, We check if a branchcache is still valid regarding the current state of the repository. This changeset puts this logic in a method of the object that can be reused when necessary. A branch map is considered valid whenever it is up to date or a strict subset of the repository state. The change will help making branchcache aware of filtered revision. The change in keyword is expected. the branch cache is actually invalid after the amend. The previous check did not detected it.

branchmap: make update a method

branchmap: make update responsible to update the cache key The update function have all necessary data to keep the branchcache key up to date with its value. This saves assignment to the cache key that each caller of update had to do by hand. The strip case is a bit more complicated to handles from inside the function but I do not expect any impact.

branchmap: factorise changelog access in update This both improves readability and performance. Access to changelog of filtered repository currently have a minor overhead.

branchmap: make write a method on the branchmap object

branchmap: simplify write signature All necessary data (cache value and key) are now stored in the branchcache object. Any extra parameter is superfluous.

branchmap: add the tiprev (cache key) on the branchmap object The actual cache key used on disk is the (tipnode, tiprev) pair. There is no reason not to use the revision number for the in memory version.

branchmap: add the tipnode (cache key) on the branchcache object Gathering data and cache key paves the way to a lot of simplification.

branchmap: store branchcache in a dedicated object Value and key of branchcache would benefit from being hold by the same object. Moreover some logic (update, write, validation) could be move on such object. The creation of this object is the first step toward this move. The result will clarify branchcache related code and hide most of the detail in the class itself. This encapsulation will greatly helps implementation of branchcache for filtered view of the repo.

branchmap: `stream_in` write remote branchcache to local one The previous code was writing it to a non existent `branchcache` attribute. We now write is to the proper `_branchcache` attribute and initialize the `_branchcachetip` at the same time. We keep writing it to disk, the previous code had this part right.

bookmarks: fix head selection for merge with two bookmarked heads A type mismatch caused the search for the other head to fail. The code is fragile, and instead it ended up using the 'first' bookmark head, but the ordering is undefined and it could thus randomly use the wrong bookmarkhead and fail with: $ hg up -q -C e@diverged $ hg merge abort: merging with a working directory ancestor has no effect

branchmap: extract updatebranchcache from repo

branchmap: extract _updatebranchcache from repo

branchmap: _updatebranchmap does not need to be filtered The `_updatebranchmap` method on repo does not need to be filtered as all callers are already handling filtering themself. The fact it is filtered may had even lead to buggy behaviors, but by chances the method make very sparse use of the repo object.

branchmap: extract read logic from repo

branchmap: extract write logic from localrepo

branchmap: create a mercurial.branchmap module This is the foundation stone for an extraction of branches map logic from local repository class. Most of the branches map logic have very few caller and therefor does not fit in the current criteria for code held by the localrepo class. Important change will be made to this code in relation with revision filtering. So we extract it in a dedicated module before adding additional complexity. Follow up commit do the actual code movement.

bundlerepo: use _cacheabletip mechanism in bundlerepo Instead of preventing any cache write we allow writing cache for all content of the original repo. The motivation for this change is to drop the custom _writebranchcache of bundlerepo to help extraction of the branchmap logic out of localrepo.

branchmap: merge _branchtags into updatebranchcache Now that nobody overwrite it, there is no reasons for `_branchtags` to remains separated from `updatebranchcache`.

branchmap: factorise access to changelog in updatebranchcache This prepares merge of `updatebranchcache` and `_branchtags`.

branchmap: simplify _branchtags using a new _cacheabletip method The current _branchtags method is a remnant of an older, much larger function. Its only remaining role is to help MQ to alter the version of branchcache we store on disk. As MQ mutates the repository it ensures persistent cache does not contain anything it is likely to alter. This changeset makes explicit the stable vs volatile part of repository and reduces the MQ specific code to the computation of this limit. The main _branchtags code now handles this possible limit in all cases. This will help to extract the branchmap logic from the repository. The new code of _branchtags is a bit duplicated, but as I expect major refactoring of this section I'm not keen to setup factorisation function here.

hook: disable demandimport before importing hooks This solved an obscure bug for me. In upgrading Distribute on the server, a patch was added to has a try: import a except ImportError thing that's only supposed to work with Python 3.3. I'm using 2.7. My hook failed with an ImportError because of this. It seems kind of sensible to turn off demandimport before importing the hook, since the except ImportError pattern is used quite a bit in Python code (including in other Distribute code).

test-pathencode: accept --seed parameter in hex as well test-pathencode.py outputs the seed value in hex if it finds a deviation. This change allows to specify the seed value as a command line parameter for test-pathencode.py in hex as well.

subrepo: append subrepo path to subrepo error messages This change appends the subrepo path to subrepo errors. That is, when there is an error performing an operation a subrepo, rather than displaying a message such as: pushing subrepo MYSUBREPO to PATH searching for changes abort: push creates new remote head HEADHASH! hint: did you forget to merge? use push -f to force mercurial will show: pushing subrepo MYSUBREPO to PATH searching for changes abort: push creates new remote head HEADHASH! (in subrepo MYSUBREPO) hint: did you forget to merge? use push -f to force The rationale for this change is that the current error messages make it hard for TortoiseHg (and similar tools) to tell the user which subrepo caused the push failure. The "(in subrepo MYSUBREPO)" message has been added to those subrepo methods were it made sense (by using a decorator). We avoid appending "(in subrepo XXX)" multiple times when subrepos are nexted by throwing a "SubrepoAbort" exception after the extra message is appended. The decorator will then "ignore" (i.e. just re-raise) the exception and never add the message again. A small drawback of this method is that part of the exception trace is lost when the exception is catched and re-raised by the annotatesubrepoerror decorator. Also, because the state() function already printed the subrepo path when it threw an error, that error has been changed to avoid duplicating the subrepo path in the error message. Note that I have also updated several subrepo related tests to reflect these changes.

tests: fix some slash-based Windows failures

inotify: on Python < 2.6, socket.error lacks errno

hidden: invalidate hiddenrevs when needed The `hiddenrevs` cache is volatile too (It use content from `obscache`). When unsure it is invalidated when necessary. In a near future, the cache will probably be moved to `revsfiltercache`

cache: group obscache and revsfiltercache invalidation in a single function Both caches are very volatile and needs invalidation on the same kind of event. revsfiltercache actually depends on the content of revsfiltercache.

clfilter: use filtering in `visibleheads` This is the second real use of changelog filtering. The change is very small to allow testing the new filter with a setup close to the original one. We replace custom post processing on `heads`function by call to the standard code pass on a filtering repo. In later coming will have wider usage of filtering that will make the dedicated function useless.

clfilter: use filtering in `visiblebranchmap` Here is the first real use of changelog filtering. The change is very small to allow testing the new filter with a setup close to the original one. We replace custom post processing on branchmap function by call to the standard code pass on a filtering repo. In later coming will have wider usage of filtering that will make the dedicated function useless.

clfilter: introduce a "unserver" filtering mode This mode is for repository used as a server. It filter secret and hidden changeset out. It is put to use in later changeset.

clfilter: add a cache on repo for set of revision to filter for a given set. Recomputing the filtered revisions at every access to changelog is far too expensive. This changeset introduce a cache for this information. This cache is hold by the repository (unfiltered repository) and invalidated when necessary. This cache is not a protected attribute (leading _) because some logic that invalidate it is not held by the local repo itself.

clfilter: add actual repo filtering mechanism We add a `filtered` method on repo. This method return an instance of `repoview` that behaves exactly as the original repository but with a filtered changelog attribute. Filters are identified by a "name". Planned filter are `unserved`, `hidden` and `mutable`. Filtering the repository in place what out of question as it wont not allows multiple thread to share the same repo. It would makes control of the filtering scope harder too. See the `repoview` docstring for details. A mechanism to compute filtered revision is also installed. Some caches will be installed in later commit.

inotify: pacify pestiferous pyflakes precipitously

tests: make test-inotify-issue1208.t disappear

posix: move server side of unix domain sockets out of inotify We also turn the unix domain socket into a class, so that we have a sensible place to hang its logically related attributes and behaviour. We'll shortly want to reuse this in other code.

inotify: don't fall over just because of a dangling symlink Previously, the inotify server failed to start if .hg/inotify.sock was a symlink that pointed to a non-existent path. This behaviour does not seem to make any sense. Now, if we encounter a broken symlink, we unlink it and continue.

test-inotify: test symlink indirection for unix sockets The inotify code performs a delicate dance to work around the 108-byte limit on unix domain socket path names on Linux. This change sets us up to safely refactor that code without breaking it. (It is redundant with part of test-inotify-issue1208.t, but we will shortly make that test go away.)

test-pathencode: compare current pathencoding implementations We already have two implementations of the pathencoding (C and Python) and this test can perfectly well be used to probabilistically test them instead of just wasting CPU cycles and test time.

rebase: use lazy ancestor membership testing For a repository with over 400,000 commits, rebasing one revision near tip, this avoids one walk up the DAG, speeding the operation up by around 0.8 seconds.

localrepo: use lazy ancestor membership testing For a repository with over 400,000 commits, rebasing one revision near tip, this avoids two treks up the DAG, speeding the operation up by around 1.6 seconds.

ancestor: add lazy membership testing to lazyancestors This also makes the perfancestorset command use lazy membership testing. In a linear repository with over 400,000 commits, without this patch, hg perfancestorset takes 0.80 seconds no matter how far behind we're looking. With this patch, hg perfancestorset -- X takes: Rev X Time -1 0.00s -4000 0.01s -20000 0.04s -80000 0.17s -200000 0.43s -300000 0.69s 0 0.88s Thus, for revisions close to tip, we're up to several orders of magnitude faster. At 0 we're around 10% slower.

revlog: move ancestor generation out to a new class This refactoring is to prepare for implementing lazy membership.

ignore: process hgignore files in deterministic order Previously, we processed them in whatever order the dict iterator gave us.

ignore: only read an ignore file once

ignore: refactor ignore into two functions This prepares us for eventually being able to hash the list of patterns in use.

clfilter: fix `nodemap` usage in `getbundle` With the current implementation, `changelog.nodemap` is not filtered. So some filtered changeset in common are not filtered by `n in nodemap`. This leads to crash lower in the stack when the bundle generation try to access those node on a filtered changelog.

test: fix truncated comment in test The push is made to ensure repository are related. The comment in the initial changeset got truncated somehow.

clfilter: ensure context raise RepoLookupError when the revision is filtered Currently the code path of `changectx(filteredrepo, rev)` call `filteredrepo.changelog.node(rev)`. When `rev` is filtered this raise an unhandled `IndexError`. This case now raise a `RepoLookupError` as other error case do.

revlog: remove incancestors since it is no longer used