branchmap: use a different file name for filtered view of repo

clfilter: ensure unfiltered repo have a filtername attribute too That will allows to use `repo.filtername` for dispatch purpose.

branchmap: move the cache file name into a dedicated function Filtered view of the repo will want to write they file name in a different file.

branchmap: read and write key part related to filtered revision Now that we have a third part for the cache key we need to write and read it on disk. It is only written when there is filtered revision. This keep the format compatible with older version. Notes that, at this state, filtered repository does not use any disk caches yet.

check-code: disallow defunct <> operator Added a test for that and one other python3 check in test-check-code.t.

largefiles: stop using <> operator in favor of != <> has been deprecated for a while, and != is futureproof

commands: fix implicit tuple that is invalid syntax in Python3

check-code: disallow two-argument form of raise Using this old form makes any attempt to port to Python 3 harder, and the new syntax is supported in 2.4 already.

httpclient: apply change df9aea1def3e: remove use of two-argument raise

test-symlink-os-yes-fs-no.py: clean up use of two-argument raise This makes any attempt to port to Python 3 harder, and the new syntax is supported in 2.4 already.

scmutil: clean up use of two-argument raise This makes any attempt to port to Python 3 harder, and the new syntax is supported in 2.4 already.

url: clean up use of two-argument raise This makes any attempt to port to Python 3 harder, and the new syntax is supported in 2.4 already.

win32: clean up use of two-argument raise This makes any attempt to port to Python 3 harder, and the new syntax is supported in 2.4 already.

commandserver: clean up use of two-argument raise This makes any attempt to port to Python 3 harder, and the new syntax is supported in 2.4 already.

test-command-template.t: fix test so it all year This test started failing for me after midnight UTC on December 31st. Fixed it by specifying a date 7 years in the future more precisely (rather than just adding 8 to the year and specifying January 1st), which allows the test to pass both now and on 2012-12-01 at the same time.

cmdutil: make getgraphlogrevs limit-aware For a repository with over 400,000 changesets, this speeds up graphlog with a small limit by around 0.05 seconds (~50%).

cmdutil: stop pretending we can calculate revs for graphlog lazily cmdutil.getgraphlogrevs does a ton of work trying to build a graphlog lazily, and then cmdutil.graphlog comes along and destroys all of that. graphmod.dagwalker requires that it be given the full list of revs upfront so that it can perform filtering and tests against known revs. For a repository with over 400,000 changesets, this speeds up graphlog by around 0.02 seconds (~20% with a small limit).

cmdutil: store a local ref to repo.hiddenrevs in getgraphlogrevs On a repository with over 400,000 changesets, this speeds graphlog up by around 0.03 seconds (~20% with a small limit).

cmdutil: make getgraphlogrevs return revs in descending order

branchmap: takes filtered revision in account for cache calculation Tracking tipnode and tiprev is not enough to ensure validaty of the cache as they do not help distinguish a cache that ignored various revisions below tiprev. To detect such difference, we build a hash of all ignored revisions. This hash is then used when checking the validity of a cache for a repo.

branchmap: improve computation of target tip With revision filtering the effective revision number of "tip" may be lower than: len(changelog) - 1 We now use a more correct version preventing useless writing on disk in some case.

branchmap: improve invalid cache message when reading This factors out the generation of the message. This helps future error reporting when reading cache for filtered repository.

histedit: allow operation from non-head if obsolete is enabled Obsolescence markers can represent this situation just fine. Rewritten revisions are marked as precursors of the ones creates by histedit. Unaffected descendants become "unstable". If obsolescence is not enabled we keep the current behavior of aborting. This new behavior only applies when obsolete is enabled and is subject to future discussion and changes.

rebase: allow non-head rebase-set when obsolete is enabled Obsolescence markers can represent this situation just fine. Rebased revisions are marked as precursors of the ones create by rebase. Unrebased descendants becomes "unstable". If obsolescence is not enabled we keep the current behavior of aborting. This new behavior only applies when obsolete is enabled and is subject to future discussion and changes.

amend: allow amend of non-head when obsolete is enabled Obsolescence marker can represent this situation just fine. The old version is marked as precursor of the new changeset. All its descendants become "unstable". If obsolescence is not enabled we keep the current behavior of aborting. This new behavior only applies when obsolete is enabled and is subject to future discussion and changes.

obsolete: factorise troubles detection during push The use of the two methods introduced earlier allows a factorisation of the push code preventing push of troubled changeset.

obsolete: introduce a troubles method on context A troubled changeset may be affected by multiple trouble at the same time. This new method returns a list of all troubles affecting a changes.

obsolete: introduce a troubled method on context Allows to quickly check if a changeset is affected by any troubles. (troubles are: unstable, bumped and divergent)

tests: fix windows test failures

merge with stable

tests: fix windows test failures

largefiles: better test coverage of wireproto and the http protocol

largefiles: cleanup of warnings on errors getting largefiles Especially the "no default or default-push path set in hgrc" was often very misleading and didn't give any hint where it actually was looking. A long error messages is better than several multi-line messages.

largefiles: remove trivial portability wrappers

largefiles: remove reporemove portability wrapper

largefiles: remove findoutgoing portability wrapper

largefiles: use constant for '.hglf/' Problem: '.hglf' + '/' was computed in some tight loops. There is no need for that.

largefiles: use plain wjoin instead of the complex pathto

largefiles: unindent code

largefiles: simplify lfdirstate ignore handling - it is only for tracking .hglf

largefiles: rename 'admin' to more descriptive 'lfstoredir'

largefiles: simplify lfutil.getstandinmatcher by inlining getmatcher

largefiles: remove overly complex handling of ignored and unknown files

largefiles: introduce basic debugstate --large functionality Very useful for debugging largefiles "performance" issues. This is used for testing 03faf12fbee7.

util: fold ENOENT check into unlinkpath, controlled by new ignoremissing flag Refactor a common pattern.

merge with stable

largefiles: don't walk through all ignored files Problem: 'hg status' with largefiles enabled would walk through all the files that .hgignore said should be ignored. That made it slow if a lot of files were .hgignored or the cache was cold. It seems like there was a reason to this, but other improvements has rendered this unnecessary. Solution: .hgignore is now only ignored when that is requested (--ignore). This is a minimal 'stable' change. There is room for other improvement.

largefiles revert: update lfdirstate with result from first cleanliness check Largefiles revert do for some reason have two lfdirstates and lfdirstatestatus invocations in one function. The result from the first lfdirstate check was however not written back to the lfdirstate, and some files was thus checked twice.

largefiles status: update lfdirstate with result from cleanliness check Problem: 'hg status' kept checking largefiles with an unknown state until some other command wrote the updated dirstate. Solution: Add missing lfdirstate.write().

bundlerepo: don't return the peer without bundlerepo from getremotechanges Problem: getremotechanges would return the 'other' repo if nothing was incoming and there thus wasn't any bundle to base the repo on. The 'other' could be a http peer which only implement the functionality available over the http protocol. Transplant could thus fail with TypeError: argument of type 'httppeer' is not iterable Solution: Return the local repo instead of the remote peer if there is no reason to place a bundlerepo on top of the local repo.

strip: do not update branchcache during strip (issue3745) At this moment, the cache is invalid, and will be thrown away. Later the strip function will call the `localrepo.destroyed` method that will update the branchmap cache.

copies: do not track backward copies, only renames (issue3739) The inverse of a rename is a rename, but the inverse of a copy is not a copy. Presenting it as such -- in particular, stuffing it into the same dict as real copies -- causes bugs because other code starts believing the inverse copies are real. The only test whose output changes is test-mv-cp-st-diff.t. When a backwards status -C command is run where a copy is involved, the inverse copy (which was hitherto presented as a real copy) is no longer displayed. Keeping track of inverse copies is useful in some situations -- composability of diffs, for example, since adding "a" followed by an inverse copy "b" to "a" is equivalent to a rename "b" to "a". However, representing them would require a more complex data structure than the same dict in which real copies are also stored.

copies: make debug messages more sensible The -> in debug messages is currently overloaded to mean both source to dest and dest to source. To fix this, we add explicit labels and make the arrow direction consistent.

copies: separate moves via directory renames from explicit copies Currently the "copy" dict contains both explicit copies/moves made by a context and pending moves that need to happen because the other context moved the directory the file was in. For explicit copies, the dict stores a destination to source map, while for pending moves via directory renames, it stores a source to destination map. The merge code uses this fact in a non- obvious way to differentiate between these two cases. We make this explicit by storing these pending moves in a separate dict. The dict still has a source to destination map, but that is called out in the docstring.

merge with stable

branchmap: move validity logic in the object itself In several place, We check if a branchcache is still valid regarding the current state of the repository. This changeset puts this logic in a method of the object that can be reused when necessary. A branch map is considered valid whenever it is up to date or a strict subset of the repository state. The change will help making branchcache aware of filtered revision. The change in keyword is expected. the branch cache is actually invalid after the amend. The previous check did not detected it.

branchmap: make update a method

branchmap: make update responsible to update the cache key The update function have all necessary data to keep the branchcache key up to date with its value. This saves assignment to the cache key that each caller of update had to do by hand. The strip case is a bit more complicated to handles from inside the function but I do not expect any impact.

branchmap: factorise changelog access in update This both improves readability and performance. Access to changelog of filtered repository currently have a minor overhead.

branchmap: make write a method on the branchmap object

branchmap: simplify write signature All necessary data (cache value and key) are now stored in the branchcache object. Any extra parameter is superfluous.

branchmap: add the tiprev (cache key) on the branchmap object The actual cache key used on disk is the (tipnode, tiprev) pair. There is no reason not to use the revision number for the in memory version.

branchmap: add the tipnode (cache key) on the branchcache object Gathering data and cache key paves the way to a lot of simplification.

branchmap: store branchcache in a dedicated object Value and key of branchcache would benefit from being hold by the same object. Moreover some logic (update, write, validation) could be move on such object. The creation of this object is the first step toward this move. The result will clarify branchcache related code and hide most of the detail in the class itself. This encapsulation will greatly helps implementation of branchcache for filtered view of the repo.

branchmap: `stream_in` write remote branchcache to local one The previous code was writing it to a non existent `branchcache` attribute. We now write is to the proper `_branchcache` attribute and initialize the `_branchcachetip` at the same time. We keep writing it to disk, the previous code had this part right.

bookmarks: fix head selection for merge with two bookmarked heads A type mismatch caused the search for the other head to fail. The code is fragile, and instead it ended up using the 'first' bookmark head, but the ordering is undefined and it could thus randomly use the wrong bookmarkhead and fail with: $ hg up -q -C e@diverged $ hg merge abort: merging with a working directory ancestor has no effect

branchmap: extract updatebranchcache from repo

branchmap: extract _updatebranchcache from repo

branchmap: _updatebranchmap does not need to be filtered The `_updatebranchmap` method on repo does not need to be filtered as all callers are already handling filtering themself. The fact it is filtered may had even lead to buggy behaviors, but by chances the method make very sparse use of the repo object.

branchmap: extract read logic from repo

branchmap: extract write logic from localrepo

branchmap: create a mercurial.branchmap module This is the foundation stone for an extraction of branches map logic from local repository class. Most of the branches map logic have very few caller and therefor does not fit in the current criteria for code held by the localrepo class. Important change will be made to this code in relation with revision filtering. So we extract it in a dedicated module before adding additional complexity. Follow up commit do the actual code movement.

bundlerepo: use _cacheabletip mechanism in bundlerepo Instead of preventing any cache write we allow writing cache for all content of the original repo. The motivation for this change is to drop the custom _writebranchcache of bundlerepo to help extraction of the branchmap logic out of localrepo.

branchmap: merge _branchtags into updatebranchcache Now that nobody overwrite it, there is no reasons for `_branchtags` to remains separated from `updatebranchcache`.

branchmap: factorise access to changelog in updatebranchcache This prepares merge of `updatebranchcache` and `_branchtags`.

branchmap: simplify _branchtags using a new _cacheabletip method The current _branchtags method is a remnant of an older, much larger function. Its only remaining role is to help MQ to alter the version of branchcache we store on disk. As MQ mutates the repository it ensures persistent cache does not contain anything it is likely to alter. This changeset makes explicit the stable vs volatile part of repository and reduces the MQ specific code to the computation of this limit. The main _branchtags code now handles this possible limit in all cases. This will help to extract the branchmap logic from the repository. The new code of _branchtags is a bit duplicated, but as I expect major refactoring of this section I'm not keen to setup factorisation function here.

hook: disable demandimport before importing hooks This solved an obscure bug for me. In upgrading Distribute on the server, a patch was added to has a try: import a except ImportError thing that's only supposed to work with Python 3.3. I'm using 2.7. My hook failed with an ImportError because of this. It seems kind of sensible to turn off demandimport before importing the hook, since the except ImportError pattern is used quite a bit in Python code (including in other Distribute code).

test-pathencode: accept --seed parameter in hex as well test-pathencode.py outputs the seed value in hex if it finds a deviation. This change allows to specify the seed value as a command line parameter for test-pathencode.py in hex as well.

subrepo: append subrepo path to subrepo error messages This change appends the subrepo path to subrepo errors. That is, when there is an error performing an operation a subrepo, rather than displaying a message such as: pushing subrepo MYSUBREPO to PATH searching for changes abort: push creates new remote head HEADHASH! hint: did you forget to merge? use push -f to force mercurial will show: pushing subrepo MYSUBREPO to PATH searching for changes abort: push creates new remote head HEADHASH! (in subrepo MYSUBREPO) hint: did you forget to merge? use push -f to force The rationale for this change is that the current error messages make it hard for TortoiseHg (and similar tools) to tell the user which subrepo caused the push failure. The "(in subrepo MYSUBREPO)" message has been added to those subrepo methods were it made sense (by using a decorator). We avoid appending "(in subrepo XXX)" multiple times when subrepos are nexted by throwing a "SubrepoAbort" exception after the extra message is appended. The decorator will then "ignore" (i.e. just re-raise) the exception and never add the message again. A small drawback of this method is that part of the exception trace is lost when the exception is catched and re-raised by the annotatesubrepoerror decorator. Also, because the state() function already printed the subrepo path when it threw an error, that error has been changed to avoid duplicating the subrepo path in the error message. Note that I have also updated several subrepo related tests to reflect these changes.

tests: fix some slash-based Windows failures

inotify: on Python < 2.6, socket.error lacks errno

hidden: invalidate hiddenrevs when needed The `hiddenrevs` cache is volatile too (It use content from `obscache`). When unsure it is invalidated when necessary. In a near future, the cache will probably be moved to `revsfiltercache`

cache: group obscache and revsfiltercache invalidation in a single function Both caches are very volatile and needs invalidation on the same kind of event. revsfiltercache actually depends on the content of revsfiltercache.

clfilter: use filtering in `visibleheads` This is the second real use of changelog filtering. The change is very small to allow testing the new filter with a setup close to the original one. We replace custom post processing on `heads`function by call to the standard code pass on a filtering repo. In later coming will have wider usage of filtering that will make the dedicated function useless.

clfilter: use filtering in `visiblebranchmap` Here is the first real use of changelog filtering. The change is very small to allow testing the new filter with a setup close to the original one. We replace custom post processing on branchmap function by call to the standard code pass on a filtering repo. In later coming will have wider usage of filtering that will make the dedicated function useless.

clfilter: introduce a "unserver" filtering mode This mode is for repository used as a server. It filter secret and hidden changeset out. It is put to use in later changeset.

clfilter: add a cache on repo for set of revision to filter for a given set. Recomputing the filtered revisions at every access to changelog is far too expensive. This changeset introduce a cache for this information. This cache is hold by the repository (unfiltered repository) and invalidated when necessary. This cache is not a protected attribute (leading _) because some logic that invalidate it is not held by the local repo itself.

clfilter: add actual repo filtering mechanism We add a `filtered` method on repo. This method return an instance of `repoview` that behaves exactly as the original repository but with a filtered changelog attribute. Filters are identified by a "name". Planned filter are `unserved`, `hidden` and `mutable`. Filtering the repository in place what out of question as it wont not allows multiple thread to share the same repo. It would makes control of the filtering scope harder too. See the `repoview` docstring for details. A mechanism to compute filtered revision is also installed. Some caches will be installed in later commit.

inotify: pacify pestiferous pyflakes precipitously

tests: make test-inotify-issue1208.t disappear

posix: move server side of unix domain sockets out of inotify We also turn the unix domain socket into a class, so that we have a sensible place to hang its logically related attributes and behaviour. We'll shortly want to reuse this in other code.

inotify: don't fall over just because of a dangling symlink Previously, the inotify server failed to start if .hg/inotify.sock was a symlink that pointed to a non-existent path. This behaviour does not seem to make any sense. Now, if we encounter a broken symlink, we unlink it and continue.

test-inotify: test symlink indirection for unix sockets The inotify code performs a delicate dance to work around the 108-byte limit on unix domain socket path names on Linux. This change sets us up to safely refactor that code without breaking it. (It is redundant with part of test-inotify-issue1208.t, but we will shortly make that test go away.)

test-pathencode: compare current pathencoding implementations We already have two implementations of the pathencoding (C and Python) and this test can perfectly well be used to probabilistically test them instead of just wasting CPU cycles and test time.

rebase: use lazy ancestor membership testing For a repository with over 400,000 commits, rebasing one revision near tip, this avoids one walk up the DAG, speeding the operation up by around 0.8 seconds.

localrepo: use lazy ancestor membership testing For a repository with over 400,000 commits, rebasing one revision near tip, this avoids two treks up the DAG, speeding the operation up by around 1.6 seconds.

ancestor: add lazy membership testing to lazyancestors This also makes the perfancestorset command use lazy membership testing. In a linear repository with over 400,000 commits, without this patch, hg perfancestorset takes 0.80 seconds no matter how far behind we're looking. With this patch, hg perfancestorset -- X takes: Rev X Time -1 0.00s -4000 0.01s -20000 0.04s -80000 0.17s -200000 0.43s -300000 0.69s 0 0.88s Thus, for revisions close to tip, we're up to several orders of magnitude faster. At 0 we're around 10% slower.

revlog: move ancestor generation out to a new class This refactoring is to prepare for implementing lazy membership.

ignore: process hgignore files in deterministic order Previously, we processed them in whatever order the dict iterator gave us.

ignore: only read an ignore file once

ignore: refactor ignore into two functions This prepares us for eventually being able to hash the list of patterns in use.

clfilter: fix `nodemap` usage in `getbundle` With the current implementation, `changelog.nodemap` is not filtered. So some filtered changeset in common are not filtered by `n in nodemap`. This leads to crash lower in the stack when the bundle generation try to access those node on a filtered changelog.

test: fix truncated comment in test The push is made to ensure repository are related. The comment in the initial changeset got truncated somehow.

clfilter: ensure context raise RepoLookupError when the revision is filtered Currently the code path of `changectx(filteredrepo, rev)` call `filteredrepo.changelog.node(rev)`. When `rev` is filtered this raise an unhandled `IndexError`. This case now raise a `RepoLookupError` as other error case do.

revlog: remove incancestors since it is no longer used

transplant: replace incancestors uses with ancestors

revlog.ancestors: add support for including revs This is in preparation for an upcoming refactoring. This also fixes a bug in incancestors, where if an element of revs was an ancestor of another it would be generated twice.

perf: add command to test performance of membership in ancestor set The new command, perfancestorset, takes an argument denoting which revset to test the membership of. Currently this runs through all the ancestors and converts them into a set. The primary purpose of having this is to compare this approach, currently used in several places, against the upcoming lazy approach.

ancestor: move missingancestors doctest out into a separate file This is in preparation for upcoming patches which will reuse the same graph for tests.

merge with crew-stable

dirstate: remove obsolete comment from setbranch This comment should have been removed in b74361cf7c0a, when the call to scmutil.checknewlabel was removed.

dirstate: don't rename branch file if writing it failed

commands: 'hg bookmark NAME' should work even with ui.strict=True Before this patch, enabling strict command processing (ui.strict=True) meant that 'hg bookmark NAME', as referenced several places in the documentation, would not work. This adds 'bookmark' as an explicit alias to 'bookmarks'.

tests: improve test-mv-cp-st-diff.t readability

obsolete: refuse to push divergent changeset As other troubles `unstable` and `bumped`. Followup patches may simplify the push code with unification of "obsolescence troubles" handling.

obsolete: add a divergent method on context The same we have `unstable` and `bumped`. Convenient method to access troubles information in general may land later. This get actual use and testing in the next changesets.

obsolete: add revset and test for divergent changesets This changesets add a new `divergent()` revset similar to `unstable()` and `bumped()` one. Introducting this revset allows actuall test of the divergent detection.

obsolete: detect divergent changesets Divergent changeset are final successors (non obsolete) of a changeset who compete with another set of final successors for this same changeset. For example if you have two obsolescence markers A -> B and A -> C, B and C are both "divergent" because they compete to be the one true successors of A. Public revision can't be divergent. This function is used and tested in the next changeset.

obsolete: drop successors sets which are subset of another one If both "(B,)" and "(B, C)" are successors set of "A", "(B,)" is dropped. We won't be interrested in detection such divergence scenario.

obsolete: compute successors set Successors set are an important part of obsolescence. It is necessary to detect and solve divergence situation. This changeset add a core function to compute them, a debug command to audit them and solid test on the concept. Check function docstring for details about the concept.