smartset: add first and last methods In multiple places in the code, we use `someset[0]` or `someset[-1]`. This works only because the `someset` is usually a baseset. For the same reason we introduce a `first` and `last` methods to be implemented for all smartset classes.

getgraphlogrevs: remove user of baseset.append A `baseset` has multiple cached results and will get even more in the future. Making it an object "populated once" like the other smartsets makes it both safer and simpler. The append method will be removed at some point.

getlogrevs: remove user of baseset.append A `baseset` has multiple cached results and will get even more in the future. Making it an object "populated once" like the other smartsets makes it both safer and simpler. The append method will be removed at some point.

revset-last: remove user of baseset.append A `baseset` has multiple cached results and will get even more in the future. Making it an object "populated once" like the other smartsets makes it both safer and simpler. The append method will be removed at some point.

revset-limit: remove user of baseset.append A `baseset` has multiple cached results and will get even more in the future. Making it an object "populated once" like the other smartsets makes it both safer and simpler. The append method will be removed at some point.

mq: use `revs.sort()` to ensure the set is ascending Sorting is super-cheap with the new smartset class, so we can use it to enforce the order. Otherwise all smartset classes would have to allow direct indexing.

baseset: use default value instead of [] when possible For pure cleanup purposes, we replace all the occurences of `baseset([])` with `baseset()`.

generatorset: implement isascending and isdescending

generatorset: explicitly track iteration order The expected iteration order may be different than the fast iteration order (eg: ancestors(42) is expected to be iterated upward but is fast/lazy to compute downward. So we explicitly track the iteration order and enforce it if the manual iteration is requested. Default expected iteration order of a generator set is ascending because I'm not aware of any descending revset that need a generatorset. The first to find such descending revset will have the pleasure to make this configurable.

addset: drop caching through generatorset The utility of this cache is debatable (no visible benchmark impact) and using generatorset for such purpose makes the code complicated. We drop it for now. Someone can reintroduce a smart version of it in the future if it is detected to be relevant.

generatorset: get list-based fast iterations after the generator is consumed When all revisions are known, we shortcut most of the class logic to use list iteration instead. The cost of the sort is expected to be non-significant. The list creation and sorting could be done lazily in the future. We have to copy the list to not break existing iterator created before we finished consuming the generator.

generatorset: move iteration code into _iterator _iterator handles the generator iteration. The `__iter__` method will need changes to handle ordering-related information.

generatorset: stop using a base as the _genlist It does not add anything and makes it more complicated to have a simple baseset implementation.

generatorset: drop the leading underscore in the class name This is a real smart set now.

generatorset: update the docstring now that it is a smartset The documentation was still stating that this class was not a smartset. We drop that part.

addset: drop the leading underscore from the class name This class is now a real smartset.

addset: this is a smartset, update the docstring The documentation was still stating that this class is a not a smartset. We drop that part.

addset: use the ascending argument in _iterordered Fix a bug where fastasc and fastdesc were iterator in the same order as self._ascending.

rebase: add help examples

rebase: attempt to clarify --base

manifest: rearrange add() method and add comments for clarity Omit the check of bool(p1) since it's always true in practice: it will either be nullid or some valid manifest sha, and we know nullid won't ever be in the cache so we can simplify understanding of this code.

manifest: simplify manifest.add() by making args required I verified that changed was never false (it was always a 2-tuple) by adding @@ -220,6 +225,8 @@ class manifest(revlog.revlog): def add(self, map, transaction, link, p1=None, p2=None, changed=None): + if not changed: + assert False, 'changed was %r' % changed # if we're using the cache, make sure it is valid and # parented by the same node we're diffing against if not (changed and p1 and (p1 in self._mancache)): and observing that the test suite still passed. Making all the arguments required should help future readers understand what's going on here.

manifest: move manifest parsing to module-level We'll need this in the sharded manifest hashing routine, and I need to tweak it anyway, so make it module-level now.

revlog: move references to revlog.hash to inside the revlog class This will make it possible for subclasses to have different hashing schemes when appropriate. I anticipate using this in manifests. Note that there's still one client of mercurial.revlog.hash() outside of revlog: mercurial.context.memctx uses it to construct the file entries in an in-memory manifest. I don't think this will be a problem in the immediate future, so I've left it as-is.

revlog: mark nullhash as module-private No other module should ever need this, so mark it with _ so nobody tries to use it.

ui: disable echo back of prompt input if ui is set to non-tty purposely 9ab18a912c44 is nice for test output, but it also affects command-server channel. Command-server client shouldn't receive echo-back message, which makes it harder to parse the output.

dirstate: cache util.normcase while constructing the foldmap This is a small win on OS X. hg perfdirstatefoldmap: before: wall 0.399708 comb 0.410000 user 0.390000 sys 0.020000 (best of 25) after: wall 0.386331 comb 0.390000 user 0.370000 sys 0.020000 (best of 25)

normcase: for darwin, use fast ASCII lower Constructing the foldmap is much faster on OS X now. For a large real-world repo, hg perfdirstatefoldmap: before: wall 0.805278 comb 0.800000 user 0.790000 sys 0.010000 (best of 13) after: wall 0.399708 comb 0.410000 user 0.390000 sys 0.020000 (best of 25) This is a nice boost to 'hg status', especially with the third-party hgwatchman extension enabled (which eliminates stat calls). For the above repo, 'hg status' goes from 1.17 seconds to 0.74.

perf: add a way to measure the perf of constructing the foldmap Constructing the foldmap is a necessary part of operations like 'hg status' on OS X. This command allows us to measure the perf of constructing it.

encoding.lower: use fast ASCII lower This benefits, among other things, the case collision auditor. On a Linux system with a large real-world repo where all filenames are ASCII, hg perfcca: before: wall 0.260157 comb 0.270000 user 0.230000 sys 0.040000 (best of 38) after: wall 0.164616 comb 0.160000 user 0.160000 sys 0.000000 (best of 54)

parsers: add a function to efficiently lowercase ASCII strings We need a way to efficiently lowercase ASCII strings. For example, 'hg status' needs to build up the fold map -- a map from a canonical case (for OS X, lowercase) to the actual case of each file and directory in the dirstate. The current way we do that is to try decoding to ASCII and then calling lower() on the string, labeled 'orig' below: str.decode('ascii') return str.lower() This is pretty inefficient, and it turns out we can do much better. I also tested out a condition-based approach, labeled 'cond' below: (c >= 'A' && c <= 'Z') ? (c + ('a' - 'A')) : c 'cond' turned out to be slower in all cases. A 256-byte lookup table with invalid values for everything past 127 performed similarly, but this was less verbose. On OS X 10.9 with LLVM version 6.0 (clang-600.0.51), the asciilower function was run against two corpuses. Corpus 1 (list of files from real-world repo, > 100k files): orig: wall 0.428567 comb 0.430000 user 0.430000 sys 0.000000 (best of 24) cond: wall 0.077204 comb 0.070000 user 0.070000 sys 0.000000 (best of 100) lookup: wall 0.060714 comb 0.060000 user 0.060000 sys 0.000000 (best of 100) Corpus 2 (mozilla-central, 113k files): orig: wall 0.238406 comb 0.240000 user 0.240000 sys 0.000000 (best of 42) cond: wall 0.040779 comb 0.040000 user 0.040000 sys 0.000000 (best of 100) lookup: wall 0.037623 comb 0.040000 user 0.040000 sys 0.000000 (best of 100) On a Linux server-class machine with GCC 4.4.6 20120305 (Red Hat 4.4.6-4): Corpus 1 (real-world repo, > 100k files): orig: wall 0.260899 comb 0.260000 user 0.260000 sys 0.000000 (best of 38) cond: wall 0.054818 comb 0.060000 user 0.060000 sys 0.000000 (best of 100) lookup: wall 0.048489 comb 0.050000 user 0.050000 sys 0.000000 (best of 100) Corpus 2 (mozilla-central, 113k files): orig: wall 0.153082 comb 0.150000 user 0.150000 sys 0.000000 (best of 65) cond: wall 0.031007 comb 0.040000 user 0.040000 sys 0.000000 (best of 100) lookup: wall 0.028793 comb 0.030000 user 0.030000 sys 0.000000 (best of 100) SSE instructions might help even more, but I didn't experiment with those.

match: remove unnecessary setting of self._always The 'always' class calls its parent constructor with an empty list of patterns, which will result in a matcher that always matches. The parent constructor will set self._always to True in such cases, so there is no need to set it again.

bookmarks: port to generic templater

bookmarks: split ui.write() so that it can be easily ported to formatter api Test output changes because color labels are applied separately.

bookmarks: iterate bookmarks list even if it is known to be empty This clarifies that "no bookmarks set" is displayed in addition to the list of bookmarks. In JSON output, for example, [] should be written if empty, and "no bookmarks set" message should be skipped.

repoview: remove hiddencache verification We have been running hiddencache verification since 3.1.1 and so far not received a bug report concerning it. Therefore we remove the verification code and make the hiddencache authoritive. That way we get the intended speedup.

color: reorganise and sectionify the help text The color docstring was getting long. This splits it up into bite-sized sections and rearranges the order of the paragraphs a little to match these sections.

color: update description of the extension The color extension long ago ceased to work only for the status and qseries commands.

help: show all nested subsections of a section with `hg help foo.section` Used to be that `hg help hgrc.paths` would show "paths" ------- Assigns symbolic names to repositories. The left side is the symbolic name, and the right gives the directory or URL that is the location of the repository. Default paths can be declared by setting the following entries. and stop there. Obviously the result seems better as shown in the attached test.

help: fix output of sections in `hg help foo.somesection` There was a bug in c3c3dd31fe1c. The block that added definitions to getsections should have been an elif, not an if. Otherwise section titles get added twice, since the else clause would always get executed for section titles.

log: show phase in hg log -v with the phase template It seems weird that `hg log -v -T phases` would be *less* verbose than `hg log -T phases`. This cset corrects that oversight.

log: add labels to the phase template This copies the labelled default template and just adds an extra {phase} keyword as necessary.

log: rewrite default template to use labels (issue2866) This is a complete rewrite of the default template to use labels. This seems ultimately useless to me in most cases. The biggest benefit of this patch to me seems to be a fairly complicated example of the templating engine. It was a lot of hard work to figure out the precise acceptable syntax, since it's almost undocumented. Hat tip to Steve Losh's smartlog template, which helped me figure out a lot of the syntax. Hopefully later I can use the present default log template as an example for documenting the templating engine. A test is attached. My goal was to match the --color=debug output, which may differ slightly in newlines from the actual ANSI escape codes output. I consider this an acceptable invisible deviation. There seems to be a considerable slowdown with this rewrite. Before: $ time hg log -T default -r .~100::. > /dev/null real 0m0.882s user 0m0.812s sys 0m0.064s $ time hg log -T default -r .~100::. > /dev/null real 0m0.872s user 0m0.796s sys 0m0.068s $ time hg log -T default -r .~100::. > /dev/null real 0m0.917s user 0m0.836s sys 0m0.076s After: $ time hg log -T default -r .~100::. > /dev/null real 0m1.480s user 0m1.392s sys 0m0.072s $ time hg log -T default -r .~100::. > /dev/null real 0m1.500s user 0m1.400s sys 0m0.088s $ time hg log -T default -r .~100::. > /dev/null real 0m1.462s user 0m1.364s sys 0m0.092s Following the maxim, "make it work, profile, make it faster, in that order", I deem this slowdown acceptable for now. I suspect but have not confirmed that a big slowdown comes from calling keywords twice in the file templates, once to test the existence of output and again to actually list the output. If so, a simple speedup might be to improve the templating engine to cache keywords when called more than once on the same revision. TODO: I found a bug while working on this. The following stack traces: hg log -r . -T '{ifcontains(phase, "secret public", "lol", "omg")}\n'

log: do not hide the public phase in debug mode (BC) When 51fc43253a52 introduced phases to the `hg log --debug` output, it also disabled outputting public phase. At the same time, it always shows phases in the default template, `hg log --debug -T default`. Those two should produce the same output, but they don't. I think it makes a lot more sense to always show all phases. There's already loss of backwards compatibility in this case when using a newer hg on an old hg repo, since draft commits will show up in the output of `hg log --debug`. Finally, I just don't think that any sort of information should be hidden with --debug. This flag should be about showing as much information as possible.

templater: set the correct phase for parents Akin to f6371cc62d2a which did this for `hg log`, the following sets the correct phase for the {phase} keyword when the context is a parent of the current cset. This allows templates such as the following to be defined, parent = '{label("log.parent changeset.{phase}", "parent: {rev}:{node|formatnode}")}\n' which when called on a parent (e.g. with the `parents` template keyword), will produce the correct phase.

color: omit debug label output on empty strings This is most noticeable when using custom templates. Before this patch, a template like {label("foo.bar", baz)} would emit [foo.bar|] whenever baz was empty. This cset simply omits all output when baz is empty.

tests: make hghave list features alphabetically

revset: remove the now unused _descgeneratorset class

revset: use _generatorset in _revancestors The _descgeneratorset class is going away.

revset: remove now unused class _ascgeneratorset

revset: use _generatorset directly in _revdescendant _ascgeneratorset is going away.

generatorset: move membership testing on ordered gen to the main class We are phasing out the ordered version of the class to simplify the code.

generatorset: make use of the new mechanism in the subclass Until we remove them, we use the new parameter of _generatorset to make sure the code is run.

generatorset: make it possible to use gen as fastasc or fastdesc We gain a parameter to inform that the generator is ascending or descending. If the generator is ordered, it is also used for the `fastasc` or `fastdesc` version. The _ascgeneratorset and _descgeneratorset class will be removed soon.

baseset: rely on the abstractsmartset implementation for filter

_orderedsetmixin: drop this now unused class All my friends are dead.

spanset: drop _orderedsetmixin inheritance The min/max method are as well provided by abstractsmartset.

orderedlazyset: drop this now unused class All my friends are dead.

_descendant: use filteredset instead of orderedlazyset The orderedlazyset class is going away. Filteredset gives the same service.

addset: use the base implementation for ascending and descending

addset: use base implementation for __filter__

addset: use base implementation for __add__

addset: use base implementation for __sub__

addset: use base implementation for __and__

addset: promote to real smartset Better revset performance are also achieved with less overlay. There is no good reason for addset to not be a smartset. We can replace the `_orderedsetmixin` inheritance since `abstractsmartset` has efficient min and max too.

addset: add a __nonzero__ method This is required to be a full smartset (not sure what was happening before that...)

addset: offer a fastasc and fastdesc methods If the underlying object offers fast iterators, we use them to provide fast iterators too.

addset: split simple and ordered iteration We have two goals here. First, we would like to restore the former iteration order we had in 2.9. Second, we want this logic to be reusable for `fastasc` and `fastdesc` methods.

generatorset: promote to smartset This is not going to be efficient but we need all basic set classes to be smartsets for the other classes to work.

generatorset: implement __nonzero__ This is necessary to become a real smartset.

spanset: use base implementation for __add__

spanset: use base implementation for __sub__

spanset: use base implementation for __and__

spanset: use base implementation for filter

filteredset: use base implementation for filter

filteredset: use base implementation for __add__

filteredset: use base implementation for __sub__

filteredset: use base implementation for __and__

abstractsmartset: add default implementation for __sub__

abstractsmartset: add default implementation for __add__

abstractsmartset: add default implementation for __and__

abstractsmartset: add default implementation for filter

lazyset: rename the class to filteredset All smartsets try to be lazy. The purpose of this class is to apply a filter on another set. So we rename the class (and all its occurences) to `filteredset`.

lazyset: add order awareness to the class Just a bit of extra code makes the lazyset aware of order. This renders orderedlazyset useless. At some point, the `subset` will become responsible for this ordering logic. But we are not there yet because the various objects used as subsets are not good enough.

lazyset: remove min/max This is now handled by abstractsmartset.

baseset: remove min/max methods This is now handled by the base class.

abstractsmartset: add a default implementation for min and max This default implementation takes advantage of the fast iterator if available.

lazyset: drop now useless ascending/descending definition

lazyset: inherit the fastasc and fastdesc method from subset When the filtered subset has such methods, we can use them. It is implemented as properties to be able to quickly return None if no corresponding fastasc exists on the subset.

lazyset: split the iteration logic from the condition filtering logic So that the filter can be reused by `fastasc` or `fastdesc`.

spanset: do a single range check in __contains__ Now that `start <= end` is always true, we can simplify this function.

spanset: enforce the order lazily to gain `fastasc` and `fastdesc` methods Instead of having the direction of iteration enforced through the ordering of `start` and `end` attributes of spanset, we encode the iteration direction in an explicit attribute and always store start < end. The logic for sort and reverse has to be updated. The __iter__ is now based on the newly introduced `fastasc` and `fastdesc` methods. This will allow other code simplifications in the future.

abstractsmartset: document the `fastasc` and `fastdesc` attributes/methods See the in-line documentation for details. (This is the beginning of a massive overhaul of revset).

spanset: remove ascending/descending implementation We can rely on their implementation in abstractsmartset.

baseset: remove ascending/descending redefinition We can rely on the abstractsmartset implementation.

abstractsmartset: default implementation for `ascending` and `descending` These two methods are actually silly aliases for `sort()` and `sort(reverse=True)`. So we get that aliasing at the abstractsmartset level. We will slowly phase out all the custom implementations and eventually remove any mentions of it from the code.

revert: bring back usage of `subset & ps` in `parents` Changeset 95af98616aa7 switched the order of the operand of the "&" computation to work around an issue from repo-wide spanset. The need for a workaround has been alleviated by the introduction of `fullreposet`. So we restore it to normal. The benchmark shows no significant changes as expected. We also revert the bogus test change introduced by 95af98616aa7. The order is actually important.

color: update docstring for debug option 964dd1c491ca updated the format for the --color=debug option. This updates the documentation to match this new format.

color: document the possibility to colourise tabs in diffs This is a no-op change that simply suggests that tabs can be colourised, but defaults to no colour effects for tabs. This complements cset c343557a8442. Like the similar effects for phases from f8e2aebbb24c, we set the default effects to '' instead of None, so that this is a true noop change. Otherwise, the diff.tab effect would override (i.e. neutralise) the effect of the surrounding label.

contrib/synthrepo: walk a repo's directory structure during analysis Augments the analyze command to additionally walk the repo's current directory structure (or of any directory tree), counting how many files appear in which paths. This data is saved in the repo model to be used by synthesize, for creating an initial commit with many files. This change is aimed at developing, testing and measuring scaling improvements when importing/converting a large repository to mercurial.

contrib/synthrepo: generate initial repo contents using directory shape model Augments the synthesize command to use an additional parameter to the analyzed repo model: the number of files in each directory at a given snapshot. Before synthesizing history, an arbitrary number of files will be generated in a distribution matching the analyzed directory structure. Intended for developing, testing and measuring scaling improvements when importing/converting a large repository to Mercurial.

filemerge: switch the default name for internal tools from internal:x to :x

filemerge: introduce :x as short version of internal:x merge tool name "internal:" is so much typing - it is frequently specified on the command line and gives options that are longer than any other I have seen.

branches: include active, closed and current flags in template output

branches: merge white space to format string Diffs of test output should be harmless. A white space character is moved into "log.changeset" color region.

branches: port to generic templater

branches: format rev as integer that is necessary for generic templater

formatter: add general way to switch hex/short functions This seems a bit awkward, but it can avoid duplicates in annotate, tags, branches and bookmarks. I guess fm.hexfunc can eventually be removed (or redesigned) when it gets template backend.

locarepo: remove the `pull` method (API) All the logic of this function is in the `exchange.pull` function for some time. We just stop calling `localrepo.pull` in `command.pull` to have access to more information. Leaving `localrepo.pull` in place will let third-party extensions wrap it but it would never be called by `hg pull` making the wrapping useless. Therefore, the method is removed so that third-party code fail noisily and get properly upgraded.

transplant: use exchange.pull localrepo.pull is going away. See 4d52e6eb98ea for details.

convert-hg: use localrepo.pull localrepo.pull is going away. See 4d52e6eb98ea for details.

fetch: use exchange.pull localrepo.pull is going away. See 4d52e6eb98ea for details.

localrepo: use exchange.pull when cloning localrepo.pull is going away. See 4d52e6eb98ea for details.

subrepo: use exchange.pull localrepo.pull is going away, see 4d52e6eb98ea for details.

commands: directly use exchange.pull localrepo.pull is going away. See explanations in 4d52e6eb98ea.

exchange: have `pull` return the pulloperation object We mimic what was done for `push` for similar reason. We are about to drop `localrepo.pull` (for consistency with dropping `localrepo.push` and we better have an API as extensible as `push` is. Find explanations about localrepo.push removal in 4d52e6eb98ea.

revset: introduce an abstractsmartset class This class documents all methods required by a smartset. This makes it easier for people to respect the API and ensure we fail loudly when something does not. It will later also contain common default implementations for multiple methods, making it easier to have smartset classes with minimal work.

revset: add a `__nonzero__` to baseset We are about to add a base class for `baseset` with an abstract `__nonzero__` method. So we need this method to be explicitly defined to avoid issues. The built-in list object apparently does not have a `__nonzero__` and relies on `__len__` for this purpose?

revset: drop isinstance(baseset) in spanset.__sub__ As baseset now has a fast __contains___ operator, this `baseset.set()` dance is no longer needed. No regressions are visible in the benchmark.

revset: drop isinstance(baseset) in spanset.__and__ As baseset now has a fast __contains___ operator, this `baseset.set()` dance is no longer needed. No regressions are visible in the benchmark.

revset: drop isinstance(baseset) from baseset.__and__ As baseset now has a fast __contains___ operator, this `baseset.set()` dance is no longer needed. No regressions are visible in the benchmark.

revset: use direct access to __contains__ in spanset.__sub__ Using `x.__contains__(r)` instead of `r in x` does not matter for built-in type (set) but have a positive impact for all other classes. This will let us drop some usage of baseset.set() in future patches. This also probably improves some performance.

revset: use a single return statement in matcher function This makes it easy to insert post processing and debug code on the returned value.

obsolete: replace "nb" notation with "num" (for "number") As requested by Matt Mackall.

shelve: remove unused status variables

gpg: use an abort hint and don't mention --force

gpg: move test of force before status call

gpg: drop unnecessary slicing of status array The call to repo.status() does not request status for clean files, so there is no reason to slice it out from the result. Leaving the tuple untouched will simplify a future change.

tag: use an abort hint

tag: only check the status of .hgtags

tag: properly abort if an unknown or ignored .hgtags is present This was a regression introduced by 4faaa0535ea7 in 2008.

summary: remove unused code for listing ignored files The call to repo.status() does not request ignored files to be listed, so remove the code for printing them.

fetch: use cmdutil.bailifchanged() Use the existing method cmdutil.bailifchanged() instead of implementing it again in fetch.py. An effect of this is that the error messages in case of uncommited changes will be different.

ui: fix comment about non-interactive prompts This fixes my earlier in-flight addition to Mads' change.

formatter: convert booleans to json It will be used in branches output.

merge: mute the status message when bid merge kicks in Bid merge is now the default and it is not necessary to tell the user that an experimental feature kicked in. (It could however still be relevant to get a notice that it is one of the rare criss-cross merge situations so the user is warned that the situation is more tricky than usual.)

merge: use bid merge by default (BC) In most cases merges will work exactly as before. The only difference is in criss-cross merge situations where there is multiple ancestors. Instead of picking an more or less arbitrary ancestor, it will consider both ancestors and pick the best bids. Bid merge can be disabled with --config merge.preferancestor='!'.

changectx: skip all invalid merge.preferancestor values A better fix for 17011b36aac7 that will ignore other kinds of "invalid" revisions.

revset: rely on built in iterator when possible in _generatorset.__iter__ Doing manual iteration is expensible. We rely on built in list iteration whenever possible. The other case has to become a closure we cannot have a both yield and return in the same function.

revset: prefetch an attribute in _generatorset.__iter__ Python's attribute lookup are expensible, lets do less of them. This gives us a 7% speedup on this revset iteration (from 0.063403 to 0.059032)

templater: add count template filter, plus tests Previously there was no way of telling how much children or bookmarks or tags a certain changeset has in a template. It was possible to tell if a changeset has either 0 or not 0 bookmarks, but not to tell if it has 1 or 2 of them, for example. This filter, simply named count, makes it possible to count the number of items in a list or the length of a string (or, anything that python's len can count). E.g.: {children|count}, {bookmarks|count}, {file_adds|count}. Testing the filter on node hash and shortened node hash is chosen because they both have defined length. As for lists of strings - children, tags and file_adds are used, because they provide some variety and also prove that what's counted is the number of string items in the list, and not the list stringified (they are lists of non-empty, multi-character strings). Additionally, revset template function is used for testing the filter, since the combination is very flexible and will possibly be used together a lot. (The previous version of this patch had an incorrect email subject and was apparently lost - patchwork says the patch has been accepted, but it's not so. The changes between that and this patch are minimal: now the filter does not disturb the alphabetical order of function definitions and dict keys.)

push: update bookmarks (on server) within a transaction A nice side effect is that bookmarks sent through bundle2 are updated within the same transaction as all other changes.

pull: perform bookmark updates in the transaction

bookmark: add a `bmstore.recordupdate` to plug bookmarks into the transaction Instead of manually writing bookmarks when they are updated, we can just record this update to the transaction and rely on it to update the on-disk file.

bookmarks: split bookmark serialization and file handling If we want to handle bookmarks in a transaction we need to decouple the file handling and the actual production of the content. This is similar to how we handle phases in transaction.

transaction: allow generating file outside of store We allow a vfs argument to `addfilegenerator`. This allows generating files outside of the store directory like bookmarks. However, this is not really working since we do not have the infrastructure to backup and restore files outside of store. By chance, the bookmark file is already backed up by another mechanism so we can restrict this new feature to bookmarks (which is our only interest here) and proceed.

transaction: work around and document issue with file backup The backup restoration is actually hard-coded for the main file. And this hard-coded list is the only one used when repairing an interrupted transaction from another process. Solving this problem is out of the scope of this series so we document it and work around its implications.

bundle2: remove an explicit packing The final writing of the empty part was done explicitly. We now using proper pack call using symbolic constant. This open simple change in the bundle2 format.

bundle2: split test in two We split the test between the one dedicated to the binary format and the one dedicated to checking the exchange of data using pull and push.

pull: merge bookmark updates and imports We do all the things in one go now, updating existing bookmark, adding new ones, and overwriting the ones explicitly specified for --bookmark. This impacts the tests by removing some duplicated or unnecessary output.

pull: gather explicit bookmark pulls with bookmark updates There is no reason to make them at different times. So we gather them. This is the first step toward merging them.

subrepo: stop pulling bookmark manually Bookmark pulling is now done with all the other pull steps in the `exchange.pull` function.

pull: retrieve bookmarks through bundle2 We can retrieve any pushkey namespace through bundle2. So we also ask for bookmark data and use them to update the local repo.

pull: retrieve bookmarks before obsmarkers Retrieving bookmarks before obsmarkers will avoid turning some changesets hidden right before making them visible again if a bookmark keeps them visible.

pull: move bookmark pulling into its own function This requires adding an attribute on the pulloperation object. The bookmark pulling is protected behind a "todostep" as other steps of pull.

pull: perform the todostep inside functions handling old way of pulling This matches what is done during push.

bookmark: remove now unused `pushtoremote` function Everything is unified in the push discovery now.

push: gather all bookmark decisions together The discovery phases for bookmarks now use the list of explicitly pushed bookmarks to do addition, removal and overwriting. Tests are impacted because this reduces the amount of listkeys calls issued, removes some duplicated messages and improves the accuracy of some messages.

push: prepare the issue of multiple kinds of messages To gather all the bookmark pushing actions together, we need code performing those actions to be ready for them. We need to be able to produce different messages for different actions.

push: set bkresult when pushing bookmarks through bundle2

clone: remove duplicated bookmark pulling Now that all clone methods (copy, pull and push) also transport bookmarks, we can safely drop the manual pulling that was performed during clone.

clone: explicitly push bookmarks when cloning from local to remote We need to explicitly push all local bookmarks when doing the clone from a local repo to a remote one through ssh. This will let us remove the manual export of bookmarks in clone and rely on the official exchange in push and pull instead.

clone: copy `.hg/bookmarks` during copy clone Now that the standard pull function includes bookmarks, we need to ensure that a copy clone also copies the bookmarks. This will let us drop the manual bookmark pulling done independently during a clone.

pull: move bookmark movements inside the `exchange.pull` There is no reason for bookmarks to get a special treatment. As a first step we move the code as is in the `exchange.pull` function. Integration with the rest of the flow will come later. Adding bookmarks to pull means that most clone paths are now pulling bookmarks through pull. We ensure that bookmark-update messages are properly suppressed in that case. In test-pull-http.t the 'requesting all changes' message disappear because we now get the authentication error on the `listkeys`command before such message is printed.

bookmarks: allow `updatefromremote` to be quiet This will be useful to use the function during clone (clone is not printing any bookmark data right now)

util: fix sorteddict.pop When using `.pop` on such object the list was not cleared of the popped key, leading to crash.

localrepo: pass arbitrary kwargs from `repo.pull` to `exchange.pull` We'll add bookmark-related arguments to `repo.pull` so we need to widen the signature of `repo.pull`. We should probably kill `repo.pull` now that `repo.push` is dead but this is outside the scope of this series.

pull: use `other.url()` as the source of a bookmark pull We want to move the bookmarks movement into `exchange.pull`, for this purpose we need to stop relying on variables from `command.pull`.

setup: set mode 644 or 755 on installed files

branches: reduce nesting in for loop

i18n: use datapath for i18n like for templates and help To avoid circular module dependencies we initialize i18n from util when datapath is found.

help: don't search randomly for help data - trust util.datapath The search was introduced in c904e76e3834 without a convincing explanation why it should be necessary ... except for consistency with templater handling. Now, just keep it simple.

templater: don't search randomly for templates - trust util.datapath The search was introduced in 2653740d8118. It might have been necessary back then when using __file__ directly and frozen-ness wasn't considered. Now we should know exactly where the templates can be found.

templater: inline global 'path' list in templatepaths

templater: introduce templatepaths for getting paths searched for templates Avoid function with different return types depending on parameters.

util: introduce datapath for getting the location of supporting data files templates, help and locale data is normally stored as sub folders in the directory containing the source of the mercurial module. In a frozen build they live as sub folders next to 'hg.exe' and 'library.zip'. These different kind of data were handled in different ways. Unify that by introducing util.datapath. The value is computed from the environment and is always used, so we just calculate the value on module load.

util: move _hgexecutable a few lines, closer to where it is used

hgk: define bookmark colour explicitly, as Tk 8.6 has changed their meaning

largefiles: remove 'forget' list that's always empty

bookmarks: fix divergent bookmark path normalization

exchange: remove a broken i18n abuse

bookmark: make the search for divergent names more robust We translate the "url we update from" and "the url in the config" into their canonical representation. This is useful for urls that have multiple equivalent forms: /foo/bar/ == file:/foo/bar/ == file:///foo/bar eg: hg pull --config path.bar=/foo/bar/ file:/foo/bar

pull: gather all bookmark-pulling code together Pulling bookmarks is done in two rounds. First we do a simple update, then we use the content of the `bookmark` argument to possibly overwrite some bookmark with their remote location. The second step was not done right after the first one for some obscure reason. We now perform them one after the other.

push: perform bookmark export in the push function This part is responsible for adding new bookmarks on the remote. Before that, it was done on its own in `commands.push`. The export is still not integrated with the rest of the push process, but at least it now dwells in the right function.

push: add `pushoperation.bkresult` This attribute will be used to carry the result of the bookmark push (when the `push` function will effectively be doing any bookmarks pushes)

push: pass list of bookmark to `exchange.push` Currently stored but not unused. This parameter will control bookmarks explicitly pushed (added to the server if missing).

exchange: import bookmarks as bookmod Using the original names makes it difficult to use `bookmarks` as a variable name.

push: sanitize handling of bookmark push return value Mixing return and assignment does not make sense, let's unify this.

locarepo: remove the `push` method (API) All the logic of this function is in the exchange.push function for some time. We just stop calling `localrepo.push` in `command.push` to have access to more information. Leaving `localrepo.push` in place will let third-party extensions wrap it but it would never be called by `hg push` making the wrapping useless. Therefore, the method is removed so that third-party code fail noisily and get properly upgraded.

push: `exchange.push` instead of `localrepo.push` The latter is going away.

clone: use `exchange.push` instead of `localrepo.push` The latter is going away.

push: use `exchange.push` in `commands.push` To gain access to all results from the push, we need to have access to the `pushoperation` object. We call `exchange.push` to do so. It is impossible to just change the `localrepo.push` signature because the change may be too subtle to be caught by external extension wrapping `localrepo.push`. This mean we'll have to kill `localrepo.push` because just using `exchange.push` in `commands.py` would silently disable all wrapping around `localrepo.push` by third-party extensions. So we'll remove it in a later changeset to get such extensions to fail noisily.

push: `exchange.push` now returns the `pushoperation` object Returning the pushop object gives access to more information (upcoming bookmark push result for example). `localrepo.push` currently extracts the `cgresult` for callers.

push: rename `pushop.ret` to `pushop.cgresult` We are about to introduce more results-related attributes on pushop (for bookmarks) so we need a more distinctive name. We now use `cgresult` as `pulloperation` does.

obsolete: use version constants in the format mapping

obsolete: use the `version` argument in encodemarkers Recent scientific studies show that assigning a variable have no effect on a unrelated constant. We therefore use the variable where we intended to in the first place.

obsolete: gather all contents related to format version 0 in a single place All contents includes documentation, constants, and functions, so we gather all of those things into a single location. The diff is confusing because it cannot understand what code is semantically moved around in this grand migration.

revert: properly back up added files with local modification These files were previously not backed up because the backup mechanism was not smart enough. This leads to data lose for the user since uncommitted contents were discarded. We now properly move the modified version to <filename>.orig before deleting it. We have to use a small hack to do a different action if "--no-backup" is specified. This is needed because the backup process is actually a move (not a copy) so the file is already missing when we backup. The internet kitten is a bit disapointed about that, but such is life. This patch concludes the "lets refactor revert" phases. We can now open the "Lets find stupid bug with renames and merge" phases. I'm sure that now that the code is clearer we could do it in another simpler way, but I consider the current improvement good enough for now.

revert: track added files with local modifications Those files need to be backed up but are currently not. We compute the set of them to be able to use a different backup strategy in the next changeset.

revert: distinguish between "check" and "backup" strategy "check" behaves as backup did before. We check if the current file differs from destination and we create a backup if it does. This is used for untracked files that will be overwritten by formerly-deleted files. We have to do the manual check since no status output can provide the content comparison. "backup" is now doing unconditional backup. This can be used for files seen as modified compared to both the target and the working directory. In such a case, we know that the file differs from target without actually comparing any content. This new "backup" strategy will be especially useful in the case of files added between the target and the working directory -parent- with additional modifications in the working directory -itself-. In that case we know we need to back it up, but we cannot run the content check as the files does not exists in target.

revert: small refactoring in the way backup value are handled The current backup value may have two different values: 1. Do not try to do backup 2. Do backup if applicable We are about to move to: 1. Do not try to do backup 2. Do backup if applicable 3. Do backup in all cases So we change the current values to make room for the new one.

merge with stable

Added signature for changeset f768c888aaa6

Added tag 3.1.2 for changeset f768c888aaa6

parsers: fix Py2.4 argument parsing issue Since fa53d66b45a8, we were getting this strange message with Py2.4: TypeError: argument 1 must be impossible<bad format char>, not int ..because we were using the 'n' type specifier introduced in 2.5. It turns out that offset is actually a revision number index, which ought to be an int anyway. So we store it in an int, use the 'i' specifier, rely on Py_ParseTuple for range checking, and rename it to avoid type confusion.

merge with i18n

diff: document the nobinary option Since 3fbef7ac26f0, we have a diff.nobinary option. This is handy, but the only way I found out about it was by looking at the release notes for 3.1, which is not something I normally do.

i18n-pt_BR: minor fixes and rewording on histedit help text

i18n-pt_BR: synchronized with c712238c4f9b

merge with i18n

i18n-pt_BR: synchronized with 802dffd62de5

filelog: censored files compare against empty data, have 0 size To support "status" operations against working directories that are the children of censored revisions, filelog must define "cmp" and "size" for censored content.

filelog: raise CensoredNodeError when hash checks fail with censor metadata With this change, when a revlog revision hash does not match its content, and the content is empty with a special metadata key, the integrity failure is assumed to be intentionally caused to remove sensitive content from repository history. To allow different Mercurial functionality to handle this scenario differently a more specific exception is raised than "ordinary" hash failures. Alternatives to this approach include, but are not limited to: - Calling a hook when hashes mismatch to allow arbitrary tombstone validation. Cons: Irresponsibly easy to disable integrity checking altogether. - Returning empty revision data eagerly instead of raising, masking the error. Cons: Push/pull won't roundtrip the tombstone, so client repos are unusable. - Doing nothing differently at this layer. Callers must do their own detection of tombstoned data if they want to handle some hash checks and not others. - Impacts dozens of callsites, many of which don't have the revision data - Would probably be missing one or two callsites at any given time - Currently we throw a RevlogError, as do 12 other places in revlog.py. Callers would need to parse the exception message and/or ensure RevlogError is not thrown from any other part of their call tree.

error: add CensoredNodeError, will be thrown when content deliberately erased This change introduces the error plus a corresponding catch in dispatch, to provide localized error messages. The verb "censor" is used in this commit and all following to refer to erasing the content of a revlog revision (filelog, for now) without recalculating node IDs, leaving that revision invalid. Further work must be done to safely share such revision data with compliant clients. I find the analogy to censorship straightforward; for less politically charged options, consider "erase", "excise", "expunge", or "blackhole".

files: cache repo.dirstate For a large repo, 'hg files' goes from 2.27 seconds to 1.92.

files: only check for removed, not unknown or missing ctx.matches() doesn't return unknown files, and repo.dirstate[f] never returns '!' for an answer.

files: use ctx.matches instead of ctx.walk ctx.matches() is an optimized form of ctx.walk() when we don't care about the state of files on disk. For a large repo, 'hg files > /dev/null' drops from 3.7 seconds to 2.3.

files: actually filter out removed files 'hg files' makes an attempt to filter out removed files, but that doesn't work because repo.dirstate[f] returns lowercase 'r', not uppercase.

subrepo: remove superfluous newline from subrepo prompt

ui: show prompt choice if input is not a tty but is forced to be interactive The tests often set ui.interactive to control normally interactive prompts from stdin. That gave an output where it was non-obvious what prompts got which which response, and the output lacked the newline users would see after input. Instead, if the input not is a tty, write the selection and a newline.

merge with stable

help: basic support for showing only specified topic sections For instance, 'hg help config.files' will show only the Files section.

help: use OS containers for config file lists

help: support OS-specific help sections Containers of the form: .. container:: verbose.<os> are shown by default on the given OS (or with -v).

minirst: allow multiple container types for prune If we have a container of type x.y, then we can preserve it by keeping x or y.

config: don't read the same config file twice In some cases some config files would be read twice and shown twice in showconfig --debug.

templater: fix precedence of --style and --template options Since e3eb480a9391, --template option is ignored if --style is specified, which is wrong according to the doc of show_changeset(): Display format will be the first non-empty hit of: 1. option 'template' 2. option 'style' ...

shelve: avoid writing file that is never read from The contents of the .files file has not been used since 1d7a36ff2615 (shelve: use rebase instead of merge (issue4068), 2013-10-23), so stop writing it. Where we currently use the presence of the file as a check for a valid shelve name, switch to checking for the .patch file.

hgk: don't break on repositories with obsolete changesets Check the existence of a changeset before adding it to the list returned by debug-rev-list command.

test: protect the run-tests.py --json test behind an hghave rule We add a rules to detect availability of a json module and skip if json is not available.

merge with stable

hgweb: refresh hgweb.repo on phase change (issue4061) Make hgweb.refresh() also look at phaseroots file (in addition to 00changelog.i file) and reload the repo when os.stat returns different mtime or size than cached, signifying the file was modified. This way if user changes phase of a changeset (secret <-> draft), there's no need to restart hg serve to see the change.

help: fix typo in log examples

ssl: on OS X, use a dummy cert to trick Python/OpenSSL to use system CA certs This will give PKI-secure behaviour out of the box, without any configuration. Setting web.cacerts to any value or empty will disable this trick. This dummy cert trick only works on OS X 10.6+, but 10.5 had Python 2.5 which didn't have certificate validation at all.

ssl: refactor sslkwargs - move things around a bit, preparing for next change

revert: special case 'hg revert --all' On large repos, hg revert --all can take over 13 seconds. This is mainly due to it walking the tree three times: once to find the list of files in the dirstate, once to find the list of files in the target, and once to compute the status from the dirstate to the target. This optimizes the hg revert --all case to only require the final status. This speeds it up to 1.3 seconds or so (with hgwatchman enabled). Further optimizations could be done for the -r NODE and pattern cases, but they are significantly more complex.

test-commandserver: make runcommand message bolder It seems ' runcommand' is difficult to distinguish from command output. '*** runcommand' is slightly better.

test-commandserver: rewrite manual substitution of output by (glob) match

test-commandserver: remove redundant banner output Since test output was inlined, "testing <func>" message should no longer be necessary.

test-commandserver: remove .py test which was ported to .t