hgk: add .diff extension when exporting commits Append the .diff extension automatically.

localrepo: move the addchangegroup method in changegroup module This is a gratuitous code move aimed at reducing the localrepo bloatness. The method had few callers, not enough to be kept in local repo.

localrepo: move the addchangegroupfiles method in changegroup module This is a gratuitous code move aimed at reducing the localrepo bloatness. The method had a single caller, far too few for being kept in local repo.

localrepo: move the changegroup method in changegroup module This is a gratuitous code move aimed at reducing the localrepo bloatness. The method had few callers, not enough to be kept in local repo. The peer API stay unchanged.

localrepo: move the getbundle method in changegroup module This is a gratuitous code move aimed at reducing the localrepo bloatness. The method had few callers, not enough to be kept in local repo. The peer API remains unchanged.

store: drop unused existing list

localrepo: move the getlocalbundle method in changegroup module This is a gratuitous code move aimed at reducing the localrepo bloatness. The method had 3 callers total, far too few for being kept in local repo.

localrepo: move the changegroupsubset method in changegroup module This is a gratuitous code move aimed at reducing the localrepo bloatness. The method had few callers, not enough to be kept in local repo. The peer API remains unchanged.

localrepo: move the changegroupinfo method in changegroup module This is a gratuitous code move aimed at reducing the localrepo bloatness. The method had a single caller... already in this changegroup module.

localrepo: move the _changegroupsubset method in changegroup module This is a gratuitous code move aimed at reducing the localrepo bloatness. The method had 3 callers total, far too few for being kept in local repo.

push: pass a `pushoperation` object to localrepo.checkpush The `pushoperation` object contains strictly more data the arguments currently passed to `localrepo.checkpush` we pass the new object instead. This function is used by MQ to abort push that includes MQ changesets. Note: extension that may use this function will have to align.

wireproto: use decorator for the ubundle command

wireproto: use decorator for the stream command

wireproto: use decorator for the pushkey command

wireproto: use decorator for the lookup command

wireproto: use decorator for the listkeys command

wireproto: use decorator for the known command

wireproto: use decorator for the hello command

wireproto: use decorator for the heads command

wireproto: use decorator for the getbundle command

wireproto: use decorator for the debugwireargs command

wireproto: use decorator for the changegroupsubset command

wireproto: use decorator for the changegroup command

wireproto: use decorator for the capabilities command

wireproto: use decorator for the branches command

wireproto: use decorator for the branchmap command

wireproto: use decorator for the between command

wireproto: use decorator for the batch command

wireproto: add decorator for wire protocol command Move move in the same direction we took for command line commands. each wire protocol function will be decorated with its name and arguments. Beside beside easier to read, this open the road to easily adding more metadata (like security level or return type)

wireproto: drop the _decompress method in favor a new call type We already have multiple call function for multiple return type. The `_decompress` function is only used for http and seems like a layer violation. We drop it in favor of a new call type dedicated to "stream that may be useful to compress".

wireproto: document protocol specific function of wirepeer Those function are explicitly declared and documented now.

wireproto: introduce an abstractserverproto class sshserver and webproto now inherit from an abstractserverproto class. This class is introduced for documentation purpose.

wireproto: document possible return type The wireprotocole command use a small set of class as return value. We document the meaning of each of them. AS my knowledge of wire protocol is fairly shallow, the documentation can probably use improvement. But this is a better than nothing.

pull: add a set of steps that remain to be done during the pull With bundle2 we'll slowly move current steps into a single bundle2 building and call (changegroup, phases, obsmarkers, bookmarks). But we need to keep the old methods around for servers that do not support bundle2. So we introduce this set of steps that remains to be done.

pull: put discovery step in its own function The discovery is a full step that will grow to contain more than changesets. We set it in its own function like all other steps.

pull: move the cgresult logic in _pullchangeset Logical follow up of the previous changeset. The `_pullchangeset` is now responsible of setting the `pullop.cgresult`.

pull: move return code in the pull operation object With bundle2 we'll have multiple code path susceptible to be responsible for adding changeset during the pull. We move it to the pull operation to simplify this the coming logic. The one doing the adding the changegroup will set the value.

merge: use ancestor filename from planning phase instead of filectx ancestor test-merge-types.t changes a bit in flag merging. It relied on the implementation detail that 100% identical revlog entries are reused. The revlog reuse did that fctx.ancestor() saw an ancestor where there really not was one.

tests: prevent test-hup from hanging forever if the transaction does not start If the transaction fails to start because of some programing errors, the test would stay stuck in this while loop forever. We now added a basic safeguard that abort the loop if nothing happen in 10 seconds. 10 seconds sounds enough to let the server catch up and open a transaction.

revset: improve roots revset performance Previously we would iterate over every item in the subset, checking if it was in the provided args. This often meant iterating over every rev in the repo. Now we iterate over the args provided, checking if they exist in the subset. On a large repo this brings setting phase boundaries (which use this revset roots(X:: - X::Y)) down from 0.8 seconds to 0.4 seconds. The "roots((tip~100::) - (tip~100::tip))" revset in revsetbenchmarks shows it going from 0.12s to 0.10s, so we should be able to catch regressions here in the future. This actually introduces a regression in 'roots(all())' (0.2s to 0.26s) since we're now using spansets, which are slightly slower to do containment checks on. I believe this trade off is worth it, since it makes the revset O(number of args) instead of O(size of repo).

revset: improve _descendants performance Previously revset._descendants would iterate over the entire subset (which is often the entire repo) and test if each rev was in the descendants list. This is really slow on large repos (3+ seconds). Now we iterate over the descendants and test if they're in the subset. This affects advancing and retracting the phase boundary (3.5 seconds down to 0.8 seconds, which is even faster than it was in 2.9). Also affects commands that move the phase boundary (commit and rebase, presumably). The new revsetbenchmark indicates an improvement from 0.2 to 0.12 seconds. So future revset changes should be able to notice regressions. I removed a bad test. It was recently added and tested '1:: and reverse(all())', which has an amibiguous output direction. Previously it printed in reverse order, because we iterated over the subset (the reverse part). Now it prints in normal order because we iterate over the 1:: . Since the revset itself doesn't imply an order, I removed the test.

revsetbenchmark: remove python 2.7 dependency revsetbenchmark.py used check_output which only exists in python 2.7. This fixes it.

bundle2: read the whole bundle from stream on abort When the bundle processing abort on unknown mandatory parts, we now makes sure all the bundle content is read. This avoid leaving the communication channel in an unrecoverable state.

bundle2: add some distinction between mandatory and advisory part Mandatory part cannot be ignored when unknown. We raise a simple KeyError exception when this happen. This is very early version of this logic, see inline comment for future improvement lead.

bundle2: introduce a `parthandler` decorator Simple syntax sugar to register an handler for a new part type.

bundle2: first version of a bundle processing We now have a function that interpret part content. This is a version early version of this function. It'll see major changes in scope and API in future development. As for previous I'm just focussing on getting minimal logic setup. Refining will happen with real world usage.

bundle2: rename unbundle2 test command to statbundle2 We will introduce object to actually process the bundle, we need to keep the simplistic unbundle around for the test.

bundle2: safely read unpack data from part header We use the same approach that the other unpack, as function is given the struct format and his both responsible for reading the right amount of data from the header and unpack the struct. This give use flexibility if we decide to change the size of something in the format before the release.

transaction: drop extra import caught by pyflakes

fncache: clean up fncache during strips Previously the fncache was cleaned up at read time by noticing when it was out of sync. This caused writes to happen outside the scope of transactions and could have caused race conditions. With this change, we'll keep the fncache up-to-date as we go by removing old entries during repair.strip.

caches: invalidate store caches when lock is taken The fncache was not being properly invalidated each time the lock was taken, so in theory it could contain old data from prior to the caller having the lock. This changes it to be invalidated as soon as the lock is taken (same as all our other caches).

fncache: move fncache writing to be in a transaction Previously the fncache was written at lock.release time. This meant it was not tracked by a transaction, and if an error occurred during the fncache write it would fail to update the fncache, but would not rollback the transaction, resulting in an fncache that was not in sync with the files on disk (which causes verify to fail, and causes streaming clones to not copy all the revlogs). This uses the new transaction backup mechanism to make the fncache transacted. It also moves the fncache from being written at lock.release time, to being written at transaction.close time.

transaction: add support for non-append files This adds support for normal, non-append-only files in transactions. For example, .hg/store/fncache and .hg/store/phaseroots should be written as part of the transaction, but are not append only files. This adds a journal.backupfiles along side the normal journal. This tracks which files have been backed up as part of the transaction. transaction.addbackup() creates a backup of the file (using a hardlink), which is later used to recover in the event of the transaction failing. Using a seperate journal allows the repository to still be used by older versions of Mercurial. A future patch will use this functionality and add tests for it.

transaction: add onclose/onabort hook for pre-close logic Adds an optional onclose parameter to transactions that gets called just before the transaction is committed. This allows things that build up data over the course of the transaction (like the fncache) to commit their data. Also adds onabort. It's not used, but will allow extensions to hook into onclose and onabort to provide transaction support.

clone: put streaming clones in a transaction Streaming clones were writing to files outside of a transaction. Soon the fncache will be written at transaction close time, so we need streaming clones to be in a transaction.

fncache: remove the rewriting logic The fncache could rewrite itself during a read operation if it noticed any entries that were no longer on disk. This was problematic because it caused Mercurial to perform write operations outside the scope of a lock or transaction, which could interefere with any other pending writes. This will be replaced in a future patch by logic that cleans up the fncache as files are deleted during strips.

pull: prevent duplicated entry in `op.pulledsubset` In the bare pull case we could add the same node multiple time to the `pulloperation.pulledsubset`. Beside being a bit wrong this confused the new revset implementation of `revset._revancestor` into giving bad result. This changeset fix the pull operation part. The fix for the revset itself will come in another changeset.

bundle2: part params

bundle2: support for bundling and unbundling payload We add the ability to bundle and unbundle a payload in parts. The payload is the actual binary data of the part. It is used to convey all the applicative data. For now we stick to very simple implementation with all the data fit in single chunk. This open the door to some bundle2 testing usage. This will be improved before bundle2 get used for real. We need to be able to stream the payload in multiple part to exchange any changegroup efficiently. This simple version will do for now. Bundling and unbundling are done in the same changeset because the test for parts is less modular. However, the result is not too complex.

merge with stable

Added signature for changeset 3f83fc5cfe71

Added tag 2.9.2 for changeset 3f83fc5cfe71

backout: correct commit status of no changes made (BC) (issue4190) If backout generated no changes to commit, it showed wrong status, "changeset <target> backs out changeset <target>", and raised TypeError with -v option. This changes the return code to 1, which is the same as "hg commit" and "hg rebase".

backout: document return code of merge conflict

merge with stable

i18n: fix "% inside _()" problems Before this patch, "contrib/check-code.py" can't detect these problems, because the regexp pattern to detect "% inside _()" doesn't suppose the case that format string consists of multiple string components concatenated implicitly or explicitly, This patch does below for that regexp pattern to detect "% inside _()" problems in such case. - put "+" into separator part ("[ \t\n]") for explicit concatenation ("...." + "...." style) - enclose "component and separator" part by "(?:....)+" for concatenation itself ("...." "...." or "...." + "....")

i18n: fix "% inside _()" problems Before this patch, "contrib/check-code.py" can't detect these problems, because the regexp pattern to detect "% inside _()" doesn't suppose the case that the format string and "%" aren't placed in the same line. This patch replaces "\s" in that regexp pattern with "[ \t\n]" to detect "% inside _()" problems in such case. "[\s\n]" can't be used in this purpose, because "\s" is automatically replaced with "[ \t]" by "_preparepats()" and "\s" in "[]" causes nested "[]" unexpectedly.

merge: fix lack of "%s" in format string causing TypeError at runtime

i18n-pt_BR: synchronized with c57c9cece645

i18n-ja: synchronized with e259d4c462b5

bundle2: support unbundling empty part We augment the unbundler to make it able to unbundle the empty part we are now able to bundle.

revset: raise ValueError when calling min or max on empty smartset min([]) raise a ValueError, we do the same thing in smartset.min() and smartset.max() for the sake of consistency. The min/amax test are greatly improved in the process to prevent this familly of regression

revpair: smartset compatibility Since recent revset changes, revrange now return a smartset. This smart set probably does not support indexing (_addset does not). This led to crash. Instead when the smartset is ordered we use the `min` and `max` method of smart set. Otherwise we turn is into a list and use indexing on it. The tests have been updated to catch such regression.

revsetbenchmark: add entry for ::rev:: Revsets of the form ::rev:: were identified as the source behind the regressions in issue 4201. Ensure we have explicit coverage of that revset.

merge with stable This should correct an earlier couple of bad merges (5433856b2558 and 596960a4ad0d, now pruned) that accidentally brought in a change that had been marked obsolete (244ac996a821).

tests: use TESTTMP instead of TESTDIR In 57d0c8c3b947, f042d4b263f4, 1e686e55780c, and 5d22cadd1938, new tests were added that used TESTDIR instead of TESTTMP thereby leading to polluting the working directory with these temporary files. Now, we use TESTTMP so that they will be cleaned up properly.

hg: introduce "wirepeersetupfuncs" to setup wire peer by extensions (issue4109) Since changeset 6f72e7d28b35, "reposetup()" of each extensions is invoked only on repositories enabling corresponded extensions. This causes that largefiles specific interactions between the repository enabling largefiles locally and remote (wire) peer fail, because there is no way to know whether largefiles is enabled on the remote repository behind the wire peer, and largefiles specific "wireproto functions" are not given to any wire peers. To avoid this problem, largefiles should be enabled in wider scope than each repositories (e.g. user-wide "${HOME}/.hgrc"). This patch introduces "wirepeersetupfuncs" to setup wire peer by extensions already enabled. Functions registered into "wirepeersetupfuncs" are invoked for all wire peers. This patch uses plain list instead of "util.hooks" for "wirepeersetupfuncs", because the former allows to control order of function invocation by order of extension enabling: it may be useful for workaround of problems with combination of enabled extensions

templater: raise error for unknown func Previously, if a template '{foo()}' was given, the buildfunc would not be able to match it and hit a code path that would not return so it would error out later in the templater stating that NoneType was not iterable. This patch makes sure that a proper error is raised so that the user can be informed. Tests have been updated.

bundle2: support bundling of empty part (with a type) Here start the work on bundle2 parts. Our first step is to be able to bundle a simplistic part that just have a type, no parameters, empty payload.

revsetbenchmark: add a summary at the end of execution The summary list timing per revset making it much more easier to compare revision to each other.

revsetbenchmark: retrieve the benchmark value in python We retrieve the output of the perf extension and print it ourself. This open the door to processing of this data in the script.

revsetbenchmark: get revision to benchmark in a function And move it to proper subprocess call.

revsetbenchmark: convert revision display to proper subprocesscall

revsetbenchmark: convert performance call to proper subprocess call

revsetbenchmark: convert update to proper subprocess call

revsetbenchmark: fix semicolon

revsetbenchmark: simplify and convert the script to python The script is now in python. That translation is very raw, more improvement to comes: The "current code" and "base" entry have been dropped. This is trivial to get same result using a tagged revision or "." in the list of benchmarked revision.

revsetbenchmapi: fix template The revision description missed an \n

perf: unroll the result in perfrevset With the new lazy revset implementation, we need to actually read all elements to trigger all the computations. Otherwise a no-op if of course much faster than the full work.

_addset: add a __len__ method Back in the time where repo.revs(...) returned a list, calling `len(...)` on the result was quite common. We reinstall this on _addset. There is absolutely no easy way to test this from the command line. The commands using this in the evolve extension will eventually land into core.

bundle2: implement the mandatory/advisory logic for parameter Parameter starting with an upper case are mandatory, the one starting with a lower case are advisory and may be ignored.

bundle2: print debug information during unbundling The unbundler class is now feed with an ui object and use it to transmit data about the unbundling process.

bundle2: print debug information during bundling The bundler class is now feed with an ui object and use it to transmit data about the bundling process.

bundle2: add ability to write to a file to the test Next changesets are about improving debug output during bundling. We won't be able to simply bundle to stdin in this case. The bundle2 stream would be interleaved with debug output.

subrepo: factor out Git version check to add doctests Followup to 6a2acb0d9352::e5641536e4d5.

contrib: add "hgperf" command to measure performance of commands easily Newly added "hgperf" command repeats "dispatch.runcommand()" for specified Mercurial command and measure performance of it in the same manner of "contrib/perf.py" extension. Users (mainly developers) can examine performance of the target command easily, without adding new entry for it to "contrib/perf.py" extension.

grep: exit loop immediately, if matching is found in the file for "hg grep -l" Before this patch, internal function "display()" of "hg grep" is not efficient for "-l"/"--files-with-matches", because loop is continued, even after the first matching is found in the specified file. This patch exits loop immediately, if matching is found for "--files-with-matches". In this case, "before is None" is equal to "opts.get('files_with_matches')".

grep: use "found" instead of "filerevmatches" examination for efficiency Before this patch, internal function "display()" of "hg grep" stores whether matching is already found or not into the dictionary "filerevmatches" by "(fn, rev)" tuple as the key. But this is redundant, because: - "filerevmatches" is local variable of "display()", so each "display()" invocations don't affect others - both "fn" and "rev" (gotten from "ctx" argument) are never changed in each "display()" invocations Then, "filerevmatches" should have only one entry at most, and "(fn, rev) in filerevmatches" should be equal to "found". This patch uses "found" instead of "filerevmatches" examination for efficiency.

grep: reuse the first "util.binary()" result for efficiency Before this patch, to check whether the file in the specified revision is binary or not, "util.binary()" is invoked via internal function "binary()" of "hg grep" once per a line of "hg grep" output, even though binary-ness is not changed in the same file. This patch reuses the first "util.binary()" invocation result by annotating internal function "binary()" with "@util.cachefunc". Performance improvement measured by "hgperf grep -r 88d8e568add1 vfs mercurial/scmutil.py": before this patch: ! wall 0.024000 comb 0.015600 user 0.015600 sys 0.000000 (best of 118) after this patch: ! wall 0.023000 comb 0.015600 user 0.015600 sys 0.000000 (best of 123) Status of recent(88d8e568add1) "mercurial/scmutil.py": # of lines: 919 (may affect cost of search) # of bytes: 29633 (may affect cost of "util.binary()") # of matches: 22 (may affect frequency of "util.binary()")

util: add the code path to "cachefunc()" for the function taking no arguments Before this patch, "util.cachefunc()" caches the value returned by the specified function into dictionary "cache", even if the specified function takes no arguments. In such case, "cache" has at most one entry, and distinction between entries in "cache" is meaningless. This patch adds the code path to "cachefunc()" for the function taking no arguments for efficiency: to store only one cached value, using list "cache" is a little faster than using dictionary "cache".

help: add examples to incoming

revset: fix generatorset race condition If two things were iterating over a generatorset at the same time, they could miss out on the things the other was generating, resulting in incomplete results. This fixes it by making it possible for two things to iterate at once, by always checking the _genlist at the beginning of each iteration. I was only able to repro it with pending changes from my other commits, but they aren't ready yet. So I'm unable to add a test for now.

tests: don't hardcode path to bash interpreter Use the env binary to figure out the correct bash to use. Certain systems ships with an ancient version of bash, but the user might have installed a newer one that is earlier in $PATH. For example the current version of Mac OS X ships version 3.2.51 of bash, which does not understand new fancy builtins such as readarray. A user might install a newer version of bash, use that as their shell and add that path before bin.

contrib: don't hardcode path to bash interpreter Use the env binary to figure out the correct bash to use. Certain systems ships with an ancient version of bash, but the user might have installed a newer one that is earlier in $PATH. For example the current version of Mac OS X ships version 3.2.51 of bash, which does not understand new fancy builtins such as readarray. A user might install a newer version of bash, use that as their shell and add that path before bin.

contrib: explicitly enable perf extension for revset tests If a developer doesn't have the perf extension enabled the revset benchmarking will fail. This patch explicitly enables the perf extension when running the tests.

merge with stable

revset: improve performance of _generatorset.__contains__ (issue 4201) _generatorset.__contains__ and __contains__ from child classes were calling into __iter__ to look for values. Since all previously-encountered values from the generator were cached and checked in __contains__ before this iteration, __contains__ was effectively performing iteration busy work which could lead to an explosion of redundant work. This patch changes __contains__ to be more intelligent. Instead of looking at all values via __iter__, __contains__ will instead go straight to "new" values from the underlying generator. On a clone of the Firefox repository with around 200,000 changesets, this patch decreases the execution time of the revset '::(200067)::' from ~100s to ~4s on the author's machine. Rebase operations (which use the aforementioned revset), speed up accordingly.

localrepo: omit ".hgsubstate" also from "added" files Before this patch, "localrepository.commit()" omits ".hgsubstate" from "modified" (changes[0]) and "removed" (changes[2]) file list before checking subrepositories, but leaves one in "added" (changes[1]) as it is. Then, "localrepository.commit()" adds ".hgsubstate" into "modified" or "removed" list forcibly, according to subrepository statuses. If "added" contains ".hgsubstate", the committed context will contain two ".hgsubstate" in its "files": one from "added" (not omitted one), and another from "modified" or "removed" (newly added one). How many times ".hgsubstate" appears in "files" changes node hash, even though revision content is same, because node hash calculation uses the specified "files" directly (without duplication check or so). This means that node hash of committed revision changes according to existence of ".hgsubstate" in "added" at "localrepository.commit()". ".hgsubstate" is treated as "added", not only in accidental cases, but also in the case of "qpush" for the patch adding ".hgsubstate". This patch omits ".hgsubstate" also from "added" files before checking subrepositories. This patch also omits ".hgsubstate" exclusion in "qnew"/"qrefresh" introduced by changeset d666da075b91, because this patch makes them meaningless. "hg parents --template '{files}\n'" newly added to "test-mq-subrepo.t" enhances checking unexpected multiple appearances of ".hgsubstate" in "files" of created/refreshed MQ revisions.

dispatch: only do __import__(debugger) when a debugger is requested When having ui.debugger=somedebugger in one's ~/.hgrc, this then somedebugger would be imported for every hg command. With this patch, this import only happens if the --debugger parameter is passed.

hg: use "os.path.join()" to join path components which may be empty (issue4203) Changset 2d0ab571b822 rewriting "hg.copystore()" with vfs uses 'dstbase + "/lock"' instead of "os.path.join()", because target files given from "store.copyfiles()" already uses "/" as path separator But in the repository using revlog format 0, "dstbase" becomes empty ("data" directory is located under ".hg" directly), and 'dstbase + "/lock"' is treated as "/lock": in almost all cases, write access to "/lock" causes "permission denied". This patch uses "os.path.join()" to join path components which may be empty in "hg.copystore()".

revset: document the regular expression support for tag(name) This has been supported since 20f55613fb2a, in 2.3.

help: allow -k to find debug commands

help: let 'hg help debug' show the list of secret debug commands The names of the debug commands are not easy to remember and they are not easy to find.

tests: add run-tests --changed option for running tests changed in revisions Convenient when polishing patches and changing details of how they change test output. This will probably break in weird ways for revsets with special quoting ... but it is good enough for run-tests. Usage example: yes | ./run-tests.py -li --changed qparent

repo: rephrase the "missing requirement" error message Unknown requirements will now be reported as: abort: repository requires features unknown to this Mercurial: largefiles! (see http://mercurial.selenic.com/wiki/MissingRequirement for more information) Some features of this phrasing: * avoid double ':' in abort message * make it more clear who requires and knows what * don't quote the requirement names - it is not something the user entered or need the exact spelling of ... and it is "identifiers" that are unambiguous anyway * remove double hint by removing "(upgrade Mercurial)" comment * don't mention upgrading Mercurial without mentioning enabling the feature - instead, just refer to wiki page for both * don't just talk about "details", talk about "more information"

revpair: drop useless conditional The `if not revs:` case is tested at the beginning of the function.

merge with stable

subrepo: add trailing newlines to warnings

subrepo: convert matched string to integer before comparing Git version (1, '4') is greater than (1, 5) so the version check never actually worked.

subrepo: only retrieve the first two components of the Git version This makes the version detection compatible with Git versions like '1.9-rc0'. We only cared about the first two components of the version anyway.