revlog: use context ancestor instead of changelog ancestor We want to move in this direction.

copies: guard debug section with ui.debugflag

copies: remove _checkcopies wrapper - it does no good mergecopies might be doomed but it is not dead yet ...

transplant: use context ancestor instead of changelog ancestor We want to move in this direction. ctx.ancestor is in a better position for handling a situation with multiple ancestors.

commit: use revlog.commonancestors instead of .ancestor This do probably not make any real difference but is slightly more correct and we would like to get rid of flog.ancestor.

backout: don't limit heritage check to a single ancestor This do probably not make any real difference but is slightly more correct and we would like to get rid of changelog.ancestor.

ancestor: remove unused genericancestor

context: remove unused filectx.ancestor

shelve: add "writebundle()" to invoke "writebundle()" with relative path and vfs

shelve: add "applybundle()" to invoke "readbundle()" with relative path and vfs "applybundle()" also includes "addchangegroup()" invocation to encapsulate "vfs.join()" inside it.

bundlerepo: treat temporarily extracted bundle file via vfs

vfs: add "mkstemp()"

repair: make "strip()" treat bundle files via vfs This patch makes "repair.strip()" treat bundle files via vfs. This patch also avoids applying "vfs.join()" on the value returned by "changegroup.writebundle()", to get relative path from "_bundle()".

changegroup: add "vfs" argument to "readbundle()" to pass relative filename

repair: make paths in "_bundle()" relative to ".hg" This patch makes paths below in "_bundle()" relative to ".hg": - backup directory ("strip-backup"), and - bundle file under backup directory "vfs" is passed to "changegroup.writebundle()" to use relative path directly. This patch applies "vfs.join()" on the value returned by "_bundle()", because the caller expect it to return absolute path. This will be changed by succeeding patch changing the caller side.

changegroup: add "vfs" argument to "writebundle()" for relative access via vfs Before this patch, filename specified to "changegroup.writebundle()" should be absolute one. In some cases, they should be relative to repository root, store and so on (backup before strip, for example). This patch adds "vfs" argument to "writebundle()", and makes "writebundle()" open (and unlink) "filename" via vfs for relative access, if both filename and vfs are specified.

localrepo: make "undofiles()" return list of tuples "(vfs, relative filename)" Before this patch, "localrepository.undofiles()" returns list of absolute filename of undo files. This patch makes it return list of tuples "(vfs, relative filename)" to access undo files via vfs. This patch also changes "repair.strip()", which is the only user of "localrepository.undofiles()".

exchange: fix pyflakes import complaint

hgweb: override the charset attribute in the http request The default http request character set is UTF-8. If the message is not encoded in UTF-8, such as big5, it cannot be shown correctly. The 'charset' is overridden by the root document's, such that the user can select the proper encoding in the browser.

diff: use second filename for --stat reporting on git patches (issue4221)

exchange: drop useless line break

subrepo: check return code for git push (issue4223)

localrepo: add unbundle support Localrepo now supports the unbundle method of pushing changegroups. We plan to use the unbundle call for bundle2 so it is important that all peers supports it. The `peer.unbundle` and `peer.addchangegroup` code path have small difference so cause some test output changes. None of those changes seems problematic.

unbundle: extract the core logic in another function The `exchange` module now contains an `unbundle` function that holds the core unbundle logic. The wire protocol keeps its own unbundle function. It enforces wireprotocol-specific logic and then calls the extracted function. This aims at implementing unbundle for localrepo.

unbundle: extract checkheads in its own function We are going to refactor the unbundle function to have it working on a local repository too. Having this function extracted will ease the process. In the case of non-matching heads, the function now directly raises an exception. The top level of the function is catching it.

phase: apply publishing enforcement for "serve" source When a changegroup is added by a push on a publishing server, we ensure they are added as public. This is used to enforce publishing on server when the client is not aware of phases. It also prevents race conditions where a reader could see the changesets as draft before they get turned public by the client. Finally, this save rounds trip as the client does not need additional request to turn them public. However, this logic was only enforced when the changegroup was from a "push" source. And "push" is used for local pushes only. Wireprotocol push uses "serve" as source since Mercurial 1.9. We now enforce this logic for both "push" and "serve" sources. One could note that this logic was mainly useful during wireprotocol exchanges. So this code is finally put into good use, 9 versions after its introduction.

merge with stable

http: reuse authentication info after the first failed request (issue3567) [This was applied in 181108726ea5 but backed out again in af02783dea65 because of Python 2.4 issues. This edition and test-http.t works with Python 2.4.] Context: mercurial access to repository server with http access, and this server is protected by basic auth. Before patch: * mercurial try an anonymous access to server, server return 401 response and mercurial resend request with login / password information After patch: * mercurial try an anonymous access to server, server return 401 response. For all next requests, mercurial keep in memory this information (this server need basic auth information). This patch reduce the number of http access against mercurial server. Example, before patch: 10.10.168.170 - - [25/Oct/2013:15:44:51 +0200] "GET /hg/testagt?cmd=capabilities HTTP/1.1" 401 260 "-" "mercurial/proto-1.0" 10.10.168.170 - - [25/Oct/2013:15:44:52 +0200] "GET /hg/testagt?cmd=capabilities HTTP/1.1" 200 147 "-" "mercurial/proto-1.0" 10.10.168.170 - - [25/Oct/2013:15:45:00 +0200] "GET /hg/testagt?cmd=capabilities HTTP/1.1" 401 260 "-" "mercurial/proto-1.0" 10.10.168.170 - - [25/Oct/2013:15:45:01 +0200] "GET /hg/testagt?cmd=capabilities HTTP/1.1" 200 147 "-" "mercurial/proto-1.0" 10.10.168.170 - - [25/Oct/2013:15:45:03 +0200] "GET /hg/testagt?cmd=batch HTTP/1.1" 401 260 "-" "mercurial/proto-1.0" 10.10.168.170 - - [25/Oct/2013:15:45:04 +0200] "GET /hg/testagt?cmd=batch HTTP/1.1" 200 42 "-" "mercurial/proto-1.0" 10.10.168.170 - - [25/Oct/2013:15:45:06 +0200] "GET /hg/testagt?cmd=getbundle HTTP/1.1" 401 260 "-" "mercurial/proto-1.0" 10.10.168.170 - - [25/Oct/2013:15:45:07 +0200] "GET /hg/testagt?cmd=getbundle HTTP/1.1" 200 61184 "-" "mercurial/proto-1.0" 10.10.168.170 - - [25/Oct/2013:15:45:09 +0200] "GET /hg/testagt?cmd=listkeys HTTP/1.1" 401 260 "-" "mercurial/proto-1.0" 10.10.168.170 - - [25/Oct/2013:15:45:10 +0200] "GET /hg/testagt?cmd=listkeys HTTP/1.1" 200 15 "-" "mercurial/proto-1.0" 10.10.168.170 - - [25/Oct/2013:15:45:12 +0200] "GET /hg/testagt?cmd=listkeys HTTP/1.1" 401 260 "-" "mercurial/proto-1.0" 10.10.168.170 - - [25/Oct/2013:15:45:12 +0200] "GET /hg/testagt?cmd=listkeys HTTP/1.1" 200 - "-" "mercurial/proto-1.0" Example after patch: 10.10.168.170 - - [28/Oct/2013:11:49:14 +0100] "GET /hg/testagt?cmd=capabilities HTTP/1.1" 401 260 "-" "mercurial/proto-1.0" 10.10.168.170 - - [28/Oct/2013:11:49:15 +0100] "GET /hg/testagt?cmd=capabilities HTTP/1.1" 200 147 "-" "mercurial/proto-1.0" 10.10.168.170 - - [28/Oct/2013:11:49:17 +0100] "GET /hg/testagt?cmd=batch HTTP/1.1" 200 42 "-" "mercurial/proto-1.0" 10.10.168.170 - - [28/Oct/2013:11:49:19 +0100] "GET /hg/testagt?cmd=getbundle HTTP/1.1" 200 61184 "-" "mercurial/proto-1.0" 10.10.168.170 - - [28/Oct/2013:11:49:22 +0100] "GET /hg/testagt?cmd=listkeys HTTP/1.1" 200 15 "-" "mercurial/proto-1.0" 10.10.168.170 - - [28/Oct/2013:11:49:24 +0100] "GET /hg/testagt?cmd=listkeys HTTP/1.1" 200 - "-" "mercurial/proto-1.0" In this last example, you can see only one 401 response.

bundle2: use discard to remove bundle2 cap caps.remove('bundle2') was throwing an exception if bundle2 wasn't present in the capabilities. This was causing test-static-http.t to hang. Let's just use discard, so we don't get an exception.

statichttp: respect localrepo _restrictcapabilities The static http repository was doing his own filtering of capability ignoring the filtering done in the local repo main class. This led to static http using the current draft of bundle2. We now apply both.

tests: make unshelve tests more tricky - don't depend on size change One reason shelve and largefiles doesn't work could be rapidly changing standin files. Prove that shelve in general doesn't have problems with that.

shelve: introduce secret option for using fixed date for temporary commit Using a fixed date makes hashes stable and makes debugging simpler. The date and hashes of this changeset are normally not exposed.

mq: repo['.'] is not a wctx, repo[None] is The parameters passed to subrepo.submerge are confusing anyway.

shelve: repo['.'] is not a wctx but a pctx Don't confuse hackers!

revlog: deal with chunk ranges over 2G on Windows (issue4215) Python uses a C long (32 bits on Windows 64) rather than an ssize_t in read(), and thus has a 2G size limit. Work around this by falling back to reading one chunk at a time on overflow. This approximately doubles our headroom until we run back into the size limit on single reads.

exchange: pass bundlecaps through to changegroup The bundlecaps passed to exchange.getbundle were being dropped completely. We should pass them on through to the changegroup. This affected the remotefilelog extension, since it relies on those bundlecaps.

bundle2: allow pulling changegroups using bundle2 This changeset refactors the pull code to use a bundle2 when available. We keep bundle2 disabled by default. The current code is not ready for prime time. Ultimately we'll want to unify the API of `bunde10` and `bundle20` to have less different code. But for now, testing the bundle2 exchange flow is an higher priority.

bundle2: add an exchange.getbundle function This function can return a `HG10` or `HG20` bundle. It uses the `bundlecaps` parameters to decides which one to return. This is a distinct function from `changegroup.getbundle` for two reasons. First the APIs of `bundle10` and `bundle20` are not compatible yet. The two functions may be reunited in the future. Second `exchange.getbundle` will grow parameters for all kinds of data (phases, obsmarkers, ...) so it's better to keep the changegroup generation in its own function for now. This function will be used it in the next changesets.

localpeer: propagate bundlecaps in getbundle call Best arguments are the ones in use...

bundle2: make it possible have a global transaction for the unbundling We use the `gettransaction` method approach already used for pull. We need this because we do not know beforehand if the bundle needs a transaction to be created. And (1) we do not want to create a transaction for nothing. (2) Some bundle2 bundles may be read-only and do not require any lock or transaction to be held.

tests: fix test output typo

bundle2: first crude version of bundling changeset with bundle2 The current changegroup format is put in a "changegroup" part and processed by an appropriate handlers. This is not production ready code, but let us start smoke testing.

bundle2: record processing results in the bundleoperation object Part handlers can now add records to the `bundleoperation` object. This can be used to help other parts or to let the caller of the unbundling process react to the results.

bundle2: introduce a bundleoperation object This object will hold all data and state gathered through the processing of a bundle. This will allow: - each handler to be aware of the things unbundled so far - the caller to retrieve data about the execution - bear useful object and logic (like repo, transaction) - bear possible bundle2 reply triggered by the unbundling. For now the object is very simple but it will grow at the same time as the bundle2 implementation.

bundle2: feed a unbundle20 to the `processbundle` function The unbundle can comes from multiple sources. (on disk file, peer, etc) and (ultimately) of multiple type (bundle10, bundle20). The `processbundle` is no longer in charge of creating the bundle.

bundle2: lock the repo during unbundle test We forgot the lock the repo. It's not too bad during tests, but the lock also still carry some transaction logic. (yes we need to get rid of this)

merge: keep destination filename as key in filemerge actions Gives more readable debug output and makes it possible to compare/merge actions later.

merge: keep destination filename as key in actions for merge with dir rename Avoids None: local renamed directory to b/c -> d updating:None 1/1 files (100.00%) and makes it possible to compare/merge actions later.

merge: include ancestor node in merge actions

merge with stable

repoview: use _gethiddenblockers in computehidden No functionality has changed, since we've only extracted the code into its own function. Now extensions can wrap _gethiddenblockers to provide their own blocker without polluting bookmarks or local tags.

repoview: add _gethiddenblockers method This is a standalone function that will provide the ability for extensions to wrap.

repoview: improve performance for computehidden (issue4206) For repos with a large number of heads (including hidden heads), a stale tag cache would cause computehidden to be drastically slower because of a the call to repo.tags() (which would build the tag cache). We actually don't need the tag cache for computehidden because we filter out global tags. This patch replaces the call to repo.tags with readlocaltags so as to avoid the tag cache.

tests: remove unused variables from test-pathencode.py caught by pyflakes

shelve: remove unused variable caught by pyflakes

changegroup: remove unused variable caught by pyflakes

hgk: use hg export to write commits Instead of calling internal command, use hg export to produce changeset diffs compatible with hg import directly.

hgk: add .diff extension when exporting commits Append the .diff extension automatically.

localrepo: move the addchangegroup method in changegroup module This is a gratuitous code move aimed at reducing the localrepo bloatness. The method had few callers, not enough to be kept in local repo.

localrepo: move the addchangegroupfiles method in changegroup module This is a gratuitous code move aimed at reducing the localrepo bloatness. The method had a single caller, far too few for being kept in local repo.

localrepo: move the changegroup method in changegroup module This is a gratuitous code move aimed at reducing the localrepo bloatness. The method had few callers, not enough to be kept in local repo. The peer API stay unchanged.

localrepo: move the getbundle method in changegroup module This is a gratuitous code move aimed at reducing the localrepo bloatness. The method had few callers, not enough to be kept in local repo. The peer API remains unchanged.

store: drop unused existing list

localrepo: move the getlocalbundle method in changegroup module This is a gratuitous code move aimed at reducing the localrepo bloatness. The method had 3 callers total, far too few for being kept in local repo.

localrepo: move the changegroupsubset method in changegroup module This is a gratuitous code move aimed at reducing the localrepo bloatness. The method had few callers, not enough to be kept in local repo. The peer API remains unchanged.

localrepo: move the changegroupinfo method in changegroup module This is a gratuitous code move aimed at reducing the localrepo bloatness. The method had a single caller... already in this changegroup module.

localrepo: move the _changegroupsubset method in changegroup module This is a gratuitous code move aimed at reducing the localrepo bloatness. The method had 3 callers total, far too few for being kept in local repo.

push: pass a `pushoperation` object to localrepo.checkpush The `pushoperation` object contains strictly more data the arguments currently passed to `localrepo.checkpush` we pass the new object instead. This function is used by MQ to abort push that includes MQ changesets. Note: extension that may use this function will have to align.

wireproto: use decorator for the ubundle command

wireproto: use decorator for the stream command

wireproto: use decorator for the pushkey command

wireproto: use decorator for the lookup command

wireproto: use decorator for the listkeys command

wireproto: use decorator for the known command

wireproto: use decorator for the hello command

wireproto: use decorator for the heads command

wireproto: use decorator for the getbundle command

wireproto: use decorator for the debugwireargs command

wireproto: use decorator for the changegroupsubset command

wireproto: use decorator for the changegroup command

wireproto: use decorator for the capabilities command

wireproto: use decorator for the branches command

wireproto: use decorator for the branchmap command

wireproto: use decorator for the between command

wireproto: use decorator for the batch command

wireproto: add decorator for wire protocol command Move move in the same direction we took for command line commands. each wire protocol function will be decorated with its name and arguments. Beside beside easier to read, this open the road to easily adding more metadata (like security level or return type)

wireproto: drop the _decompress method in favor a new call type We already have multiple call function for multiple return type. The `_decompress` function is only used for http and seems like a layer violation. We drop it in favor of a new call type dedicated to "stream that may be useful to compress".

wireproto: document protocol specific function of wirepeer Those function are explicitly declared and documented now.

wireproto: introduce an abstractserverproto class sshserver and webproto now inherit from an abstractserverproto class. This class is introduced for documentation purpose.

wireproto: document possible return type The wireprotocole command use a small set of class as return value. We document the meaning of each of them. AS my knowledge of wire protocol is fairly shallow, the documentation can probably use improvement. But this is a better than nothing.

pull: add a set of steps that remain to be done during the pull With bundle2 we'll slowly move current steps into a single bundle2 building and call (changegroup, phases, obsmarkers, bookmarks). But we need to keep the old methods around for servers that do not support bundle2. So we introduce this set of steps that remains to be done.

pull: put discovery step in its own function The discovery is a full step that will grow to contain more than changesets. We set it in its own function like all other steps.

pull: move the cgresult logic in _pullchangeset Logical follow up of the previous changeset. The `_pullchangeset` is now responsible of setting the `pullop.cgresult`.

pull: move return code in the pull operation object With bundle2 we'll have multiple code path susceptible to be responsible for adding changeset during the pull. We move it to the pull operation to simplify this the coming logic. The one doing the adding the changegroup will set the value.

merge: use ancestor filename from planning phase instead of filectx ancestor test-merge-types.t changes a bit in flag merging. It relied on the implementation detail that 100% identical revlog entries are reused. The revlog reuse did that fctx.ancestor() saw an ancestor where there really not was one.

tests: prevent test-hup from hanging forever if the transaction does not start If the transaction fails to start because of some programing errors, the test would stay stuck in this while loop forever. We now added a basic safeguard that abort the loop if nothing happen in 10 seconds. 10 seconds sounds enough to let the server catch up and open a transaction.

revset: improve roots revset performance Previously we would iterate over every item in the subset, checking if it was in the provided args. This often meant iterating over every rev in the repo. Now we iterate over the args provided, checking if they exist in the subset. On a large repo this brings setting phase boundaries (which use this revset roots(X:: - X::Y)) down from 0.8 seconds to 0.4 seconds. The "roots((tip~100::) - (tip~100::tip))" revset in revsetbenchmarks shows it going from 0.12s to 0.10s, so we should be able to catch regressions here in the future. This actually introduces a regression in 'roots(all())' (0.2s to 0.26s) since we're now using spansets, which are slightly slower to do containment checks on. I believe this trade off is worth it, since it makes the revset O(number of args) instead of O(size of repo).

revset: improve _descendants performance Previously revset._descendants would iterate over the entire subset (which is often the entire repo) and test if each rev was in the descendants list. This is really slow on large repos (3+ seconds). Now we iterate over the descendants and test if they're in the subset. This affects advancing and retracting the phase boundary (3.5 seconds down to 0.8 seconds, which is even faster than it was in 2.9). Also affects commands that move the phase boundary (commit and rebase, presumably). The new revsetbenchmark indicates an improvement from 0.2 to 0.12 seconds. So future revset changes should be able to notice regressions. I removed a bad test. It was recently added and tested '1:: and reverse(all())', which has an amibiguous output direction. Previously it printed in reverse order, because we iterated over the subset (the reverse part). Now it prints in normal order because we iterate over the 1:: . Since the revset itself doesn't imply an order, I removed the test.

revsetbenchmark: remove python 2.7 dependency revsetbenchmark.py used check_output which only exists in python 2.7. This fixes it.

bundle2: read the whole bundle from stream on abort When the bundle processing abort on unknown mandatory parts, we now makes sure all the bundle content is read. This avoid leaving the communication channel in an unrecoverable state.

bundle2: add some distinction between mandatory and advisory part Mandatory part cannot be ignored when unknown. We raise a simple KeyError exception when this happen. This is very early version of this logic, see inline comment for future improvement lead.

bundle2: introduce a `parthandler` decorator Simple syntax sugar to register an handler for a new part type.

bundle2: first version of a bundle processing We now have a function that interpret part content. This is a version early version of this function. It'll see major changes in scope and API in future development. As for previous I'm just focussing on getting minimal logic setup. Refining will happen with real world usage.

bundle2: rename unbundle2 test command to statbundle2 We will introduce object to actually process the bundle, we need to keep the simplistic unbundle around for the test.

bundle2: safely read unpack data from part header We use the same approach that the other unpack, as function is given the struct format and his both responsible for reading the right amount of data from the header and unpack the struct. This give use flexibility if we decide to change the size of something in the format before the release.

transaction: drop extra import caught by pyflakes

fncache: clean up fncache during strips Previously the fncache was cleaned up at read time by noticing when it was out of sync. This caused writes to happen outside the scope of transactions and could have caused race conditions. With this change, we'll keep the fncache up-to-date as we go by removing old entries during repair.strip.

caches: invalidate store caches when lock is taken The fncache was not being properly invalidated each time the lock was taken, so in theory it could contain old data from prior to the caller having the lock. This changes it to be invalidated as soon as the lock is taken (same as all our other caches).

fncache: move fncache writing to be in a transaction Previously the fncache was written at lock.release time. This meant it was not tracked by a transaction, and if an error occurred during the fncache write it would fail to update the fncache, but would not rollback the transaction, resulting in an fncache that was not in sync with the files on disk (which causes verify to fail, and causes streaming clones to not copy all the revlogs). This uses the new transaction backup mechanism to make the fncache transacted. It also moves the fncache from being written at lock.release time, to being written at transaction.close time.

transaction: add support for non-append files This adds support for normal, non-append-only files in transactions. For example, .hg/store/fncache and .hg/store/phaseroots should be written as part of the transaction, but are not append only files. This adds a journal.backupfiles along side the normal journal. This tracks which files have been backed up as part of the transaction. transaction.addbackup() creates a backup of the file (using a hardlink), which is later used to recover in the event of the transaction failing. Using a seperate journal allows the repository to still be used by older versions of Mercurial. A future patch will use this functionality and add tests for it.

transaction: add onclose/onabort hook for pre-close logic Adds an optional onclose parameter to transactions that gets called just before the transaction is committed. This allows things that build up data over the course of the transaction (like the fncache) to commit their data. Also adds onabort. It's not used, but will allow extensions to hook into onclose and onabort to provide transaction support.

clone: put streaming clones in a transaction Streaming clones were writing to files outside of a transaction. Soon the fncache will be written at transaction close time, so we need streaming clones to be in a transaction.

fncache: remove the rewriting logic The fncache could rewrite itself during a read operation if it noticed any entries that were no longer on disk. This was problematic because it caused Mercurial to perform write operations outside the scope of a lock or transaction, which could interefere with any other pending writes. This will be replaced in a future patch by logic that cleans up the fncache as files are deleted during strips.

pull: prevent duplicated entry in `op.pulledsubset` In the bare pull case we could add the same node multiple time to the `pulloperation.pulledsubset`. Beside being a bit wrong this confused the new revset implementation of `revset._revancestor` into giving bad result. This changeset fix the pull operation part. The fix for the revset itself will come in another changeset.

bundle2: part params

bundle2: support for bundling and unbundling payload We add the ability to bundle and unbundle a payload in parts. The payload is the actual binary data of the part. It is used to convey all the applicative data. For now we stick to very simple implementation with all the data fit in single chunk. This open the door to some bundle2 testing usage. This will be improved before bundle2 get used for real. We need to be able to stream the payload in multiple part to exchange any changegroup efficiently. This simple version will do for now. Bundling and unbundling are done in the same changeset because the test for parts is less modular. However, the result is not too complex.

merge with stable

Added signature for changeset 3f83fc5cfe71

Added tag 2.9.2 for changeset 3f83fc5cfe71

backout: correct commit status of no changes made (BC) (issue4190) If backout generated no changes to commit, it showed wrong status, "changeset <target> backs out changeset <target>", and raised TypeError with -v option. This changes the return code to 1, which is the same as "hg commit" and "hg rebase".