sortdict: add iteritems method Future patches will start using sortdict for log operations where order is important. Adding iteritems removes the headache of having to remember to use items() if the object is a sortdict.

addremove: add back forgotten files (BC) After running "hg forget README && hg addremove", README will still be reported as removed, while "hg forget README && hg add README" adds it back so it gets reported as clean. It seems like they should behave the same. Furthermore, it seems like no files should remain untracked after 'hg addremove && hg commit' (or 'hg commit -A'). For these reasons, change the behavior of addremove so it does add forgotten files back. The problem is with scmutil._interestingfiles(), which reports the file as removed, so scmutil.addremove() does not add it. Fix by teaching _interestingfiles() to report forgotten files separately from removed files and make addremove() add forgotten files back. However, do not treat forgotten files as sources for rename detection. Note that since removed and forgotten files are treated the same before this change, forgotten files were considered sources for rename detection. Also update the other caller, marktouched(), in the same way as addremove().

add: add back forgotten files even when not matching exactly (BC) I accidentally did 'hg forget .' and tried to undo the operation with 'hg add .'. I expected the files to be reported as either modified or clean, but they were still reported as removed. It turns out that forgotten files are only added back if they are listed explicitly, as shown by the following two invocations. This makes it hard to recover from the mistake of forgetting a lot of files. $ hg forget README && hg add README && hg status -A README C README $ hg forget README && hg add . && hg status -A README R README The problem lies in cmdutil.add(). That method checks that the file isn't already tracked before adding it, but it does so by checking the dirstate, which does have an entry for forgotten files (state 'r'). We should instead be checking whether the file exists in the workingctx. The workingctx is also what we later call add() on, and that method takes care of transforming the add() into a normallookup() on the dirstate. Since we're changing repo.dirstate into wctx, let's also change repo.walk into wctx.walk for consistency (repo.walk calls wctx.walk, so we're simply inlining the call).

context.status: explain "caching reasons" more fully Where we "load earliest manifest first for caching reasons", elaborate on what "caching reasons" refers to. Text provided by Matt in http://thread.gmane.org/gmane.comp.version-control.mercurial.devel/73235/focus=73578.

localrepo: rename revlog.maxchainlen to format.maxchainlen This is more consistent with other option names, as spotted by Pierre-Yves. Thanks!

revlog: add config variable for limiting delta-chain length The current heuristic for deciding between storing delta and full texts is based on ratio of (sizeofdeltas)/(sizeoffulltext). In some cases (for example a manifest for ahuge repo) this approach can result in extremely long delta chains (~30,000) which are very slow to read. (In the case of a manifest ~500ms are added to every hg command because of that). This commit introduces "revlog.maxchainlength" configuration variable that will limit delta chain length.

debugrevlog: fix computing chain length in debugrevlog -d The chain length was computed correctly only when generaldelta feature was enabled. Now it's fixed. When generaldelta is disabled the base revision in revlog index is not the revision we have delta against - it's always previous revision. Instead of incorrect chainbaseandlen in command.py we are now using two single-responsibility functions in revlog.py: - chainbase(rev) - chainlen(rev) Only chainlen(rev) was missing so it was written to mimic the way the chain of deltas is actually found during file reconstruction.

transaction: factorise append-only file registration The addition is done in two different places but differs slightly. We factorise this addition to ensure it is consistent in all places.

transaction: document `tr.add`

transaction: drop backupentries logic from startgroup and endgroup The `startgroup` and `endgroup` methods are used in a very specific context to wrap a very specific operation (revlog truncation). It does not make sense to perform any other operations during such a "group" (eg:file backup). There is currently no user of backupfile during a "group" so we drop the group-specific code and restrict authorized operations during "group".

transaction: document startgroup and endgroup These enigmatic methods are only used in repair. We document them to clarify there purpose and user.

transaction: mark backup-related attributes private As the transaction is gaining more functions and attributes, it is important to clarify what is part of the public API.

transaction: document the contents of `tr.backupentries` Now that all items are known we can document it.

transaction: drop the third item in `tr.backupentries` This third item is always None and never used.

rebase: fix rebase with no common ancestors (issue4446) The new rebase revset didn't check for the case when there are no common ancestors. Now it does. The new behavior should be the same as the pre-3.2 behavior. Added a test.

test-run-tests: accept more levels of precision and trailing ws (issue4440) simplejson produces slightly different output from the built-in json module, specifically: * It uses 0.000 instead of 0.0000 * It likes to put a trailing space after a comma This change works around both of those variations.

merge with stable

changegroup.cg2packer: lookup 'group' via inheritance chain This lets extensions insert themselves in the class hierarchy.

context.status: only filter suspect symlinks in the dirstate status We don't care about filtering out symlinks that have already been committed with full content, only those that have been accidentally resolved in the working directory.

context.status: inline _poststatus() By inlining _poststatus() into _buildstatus(), it becomes clearer that it is only called for the workingctx.

context.status: remove now-empty _prestatus()

context.status: call _dirstatestatus() from within _buildstatus() By making the call to _dirstatestatus() within _buildstatus(), it becomes clearer that it's called only for the workingctx.

context.status: move manifest caching trick to _buildstatus() In basectx._buildstatus(), we read the manifests for the two revisions being compared. For "caching reasons" unknown to me, it is better to read the earlier manifest first, which basectx._prestatus() takes care of. However, if the 'self' context is a committablectx and the 'other' context is the parent of the working directory (as in the very common case of plain "hg status"), there is no need to read any manifests at all -- all that's needed is the dirstate status. To avoid reading the manifests, _prestatus() is overridden in committablectx and avoids calling its super method, and _buildstatus() calls its super method only if the 'other' context is not the parent of the working directory. It seems easier to follow what's happening if we move the pre-fetching to _buildstatus() just before the place where the manifests are fetched. We just need to add an extra check that the revision is not None to handle the case that was previously handled by subclass overriding. That also makes it safe for committablectx._prestatus() to call its parent, although the latter now becomes empty, so we won't bother.

context.status: remove unused arguments from _matchstatus()

context.status: remove overriding in workingctx The workingctx method simply calls the super method. The only effect it has is that it uses a different default argument for the 'other' argument. The only in-tree caller is patch.diff, which always passes an argument to the method, so it should be safe to remove the overriding. Having the default argument depend on the type seems rather dangerous anyway.

synthrepo: when adding files, ensure new path is not a directory

synthrepo: synthesized dates must be positive, fit in 32-bit signed ints

discovery: test coverage for issue4438 / 86c35b7ae300 / 73cfaa348650 The randomness in the discovery protocol made this problem hard to reproduce. The test mocks random.sample to make sure we hit the problem every time. The set iteration order also made the output unstable ... but with the issue fixed, it is stable.

hgweb: fix a crash when using web.archivesubrepos A matcher is required when enabling the subrepo option on archival.archive(), because that calls match.narrowmatcher(), which accesses fields on the object. It's therefore probably a bad idea to default the matcher to None on archive(), but that's a fix for default.

tests: introduce a subrepository to test-archive.t This will be used in an upcoming patch to add coverage for web.archivesubrepos.

test-status-rev: add tests for plain dirstate and inter-revision status We have tests for the status across from '.^' to the working copy. It makes sense to have the similar tests for the inter-revision status between '.^' and '.' and for the dirstate status in the same place.

test-status-rev: remove unnecessary initial commit The initial commit was there when we had a group of tests that compared against an empty base, but since those tests no longer exist, we can drop the empty commit.

test-status-rev: use one glob for each expected status It's getting a little hard to read the ~30 calls to 'hg status' with one per file. Instead, let's use one glob for each expected status. For example, modified files can be listed with 'glob:content1_*_content[23]-tracked'. That also nicely becomes an explanation for why each status is expected.

test-status-rev: remove duplicate tests The second group of tests in test-status-rev compare to an empty revision. The first group of tests that compare to the first commit should be testing all the same states with the missing_* files, so drop the second group of tests.

changegroup: sparsely populate fnodes Previously, fnodes had a key and empty dict value for every element in changedfiles. This is somewhat wasteful. Empty dicts in CPython consume a lot more memory than you would expect - 280 bytes. On mozilla-central, which has ~190,000 files/fnodes keys, the previous loop populating fnodes allocated 91,924 KB of memory, most of that for the empty dicts. With this patch in place, our peak RSS during mozilla-central clone drops: before: 364,356 KB after: 326,008 KB delta: -38,348 KB When combined with the previous patch, total peak RSS decrease is now 190,116 KB.

changegroup: don't store unused value on fnodes (issue4443) The contents of fnodes are only accessed once per key. It is wasteful to cache the value since nobody will use it. Before this patch, the caching of unused data in fnodes was effectively causing a memory leak during the file streaming part of bundle creation. On mozilla-central (which has ~190,000 entries in fnodes), this patch has a significant impact on RSS at the end of generate(): before: 516,124 KB after: 364,356 KB delta: -151,768 KB The origin of this code can be traced back to 627cd7842e5d and has been with us since the 2.7 release.

changegroup: don't define lookupmf() until it is needed lookupmf() is currently defined earlier than when it is needed. Future patches further refactoring this code will be easier to read when lookupmf() is in its new home.

mail: actually use the verifycert config value The mail module only verifies the smtp ssl certificate if 'verifycert' is enabled (the default). The 'verifycert' can take three possible values: - 'strict' - 'loose' - any "False" value, eg: 'false' or '0' We tested the validity of the third value, but never converted it to actual falseness, making 'False' an equivalent for 'loose'. This changeset fixes it.

exchange: use the postclose API on transaction As with changegroup, we should wait for the transaction to be really closed before scheduling hook execution.

changegroup: use the 'postclose' API on transaction The post-transaction hooks run after the lock release (because hooks may want to touch the repository), but they must only run if the transaction is successfully closed. We use the new 'addpostclose' method on transaction to register a callback installing this post-lock-release call.

transaction: allow registering a post-close callback The addchangegroup code considers the transaction done after a 'tr.close()' call and schedules the hook's execution for after lock release. In the nested transaction case, the transaction is not yet committed and we must delay this scheduling. We add an 'addpostclose' method (like the 'addpending' and 'addfinalize' ones) that registers code to be run if the transaction is successfully committed.

exchange: swap "push" for "pull" in pulloperation docstring

exchange: prepare kwargs for bundle2 part generation exactly once

exchange: fix indentation in _pullchangeset

dagutil: fix id/ix typos in docstrings

patchbomb: extract 'getpatchmsgs' closure into its own function Keep marching toward the promised land of simplification!

patchbomb: extract 'makeintro' closure into its own function Keep marching toward the promised land of simplification!

patchbomb: extract 'getbundlemsgs' closure in its own function Keep marching toward the promised land of simplification!