histedit: add torule method to histedit action objects To make histedit action objects responsible for understanding the format of their action lines we are adding a torule method which for a histedit action will return a string which can be saved in histedit state or shown in text editor when editing the plan.

histedit: add verify() to histeditaction This commits splits the parsing of the histedit rule from its semantic analysis. It's necessary because sometimes we want to do first without doing the former (reading the histedit state).

histedit: add addhisteditaction decorator This decorator will is allowing us to move the registering the action in actiontable closer to the action code. Also it is storing the verb inside histedit action so the action is aware of the verb needed to trigger it.

histedit: add actions property to histedit state I want to refactor histedit to use action objects instead of (verb, rest) pairs whenever possible. At the end of this series I want the rules to be translated into action objects when reading state and translated back when writing state. All histedit internals should use action objects instead of state rules. To migrate histedti internals sequentially I'm introducing the state.actions property to translate rules on the fly so we can use both state.actions and state.rules until refactoring is done.

check-commit: remove confusion between summary line and other headers The pull url header can easily grow over 80 chars. The check-commit script was confusing this with a too long summary line. We update the regular expression to not match other header.

shelve: execute checkunfinished inside wlock scope Before this patch, "hg shelve" of shelve extension executes 'cmdutil.checkunfinished()' before acquisition of wlock. It may cause unintentional result, if another command runs parallelly (see also issue4368). To avoid this issue, this patch executes 'cmdutil.checkunfinished()' inside wlock scope of "hg shelve". This also fixes issue4957, because now 'cmdutil.checkunfinished()' isn't invoked at "hg shelve" with options below: --cleanup --delete --list --patch --stat

shelve: widen wlock scope of shelve for consistency while processing Before this patch, "hg shelve" of shelve extension executes/refers below before acquisition of wlock: - 'repo.dirstate.parents()' via 'repo[None].parents()' - 'repo._activebookmark' It may cause unintentional result, if another command runs parallelly (see also issue4368). This patch widens wlock scope of "hg shelve" of shelve extension for consistency while processing.

gpg: make sign acquire wlock before processing Before this patch, "hg sign" of gpg extension executes/evaluates below without acquisition of wlock. - repo.dirstate.parents() - '.hgsigs' not in repo.dirstate It may cause unintentional result, if another command runs parallelly (see also issue4368). To avoid this issue, this patch makes "hg sign" of gpg extension acquire wlock before processing.

commands: execute checkunfinished and bailifchanged inside wlock scope Before this patch, "hg import" executes below before acquisition of wlock: - cmdutil.checkunfinished() - cmdutil.bailifchanged() It may cause unintentional result, if another command runs parallelly (see also issue4368). To avoid this issue, this patch executes 'cmdutil.checkunfinished()' and 'cmdutil.bailifchanged()' inside wlock scope of "hg import".

commands: widen wlock scope of graft for consitency while processing Before this patch, "hg graft" executes below before acquisition of wlock. - cmdutil.checkunfinished() - cmdutil.bailifchanged() - repo.dirstate.parents() via 'repo["."]' - unlinking '.hg/graftstate' It may cause unintentional result, if another command runs parallelly (see also issue4368). This patch widens wlock scope of "hg graft" for consitency while processing.

commands: make backout acquire locks before processing Before this patch, "hg backout" executes below before acquisition of wlock. - cmdutil.checkunfinished() - cmdutil.bailifchanged() - repo.dirstate.parents() It may cause unintentional result, if another command runs parallelly (see also issue4368). In addition to it, "hg backout" refers changelog for purposes below without acquisition of store lock (slock), and it may cause unintentional result, if store is updated parallelly. - show and update to the revision by 'repo.changelog.tip()' - examine for "created new head" by 'repo.branchheads()' and 'cmdutil.commitstatus()' To avoid this issue, this patch makes "hg backout" acquire wlock and slock before processing.

commands: make commit acquire locks before processing (issue4368) Before this patch, "hg commit" (process A) executes steps below: 1. get current branch heads via 'repo.branchheads()' - cache 'repo.changelog' 2. invoke 'repo.commit()' 3. acquire wlock - invalidate 'repo.dirstate' 4. access 'repo.dirstate' - re-read '.hg/dirstate' - check validity of parent revisions with 'repo.changelog' 5. invoke 'repo.commitctx()' 6. acquire store lock (slock) - invalidate 'repo.changelog' 7. do committing 8. release slock 9. release wlock 10. check new branch head (via 'cmdutil.commitstatus()') If acquisition of wlock at (3) above waits for another "hg commit" (process B) or so running parallelly to release wlock, process A causes creating orphan revision, because: - '.hg/dirstate' refers the revision, which is newly added by process B, as its parent - but already cached 'repo.changelog' doesn't contain such revision - therefore, validating parents of '.hg/dirstate' at (4) above replaces such revision with 'nullid' Then, process A creates "orphan" revision, of which parent is "null" revision. In addition to it, "created new head" may be shown at the end of process A unintentionally, if store is updated parallelly, because both getting branch heads (1) and checking new branch head (10) are executed outside slock scope. To avoid this issue, this patch makes "hg commit" acquire wlock and slock before processing. This patch resolves the issue between "hg commit" processes, but not one between "hg commit" and other commands. Subsequent patches resolve the latter. Even after this patch, there are still corner case problems below: - filecache may overlook changes of '.hg/dirstate', and it causes similar issue (see below for detail) https://bz.mercurial-scm.org/show_bug.cgi?id=4368#c10 - 3rd party extension may cause similar issue, if it directly uses 'repo.commit()' without acquisition of wlock and slock This can be fixed by acquisition of slock at the beginning of 'repo.commit()', but it seems suitable for "default" branch In fact, acquisition of slock itself is already introduced at "default" branch by 4414d500604f, but acquisition is not at the beginning of 'repo.commit()'. This patch also changes some tests: - test-fncache.t needs this tricky wrapping, to release (= forced failure of) wlock certainly - order of "hg commit" output is changed by widening scope of locks, because some hooks are fired after releasing wlock

addrevision: use general delta when the incoming base delta is bad We unify the delta selection process to be a simple three options process: - try to use the incoming delta (if lazydeltabase is on) - try to find a suitable parents to delta against (if gd is on) - try to delta against the tipmost revision The first of this option that yield a valid delta will be used. The test change in 'test-generaldelta.t' show this behavior as we use a delta against the parent instead of a full delta when the incoming delta is not suitable. This as some impact on 'test-bundle.t' because a delta somewhere changes. It does not seems to change the test semantic and have been ignored.

test: use a bigger manifest in general delta test The currently used manifest is too small and cannot sustain a chain length above "1". This make testing the 'lazybasedelta' behavior hard. So we add an extra file in the manifest to help testing in the next changeset. The semantic of existing tests have been checked and is not changed.

addrevision: rework generaldelta computation The old code have multiple explicit tests and code duplications. This makes it hard to improve the code. We rewrite the logic in a more generic way, not changing anything of the computed result. The final goal here is to eventually be able to: - factor out the default fallback case "try against 'prev'" in a single place - allow 'lazydeltabase' case to use the smarter general delta code path when the incoming base does not provide us with a good delta.

bmstore: close file in a finally block in _writerepo Also rename the variable to file_ to avoid shadowing a builtin.

bmstore: add basic clean-state tracking I'm about to move active-bookmark management into the bmstore. I'd like to avoid re-writing the bookmarks data (as distinct from the active bookmark file) if possible, so let's introduce some dirty-tracking early.

bookmarks: hoist getbkfile out of bmstore class It's totally fine that this hook exists, but I don't see a need for it to live inside the bmstore class.

bookmarks: document getbkfile method I'm working on bmstore again, and this function gave me a moment's pause. Document it to save future readers from any undue confusion.

hgweb: load server settings from --web-conf (issue4699) It copies the ui before loading the webconf and passes the copied ui only to the service. This way, the hgwebdir app can reload configs cleanly.

extdiff: correctly handle deleted subrepositories (issue3153) Previously, when extdiff was called on two changesets where a subrepository had been removed, an unexpected KeyError would be raised. Now, the missing subrepository will be ignored. This behavior mirrors the behavior in diffordiffstat from cmdutil.py line ~1138-1153. The KeyError is caught and the revision is set to None. try/catch of LookupError around matchmod.narrowmatcher and sub.status is removed, as LookupError is not raised anywhere within those methods or deeper calls.

hgweb: make sure command options are set to all ui objects Before this patch, it was unclear why the httpservice object could read the server options (e.g. --port) from 'ui'. It just worked because repo.ui is ui.

hgweb: eliminate duck-typing to select hgweb or hgwebdir by command option Since createservice() was moved to hgweb and hgweb imports both hgweb_mod and hgwebdir_mod, we no longer have to force hgweb() function to select one of them by the type of 'o' variable. Let's be explicit! This patch does not change hgweb() function because it is the interface of existing WSGI and CGI scripts.

addrevision: only use the incoming base if it is a good delta (issue4975) Before this change, the 'lazydeltabase' would blindly build a delta using the base provided by the incoming bundle and try to use it. If that base was far down the revlog, the delta would be seen as "no good" and we would fall back to a full text revision. We now check if the delta is good and fallback to a computing a delta again the tipmost revision otherwise (as we would do without general delta). Later changesets will improve the logic to compute the fallback delta using the general delta logic.

addrevision: handle code path not producing delta We would like to be able to exit the delta generation block without a valid delta (for a more flexible control flow). So we make sure we do not expand the "delta" content unless we actually have a delta. We can do it one level lower because 'delta' is initialised at None anyway. Not adding a level to the assignment prevent a line length issue.

addrevision: rename 'd' to 'delta' That variable is quite central to the whole function. Giving it a more explicit name help with code readability.

strip: add a --keep test related to removing files from dirstate When strip builds the list of changedfiles to pass into dirstate.rebuild, it adds files blindly, including those that have been removed. This tests ensures that rebuild can handle this case.

dirstate: change debugrebuilddirstate --minimal to use dirstate.rebuild When debugrebuilddirstate --minimal is called, rebuilding the dirstate was done outside of the appropriate rebuild function. This patch makes debugrebuilddirstate use dirstate.rebuild. This was done to allow our extension to become aware debugrebuilddirstate --minimal

debugrebuilddirstate: added tests for --minimal flag Added tests for debugrebuilddirstate --minimal.

debugdirstate: add command to rebuildstate test to modify dirstate Debugging the dirstate helps if you have options to add files for normal lookup or drop them from the dirstate. This patch adds a convenience command to test-rebuilddirstate.t to modify the dirstate. It will be used in the next patch to write proper tests for debugrebuilddirstate --minimal