bundlerepo: disable filtering of changelog while constructing revision text This avoids the following error that happened if base revision of bundle file was hidden. bundlerevlog needs it to construct revision texts from bundle content as revlog.revision() does. File "mercurial/context.py", line 485, in _changeset return self._repo.changelog.read(self.rev()) File "mercurial/changelog.py", line 319, in read text = self.revision(node) File "mercurial/bundlerepo.py", line 124, in revision text = self.baserevision(iterrev) File "mercurial/bundlerepo.py", line 160, in baserevision return changelog.changelog.revision(self, nodeorrev) File "mercurial/revlog.py", line 1041, in revision node = self.node(rev) File "mercurial/changelog.py", line 211, in node raise error.FilteredIndexError(rev) mercurial.error.FilteredIndexError: 1

merge: run update hook after the last wlock release There were 2 test failures in 3.4-rc when running test-hook.t with the largefiles extension enabled. For context, the first is a commit hook: @@ -618,9 +621,9 @@ $ echo 'update = hg id' >> .hg/hgrc $ echo bb > a $ hg ci -ma - 223eafe2750c tip + d3354c4310ed+ $ hg up 0 - cb9a9f314b8b + 223eafe2750c+ tip 1 files updated, 0 files merged, 0 files removed, 0 files unresolved make sure --verbose (and --quiet/--debug etc.) are propagated to the local ui In both cases, largefiles acquires the wlock before calling into core, which also acquires the wlock. The first case was fixed in 57f1dbc99631 by ensuring the hook only runs after the lock has been fully released. The full release is important, because that is what writes dirstate to the disk, allowing external hooks to see the result of the update. This simply changes how the update hook is called, so that it too is deferred until the lock is finally released. There are many uses of mergemod.update(), but in terms of commands, it looks like the following commands take wlock while calling mergemod.update(), and therefore will now have their hook fired at a later time: backout, fetch, histedit, qpush, rebase, shelve, transplant Unlike the others, fetch immediately unlocks after calling update(), so for all intents and purposes, its hook invocation is not deferred (but the external hook still sees the proper state).

censor: remove meaningless explanation about .hgcensored There is no code path handling ".hgcensored" in Mercurial source tree. This meaningless explanation may make users misunderstand about censor.

parsers: avoid signed integer overflow in calculation of leaf-node index If v = -INT_MAX - 1, -v would exceed INT_MAX. I don't think this would cause problems such as issue4627, but we can't blame it as a compiler bug because signed integer overflow is undefined in C.

bundle2: disable ouput capture unless we use http (issue4613 issue4615) The current bundle2 processing was capturing all output. This is nice as it provide better meta data about what output what, but this was changing two things: 1) adding a prefix "remote: " to "other" output during local push (issue4613) 2) local and ssh push does not provide real time output anymore (issue4615) As we are unsure about what form should be used in (1) and how to solve (2) we disable output capture in this two cases. Output capture can be forced using an experimental option.

subrepo: propagate the --hidden option to hg subrepositories With many commands accepting a '-S' or an explicit path to trigger recursing into subrepos, it seems that --hidden needs to be propagated too. Unfortunately, many of the subrepo layer methods discard the options map, so passing the option along explicitly isn't currently an option. It also isn't clear if other filtered views need to be propagated, so changing all of those commands may be insufficient anyway. The specific jam I got into was amending an ancestor of qbase in a subrepo, and then evolving. The patch ended up being hidden, and outgoing said it would only push one unrelated commit. But push aborted with an 'unknown revision' that I traced back to the patch. (Odd it didn't say 'filtered revision'.) A push with --hidden worked from the subrepo, but that wasn't possible from the parent repo before this. Since the underlying problem doesn't actually require a subrepo, there's probably more to investigate here in the discovery area. Yes, evolve + mq is not exactly sane, but I don't know what is seeing the hidden revision. In lieu of creating a test for the above situation (evolving mq should probably be blocked), the test here is a marginally useful case where --hidden is needed in a subrepo: cat'ing a file in a hidden revision. Without this change, cat aborts with: $ hg --hidden cat subrepo/a skipping missing subrepository: subrepo [1]

subrepo: don't pass the outer repo's --rev or --branch to subrepo incoming() When passing a --rev, 'hg incoming -S' previously suffered from the same output truncation or abort that was fixed for 'hg outgoing -S' in the previous patch, for the same reasons. Unlike push, subrepos are currently only pulled when the outer repo is updated, not when the outer repo is pulled. That makes matching 'hg pull' behavior impossible. Listing all incoming csets in the subrepo seems like the most useful behavior, and is consistent with 'hg outgoing -S'.

subrepo: don't pass the outer repo's --rev or --branch to subrepo outgoing() The previous behavior didn't reflect what would actually be pushed- push will ignore --rev and --branch in the subrepo and push everything. Therefore, 'push -r {rev}' would not list everything, unless {rev} was also the revision of the subrepo's tip. Worse, if a hash was passed in, the command would abort because that hash would either not be in the outer repo or not in the subrepo.

subrepo: update the help text to account for diff -I/-X gitsubrepo support The functionality was added in a48b65ab428d.

revert: make the interactive mode experimental While fixing issue4304: "record: allow editing new files" we introduced changes in record/crecord. These changes need to be matched with changes in any command using record. Revert is one of these commands and the changes have not been made for this release. Therefore, revert -i should be an experimental feature for this release.

shelve: make the interactive mode experimental While fixing issue4304: "record: allow editing new files" we introduced changes in record/crecord. These changes need to be matched with changes in any command using record. Shelve is one of these commands and the changes have not been made for this release. Therefore, shelve -i should be an experimental feature for this release.

help: also hide options marked EXPERIMENTAL Similar to DEPRECATED, add a way to indicate that options are EXPERIMENTAL.

rebase: don't forward "source" argument to rebase (issue4633) `hg pull` takes an optional "source" argument to define the path/url to pull from. Under some circumstances, this option could get proxied to rebase and interpretted as the --source argument to rebase, leading to unexpected behavior. In my local environment, "source" always appears in "opts" in pullrebase. However, when attempting to write a test, I couldn't reproduce this. Instead, the source is being captured as a positional argument in "args." I suspect an interaction between **kwargs and an extension is to blame for the differences in behavior. This is why no test has been written. I have tested behavior locally and the patch has the intended side-effect of making `hg pull --rebase` work again.

histedit: fix reST syntax problem of example code in help document To show example code correctly in online help, text block of it should be preceded by "::" and indented.

webcommands: fix description of manifest default behavior

webcommands: fix typo in changelog documentation

record: fix adding new file with record from within a subdir (issue4626) In my latest change on record (edit newly added file), I forgot the repo.wjoin() so that record was not computing the paths properly to delete the backups and was crashing.

tests: fix backslashes in test-casefolding

cmdutil: avoid wrapping ctx.phasestr() by _() This wrapping seems meaningless, because: - there is no "_()" invocation to prepare for extracting phase names to be translated "make update-pot" doesn't extract msgids for phase names - phase names are kine of reserved keywords like as branch name "default"

repair: avoid string concatenation by + operator String concatenation by "+" operator causes failure of extracting messages to be translated. Python automatically concatenates strings separated by whitespaces into one string.

annotate: always adjust linkrev before walking down to parents (issue4623) This should avoid the bad performance in the following scenario. Before this patch, on "hg annotate -r10000", p.rev() would walk changelog from 10000 to 3 because _descendantrev was 10000. With this patch, it walks from 5 to 3. 1 -- 2 -- 4 -- 5 -- ... -- 10000 \ 'p' 'f' - 3 (grafted 3 to 4) 'p' repo: https://hg.mozilla.org/releases/mozilla-beta/#4f80fecda802 command: hg annotate -r b0a57152fd14 browser/app/profile/firefox.js before: 83.120 secs after: 3.820 secs This patch involves extra calls of narrow _adjustlinkrev(), but the cost of them seems relatively small compared to wide _adjustlinkrev() calls eliminated by this patch. repo: http://selenic.com/repo/hg/#8015a3cf1380 command: hg annotate mercurial/commands.py before: 7.380 secs after: 7.320 secs repo: https://hg.mozilla.org/mozilla-central/#f214df6ac75f command: hg annotate layout/generic/nsTextFrame.cpp before: 5.070 secs after: 5.050 secs repo: https://hg.mozilla.org/releases/mozilla-beta/#4f80fecda802 command: hg annotate -r 4954faa47dd0 gfx/thebes/gfxWindowsPlatform.cpp before: 1.600 secs after: 1.620 secs

test-commit-interactive: stablize output for no-execbit platforms

test-bundle2-exchange: make hooks compatible with Windows The cmd.exe process doesn't fail the hook when "; false" is appended, and its echo command prints out the quote characters.

i18n: extract doc string of each web commands as translatable one Before this patch, doc string of each web commands isn't extracted as translatable one, even though web commands are listed up in "hg help hgweb". This patch adds "mercurial/hgweb/webcommands.py" on to arguments of "i18n/hggettext". "i18nfunctions" added into "webcommands.py" is used by "i18n/hggettext" to get the list of functions having translatable doc string.

subrepo: don't write .hgsubstate lines with empty subrepo state (issue4622) The '' that is used to represent the state of a not-yet-committed subrepo cannot be written to the file, because the code that parses the file splits on ' ' and expects two parts. Given that the .hgsubstate file is automatically rewritten on commit, it seems a little strange that the file is written out during a merge.

revert: restore the ability to revert across case only renames (issue4481) This regressed in 5e16fe6fdd32, in what looks like an unrelated change. It seems sufficient to pass 'ignoremissing=True', but the restored try/except has been there for six years since 41bb88cb913e, so this seems safer for now. Note that renaming directories in the filename doesn't appear to work- not sure if this would end up throwing a different type of error when that is fixed.

convert: avoid traceback in subversion sink We've had a couple reports of subversion tracebacks that trigger when the parents list is empty, and thus block showing what the commit failure was on the next two lines.

merge with i18n

i18n-pt_BR: synchronized with 5947a68fa271

template-phases: do not show description or summary if empty This is a copy of c1cb6523e968, but applied to the phases template, which is itself almost a copy of the default template. Perhaps we should rewrite these templates to use the %include syntax afforded by config files. This change seems a bit too big for stable, though.

test-command-template: add a short description of a temporary snippet This snippet was introduced by c1cb6523e968. Its purpose seems a bit obscure without historical context.

bundle2: also save output when error happens during part processing Until this changeset, we were only able to save output if an error happened during the 'transaction.close()' phase. If the 'processbundle' call raised an exception, the 'bundleoperation' object was never returned, so the reply bundle was never accessible and no output could be salvaged. We introduce a quick (but not very elegant) fix to gain access to any reply created during the processing. This conclude this output related series. We should hopefully be able client-side to see the whole server output, in a proper order. The code is now complex enough that a refactoring of it would make sense on default.

bundle2: capture output issue during transaction close We were capturing all output issue during bundle2 processing, and all output issue during transaction rollback in case of failure. However, the output issue during transaction commit was still roaming the land freely. It is now put back in line. This let the user see output from 'pretxnclose' and 'txnclose' (and related) in the right order.

bundle2: also capture hook output during processing External hook used to directly write on stdout and stderr. As a result their output was not captured by the bundle2 processing. This resulted in confusing out of order output on the client side. We are now capturing hooks output in this context.

ui: allow capture of subprocess output We want to capture hooks output during bundle2 processing. For this purpose we introduce a new 'subproc' argument to 'ui.pushbuffer'. When set, the output of sub process created through 'ui.system' will be captured in the buffer too. This will be used in the next changeset.

bundle2: capture transaction rollback message output (issue4614) The output from the transaction rollback was not included into the reply bundle. It was eventually caught by the usual 'unbundle' output capture and sent to the client but the result was out of order on the client side. We now capture the output for the transaction release and transmit it the same way as all other output. We should probably rethink the whole output capture things but this would not be appropriate for stable. The is still multiple cases were output failed to be properly capture, they will be fixed in later changesets.

bundle2: issue remote output as "status" (issue4612) Remote output should be silenced by --quiet. The issue was found while running `test-largefiles-cache.t` so it will get tested once we switch bundle2 by default.

record: edit patch of newly added files (issue4304) I tried to fix this issue in the past and had to revert the fix. This is a second attempt without the regression we found with the first one. record defines special headers (of file) as headers whose hunk are not shown to the user for editing, they are used to represent deleted, moved and new files. Since we want to authorize editing the patch of newly added file we make the newly added file with some content not special anymore. This entails that we have to save their content before applying the backup to be able to revert it if the patch does not apply properly. We reintroduce the test showing that newly added files can be edited and that their content is shown to the user.

record: remove useless line in test Remove a useless line in test to make the next patch cleaner.

revert: apply normallookup on reverted file if size isn't changed (issue4583) Before this patch, reverting a file to the revision other than the parent doesn't update dirstate. This seems to expect that timestamp and/or size will be changed by reverting. But if (1) dirstate of file "f" is filled with timestamp before reverting and (2) size and timestamp of file "f" isn't changed at reverting, file "f" is recognized as CLEAN unexpectedly. This patch applies "dirstate.normallookup()" on reverted file, if size isn't changed. Making "localrepository.wwrite()" return length of written data is needed to avoid additional (and redundant) "lstat(2)" on the reverted file. "filectx.size()" can't be used to know it, because data may be decoded at being written out. BTW, interactive reverting may cause similar problem, too. But this patch doesn't focus on fixing it, because (1) interactive (maybe slow) reverting changes one (or both) of size/timestamp of reverted files in many usecases, and (2) changes to fix it seems not suitable for stable branch.

win32: remove cacert.pem file from Inno Setup installer Duplicate the modification done in 6e38b6fc4123 for wix installer so that CA certificates loading works fine with Python 2.7.9+.

revert: accept just -I/-X without paths or -a/-i (issue4592) 'hg revert -I foo' currently fails with abort: no files or directories specified (use --all to revert all files, or 'hg update 1' to update) It doesn't seem intentional that -I/-X without other paths or --all/--interactive should fail, and it doesn't seem that harmful to allow it either, so let's just do that.

record: change wording for record curses interface We are using record and crecord in different context, not just for commiting changes but also reverting and shelving changes. This diff changes the wording from commiting to confirming changes to avoid confusing the users with what they are doing.

wix: remove cacert.pem from Windows distribution It should not be included in the Windows installers because it prevents loading CA certificates from the system store on Python 2.7.9, implemented by 760a86865f80. The msi packages bundles Python 2.7.9, so cacert.pem is no longer necessary. Backed out changeset e5c2338d76b5

tests: use double quote to quote arguments in hook for portability On windows, single quote doesn't work as quote character in hook command line, because "cmd.exe" doesn't recognize it as quoting character. And this causes failure of test. This patch uses double quote to quote arguments in hook instead of single quote for portability. Even though single quotes for "[hooks] pretxncommit" in test-clone-pull-corruption.t seems to work correctly (may MinGW sh treat specially ?) AFAIK, this patch also replaces them by double quotes for consistency.

record: fix record with change on moved file crashes (issue4619) reverting 79fceed67676, add a test to prevent the issue from coming back.

check-code: check os.path.join(*, '') not working correctly with Python 2.7.9 Since Python 2.7.9, "os.path.join(path, '')" doesn't append "os.sep" for UNC path (see issue4557 for detail).

unionrepo: use pathutil.normasprefix to ensure os.sep at the end of cwd Since Python 2.7.9, "os.path.join(path, '')" doesn't add "os.sep" at the end of UNC path (see issue4557 for detail). This makes unionrepo incorrectly work, if: 1. cwd is the root of UNC share (e.g. "\host\share"), and 2. mainreporoot is near cwd (e.g. "\host\sharefoo\repo") - host of UNC path is same as one of cwd - share of UNC path starts with one of cwd 3. "repopath" isn't specified in URI (e.g. "union:path/to/repo2") For example: $ hg --cwd \host\share -R \host\sharefoo\repo incoming union:path\to\repo2 In this case: - os.path.join(r"\host\share", "") returns r"\host\share", - r"\host\sharefoo\repo".startswith(r"\host\share") returns True, then - r"foo\repo" is treated as repopath of unionrepo instead of r"\host\sharefoo\repo" This causes failure of combining "\host\sharefoo\repo" and another repository: in addition to it, "\host\share\foo\repo" may be combined with another repository, if it accidentally exists. This patch uses "pathutil.normasprefix()" to ensure "os.sep" at the end of cwd safely, even with some problematic encodings, which use 0x5c (= "os.sep" on Windows) as the tail byte of some multi-byte characters. BTW, normalization before "pathutil.normasprefix()" isn't needed in this case, because "os.getcwd()" always returns normalized one.

bundlerepo: use pathutil.normasprefix to ensure os.sep at the end of cwd Since Python 2.7.9, "os.path.join(path, '')" doesn't add "os.sep" at the end of UNC path (see issue4557 for detail). This makes bundlerepo incorrectly work, if: 1. cwd is the root of UNC share (e.g. "\host\share"), and 2. mainreporoot is near cwd (e.g. "\host\sharefoo\repo") - host of UNC path is same as one of cwd - share of UNC path starts with one of cwd 3. "repopath" isn't specified in bundle URI (e.g. "bundle:bundlefile" or just "bundlefile") For example: $ hg --cwd \host\share -R \host\sharefoo\repo incoming bundle In this case: - os.path.join(r"\host\share", "") returns r"\host\share", - r"\host\sharefoo\repo".startswith(r"\host\share") returns True, then - r"foo\repo" is treated as repopath of bundlerepo instead of r"\host\sharefoo\repo" This causes failure of combining "\host\sharefoo\repo" and bundle file: in addition to it, "\host\share\foo\repo" may be combined with bundle file, if it accidentally exists. This patch uses "pathutil.normasprefix()" to ensure "os.sep" at the end of cwd safely, even with some problematic encodings, which use 0x5c (= "os.sep" on Windows) as the tail byte of some multi-byte characters. BTW, normalization before "pathutil.normasprefix()" isn't needed in this case, because "os.getcwd()" always returns normalized one.

dirstate: use pathutil.normasprefix to ensure os.sep at the end of root 3cc630be5f09 replaced "os.path.join(root, '')" by "root.endswith(os.sep)" examination, because Python 2.7.9 changes behavior of "os.path.join(path, '')" on UNC path. But some problematic encodings use 0x5c (= "os.sep" on Windows) as the tail byte of some multi-byte characters, and replacement above prevents Mercurial from working on the repository, of which root path ends with such multi-byte character, regardless of enabling win32mbcs. This patch uses "pathutil.normasprefix()" instead of "root.endswith(os.sep)" examination, to ensure "os.sep" at the end of "dirstate._rootdir" even with problematic encodings. "root" of dirstate can be passed to "pathutil.normasprefix()" without normalization, because it is always given from "repo.root" = "repo.wvfs.base", which is normalized by "os.path.realpath()". Using "util.endswithsep()" instead of "str.endswith(os.sep)" also fixes this problem, but this patch chooses "pathutil.normasprefix()" to centralize "adding os.sep if endswith(os.sep)" logic into it.

bookmarks: add i18n hints to bookmark sync states

histedit: fix typo in abort message

help: fix typos in txnclose and txnabort documentation

util: fix the check for non-C99 compilers (issue4605)

histedit: fix rollup prompting for a commit message (issue4606) Commit 090da03361c5 broke histedit's rollup by causing it to open the editor. Turns out I missed a spot where the rollup option was read. This fixes that and adjusts the test to catch this case.

test-hook.t: don't directly use redirect to /dev/null in hook for Windows This goes with 57f1dbc99631. External hooks are run in cmd.exe, which doesn't know about /dev/null, but sh can handle it.

mq: avoid silent failure when single patch doesn't apply (issue4604) The error-handling here is quite byzantine. self._apply raises an AbortNoCleanup, but self.apply was swallowing the exception and returns 2. In self.push, we catch all exceptions.. and cleanup. We try to print a message to clean up.. but that relies on having a top-of-stack. Instead, we re-raise the abort in self.apply, and avoid cleanup on AbortNoCleanup in self.push by adding a trivial new except clause. We also modernize the now-visible abort message.

pushkey: flush pending data before running a pre-pushkey hook (issue4607) The pre-pushkey hook will likely validate the pushkey based on element previously changed in the same transaction. We need to make theses data available for the hook.

pushkey: use hook arguments from transaction when one exists When pushkey is called during a transaction, we include its 'hookargs' when running the pre-pushkey hooks. Having more data cannot hurt, especially the transaction ID.

util.h: also sniff for C99 before trying to use stdbool.h (issue4605) stdbool.h isn't required until C99. Sigh.

changelog: fix readpending if no pending data exist (issue4609) Since transaction are used for more than just changesets, it is possible to have a transaction without new changesets at all. In this case no ''00changelog.i.a' are written. In all cases the 'changelog.readpending' method is called if the repository has any pending data. The 'revlog' logic provides empty content if the file is missing, so the whole operation resulted in an empty changelog. We now skip reading the pending file if it is missing.

afterlock: add the callback to the top level lock (issue4608) If 'wlock' is taken, we should add 'afterlock' callback to the 'wlock' instead. Otherwise, running post transaction hook after 'lock' is release but 'wlock' is still taken lead to a deadlock (eg: 'hg update' during a hook). This situation is much more common since: 5dc5cd7abbf5 push: acquire local 'wlock' if "pushback" is expected (BC) (issue4596)

debuglock: access the lock file using the proper vfs Before this fix, 'wlock' was always reported as "free" because '.hg/store/wlock' have never been a thing.

status: pass matcher to pathcopies() Just like a7f8e3584ef3 (diff: pass the diff matcher to the copy logic, 2015-04-16) sped up 'hg diff --git $path', let's speed up 'hg st -C $path'. On the Firefox repo, this speeds up hg st --rev tip~40000 --rev tip -C python from 16s to 1.8s. Those two revisions differ in 100k files, out of which 1k is in python/.