dagparser: use absolute_import

crecord: use absolute_import

hg: use absolute_import

fileset: use absolute_import

demandimport: support lazy loading for absolute_import Before, we didn't support lazy loading if absolute_import was in effect and a fromlist was used. This meant that "from . import X" wasn't lazy and performance could suffer as a result. With this patch, we now support lazy loading for this scenario. As part of developing this, I discovered issues when module names are defined. Since the enforced import style only allows "from X import Y" or "from .X import Y" in very few scenarios when absolute_import is enabled - scenarios where Y is not a module and thus there is nothing to lazy load - I decided to drop support for this case instead of chasing down the errors. I don't think much harm will come from this. But I'd like to take another look once all modules are using absolute_import and I can see the full extent of what is using names in absolute_import mode.

demandimport: support keyword arguments on _hgextimport __import__ supports keyword arguments since Python 2.5. This proxy should too. An upcoming patch will use this feature.

demandimport: refactor logic and add documentation demandimport doesn't currently support absolute imports (level >= 0). In preparation for this, we add some documentation and a code branch to handle the absolute_import case.

demandimport: add __future__ to ignore list __future__ is special. We should definitely not be trying to lazy load it.

demandimport: remove support for Python < 2.5 The removed code was to support an __import__ function that doesn't support the "level" argument. This argument was added in Python 2.5, which we no longer support.

dispatch: use absolute_import A mixed, ambiguous import has been removed!

commands: break import cycle This was the easiest link in the chain to break as there was only one use of commandserver in commands.py. Other files had multiple users.

cmdutil: break import cycle This was the easiest place to break the chain, as there were only 2 uses of the imported module in the file.

revset: make balanced addsets by orset() without using _combinesets() As scmutil.revrange() was rewritten to not use _combinesets(), we no longer need _combinesets().

revrange: evaluate all revset specs at once This provides an opportunity for revset to optimize given expressions. For example, "-r0 -r1 -r2" can be optimized to "_list(0 1 2)".

revset: add matchany() to construct OR expression from a list of specs This will allow us to optimize "-rREV1 -rREV2 ..." command-line options.

revset: split post-parsing stage from match() _makematcher() will be reused by new matchany(ui, specs, repo=None) function I'll add by the next patch.

filesets: ignore unit case in size() predicate for single value When specifying one plain value in size(), e.g. size(1k), fileset tries to guess the upper bound automatically (see the comment in _sizetomax()). It didn't ignore the specified unit's case, and so size("1 GB"), for example, produced this error: hg: parse error: couldn't parse size: 1 GB Let's do the same thing that util.sizetoint() does: .lower(). The two test lines without output just check that there are no parse errors.

copies: use absolute_import

config: use absolute_import

changelog: use absolute_import

changegroup: use absolute_import

bundlerepo: use absolute_import

bundle2: use absolute_import

branchmap: use absolute_import

bookmarks: use absolute_import

archival: use absolute_import

ancestor: use absolute_import A few months ago, import-checker.py was taught to enforce a more well-defined import style for files with absolute_import. However, we stopped short of actually converting source files to use absolute_import because of problems with certain files. Investigation revealed the following problems with switching to absolute_import universally: 1) import cycles result in import failure on Python 2.6 2) undetermined way to import C/pure modules While these problems need to be solved, they can be put off. This patch starts a series of converting files to absolute_import that won't exhibit any of the aforementioned problems.

discovery: always use batching now that all peers support batching Some peers will transparently downgrade batched requests to non-batched ones, but that simplifies code for everyone using batching.

wireproto: make wirepeer look-before-you-leap on batching This means that users of request batching don't need to worry themselves with capability checking. Instead, they can just use batching, and if the remote server doesn't support batching for some reason the wirepeer code will transparently un-batch the requests. This will allow for some slight simplification in a handful of places. Prior to this change, largefiles would have been silently broken against a server which did not support batching.

batching: migrate basic noop batching into peer.peer "Real" batching only makes sense for wirepeers, but it greatly simplifies the clients of peer instances if they can be ignorant to actual batching capabilities of that peer. By moving the not-really-batched batching code into peer.peer, all peer instances now work with the batching API, thus simplifying users. This leaves a couple of name forwards in wirepeer.py. Originally I had planned to clean those up, but it kind of unclarifies other bits of code that want to use batching, so I think it makes sense for the names to stay exposed by wireproto. Specifically, almost nothing is currently aware of peer (see largefiles.proto for an example), so making them be aware of the peer module *and* the wireproto module seems like some abstraction leakage. I *think* the right long-term fix would actually be to make wireproto an implementation detail that clients wouldn't need to know about, but I don't really know what that would entail at the moment. As far as I'm aware, no clients of batching in third-party extensions will need updating, which is nice icing.

parsers: fix memory leak in compute_phases_map_sets PySet_Add increments the reference of the added object to the set, see: https://hg.python.org/cpython/file/2.6/Objects/setobject.c#l379 Before this patch we were forgetting to decrement the reference count after adding objects to the phaseset. This patch fixes the issue and makes the reference count right so that these objects can be properly garbage collected.

histedit: do not stay on a cleaned nodes on abort There is case where nodes are neither in tmpnodes nor leaf but still get removed. For example, if you used the "edit" action, made a commit and run --abort. The commit you made is not tracked by histedit, yet it will likely be cleaned up with its parent. The commit may not tracked because no replacements computations are done in the --abort case.

histedit: also update away from tmpnodes The working copy may be on a tmpnodes, we need to update away before it is stripped from the repository.

histedit: use revset to check if we need to update during abort The for loop is already quite more complicated than necessary and we are about to add some logic. Instead, we use a simple revset. Revset laziness should provide us with similar performance.

histedit: remove useless 'else' clause This 'else: pass' clause have no effect. We drop it for clarity.

histedit: make cleanupnode more robust The goal of this function is to strip content out of the repository. We do not really care if this content is visible or cleanup node not and we should proceed anyway. None of the internal actions are subject to this, however, a third party extension running arbitrary commands during histedit is affected by this.

convert: fix git convert using servers branches The conversion from git to hg was reading the remote branch list directly from the origin server. If the origin's branch had moved forward since the last git fetch, it would return a git hash which didn't exist locally, and therefore the branch was not converted. This changes it to rely on the local repo's refs/remotes list of branches instead, so it's completely cut off from the server.

revrange: drop old-style parser in favor of revset (API) Now revset can parse nullary ":" operator and existing "foo+bar" tags, we don't need the old-style parser. This means scmutil.revsingle(), revpair() and revrange() no longer accept a binary nodeid. An integer revision is still allowed as it isn't ambiguous.

tag: do not pass binary nullid to scmutil.revsingle() Future patches will remove the old-style parser that happen to accept a binary nodeid. A binary nodeid shouldn't be passed to scmutil.revrange() because it is ambiguous. For example, bin('20' * 19 + '30') is valid binary nodeid, but it can also be parsed as a revset expression, '0'.

revset: port parsing rule of old-style ranges from scmutil.revrange() The old-style parser will be removed soon.

debugrevspec: pass lookup function to visualize fallback mechanism The next patch will move the exceptional parsing of old-style ranges to revset.tokenize(). This patch will allow us to see the result tree. Note that the parsing result of '-a-b-c-' is incorrect at this changeset. It will be fixed soon.

help: fix typo familar -> familiar

highlight: exit early on textual and unknown files (issue3005) When highlight extension encountered files that pygments didn't recognize, it used to fall back to text lexer. Also, pygments uses TextLexer for .txt files. This lexer is noop by design. On bigger files, however, doing the noop highlighting resulted in noticeable extra CPU work and memory usage: to show a 1 MB text file, hgweb required about 0.7s more (on top of ~3.8s, Q8400) and consumed about 100 MB of RAM more (on top of ~150 MB). Let's just exit the function when it's clear that nothing will be highlighted. Due to how this pygmentize function works (it modifies the template in-place), we can just return from it and everything else will work as if highlight extension wasn't enabled.

histedit: extract a simpler function to process replacement on abort The process replacement is building a full mapping to allow moving bookmarks and creating obsolescence marker. We do not need the full logic for abort so we extract it. It will be useful as abort is missing some data about the replacement and can crash when third party extensions push it a bit too far.

merge with stable

exchange: fix dead assignment The assignment of the value from bundle2.processbundle() to 'r' is unused. It is currently the same as its third argument (if given), and since that argument may eventually go away (according to the method's docstring), let's reassign the return value to 'op' instead to better prepare for that.

exchange: s/phase/bookmark/ in _pushb2bookmarks()

histedit: backout ebb5bb9bc32e The faulty changeset use obsolescence marker to roll the repository back on --abort. This is a problematic approach because --abort should be as close as an actually transaction rollback as possible stripping all created data from the repository (cf `hg rebase --abort` stripping all created changesets). Instead ebb5bb9bc32e made all content created during the aborted histedit still available in the repository adding obsolescence marker to make them hidden. This will cause trouble to evolution user as a re-run of the same histedit (with success) will likely result in the very same node to be "recreated" while obsolescence marker would be in place for them. And canceling an obsoletion is still a fairly complicated process. This also rollback using obsmarkers instead of strip to clean up temporary node on successful histedit run because the two change were not split in separated changeset. Rolling that part back does not have significant consequence a will have to be resubmitted independently

test-bookmarks.t: avoid nested repo This is (a) pretty unnecessary and (b) breaks tests for the third-party hgwatchman extension, which doesn't support nested repos.

revlog: remove unused shaoffset constants Call sites were removed at 61c9bc3da402, "revlog: remove lazy index".

revlog: correct comment about size of v0 index format

merge with stable

Added signature for changeset 21aa1c313b05

Added tag 3.5 for changeset 21aa1c313b05

i18n: fix unclosed inline span in pt_BR This was causing test-gendoc.t to complain: WARNING: Inline interpreted text or phrase reference start-string without end-string.

merge with i18n

i18n-ja: synchronized with 7fcad0c4ef8c

convert: when converting from Perforce use original local encoding by default On Windows Perforce command line client uses default system locale to encode output. Using 'latin_1' causes locale-specific characters to be replaced with question marks. With this patch we will use default locale by default whilst allowing to specify it explicity with 'convert.p4.encoding' config option. This is a potentially breaking change for any scripts relying on output treated as in 'latin_1' encoding. Also because hgext.convert.convcmd overwrites detected default system locale with UTF-8 we had to introduce an import cycle in hgext.convert.p4 to retrieve originally detected encoding from hgext.convert.convcmd.

i18n-pt_BR: synchronized with 3e84f40232c7

convert: when getting file from Perforce concatenate data at the end As it turned out, even when getting relatively small files, concatenating string data every time when new chunk is received is very inefficient. Maintaining a string list of data chunks and concatenating everything in one go at the end seems much more efficient - in my testing it made getting 40 MB file 7 times faster, whilst converting of a particularly big changelist with some big files went down from 20 hours to 3 hours.

help: scripting help topic There are a lot of non-human consumers of Mercurial. And the challenges and considerations for machines consuming Mercurial is significantly different from what humans face. I think there are enough special considerations around how machines consume Mercurial that a dedicated help topic is warranted. I concede the audience for this topic is probably small compared to the general audience. However, lots of normal Mercurial users do things like create one-off shell scripts for common workflows that I think this is useful enough to be in the install (as opposed to, say, a wiki page - which most users will likely never find). This text is by no means perfect. But you have to start somewhere. I think I did cover the important parts, though.

convert: document convert.hg.startrev

transplant: restore dirstate correctly at unexpected failure Before this patch, transplant can't restore dirstate as expected at failure other than one while patching. This causes: - unexpected file status - dirstate refers already rollback-ed parent (only at failure of transplanting the 2nd or later revision) To restore dirstate correctly also at unexpected failure, this patch encloses scope of store lock and transaction by 'dirstateguard'. This is temporary fixing for stable branch. See DirstateTransactionPlan wiki page for detail about the future plan to treat dirstate consistently around scope boundary of transaction. https://mercurial.selenic.com/wiki/DirstateTransactionPlan This patch also adds 'if lock' examination for safety 'lock.release()', because creating 'dirstateguard' object may fail unexpectedly (e.g. IOError for saving dirstate). BTW, in the test script, putting section header '[extensions]' into '.hg/hgrc' is needed to fix incomplete disabling 'abort' extension at 4d1382fd96ff.

localrepo: make journal.dirstate contain in-memory changes before transaction Before this patch, in-memory dirstate changes aren't written out at opening transaction, even though 'journal.dirstate' is created directly from '.hg/dirstate'. Therefore, subsequent 'hg rollback' uses incomplete 'undo.dirstate' to restore dirstate, if dirstate is changed and isn't written out before opening transaction. In cases below, the condition "dirstate is changed and isn't written out before opening transaction" isn't satisfied and this problem doesn't appear: - "wlock scope" and "transaction scope" are almost equivalent e.g. 'commit --amend', 'import' and so on - dirstate changes are written out before opening transaction e.g. 'rebase' (via 'dirstateguard') and 'commit -A' (by separated wlock scopes) On the other hand, 'backout' may satisfy the condition above. To make 'journal.dirstate' contain in-memory changes before opening transaction, this patch explicitly invokes 'dirstate.write()' in 'localrepository.transaction()'. 'dirstate.write()' is placed before not "writing journal files out" but "invoking pretxnopen hooks" for visibility of dirstate changes to external hook processes. BTW, in the test script, 'touch -t 200001010000' and 'hg status' are invoked to make file 'c' surely clean in dirstate, because "clean but unsure" files indirectly cause 'dirstate.write()' at 'repo.status()' in 'repo.commit()' (see fe03f522dda9 for detail) and prevents from certainly reproducing the issue.

dirstate: ensure mv source is marked deleted when walking icasefs (issue4760) Previously, importing a case-only rename patch on a case insensitive filesystem caused the original file to be marked as '!' in status. The source was being forgotten properly in patch.workingbackend.close(), but the call it makes to scmutil.marktouched() then put the file back into the 'n' state (but it was still missing from the filesystem). The cause of this was scmutil._interestingfiles() would walk dirstate, and since dirstate was able to lstat() the old file via the new name, was treating this as a forgotten file, not a removed file. scmutil.marktouched() re-adds forgotten files, so dirstate got out of sync with the filesystem. This could be handled with less code in the "kind == regkind or kind == lnkkind" branch of dirstate._walkexplicit(), but this avoids filesystem accesses unless case collisions occur. _discoverpath() is used instead of normalize(), since the dirstate case is given first precedence, and the old file is still in it. What matters is the actual case in the filesystem.

extdiff: allow modifications in subrepos to be copied back This check was a legacy bit from when the file data was being fetched manually with 'ctx[wfn]', but archive() does that now. 49966b5ab16f seems to indicate that this avoided a problem where a merge adds a file to another branch, and that test still passes. Unfortunately, I don't see a way to create a test that modifies the file in the temporary directory before the command exits. I wonder if the os.lstat() call needs to be wrapped in an exception handler for the case where archive didn't create a file because the file didn't exist in that revision. But I wasn't able to trigger a problem without it on a real repository.

ignore: fix path concatenation of .hgignore on Windows Since 3de48ff62733, .hgignore is ignored on Windows because a pat may have a drive letter, but pathutil.join is posixpath.join. "z:\foo\bar/z:\foo\bar\.hgignore" Instead, this patch uses os.path.join() and util.localpath() to process both parts as file-system paths. Maybe we can remove os.path.join() at dirstate._ignore because 'include:' is resolved relative to repo root? It was introduced by a04c7b74b3d5.

repair: fix typo in warning message

monoblue: link to branches on summary page "Tags" and "Bookmarks" sections on this page already have the similar links, and so does the "Branches" on summary page in gitweb, so let's do the same in monoblue.

monoblue: fix branches link on help topics page

hgweb: show ages in repos' Last modified column in monoblue and gitweb Index page, which shows the list of available repositories, has a column where the last modification date for each repo is shown. paper, coal and spartan already show the dates in relative format (e.g. "2 weeks ago"), because these styles have the required process_dates() js function call in their footer templates, which are included on every page. But monoblue and gitweb styles have more things in the footer templates, such as repo name and its atom/rss links, so they don't include the footer on index page (as this page doesn't have a single repo context). Let's call process_dates() without including the footer.

ignore: fix include: rules depending on current directory (issue4759) When reading pattern files, we just call open(path), which is relative to the current directory. Let's fix this by resolving the paths before attempting to read the file.

test: move ignore test run into a subdirectory Previously the hgignore test just called hg init in the test directory. A future patch needs to test hgignore stuff from outside of the repo, so let's move the entire test repo into a subdirectory.

monoblue: show relative age of bookmarks In b24e5a708fad, age calculation was made dynamic (i.e. in javascript), but for some reason bookmarkentry template in monoblue/map got a wrong class. It resulted in /summary and /bookmarks pages always showing exact dates for bookmarks, no age calculation was performed. Let's fix this by using "age" class that is already used in branchentry and tagentry templates in the same map file. As usual, the exact date for such elements is still available in title attribute, so it shows in a tooltip on hover.

highlight: produce correct markup when there's a blank line just before EOF Due to how the colorized output from pygments was stripped of <pre> elements, when there was an empty line at the end of a file, highlight extension produced an incorrect markup (no closing tags from the fileline/annotateline template). It wasn't usually noticeable, because browsers were smart enough to see where the missing tags should've been, but in monoblue style it resulted in the last line having twice the normal height. Instead of awkwardly trying to strip outer <pre></pre> tags, let's make the formatter with nowrap=True, which should do what we need in pygments since at least 0.5 (2006-10-30). Example from monoblue style: Before: <div class="source"> <div style="font-family:monospace" class="parity0"> <pre><a class="linenr" href="#l1" id="l1"> 1</a> </pre> </div> <div style="font-family:monospace" class="parity1"> <pre><a class="linenr" href="#l2" id="l2"> 2</a> </div> Now: <div class="source"> <div style="font-family:monospace" class="parity0"> <pre><a class="linenr" href="#l1" id="l1"> 1</a> </pre> </div> <div style="font-family:monospace" class="parity1"> <pre><a class="linenr" href="#l2" id="l2"> 2</a> </pre> </div> </div> (Notice the missing </pre></div> now in place)

test-check-config: convert directory separators to '/' for MSYS The output of the files command uses native separator. MSYS then seems to drop the '\' on Windows when invoking python: --- c:/Users/Matt/Projects/hg/tests/test-check-config-hg.t +++ c:/Users/Matt/Projects/hg/tests/test-check-config-hg.t.err @@ -6,22 +6,10 @@ $ hg files "set:(**.py or **.txt) - tests/**" | > xargs python contrib/check-config.py Traceback (most recent call last): File "contrib/check-config.py", line 93, in <module> sys.exit(main(sys.argv[1:])) File "contrib/check-config.py", line 24, in main for l in open(f): IOError: [Errno 2] No such file or directory: 'contriball-revsets.txt' [123]

test-subrepo: fix globs for Windows It looks like these changes originated with 2538b87660be. I'm not sure that it was intentional, but is seems harmless enough for an error message.

test-clone: fix globs for Windows

tests: diff -r on Solaris emits "Common subdirectories:" When using the -r option to Solaris diff, any directores that compare identically are mentioned in the output. We don't really care about these directories for the purposes of this test, so ignore them.

tests: work around differing hunk headers between GNU and Solaris diff The hunk headers specifying what lines the hunk apply to differ. They're irrelevant to the test, so just eliminate them from the output.

tests: use egrep with regular expression branches for compatibility GNU grep allows you to use "a\|b" in a regular expression to match either "a" or "b", but at least Solaris grep does not; only egrep allows for that. And egrep considers "a+" to be "a{1,}" instead of an "a" and a literal plus sign, so escape that as well.

parsers: silence warning of implicit integer conversion issued by clang "-Wshorten-64-to-32" is enabled by default on Mac OS X. Because "len" should be represented in 32bit integer, this patch simply cast ssize_t to int.

tests: disable test of buffer overflow in parsers.c if --pure It fails with AttributeError and there's no benefit to make it runnable with pure Python code.

win32: update Inno Setup script after the changes done in 95e042d77a5f

Added signature for changeset 96a38d44ba09

Added tag 3.5-rc for changeset 96a38d44ba09

merge with default for code freeze

tests: add a check-config pass

hgweb: document web.certificates option

shelve: make maxbackup doc check-config friendly

share: make option docs more check-config friendly

perf: mark experimental option presleep

check-config: don't continue prematurely The early check for section headers like ^``foo`` was missing actual options like ^``foo.bar``.

subrepo: mark internal-only option

progress: mark experimental option

censor: mark experimental option

bundle2: fix type of experimental option

merge: make merge.preferancestor type and default consistent (and mark it)

merge: mark ancient debugging option

email: fix config default value inconsistency

bookmarks: mark internal-only option

commit: mark internal-only option

localrepo: mark format options

formatter: mark developer options

http2: mark experimental and developer options

bookmarks: mark internal-only config option

filemerge: mark internal-only config option

profiler: mark developer-only config option

debugger: mark developer-only option (and rearrange comment)

commandserver: mark developer-only logging option

generaldelta: mark experimental reordering option

bundlerepo: mark internal-only config variable

win32text: mark deprecated extension option deprecated

transplant: mark some undocumented options deprecated

mq: tweak config reading to make check-config happy The not-really-a-bool handling here upsets the type checker.

patchbomb: make sure all users of smtp.verifycert agree on the default

patchbomb: mark ancient option deprecated This just exists for backwards compatibility with the earliest versions of patchbomb.

histedit: mark defaultrev option experimental

changegroup: compute seen files as changesets are added (issue4750) Before this patch, addchangegroup() would walk the changelog and compute the set of seen files between applying changesets and applying manifests. When cloning large repositories such as mozilla-central, this consumed a non-trivial amount of time. On my MBP, this walk takes ~10s. On a dainty EC2 instance, this was measured to take ~125s! On the latter machine, this delay was enough for the Mercurial server to disconnect the client, thinking it had timed out, thus causing a clone to abort. This patch enables the changelog to compute the set of changed files as new revisions are added. By doing so, we: * avoid a potentially heavy computation between changelog and manifest processing by spreading the computation across all changelog additions * avoid extra reads from the changelog by operating on the data as it is added The downside of this is that the add revision callback does result in extra I/O. Before, we would perform a flush (and subsequent read to construct the full revision) when new delta chains were created. For changelogs, this is typically every 2-4 revisions. Using the callback guarantees there will be a flush after every added revision *and* an open + read of the changelog to obtain the full revision in order to read the added files. So, this increases the frequency of these operations by the average chain length. In the future, the revlog should be smart enough to know how to read revisions that haven't been flushed yet, thus eliminating this extra I/O. On my MBP, the total CPU times for an `hg unbundle` with a local mozilla-central gzip bundle containing 251,934 changesets and 211,065 files did not have a statistically significant change with this patch, holding steady around 360s. So, the increased revlog flushing did not have an effect. With this patch, there is no longer a visible pause between applying changeset and manifest data. Before, it sure felt like Mercurial was lethargic making this transition. Now, the transition is nearly instantaneous, giving the impression that Mercurial is faster. Of course, eliminating this pause means that the potential for network disconnect due to channel inactivity during the changelog walk is eliminated as well. And that is the impetus behind this change.

revlog: add support for a callback whenever revisions are added A subsequent patch will add a feature that performs iterative computation as changesets are added from a changegroup. To facilitate this type of processing in a generic manner, we add a mechanism for calling a function whenever a revision is added via revlog.addgroup(). There are potential performance concerns with this callback, as using it will flush the revlog after every revision is added.