Merge crew and main.

merge: don't fiddle with name lookups or i18n in hot loops We perform attribute dereferences and i18n lookups before looping.

merge: apply non-interactive working dir updates in parallel This has a big effect on the performance of working dir updates. Here are the results of update from null to the given rev in several repos, on a Linux 3.2 system with 32 cores running ext4, with the progress extension enabled. repo rev plain parallel speedup hg 7068089c95a2 0.9 0.3 3 mozilla-central fe1600b22c77 42.8 7.7 5.5 linux-2.6 9ef4b770e069 31.4 4.9 6.4

worker: allow a function to be run in multiple worker processes If we estimate that it will be worth the cost, we run the function in multiple processes. Otherwise, we run it in-process. Children report progress to the parent through a pipe. Not yet implemented on Windows.

worker: partition a list (of tasks) into equal-sized chunks

worker: estimate whether it's worth running a task in parallel Not implemented for Windows yet.

worker: count the number of CPUs This works on the major platforms, and falls back to a safe guess of 1 elsewhere.

tests: getremove test output changes (fold into previous patch)

merge: report non-interactive progress in chunks Instead of a monotonic count, getupdates yields the number of files it has updated since it last reported, and its caller sums the numbers when updating progress. Once we run these updates in parallel, this will allow worker processes to report progress less often, reducing overhead.

merge: handle subrepo merges and .hgsubstate specially In an upcoming patch, we will update .hgsubstate in a non-interactive worker process. Merges of subrepo contents will still need to occur in the master process (since they may be interactive), so we move that code into a place where it will always run in what will become the master process.

tests: update test output (will be folded into parent)

merge: split out mostly-non-interactive working dir updates In a later patch, we'll run these updates in parallel.

extensions: obsolete and remove interhg extension With the addition of the websub filter extension this extension is no longer needed. We maintain a sort of backwards compatibility by reading the [interhg] section and using it as we would use the [websub] section.

hgweb: apply the websub filter to revision descriptions In order to use this, add a [websub] section to your configuration and add websub expressions such as: italic = s/\b_(\S+)_\b/<i>\1<\/i>/ bold = s/\*\b(\S+)\b\*/<b>\1<\/b>/ issues = s|issue(\d+)|<a href="http://bts.example.org/issue\1">issue\1</a>|i bugzilla = s!((?:bug|b=|(?=#?\d{4,}))(?:\s*#?)(\d+))!<a href="http://bz.selenic.com/\2">\1</a>!i This also adds documentation (proofed by Kevin!) to the config help section.

hgweb: add websub template filter The purpose of this new filter is to make it possible to partially replace the functionality of the interhg extension. The idea is to be able to define regular expression based substitutions on a new "websub" config section. hgweb will then be able to apply these substitutions wherever the "websub" filter is used on a template. This first revision just adds the code necessary to load the websub expressions and adds the websub filter, but it does not add any calls to the websub filter itself on any of the templates. That will be done on the following revisions.

addremove: don't audit the path for paths already in the dirstate Now that dirstate.walk returns None for paths under symlink directories, addremove doesn't need to validate each path it sees to look for files under symlinks. On a large repository this brings addremove from 6.3 seconds down to 3.65 (42%) since addremove no longer has to stat every directory of every file to determine if the file is inside a symlink directory. I put it through our benchmark and see no perf hit to any other commands.

dirstate: walk returns None for files that have a symlink in their path Previously dirstate.walk would return a stat object for files in the dmap that have a symlink to a directory in their path. Now it will return None to indicate that they are no longer considered part of the repository. This currently only affects walks that traverse the entire directory tree (ex: hg status) and not walks that only list the contents of the dmap (ex: hg diff). In a situation like this: mkdir foo && touch foo/a && hg commit -Am "a" mv foo bar ln -s bar foo 'hg status' will now show '! foo/a', whereas before it incorrectly considered 'foo/a' to be unchanged. In addition to making 'hg status' report the correct information, this will allow callers to dirstate.walk to not have to detect symlinks themselves, which can be very expensive.

pathauditor: add check() method The pathauditor currently throws exceptions when it encounters an invalid path. This change adds a method to allow people to treat it as a boolean. This is currently used by scmutil.addremove and in a subsequent patch it will be used by dirstate.walk

summary: add missing space for updated active bookmark display

summary: show active bookmark even if not at current changeset Before this change, 'hg summary' would not show the active bookmark unless it pointed to the working directory parent. After this change, it will show it in parentheses, like so: parent: 18581:f0ff45fe6700 tip summary: simplify handling of active bookmark branch: default bookmarks: [crew] commit: (clean) update: (current)

summary: simplify handling of active bookmark

summary: test that current bookmark isn't shown This exposes the current behavior in a test. A later change will make summary show when the active bookmark has moved out from under us.

merge crew heads

merge with stable

outgoing: fix possible filtering crash in outgoing (issue3814) If there is no outgoiing changesets but we have filtered revision in outgoing.excluded We run into a filtering related crash. The excluded revision should not be there in the first place but discovery need cleanup in default, not stable.

test: display used python hash seed We keep using a random seed for each run, but we "compute" it ourself to be able to reproduce a failed test run.

outgoing: fix possible filtering crash in outgoing (issue3814) If there is no outgoiing changesets but we have filtered revision in outgoing.excluded We run into a filtering related crash. The excluded revision should not be there in the first place but discovery need cleanup in default, not stable.

parsedate: understand "now" as a shortcut for the current time

export: clobber files with -o (bc) (issue3652) This violated user expectation. Updated the code to clobber files, but preserve the behavior of appending multiple patches requested in a single export. Includes tests.

merge: don't call copies.mergecopies unless we need to This reduces the amount of time we spend calculating when doing a clean non-merge update. In a large repo, the time dropped from 10.1 seconds to 3.4.

merge: rename p1 to wctx in manifestmerge This is always a workingctx, and this name is more in line with other functions in this module.

largefiles: fix test and check code

histedit: always abort with same message when working dir is wrong. Using only one message makes live easier for translators. The clearer error message was supposed by Augie Fackler.

histedit: report when revisions to edit are not ancestors of working dir Editing the history only is possible when the working dir is a descendant of the revisions to edit. When this happens explain it by writing abort: %s is not an ancestor of working directory

Merge

manifestmerge: handle abort on local unknown, remote created files This replaces the _checkunknown call in calculateupdates with a more performant version. On a repository with over 150,000 files, this speeds up an update by 0.6-0.8 seconds, which is up to 25%. This does not introduce any UI changes. There is existing test coverage for every case, mostly in test-merge*.t.

manifestmerge: pass in branchmerge and force separately This will be used in an upcoming patch.

manifest: use a size 3 LRU cache to store parsed manifests Previously, the manifest cache would store the last manifest parsed. We could run into situations with operations like update where we would try parsing the manifest for a revision r1, then r2, then r1 again. This increases the cache size to 3 to avoid that bit of performance fragility.

util: add an LRU cache dict In certain cases we would like to have a cache of the last N results of a given computation, where N is small. This will be used in an upcoming patch to increase the size of the manifest cache from 1 to 3.

tests: remove last two check-code warnings about killdaemons Removes the last of the warnings in test-check-code-hg.t introduced in 667063b22a69.

tests: guard against obsolete markers in the hg repo itself If obsolete markers appear in the hg repo (because of enabling evolve), then tests that run hg against the hg repo itself will see warnings like: obsolete feature not enabled but 4 markers found! As far as I can tell, this only occurs in test-check-code-hg.t -- in particular, it will -not- show up on tests that run against test-created repos, as most of the test suite does.

largefiles: make caching largefiles message more explicit In some cases, caching largefiles may take a long time (if the user has pulled a lot of new heads). This patch makes it more clear what is happening, by showing the number of heads we are caching largefiles for.

largefiles: document behavior of caching largefiles for new heads

check-code: add Makefile target to run check-code

byterange: remove now-unused sys import

test-obsolete.t: migrate to killdaemons from kill `cat pidfile`

test-inotify.t: migrate to killdaemons from kill `cat pidfile`

test-inotify-lookup.t: migrate to killdaemons from kill `cat pidfile`

test-inotify-issue1556.t: migrate to killdaemons from kill `cat pidfile`

test-inotify-issue1542.t: migrate to killdaemons from kill `cat pidfile`

test-inotify-issue1371.t: switch to killdaemons from kill `cat pidfile`

test-inotify-debuginotify.t: migrate to killdaemons from kill `cat pidfile`

test-hgweb-raw.t: use killdaemons instead of kill `cat pidfile`

test-https.t: stop using kill `cat $pidfile`

changelog: hexlify node when throwing a LookupError on a filtered node The non-hexlified node was leaking all the way out to the web interface, and wasn't consistent with the behavior for nonexistent nodes.

byterange: remove old two-arg raise trick We don't care about the original traceback, and this was frustrating any efforts to make the code work on Python 3.3.

revlog: document v0 format

merge crew heads

merge with mpm stable

templater: add get() function to access dict element (e.g. extra)

templates: export extra as a dict to templates Currently only useful with the json filter.

Added signature for changeset 7511d4df752e

Added tag 2.5.1 for changeset 7511d4df752e

merge with i18n

i18n-de: fix duplicated shortcut key b for record (edit and quit) Use e for edit instead.

check-code: fix platform-specific error code variance test-check-code-hg.t uses xargs to invoke check-code.py on every file in 'hg manifest'. The return code from xargs varies between BSD xargs and GNU xargs: BSD will return 1 if any invocation exits with an error code; GNU xargs will return 123 in this case. This normalizes the exit code back to 1.

check-code: warn to use killdaemons instead of kill `cat PIDFILE` We have a bunch of tests that still use kill `cat hg.pid` or worse, kill `cat hg.pid`; while kill -0 `cat hg.pid`; sleep 0; done Cleaning these up to use tests/killdaemons.py is non-trivial, so for now we just add a warning.

largefiles: docstrings for verify methods

largefiles: fold oddly named _verify into remotestore.exists

git convert: some versions of git use fatal: instead of error: I saw this behavior with git 1.7.12 on my Mac.

test-convert-git: stabilize corruption of test git repo This error would show up only intermittently since the test depended on the order of the directories returned by os.walk. The damage repository test would delete the first object file it came across. However, the order of the directory listing is arbitrary (it seems to depend on the filesystem). This meant that sometimes a commit object was deleted, sometimes a blob object and sometimes a tree object. So, fix by hardcoding which object to delete. Delete a commit object, a blob object and a tree object in three separate tests.

convert/git: catch errors from modern git-ls-remote (issue3428) Since git v1.7.8.2-327-g926f1dd (the change was first released in git 1.7.10), git does not return non-zero when "git ls-remote --tags ..." is run and the repository is damaged. This causes the "damaged repository with missing commit" test in test-convert-git.t to unexpectedly succeed. Fix by aborting if git outputs any lines beginning with "error:", which required adding some subprocess use in convert/git.py.

merge with stable

incoming: fix incoming when a local head is remotely filtered (issue3805) In its current state discovery may return (remotely) filtered elements in "common". This has usually no impact as "missing" is kept clear of filtered elements. However when the "remote" repo is a local repo (disk accessible, and directly created in memory) the incoming code takes a shortcut and directly uses the "remote" repo to generate the incoming output. When some common elements are filtered this led to a crash. We now ensure we use an unfiltered repository to generate the incoming output. This does not change the behavior as missing is clear of filtered revision. Now that we have proper low level filtering, incoming code needs a deeper cleanup but it is already planned.

dirstate: move pure python dirstate packing to pure/parsers.py

bookmark: don't allow integers as bookmark/branch/tag names Bookmarks/branches/tags shouldn't be allowed to be integers because that overlaps with revision numbers. Right now if a user created one they can't use it anyway because the revision numbers take precedence. The check only happens when creating a new bookmark/etc from a command so it shouldn't affect existing bookmarks/branches/tags or importing branches from git. This fix was prompted by us having a user create a bookmark named "404" then accidentally checkout a very old version of our repository.

run-tests: do not fail on empty tsttest file Initialize n for not failing on empty tsttest files.

merge with stable

hgweb: make 'summary' work with hidden changesets (issue3810) Since the 'summary' view used by e.g. gitweb and monoblue shows both a changelog and a bookmarks list, the same changes are needed here as were made to the 'changelog' and 'bookmarks' web commands (56ca4443a343 and 886936ecc21b, respectively).

merge with stable

hgk: support the old way of getting the current Ttk theme (issue3808) It seems like the API has changed somewhere around 8.5.7, so the preferred way of getting the current theme is now [ttk::style theme use], while the deprecated (but still working) is $::ttk::currentTheme.

addremove: don't perform m.exact/rel until needed Moves the m.exact and m.rel calls within the conditionals they are used in. On a large repo this brings addremove from 7.1 seconds down to 6.3 (13%).

addremove: don't call lexists, isdir, and islink The dirstate walk results contain the stat information for each path, so we don't need to query it again. On a large repo this makes addremove go from 8.35 seconds to 7.1 (15%).

addremove: only query dirstate once per path Previously the addremove code queried the dirstate 4 times per path. Now it only does so once. On a large repo this brings addremove from 9.5 seconds to 8.35 seconds (12%).

tests: fix windows test failure with largefiles

merge with stable

tests: fix windows test failure in test-subrepo.t

merge with stable

largefiles: don't crash when trying to find default dest for url without path b32e55e6c3c7 introduced a crash when cloning a url without path - where util.url().path would be None. This None will now be handled as ''. clone will thus abort with 'repository / not found' as before.

hgweb.cgi: fix internal WSGI emulation (issue3804) The internal WSGI emulation in wsgicgi.py was not fully WSGI compliant and assumed that all responses sent a body. With 3fbdbeab38cc that caused a real bug when using hgweb.cgi. wsgicgi.py will now make sure headers always are sent, using the pattern from PEP 333 and similar to how it is done in c007e5c54b16.

bdiff: simplify overflow checks Rather than check that each delta start, end, and length is within 32 bits, we simply check that the input strings are under 4GB.

Merge crew-stable into crew

rebase: derive node from target rev (issue3802) dest.rev() is the same as target when a new rebase is run, but dest isn't set when rebase --continue is run. Bug introduced in 2a1fac3650a5, which fixed issue3685.

profiling: introduce limit configuration option The number of output lines was hardcoded to 30. There was a 'nested' configuration options that controlled something else related to counting the number of output lines. This introduces the profiling.limit configuration option for controlling the number of profiling output to show.

largefiles: make verify --lfa and --lfc work without --large The slightly obscure --lfa and --lfc only worked as modifiers to --large and could be combined. The documentation was however not clear what they did. Instead they now imply --large and the description is updated.

largefiles: verify status should be written as status, not as write Make 'hg verify --large -q' quiet when no errors.

largefiles: report localstore errors with single line warnings messages The actual hash is also not relevant to report. This makes verify output with multiple failures easier to process.

merge: inline act() The act function had become very trivial and mainly shuffled arguments around and made it harder to see what really was going on.

merge: don't indent "local changed %s which remote deleted" prompt It was usually not shown in a context where indentation helped readability and it was inconsistent with other prompts.

merge: backout manifest sorting and workarounds for cost of it Backout 8a811fa9a9c0 and 760c0d67ce5e which no longer are needed.

merge: delay debug messages for merge actions Show messages at a point where the actions have been sorted, thus preparing for backout of 760c0d67ce5e. This makes manifestmerge more of a silent operation, just like 'copies' is. Indent 'preserving' messages to make them subordinate to the action logging so they fit in the new context. (The 'preserving' messages are quite redundant and could also be removed completely.)

merge: make all action tuples have the same length - keep args as tuple

merge: delay prompts a bit and show them in (extra) sorted order Preparing for backout of 760c0d67ce5e. The number of prompts will for all relevant cases be significantly smaller than the total number of files in the manifests. We can thus afford to sort the prompts more than we can afford to sort the manifests.

commit: show active bookmark in commit editor helper text If there is an active bookmark while committing, the bookmark name will be visible inside the commit message helper, below the branch name. This should make easier for the user to detect a mistaken commit parent, while working for example with a bookmark centric workflow like topic branches. The active bookmark is checked to be in the working directory, as pointed by Kevin Bullock, because otherwise committing would not advance it. In other words, this would not show the active bookmark name if the user changed the working tree parents with 'hg debugsetparents', for example.

dates: support 'today' and 'yesterday' in parsedate (issue3764) Adding support to parsedate in util module to understand the more idiomatic dates 'today' and 'yesterday'. Added unified tests and docstring tests for added functionality.

revset: change ancestor to accept 0 or more arguments (issue3750) Change ancestor to accept 0 or more arguments. The greatest common ancestor of a single changeset is that changeset. If passed no arguments, the empty list is returned.

merge with stable

Added signature for changeset a6088c05e43a

Added tag 2.5 for changeset a6088c05e43a

test-obsolete: now gets 404 for hidden change

merge with i18n

i18n-pt_BR: added message from crew 0324a1d88a53

i18n-pt_BR: added message from crew 9fbeb61b8ad2

i18n-ja: synchronized with 2a1fac3650a5

i18n-pt_BR: synchronized with 68eecbaf1bd3

hgweb: urlescape all urls, HTML escape repo/tag/branch/... names Without this, repository paths or names containing e.g. & characters or html tags yielded strange results, possibly allowing cross-site scripting attacks.

merge with crew

hgweb: rename 'currentbaseline' template keyword to 'basenode' Shorter and clearer. This keyword represents the node we're currently diffing against.

hgweb: rename 'changesetbaseline' template to 'difffrom' More accurately reflects what it will be used for, and is also shorter. This template is used to change which rev the current rev is diff'd against. For example, if you're at '/rev/P1:REV', this would link to a path like '/rev/P2:REV'. Example usage in a template: {parent%difffrom}

hgweb: add a `web.view` to control filtering This options add a new `web.view` to control filter level of hgweb. This option have two purposes: 1) Allow fall back to unfiltered version in case a yet undetected by critical bug is found in filtering after 2.5 release 2) People use hgweb as a local repoviewer. When they have secret changesets, they wants to use "visible" filter not "served" (modified by mpm, documentation deferred)