util: remove no-op assignment Found by Cython.

merge more evolve stuff away (what?)

synthrepo: add missing import of sys Found using Cython.

transplant: pass source through to recover

repoview: remove more unreachable code Found using Cython.

merge away some failed evolve fat-fingering

pager: add missing imports of errno and shlex Found using Cython.

largefiles: quiet (and document) undefined name errors (issue3886) These names were found using Cython; I was completely puzzled until I searched the rest of the tree. It's icky to mess with another module's namespace, but ickier yet to do so without a comment :-)

color: add missing import of error Found using Cython.

pvec: use the correct name for an identifier Found using Cython.

repoview: remove unreachable code Found using Cython.

pvec: use the correct name for an identifier Found using Cython.

repoview: remove unreachable code Found using Cython.

mail: add missing import of sys Found using Cython.

statichttprepo: add missing import of os Found using Cython.

worker: add missing import of errno Found using Cython.

histedit: support editing of the first commit (issue3767)

test-commit-amend.t: fix check-code violation from 3a72c89a83ec

util: improve doc for checkcase

scheme: don't crash on invalid URLs

amend: support amending merge changesets (issue3778)

worker: catch all exceptions, try to exit usefully/safely

lock: if we fork, ensure that only the parent releases This prevents us from having a bunch of errant worker processes all try to release a lock if a problem occurs. (Releasing the lock more than once is harmless; it's invoking the associated callbacks we want to avoid.)

tag: clarify cryptic error message when tagging null revision

setup: make error message for missing Python headers more helpful

obsolete: ensure all markers have a date Obsolescence creates a sparse DAG mostly composed of a lot of small independent chain of markers. Date is the only simple and "reliable" way to sort them. The existence of a date is now enforced at creation time as I'm more and more convinced that date will have a key role in obsolescence markers exchange.

zsh_completion: make use of `debuglabelcomplete` command Use `debuglabelcomplete` command when populating labels list, instead of calling `hg` three times: for branches, bookmarks and tags. Do not pass string being completed to `hg debuglabelcomplete` (as `$words[$CURRENT]`), since it breaks `_hg_revrange` completion (`--rev 2.5:2.5.<Tab>`) for no apparent benefit. Also complete `hg view` with labels, not just tags, and drop unused `_hg_tags`.

dirs: use mutable strings internally perfdirs results for a working dir with 170,000 files: Python 638 msec C 244 C+int 192 C+int+str 168 In the large repo above, the nearly 0.5 second time improvement is visible in commands like "hg add" and "hg update". hg add Python 1100 msec C+int+str 600 hg update (with nothing to do) Python 2800 msec C+int+str 2240

dirs: use mutable integers internally These integers are not visible to Python code, so this is safe. perfdirs results for a working dir with 170,000 files: Python 638 msec C 244 C+int 192

scmutil: rewrite dirs in C, use if available This is over twice as fast as the Python dirs code. Upcoming changes will nearly double its speed again. perfdirs results for a working dir with 170,000 files: Python 638 msec C 244

scmutil: use new dirs class in dirstate and context The multiset-of-directories code was open coded in each of these modules; this change gets rid of the duplication.

scmutil: add a dirs class This encapsulates the "multiset of directories" structures that are currently open-coded (and duplicated) in both the dirstate and context modules. This will be used, and optionally replaced by a C implementation, in upcoming changes.

scmutil: migrate finddirs from dirstate

merge

manifestmerge: handle workdir removed, remote removed with flags This can happen when a file with flags is removed or deleted in the working directory and also not present in m2. The obvious solution is to add a __delitem__ override to manifestdict that removes the file from flags if necessary, but that has a significant performance cost in some cases, e.g. hg status --rev rev1 --rev rev2 <file>.

dicthelpers.diff: compare against default for missing values This is not only a bit faster, but also aligns with callers' expectations better since we can legitimately have manifestdict's _flags set to '' instead of unset. hg perfmergecalculate -r . before: ! wall 0.139582 comb 0.140000 user 0.140000 sys 0.000000 (best of 59) after: ! wall 0.126154 comb 0.120000 user 0.120000 sys 0.000000 (best of 74) hg perfmergecalculate -r .^ before: ! wall 0.236333 comb 0.240000 user 0.240000 sys 0.000000 (best of 36) after: ! wall 0.212265 comb 0.210000 user 0.210000 sys 0.000000 (best of 45)

templater: back out 0615b22da148, it breaks schemes ({1})

debuglabelcomplete: compute active branch heads correctly The previous computation was simply wrong.

merge with i18n Note: i18n work should normally be done on stable

hg-i18n-it: minor fixes

templater: fix check-code error

smtp: verify the certificate of the SMTP server for STARTTLS/SMTPS Before this patch, the certificate of the SMTP server for STARTTLS or SMTPS isn't verified. This may cause man-in-the-middle security problem (stealing authentication information), even though SMTP channel itself is encrypted by SSL. When "[smtp] tls" is configured as "smtps" or "starttls", this patch: - uses classes introduced by preceding patches instead of "SMTP" or "SMTP_SSL" of smtplib, and - verifies the certificate of the SMTP server, if "[smtp] verifycert" is configured as other than False "[smtp] verifycert" can be configured in 3 levels: - "strict": This verifies peer certificate, and aborts if: - peer certification is not valid, or - no configuration in "[hostfingerprints]" and "[web] cacerts" This is default value of "[smtp] verifycert" for security. - "loose": This verifies peer certificate, and aborts if peer certification is not valid. This just shows warning message ("certificate not verified"), if there is no configuration in "[hostfingerprints]" and "[web] cacerts". This is as same as verification for HTTPS connection. - False(no verification): Peer certificate is not verified. This is as same as the behavior before this patch series. "hg email --insecure" uses "loose" level, and ignores "[web] cacerts" as same as push/pull/etc... with --insecure. Ignoring "[web] cacerts" configuration for "hg email --insecure" is already done in "dispatch._dispatch()" by looking "insecure" up in the table of command options.

sslutil: abort if peer certificate is not verified for secure use Before this patch, "sslutil.validator" may returns successfully, even if peer certificate is not verified because there is no information in "[hostfingerprints]" and "[web] cacerts". To prevent from sending authentication credential to untrustable SMTP server, validation should be aborted if peer certificate is not verified. This patch introduces "strict" optional argument, and "sslutil.validator" will abort if it is True and peer certificate is not verified.

smtp: add the class to verify the certificate of the SMTP server for SMTPS Original "smtplib.SMTP_SSL" has no route to pass "ca_certs" and "cert_reqs" arguments to underlying SSL socket creation. This causes that "getpeercert()" on SSL socket returns empty dict, so the peer certificate for SMTPS can't be verified. This patch introduces the "SMTPS" class derived from "smtplib.SMTP" to pass "ca_certs" and "cert_reqs" arguments to underlying SSL socket creation. "SMTPS" class is derived directly from "smtplib.SMTP", because amount of "smtplib.SMTP_SSL" definition derived from "smtplib.SMTP" is as same as one needed to override it. This patch defines "SMTPS" class, only when "smtplib.SMTP" class has "_get_socket()" method, because this makes using SSL socket instead of normal socket easy. "smtplib.SMTP" class of Python 2.5.x or earlier doesn't have this method. Omitting SMTPS support for them is reasonable, because "smtplib.SMTP_SSL" is already unavailable for them before this patch. Almost all code of "SMTPS" class is imported from "smtplib.SMTP_SSL" of Python 2.7.3, but it differs from original code in point below: - "ssl.wrap_socket()" is replaced by "sslutil.ssl_wrap_socket()" for compatibility between Python versions

smtp: add the class to verify the certificate of the SMTP server for STARTTLS Original "smtplib.SMTP" has no route to pass "ca_certs" and "cert_reqs" arguments to underlying SSL socket creation. This causes that "getpeercert()" on SSL socket returns empty dict, so the peer certificate for STARTTLS can't be verified. This patch introduces the "STARTTLS" class derived from "smtplib.SMTP" to pass "ca_certs" and "cert_reqs" arguments to underlying SSL socket creation. Almost all code of "starttls()" in this class is imported from "smtplib.SMTP" of Python 2.7.3, but it differs from original code in points below: - "self.ehlo_or_helo_if_needed()" invocation is omitted, because: - "ehlo_or_helo_if_needed()" is available with Python 2.6 or later, and - "ehlo()" is explicitly invoked in "mercurial.mail._smtp()" - "if not _have_ssl:" check is omitted, because: - "_have_ssl" is available with Python 2.6 or later, and - same checking is done in "mercurial.sslutil.ssl_wrap_socket()" - "ssl.wrap_socket()" is replaced by "sslutil.ssl_wrap_socket()" for compatibility between Python versions - use "sock.recv()" also as "sock.read()", if "sock" doesn't have "read()" method with Python 2.5.x or earlier, "sslutil.ssl_wrap_socket()" returns "httplib.FakeSocket"-ed object, and it doesn't have "read()" method, which is invoked via "smtplib.SSLFakeFile".

template: allow unquoted int function arguments

Merge with main

graft: use missing ancestors algorithm to find earlier grafts When the revisions to graft are numerically close to the destination, this avoids two walks up the DAG, which for a repository with over 470,000 changesets translates to around 2.2 seconds.

graft: find ancestors of destination lazily When the revisions to graft are numerically close to the destination, this avoids one walk up the DAG, which for a repository with over 470,000 changesets translates to around 1.1 seconds.

merge with crew

sslutil: try harder to avoid getpeercert problems We wrap both calls to getpeercert in a try/except to make sure we catch its bogus AttributeError.

copies._forwardcopies: use set operations to find missing files This is a performance win for a number of reasons: - We don't iterate over contexts, which avoids a completely unnecessary sorted call + the O(number of files) abstraction cost of doing that. - We don't check membership in a context, which avoids another O(number of files) abstraction cost. - We iterate over the manifests in C instead of Python. For a large repo with 170,000 files, this improves perfpathcopies from 0.34 seconds to 0.07. Anything that uses pathcopies, such as rebase or diff --git between two revisions, benefits.

perf: add a command to test copies.pathcopies perf An upcoming patch will improve the performance of this function, and this command will be used to demonstrate that improvement.

merge with stable

Added signature for changeset 8a7bd2dccd44

Added tag 2.5.4 for changeset 8a7bd2dccd44

applyupdates: assign variable before we try to use it (issue3855) The variable 'fd' was getting used with a value left over from a prior iteration, causing a KeyError: '.hgsubstate'.

merge with mpm

perf: add a command to test addremove performance Upcoming patches will improve addremove performance. This command will be used to demonstrate that improvement.

merge with crew

dirstate: only call lstat once per flags invocation This makes a big difference to performance in some cases. hg --time locate 'set:symlink()' mozilla-central (70,000 files): before: 2.92 sec after: 2.47 another repo (170,000 files): before: 7.87 sec after: 6.86

util: add functions to check symlink/exec bits These are not yet used.

Merge with stable.

setup.py: properly discard trust warning. This modification was missing from the fa91ddfc3f36 changeset.

scmutil.addremove: use iteritems on walk results Now that we no longer sort all the walk results, using iteritems becomes possible. This is a relatively minor speedup: on a large repository with 170,000 files, perfaddremove goes from 2.13 seconds to 2.10.

scmutil.addremove: stop sorting all walk results The only place where the order matters is in printing out added or removed files. We already sort that set. On a large repository with 170,000 files, this speeds up perfaddremove from 2.34 seconds to 2.13.

scmutil.addremove: pull ui.status printing out of the loop This will let us stop sorting all the results in an upcoming patch. This also permits future refactorings where the same code consumes dirstate.walk results but doesn't print anything out. An argument could be made that printing out results as we go along is more responsive UI-wise. However, at this point iterating through walk results is actually faster than sorting them, so once we stop sorting all the results the argument ceases to be valid.

scmutil.addremove: remove redundant directory and symlink checks dirstate.walk only does lstats and never returns stat objects for directories. On a large repository with 170,000 files, this speeds perfaddremove up from 2.40 seconds to 2.34.

scmutil.addremove: pull repo.dirstate fetch out of the loop On a large repository with 170,000 files, this speeds up perfaddremove from 2.78 seconds to 2.40.

scmutil: remove dead updatedir code This code has not been used in a long while.

merge with mpm

merge with stable

Added signature for changeset 50c922c1b514

Added tag 2.5.3 for changeset 50c922c1b514

hgweb: show correct error message for i18n environment If exception is error.LookupError and running in i18n environment, below condition is always true. Because msg is translated and dosen't contain 'manifest'. if util.safehasattr(err, 'name') and 'manifest' not in msg: This patch creates a new exception class and uses it instead of string match.

commands: fix typo in debugobsolete docstring

duplicatecopies: do not mark items not in the dirstate as copies Consider the following repo: 0 -- 1 (renames a to b) \ - 2 If we're rebasing 2 onto 1, then duplicatecopies is called with arguments (2, 1). copies.pathcopies goes backwards from 1 to 0 and returns the pair dst = a, src = b. Of course, since we're working on top of 2, at this point a doesn't exist in the dirstate. Extra entries in the copymap are currently harmless because the copymap is only queried for items in the dirstate map. However, if the dirstate.copy method becomes one of the sources used to determine which files have changed, this will prove problematic. Note that we can't avoid going backwards in general -- consider this repo: 0 -- 1 (renames a to b) \ - 2 (renames a to c) Rebasing 2 onto 1 should produce a rename from b to c.

duplicatecopies: fix arg name and docstring While most callers do indeed use duplicatecopies with a rev and its parent, rebase doesn't.

pull: list bookmarks before pulling changesets (issue3873) Consider a bookmark B that exists both locally and remotely. If B is updated remotely, and then a pull is performed where the pull set contains the new location of B, the bookmark is updated locally. However, if remote B is updated in the middle of a pull to a location not in the pull set, the bookmark won't be updated locally at all. To fix this, list bookmarks before pulling in changesets, not after. This still leaves a race open if B gets moved in between listing bookmarks and pulling in changesets, but the race window is much smaller. Fixing the race properly would require a bundle format upgrade. test-hook.t's output changes because we no longer do two listkeys calls during pull, just one. test-pull-http.t's output changes because we now search for bookmarks before searching for changes.

test-hook.t: remove prelistkeys.forbid hook before moving on An upcoming patch will change the order of operations and perform a listkeys before a changegroup fetch. This will cause a few tests to print out the wrong error message.

bookmarks.updatefromremote: rename local rb to remotemarks This local will become a parameter in an upcoming patch, so give it a more descriptive name. The name stays similar to the existing localmarks.

pull: rename local rb to remotebookmarks This local will become more important in an upcoming patch, so give it a more descriptive name.

dicthelpers: inline diff and join code mpm suggested this change since it improves performance slightly. Benchmarking hg perfcalculate -r . Before: ! wall 0.141173 comb 0.140000 user 0.140000 sys 0.000000 (best of 66) After: ! wall 0.138619 comb 0.140000 user 0.140000 sys 0.000000 (best of 69)

dicthelpers: add docstrings for diff and join

perf: add perfdirs command This measures the cost of computing the dirstate's dirs structure, which becomes very expensive in a large working directory.

merge with stable

zsh_completion: fix trailing carriage return spoiling tag completion This was giving the following error: % hg up 2.5-r<Tab> (eval):1: bad substitution % hg up 2.5-r

filesets: add eol predicate

merge with stable

merge with i18n

merge with i18n

i18n-ja: synchronized with 61c8327ced50

perf: remove mysterious trailing newlines

blackbox: fix a few check-code portability errors

check-code: fix a check-code failure in check-code The irony is delicious.

check-code: re-add check for missing glob As suggested by mpm reintroducing this test failure on windows is prevented by a code-check. One line in a unix test is ignored. This pattern matches 20 windows glob lines.

tests: append missing glob to filename output and check-code it

check-code: extract windows glob warning message it will be used often

blackbox: defer opening a log file until needed (issue3869) Previously, we opened the log file when creating a repo object. This was inefficient (not all repo creation is going to result in a need to log something), but more importantly it broke subrepo updates when used on NFS. * perform an update in the master repo that triggers a subrepo clone * empty subrepo already exists, and has an open, empty blackbox.log file due to it being opened eagerly/prematurely * hg decides to blow away the skeletal subrepo (see use of shutil.rmtree in subrepo._get) * we crash, due to NFS treating a delete of an open file as really a rename to a hidden ".nfs" file Now that we open the blackbox log file on demand, no file exists at the time the empty subrepo is deleted, so the above problem does not occur.

patch: match 'diff --git a/' instead of 'diff --git' This reduces the likelihood of a traceback when trying to email a patch that happens to have 'diff --git' at the beginning of a line in the description, as this patch did: http://markmail.org/message/wxpgowxd7ucxygwe

color: drop unnecessary global statement

serve: pass on the repo instad of recreating it in hgweb When we pass on the path to the repo, the repo is created in hgweb. But the repo is already here, so pass it on.

serve: pass the prepared baseui to hgweb The baseui was carefully prepared but not used. ui can contain repo specific settings which can have unwanted effects.

hgweb: do not pass on repo.ui when recreating a repo Recreate the repo with the global configuration in repo.baseui. The repo configuration is reread anyway. And now deleted repo configuration is reset to the default value.

repo: repo isolation, do not pass on repo.ui for creating new repos A repo should not get the configuration from an other repo, so create it with the global configuration in repo.baseui. This is done too when recreating a repo. The repo configuration is reread anyway. And now deleted repo configuration does not persist.

diff: fix binary file removals in git mode. With the previous version, a binary file removal diff generated with

manifestmerge: drop redundant flags calls

manifestmerge: use dicthelpers.diff and join This patch improves manifestmerge performance significantly. In a repository with 170,000 files, the following results were observed on a clean working directory. Revision '.' adds one file. hg perfmergecalculate -r . - before: 0.41 seconds - after: 0.13 seconds hg perfmergecalculate -r .^ - before: 0.53 seconds - after: 0.24 seconds Comparing against '.' is much faster than comparing against '.^' because with '.', the wctx and p2 manifest strings have the same identity, so comparisons are simply pointer equality. With '.^', the strings have different identities so we need to perform memcmps. Any operation that uses manifestmerge benefits. - hg update . goes from 2.04 seconds to 1.75 - hg update .^ goes from 2.52 seconds to 2.25 - hg rebase -r . -d .~6 (involves 4 merges) goes from 11.8 seconds to 10.8

manifestdict: add a method to diff _flags This will be used in an upcoming patch.

mercurial: implement diff and join for dicts Given two dicts, diff returns a dict containing all the keys that are present in one dict but not the other, or whose values are different between the dicts. The values are pairs of the values from the dicts, with missing values being represented as an optional argument, defaulting to None. Given two dicts, join performs what is known as an outer join in relational database land: it returns a dict containing all the keys across both dicts. The values are pairs as above, except they aren't compared to see if they're the same.

convert: add closesort algorithm to mercurial sources If you actively work with branches, sometimes you need to close old branches which last commited hundreds revisions ago. After close you will see long lines in graph visually spoiling history. This sort only moves closed revisions as close as possible to parents and does not increase storage size as datesort do.

manifestmerge: rename n to n1 and n2 An upcoming patch will combine the two loops into one, so it's important to distinguish between nodes in m1 and nodes in m2.

perf: add a command to measure merge.calculateupdates perf The performance of merge.manifestmerge will be improved in upcoming patches. This command will be used to demonstrate the improvement.

hgk: fix a pyflakes error

dirstate.walk: fast path none-seen + match-always case for step 3 This case is a common one -- e.g. `hg diff`. For a repository with 170,000 files, this speeds up perfstatus from 0.95 seconds to 0.88.

dirstate.walk: fast path match-always case during traversal This case is a common one -- e.g. `hg status`. For a repository with 170,000 files, this speeds up perfstatus --unknown from 2.15 seconds to 2.09.

largefiles: fix _always for match overrides Upcoming patches will speed dirstate.walk up by not filtering based on the match function when match.always() is True. For that to work, match.always() needs to be accurate. Previously it wasn't so for largefiles.

dirstate.walk: remove subrepo and .hg from results before step 3 An upcoming patch will speed dirstate.walk up by not querying the results dict when it is empty. This ensures it is in some common cases. This should be safe because subrepos and .hg aren't part of the dirstate.

merge with stable

blackbox: account for another source of errors

hgk: remove regsub, split on \r instead No need to regsub all \r\n's to \n's, we can just split on \r as well; this may produce extra empty elements in the list, but they won't match anyway.

hgk: simplify tags parser As when hg tags is being called without -v option, it returns lines with two elements in each, we can just interate them as if it were a usual Tcl list using foreach and two variables. Line endings and whitespace don't matter when doing so, so we may keep them as is. When we're processing the lines, tag variable is assigned a tag name, and rev is a string in form of revision:hash which we can split on colon. As Tcl8.4 lacks lassign command, and using lindex makes code a bit less readable, we use foreach to iterate over two-element list.

hgk: add support for phases Add support for phases. Update tests.

hgk: in popups, replace labels with window titles Replace labels with window titles. That also requires to change grid placement manager settings to compensate padding which used to be applied to now non-existent labels.

hgk: update backgroud colour when Ttk is available Ttk doesn't automatically set up Tk colour palette. Because of that, Ttk controls look differently when used together with old Tk controls. When we use Ttk controls, we first query if we have any setting for the client background, and if we do, we update Tk palette as well.

hgk: fix pop-up windows Create pop-up windows properly so they go to the floating layer in tiling window managers.

hgk: don't use fixed format for dates Don't pass -format with a date format string, use default instead so users can use their own preferences.

hgk: reformat changsets fields Rename Author to User. Rename Revision to Changeset. Put Date on its own line.

hgk: drop committer field from the UI Remove Committer field, it doesn't make sense in Hg.

test-completion: test completion of tags

debugpathcomplete: satisfy the code checker

tests: test debugpathcomplete and debuglabelcomplete

tests: rename test-debugcomplete.t to test-completion.t

pathcomplete: complete directories more conservatively Suppose we want to perform a single-level completion (i.e. without --full) of "fi" in a repo containing "fee", "fie/dead", "fie/live", and "foe". If we give back "fie/" as the only answer, the shell will consider the completion to be unambiguous, and will append a space after the completion. We can't complete "fie/live" or "fie/dead" without first backspacing over that space. We used to thus create two fake names, "fie/a" and "fie/b", to force the shell to consider the completion to be ambiguous. It would then stop at "fie/" without appending a space, allowing us to hit tab again to complete "fie/live" or "fie/dead". The change here arises from realising that we only need to force the shell to consider a completion as ambiguous if we have exactly one directory and zero files as possible completions. This prevents spurious names from showing up as possible completions when they don't need to be invented in the first place.

sadclown: another test failure eluded me

bash_completion: recognize normal command abbreviations In many common cases, this eliminates a call to "hg help" to get a complete command name, thus improving responsiveness.

completion: selectively use debugpathcomplete in bash_completion The current bash_completion code can be very slow in a large working directory. It always uses "hg status" to generate possibly matching files, which checks the status of every file. We often don't care about status when completing, so that cost is very high. As the new debugpathcomplete command does not check the status of files, it offers much better performance for commands that only care about completing names.

completion: add a debugpathcomplete command The bash_completion code uses "hg status" to generate a list of possible completions for commands that operate on files in the working directory. In a large working directory, this can result in a single tab-completion being very slow (several seconds) as a result of checking the status of every file, even when there is no need to check status or no possible matches. The new debugpathcomplete command gains performance in a few simple ways: * Allow completion to operate on just a single directory. When used to complete the right commands, this considerably reduces the number of completions returned, at no loss in functionality. * Never check the status of files. For completions that really must know if a file is modified, it is faster to use status: hg status -nm 'glob:myprefix**' Performance: Here are the commands used by bash_completion to complete, run in the root of the mozilla-central working dir (~77,000 files) and another repo (~165,000 files): All "normal state" files (used by e.g. remove, revert): mozilla other status -nmcd 'glob:**' 1.77 4.10 sec debugpathcomplete -f -n 0.53 1.26 debugpathcomplete -n 0.17 0.41 ("-f" means "complete full paths", rather than the current directory) Tracked files matching "a": mozilla other status -nmcd 'glob:a**' 0.26 0.47 debugpathcomplete -f -n a 0.10 0.24 debugpathcomplete -n a 0.10 0.22 We should be able to further improve completion performance once the critbit work lands. Right now, our performance is limited by the need to iterate over all keys in the dirstate.

tests: add new command to test-debugcomplete

completion: add a debuglabelcomplete command When completing a "label" (a symbolic name for a commit), the bash_completion script currently has to invoke hg three times. For a large repository, the cost of starting up and loading all the necessary context over and over is very high. For instance, in mozilla-central: time (export HGPLAIN=1; hg tags -q; hg bookmarks -q; hg branches) >/dev/null 0.446 sec Compare with the debuglabelcomplete command that this commit adds: time hg debuglabelcomplete >/dev/null 0.148 sec This greatly helps responsiveness.

bash_completion: nuke a duplicated function

run-tests: only sort files when not given as argument os.listdir returns the files in any order. This has to be sorted. But when given as argument, the user should be allowed to set any order. This restores the behaviour before 9848a94e2a.

blackbox: use util.getuser for portability On Windows, our implementation is more robust than Python's version.

blackbox: prevent failed I/O from causing hg to abort Instead, we simply print a warning message if opening the blackbox log file fails, or if writing to it fails.

merge with crew-stable

largefiles: don't query the dirstate for key None lfutil.splitstandin(f) can be None, and we query the dirstate for that without checking if it is. This will cause problems with the upcoming move to critbit- based dicts, since they only support strings as keys.

merge

bookmarks: moving the active bookmark deactivates it After this change, moving the active bookmark somewhere other than the current changeset (i.e., with --rev) deactivates it. Previously it would remain in .hg/bookmarks.current, which seems like a bug.

bookmarks: allow (re-)activating a bookmark on the current changeset Allow a bookmark that points to the current changeset to be made the active bookmark without requiring --force. Previously, this would've aborted with: abort: bookmark 'Z' already exists (use -f to force)

applyupdates: assign variable before we try to use it (issue3855) The variable 'fd' was getting used with a value left over from a prior iteration, causing a KeyError: '.hgsubstate'.

revset: don't abort when regex to tag() matches nothing (issue3850) This makes the tag("re:...") revset consistent with branch("re:...").

rebase: fix --collapse when a file was added then removed When a series of commits first adds a file and then removes it, hg rebase --collapse prompts whether to keep the file or delete it. This is due to it reusing the branch merge code. In a noninteractive terminal it defaults to keeping the file, which results in a collapsed commit that is has a file that should be deleted. This bug resulted in developers accidentally commiting unintentional changes to our repo twice today, so it's fairly important to get fixed. This change allows rebase --collapse to tell the merge code to accept the latest version every time without prompting. Adds a test as well.

Merge

grep: use re2 if possible Since re2 is enormously faster than Python's re module, this should help performance, ceteris paribus. grep currently has other huge performance problems that mask any gain :-(

util: add flag support to compilere

bookmarks: fix test broken by 0bba1ff2ac7b The added test reflects an error in printing the origin node of the bookmark that I fixed before committing, but I forgot to update the test (whoops).

bookmarks: allow moving a bookmark forward to a descendant Allow 'hg bookmark MARK', with an existing bookmark MARK, to move the bookmark forward to the current or specified revision, if the target revision is a descendant of the revision the bookmark currently points to. Prints a status message including the revision the bookmark was formerly at: $ hg bookmark Z moving bookmark 'Z' forward from 663762316562 Test coverage is added.

hgweb: change manifest archive links to only archive the current directory When the web server shows the manifest for a single, non top directory, append the path to the directory to the archive links. This makes the web server generate archive files that only include the current directory (and its subdirectories). Note that archive links in other pages (e.g. changeset) or at the top of the manifest are unchanged. Directory archive links have an extra "/" at the end which does not impact the result of the archive operation. Keeping it there made the implementation of this feature simpler.

hgweb: teach archive how to download a specific directory or file The archive web command now takes into account the "file" request entry, if one is provided. The provided "file" is processed as a "path" corresponding to a directory or file that will be downloaded. With this change hgweb can to process requests such as: http://mercurial.selenic.com/hg/archive/tip.zip/mercurial/templates This will download all files on the mercurial/templates directory as a zip file. It is not possible to specify file patterns ('glob', 'relglob', 'path', 'relpath', 're', 'relre' nor 'set'). The server will reject those with a 403 HTTP error response. Note that this is a first step to add support for downloading directories from the web interface. A following patch will modify the archiveentry map entry on the different templates so that it adds the current folder path to the archive links.

test-archive: gracefully handle HTTPErrors on get-with-headers This avoids pritting out a traceback when a get-with-headers call causes hgweb to respond with an HTTPError code.

bash_completion: tell an editor what type of file this is

bash_completion: allow remove to complete normal files Previously, we only completed files that had already been manually deleted. That behaviour made no sense. We now complete unmodified, modified, and deleted files.

bash_completion: match more narrowly This greatly helps completion performance for most commands that deal with files. In a working dir with 150,000 files, where we want to complete the name of a modified file under a path beginning with "a", from the root of the working dir: (old) hg status -nm . 1.7 sec (new) hg status -nm "glob:a**" 0.3 Even "hg add" becomes a little faster, in spite of being the worst case (matching untracked files).

blackbox: add backup bundle paths to blackbox logs Writes the backup bundle paths to the blackbox so it's easy to see which backup bundle is associated with which command when you are debugging an issue. Example output: 2013/03/13 10:39:56 durham> strip tip 2013/03/13 10:39:59 durham> saved backup bundle to /data/users/durham/www-hg/.hg/strip-backup/e5fac262363a-backup.hg 2013/03/13 10:40:03 durham> strip tip exited 0 after 7.97 seconds

tests: fix test-profile to not depend on HGPROF environment variable The test-profile test would fail if the user had HGPROF set to another profiler in their environment. This fix makes the test independent of that environment variable. Reverts the previous attempt to fix this, which was not cross platoform.

merge with crew-stable

tests: fix test-profile to not depend on HGPROF environment variable The test-profile test would fail if the user had HGPROF set to another profiler in their environment. This fix makes the test independent of that environment variable.

check-code: do not prepend "warning" to a failure message The prefix has not been removed when this check changed from a warning to a failure.

merge with crew-stable

strip: make --keep option not set all dirstate times to 0 hg strip -k was using dirstate.rebuild() which reset all the dirstate entries timestamps to 0. This meant that the next time hg status was run every file was considered to be 'unsure', which caused it to do expensive read operations on every filelog. On a repo with >150,000 files it took 70 seconds when everything was in memory. From a cold cache it took several minutes. The fix is to only reset files that have changed between the working context and the destination context. For reference, --keep means the working directory is left alone during the strip. We have users wanting to use this operation to store their work-in-progress as a commit on a branch while they go work on another branch, then come back later and be able to uncommit that work and continue working. They currently use 'git reset HARD^' to accomplish this in git.

sshpeer: store subprocess so it cleans up correctly When running 'hg pull --rebase', I was seeing this exception 100% of the time as the python process was closing down: Exception TypeError: TypeError("'NoneType' object is not callable",) in <bound method Popen.__del__ of <subprocess.Popen object at 0x937c10>> ignored By storing the subprocess on the sshpeer, the subprocess seems to clean up correctly, and I no longer see the exception. I have no idea why this actually works, but I get a 0% repro if I store the subprocess in self.subprocess, and a 100% repro if I store None in self.subprocess. Possibly related to issue 2240.

blackbox: fix exception when logging commands with format characters When running commands like 'hg export -o mypatch-%N.patch', the blackbox would throw an exception because it tried to format %N. This change prevents it from trying to format the command string.

localrepo: always write the filtered phasecache when nodes are destroyed (issue3827) When the strip command is run, it calls repo.destroyed, which in turn checks if we read _phasecache, and if we did calls filterunknown on it and flushes the changes immediately. But in some cases, nothing causes _phasecache to be read, so we miss out on this and the file remains the same on-disk. Then a call to invalidate comes, which should refresh _phasecache if it changed, but it didn't, so it keeps using the old one with the stripped revision which causes an IndexError. Test written by Yuya Nishihara.

merge with stable

rebase: restore active bookmark after rebase --continue When a rebase has conflicts and the user uses rebase --continue, the previously active bookmark was not being made active once again. With this change that bookmark is made active again, just as if the rebase had never been interrupted. This changes the rebasestate file format, but should handle old formats correctly. Since the file is transient, this is even less of a problem. Adds a test to verify the new behavior. I manually tested continuing rebases with and without an active bookmark, and with and without being on the bookmark being rebased.

setup.py: ignore warnings from obsolete Fix version detection with recent hg versions.

setup.py: add metadata to register package to PyPI This allows updating PyPI with just 'setup.py register' (assuming the user is an owner or maintainer of the PyPI package).

minirst: optimize HTML table generation a bit avoid a couple of array copies and string interpolations

minirst: HTML formatter tweaks output table rows on distinct lines don't make the first row a table header

minirst: CGI escape strings prior to embedding it in the HTML

minirst: don't generate TH elements for the first row in table We mainly use them for option tables -- browsers bolding and centering them looks wrong.

help: use a full header for topic titles ...for prettier HTML!

hgweb: generate HTML documentation It's generated from the raw ReST source, as returned from help.help_().

help: move the majority of the help command to the help module We move the logic for generating the unformatted ReST source to the help module, in order to eventually avoid calling commands.help_() from hgweb. No functionality change.

hgweb help: split up long lines (in generated output)

hgweb help: add tests

merge with stable

merge with crew

Added signature for changeset 5b7175377bab

Added tag 2.5.2 for changeset 5b7175377bab

setparents: drop copies from dropped p2 (issue3843)

httppeer: avoid large dumps when we don't see an hgweb repo When we don't get an hgweb protocol response, we dump the response to the user for diagnostic purposes (it might be a cgitb message, for instance). But if we try to clone a bundle, we don't want to show the entire bundle in the error message. Also, we don't want fetch the full bundle multiple times during fallback. So we only fetch 1k here.

httppeer: improve protocol check Pre-0.6c hgweb used text/plain for protocol responses. This meant that a web server could serve a static file and confuse a client into generating a nasty traceback. Now we insist that text/plain protocol responses not include a Content-Length, which older hgweb didn't generate but will typically be produced for static files.

util: add a timed function for use during development I often want to measure the cost of a function call before/after an optimization, where using top level "hg --time" timing introduces enough other noise that I can't tell if my efforts are having an effect. This decorator allows a developer to measure a function's cost with finer granularity.

util: generalize bytecount to unitcountfn This gives us a function we can reuse to count units of other kinds.

help: new SHA-1 fingerprint of hg.intevation.org in hostfingerprints example The certificate was updated in March 2012. You can verify the certificate by using the Root CA certificate downloadable from https://ssl.intevation.de/ The intermediate CA is sent by https://hg.intevation.org/

tests: work around msys bash mangling of paths in test-mq.t

largefiles: remove wrong comment on standin matcher and code for creating dirs The existence of these directories could not be relied on anyway and all dependencies on them should have been removed.

merge with stable

templatefilters: add missing import of _

largefiles: updatelfiles should use working dir standins, not standins from p1 This makes a difference when working directory is dirty, especially when merging with a revision for which we don't have largefiles.

largefiles: simplify cachelfiles - don't spend a lot of time checking hashes cachelfiles jumped through loops to handle merges and modified files ... but it did apparently no longer have a valid reason to do so. It should just always make sure that the largefiles referenced from the standins are present - no matter which actual largefile is stored in the working directory. If there is no standin then there is nothing to fetch. The old code usually verified the hash of all largefiles every time this function was invoked - for examply by 'update'. This change makes a trivial noop update 5-10 seconds faster on our repo (with the other 50% spent doing another unnecessary hashing of all largefiles).

largefiles: don't let update leave wrong largefiles in wd if fetch fails Situations where a largefile for some reason wasn't available sometimes caused wrong largefile content and state. It has mostly been seen when interrupting download of largefiles ... and when introducing programming errors. Instead we now make sure to delete the old and wrong largefile. A missing file is a well-known error condition and much more reasonable way to handle the situation.

largefiles: missing largefiles should not be committed as removed Largefiles can easily become missing - for example if it simply isn't available or the download fail. It might even be convenient to be able to work that way in some cases. But commiting missing largefiles as if they had been 'hg remove'd is plain wrong.

largefiles: don't assume that .hg/largefiles/ still exists It might not have been created and it might have been removed.

largefiles: getstandinmatcher should not depend on existence of directories Looking for a (potentially empty) directory was not reliable - both because it is a reasonable assumption that empty directories can be removed and because it wasn't created in all cases ... such as when pulling to an existing repository.

tests: don't rely on broken behaviour in test-largefiles-cache.t The test relied on the bug that 'pull largefiles from branchheads' didn't pull any largefiles from tip revision when it seemed like no largefiles had been checked out before.

largefiles: fix download of largefiles from an empty list of changesets The empty list was interpreted as all revisions - just like None is. The empty list is now handled explicitly.

largefiles: fix off-by-one error on pull --all-largefiles Test output is changed in a case where one revision was pulled, but because of the off-by-one error it thought that 0 revisions were pulled ... and because of another bug it thus (tried to) fetch largefiles for all revisions. After this change it no longer reports failure when it failed while trying to fetch largefiles it shouldn't fetch. Largefiles that it shouldn't fetch but managed to fetch anyway will now correctly be missing later on. This change thus resolves some of unexplained test output introduced in 1e4eb1faba6e.

tests: remove glob lines which unnecessary match / for \ on windows This lines were reported as unnecessary when running the tests on windows because the path was already printed with a slash and not a backslash.

tests: append glob to filename output when required for windows The test failed on windows before this patch.

convert: stabilize cvsps commitid sort order

pager: catch ctrl-c on exit (issue3834)

merge with crew

templater: properly handle file_copies with %

localrepo: iterate over manifest key/value pairs in status This saves us a couple of dict lookups in the common case, and improves the performance of the status method by 5% (measured with util.timed) in a repo with a large manifest.

match: more accurately report when we're always going to match This improves the performance of log --patch and --stat by about 20% for moderately large manifests (e.g. mozilla-central) for the common case of no -I/-X patterns.

scmutil: fix NameError on windows Forgot to import util in mercurial.scmwindows. Error was introduced in: changeset: 18862:4c6f7f0dadab user: Kevin Bullock <kbullock@ringworld.org> date: Tue Feb 12 11:36:21 2013 -0600 summary: scmutil: split platform-specific bits into their own modules

commands: exit from the log loop at the right time Previously, we'd run the iterator one more time than necessary, potentially doing a lot of extra work in the process.

cmdutil: use a small initial window with --limit In a large repo, running a command like "log -l1 -p" was expensive because it would always traverse 8 commits, as 8 was the initial window size. We now choose the lesser of 8 or the limit, speeding up the "log -l1 -p" case by a factor of 5.

worker: handle worker failures more aggressively We now wait for worker processes in a separate thread, so that we can spot failures in a timely way, wihout waiting for the progress pipe to drain. If a worker fails, we recover the pre-parallel-update behaviour of failing early by killing its peers before propagating the failure.

worker: fix a race in SIGINT handling This is almost impossible to trigger due to the tiny time window involved.

worker: on error, exit similarly to the first failing worker Previously, if a worker failed, we exited with status 1. We now exit with the correct exit code (killing ourselves if necessary).

merge with stable

merge with main

largefiles: don't cache largefiles for pulled heads by default After discussion, we've agreed that largefiles for newly pulled heads should not be cached by default. The use case for this is using largefiles repos with multiple remote servers (and therefore multiple remote largefiles caches), where users will be pulling from non-default locations on a regular basis. We think this use case will be significantly less common than the use case where all largefiles are stored on the same central server, so the default should be no caching. The old behavior can be obtained by passing the --cache-largefiles flag to pull.

merge with stable

merge with crew

bundle: treat branches created newly on the local correctly (issue3828) Before this patch, "hg bundle --branch foo other" fails to create bundle file, if specified "foo" branch is created newly on the local repository. "hg bundle" uses "hg.addbranchrevs(repo, other, ...)" to look branch names up, even though other outgoing-like implementation uses "hg.addbranchrevs(repo, repo, ...)". In the former invocation, "other" repository recognizes such branches as unknown, so execution is aborted. This patch uses "hg.addbranchrevs(repo, repo, ..)" in "hg bundle" to bundle revisions on such branches correctly.

merge with stable

mergetools: refine vimdiff warning message We explicitly redraw before echoing the message so that it simply displays at the bottom of the window. Also simplifies the message printing by using 'echomsg' (which uses 'echohl' internally) and adds the names of the software involved for improved Googleability.

mergetools: vimdiff issue a warning explaining how to abort Adds a message displayed at each vimdiff invocation: merge conflict detected, type ":cq" to abort Vimdiff is very confusing for non-vim user (not to speak about vim user confused anyway. However it is very likely that vimdiff is picked as the mergetool of choice when using the default config: - vim is available on all UNIX system. - Its one of the rare non graphical merge tools.

httppeer: improve protocol check Pre-0.6c hgweb used text/plain for protocol responses. This meant that a web server could serve a static file and confuse a client into generating a nasty traceback. Now we insist that text/plain protocol responses not include a Content-Length, which older hgweb didn't generate but will typically be produced for static files.

httppeer: avoid large dumps when we don't see an hgweb repo When we don't get an hgweb protocol response, we dump the response to the user for diagnostic purposes (it might be a cgitb message, for instance). But if we try to clone a bundle, we don't want to show the entire bundle in the error message. Also, we don't want fetch the full bundle multiple times during fallback. So we only fetch 1k here.

mergetools: refine vimdiff warning message We explicitly redraw before echoing the message so that it simply displays at the bottom of the window. Also simplifies the message printing by using 'echomsg' (which uses 'echohl' internally) and adds the names of the software involved for improved Googleability.

mergetools: vimdiff issue a warning explaining how to abort Adds a message displayed at each vimdiff invocation: merge conflict detected, type ":cq" to abort Vimdiff is very confusing for non-vim user (not to speak about vim user confused anyway. However it is very likely that vimdiff is picked as the mergetool of choice when using the default config: - vim is available on all UNIX system. - Its one of the rare non graphical merge tools.

dispatch: also a separate warning message on aliases with --config As mentioned in bug 2043, --config is also not supported in an alias. So report this the same way as the other "early" options. Example with alias.broken = stat --config a.config=1 Before: $ hg broken abort: Option --config may not be abbreviated! After: $ hg broken error in definition for alias 'broken': --config may only be given on the command line

extensions: remove erroneous comment We actually -do- use the 'ui' argument to print a debug statement.

blackbox: do not translate the log messages User 'timeless' in irc mentioned that having the blackbox be translated would result in logs that: - may be mixed language, if multiple users use the same repo - are not google searchable (since searching for english gives more results) - might not be readable by an admin if the employee is using hg in his native language And therefore we should log everything in english.

scmutil: split platform-specific bits into their own modules This parallels what's done for the util module, which imports either mercurial.posix or mercurial.windows as 'platform' and then slurps the appropriate functions into its own namespace.

backout: call cmdutil.commit directly instead of commands.commit This cleans up the messiness of having one command call another, and makes the backout command robust against changes to the commit command.

commit: factor out status printing into a helper function We create a new function commitstatus() in cmdutil that handles printing the status message(s) after a commit. This will allow other commit-like commands to use it, and in particular is step 2 towards removing backout's call to commands.commit.

backout: remove unnecessary dict copy This is step 1 to remove backout's call to commands.commit. We don't use the options again anywhere below except for backout's own purposes, specifically choosing a merge tool, so we just write the commit options in directly.

backout: remove unnecessary frobbing of addremove option There's no way for addremove to show up in backout's opts dictionary. It was being set manually because cmdutil.commit expected it to be there (and would throw an exception if it wasn't). This was fixed waaaaaaay back in: changeset: 5829:784073457a0f user: Kirill Smelkov <kirr@mns.spb.ru> date: Thu Jan 10 12:07:18 2008 +0300 summary: cmdutil.commit: extract 'addremove' from opts carefully

backout: use cmdutil.revert directly instead of commands.revert Before this change, backout would explicitly set the options it passed to commands.revert in order to fall thru most of its logic and call cmdutil.revert. This change makes it clearer what backup is trying to accomplish and makes it robust against changes to the revert command.

merge with stable

check-code: warn about line glob match with no glob character (?*/)

tests: remove glob from output lines containing no glob character

tests: inform on Windows about unnecessary glob lines When glob lines directly match on windows, "/" (and not "\") was output in the path on the line. No glob matching is necessary in this case. The test output will look like this (when 5 tests have passed and no 4 has an unnecessary glob): ... Info, unnecessary glob: info about some/thing (glob) ..

tests: quickly check if the glob line already matches the output This happens when a path with "/" as only glob char is matched on a non windows platform. (Currently one third of all glob matches.) The slowdown on windows and the speedup on other os are neglectable.

tests: append glob to filename output when required (windows) This lines were introduced in cd403d6d96ef and made the test fail on windows.

util: make ensuredirs safer against races

blackbox: only show new heads on incoming The blackbox was logging every head after every incoming group. Now we only log the heads that have changed. Added a test. Moved the hooks test to the bottom of the file since the hooks interfer with the tests after it.

blackbox: fix copyright

blackbox: fix a failing pyflakes test

blackbox: tests for the blackbox extension A few tests to cover the blackbox extension. Covers commands, hooks, and incoming changes.

blackbox: adds a 'blackbox' command for viewing recent logs Adds a 'hg blackbox' command for viewing the latest entries in the blackbox log. By default it shows the last 10 entries, but -l allows the user to specify.

blackbox: log incoming changes via ui.log() Logs incoming changes to a repo to ui.log(). Includes the number of changes and the hashes of the heads after the new changes. Example log line: 2013/02/09 08:35:19 durham> 1 incoming changes - new heads: cb9a9f314b8b Currently the blackbox logs the unix user that is performing the push/pull. It would be nice to log the http authorized user as well so it works with hgweb, but that's outside the scope of this commit.

blackbox: logs python and extension hooks via ui.log() Logs python and extension hooks to ui.log() for viewing in the blackbox. Example log lines: 2013/02/09 08:35:19 durham> pythonhook-preupdate: hgext.eol.preupdate finished in 0.01 seconds 2013/02/09 08:35:19 durham> exthook-update: echo hooked finished in 0.02 seconds

blackbox: log the commands that are run Uses ui.log to log which commands are run, their exit code, the time taken, and any unhandled exceptions thrown. Example log lines: 2013/02/09 08:35:19 durham> add foo 2013/02/09 08:35:19 durham> add exited 0 after 0.02 seconds Updates the progress tests because they use a mocked time.time() which these changes affect.

blackbox: adds a blackbox extension Adds a blackbox extension that listens to ui.log() and writes the messages to .hg/blackbox.log. Future commits will use ui.log() to log commands, unhandled exceptions, incoming changes, and hooks. The extension defaults to logging everything, but can be configured via blackbox.track to only log certain events. Log lines are of the format: "date time user> message" Example log line: 2013/02/09 08:35:19 durham> 1 incoming changes - new heads: d84ced58aaa

scmutil: create directories in a race-safe way during update With the new parallel update code, it is possible for multiple workers to try to create a hierarchy of directories at the same time. This is hard to trigger in general, but most likely during initial checkout. To deal with these races, we introduce a new ensuredirs function whose contract is to ensure that a directory hierarchy exists - it will ignore a failure that implies that the desired directory already exists.

Merge

test-atomictempfile: convert to unit test

tests: add a test runner utility that prints nothing when all tests pass This will be used to run tests through run-tests, which will expect no output for a unit test that passes successfully. The motivation for using unit tests instead of the current Python tests is that they don't require an output file for comparison and that they're easier to write because of the available tools from unittest (setup, asserts).

hgweb: consistent author name width Paper & coal use a sligtly shorter name length of 12em in the log view than gitweb's 15em. Names too long for 12em is not entirely unheard of, and hence increasing the length is appropriate; hgweb should remain usable on 1024x768.

dirstate: fix generator/list error when using python 2.7 util.statfiles returns a generator on python 2.7 with c extensions disabled. This caused util.statfiles(...) [0] to break in tests. Since we're only stat'ing one file, I just changed it to call lstat directly.

mq: comply with filtering when injecting fake tags (issue3812) mq was injecting fake tags whenever the revisions were accessible to the filtering level. This issue impacts hgweb since it's common to have "secret" mq patches. As secret changesets are filtered by hgweb, the tags computation could break.

commit: factor out post-commit cleanup into workingctx This pulls some of the logic for the cleanup that needs to happen after a commit has been made otu of localrepo.commit and into workingctx. This is part of a larger refactoring effort that will eventually allow us to perform some types of merges in-memory.

localrepo: use workingctx for validation in commit This changes localrepo.commit to use the workingctx it creates form the munged output of localrepo.status while running some precommit validation. Specifically, it uses functions that were already present on the workingctx. I believe this is a net readabilty improvement, and that this makes these lines consistent with the refactoring in a subsequent patch that pulls some of the validation logic into workingctx so that it can be reused elsewhere.

localrepo: create context used for actual commit earlier localrepo.commit creates a workingctx, calls self.status, does some munging on the changes status returns, does some validation on those changes, and then creates a new workingctx from the changes. This moves the creation of the new workginctx ahead of some validation, with the intention of refactoring some of that validation logic into the workingctx, so that it can be reused elsewhere.

merge with crew-stable

debugobsolete: improve command help The behavior without argument was not documented.

import: don't rollback on failed import --exact (issue3616) The checkexact() helper function was calling repo.rollback() from inside an open transaction. In addition to being insane, this is unnecessary because import will release the transaction on an exception. It turns out that this has been broken since the feature was first introduced, first released in v1.0: changeset: 4263:47ba52121433 user: Brendan Cully <brendan@kublai.com> date: Thu Mar 22 10:44:59 2007 -0700 files: mercurial/commands.py mercurial/patch.py description: Add import --exact. When this option is set, import will apply the patch (which must be generated by export) to the parents specified in the patch, and check that the node produced by the patch matches the node ID in the patch.

Merge

Merge

merge crew and main

merge crew and main

manifestmerge: fix order in which manifests are fetched If the manifest of an earlier revision on the same delta chain is read before that of a later revision, the revlog remembers that we parsed the earlier revision and continues applying deltas from there onwards. If manifests are parsed the other way round, we have to start over from the fulltext. For a fresh clone of mozilla-central, updating from 29dd80c95b7d to its parent aab96936a177 requires approximately 400 fewer zlib.decompress calls, which results in a speedup from 1.10 seconds to 1.05.

merge: run _forgetremoved after manifestmerge _forgetremoved can trigger manifest construction, but we only want it to happen after manifestmerge, so that our attempt to read the manifests in the right order in an upcoming patch actually works.

dirstate: disable gc while parsing the dirstate This prevents a performance regression an upcoming patch would otherwise introduce because it indirectly delays parsing the dirstate a bit.

export: show 'Date' header in a format that also is readable for humans 'export' is the official export format and used by patchbomb, but it would only show date as a timestamp that most humans might find it hard to relate to. It would be very convenient when reviewing a patch to be able to see what timestamp the patch will end up with. Mercurial has always used util.parsedate for parsing these headers. It can handle 'all' date formats, so we could just as well use a readable one. 'export' will now use the format used by 'log' - which is the format described as 'Unix date format' in the templating help. We assume that all parsers of '# HG changeset patch'es can handle that.

factotum: fix urllib2 import so it no longer relies on a demandimport bug demandimport will do that urllib2 can be imported 'from mercurial' even though it doesn't exist there.

hgweb: make the test suite use hgweb in a more WSGI compliant way - as checked by wsgiref.validate. This makes sure that we don't optimize hgweb for invalid use cases.

hgweb: simplify internal staticfile return codes

spelling: fix some minor issues found by spell checker

bundlerepo: replace basemap with the base field in the index Bundle revisions had some info in their fake revlog intries and some info in a dict with all the bundle revisions. This dict was used to get the stored data and to distinguish repo revisions from bundle revisions. Real repo revisions and bundle revisions will now be distinguished by comparing with the tip revision of the original repo. This reintroduces something similar to disktiprev which was unused and removed in a928865b4a4f and let that replace the O(reposize) dict.

profiling: replace '+' markup of nested lines with indentation The display of nested lines for hg --profile was very non-obvious and made it look like sort didn't work. The '+' immediately before CallCount was not related to the CallCount and did not mean plus in any integer sense. The '+' before module looked like a part of the module name and not like ascii art. Instead we now indent the subordinate module names to clearly show the structure.

Merge crew and main.

merge: don't fiddle with name lookups or i18n in hot loops We perform attribute dereferences and i18n lookups before looping.

merge: apply non-interactive working dir updates in parallel This has a big effect on the performance of working dir updates. Here are the results of update from null to the given rev in several repos, on a Linux 3.2 system with 32 cores running ext4, with the progress extension enabled. repo rev plain parallel speedup hg 7068089c95a2 0.9 0.3 3 mozilla-central fe1600b22c77 42.8 7.7 5.5 linux-2.6 9ef4b770e069 31.4 4.9 6.4

worker: allow a function to be run in multiple worker processes If we estimate that it will be worth the cost, we run the function in multiple processes. Otherwise, we run it in-process. Children report progress to the parent through a pipe. Not yet implemented on Windows.

worker: partition a list (of tasks) into equal-sized chunks

worker: estimate whether it's worth running a task in parallel Not implemented for Windows yet.

worker: count the number of CPUs This works on the major platforms, and falls back to a safe guess of 1 elsewhere.

tests: getremove test output changes (fold into previous patch)

merge: report non-interactive progress in chunks Instead of a monotonic count, getupdates yields the number of files it has updated since it last reported, and its caller sums the numbers when updating progress. Once we run these updates in parallel, this will allow worker processes to report progress less often, reducing overhead.

merge: handle subrepo merges and .hgsubstate specially In an upcoming patch, we will update .hgsubstate in a non-interactive worker process. Merges of subrepo contents will still need to occur in the master process (since they may be interactive), so we move that code into a place where it will always run in what will become the master process.

tests: update test output (will be folded into parent)

merge: split out mostly-non-interactive working dir updates In a later patch, we'll run these updates in parallel.

extensions: obsolete and remove interhg extension With the addition of the websub filter extension this extension is no longer needed. We maintain a sort of backwards compatibility by reading the [interhg] section and using it as we would use the [websub] section.

hgweb: apply the websub filter to revision descriptions In order to use this, add a [websub] section to your configuration and add websub expressions such as: italic = s/\b_(\S+)_\b/<i>\1<\/i>/ bold = s/\*\b(\S+)\b\*/<b>\1<\/b>/ issues = s|issue(\d+)|<a href="http://bts.example.org/issue\1">issue\1</a>|i bugzilla = s!((?:bug|b=|(?=#?\d{4,}))(?:\s*#?)(\d+))!<a href="http://bz.selenic.com/\2">\1</a>!i This also adds documentation (proofed by Kevin!) to the config help section.

hgweb: add websub template filter The purpose of this new filter is to make it possible to partially replace the functionality of the interhg extension. The idea is to be able to define regular expression based substitutions on a new "websub" config section. hgweb will then be able to apply these substitutions wherever the "websub" filter is used on a template. This first revision just adds the code necessary to load the websub expressions and adds the websub filter, but it does not add any calls to the websub filter itself on any of the templates. That will be done on the following revisions.

addremove: don't audit the path for paths already in the dirstate Now that dirstate.walk returns None for paths under symlink directories, addremove doesn't need to validate each path it sees to look for files under symlinks. On a large repository this brings addremove from 6.3 seconds down to 3.65 (42%) since addremove no longer has to stat every directory of every file to determine if the file is inside a symlink directory. I put it through our benchmark and see no perf hit to any other commands.

dirstate: walk returns None for files that have a symlink in their path Previously dirstate.walk would return a stat object for files in the dmap that have a symlink to a directory in their path. Now it will return None to indicate that they are no longer considered part of the repository. This currently only affects walks that traverse the entire directory tree (ex: hg status) and not walks that only list the contents of the dmap (ex: hg diff). In a situation like this: mkdir foo && touch foo/a && hg commit -Am "a" mv foo bar ln -s bar foo 'hg status' will now show '! foo/a', whereas before it incorrectly considered 'foo/a' to be unchanged. In addition to making 'hg status' report the correct information, this will allow callers to dirstate.walk to not have to detect symlinks themselves, which can be very expensive.

pathauditor: add check() method The pathauditor currently throws exceptions when it encounters an invalid path. This change adds a method to allow people to treat it as a boolean. This is currently used by scmutil.addremove and in a subsequent patch it will be used by dirstate.walk

summary: add missing space for updated active bookmark display

summary: show active bookmark even if not at current changeset Before this change, 'hg summary' would not show the active bookmark unless it pointed to the working directory parent. After this change, it will show it in parentheses, like so: parent: 18581:f0ff45fe6700 tip summary: simplify handling of active bookmark branch: default bookmarks: [crew] commit: (clean) update: (current)

summary: simplify handling of active bookmark

summary: test that current bookmark isn't shown This exposes the current behavior in a test. A later change will make summary show when the active bookmark has moved out from under us.

merge crew heads

merge with stable

outgoing: fix possible filtering crash in outgoing (issue3814) If there is no outgoiing changesets but we have filtered revision in outgoing.excluded We run into a filtering related crash. The excluded revision should not be there in the first place but discovery need cleanup in default, not stable.

test: display used python hash seed We keep using a random seed for each run, but we "compute" it ourself to be able to reproduce a failed test run.

outgoing: fix possible filtering crash in outgoing (issue3814) If there is no outgoiing changesets but we have filtered revision in outgoing.excluded We run into a filtering related crash. The excluded revision should not be there in the first place but discovery need cleanup in default, not stable.

parsedate: understand "now" as a shortcut for the current time

export: clobber files with -o (bc) (issue3652) This violated user expectation. Updated the code to clobber files, but preserve the behavior of appending multiple patches requested in a single export. Includes tests.

merge: don't call copies.mergecopies unless we need to This reduces the amount of time we spend calculating when doing a clean non-merge update. In a large repo, the time dropped from 10.1 seconds to 3.4.

merge: rename p1 to wctx in manifestmerge This is always a workingctx, and this name is more in line with other functions in this module.

largefiles: fix test and check code

histedit: always abort with same message when working dir is wrong. Using only one message makes live easier for translators. The clearer error message was supposed by Augie Fackler.

histedit: report when revisions to edit are not ancestors of working dir Editing the history only is possible when the working dir is a descendant of the revisions to edit. When this happens explain it by writing abort: %s is not an ancestor of working directory

Merge

manifestmerge: handle abort on local unknown, remote created files This replaces the _checkunknown call in calculateupdates with a more performant version. On a repository with over 150,000 files, this speeds up an update by 0.6-0.8 seconds, which is up to 25%. This does not introduce any UI changes. There is existing test coverage for every case, mostly in test-merge*.t.

manifestmerge: pass in branchmerge and force separately This will be used in an upcoming patch.

manifest: use a size 3 LRU cache to store parsed manifests Previously, the manifest cache would store the last manifest parsed. We could run into situations with operations like update where we would try parsing the manifest for a revision r1, then r2, then r1 again. This increases the cache size to 3 to avoid that bit of performance fragility.

util: add an LRU cache dict In certain cases we would like to have a cache of the last N results of a given computation, where N is small. This will be used in an upcoming patch to increase the size of the manifest cache from 1 to 3.

tests: remove last two check-code warnings about killdaemons Removes the last of the warnings in test-check-code-hg.t introduced in 667063b22a69.

tests: guard against obsolete markers in the hg repo itself If obsolete markers appear in the hg repo (because of enabling evolve), then tests that run hg against the hg repo itself will see warnings like: obsolete feature not enabled but 4 markers found! As far as I can tell, this only occurs in test-check-code-hg.t -- in particular, it will -not- show up on tests that run against test-created repos, as most of the test suite does.

largefiles: make caching largefiles message more explicit In some cases, caching largefiles may take a long time (if the user has pulled a lot of new heads). This patch makes it more clear what is happening, by showing the number of heads we are caching largefiles for.

largefiles: document behavior of caching largefiles for new heads

check-code: add Makefile target to run check-code

byterange: remove now-unused sys import

test-obsolete.t: migrate to killdaemons from kill `cat pidfile`

test-inotify.t: migrate to killdaemons from kill `cat pidfile`

test-inotify-lookup.t: migrate to killdaemons from kill `cat pidfile`

test-inotify-issue1556.t: migrate to killdaemons from kill `cat pidfile`

test-inotify-issue1542.t: migrate to killdaemons from kill `cat pidfile`

test-inotify-issue1371.t: switch to killdaemons from kill `cat pidfile`

test-inotify-debuginotify.t: migrate to killdaemons from kill `cat pidfile`

test-hgweb-raw.t: use killdaemons instead of kill `cat pidfile`

test-https.t: stop using kill `cat $pidfile`

changelog: hexlify node when throwing a LookupError on a filtered node The non-hexlified node was leaking all the way out to the web interface, and wasn't consistent with the behavior for nonexistent nodes.

byterange: remove old two-arg raise trick We don't care about the original traceback, and this was frustrating any efforts to make the code work on Python 3.3.

revlog: document v0 format

merge crew heads

merge with mpm stable

templater: add get() function to access dict element (e.g. extra)

templates: export extra as a dict to templates Currently only useful with the json filter.

Added signature for changeset 7511d4df752e

Added tag 2.5.1 for changeset 7511d4df752e

merge with i18n

i18n-de: fix duplicated shortcut key b for record (edit and quit) Use e for edit instead.

check-code: fix platform-specific error code variance test-check-code-hg.t uses xargs to invoke check-code.py on every file in 'hg manifest'. The return code from xargs varies between BSD xargs and GNU xargs: BSD will return 1 if any invocation exits with an error code; GNU xargs will return 123 in this case. This normalizes the exit code back to 1.

check-code: warn to use killdaemons instead of kill `cat PIDFILE` We have a bunch of tests that still use kill `cat hg.pid` or worse, kill `cat hg.pid`; while kill -0 `cat hg.pid`; sleep 0; done Cleaning these up to use tests/killdaemons.py is non-trivial, so for now we just add a warning.

largefiles: docstrings for verify methods

largefiles: fold oddly named _verify into remotestore.exists

git convert: some versions of git use fatal: instead of error: I saw this behavior with git 1.7.12 on my Mac.

test-convert-git: stabilize corruption of test git repo This error would show up only intermittently since the test depended on the order of the directories returned by os.walk. The damage repository test would delete the first object file it came across. However, the order of the directory listing is arbitrary (it seems to depend on the filesystem). This meant that sometimes a commit object was deleted, sometimes a blob object and sometimes a tree object. So, fix by hardcoding which object to delete. Delete a commit object, a blob object and a tree object in three separate tests.

convert/git: catch errors from modern git-ls-remote (issue3428) Since git v1.7.8.2-327-g926f1dd (the change was first released in git 1.7.10), git does not return non-zero when "git ls-remote --tags ..." is run and the repository is damaged. This causes the "damaged repository with missing commit" test in test-convert-git.t to unexpectedly succeed. Fix by aborting if git outputs any lines beginning with "error:", which required adding some subprocess use in convert/git.py.

merge with stable

incoming: fix incoming when a local head is remotely filtered (issue3805) In its current state discovery may return (remotely) filtered elements in "common". This has usually no impact as "missing" is kept clear of filtered elements. However when the "remote" repo is a local repo (disk accessible, and directly created in memory) the incoming code takes a shortcut and directly uses the "remote" repo to generate the incoming output. When some common elements are filtered this led to a crash. We now ensure we use an unfiltered repository to generate the incoming output. This does not change the behavior as missing is clear of filtered revision. Now that we have proper low level filtering, incoming code needs a deeper cleanup but it is already planned.

dirstate: move pure python dirstate packing to pure/parsers.py

bookmark: don't allow integers as bookmark/branch/tag names Bookmarks/branches/tags shouldn't be allowed to be integers because that overlaps with revision numbers. Right now if a user created one they can't use it anyway because the revision numbers take precedence. The check only happens when creating a new bookmark/etc from a command so it shouldn't affect existing bookmarks/branches/tags or importing branches from git. This fix was prompted by us having a user create a bookmark named "404" then accidentally checkout a very old version of our repository.

run-tests: do not fail on empty tsttest file Initialize n for not failing on empty tsttest files.

merge with stable

hgweb: make 'summary' work with hidden changesets (issue3810) Since the 'summary' view used by e.g. gitweb and monoblue shows both a changelog and a bookmarks list, the same changes are needed here as were made to the 'changelog' and 'bookmarks' web commands (56ca4443a343 and 886936ecc21b, respectively).

merge with stable

hgk: support the old way of getting the current Ttk theme (issue3808) It seems like the API has changed somewhere around 8.5.7, so the preferred way of getting the current theme is now [ttk::style theme use], while the deprecated (but still working) is $::ttk::currentTheme.

addremove: don't perform m.exact/rel until needed Moves the m.exact and m.rel calls within the conditionals they are used in. On a large repo this brings addremove from 7.1 seconds down to 6.3 (13%).

addremove: don't call lexists, isdir, and islink The dirstate walk results contain the stat information for each path, so we don't need to query it again. On a large repo this makes addremove go from 8.35 seconds to 7.1 (15%).

addremove: only query dirstate once per path Previously the addremove code queried the dirstate 4 times per path. Now it only does so once. On a large repo this brings addremove from 9.5 seconds to 8.35 seconds (12%).

tests: fix windows test failure with largefiles

merge with stable

tests: fix windows test failure in test-subrepo.t

merge with stable

largefiles: don't crash when trying to find default dest for url without path b32e55e6c3c7 introduced a crash when cloning a url without path - where util.url().path would be None. This None will now be handled as ''. clone will thus abort with 'repository / not found' as before.

hgweb.cgi: fix internal WSGI emulation (issue3804) The internal WSGI emulation in wsgicgi.py was not fully WSGI compliant and assumed that all responses sent a body. With 3fbdbeab38cc that caused a real bug when using hgweb.cgi. wsgicgi.py will now make sure headers always are sent, using the pattern from PEP 333 and similar to how it is done in c007e5c54b16.

bdiff: simplify overflow checks Rather than check that each delta start, end, and length is within 32 bits, we simply check that the input strings are under 4GB.

Merge crew-stable into crew

rebase: derive node from target rev (issue3802) dest.rev() is the same as target when a new rebase is run, but dest isn't set when rebase --continue is run. Bug introduced in 2a1fac3650a5, which fixed issue3685.

profiling: introduce limit configuration option The number of output lines was hardcoded to 30. There was a 'nested' configuration options that controlled something else related to counting the number of output lines. This introduces the profiling.limit configuration option for controlling the number of profiling output to show.

largefiles: make verify --lfa and --lfc work without --large The slightly obscure --lfa and --lfc only worked as modifiers to --large and could be combined. The documentation was however not clear what they did. Instead they now imply --large and the description is updated.

largefiles: verify status should be written as status, not as write Make 'hg verify --large -q' quiet when no errors.

largefiles: report localstore errors with single line warnings messages The actual hash is also not relevant to report. This makes verify output with multiple failures easier to process.

merge: inline act() The act function had become very trivial and mainly shuffled arguments around and made it harder to see what really was going on.

merge: don't indent "local changed %s which remote deleted" prompt It was usually not shown in a context where indentation helped readability and it was inconsistent with other prompts.

merge: backout manifest sorting and workarounds for cost of it Backout 8a811fa9a9c0 and 760c0d67ce5e which no longer are needed.

merge: delay debug messages for merge actions Show messages at a point where the actions have been sorted, thus preparing for backout of 760c0d67ce5e. This makes manifestmerge more of a silent operation, just like 'copies' is. Indent 'preserving' messages to make them subordinate to the action logging so they fit in the new context. (The 'preserving' messages are quite redundant and could also be removed completely.)

merge: make all action tuples have the same length - keep args as tuple

merge: delay prompts a bit and show them in (extra) sorted order Preparing for backout of 760c0d67ce5e. The number of prompts will for all relevant cases be significantly smaller than the total number of files in the manifests. We can thus afford to sort the prompts more than we can afford to sort the manifests.

commit: show active bookmark in commit editor helper text If there is an active bookmark while committing, the bookmark name will be visible inside the commit message helper, below the branch name. This should make easier for the user to detect a mistaken commit parent, while working for example with a bookmark centric workflow like topic branches. The active bookmark is checked to be in the working directory, as pointed by Kevin Bullock, because otherwise committing would not advance it. In other words, this would not show the active bookmark name if the user changed the working tree parents with 'hg debugsetparents', for example.

dates: support 'today' and 'yesterday' in parsedate (issue3764) Adding support to parsedate in util module to understand the more idiomatic dates 'today' and 'yesterday'. Added unified tests and docstring tests for added functionality.

revset: change ancestor to accept 0 or more arguments (issue3750) Change ancestor to accept 0 or more arguments. The greatest common ancestor of a single changeset is that changeset. If passed no arguments, the empty list is returned.

merge with stable

Added signature for changeset a6088c05e43a

Added tag 2.5 for changeset a6088c05e43a

test-obsolete: now gets 404 for hidden change

merge with i18n

i18n-pt_BR: added message from crew 0324a1d88a53

i18n-pt_BR: added message from crew 9fbeb61b8ad2

i18n-ja: synchronized with 2a1fac3650a5

i18n-pt_BR: synchronized with 68eecbaf1bd3

hgweb: urlescape all urls, HTML escape repo/tag/branch/... names Without this, repository paths or names containing e.g. & characters or html tags yielded strange results, possibly allowing cross-site scripting attacks.

merge with crew

hgweb: rename 'currentbaseline' template keyword to 'basenode' Shorter and clearer. This keyword represents the node we're currently diffing against.

hgweb: rename 'changesetbaseline' template to 'difffrom' More accurately reflects what it will be used for, and is also shorter. This template is used to change which rev the current rev is diff'd against. For example, if you're at '/rev/P1:REV', this would link to a path like '/rev/P2:REV'. Example usage in a template: {parent%difffrom}

hgweb: add a `web.view` to control filtering This options add a new `web.view` to control filter level of hgweb. This option have two purposes: 1) Allow fall back to unfiltered version in case a yet undetected by critical bug is found in filtering after 2.5 release 2) People use hgweb as a local repoviewer. When they have secret changesets, they wants to use "visible" filter not "served" (modified by mpm, documentation deferred)

hgweb: returns 404 for unknow revision instead of 500 I noticed that access to filtered revision returned HTTP 500 code (internal server error). Investigation shown that it was the case for unknown revision too. That wrong and we now properly return a 404 for revision not found.

subrepo: allows to drop courtesy phase sync (issue3781) Publishing server may contains draft changeset when they are created locally. As publishing is the default, it is actually fairly common. Because of this "inconsistency" phases synchronization may be done even to publishing server. This may cause severe issues for subrepo. It is possible to reference read-only repository as subrepo. Push in a super repo recursively push subrepo. Those pushes to potential read only repo are not optional, they are "suffered" not "choosed". This does not break because as the repo is untouched the push is supposed to be empty. If the reference repo locally contains draft changesets, a courtesy push is triggered to turn them public. As the repo is read only, the push fails (after possible prompt asking for credential). Failure of the sub-push aborts the whole subrepo push. This force the user to define a custom default-push for such subrepo. This changeset introduce a prevention of this error client side by skipping the courtesy phase synchronisation in problematic situation. The phases synchronisation is skipped when four conditions are gathered: - this is a subrepo push, (normal push to read-only repo) - and remote support phase - and remote is publishing - and no changesets was pushed (if we pushed changesets, repo is not read only) The internal config option used in this version is not definitive. It is here to demonstrate a working fix to the issue. In the future we probably wants to track subrepo changes and avoid pushing to untouched one. That will prevent any attempt to push to read-only or unreachable subrepo. Another fix to prevent courtesy push from older clients to push to newer server is also still needed.

tests: fix toctou race in tinyproxy.py (issue3795) test-http-proxy.t sometimes failed with: File ".../tests/tinyproxy.py", line 110, in _read_write data = i.recv(8192) error: (104, 'Connection reset by peer') This might have started showing up with a9fd11ffa13f ... but it has apparently also been seen before. I don't see anything in a9fd11ffa13f that can explain it. It seems to be a race in test, in the tinyproxy helper: Tinyproxy found an incoming socket using select(). It would break the loop if an error had been detected on the socket, but there was no error and it tried to recv() from the socket. That failed - apparently because it had been reset after select(). Errors in the recv() will now be caught and will break the loop like errors detected by select() would. (send() could also fail in a similar way ... but using the same solution there and losing data we have read doesn't feel right.)

rebase: mention --rev in the help This changeset adds a small mention of it in the help to prevent confusion. This small addition references online help that is easier to update and improve at release time. Following Wagner Bruna's advice, this is added in a plain new paragraph to not invalidate current translation this close to the release.

hgweb: remove baseline info from paper template The user interface introduced in d605a82cf189 is not considered ready for prime time yet. The internal code stays in place for custom template usage. The feature is ultimately wanted and will be re-enabled soon. The current issue is only related to the visual of the current interface.

rebase: mention phases in the help Mention that Mercurial helps you not do what you've just been warned not to do, with a reference to the 'phases' help topic (not the 'phase' command help). Thanks to Pierre-Yves David <pierre-yves.david@ens-lyon.org> for motivating this change and Wagner Bruna <wagner.bruna+mercurial@gmail.com> for advising on how to do it in an i18n-friendly way.

hgwebdir: use web.prefix when creating url breadcrumbs (issue3790) The web.prefix setting was being ignored when creating the index URL breadcrumbs. We only need to fix hgwebdir and not hgweb because hgweb gets the complete URL request, including the prefix, while hgwebdir gets a "subdir" which does not include the prefix. This fix is slightly different of what was suggested on the bug tracker. In there it was suggested to hide the prefix itself from the breadcrumb. I think that would be a better solution, but it would require changing all the index templates and passing the prefix to the template engine, which may be too big a change for stable during the freeze. For now this fixes the problem, and the fix could be improved during the next cycle.

rebase: delete divergent bookmarks on destination (issue3685) Similar to merge, divergent bookmarks are only deleted when the bookmark is on the destination parent.

bookmarks: factor out delete divergent code Deleting divergent bookmarks is more generally useful than just in bookmarks.update.

rebase: remove bogus nullmerge check in updatebookmarks nstate[v] is a node, not an int, and the nullmerge check was done while building nstate anyway.

share: backout fd903f89e42b, except the test Locating the share source when no default path is available is now handled in subrepo._abssource(), so unconditionally setting a default path (and the associated problems) can be avoided. The test change reflects the fact that a default path is no longer set on the resulting share.

subrepo: use sharepath if available when locating the source repo This is an alternative fix for issue3518, enabling sharing of repositories with subrepos, without unconditionally setting the default path in the resulting repo's hgrc file. Better test coverage is added here, but won't prove this code is working until fd903f89e42b is backed out. The problem with the original fix is, if a default path is not available to be copied over from the share source, the default path on the resulting repo is set to the source location. Since that's where the actual repository is stored, the path is essentially self-referential, so push, pull, incoming and outgoing effectively operate on itself. While incoming and outgoing make it look like nothing was changed, push currently hangs (see issue3657). In this case where there is not a real default path, these operations should abort with "default(-push) not found", like the source repo would. Note this problem with the original fix affected repos without subrepos too.

test-histedit: add tests for dropping head changeset I got bug report from user in this specific case. I was unable to reproduce in test situation. Testing this situation is still valuable.

tests: use `pwd` instead of ${PWD} in test-convert-git.t - because of Solaris

histedit: prevent parent guessed via --outgoing from being a revset (issue3770) If the binary hash of the parent node guessed via --outgoing happened to contain a special revset character (":" was specified in the bug), the revset parser would abort. Hexlifying the node before passing it to the revsingle call should fix that.

tests: fix for windows - slashes and no serve

merge: fix UnboundLocalError (issue3791) A wrong variable name was introduced in 384df4db6520 for a case without test coverage. The variable name is fixed and a test case is introduced.

parsers: fix memleak of revlog cache entries on strip Since 12a852c7c763, raw_length can be reduced on strip, but corresponding cache entries still have refcount. They are not dereferenced by _index_clearcache(), and never freed. To reproduce the problem, run "hg pull" and "hg strip null" several times in the same process.

hgweb: fix navigation label (issue3792) Latest refactoring (6da1e979340a) was buggy and used a variable from another loop. Tests are run on repo too small to cache that.

profiling: add documentation of lsprof 'sort' and 'nested'

OS X: try cheap ascii .lower() in normcase before making full unicode dance This is similar to what is done in encoding.lower, introduced in c481761033bd. This has been seen making 'hg up' and 'hg st' in a 50000+ files repo 13% faster. This might make Mercurial slightly slower for users who mainly use non-ASCII filenames. That is a reasonable trade-off.

run-tests.py: inherit PYTHONHASHSEED from environment if set This makes it possible to fix the seed by using for instance PYTHONHASHSEED=7 ./run-tests.py ... This can be very convenient when trying to debug problems that are influenced by hash values. Try different seed values until you find one that triggers the bad behaviour and then keep that while debugging. The value 0 will restore default Python behavior and disable randomization.

test-obsolete: validate that bundle is not affected by issue3788 Bundle might have been affected by the same kind of error than pull (issue3788). Testing show it is not the case.

pull: fix crash when pulling changeset that get hidden locally (issue3788) When you have obsolescence marker that apply to a pulled changesets, the added changeset is immediately filtered. Then the list of added changeset needs to be build against and unfiltered repo.

hgweb: prevent traceback during search when filtered (issue3783) The search needs to iterate over the repo using changelog.revs like the rest of the Mercurial code.

bookmarks: hide bookmarks on filtered revs from listkeys Don't expose unserved changesets to remote repos. Thanks to Sean Farley <sean.michael.farley@gmail.com> for tracking down the issue and Pierre-Yves David <pierre-yves.david@ens-lyon.org> for the fix.

bookmarks: don't use bookmarks.listbookmarks in local computations bookmarks.listbookmarks is for wire-protocol use. The normal way to get all the bookmarks on a local repository is repo._bookmarks.

filtering: test that bookmarks prevent hiding of changesets

discovery: outgoing pass unfiltered repo to findcommonincoming (issue3776) We noa pass an unfiltered repo in the same way `localrepo.push` does. This does not alter outgoing behavior and prevents possible crash with computing common/missing. The `findcommonincoming` code could be simplified to make this unnecessary, but this is too much change for the freeze.

test: minor documentation fix The related test check push, not pull.

largefiles: fix cat when using relative paths from subdirectory

largefiles: fix commit when using relative paths from subdirectory Remove cwd handling from getstandinmatcher - it did not belong there, as proven by the tests.

largefiles: allow use of urls with #revision largefiles tried to create a peer directly with the specified url. That caused abort: unsupported URL component: "..." if a revision was specified in the url. The branch name do not matter for largefiles' use of remote peers. Largefiles will be shared among all branches anyway.

largefiles: don't verify largefile hashes on servers when processing statlfile When changesets referencing largefiles are pushed then the corresponding largefiles will be pushed too - unless the target already has them. The client will use statlfile to make sure it only sends largefiles that the target doesn't have. The server would however on every statlfile check that the content of the largefile had the expected hash. What should be cheap thus became an expensive operation that trashed the disk and the cache. Largefile hashes are already checked by putlfile before being stored on the server. A server should thus be able to keep its largefile store free of errors - even more than it can keep revlogs free of errors. Verification should happen when running 'hg verify' locally on the server. Rehashing every largefile on every remote stat is too expensive. Clients will also stat lfiles before downloading them. When the server verified the hash in stat it meant that it had to read the file twice to serve it. With this change the server will assume its own hashes are ok without checking them on every statlfile. Some consequences of this change: - in case of server side corruption the problem will be detected by the existing check on the client side - not on server side - clients that could upload an uncorrupted largefile when pushing will no longer magically heal the server (and break hardlinks) - a client will now only upload its uncorrupted files after the corrupted file has been removed on the server side - client side verify will no longer report corruption in files it doesn't have (Issue3123 discussed related problems - and how they have been fixed.)

tests: clarify test for pushing corrupted largefile The test no longer tested that the server prevented pushing a corrupt largefile. At the same time it tested what happened when the server already had a corrupt largefile. These two cases are now separated.

largefiles: verify all files in each revision and report errors in any revision Verify used 'any' and would stop verifying after the first failure in each changeset. The exit code only reported the result from the last changeset.

tests: better test coverage of largefiles localstore verify This demonstrates problems that will be fixed later.

largefiles: adapt remotestore._getfile to batched statlfile 9e1616307c4c introduced batching of statlfile, but not all codepaths got converted. _getfile gave _stat garbage and got garbage back. The garbage didn't match the expected error codes and was thus interpreted as success. It could thus end up trying to fetch a largefile that didn't exist. Instead we now pass _stat valid input and handle both correct and invalid output correctly. This makes the code work as intended ... but it would probably be better if it didn't abort on missing largefiles, just like it happened to do before.

largefiles: don't allow corruption to propagate after detection basestore.get uses util.atomictempfile when checking and receiving a new largefile ... but the close/discard logic was too clever for largefiles. Largefiles relied on being able to discard the file and thus prevent it from being written to the store. That was however too brittle. lfutil.copyandhash closes the infile after writing to it ... with a 'blecch' comment. The discard was thus a silent noop, and as a result of that corruption would be detected ... and then the corrupted files would be used anyway. Instead we now use a tmp file and rename or unlink it after validating it. A better solution should be implemented ... but not now.

largefiles: adapt verify to batched remote statlfile (issue3780) 9e1616307c4c introduced batching of statlfile, but not all codepaths got converted. 'hg verify' with a remotestore could thus crash with TypeError: 'builtin_function_or_method' object is not iterable Also, the 'hash' variable was used without assigning to it. Don't use variable names that collide with Python built-in functions. Instead we use 'expecthash' as in localstore. The tests for this issue covers an untested area. The tests happens to also reveal incorrect attempts at getting non-existing largefiles, bad server side handling of that, and corruption issues - all to be fixed later.

largefiles: let wirestore._stat return stats as expected by remotestore verify - preparing for fixing verify crash.

tests: improve description of hgweb secret bookmarks test Added in 886936ecc21b with only an issue number to describe it.

bookmarks: show active bookmark even if not at working dir If the active bookmark doesn't point at a parent of the working dir (e.g. a pull moved it out from under us), we nonetheless show it as active. This follows on 2096e025a728 in removing the dichotomy (at least in the UI) between "current" and "active" bookmarks.

hgweb: don't attempt to show hidden bookmarks (issue3774) localrepository._bookmarks is unfiltered, but hgweb gets a filtered repo. This fixes the resulting traceback on the 'bookmarks' page.

hgweb: fetch tipmost unfiltered rev thru the changelog This fixes a traceback when tip is filtered (e.g. because it's secret). See issue3783, for which this is a partial fix.

help: update verbose 'clone' help to include '@' bookmark

tests: fix test-help.t for '@' bookmark documentation e031e10cdc06 unexpectedly introduced bookmarks into the results for 'hg help -k clone'. Mea culpa, miserere &c.

help: document '@' bookmark in 'help bookmarks' and 'help clone'

revset: evaluate sub expressions correctly (issue3775) Before this patch, sub expression may return unexpected result, if it is joined with another expression by "or": - "^"/parentspec(): "R or R^1" is not equal to "R^1 or R". the former returns only "R". - "~"/ancestorspec(): "R or R~1" is not equal to "R~1 or R". the former returns only "R". - ":"/rangeset(): "10 or (10 or 15):" is not equal to "(10 or 15): or 10". the former returns only 10 and 15 or grater (11 to 14 are not included). In "or"-ed expression "A or B", the "subset" passed to evaluation of "B" doesn't contain revisions gotten from evaluation of "A", for efficiency. In the other hand, "stringset()" fails to look corresponding revision for specified string/symbol up, if "subset" doesn't contain that revision. So, predicates looking revisions up indirectly should evaluate sub expressions of themselves not with passed "subset" but with "entire revisions in the repository", to prevent "stringset()" from unexpected failing to look symbols in them up. But predicates in above example don't so. For example, in the case of "R or R^1": 1. "R^1" is evaluated with "subset" containing revisions other than "R", because "R" is already gotten by the former of "or"-ed expressions 2. "parentspec()" evaluates "R" of "R^1" with such "subset" 3. "stringset()" fails to look "R" up, because "R" is not contained in "subset" 4. so, evaluation of "R^1" returns no revision This patch evaluates sub expressions for predicates above with "entire revisions in the repository".

test-rebase: add another test for rebase with multiple roots This test the case when a Merge is dropped.

update: update to current bookmark if it moved out from under us (issue3682) If the current bookmark (the one listed in .hg/bookmarks.current) doesn't point to a parent of the working directory, e.g. if it was moved by a pull, use that as the update target instead of the tipmost descendent. A small predicate is (finally) added to the bookmarks module to check whether the current bookmark is also active.

test-bookmarks-pushpull.t: don't set bookmark active unnecessarily The test in question doesn't have anything to do with having an active bookmark. This change makes the test change the two bookmarks it affects without making them active. It clears the way for adding a test for updating to an active bookmark that moved out from under us.

doc: use "tag" revset predicate instead of "tagged" for example in help "tag" predicate is officially described in help, but "tagged" is not, even though the latter works as same as the former.

tests: add regression tests for another revrange edge case These tests would've passed before ac0c12123743 and e441657b372b. Inserting them to make sure that continues to be the case.

graphmod: don't try to visit nullrev (issue3772)

log: remove any ancestors of nullrev (issue3772) For the special case, ":null" we remove the implied revision 0 since that wouldn't make any sense here. A test case is added to make sure only nullrev is shown.

help: add documentation for new template functions

changectx: fix the handling of `tip` We can not use `len(repo,changelog)`, it may be a filtered revision. We now use `repo,changelog.tip()` to fetch this information. The `tip` command is also fixed and tested Thanks goes to Idan Kamara for the initial report.

bisect: use changelog for iteration With changelog filtering, we can not use xrange anymore. We have to use the changelog to do the iteration. This way, the changelog excludes filtered revision and we can safely use what we iterate over. Without this changes, bisect crash with a traceback if there is filtered revision in the repo. Tests have been updated.

documentation: update to new filter names Changeset f3b21beb9802 change filter names but forgot some documentation updates.

largefiles: enhance error message to make it more i18n-friendly

merge with i18n

largefiles: fix largefiles+subrepo update (issue3752) Override updaterepo() instead of individual methods that may not be called for each subrepo. Add test. Based on patch from Matt Harbison. Changes the order of update-related messages (now largefiles comes before the global status).

debugsuccessorssets: fix typos in docstring

i18n-pt_BR: synchronized with f5fbe15ca744

merge: only sort manifests in debug mode (issue3769)

Added signature for changeset f5fbe15ca744

Added tag 2.5-rc for changeset f5fbe15ca744

merge default into stable for 2.5 code freeze

pathencode: don't use alloca() for safety/portability

branchmap: display filtername when `updatebranch` fails to do its jobs We have a very handy assert at the ends of `branchmap.updatecache` that check the resulting branchmap is actually valid. I know we do not like assert in mercurial but this one is very handy for debugging. There is really not reason for `branchmap.updatecache` to have this kind of issue but this happened and handful of time during the development of this or introduction of other related feature. I advice to keep it around until we are a bit more confident with the new code.

scmutil: localize and improve 'not under root' message

run-tests.py: don't let hg run interactively in debug mode In normal test mode stdin is closed and hg is thus not interactive. In --debug mode stdin is inherited from the running console and to the tests, and hg could thus wait in prompts when running on Windows. See http://selenic.com/pipermail/mercurial-devel/2013-January/047548.html . Instead set ui.interactive=False to make Mercurial non-interactive. Other commands might still work differently in the --debug environment. This should solve the problem with hg waiting for input but still make it possible to add --debugger to hg in a test and run run-tests.py with --debug.

run-tests.py: backout "don't use console for stdin when running in debug mode" f5842787a958 caused that some kind of interactive debugging no longer was possible - such as running hg with --debugger in a test run with run-tests.py --debug .

rebase: properly handle unrebased revision between rebased one With rebase taking multiple roots it is possible to have revision in the "rebase domain" not rebased themself. We do not want rebased revision above them to be detached. We want such revision to be rebased on the nearest rebased ancestors. This allows to preserve the topology of the rebase set as much a possible To achieve this we introduce a new state `revignored` which informs `defineparents` of the situation. The test in `test-rebase-obsolete.t` was actually wrote and his now fixed.

rebase: lose the comparison to `nullmerge` For a proper behavior of the `--rev` revision we will need another possible state for revision ignored by rebase. We alter the comparison to `nullmerge` to match this future lower state too.

repoview: cache filtered changelog Creating a new changelog object for each access is costly and prevents efficient caching changelog side. This introduced a x5 performance regression in log because chunk read from disk were never reused. We were jumping from about 100 disk read to about 20 000. This changeset introduce a simple cache mechanism that help the last changelog object created by a repoview. The changelog is reused until the changelog or the filtering changes. The cache invalidation is much more complicated than it should be. But MQ test show a strange cache desync. I was unable to track down the source of this desync in descent time so I'm not sure if the issue is in MQ or core. However given the proximity to the 2.5 freeze, I'm choosing the inelegant but safe route that makes the cache mechanism safer.

rebase: do not invent successor to skipped changeset When rebase results in an empty a changeset it is "skipped" and no related changeset is created at all. When we added obsolescence support to rebase (in fc2a6114f0a0) it seemed a good idea to use its parent successor as the successors for such dropped changesets. (see old version of the altered test). This option was chosen because it seems a good way to hint about were the dropped changeset "intended" to be. Such hint would have been used by automatic evolution mechanism to rebase potential unstable children. However, field testing of this version are not conclusive. It very often leads to the creation of (totally unfounded) evolution divergence. This changeset changes this behavior and mark skipped changesets as pruned (obsolete without successors). This prevents the issue and seems semantically better probably a win for obsolescence reading tool. See example bellow for details: User Babar has five changesets of interest: - O, its current base of development. - U, the new upstream - A and C, some development changesets - B another development changeset independent from A O - A - B - C \ U Babar decides that B is more critical than the A and C and rebase it first $ hg rebase --rev B --dest U B is now obsolete (in lower case bellow). Rebase result, B', is its successors.(note, C is unstable) O - A - b - C \ U - B' Babar is now done with B', and want to rebase the rest of its history: $ hg rebase --source A --dest B' hg rebase process A, B and C. B is skipped as all its changes are already contained in B'. O - U - B' - A' - C' Babar have the expected result graph wise, obsolescence marker are as follow: B -> B' (from first rebase) A -> A' (from second rebase) C -> C' (from second rebase) B -> ?? (from second rebase) Before this changeset, the last marker is `B -> A'`. This cause two issues: - This is semantically wrong. B have nothing to do with A' - B has now two successors sets: (B',) and (A',). We detect a divergent rewriting. The B' and A' are reported as "divergent" to Babar, confusion ensues. In addition such divergent situation (divergent changeset are children to each other) is tricky to solve. With this changeset the last marker is `B -> ø`: - This is semantically better. - B has a single successors set (B',) This scenario is added to the tests suite.

repoview: protect `base` computation from weird phase root If for some reason the phase roots contains nullid, the set of filtered revs will contains -1. That confuse Mercurial a lot. In particular this corrupt the branchcache. Standard code path does not result in nullid phase root. It can only result from altered `.hg/store/phaseroots` or buggy extension. However better safe than sorry.

posix: don't compare atime when determining if a file has changed A file's atime might change even if the file itself doesn't change. This might cause us to invalidate caches more often than necessary. Before this change, hg add often resulted in the dirstate being parsed twice on systems that track atime. After this change, it is only parsed once. For a repository with over 180,000 files, this speeds up hg add from 1.2 seconds to 1.0.

hg: replace DirCleanup class with normal try/finally use

histedit: proper phase conservation (issue3724) Before this changeset, histedit created all new changesets according phases.new-commit option without any regards for the phases of the original changesets. This changeset fix that using the phase of rewritten changeset to decide the phase of the resulting changeset. In case of reordering or folding, we keep secret item secret as it seems the safer path. temporary commit creation are not affected. They are head only and stripped at the end of the histedit. As for the resolution of issue3681 (obsolescence cycle prevention), we do not handle changesets created by edit command.

test-histedit: reorder phases test and prepare for more We are going to add a lot regarding phase of test while fixing issue3724. This movement allows to put them after this first phase test.

test-histedit: fix instability creation test The current test does not rewrite anything and therefor does not create any instability. We also clean up the repo state after the test. This required the rebase extension.

histedit: record histedit source (issue3681) Have histedit record the hex of the original changeset as already done by: - graft - commit --amend - rebase My main motivation for adding this is to prevent the creation of obsolescence cycle (see issue3681). Note that commit created during edit are not affected yet.

histedit: factor most commit creation in a function Later commits add two important items to histedit: - Obsolescence cycle prevention - Proper phase conservation Those logics must be applied to all simple operations (pick, edit, mess) and will require verbose code. So we introduce a new function that will provide an entry point for this new. logic. The function build a closure to have a clear distinction between commit arguments and data provided to the function to fulfil its logic.

store: switch to C-based hashed path encoding

pathencode: implement both basic and hashed encoding in C

pathencode: implement hashed encoding in C This will be used by an upcoming patch.

pathencode: implement the "mangling" part of hashed encoding in C This will be used by an upcoming patch.

pathencode: add a SHA-1 hash function This will be used by an upcoming patch. This calls out to the Python hash implementation. An earlier version of this function implemented SHA-1 directly, but the amount of extra code didn't seem like a good tradeoff compared to the small big-picture increase in performance (long paths are uncommon).

store: implement lowerencode in C

hgweb: run with "served" filter This changeset enable the "served" filter on all repo used by hgweb. Hgweb misbehave in a lot of when filtering changeset are present but I do not expect normal people to have secret or obsolete changeset on they server. Misbehavior will be gradually fixed later.

hgweb: walk the graph through the changelog This is necessary to enforce filtering. The result is a bit buggy (may provide less changeset than expected, but it will stop crashing on filtered revision access. Note that changelog.revs can not represents empty iteration like xrange did. So we have to explicitly prevent call when there is nothing to do.

hgweb: use changelog for iteration Iterating through changelog is necessary to enforce filtering.

hgweb: ignore filtered revision in revnav This changeset checks that a revision is known before adding it to the navigation. This will prevent traceback on filtered repository. This changeset result in an incorrect behaviors, Navigation link may be dropped without any replacement. However this bad navigation generation is much better than a crash

hgweb: generate revnav in two phase We first generate revision to link. Then we dispatch them in before, after list. This will help filtering of unknown revision.

rebase: support multiple roots for rebaseset We have all the necessary mechanism to rebase a set with multiple roots, we only needed a proper handling of this case we preparing and concluding the rebase. This changeset des that. Rebase set with multiple root allows some awesome usage of rebase like: - rebase all your draft on lastest upstream hg rebase --dest @ --rev 'draft()' - exclusion of specific changeset during rebase hg rebase --rev '42:: - author(Babar)' - rebase a set of revision were multiple roots are later merged hg rebase --rev '(18+42)::'

clfilter: stronger detection of filtered changeset in changectx.__init__ We previously let some IndexError spill out of this function. A new tests is added to check the command that spotted the error.

validate: check for spurious incoming filelog entries Accepting those will lead to "mild corruption", correctly reported as an error by hg verify, but often not a problem in practice. Enabled when server.validate is switched on.

zsh_completion: add descriptive branch names to head revisions

zsh_completion: add completion of branch names

zsh_completion: fix issue with overlong branch/tag names

bundlerepo: remove old style check of file names The old check is no longer needed: The checked paths will be passed on to filelog.__init__ which will prefix the path with 'data/'. A leading '/' will thus not do any harm. And: The paths will be used by an opener which will use a pathauditor. And finally: The old check did not consider Windows paths and was thus insufficient.

bundlerepo: use revlog._checkhash The error message will mention datafile instead of indexfile.

bundlerepo: store validated deltabase rev in basemap instead of node This avoids some lookups and aligns bundlerepo more with what revlog do.

bundlerepo: use rev instead of node for iteration in revision() Avoids some lookups and avoids using rev while iterating.

bundlerepo: inline inbundle() It didn't make it faster, it didn't make the code smaller, it didn't make the code simpler. -1 is never in basemap.

bundlerepo: drop bundlebase wrapper function for basemap

bundlerepo: remove unused disktiprev attribute

bundlerepo: improve performance for bundle() revset expression Create the set of revision numbers directly instead of creating a list of nodes first.

bundlerepo: fix outdated comment Comment was made invalid by 01ee43dda681.

hgweb: pass repo object to revnav construction For compatibility with changelog filtering we need access to the changelog, a simple nodefunc is not sufficient, only the changelog and repo have access the filteredrevs information. For the filerevnav version, we use an unfiltered changelog. Linkrev is currently broken with filtering and we need some failsafe to prevent traceback. This is the same approach as the one used in 518c1403838f. The use of filectx.changectx() allowed the previous code to use the 518c1403838f hack. This changeset may result in an incorrect behaviors, Navigation link may point to missing revision. However this bad navigation generation is much better than a plain crash

hgweb: introduction a filerevnav subclass It'll be use to implement the file specific behavior.

hgweb: simplify addition of "(0) navigation entry"

hgweb: simplify the handling of empty repo This abstraction have two advantages: - If the revlog is empty, None of the code bellow is relevant, early returns seems a win. - Abtraction of the 'emptiness' check will help later when we stop relying on nodefunc. A bonus, with filtering, a non-empty revlog may not have '0' revision accessible. It'll be easier to handle with the emptiness test in a dedicated function

hgweb: move hex creation into an object method This is clearer and allow later overwrite.

hgweb: pass nodefunc to the revnav object The issue between hgweb and filtering lay in this function. Moving it into the object itself helps to abstract the erroneous bit.

hgweb: move revnavgen into an object For later compatibility with changelog filtering some part of the navigation generation logic will be altered. Those altered part will be different when in the changelog case and in the filelog case. Moving this into an object will allow to use inheritance to override just the part of the logic we need. The aimed logic are for example: - generation of revision 'hex' (different logic for changelog and filelog) - revlog emptyness test - fetching of the first revision of a revlog (may not be 0)

hgweb: `limit` argument is actually `latestonly` renames and enforce The `limit` argument of several generator have only two possible values in practice: 0 and 1. We rename this parameter to `latestonly` and simplify it's handling. The simplification allows us to save fetching of data that we are sure to not consume. Having a function minimal function perimeter will helps future refactoring.

hgweb: add test regarding filtering and hgweb Since 4d1671b39168 hgweb is broken with filtering. This changeset add test that should pass once it is fixed. Test currently broken are commented and will be uncommented by changeset that fix them. The filelog test is currently passing because we already have some hack in core regarding filelog (see 518c1403838f).

get-with-headers: add a --headeronly switch In some case we do not care about the actual rendering.

tests: more comment cleanup related to obsolescence

test: fix in-test comments related to obsolescence The `ui.prevent-unstable` option never made it into core. It always behaves this way when obsolescence feature is enabled. See changesets c5bd753c5bc6, bacf55bd8f90 and 0f5a0a2073a8 for details.

stream_in: write the remote branchcache for a usage as wide as possible Writing cache for unfiltered repo only is barely useful, Most repo user are now at least use the `hidden` filter. This changeset now assigns the remote cache for a filter as low as possible for a wider reuse as possible.

destroyed: update `unserved` branchcache instead Before this changesets the `destroyed` function updated the branchcache for unfiltered repository. As seen in a previous changeset, Read only repo does not cares about the unfiltered repo. We now update it for `unserved`.

destroyed: drop complex branchcache rebuilt logic The strip code used a trick to lower the cost of branchcache update after a strip. However is less necessary since we have branchcache collaboration. Invalid branchcache are likely to be cheaply rebuilt again a near subset of the repo. Moreover, this trick would need update to be relevant in the now filtered repository world. It currently update the unfiltered branchcache that few people cares about. Make it smarter on that aspect would need complexes update of the calling logic So this mechanism is: - Arguably needed, - Currently irrelevant, - Hard to update and I'm dropping it. We now update the branchcache in all case by courtesy of the read only reader. This changeset have a few expected impact on the testsuite are different cache are updated.

branchmap: update cache of 'unserved' filter on new changesets The `commitctx` and `addchangegroup` methods of repo upgrade branchcache after completion. This behavior aims to keep the branchcache in sync for read only process as hgweb. See ee317dbfb9d0 for details. Since changelog filtering is used, those calls only update the cache for unfiltered repo. One of no interest for typical read only process like hgweb. Note: By chance in basic case, `repo.unfiltered() == repo.filtered('unserved')` This changesets have the "unserved" cache updated instead. I think this is the only cache that matter for hgweb. We could imagine updating all possible branchcaches instead but: - I'm not sure it would have any benefit impact. It may even increase the odd of all cache being invalidated. - This is more complicated change. So I'm going for updating a single cache only which is already better that updating a cache nobody cares about. This changeset have a few expected impact on the testsuite are different cache are updated.

tests: make test-hgweb.t output stable Instability introduced in combination of a4d7fd7ad1f7 and e389a25e7e60.

hgweb: ensure _navseq yield strictly increasing numbers This is not hard and allows to drop hack in the customer code.

hgweb: better names for _navseq arguments The old names were misleading.

hgweb: drop recursivity in _navseq This is totally not needed.

hgweb: move the `seq` function out of the revnavgen scope There is not reason for it to be a in there. And this function could use a major reworks.

hgweb: simplify return value creation in for navgen We now have access to better syntax allowing a clearer version.

hgweb: do not access first changeset with a string There is not reason not to use an int

refactoring: use unlinkpath with ignoremissing

docs: "deprecate" checkchanged and checkconflicts in merge-tool configuration These settings were replaced by check=changed and check=conflicts in a912f26777d3. There is no reason to announce two different ways to achieve the same. The old way should be kept but not announced.

tests: run with PYTHONHASHSEED=random Python set and dict iteration order is in principle undefined but usually 'quite stable'. Setting PYTHONHASHSEED=random will make the iteration order more random in Python 2.6.8 and 2.7.3 and where it has been backported. This can thus help spot dependencies on undefined behaviour and prevent future problems.

run-tests: exit cleanly if parallel run is interrupted If interrupted while running with "--jobs N", run-tests asynchronously spewed a bunch of output and backtraces from both the master and slave processes, leaving the terminal full of goop. This patch makes it behave more sensibly.

filtering: rename filters to their antonyms Now that changelog filtering is in place, it's become evident that naming the filters according to the set of revs _not_ included in the filtered changelog is confusing. This is especially evident in the collaborative branch cache scheme. This changes the names of the filters to reflect the revs that _are_ included: hidden -> visible unserved -> served mutable -> immutable impactable -> base repoview.filteredrevs is renamed to filterrevs, so that callers read a bit more sensibly, e.g.: filterrevs('visible') # filter revs according to what's visible

clone: don't include empty revlogs in stream

serve: don't send any content headers with 304 responses Fixes HTTP protocol violation introduced in cf5c76017e11. 'hg serve' would show a stacktrace when loading pages that not had been modified. There was test coverage for this, but the wrong response headers wasn't shown and thus not detected.

tests: fix doctest stability over Python versions pprint ain't pretty in Python 2.4: Changed in version 2.5: Dictionaries are sorted by key before the display is computed; before 2.5, a dictionary was sorted only if its display required more than one line, although that wasn’t documented. Fixes issue introduced in 404feac78b8a.

tests: stabilize doctest output Avoid dependencies to dict iteration order.

tests: make test-hook.t output more stable

convert: make toposort order stable when multiple orderings are possible The output of some tests are changed. Not necessarily to the better, just other valid permutations.

convert: report cvsps branchpoints sorted

convert: process subversion branch in a sorted order

convert: process missing branches in sorted order

convert: process splicemap in sorted order

mq: stabilize update after strip of parent revision Strip will (if it updates) update to the parent of revs[0], where revs are the roots of the tree that is stripped. When revs was list(set) it was thus undefined which root parent it would update to. With sorted(set) it is at least stable what it updates to. (But it is very possible that another more useful and predictable behaviour could be defined ... such as updating to the tip-most surviving wd ancestor.)

histedit: process bookmarks in sorted order

churn: sort users with same churn by name This makes the output order well-defined and improves code readability.

largefiles: upload files in sorted order

hgweb: generate query strings with parameters sorted by key

identity: report bookmarks sorted

obsolete: process markers in a stable order Using set iteration order gave unstable debugsuccessorssets output with PYTHONHASHSEED=random.

subrepos: process subrepos in sorted order Add sorted() in places found by testing with PYTHONHASHSEED=random and code inspection. An alternative to sprinkling sorted() all over would be to change substate to a custom dict with sorted iterators...

bookmarks: process pulled remote bookmarks in sorted order

copies: report found copies sorted

discovery: process heads in sorted order

merge: process files in sorted order

commandserver: report capabilities sorted

bisect: store state sorted

localrepo: store branchheads sorted

localrepo: store requirements sorted

copies: make the loss in _backwardcopies more stable A couple of tests shows slightly more correct output. That is pure coincidence.

serve: use chunked encoding in hgweb responses 'hg serve' used to close connections when sending a response with unknown length ... such as a bundle or archive. Now chunked encoding will be used for responses with unknown length, and the connection do thus not have to be closed to indicate the end of the response. Chunked encoding is only used if the length is unknown, if the connection wouldn't be closed for other reasons, AND if it is a HTTP 1.1 request. This will not benefit other users of hgweb ... but it can serve as an example that it can be done.

serve: remove connection close hack for Python 2.3 Introduced in 276de216d2c5 and no longer needed - BaseHTTPServer handles connection closing just fine if we don't mess with its internals.

hgweb: pass the actual response body to request.response, not just the length This makes it less likely to send a response that doesn't match Content-Length.

hgweb: don't pass empty response chunks on hgweb internals will often produce empty writes - especially when returning compressed data. hgweb is no middleware application and there is thus no reason to pass them on to be processed in other layers.

hgweb: remove handling of any else than strings from request.write Iterators should be returned WSGI style, not written. And apparently all of hgweb do that.

serve: send response headers even if response has no body The headers would usually be sent anyway because the app did a number of writes of empty strings.

hgweb: simplify wsgirequest header handling Remove leaky header abstraction and prepare for other encodings.

hgweb: make type a mandatory parameter to request.respond There will thus always be headers and the runtime check can be removed.

hgweb: use Content-Length for pushres This prevents some unnecessary http connection close.

hgweb: send Content-Length 0 for zero length response Before, Content-Length wasn't sent for 0 length responses. Now it is. This could in principle prevent some unnecessary http connection close.

archival: tarit should never close the dest passed to it Some archive types closed the open file passed to it, some didn't. This could cause either missing or duplicate close and cause problems in hgweb. The fix in 14f3795a5ed7 should only have closed the compressors and archivers - not the underlying file itself if no compressor is used.

mq: checktoppatch should only check if p1 is qtip There is no way qtip in p2 could be used for anything, and there is thus no reason to check and accept it.

mq: fix qpop of working directory parent patch when not at qtip mq assumed that it had to update from qtip to qparent, and instead of updating from where it was it failed with: abort: working directory revision is not qtip

largefiles: make log match largefiles in the non-standin location too Yet another match hack.

log: make log work even if first parameter doesn't exist A slowpath optimization kicked in too often because of wrong indentation.

merge: remove "case" comments The comments introduced in f05c182430a0 seems important ... but the context has apparently been lost and they do not have any value now.

merge: merge file flags together with file content The 'x' flag and the 'l' flag are very different. It is usually not a problem to change the 'x' flag of a normal file independent of the content, but one does not simply change the type of a file to 'l' independent of the content. This removes the fmerge function that merged both 'x' and 'l' independent of content early in the merge process. This correctly introduces some conflicts instead of silent incorrect merges. 3-way flag merge will now be done in the resolve process, right next to file content merge. Conflicts can thus be resolved with (slightly inconvenient) resolve commands like 'resolve f --tool internal:other'. It thus brings us closer to be able to re-solve manifest merge after the merge and avoid prompts during merge. This also removes the "conflicting flags for a - (n)one, e(x)ec or sym(l)ink?" prompt that nobody could answer and that made it easy to mix symlink targets and file contents up. Instead it will give a file merge where a sufficiently clever merge tool can help resolving the issue.

tests: better test coverage of merges of flags This makes existing problems with merges of symlinks and files more obvious and add test coverage for tricky merges without real common ancestors.

merge: remove old pre-audit code checking for absolute paths Audit will handle this in a more elegant way.

merge: drop reference to file contents immediately after write Like 7d2aaeea67ed this reduces memory usage on large merges.

merge: changing the mode of a file is also an update It was a change in the file system that wasn't reported in the summaries.

merge: use util.unlinkpath for removing moved files - more like how removed files are removed.

merge: .hgsubstate is special as merge destination, not as merge source

merge: remove redundant unlink after merge The early prescan for move/remove and removal of moved files in applyupdates was introduced with mergestate 368a4ec603cc and rendered this chunk of code irrelevant. The impact of the chunk was reduced in 5b3383ea67d2 - but it could have been removed completely.

merge: rename list of actions from action to actions

merge: consistently use "x" instead of 'x' for internal action types This makes it simpler to search for places where the action types are handled.

merge: consistently use repo.wopener.audit instead of creating a new auditor

scmutil: simplify vfs.audit - drop wrapped vfs.auditor

util: copyfile: remove dest before copying This prevents spurious problems writing to locked files on Windows.

merge: warn when internal:merge cannot merge symlinks A follow-up to d084df89d948. internal:merge should never be picked for merging symlinks ... but in the test suite we have HGMERGE="internal:merge" which bypasses all the usual merge-tool cleverness. Without any output it can be hard to figure out what happened and where the problem is.

merge with stable

histedit: correct the number of added revisions in online help In the context of help document on which this patch focuses, the example repository, which is source of cloning, should be already histedit-ed, and contain only 3 revisions (rev # 0 to 2). So, not 3, but 4 revisions should be added to the destination repository of cloning, if it contains 7 revisions (rev # 0 to 6). This patch also adds modifier "histedit-ed" to "example repository", to make context clear.

histedit: correct changeset IDs in online help There is no '633536316234' revision in the example repository. It should be 'c561b4e977df', according to the revisions in it and explanation in help document.

convert: correct 'hooks' section name in online help The section name for hooks is not 'hook', but 'hooks'.

hgweb: document the revnavgen function

hgweb: no do not use listinsert(0, ...) This is not efficient. We now append element and either reverse the list or iterate in reverse order.

localrepo: drop unnecessary check on wlock unlock Calling out to dirstate one line before guarantees that it'll be in the filecache.

dirstate: refresh _branch cache entry after writing it

filecache: create an entry in _filecache when __set__ is called for a missing one Preserve the invariant that if P is a filecached property on X then P in X.__dict__ => P in X._filecache. Previously, it was possible for a filecached property to become out of sync with the filesystem if it was set before getting it first, since the initial filecacheentry was created in __get__. Old behaviour: repo.prop = x repo.invalidate() # prop has no entry in _filecache, it's not removed # from __dict__ repo.prop # returns x like before without checking with the # filesystem New: repo.prop = x # an empty entry is created in _filecache repo.invalidate() # prop is removed from __dict__ repo.prop # recreates prop

filecache: allow filecacheentry to be created without stating in __init__ Will be used for properties that are set without getting them first.

rollback: don't clear the filecache This was an old workaround to force the filecache to reload everything. Now that the syncing issue is fixed, we no longer need it.

destroyed: keep the filecache in sync with __dict__ (issue3335) (issue3693) (issue3743) We need to make sure that if X is in the filecache then it's also in the filecache owner's __dict__, otherwise it will go out of sync: repo.X # first access to X, records stat info in # filecache and updates __dict__ repo._filecache.clear() # removes X from _filecache but it's still in __dict__ repo.invalidate() # iterates over _filecache and removes entries # from __dict__, but X isn't in _filecache, so # it's kept in __dict__ repo.X # X is fetched from __dict__, bypassing the filecache

localrepo: write the phasecache when destroying nodes

localrepo: update the branchmap when destroying nodes This was previously called directly during strip. Moving it to destroying also means that it'll be called through _rollback, which seems harmless.

localrepo: introduce destroying function

localrepo: don't refresh filecache entries that aren't in __dict__ We call invalidate to remove properties from __dict__ because they're possibly outdated and we'd like to check for a new version. Next time the property is accessed the filecache mechanism checks the current stat info with the one recorded at the last time the property was read, if they're different it recreates the property. Previously we refreshed the stat info on all properties in the filecache when the lock is released, including properties that are missing from __dict__. This is a problem because: l = repo.lock() repo.P # stat info S for P is recorded in _filecache <changes are made to repo.P indirectly, e.g. underlying file is replaced> # P's new stat info = S' l.release() # filecache refreshes, records S' as P's stat info At this point our filecache contains P with stat info S', but P's version is from S, which is outdated. The above happens during _rollback and strip. Currently we're wiping the filecache and forcing everything to reload from scratch which works but isn't the right solution.

changelog: please check-code and remove tabs Tabs were introduced in 06185554e7e3.

branchmap: Save changectx creation during update The newly introduced `branchmap` function allows us to skip the creation of changectx objects. This speeds up the construction of the branchmap. On the mozilla repository (117293 changesets, 15490 mutable) Before: ! impactable 19.9 ! mutable 0.576 ! unserved 3.16 After: ! impactable 7.03 (2.8x faster) ! mutable 0.352 (1.6x) ! unserved 1.15 (2.7x) On the cpython repository (81418 changesets, 6418 mutable) Before: ! impactable 15.9 ! mutable 0.451 ! unserved 0.861 After: ! impactable 6.55 (2.4x faster) ! mutable 0.170 (2.6x faster) ! unserved 0.289 (2.9x faster) On the pypy repository (58852 changesets) Before: ! impactable 13.6 After: ! impactable 6.17 (2.2x faster) On my Mercurial repository (18295 changesets, 2210 mutable) Before: ! impactable 23.9 ! mutable 0.368 ! unserved 0.057 After: ! impactable 1.31 (18x faster) ! mutable 0.042 (8.7x) ! unserved 0.025 (2.2x)

changelog: add a `branch` method, bypassing changectx The only way to access the branch of a changeset is currently to create a changectx object and access its `branch()` method. Creating a new Python object is costly and has a huge impact on code doing heavy access to `branch()` (like branchmap). This change introduces a new method on changelog that allows direct access to the branch of a revision. See the next changeset for impact.

branchmap: pass revision insteads of changectx to the update function Creation of changectx objects is very slow, and they are not very useful. We are going to drop them. The first step is to change the function argument type.

perf: add perfbranchmap command The command times the update of a branchmap from its nearest subset or from scratch.

clfilter: enforce hidden filtering on all repository accesses We ensure all repositores created through `mercurial.hg.repository` are "hidden" filtered. This is an even stronger enforcement than 5bb610f87d1d. Citing Matt's response to changeset 5bb610f87d1d installing filtering in dispatch: > Unfortunately, this means that code that doesn't go through dispatch (ie all > those crazy misguided people using Mercurial as a library) are going to see > these hidden changesets. > > Might be better to instead install the filter in localrepo construction by > default and disable it in dispatch.

archival: avoid touching deprecated gzip name attribute The existing workaround didn't work when no filename was specified. If running in a context with warnings enabled and subsecond mtime it gave a warning: DeprecationWarning: use the name attribute

archival: pass integer to struct.pack int field instead of float If running in a context with warnings enabled and subsecond mtime it gave a warning: DeprecationWarning: integer argument expected, got float

largefiles: fix update from a merge with removed files A situation with this case could happen after interrupting an update. Update would fail with: abort: No such file or directory: $TESTTMP/f/.hglf/sub2/large6 Update from a merge without using clean is not possible anyway, so this patch takes a step in the right direction so it gets as far as reporting the right error.

largefiles: fix revert removing a largefile from a merge Before revert could fail with: abort: .hglf/large@33fdd332ec64: not found in manifest! The LookupError will now be caught and handled correctly.

largefiles: remove unused proto.refuseclient code Should have been removed with other pre-1.9 code in 7c604d8c7e83.

subrepo: make 'in subrepo' string easier to find by external tools This patch is meant to make it easier for tools that wrap the mercurial output (such as TortoiseHg) to find the "in subrepo MYSUBREPO" string that (since 9e3910db4e78) is appended after subrepo error messages, particularly when the mercurial output is translated to a non-English language. The message remains the same but the '%s' that was used to prepend the original error message in front of the 'in subrepo' string has been moved out of the translatable string. As an example of the usefulness of making it easy to look for "in subrepo MYSUBREPO" strings, TortoiseHg looks for these strings in error messages in order to "linkify them" (i.e. convert "MYSUBREPO" into alink to the corresponding subrepo). The original string made it hard for a tool such as TortoiseHg to look for the translated string on mercurial's output because the translated string contained the error message itself. This meant that a regular expression was required to ignore the error message part. With this change TortoiseHg can just get the translated "(in subrepo %s)" string, substitute %s for the subrepo path (which it gets from the subrepo exception) and simply search for the resulting string (no regular expression needed, or at least a much simpler regular expression could be used). Additionaly, the existing string could lead a translator mistakenly assume that it was possible invert the order of the %s (error and subrepo path) fields, which would not work because the string interpolation was position based.

subrepo: fix python2.4 compatibility after 9aa6bee6e9f9 super(SubrepoAbort, self).__init__(*args, **kw) raises TypeError: super() argument 1 must be type, not classobj

test-keyword: improve grammar and spelling in branchcache note See: da9e544c69d6

Merge with stable.

repoview: extract hideable revision computation in a dedicated function This will help extensions to plug into the hidden mechanism.

largefiles: make update with backup files in .hglf slightly less broken Largefiles update would try to copy f to f.orig if there was a .hglf/f.orig . That is in many many ways very very wrong, but it also caused an abort if f didn't exist. Now it only tries to copy f if it exists.

tests: fix up test-highlight for breadcrumb changes

color: add template label function

templater: add no-op template function 'label'

posix: fix split() for the case where the path is at the root of the filesystem posixpath.split() strips '/' from the dirname *unless it is the root*. This patch reproduces this behavior in posix.split(). The old behavior causes a crash when creating a file at the root of the repo with localrepo.wfile() when the repo is at the root of the filesystem.

record: remove unused import

cvsps: fix indentation

hghave: introduce a test (unused) for cvs >= 1.12

discovery: drop the visibleheads function It has no users left. (Using filtering explicitly is easier.)

clfilter: remove the last usage of `visibleheads` We can just inline the filtering now that all server code uses changelog filtering directly.

discovery: drop the visiblebranchmap function

clfilter: drop extra filtering in wireprotocol The repository used by wireprotocol is already filtered. We no longer need to call special functions.

clfilter: filter "unserved" on all wireprotocol command calls This ensures that unserved changesets are not exposed through the wire protocol.

clfilter: drop extra filtering in localpeer The repository used by localpeer is already filtered. We no longer need to call a special function.

clfilter: make localpeer use a repo with "unserved" filter This changeset installs a broad filter on most repos used for serving. This removes the need to use the `visiblehead`/`visiblebranchmap` functions, and ensures that changesets we should not serve are in fact never served. We do not use filtering on hgweb yet, as there is still a number of issues to solve there.

performance: speedup computation of extinct revisions In their current state, revset calls can be very costly, as we test predicates on the entire repository. This change drops revset calls in favor of direct testing of the phase of changesets. Performance test on my Mercurial checkout - 19857 total changesets, - 1584 obsolete changesets, - 13310 obsolescence markers. Before: ! extinct ! wall 0.015124 After: ! extinct ! wall 0.009424 Performance test on a Mozilla central checkout: - 117293 total changesets, - 1 obsolete changeset, - 1 obsolescence marker. Before: ! extinct ! wall 0.032844 After: ! extinct ! wall 0.000066

performance: speedup computation of suspended revisions In their current state, revset calls can be very costly, as we test predicates on the entire repository. This change drops a revset call in favor of direct testing of the phase of changesets. Performance test on my Mercurial checkout - 19857 total changesets, - 1584 obsolete changesets, - 13310 obsolescence markers. Before: ! suspended ! wall 0.014319 After: ! suspended ! wall 0.009559 Performance test on a Mozilla central checkout: - 117293 total changesets, - 1 obsolete changeset, - 1 obsolescence marker. Before: ! suspended ! wall 0.033373 After: ! suspended ! wall 0.000053

performance: speedup computation of unstable revisions In their current state, revset calls can be very costly, as we test predicates on the entire repository. This change drops revset call in favor of direct testing of the phase of changesets. Performance test on my Mercurial checkout - 19857 total changesets, - 1584 obsolete changesets, - 13310 obsolescence markers. Before: ! unstable ! wall 0.017366 After this changes: ! unstable ! wall 0.008093 Performance test on a Mozilla central checkout: - 117293 total changesets, - 1 obsolete changeset, - 1 obsolescence marker. Before: ! unstable ! wall 0.045190 After: ! unstable ! wall 0.000032

performance: speedup computation of mutable revisions In their current state, revset calls can be very costly, as we test predicates on the entire repository. The "mutable" filter is used during branch cache loading operation. We need to make it fast. This change drops revset calls in favor of direct testing of the phase of a changeset. Performance test on my Mercurial checkout - 19857 total changesets, - 1646 mutable revision Before: ! mutable ! wall 0.032405 After: ! mutable ! wall 0.001469 Performance test on a Mozilla central checkout: - 117293 total changesets, - 1 mutable changeset, Before: ! mutable ! wall 0.188636 After: ! mutable ! wall 0.000022

performance: speedup computation of unserved revisions In their current state, revset calls can be very costly, as we test predicates on the entire repository. The "unserved" filter is used in multiple applications, and in particular in some branch cache loading operations. We need to make it fast. This change drops revset calls in favor of direct testing of the phase of a changeset. Performance test on my Mercurial checkout - 19857 total changesets, - 1584 obsolete changesets, - 13310 obsolescence markers. Before: ! unserved ! wall 0.030477 After: ! unserved ! wall 0.011844 Performance test on a Mozilla central checkout: - 117293 total changesets, - 1 obsolete changeset, - 1 obsolescence marker. Before: ! unserved ! wall 0.111259 After: ! unserved ! wall 0.000084

performance: speedup computation of hidden revisions In their current state, revset calls can be very costlys, as we test predicates on the entire repository. The hidden filter is very widely used, and needs to be very fast. This change drops revset calls in favor of direct revision manipulation. Performance test on my Mercurial checkout - 19857 total changesets, - 1584 obsolete changesets, - 13310 obsolescence markers. Before: ! hidden ! wall 0.077553 After this changes: ! hidden ! wall 0.011230 Performance test on a Mozilla central checkout: - 117293 total changesets, - 1 obsolete changeset, - 1 obsolescence marker. Before: ! hidden ! wall 0.389472 After: ! hidden ! wall 0.000079

performance: speedup computation of obsolete revisions In their current state, revset calls can be very costly as we test predicates on the entire repository. As obsolete computation is used by the "hidden" filter, it needs to be very fast. This changet drops the revset call in favor of direct testing of the phase of a changeset. Performance test on my Mercurial checkout - 19857 total changesets, - 1584 obsolete changesets, - 13310 obsolescence markers. Before: ! obsolete ! wall 0.047041 After: ! obsolete ! wall 0.004590 Performance test on a Mozilla central checkout: - 117293 total changesets, - 1 obsolete changeset, - 1 obsolescence marker. Before: ! obsolete ! wall 0.001539 After: ! obsolete ! wall 0.000017

clfilter: drop unnecessary explicit filtering on histedit Hidden changeset filtering is now done at repo level. The orphaned children computation will not include any (unless you add --hidden).

clfilter: drop unnecessary explicit filtering on rebase Hidden changeset filtering is now done at repo level. The rebaseset computation will not include any (unless you add --hidden).

clfilter: ensure that hidden filtering is working on all commands Now that hidden changeset are filtered for all commands, we test the behavior of `heads` and `summary` regarding hidden changeset.

clfilter: enforce hidden changeset globally The dispatch code now enables filtering of "hidden" changesets globally. The filter is installed before command and extension invocation. The `--hidden` switch is now global and disables this filtering for any command. Code in log dedicated to changeset exclusion is removed as this global filtering has the same effect.

record: use patch.diffopts to account for user diffopts This allows user defined diff options (e.g. showfunc) to be accounted for when using record. A test has been updated accordingly.

convert: fix most test-check-code-hg violations in cvsps code

tests: update hgweb tests to include breadcrumbs

subrepo: add subrepo property to SubrepoAbort exceptions This new property contains the path of the subrepo that generated the exception. This information can then be used by GUI tools such as TortoiseHg.

tests: update test-convert-cvs*.t The preceding commit caused their outputs to change.

cvsps: use commitids (when present) to detect changesets Simplify core logic by no longer attempting to work around missing class attributes. Instead always generate the attributes and ignore the cache if the attributes are missing

hgweb, spartan: link from manifest title to changeset page

hgweb, spartan: add "URL breadcrumbs" This change adds a "URL breadcrumb" to the "title" of the pages on the spartan template. By title I mean the first line that is shown right below the page selection row, which shows the name of the page that is being viewed, along with some additional information. In doing so it standarizes those "titles" which now follow the pattern: URL breadcumb / page details

hgweb: add a "URL breadcrumb" to the index and repository pages The purpose of this change is to make it much easier to navigate up the repository tree when the hg web server is used to serve more than one repository. A "URL breadcrumb" is a path where each of the path items can be clicked to go to the corresponding path page. This lets you go up the folder hierarchy very quickly. For example, when showing the list of repositories in http://myserver/myteams/myprojects, the following "breadcrumb" will be shown: Mercurial > myteams > myprojects Clicking on "myprojects" reloads the page. Clicking on "myteams" goes up one folder. Clicking on the leftmost "Mercurial" goes to the server root. This "breadcrumb" also appears on all repository pages. For example on the summary page of the repository at http://myserver/myteams/myprojects/myrepo the following will be shown: Mercurial > myteams > myprojects > myrepo / summary This change has been applied to all templates that already had a link to the main repository page (i.e. gitweb, monoblue, paper and coal) plus to the index page of the spartan template. In order to make the breadcumb links stand out the some of the template styles have been customized.

merge: never do premerge on symlinks Simplemerge is not symlink aware and will never do the right thing on symlinks.

merge: make internal merge fail cleanly on symlinks Simplemerge is not symlink aware and will never do the the right thing on symlinks. It would read the symlink as a file and abort with 'No such file or directory' on dangling symlinks. Instead, internal:merge now simply fails to merge symlinks.

debugpushkey: list keys sorted

debugdiscovery: report heads in sorted order

hidden: drop of the repo.hiddenrevs property It does not have any user left

context: retrieve hidden from filteredrevs This prepare the dropping of the repo.hiddenrevs property

revset: retrieve hidden from filteredrevs This prepare the dropping of the `repo.hiddenrevs` property

hidden: use both parents of working directory If we are merging with and extinct revision, this extinct revision should not be hidden.

hidden: drop cache on hiddenrevs property The `filteredrevs` function already have a cache mechanism. And this cache in invalidated at the same time than the current property cache. So we drop the cache on the property. The property itself is going to be dropped soon.

hidden: move computation in filter function There is not good reason for this computation to be handle in a different way from the other. We are moving the computation of hidden revs in the filter function. In later changesets, code that access to `repo.hiddenrevs` will be updated and the property dropped.

branchcache: add note about cache invalidation to test-keyword.t [Should've been included in aff706b3a21c.] --Kevin Bullock <kbullock@ringworld.org>

clfilter: add impactable filter The `mutable` filter still have some chance to get invalidated. This will happen when: - you garbage collect hidden changeset, - public phase is moved backward, - something is changed in the filtering (this could be fixed) So we introduce an even more stable filtering set: everything with a revision number egal or higher than the first mutable changeset is filtered. The only official use of this filter is for branchcache.

clfilter: add mutable filtering It filters all mutable changesets, leaving only public changeset unfiltered. This filtering set is expected to be much more stable that the previous one as public changeset are unlikely to disapear. The only official use of this filter is for branchcache.

run-tests.py: fix handling of symlink to the right python Before: a symlink for python in BINDIR was sometimes created, but it was never updated when a different Python was used and it was never removed. An invalid python could thus be left around and used when testing with --local. Now: the symlink is removed when wrong and created when necessary. The mechanism for finding the right name (python or python.exe) also had to be simplified and made more explicit.

log: use "hidden" filtering instead of manual check at display time When log is not given the --hidden option, hidden revision are not shown. We move the implementation from manual checking at display time to changelog filtering. This is the first official usage of the hidden filtering.

clfilter: introduces a hidden filter This filter exclude all hidden revision. We plan to use this filter to hide revision instead of manually checking contents of the hidden revisions set.

perftest: allow selection of volatile set to benchmark This helps when you focus on a subset of the volatile chain.

perftest: add a command to benchmark construction of volatile cache Obsolescence and filtering related caches are critical. Having a handy way to check them is valuable.

perftest: add an option to invalidate volatile cache Some revsets are sensitive to such initialization. Being able to test the impact is great.

perftest: document the perfrevset command I'll add an argument to it.

perftest: migrate to new style command declaration Declaring synopsis and argument on the function site is much clearer.

perftest: drop duplicated codes The `perfnodelookup` lookup commands is duplicated. We drop the first version, overwritten by the seconds.

cmdutil: make options argument optional There is not reason to force passing of an empty options list.

branchmap: allow to use cache of subset Filtered repository are *subset* of unfiltered repository. This means that a filtered branchmap could be use to compute the unfiltered version. And filtered version happen to be subset of each other: - "all() - unserved()" is a subset of "all() - hidden()" - "all() - hidden()" is a subset of "all()" This means that branchmap with "unfiltered" filter can be used as a base for "hidden" branchmap that itself could be used as a base for unfiltered branchmap. unserved < hidden < None This changeset implements this mechanism. If the on disk branchcache is not valid we use the branchcache of the nearest subset as base instead of computing it from scratch. Such fallback can be cascaded multiple time is necessary. Note that both "hidden" and "unserved" set are a bit volatile. We will add more stable filtering in next changesets. This changeset enables collaboration between no filtering and "unserved" filtering. Fixing performance regression introduced by 47f00b0de337

repoview: add a subset table This will be used by branchmap collaboration. See inline documentation for more details

branchmap: add a copy method If we want branchcache of different filter to collaborate, they need a simple way to copy each other. This will ensure that each filtered have no side effect on other filter level cache.

clfilter: use empty frozenset intead of empty tuple This will allows set operation needed for cache collaboration.

run-tests.py: don't use console for stdin when running in debug mode Tests would wait for input instead of using non-interactive mode.

tests: make hghave and run-tests exit on unknown feature requirements

test-largefiles.t: fix wrong '#if hgweb' - it should be '#if serve' A bug introduced in 0c1d10351869 did that a part of the test never was run.

dispatch: show empty filename in OSError aborts Mercurial would sometimes exit with: abort: No such file or directory where str of the actual OSError exception was the more helpful: [Errno 2] No such file or directory: '' The exception will now always show the filename and quote it: abort: No such file or directory: ''

test-dispatch.t: remove incorrect "cd $dir" A line that should have been removed in 50fbe9063ff2.

merge with crew-stable

dispatch: handle empty `testedwith` value in extension When extensions had an empty `testedwith` attribute the code tried to parse it and failed. As a result the actual error were shallowed by a This crash. We now treat empty strip as 'unknown'

destroyed: filter unknown before computing branchcache Branchcache of filtered version need up to date phase data.

branchmap: disable fallback to unfiltered branchcache Disables this simple optimisation to allow coming more powerfull approach: cache collaboration. Our goal is to have branchcache collaborate. This means that unfiltered branchcache will fallback to some filtered branchcache if invalid. We can't have the filtered branchcache to use the unfiltered one. That would loop.

localrepo: filter unknown nodes from the phasecache on destroyed When commit is followed by strip (qrefresh), phasecache contains nodes that were removed from the changelog. Since phasecache is filecached with .hg/store/phaseroots which doesn't change as a result of stripping, we have to filter it manually. If we don't write it immediately, the next time it is read from disk the nodes will be filtered again. That's what happened before, but there's no reason not to write it immediately. The change in test-keyword.t is caused by the above.

phases: make _filterunknown a member function of phasecache We'd like the ability to call filterunknown on an existing phasecache instance after nodes are destroyed.

localrepo: drop `_cacheabletip` method It iss dead code now.