profiling: add documentation of lsprof 'sort' and 'nested'

OS X: try cheap ascii .lower() in normcase before making full unicode dance This is similar to what is done in encoding.lower, introduced in c481761033bd. This has been seen making 'hg up' and 'hg st' in a 50000+ files repo 13% faster. This might make Mercurial slightly slower for users who mainly use non-ASCII filenames. That is a reasonable trade-off.

run-tests.py: inherit PYTHONHASHSEED from environment if set This makes it possible to fix the seed by using for instance PYTHONHASHSEED=7 ./run-tests.py ... This can be very convenient when trying to debug problems that are influenced by hash values. Try different seed values until you find one that triggers the bad behaviour and then keep that while debugging. The value 0 will restore default Python behavior and disable randomization.

test-obsolete: validate that bundle is not affected by issue3788 Bundle might have been affected by the same kind of error than pull (issue3788). Testing show it is not the case.

pull: fix crash when pulling changeset that get hidden locally (issue3788) When you have obsolescence marker that apply to a pulled changesets, the added changeset is immediately filtered. Then the list of added changeset needs to be build against and unfiltered repo.

hgweb: prevent traceback during search when filtered (issue3783) The search needs to iterate over the repo using changelog.revs like the rest of the Mercurial code.

bookmarks: hide bookmarks on filtered revs from listkeys Don't expose unserved changesets to remote repos. Thanks to Sean Farley <sean.michael.farley@gmail.com> for tracking down the issue and Pierre-Yves David <pierre-yves.david@ens-lyon.org> for the fix.

bookmarks: don't use bookmarks.listbookmarks in local computations bookmarks.listbookmarks is for wire-protocol use. The normal way to get all the bookmarks on a local repository is repo._bookmarks.

filtering: test that bookmarks prevent hiding of changesets

discovery: outgoing pass unfiltered repo to findcommonincoming (issue3776) We noa pass an unfiltered repo in the same way `localrepo.push` does. This does not alter outgoing behavior and prevents possible crash with computing common/missing. The `findcommonincoming` code could be simplified to make this unnecessary, but this is too much change for the freeze.

test: minor documentation fix The related test check push, not pull.

largefiles: fix cat when using relative paths from subdirectory

largefiles: fix commit when using relative paths from subdirectory Remove cwd handling from getstandinmatcher - it did not belong there, as proven by the tests.

largefiles: allow use of urls with #revision largefiles tried to create a peer directly with the specified url. That caused abort: unsupported URL component: "..." if a revision was specified in the url. The branch name do not matter for largefiles' use of remote peers. Largefiles will be shared among all branches anyway.

largefiles: don't verify largefile hashes on servers when processing statlfile When changesets referencing largefiles are pushed then the corresponding largefiles will be pushed too - unless the target already has them. The client will use statlfile to make sure it only sends largefiles that the target doesn't have. The server would however on every statlfile check that the content of the largefile had the expected hash. What should be cheap thus became an expensive operation that trashed the disk and the cache. Largefile hashes are already checked by putlfile before being stored on the server. A server should thus be able to keep its largefile store free of errors - even more than it can keep revlogs free of errors. Verification should happen when running 'hg verify' locally on the server. Rehashing every largefile on every remote stat is too expensive. Clients will also stat lfiles before downloading them. When the server verified the hash in stat it meant that it had to read the file twice to serve it. With this change the server will assume its own hashes are ok without checking them on every statlfile. Some consequences of this change: - in case of server side corruption the problem will be detected by the existing check on the client side - not on server side - clients that could upload an uncorrupted largefile when pushing will no longer magically heal the server (and break hardlinks) - a client will now only upload its uncorrupted files after the corrupted file has been removed on the server side - client side verify will no longer report corruption in files it doesn't have (Issue3123 discussed related problems - and how they have been fixed.)

tests: clarify test for pushing corrupted largefile The test no longer tested that the server prevented pushing a corrupt largefile. At the same time it tested what happened when the server already had a corrupt largefile. These two cases are now separated.

largefiles: verify all files in each revision and report errors in any revision Verify used 'any' and would stop verifying after the first failure in each changeset. The exit code only reported the result from the last changeset.

tests: better test coverage of largefiles localstore verify This demonstrates problems that will be fixed later.

largefiles: adapt remotestore._getfile to batched statlfile 9e1616307c4c introduced batching of statlfile, but not all codepaths got converted. _getfile gave _stat garbage and got garbage back. The garbage didn't match the expected error codes and was thus interpreted as success. It could thus end up trying to fetch a largefile that didn't exist. Instead we now pass _stat valid input and handle both correct and invalid output correctly. This makes the code work as intended ... but it would probably be better if it didn't abort on missing largefiles, just like it happened to do before.

largefiles: don't allow corruption to propagate after detection basestore.get uses util.atomictempfile when checking and receiving a new largefile ... but the close/discard logic was too clever for largefiles. Largefiles relied on being able to discard the file and thus prevent it from being written to the store. That was however too brittle. lfutil.copyandhash closes the infile after writing to it ... with a 'blecch' comment. The discard was thus a silent noop, and as a result of that corruption would be detected ... and then the corrupted files would be used anyway. Instead we now use a tmp file and rename or unlink it after validating it. A better solution should be implemented ... but not now.

largefiles: adapt verify to batched remote statlfile (issue3780) 9e1616307c4c introduced batching of statlfile, but not all codepaths got converted. 'hg verify' with a remotestore could thus crash with TypeError: 'builtin_function_or_method' object is not iterable Also, the 'hash' variable was used without assigning to it. Don't use variable names that collide with Python built-in functions. Instead we use 'expecthash' as in localstore. The tests for this issue covers an untested area. The tests happens to also reveal incorrect attempts at getting non-existing largefiles, bad server side handling of that, and corruption issues - all to be fixed later.

largefiles: let wirestore._stat return stats as expected by remotestore verify - preparing for fixing verify crash.

tests: improve description of hgweb secret bookmarks test Added in 886936ecc21b with only an issue number to describe it.

bookmarks: show active bookmark even if not at working dir If the active bookmark doesn't point at a parent of the working dir (e.g. a pull moved it out from under us), we nonetheless show it as active. This follows on 2096e025a728 in removing the dichotomy (at least in the UI) between "current" and "active" bookmarks.

hgweb: don't attempt to show hidden bookmarks (issue3774) localrepository._bookmarks is unfiltered, but hgweb gets a filtered repo. This fixes the resulting traceback on the 'bookmarks' page.

hgweb: fetch tipmost unfiltered rev thru the changelog This fixes a traceback when tip is filtered (e.g. because it's secret). See issue3783, for which this is a partial fix.

help: update verbose 'clone' help to include '@' bookmark

tests: fix test-help.t for '@' bookmark documentation e031e10cdc06 unexpectedly introduced bookmarks into the results for 'hg help -k clone'. Mea culpa, miserere &c.

help: document '@' bookmark in 'help bookmarks' and 'help clone'

revset: evaluate sub expressions correctly (issue3775) Before this patch, sub expression may return unexpected result, if it is joined with another expression by "or": - "^"/parentspec(): "R or R^1" is not equal to "R^1 or R". the former returns only "R". - "~"/ancestorspec(): "R or R~1" is not equal to "R~1 or R". the former returns only "R". - ":"/rangeset(): "10 or (10 or 15):" is not equal to "(10 or 15): or 10". the former returns only 10 and 15 or grater (11 to 14 are not included). In "or"-ed expression "A or B", the "subset" passed to evaluation of "B" doesn't contain revisions gotten from evaluation of "A", for efficiency. In the other hand, "stringset()" fails to look corresponding revision for specified string/symbol up, if "subset" doesn't contain that revision. So, predicates looking revisions up indirectly should evaluate sub expressions of themselves not with passed "subset" but with "entire revisions in the repository", to prevent "stringset()" from unexpected failing to look symbols in them up. But predicates in above example don't so. For example, in the case of "R or R^1": 1. "R^1" is evaluated with "subset" containing revisions other than "R", because "R" is already gotten by the former of "or"-ed expressions 2. "parentspec()" evaluates "R" of "R^1" with such "subset" 3. "stringset()" fails to look "R" up, because "R" is not contained in "subset" 4. so, evaluation of "R^1" returns no revision This patch evaluates sub expressions for predicates above with "entire revisions in the repository".