diff: use binary diff when copy source is binary When a binary source has been copied or renamed into a non-binary file, we don't check whether the copy source was binary. There is a code comment explaining that a git mode will be forced anyway in order to capture the copy record (i.e. losedatafn() will be called). This is true, but forcing git mode is not the only effect binary files have: when git mode was already requested, we use the binary-ness to tell us whether to use a regular unified diff or a git binary diff. The user sees this as a "Binary file $file has changed" instead of the binary

largefiles: fix commit of a directory with no largefile changes (issue4330) When a directory is named in the commit file list, the previous behavior was to walk the list, and if no normal files in the directory were also named, add the corresponding standin for each largefile in that directory. The directory is then dropped from the list, so that committing a directory with no normal file changes works. It then added the corresponding standin directory for the first largefile seen, by prefixing it with '.hglf/'. The latter is unnecessary since each affected largefile is explicitly referenced by its standin in the list. It also caused an abort if there were no changed largefiles in the directory, because none of its standins changed: abort: .hglf/foo/bar: no match under directory! This list of files is used to tweak a matcher in lfutil.updatestandinsbymatch(), which is what is passed to commit(). The status() call that is ultimately done in the commit code with this matcher seems to have some OS specific differences. It is not necessary to append '.' for Windows to run the largefiles tests cleanly. But if '.' is not added to the list, the match function isn't called on Linux, so status() would miss any normal files that were also in a named directory. The commit then proceeds without those normal files, or says "nothing changed" if there were no changed largefiles in the directory. This is not filesystem specific, as VFAT on Linux had the same behavior as when run on ext4. It is also not an issue with lfilesrepo.status(), since that only calls the overridden implementation when paths are passed to commit. I dont have access to an OS X machine ATM to test there. Maybe there's a better way to do this. But since the standin directory for the first largefile was previously being added, and that caused the same walk in status(), there's no preformance change to this. There is no danger of erroneously committing files in '.', because the original match function is called, and if it fails, the lfutil.updatestandinsbymatch() tweaked matcher only indicates a match if the file is in the list of standins- and '.' never is. The added tests confirm this.

test-tools: update for platforms without symlink support after 5b20e4c32117 The change was triggered by removing the 'baz' hardlink.

tests: add "(glob)" to output in test-histedit-commute.t for Windows This goes with the changes in 3831e9b3750a.

Added signature for changeset db8e3f7948b1

Added tag 3.3-rc for changeset db8e3f7948b1

merge default into stable for 3.3 feature freeze

messages: quote "hg help" hints consistently

bundle2: fix parttype enforcement As spotted by Malte Helmert.

test-tools: another vfat fix

test-tools: check for unix permissions for hardlinking

tests: more fixes for f

tests: teach f not to report symlink mode bits They're not meaningful or portable

tests: teach f not to report directory size It's not meaningful or portable

test-histedit-commute: call helper script with sh Buildbot pointed out that this test wasn't passing on Linux+vfat because there's no chmod for shell scripts.

test-tools: fix portability issues

progress: add a lock to prepare for introducing a thread

progress: move update check into helper method

progress: move current topic to member variable

progress: add try/finally to make the diffs for the next commit more readable No change in behavior.

transaction: include backup file in the "undo" transaction Once the transaction is closed, we now write transaction related data for possible future undo. For now, we only do it for full file "backup" because their were not handle at all in that case. In the future, we could move all the current logic to set undo up (that currently exists in localrepository) inside transaction itself, but it is not strictly requires to solve the current situation.

transaction: pass the name of the "undo" journal to the transaction It is time for the transaction to be responsible for setting up the undo data. It is necessary to move this logic into the transaction because many more files are handled now, and the transaction is the object tracking them all. The value can be set to None if no undo should be set.

rollback: have an empty entry for the vfsmap in rollback This empty string key is used for the store. This will be needed to properly rollback backup in a future changesets.

transaction: clarify the name of 'journal' argument for transaction The argument is a string containing the journal name (used as prefix for all other transaction file). This is not the transaction file itself. So we clarify this.

transaction: use 'util.copyfile' for creating backup Using 'copyfile' (single file) instead of 'copyfiles' (tree) will ensures destination file will be overwritten. This will prevent some abort if backup file are left in place for random reason. It also seems more correct.

copyfile: allow optional hardlinking Some code paths use 'copyfiles' (full tree) for a single file to take advantage of the best-effort-hard-linking parameter. We add similar parameter and logic to 'copyfile' (single file) for this purpose. The single file version have the advantage to overwrite the destination file if it exists.

repair: add experimental option to write bundle2 files This adds an experimental option 'strip-bundle2-version' which causes backup bundles to use bundle2 formatting. Especially for generaldelta repositories, this should provide significant performance gains for any operation that needs to write a backup.

changegroup.getsubset: support multiple versions Allow a version parameter to specify which version of the packer should be used

changegroup.writebundle: HG2Y support This diff adds support to writebundle to generate a bundle2 wrapper; upcoming diffs will add an option to write a v2 changegroup part instead of v1 in these bundles.

changegroup.writebundle: provide ui The next diff will add support for writing bundle2 files to writebundle, but the bundle2 generator wants access to a ui object. This changes the signature and callsites to pass one in.

test-module-imports: use test-repo requirement

largefiles: don't rehash largefiles in updatelfiles if standin hash changed Standins are read before and after an update/merge, and all the standins that changes are handed to updatelfiles for getting their corresponding largefiles updated. updatelfiles would then hash the largefile and see if it already matched the new expected hash. If so, it would skip the update. If different, the largefile would be updated. It would happen very rarely that the largefile happened to match the new hash (and thus not the old one) and the hashing would thus be pointless ... and hashing is not cheap. Instead, when it is known that the standin hash changed (from an update), just update the standin unconditionally. If the largefile was "unsure" before the update, it was hashed at that point, so we know there is nothing to preserve. (Also, the hashing in updatelfiles was not used to preserve changes, but only to be lazy about updating the largefile, so nothing is lost by not doing this extra hashing.) There might be rare situations where we now will update largefiles that didn't have to be updated, but in all relevant cases (?) this will improve performance. Updates on a repo with some big largefiles has been seen to go from 9.19 s to 6.8 s - that is 26% less painful.

largefiles: show progress when checking standin hashes in outgoing changesets This checking can take a huge amount of time and we should give user a hint that something is going on.

unbundle: support bundle2 files This adds support for bundle2 files to the unbundle command.

pullbundle2: extract addchangegroup result combining into its own function This will also be used for 'hg unbundle'

histedit: add a test to show that issue4251 is fixed (issue4251) This will help us not regress this case in the future.

commands.debugbundle: bundle2 support This enables debugbundle to print supporting info for bundle2 files.

largefiles: cleanup overrideadd() This was a remnant of the code prior to overridding cmdutil.add().

largefiles: enable subrepo support for add The --large, --normal and --lfsize args couldn't be passed to a subrepo before, and files in the subrepos would be added silently (if -v wasn't specified) as normal files. As an added bonus, 'hg add --dry-run' no longer prints that largefiles would also be added as normal files as well.

add: pass options via keyword args The largefiles extensions needs to be able to pass --large, --normal and --lfsize to subrepos via cmdutil.add() and hgsubrepo.add(). Rather than add additional special purpose arguments, stop extracting the existing args from the **opts passed to commands.add() and just pass them along.

largefiles: don't pop largefile-specific arguments to the add command The arguments will need to stay present when making add work with subrepos.

share: replace the bookmarks.shared file with an entry on a new "shared" file cd79fb4d75fd introduced a way to share bookmarks. When a repository share that shares bookmarks was created, a .hg/bookmarks.shared file was created to mark the repository share as one that shares its bookmarks. We have plans to introduce other levels of sharing, including a "full share" mode. Rather than creating a new ".shared" file for each new thing that we may want to share It seems better to create a single "shared" file that will list what is shared for a given shared repository. This should make it much easier to get a list of everything that is shared by a given shared repository. The shared file contains a list of shared "items" (such as bookmarks). Each shared "item" is added as a new line in the file. For now the only possible entry in the file is "bookmarks".

docker: support Fedora 21

rpm: make Python 2.7.9 the default Python to include in rpms for EL 5 Use the new and more TLS support in Python 2.7.9.

contrib: make Python 2.7.9 the default in Makefile.python We should utilize (and test) the big API changes and new TLS functionality in Python 2.7.9 whenever possible.

localrepo: remove all external users of localrepo.wopener This change touches every module in which repository.wopener was being used, and changes it for the equivalent repository.wvfs. It should now be possible to remove localrepo.wopener.

localrepo: remove all external users of localrepo.sopener This change touches every module in which repository.sopener was being used, and changes it for the equivalent repository.svfs. It should now be possible to remove localrepo.sopener.

localrepo: remove all external users of localrepo.opener This change touches every module in which repository.opener was being used, and changes it for the equivalent repository.vfs. This is meant to make it easier to split the repository.vfs into several separate vfs. It should now be possible to remove localrepo.opener.

log: use namespace logname and colorname Now that we have the machinary to change the log name and the color label used, let's use that. Tests have been updated accordingly.

namespaces: add colorname member to namespace object Previously, there was no way to change the color label used for 'hg log' output. This patch just adds the member to the object, a future patch will change 'hg log' to use this.

namespaces: add logname member to namespace object Previously, there was no way to change the name used for 'hg log' output. This was inconvenient for extensions that want a template name longer than 12 characters (e.g. remotebookmarks) but a different name for 'hg log'. This patch only adds the member to the object, a future patch will update the 'hg log' code.

namespaces: use named args for namespace api This is just a style change but makes adding new arguments more robust for callers.

namespaces: make the constructor into named args None of the arguments are truly optional but this makes adding future arguments more robust and perhaps optional.

dispatch: only check compatibility against major and minor versions (BC) Extensions can declare compatibility with Mercurial versions. If an error occurs, Mercurial will attempt to pin blame on an extension that isn't marked as compatible. While all bets are off when it comes to the internal API, my experience has shown that a monthly/patch release of Mercurial has never broken any of the extensions I've written. I think that expecting extensions to declare compatibility with every patch release of Mercurial is asking a bit much and adds little to no value. This patch changes the blame logic from exact version matching to only match on the major and minor Mercurial versions. This means that extensions only need to mark themselves as compatible with the major, quarterly releases and not the monthly ones in order to stay current and avoid what is almost certainly unfair blame. This will mean less work for extension authors and almost certainly fewer false positives in the blame attribution.

test-extension: use a realistic Mercurial version Mercurial doesn't define the 3rd "patch" version field for major releases. Don't use it in tests.

test-extension: improve test readability There was a mountain of text in this file that made reading the tests difficult. Inserting some line breaks greatly improves the situation.

bundle2: enforce parttype as alphanumerical The binary format description has always stated that the parttype should be simple, but it was never really enforced. Recent discussions have convinced me we want to keep the part type simple and easy to debug. There is enough extensibility in the rest of the format.

tests: conditionalize test-tools.t for Windows support I still get the following diff on Windows 7 with NTFS. I'm not sure if it is expected and we should wildcard the link count, or if something needs to be fixed. @@ -58,7 +58,7 @@ $ ln bar baz $ f bar -n baz -l --hexdump -t --sha1 --lines=9 -B 20 - bar: file, links=2, newer than baz, sha1=612ca68d0305c821750a + bar: file, links=0, newer than baz, sha1=612ca68d0305c821750a\r (esc) 0000: 31 0a 32 0a 33 0a 34 0a 35 0a 36 0a 37 0a 38 0a |1.2.3.4.5.6.7.8.| 0010: 39 0a |9.|

dirstate: ignore negative debug.dirstate.delaywrite values - they crashed it Sleep can only travel forward in time, not back.

linkrev: use the right manifest content when adjusting linrev (issue4499) When the manifest revision is stored as a delta against a non-parent revision, '_adjustlinkrev' could miss some file update because it was using the delta only. We now use the 'fastread' method that uses the delta only when it makes sense. A test showcasing on the of possible issue have been added.

unpacker: fix missing arg for py2.4

branchcache: make _rbcrevslen handling more safe self._rbcrevslen is used to keep track of the number of good records on disk. It should thus not be updated before the records actually have been written to disk.

branchcache: add debug output whenever cache files use truncate The cache files are usually append only but will automatically be truncated and recover in exceptional situations. Add a debug notice when such exceptional situations are encountered.

tests: rework revision branch cache tests The tests that were added with the revision branch cache in 678f53865c68 had suffered from bit rot in the development iterations. They were no longer that "good". Now, the tests are rewritten and reworked to be more readable and maintainable and relevant for the actual implementation. This also utilizes the new 'f' helper tool for keeping an eye the cache files.

tests: add 'f' tool for cross platform file operations in the tests This tool is like the collection of tools found in a unix environment but are cross platform and stable and suitable for our needs in the test suite. The main reason it is "needed" now is for hexdump of revision branch cache to keep an eye on how it changes and make sure the format is stable. It is a very generic tool that can end up being used a lot in tests, so I gave it very generic name.

run-tests: include testdir in $PATH so tests easily can use helper tools The testdir is already added to $PYTHONPATH - I think it makes sense and is convenient to add it to $PATH too. The following binaries are invoked from tests using full path with $TESTDIR/ - they can now be used without specifying path: dumbhttp.py dummyssh filterpyflakes.py generate-working-copy-states.py get-with-headers.py hghave histedit-helpers.sh killdaemons.py md5sum.py notcapable printenv.py readlink.py revlog-formatv0.py run-tests.py svn-safe-append.py svnxml.py tinyproxy.py

filelog: use censored revlog flag bit to quickly check if a node is censored

revlog: verify censored flag when hashing added revision fulltext When receiving a delta via exchange, three possible storage outcomes emerge: 1. The delta is added directly to the revlog. ("fast-path") 2. A freshly-computed delta with a different base is stored. 3. The new revision's fulltext is computed and stored outright. Both (2) and (3) require materializing the full text of the new revision by applying the delta to its base. This is typically followed by a hash check. The new flags argument allows callers to _addrevision to signal that they expect that hash check to fail. We can use this opportunity to verify that expectation. If the hash fails, require the flag be set; if the hash passes, require the flag be unset. Rather than simply eliding the hash check, this approach provides some assurance that the censored flag is not applied to valid revisions. Read more at: http://mercurial.selenic.com/wiki/CensorPlan

revlog: add flags argument to _addrevision, update callers use default flags For revlog index flags to be useful to other parts of Mercurial, they need to be settable when writing revisions. The current use case for revlog index flags is the censorship feature: http://mercurial.selenic.com/wiki/CensorPlan While the censor flag could be inferred in _addrevision by interrogating the text/delta being added, that would bury the censorship logic and inappropriately couple it to all revision creation.

revlog: define censored flag for revlogng index This flag bit will be used to cheaply signal censorship presence to upper layers (exchange, verify). It indicates that censorship metadata is present but does not attest to the verifiability of that metadata. For the censorship design, see: http://mercurial.selenic.com/wiki/CensorPlan

localrepo: remove all internal uses of localrepo.wopener It has been replaced with localrepo.wvfs.

localrepo: remove all internal uses of localrepo.sopener It has been replaced with localrepo.svfs.

localrepo: remove all internal uses of localrepo.opener It has been replaced with localrepo.vfs. In the future we may split the vfs into different vfs objects to access different elements of the repository.

sslutil: drop defunct ssl version constants Nobody outside sslutil should be using these constants anyway.

sslutil: use saner TLS settings on Python 2.7.9 Asking for TLSv1 locks us out of TLSv1_2 etc. This is at least less bad. Ideally we'd use ssl.create_default_context(), but that causes more mayhem in the testsuite than I really want to deal with right now.

sslutil: drop support for clients of sslutil specifying a TLS version We really just want to support the newest thing possible, so we may as well consolidate that knowledge into this module. Right now this doesn't change any behavior, but a future change will fix the defaults for Python 2.7.9 so we can use slightly better defaults there (which is the only place it's possible at the moment.)

discovery: run discovery on filtered repository We have been running discovery on unfiltered repository for quite some time. This was aimed at two things: - save some bandwith by prevent the repushing of common but hidden changesets - allow phases changes on secret/hidden changeset on bare push. The cost of this unfiltered discovery combined with evolution is actually really high. Evolution likely create thousand of hidden heads, and the discovery is going to try to discovery if each of them are common or not. For example, pushing from my development mercurial repository implies 17 discovery round-trip. The benefit are rare corner cases while the drawback are massive. So we run the discovery on a filtered repository again. We add some hack to detect remote heads that are known locally and adds them to the common set anyway, so the good behavior of most of the corner case should remains. But this will not work in all cases. This bring my discovery phase back from 17 round-trips to 1 or 2.

revset: check for collisions between alias argument names in the declaration Before this patch, collisions between alias argument names in the declaration are ignored, and this silently causes unexpected alias evaluation. This patch checks for such collisions, and aborts (or shows a warning) when collisions are detected. This patch doesn't add a test to "test-revset.t", because a doctest is enough to test the collisions detection itself.

revset: parse alias declaration strictly by _parsealiasdecl Before this patch, alias declaration is parsed by string base operations: matching against "^([^(]+)\(([^)]+)\)$" and splitting by ",". This overlooks many syntax errors like below (see the previous patch introducing "_parsealiasdecl" for detail): - un-closed parenthesis causes being treated as "alias symbol" - symbol/function name aren't examined whether they are valid or not - invalid argument list causes unexpected argument names To parse alias declaration strictly, this patch replaces parsing implementation by "_parsealiasdecl". This patch tests only one typical declaration error case, because error detection itself is already tested in the doctest of "_parsealiasdecl". This also removes class property "args" and "error", because these are certainly initialized in "revsetalias.__init__".

revset: introduce "_parsealiasdecl" to parse alias declarations strictly This patch introduces "_parsealiasdecl" to parse alias declarations strictly. For example, "_parsealiasdecl" can detect problems below, which current implementation can't. - un-closed parenthesis causes being treated as "alias symbol" because all of declarations not in "func(....)" style are recognized as "alias symbol". for example, "foo($1, $2" is treated as the alias symbol. - alias symbol/function names aren't examined whether they are valid as symbol or not for example, "foo bar" can be treated as the alias symbol, but of course such invalid symbol can't be referred in revset. - just splitting argument list by "," causes overlooking syntax problems in the declaration for example, all of invalid declarations below are overlooked: - foo("bar") => taking one argument named as '"bar"' - foo("unclosed) => taking one argument named as '"unclosed' - foo(bar::baz) => taking one argument named as 'bar::baz' - foo(bar($1)) => taking one argument named as 'bar($1)' To decrease complication of patch, current implementation for alias declarations is replaced by "_parsealiasdecl" in the subsequent patch. This patch just introduces it. This patch defines "_parsealiasdecl" not as a method of "revsetalias" class but as a one of "revset" module, because of ease of testing by doctest. This patch factors some helper functions for "tree" out, because: - direct accessing like "if tree[0] == 'func' and len(tree) > 1" decreases readability - subsequent patch (and also existing code paths, in the future) can use them for readability This patch also factors "_tokenizealias" out, because it can be used also for parsing alias definitions strictly.

revset: store full detail into revsetalias.error for error source distinction Before this patch, any errors in the declaration of revset alias aren't detected at all, and there is no information about error source in the error message. As a part of preparation for parsing alias declarations and definitions more strictly, this patch stores full detail into "revsetalias.error" for error source distinction. This makes raising "Abort" and warning potential errors just use "revsetalias.error" without any message composing.

revset: factor out composing error message for ParseError to reuse This patch defines the composing function not in "ParseError" class but in "revset" module, because: - "_()" shouldn't be used in "ParseError", to avoid adding "from i18n import _" i18n" to "error" module - generalizing message composition of"ParseError" for all code paths other than revset isn't the purpose of this patch we should also take care of showing "unexpected leading whitespace" for some code paths, to generalize widely.

revset: make tokenize extensible to parse alias declarations and definitions Before this patch, "tokenize" doesn't recognize the symbol starting with "$" as a valid one. This prevents revset alias declarations and definitions from being parsed with "tokenize", because "$" may be used as the initial letter of alias arguments. BTW, the alias argument name doesn't require leading "$" itself, in fact. But we have to assume that users may use "$" as the initial letter of argument names in their aliases, because examples in "hg help revsets" uses such names for a long time. To make "tokenize" extensible to parse alias declarations and definitions, this patch introduces optional arguments "syminitletters" and "symletters". Giving these sets can change the policy of "valid symbol" in tokenization easily. This patch keeps original examination of letter validity for reviewability, even though there is redundant interchanging between "chr"/"ord" at initialization of "_syminitletters" and "_symletters". At most 256 times examination (per initialization) is cheaper enough than revset evaluation itself. This patch is a part of preparation for parsing alias declarations and definitions more strictly.

largefiles: make linear update set unsure largefiles normal if unchanged 'hg update' would hash all 'unsure' largefiles before performing the merge. It would update the standins but not detect the very common case where the largefile never had been changed by the user but just had been marked with an invalid dirstate mtime to make sure any changes done by the user in the same second would be detected. The largefile would remain in that state and would have to be hashed again next time even though it still not had been changed. Sad trombone. Instead, for largefiles listed as 'unsure' or 'modified', after updating the standin with the actual hash, mark the largefile as normal if it turns out to not be modified relative to the revision in the parent revision. That will prevent it from being hashed again next time.

debugdirstate: don't hide date field with --nodate, just show 'set'/'unset' The value of the dirstate date field cannot be used in tests and we thus have to use debugdirstate with --nodate. It is however still very helpful to be able to see whether the date field has been set or still is unset. The absence of that information made it hard to debug some largefile dirstate issues. This change _could_ make the test suite more unstable ... but that would be places where the test suite or the code should be made more stable. (Note: 'unset' with the magic negative sizes is reliable. 'unset' for normal sizes would probably not be reliable, but there is no such occurrences in the test suite and it should thus be reliable.) This output wastes more horizontal space in the --nodate output, but it also makes things simpler that the output format always is the same. It is just a debug command so let's keep it simple.

debugdirstate: simplify date handling after e7ed5d07cc4c used fixed format

forget: don't report rejected files as forgotten as well It seems like a mistake to report a file as forgotten and rejected. The forgotten list doesn't seem to be used by anything in core, so no test changes.

largefiles: enable subrepo support for forget

namespaces: add revset for 'named(namespace)' This patch adds functionality for listing all changesets in a given namespace via the revset language.

bundles: do not overwrite existing backup bundles (BC) Previously, a backup bundle could overwrite an existing bundle and cause user data loss. For instance, if you have A<-B<-C and strip B, it produces backup bundle B-backup.hg. If you then hg pull -r B B-backup.hg and strip it again, it overwrites the existing B-backup.hg and C is lost. The fix is to add a hash of all the nodes inside that bundle to the filename. Fixed up existing tests and added a new test in test-strip.t

https: support tls sni (server name indication) for https urls (issue3090) SNI is a common way of sharing servers across multiple domains using separate SSL certificates. As of Python 2.7.9 SSLContext has been backported from Python 3. This patch changes sslutil's ssl_wrap_socket to use SSLContext and take a server hostname as and argument. It also changes the url module to make use of this argument. The new code for 2.7.9 achieves it's task by attempting to get the SSLContext object from the ssl module. If this fails the try/except goes back to what was there before with the exception that the ssl_wrap_socket functions take a server_hostname argument that doesn't get used. Assuming the SSLContext exists, the arguments to wrap_socket at the module level are emulated on the SSLContext. The SSLContext is initialized with the specified ssl_version. If certfile is not None load_cert_chain is called with certfile and keyfile. keyfile being None is not a problem, load_cert_chain will simply expect the private key to be in the certificate file. verify_mode is set to cert_reqs. If ca_certs is not None load_verify_locations is called with ca_certs as the cafile. Finally the wrap_socket method of the SSLContext is called with the socket and server hostname. Finally, this fails test-check-commit-hg.t because the "new" function ssl_wrap_socket has underscores in its names and underscores in its arguments. All the underscore identifiers are taken from the other functions and as such can't be changed to match naming conventions.

merge with stable

unpacker: check the right exception type for 2.4

hgweb: fix diffstat links in paper/changeset.tmpl '<a .../>foo</a>' syntax is incorrect, since the first tag just "tries" to close itself and then the actual content follows. It doesn't work, either because web browsers know better than this or because there should be a whitespace before /: '<a />'. So for the hgweb users the links looked normal anyway, but now they are correct in code as well.

hgweb: close <img> elements Templates declare xhtml doctype, which means, in particular, that the document must also be valid xml. So <img> elements must be closed.

hgweb: close <p> elements <p> elements can only contain inline elements, so as soon as browser encounters a block element (e.g. block <div>) "inside" a <p>, it puts an implicit </p>. It's better to do this explicitly.

hgweb: close <th> properly in spartan/filelogentry.tmpl

revset: simplify fullreposet.__and__ to call sort() with boolean flag Note that sort() takes a boolean flag, so other.sort(reverse) was wrong. It just worked fine because there is a top-level function, reverse().

revset: fix spanset.isascending() to honor sort() or reverse() request Because spanset.isascending() ignored the ascending flag, the result of "fullreposet() & x" was always sorted in ascending order. The test case is carefully chosen to call fullreposet.__and__.

hghave: we now support Python 2.7.9's ssl for https

Makefile.python: try curl if wget fails Macs ship with curl and not wget, so this is a nice little tweak for folks testing on OS X.

test-https: glob error messages more so we pass on Python 2.7.9 Python 2.7.9 cleans up how it stringifies SSL errors, so we have to look only for the important bit (certificate verify failed) rather than looking for specific ssl module goop (which is now unstable).

filelog: fix backwards comment for 'backrevref'

filelog: remove trailing "form feed" character

filelog: remove unused variable 'lkr' It's used further down, but it's overwritten before, so it's technically a dead assignment, but unnecessary nevertheless.

branchmap: add seek() to end of file before calling tell() on append open() This is similar to 48c232873a54, which was subsequently modified in 19f5dec2d61f for 2.4. Unexpected test changes on Windows occurred without this.

tests: fix test-casefolding.t output for branchcache This belongs with 7d63398fbfd1. I assume that the failure to read is OK, because there is similar output in test-convert-svn-encoding.t.

setdiscovery: remove '_setupsample' function It is now unused.

setdiscovery: document '_takequicksample'

setdiscovery: drop '_setupsample' usage in '_takequicksample' For '_takefullsample' we can just retrieve the list of head directly and ignore the rest of the complex return values. This was the last call to the infamous '_updatesample' function.

setdiscovery: drop the 'always' argument to '_updatesample' This argument exists because of the complex code flow in '_takequicksample'. It first gets the list of heads and then calls '_updatesample' on an empty initial sample and a size limit matching the differences between the number of heads and the target sample size. Finally the heads and the sample from '_updatesample' were added. To ensure this addition result had the exact target length, the code had to ensure no elements from the heads were added to the '_updatesample' content and therefore was passing this "always included set of heads". Instead we can just update the initial heads sample directly and use the final target size as target size for the update. This removes the need for this 'always' parameter to the '_updatesample' function The test are affected because different set building order results in different random sampling.

setdiscovery: always add exponential sample to the heads As explained in a previous changeset, prioritizing heads too much behaves pathologically when there are more heads than the sample size. To counter this, we always inject exponential samples before reducing to the sample size limit. This already show some benefit in the test themselves, but on a real-world example this moves my discovery for push to pathologically headed repo from 45 rounds to 17 of them. We should maybe ensure that at least 25% of the result sample is heads, but I think the random sampling will be fine in practice.

setdiscovery: directly run '_updatesample' The heads and exponential sample are going to end up in the same set before any extra processing happens. We simplify the code by directly updating a set with heads. Changes in the order the set is built lead to small changes in the random sampling output. But after double checking, I can confirm the input data to the random sampling is consistent.

setdiscovery: stop using '_setupsample' in '_takefullsample' Very few of the return values of '_setupsample' remain in use, so we directly retrieve the value we care about and drop the '_setupsample' call.

setdiscovery: randomly pick between heads and sample when taking full sample Before this changeset, the discovery protocol was too heads-centric. Heads of the undiscovered set were always sent for discovery and any room remaining in the sample were filled with exponential samples (and random ones if any room remained). This behaved extremely poorly when the number of heads exceeded the sample size, because we keep just asking about the existence of heads, then their direct parent and so on. As a result, the 'O(log(len(repo)))' discovery turns into a 'O(len(repo))' one. As a solution we take a random sample of the heads plus exponential samples. This way we ensure some exponential sampling is achieved, bringing back some logarithmic convergence of the discovery again. This patch only applies this principle in one place. More places will be updated in future patches. One test is impacted because the random sample happen to be different. By chance, it helps a bit in this case.

setdiscovery: document the '_updatesample' function This function is central in the sample building process, having it documented help code readability a lot.

setdiscovery: avoid calling any sample building if the undecided set is small If the length of undecided is smaller than the sample size, we can just request information for all of them. This conditional was previously handled by '_setupsample'. But '_setupsample' is in my opinion a problematic function with blurry semantics. Having this conditional explicitly earlier makes the code more explicit and moves us closer to removing this '_setupsample' function.

setdiscovery: delay sample building calls to gather them in a single place Some of the logic around sample building is duplicated in the sample builders, it would clean up thing to extract it in the top function, but this requires all codes to be in the same place. This changeset mostly exists to make the next one more clear.

setdiscovery: drop unused 'initial' argument for '_takequicksample' There is a single call site, and it is always using 'initial=True'. So we just drop the argument and the associated condition.

readmarkers: add a SHA256 fixme note