transaction: reorder unlinking .hg/journal and .hg/journal.backupfiles After this reordering, absence of '.hg/journal' just before starting new transaction means also absence of '.hg/journal.backupfiles'. In this case, all temporary files for preceding transaction should be completely unlinked, and HG_PENDING doesn't cause unintentional reading stalled temporary files in. Otherwise, 'repo.transaction()' raises exception with "run 'hg recover' to clean up transaction" hint.

merge: make in-memory changes visible to external update hooks 51844b8b5017 (while 3.4 code-freeze) made all 'update' hooks run after releasing wlock for visibility of in-memory dirstate changes. But this breaks paired invocation of 'preupdate' and 'update' hooks. For example, 'hg backout --merge' for TARGET revision, which isn't parent of CURRENT, consists of steps below: 1. update from CURRENT to TARGET 2. commit BACKOUT revision, which backs TARGET out 3. update from BACKOUT to CURRENT 4. merge TARGET into CURRENT Then, we expects hooks to run in the order below: - 'preupdate' on CURRENT for (1) - 'update' on TARGET for (1) - 'preupdate' on BACKOUT for (3) - 'update' on CURRENT for (3) - 'preupdate' on TARGET for (4) - 'update' on CURRENT/TARGET for (4) But hooks actually run in the order below: - 'preupdate' on CURRENT for (1) - 'preupdate' on BACKOUT for (3) - 'preupdate' on TARGET for (4) - 'update' on TARGET for (1), but actually on CURRENT/TARGET - 'update' on CURRENT for (3), but actually on CURRENT/TARGET - 'update' on CURRENT for (4), but actually on CURRENT/TARGET Root cause of the issue focused by 51844b8b5017 is that external 'update' hook process can't view in-memory changes (especially, of dirstate), because they aren't written out until the end of transaction (or wlock). Now, hooks can be invoked just after updating, because previous patches made in-memory changes visible to external process. This patch may break backward compatibility from the point of view of "scheduling hook execution", but should be reasonable because 'update' hooks had been executed in this order before 3.4. This patch tests "hg backout" and "hg unshelve", because the former activates the transaction before 'update' hook invocation, but the former doesn't.

hook: centralize passing HG_PENDING to external hook process This patch centralizes passing HG_PENDING to external hook process into '_exthook()'. To make in-memory changes visible to external hook process, this patch does: - write (or schedule to write) in-memory dirstate changes, and - set HG_PENDING environment variable, if: - a transaction is running, and - there are in-memory changes to be visible This patch tests some commands with some hooks, because transaction activity of a same hook differs from each other ("---": "not tested"). ======== ========= ========= ============ command preupdate precommit pretxncommit ======== ========= ========= ============ unshelve o --- --- backout x --- --- import --- o o qrefresh --- x o ======== ========= ========= ============ Each hooks are examined separately to prevent in-memory changes from being visible to external process accidentally by side effect of hooks previously invoked.

cmdutil: make in-memory changes visible to external editor (issue4378) Before this patch, external editor process for the commit log can't view some in-memory changes (especially, of dirstate), because they aren't written out until the end of transaction (or wlock). This causes unexpected output of Mercurial commands spawned from that editor process. To make in-memory changes visible to external editor process, this patch does: - write (or schedule to write) in-memory dirstate changes, and - set HG_PENDING environment variable, if: - a transaction is running, and - there are in-memory changes to be visible "hg diff" spawned from external editor process for "hg qrefresh" shows: - "changes newly imported into the topmost" before 49148d7868df(*) - "all changes recorded in the topmost by refreshing" after this patch (*) 49148d7868df changed steps invoking editor process Even though backward compatibility may be broken, the latter behavior looks reasonable, because "hg diff" spawned from the editor process consistently shows "what changes new revision records" regardless of invocation context. In fact, issue4378 itself should be resolved by 800e090e9c64, which made 'repo.transaction()' write in-memory dirstate changes out explicitly before starting transaction. It also made "hg qrefresh" imply 'dirstate.write()' before external editor invocation in call chain below. - mq.queue.refresh - strip.strip - repair.strip - localrepository.transaction - dirstate.write - localrepository.commit - invoke external editor Though, this patch has '(issue4378)' in own summary line to indicate that issues like issue4378 should be fixed by this. BTW, this patch adds '-m' option to a 'hg ci --amend' execution in 'test-commit-amend.t', to avoid invoking external editor process. In this case, "unsure" states may be changed to "clean" according to timestamp or so on. These changes should be written into pending file, if external editor invocation is required, Then, writing dirstate changes out breaks stability of test, because it shows "transaction abort!/rollback completed" occasionally. Aborting after editor process invocation while commands below may cause similar instability of tests, too (AFAIK, there is no more such one, at this revision) - commit --amend - without --message/--logfile - import - without --message/--logfile, - without --no-commit, - without --bypass, - one of below, and - patch has no description text, or - with --edit - aborting at the 1st patch, which adds or removes file(s) - if it only changes existing files, status is checked only for changed files by 'scmutil.matchfiles()', and transition from "unsure" to "normal" in dirstate doesn't occur (= dirstate isn't changed, and written out) - aborting at the 2nd or later patch implies other pending changes (e.g. changelog), and always causes showing "transaction abort!/rollback completed"

dirstate: show develwarn for write() invocation without transaction This is used to detect 'dirstate.write()' invocation without the value gotten by 'repo.currenttransaction()' (mainly focused on 3rd party extensions).

dirstate: make dirstate.write() callers pass transaction object to it Now, 'dirstate.write(tr)' delays writing in-memory changes out, if a transaction is running. This may cause treating this revision as "the first bad one" at bisecting in some cases using external hook process inside transaction scope, because some external hooks and editor process are still invoked without HG_PENDING and pending changes aren't visible to them. 'dirstate.write()' callers below in localrepo.py explicitly use 'None' as 'tr', because they can assume that no transaction is running: - just before starting transaction - at closing transaction, or - at unlocking wlock

dirstate: remove layering violation around writing dirstate out This violation, which passes repo object to dirstate, was introduced by 09bb1ee7e73e. This patch uses 'False' instead of 'None' as default value of 'tr' argument, to distinguish "None as repo.currenttransaction() result" from "legacy invocation without explicit tr passing".

dirstateguard: remove layering violation around saving/restoring backup This violation, which passes 'repo' object to dirstate, was introduced by 59b5e8844eb0 and 020b12d591f3.

rebase: reduce scope of try catch in restorestate Refactoring by reduce the scope of the try catch block so that it only captures what it needs. I could have made it smaller but another patch in the series will add onto it.

rebase: on abort delete rebase state file no matter what When a user's repository is in an unfinished rebase state and they choose to abort, at a minimum, the repo should be out of that state. We've found situations where the user could not leave the state unless manually deleting the rebasestate file. This fix ensures that no matter what exception may be raised during the abort, the rebase state file will be deleted, the user will be out of the rebase state and they can get their repository into a workable condition.

contrib: make editmerge look for merge markers at the beginning of the line This fix adds a caret to the start of the regex looking for merge markers. This avoids the issue arises when you've real merge conflicts in a file that tests for the existance of merge markers in test output. Editmerge will not open on the fake/tested merge markers because they'll be indented in.

commit: abort when a committemplate is not changed If a committemplate is provided and no message is provided on the command line, and no edits are made to the commit template, then abort the commit.

localrepo.commit: disallow commit when driver-resolved files exist This code will not currently be activated because there's no code to mark files as driver-resolved in core. This point is also somewhat hard to plug into from extensions.

merge.mergestate: add a generator for driver-resolved files Just like for unresolved files above, we need to be able to tell what files are driver-resolved.

hook: for python hooks, also return whether an exception was raised The hook code treats python hooks raising an exception and returning True as the exact same. This is OK for hooks themselves, but other code that wants to invoke external code using the same underlying code is a bit more interested in making a distinction.

hook.runhooks: return a dict of result values This will be useful to other calling code that would be interested in what the individual hooks return.

hook: factor out determination of hooks from running them This will allow other code to run a predetermined series of hooks.

mq: generate patch names from first line of description Avoid the pointless numeric rev.diff patch names. Instead, do like mbox extension does and create meaningful patch names.

contrib: offer Python 2.7.10

contrib: drop Python < 2.6 from Makefile.python

mergetools.rc: find OSX FileMerge in the new location inside Xcode 4.3

exchange: don't print error codes after clone bundle failure We don't appear to print error codes elsewhere. The error codes are inconsistent between at least Linux and OS X and are more trouble than they are worth. Humans care about the error string more than the code anyway. A glob was also added to pave over differences in error strings between Linux and OS X.

debian: install bash completion as hg and not mercurial (issue4900)

merge-tools: allow marking a mergetool as completely disabled Very often in my life I'm finding that the only configured merge tool present on the system is vimdiff[0], and it's currently impossible (as far as I can tell) short of specifying `ui.merge = `[1] to actually *disable* a merge tool. This allows vimdiff-haters to put: [merge-tools] vimdiff.disable = yes in their ~/.hgrc and never see vimdiff again. I'm stopping short of putting this as a commented out entry in the sample new user hgrc (seen when a user runs `hg config --edit` with no ~/.hgrc) for now, but I might come back and do that later. 0: vimdiff is at an awkward intersection: it's usually installed by the vim package which is often installed as a vi substitute, so it's mere presence doesn't imply me wanting it, unlike (say) kdiff3. 1: There's a related problem I ran into today where specifying `ui.merge = :merge` failed because :merge isn't a command, which I think is a regression. I'll try and figure that out and at least file a bug.

exchange: add oparg to push so that extensions can wrap pushop

destmerge: extract logic based on branch heads in its own function One of the main goal of having consolidated destination function is to allow extension to play with this logic. We extract sub logic to make is wrapping more practical.

destmerge: extract logic based on bookmark into its own function One of the main goal of having consolidated destination function is to allow extension to play with this logic. We extract sub logic to make is wrapping more practical.

destupdate: have a generic and extensible way to run each step We want extension to be able to easily override or add new way to select the default update destination. We use the same list + dict approach as in other parts of the code.

destupdate: extract logic based on branch in its own function One of the main goal of having consolidated destination function is to allow extension to play with this logic. We extract sub logic to make is wrapping more practical.

destupdate: extract logic based on bookmarks in its own function One of the main goal of having consolidated destination function is to allow extension to play with this logic. We extract sub logic to make is wrapping more practical.

destupdate: extract logic based on obsolescence marker in its own function One of the main goal of having consolidated destination function is to allow extension to play with this logic. We extract sub logic to make is wrapping more practical.

destupdate: move obsolete handling first This block was overwriting any result from the previous block anyway. So we move it first to prove it is possible and we'll extract it in its own function in the next patch.

destupdate: indent bookmark and branch logic We'll move the obsolete related logic first (as it is overwriting any other anyway) to make the next patch clearer we add indentation in this one.

destupdate: extract validation logic One of the main goal of having consolidated destination function is to allow extension to play with this logic. We extract sub logic to make is wrapping more practical.

rebase: rename and test '_destrebase' We make the name consistent with the other similar revsets and make sure it has minimal tests.

rebase: directly use '_destrebase' There is little value in using the revset instead of the function.

rebase: extra default destination in its own function This makes it much simple to wrap for other extension.

revset: rename and test '_destmerge' We make the name consistent with the one used by '_destupdate' and we ensure the code is run by testing it (abort is expected and merge would).

merge: directly get destination from destutil There is no real value in using the revset over the function. The revset have no remaining users and will be taken care of in a later changesets.

destutil: move default merge destination into a function Function in destutil are much simpler to wrap and more flexible than revset. This also help consistency as 'destupdate' live here and cannot become a pure revset anyway.

revset: reintroduce and experimental revset for update destination The revset is not ready for prime time yet. However it is useful to have some version of it exposed to help candidate users to play with it and provide feedback on what we should aim at. We add a small test to make sure the code runs.

changegroup: move manifest unpacking into its own method The upcoming cg3 will need different logic for unpacking manifests.

changegroup: move manifest packing into a separate function A future change will introduce a new function on a cg3packer that can pack treemanifests as well as flatmanifests.

changegroup: rename manifest linknode closure for clarity Since I'm spending the time to understand this code, I may as well leave it clearer than I found it.

changegroup: reformat packermap and add comment I'm about to add a cg3, and it seems prudent to annotate what formats support what features. It strikes me that we may want to consider moving to a more feature-oriented model in the future, but we'll see how that looks in a little while I guess.

changegroup: document the public surface area of cg?unpackers This should help future readers at least a little.

changegroup: mark cg1unpacker.chunklength as private

changegroup: note why a few methods on cg1unpacker exist I'm not sure what to do abstraction-wise here. It might be more sensible to make a memoryrepo that could apply a bundle in-memory and then we could make the changegroup data be strictly an applyable stream, but that's an idea for Later.

revlog: rename bundle to cg to reflect its nature as a cg?unpacker The new convention is that bundles contain changegroups. bundle1 happens to *only* be a changegroup, but bundle2 is a more featureful container that isn't something you can pass to addgroup().

changegroup: mark _addchangegroupfiles as module-private I'm trying to reason about the public surface area of this module now, so it's worth tagging private things as such.

changegroup: delete now-unused addchangegroup method

localrepo: use cg?unpacker.apply() instead of changegroup.addchangegroup() This is in localpeer, so it lives. Had it been in localrepo instead, I would have tried to exterminate it.

repair: use cg?unpacker.apply() instead of changegroup.addchangegroup()

exchange: use cg?unpacker.apply() instead of changegroup.addchangegroup()

commands: use cg?unpacker.apply() instead of changegroup.addchangegroup()

bundle2: use cg?unpacker.apply() instead of changegroup.addchangegroup()

shelve: use cg?unpacker.apply() instead of changegroup.addchangegroup()

histedit: use cg?unpacker.apply() instead of changegroup.addchangegroup()

changegroup: migrate addchangegroup() to forward to cg?unpacker.apply() I'll clean up callers in subsequent patches, then remove the forwarding.

changegroup: move source check to top of addchangegroup This is preparation for some refactoring.

error: remove superfluous pass statements

hook: raise a separate exception for when loading a hook fails For easier catching.

clonebundles: advertise clone bundles feature to clients Server operators that have enabled clone bundles probably want clients to use it. This patch introduces a feature that will insert a bundle2 "output" part that advertises the existence of the clone bundles feature to clients that aren't using it. The server uses the "cbattempted" argument to "getbundle" to determine whether a client supports clone bundles and to avoid sending the message to clients that failed the clone bundle for whatever reason.

exchange: advertise if a clone bundle was attempted The client now sends a "cbattempted" boolean flag to the "getbundle" wire protocol command to tell the server whether a clone bundle was attempted. The presence of this flag will enable the server to conditionally emit a bundle2 "output" part advertising the availability of clone bundles to compatible clients that don't have it enabled.

exchange: record that we attempted to fetch a clone bundle This is needed so a subsequent patch can conditionally add a bundle2 part to the "getbundle" wire protocol command depending on whether a clone bundle was attempted.

exchange: provide hint on how to disable clone bundles If a clone bundle persistently fails to apply, users need a way to disable it so they have a hope of the clone working. Change the hint for the abort scenario to advertise the config option to disable clone bundles.

exchange: document filterclonebundleentries

wireproto: properly parse false boolean args (BC) The client represents boolean arguments as '0' and '1'. bool('0') == bool('1') == True, so a simple bool(val) isn't sufficient for converting the argument back to a bool type. Currently, "obsmarkers" is the only boolean argument to getbundle. I /think/ the only place where we currently set the "obsmarkers" argument is during bundle2 pulls. As a result of this bug, the server /might/ be sending obsolete markers bundle2 part(s) to clients that don't request them. That is why I marked this BC. Surprisingly there was no test fall out from this change. I suspect a lapse in test coverage.

bundle2: gracefully skip 'obsmarkers' part if evolution is disabled We would skip the part if it was fully unknown, so we should also skip it if we know we won't be able to apply it. This will allow us to produce bundles with obsolescence markers alongside changegroup while still being able to apply them on any client.

obsstore: make the readonly attribute accessible We want to gracefully handle the read only case in some case (current target: advisory obsmarkers parts in bundle2). So we expose the attribute in a clean way.

update: introduce a 'UpdateAbort' exception The 'postincoming' function used by 'hg pull --update' and 'hg unbundle' is catching 'Abort' exceptions to intercept failed update. This feel a bit too wide to me, so I'm introducing a more precise exception to specify update destination issues.

update: "deprecate" call to 'merge.update' without a destination Now that all internal callers pre-compute and set a destination at a higher level it feels like we can kill this API. This will allow us to simplify this function. However I feel like this is a bit too central and critical to break now. I'm adding a devel warning to let extension make catch this in the next cycle.

shelve: delete shelve statefile on any exception during abort When a user's repository is in an unfinished unshelve state and they choose to abort, at a minimum, the repo should be out of that state. We've found situations where the user could not leave the state unless manually deleting the state file. This fix ensures that no matter what exception may be raised during the abort, the shelved state file will be deleted, the user will be out of the unshelve state and they can get their repository into a workable condition.

highlight: add option to prevent content-only based fallback When Mozilla enabled Pygments on hg.mozilla.org, we got a lot of weirdly colorized files. Upon further investigation, the hightlight extension is first attempting a filename+content based match then falling back to a purely content-driven detection mode in Pygments. Sounds good in theory. Unfortunately, Pygments' content-driven detection establishes no minimum threshold for returning a lexer. Furthermore, the detection code for a number of languages is very liberal. For example, ActionScript 3 will return a confidence of 0.3 (out of 1.0) if the first 1k of the file we pass in matches the regex "\w+\s*:\s*\w"! Python matches on "import ". It's no coincidence that a number of our extension-less files were getting highlighted improperly. This patch adds an option to have the highlighter not fall back to purely content-based detection when filename+content detection failed. This can be enabled to render unlighted text instead of taking the risk that unknown file types are highlighted incorrectly. The old behavior is still the default.

highlight: inline checkfctx() It is only used once. pygmentize() is pretty small. Let's just inline it.

highlight: consolidate duplicate code I'm adding some logic in a future patch and this will make it so I only have to add it once.

rebase: properly abort when destination is public (issue4896) After rebasing a set of changes onto a public changeset and having the first one be skipped, if you try to abort, the operation fails. This fix adds a check to disallow the target rev into the dstates list within the abort function. This list is checked for immutable states before the rest of abort does its thing.

bookmarks: don't deactivate on no-op update (issue4901)

rebase: properly handle chains of markers with missing nodes As obsolescence markers can contains unknown nodes and 'allsuccessors' returns them, we have to protect again that when looking for successors of the rebase set in the destination. Test have been expanded to catch that.

rebase: use a direct reference to repo.changelog Accessing 'repo.changelog' have a small overhead because we double check that the filtering did not changed. As we make multiple use of this into loops, we should avoid doing the lookup/check every time. This also make the code tidier.

mail: drop python 2.5 support

exchange: use pushop.repo instead of repo

rebase: factor out nothing to rebase return code A rebase call that results in nothing to rebase might be considered successful in some contexts. This factors out the return code from places where hg determines that there is nothing to rebase, so an extenion might change this return code to be something that would allow scripts to run without seeing this as an error.

gitweb: visually highlight source lines when hovering over line numbers Due to how the line links now reside outside of the source lines, hovering over line numbers doesn't count as hovering over the appropriate source line. It can be worked around by using a "+" css selector. However, it's necessary to reorder the elements and put <a> before <span> (which is actually quite logical). It works without further css tweaks because <a> is already absolute-positioned and so the order doesn't matter visually.

rebase: added comments Added comments describing the state variable and constants used throughout the rebase extension

monoblue: visually highlight source lines when hovering over line numbers Due to how the line links now reside outside of the source lines, hovering over line numbers doesn't count as hovering over the appropriate source line. It can be worked around by using a "+" css selector. However, it's necessary to reorder the elements and put <a> before <span> (which is actually quite logical). It works without further css tweaks because <a> is already absolute-positioned and so the order doesn't matter visually.

monoblue: make the size of line links bigger to cover line numbers better Due to how the line numbers in monoblue are formed (via css counters), the size of the area with the numbers and the size of the actually clickable links are not tied together well enough. Before this patch, there were noticeable "gaps" between line links - clicking on the bottom part of a visible line number did nothing as opposed to selecting this line. Let's set font-size for everything in pre.sourcelines so that it also affects the links and then add a bit of padding to them so compensate for layout differences. This way the sizes are still not 100% the same, but should be very close.

gitweb: don't drop current revision context on graph page In hgweb, some pages have a context of current revision; e.g. changelog and shortlog show changesets starting from this current revision. However, some gitweb templates were dropping current revision from some urls _to_ /graph page and _on_ that page. This patch fixes it.

util: also catch IndexError This makes life so, so much easier for hgwatchman, which provides a named tuple but throws an IndexError instead of a TypeError.

exewrapper: add comments about PYTHONHOME This has been a repeating source of confusion for users of HackableMercurial. Note that users of HackableMercurial should *not* and are *not* expected to set PYTHONHOME.

exewrapper: report name of failed private DLL in error message Same as in preceding change, but this time it's for the private Python of HackableMercurial (in hg-python directory next to the hg.exe).

exewrapper: report name of failed DLL in error message This uses C string literal concatenation. Example output: $ hg version abort: failed to load Python DLL python27.dll Note that HGPYTHONLIB does not contain the trailing .dll, which works here because the documentation of LoadLibrary [1] says, that if no file name extension is specified in the filename parameter, the default library extension .dll is appended. See [2] for a motivation of this change. [1] https://msdn.microsoft.com/en-us/library/windows/desktop/ms684175(v=vs.85).aspx [2] https://selenic.com/pipermail/mercurial/2015-August/048627.html

merge: abort on file/directory case folding collisions (issue4892) File/directory case folding collisions cannot be represented on case folding systems and have to fail. To detect this and abort early, utilize that for file/directory collisions, a sorted list of case folded manifest names will have the colliding directory right after the file. (This could perhaps be optimized, but this way of doing it also has directory/directory case folding in mind ... which however not is handled yet.)

tests: test of case-folding merge collisions

copies: group bothnew with other sets

copies: rename renamedelete to renamedeleteset for clarity

copies: move _makegetfctx calls into checkcopies

copies: factor out setupctx into _makegetfctx This reduces the scope of mergecopies a bit

copies: avoid reference to c1/c2 in makectx

mq: use cmdutil.revert instead of hg.revert It's the last user.

debugmergestate: add support for printing out driver-resolved files

debugmergestate: add support for printing out merge driver

merge.mergedriver: don't try resolving files marked driver-resolved The driver is expected to take care of these.

merge.mergestate: add support for persisting driver-resolved files A driver-resolved file is a file that's handled specially by the driver. A common use case for this state would be autogenerated files, the generation of which should happen only after all source conflicts are resolved. This is done with an uppercase letter because older versions of Mercurial will not know how to treat such files at all.

merge.mergestate: add support for persisting a custom merge driver A 'merge driver' is a coordinator for the overall merge process. It will be able to control: - tools for individual files, much like the merge-patterns configuration does today - tools that can work across groups of files - the ordering of file resolution - resolution of automatically generated files - adding and removing additional files to and from the dirstate Since it is a critical part of the merge process, it really is part of the merge state. This is a lowercase character (i.e. optional) because ignoring this is fine for older versions of Mercurial -- however, if there are any files that are specially treated by the driver, we should abort. That will happen in upcoming patches. There is a potential security issue with storing the merge driver in the merge state. See the inline comments for more details.

exchange: support sorting URLs by client-side preferences Not all bundles are appropriate for all clients. For example, someone with a slow Internet connection may want to prefer bz2 bundles over gzip bundles because they are smaller and don't take as long to transfer. This is information that a server cannot know on its own. So, we invent a mechanism for "preferring" server-advertised URLs based on their attributes. We could invent a negotiation between client and server where the client sends its preferences and the sorting/filtering is done server-side. However, this feels complex. We can avoid complicating the wire protocol and exposing ourselves to backwards compatible concerns by performing the sorting locally. This patch defines a new config option for expressing preferred attributes in server-advertised bundles. At Mozilla, we leverage this feature so clients in fast data centers prefer uncompressed bundles. (We advertise gzip bundles first because that is a reasonable default.) I consider this an advanced feature. I'm on the fence as to whether it should be documented in `hg help config`.

exchange: extract bundle specification components into own attributes An upcoming patch will enable clients to prefer certain bundles over others. The idea is that we define values of attributes from manifests that are desirable. The BUNDLESPEC attribute is a complex value consisting of multiple parts. Clients may wish to only prefer one of these parts. Having to specify every combination of BUNDLESPEC would be annoying. So, we extract the components of BUNDLESPEC into their own attributes so clients can easily filter on a sub-component.

exchange: support preserving external names when parsing bundle specs This will be needed to make client-side preferences work easier.

clonebundles: filter on SNI requirement Server Name Indication (SNI) is commonly used in CDNs and other hosted environments. Unfortunately, Python <2.7.9 does not support SNI and when these older Python versions attempt to negotiate TLS to an SNI server, they raise an opaque error like "_ssl.c:507: error:14094410:SSL routines:SSL3_READ_BYTES:sslv3 alert handshake failure." We introduce a manifest attribute to denote the URL requires SNI and have clients without SNI support filter these entries.

clonebundles: filter on bundle specification Not all clients are capable of reading every bundle. Currently, content negotiation to ensure a server sends a client a compatible bundle format is performed at request time. The response bundle is dynamically generated at request time, so this works fine. Clone bundles are statically generated *before* the request. This means that a modern server could produce bundles that a legacy client isn't capable of reading. Without some kind of "type hint" in the clone bundles manifest, a client may attempt to download an incompatible bundle. Furthermore, a client may not realize a bundle is incompatible until it has processed part of the bundle (imagine consuming a 1 GB changegroup bundle2 part only to discover the bundle2 part afterwards is incompatibl). This would waste time and resources. And it isn't very user friendly. Clone bundle manifests thus need to advertise the *exact* format of the hosted bundles so clients may filter out entries that they don't know how to read. This patch introduces that mechanism. We introduce the BUNDLESPEC attribute to declare the "bundle specification" of the entry. Bundle specifications are parsed using exchange.parsebundlespecification, which uses the same strings as the "--type" argument to `hg bundle`. The supported bundle specifications are well defined and backwards compatible. When a client encounters a BUNDLESPEC that is invalid or unsupported, it silently ignores the entry.

clonebundle: support bundle2 exchange.readbundle() can return 2 different types. We weren't handling the bundle2 case. Handle it. At some point we'll likely want a generic API for applying a bundle from a file handle. For now, create another one-off until we figure out what the unified bundle API should look like (addressing this is a can of worms I don't want to open right now).

update: also use 'destupdate' for pull and unbundle Update can also be performed by 'hg pull --update' and 'hg unbundle'. We use the destupdate function in these case too.

destupdate: also include bookmark related logic For the same reason, we move the bookmark related update logic into the 'destupdate' function. This requires to extend the returns of the function to include the bookmark that needs to move (more or less) and the bookmark to activate at the end of the function. See function documentation for details on this returns.

exchange: refactor bundle specification parsing The old code was tailored to `hg bundle` usage and not appropriate for use as a general API, which clone bundles will require. The code has been rewritten to make it more generally suitable. We introduce dedicated error types to represent invalid and unsupported bundle specifications. The reason we need dedicated error types (rather than error.Abort) is because clone bundles will want to catch these exception as part of filtering entries. We don't want to swallow error.Abort on principle.

exchange: move bundle specification parsing from cmdutil Clone bundles require a well-defined string to specify the type of bundle that is listed so clients can filter compatible file types. The `hg bundle` command and cmdutil.parsebundletype() already establish the beginnings of a bundle specification format. As part of formalizing this format specification so it can be used by clone bundles, we move the specification parsing bits verbatim to exchange.py, which is a more suitable place than cmdutil.py. A subsequent patch will refactor this code to make it more appropriate as a general API.

revset: add optional offset argument to limit() predicate It's common for GUI or web frontend to fetch chunk of revisions per batch size. Previously it was possible only if revisions were sorted by revision number. $ hg log -r 'limit({revspec} & :{last_known}, 101)' So this patch introduces a general way to retrieve chunk of revisions after skipping offset revisions. $ hg log -r 'limit({revspec}, 100, {last_count})' This is a dumb implementation. We can optimize it for baseset and spanset later.

revset: port limit() to support keyword arguments The next patch will introduce the third 'offset' argument. This allows us to specify 'offset' without 'n' argument.

revset: eliminate temporary reference to subset in limit() and last()

dirstate: read from pending file under HG_PENDING mode if it exists True/False value of '_pendingmode' means whether 'dirstate.pending' is used to initialize own '_map' and so on. When it is None, neither 'dirstate' nor 'dirstate.pending' is read in yet. This is used to keep consistent view between '_pl()' and '_read()'. Once '_pendingmode' is determined by reading one of 'dirstate' or 'dirstate.pending' in, '_pendingmode' is kept even if 'invalidate()' is invoked. This should be reasonable, because: - effective 'invalidate()' invocation should occur only in wlock scope, and - wlock can't be gotten under HG_PENDING mode '_trypending()' is defined as a normal function to factor similar code path (in bookmarks and phases) out in the future easily.

dirstate: make writing in-memory changes aware of transaction activity This patch delays writing in-memory changes out, if transaction is running. '_getfsnow()' is defined as a function, to hook it easily for ambiguous timestamp tests (see also fakedirstatewritetime.py) 'if tr:' code path in this patch is still disabled at this revision, because there is no client invoking 'dirstate.write()' with repo object. BTW, this patch changes 'dirstate.invalidate()' semantics around 'dirstate.write()' in a transaction scope: before: with repo.transaction(): dirstate.CHANGE('A') dirstate.write() # change for A is written out here dirstate.CHANGE('B') dirstate.invalidate() # discards only change for B after: with repo.transaction(): dirstate.CHANGE('A') dirstate.write() # change for A is still kept in memory dirstate.CHANGE('B') dirstate.invalidate() # discards changes for A and B Fortunately, there is no code path expecting the former, at least, in Mercurial itself, because 'dirstateguard' was introduced to remove such 'dirstate.invalidate()'.